2024-03-22T05:52:43.367 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-22T05:52:43.367 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_phlogistonjohn@teuthology --archive /home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090 --name phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --verbose -- /home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090/orig.config.yaml 2024-03-22T05:52:43.415 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T05:52:43.506 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090 branch: wip-phlogistonjohn-testing-2024-03-21-1835 description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: jmulligan@redhat.com first_in_suite: false job_id: '7616090' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-phlogistonjohn-testing-2024-03-21-1835 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-phlogistonjohn-testing-2024-03-21-1835 sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 owner: scheduled_phlogistonjohn@teuthology priority: 95 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 5374 sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 sleep_before_teardown: 0 subset: 1/24 suite: orch:cephadm suite_branch: wip-phlogistonjohn-testing-2024-03-21-1835 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ea01217f8a79678e64ef6a5d68a043a4ef21ca37/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 targets: smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL7Rh2nhxdiKheDchRY+s+PZcNNLncc8JtyB/NpcJI2YkbwhsG/vLRl4Ex7h/vWFnvtTLSdRhiuvrd8TdxoU5ok= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLSrAIKihkpI38B332XvAQERRk0xewPUHrkVozw9iWrIVEJAogL1Vf9QTCD4D1w34az4Mhh1zPLDzDksRuKdlig= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/phlogistonjohn/src/git.ceph.com_ceph-c_wip-phlogistonjohn-testing-2024-03-21-1835/qa/suites/orch/cephadm/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_03:02:08 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: phlogistonjohn verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-22T05:52:43.506 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ea01217f8a79678e64ef6a5d68a043a4ef21ca37/qa; will attempt to use it 2024-03-22T05:52:43.507 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ea01217f8a79678e64ef6a5d68a043a4ef21ca37/qa/tasks 2024-03-22T05:52:43.507 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-22T05:52:43.508 INFO:teuthology.task.internal:Checking packages... 2024-03-22T05:52:43.533 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ea01217f8a79678e64ef6a5d68a043a4ef21ca37' 2024-03-22T05:52:43.533 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T05:52:43.533 INFO:teuthology.packaging:ref: None 2024-03-22T05:52:43.533 INFO:teuthology.packaging:tag: None 2024-03-22T05:52:43.534 INFO:teuthology.packaging:branch: wip-phlogistonjohn-testing-2024-03-21-1835 2024-03-22T05:52:43.534 INFO:teuthology.packaging:sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T05:52:43.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-phlogistonjohn-testing-2024-03-21-1835 2024-03-22T05:52:43.750 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2299.gea01217f 2024-03-22T05:52:43.751 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-22T05:52:43.757 INFO:teuthology.task.internal:no buildpackages task found 2024-03-22T05:52:43.757 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-22T05:52:43.780 INFO:teuthology.task.internal:Saving configuration 2024-03-22T05:52:43.799 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-22T05:52:43.805 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T05:52:43.827 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 05:47:43.154049', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL7Rh2nhxdiKheDchRY+s+PZcNNLncc8JtyB/NpcJI2YkbwhsG/vLRl4Ex7h/vWFnvtTLSdRhiuvrd8TdxoU5ok='} 2024-03-22T05:52:43.849 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 05:47:43.151797', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLSrAIKihkpI38B332XvAQERRk0xewPUHrkVozw9iWrIVEJAogL1Vf9QTCD4D1w34az4Mhh1zPLDzDksRuKdlig='} 2024-03-22T05:52:43.849 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-22T05:52:43.855 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-22T05:52:43.855 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-22T05:52:43.855 INFO:teuthology.run_tasks:Running task console_log... 2024-03-22T05:52:43.934 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f37001270d0>, signals=[15]) 2024-03-22T05:52:43.935 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-22T05:52:43.941 INFO:teuthology.task.internal:Opening connections... 2024-03-22T05:52:43.941 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T05:52:43.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T05:52:44.051 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-03-22T05:52:44.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T05:52:44.139 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-22T05:52:44.145 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2024-03-22T05:52:44.163 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2024-03-22T05:52:44.163 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2024-03-22T05:52:44.221 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream" 2024-03-22T05:52:44.221 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="9" 2024-03-22T05:52:44.221 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos" 2024-03-22T05:52:44.222 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora" 2024-03-22T05:52:44.222 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="9" 2024-03-22T05:52:44.222 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el9" 2024-03-22T05:52:44.222 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-22T05:52:44.222 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2024-03-22T05:52:44.222 INFO:teuthology.orchestra.run.smithi006.stdout:LOGO="fedora-logo-icon" 2024-03-22T05:52:44.222 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-22T05:52:44.222 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/" 2024-03-22T05:52:44.222 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-22T05:52:44.222 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-22T05:52:44.222 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-22T05:52:44.223 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2024-03-22T05:52:44.251 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-03-22T05:52:44.269 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-03-22T05:52:44.269 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-22T05:52:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-22T05:52:44.324 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-03-22T05:52:44.343 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-22T05:52:44.352 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-22T05:52:44.371 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-22T05:52:44.372 DEBUG:teuthology.orchestra.run.smithi006:> test '!' -e /home/ubuntu/cephtest 2024-03-22T05:52:44.374 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-03-22T05:52:44.390 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-22T05:52:44.397 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-22T05:52:44.397 DEBUG:teuthology.orchestra.run.smithi006:> test -z $(ls -A /var/lib/ceph) 2024-03-22T05:52:44.431 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-03-22T05:52:44.526 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-22T05:52:44.582 INFO:teuthology.run_tasks:Running task kernel... 2024-03-22T05:52:44.595 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-22T05:52:44.595 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-22T05:52:44.595 DEBUG:teuthology.orchestra.run.smithi006:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T05:52:44.595 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T05:52:44.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T05:52:44.612 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-03-22T05:52:44.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T05:52:44.613 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-03-22T05:52:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-361.el9.x86_64 2024-03-22T05:52:44.668 INFO:teuthology.task.kernel:Running kernel on smithi117: 5.14.0-361.el9.x86_64 2024-03-22T05:52:44.668 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum install -y kernel 2024-03-22T05:52:44.671 INFO:teuthology.orchestra.run.smithi006.stdout:5.14.0-361.el9.x86_64 2024-03-22T05:52:44.671 INFO:teuthology.task.kernel:Running kernel on smithi006: 5.14.0-361.el9.x86_64 2024-03-22T05:52:44.672 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum install -y kernel 2024-03-22T05:52:46.703 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-22T05:52:46.723 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-22T05:52:48.153 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-03-22T05:52:48.290 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - BaseOS 4.3 MB/s | 2.0 MB 00:00 2024-03-22T05:52:49.503 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.0 MB 00:01 2024-03-22T05:52:51.047 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.0 MB 00:02 2024-03-22T05:52:51.446 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-22T05:52:53.184 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-03-22T05:52:55.632 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.1 MB 00:00 2024-03-22T05:52:57.040 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-03-22T05:52:57.265 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-03-22T05:52:59.315 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-03-22T05:53:00.408 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-03-22T05:53:05.408 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 7.4 kB/s | 1.7 kB 00:00 2024-03-22T05:53:07.439 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-22T05:53:07.439 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-22T05:53:07.535 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T05:53:07.547 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T05:53:07.547 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-03-22T05:53:07.547 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T05:53:07.547 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T05:53:07.547 INFO:teuthology.orchestra.run.smithi117.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-22T05:53:07.547 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-22T05:53:07.547 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-22T05:53:07.547 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-22T05:53:07.547 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-22T05:53:07.547 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T05:53:07.548 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T05:53:07.548 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T05:53:07.548 INFO:teuthology.orchestra.run.smithi117.stdout:Install 4 Packages 2024-03-22T05:53:07.548 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T05:53:07.548 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 76 M 2024-03-22T05:53:07.548 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 123 M 2024-03-22T05:53:07.548 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T05:53:07.834 INFO:teuthology.orchestra.run.smithi117.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 199 kB/s | 44 kB 00:00 2024-03-22T05:53:08.250 INFO:teuthology.orchestra.run.smithi117.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 23 MB/s | 15 MB 00:00 2024-03-22T05:53:08.942 INFO:teuthology.orchestra.run.smithi117.stdout:(3/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 25 MB/s | 33 MB 00:01 2024-03-22T05:53:09.692 INFO:teuthology.orchestra.run.smithi117.stdout:(4/4): kernel-modules-core-5.14.0-430.el9.x86_6 15 MB/s | 27 MB 00:01 2024-03-22T05:53:09.695 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T05:53:09.695 INFO:teuthology.orchestra.run.smithi117.stdout:Total 35 MB/s | 76 MB 00:02 2024-03-22T05:53:10.364 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T05:53:10.718 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T05:53:10.718 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T05:53:11.735 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T05:53:11.736 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T05:53:14.252 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T05:53:14.885 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-22T05:53:14.975 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T05:53:17.709 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T05:53:17.845 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T05:53:18.258 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-03-22T05:53:22.583 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T05:53:22.828 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:53:23.347 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-22T05:53:25.646 INFO:teuthology.orchestra.run.smithi006.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-22T05:53:25.646 INFO:teuthology.orchestra.run.smithi006.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-22T05:53:25.729 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T05:53:25.735 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T05:53:25.736 INFO:teuthology.orchestra.run.smithi006.stdout:Install 4 Packages 2024-03-22T05:53:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T05:53:25.736 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 76 M 2024-03-22T05:53:25.736 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 123 M 2024-03-22T05:53:25.736 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T05:53:26.457 INFO:teuthology.orchestra.run.smithi006.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 66 kB/s | 44 kB 00:00 2024-03-22T05:53:27.582 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:53:45.414 INFO:teuthology.orchestra.run.smithi006.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 777 kB/s | 15 MB 00:19 2024-03-22T05:53:56.939 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:53:56.939 INFO:teuthology.orchestra.run.smithi117.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-22T05:53:56.940 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T05:53:56.984 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:53:57.312 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:53:57.312 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-22T05:53:57.312 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T05:53:57.312 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T05:53:57.691 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:53:57.691 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T05:53:57.691 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T05:53:57.691 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-22T05:53:57.691 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-22T05:53:57.691 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-22T05:53:57.691 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-22T05:53:57.691 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T05:53:57.691 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T05:53:58.014 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T05:53:58.122 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T05:53:58.123 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T05:53:58.124 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-22T05:53:58.124 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-22T05:53:58.124 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-22T05:53:58.124 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-03-22T05:53:58.124 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=None, version=distro) 2024-03-22T05:53:58.124 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T05:53:58.203 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T05:53:58.204 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T05:53:58.204 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -qi grub2-tools 2024-03-22T05:53:58.245 INFO:teuthology.orchestra.run.smithi117.stdout:Name : grub2-tools 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Epoch : 1 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Version : 2.06 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Release : 68.el9 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: x86_64 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Group : Unspecified 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Size : 8264254 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:License : GPLv3+ 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-22T05:53:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout:Packager : builder@centos.org 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout:Vendor : CentOS 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout:Summary : Support tools for GRUB. 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout:Description : 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout:hardware devices. 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T05:53:58.247 INFO:teuthology.orchestra.run.smithi117.stdout:This subpackage provides tools for support of all platforms. 2024-03-22T05:53:58.248 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-22T05:53:58.248 INFO:teuthology.task.kernel:Updating grub on smithi117 to boot 5.14.0-430.el9.x86_64 2024-03-22T05:53:58.248 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-22T05:53:58.848 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-03-22T05:54:00.209 INFO:teuthology.orchestra.run.smithi117.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-22T05:54:00.229 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-03-22T05:54:00.232 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-03-22T05:54:00.247 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.yyNFLTwgrs 2024-03-22T05:54:00.247 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yyNFLTwgrs 2024-03-22T05:54:00.316 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.yyNFLTwgrs 2024-03-22T05:54:00.453 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.yyNFLTwgrs is 6KB 2024-03-22T05:54:00.472 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.yyNFLTwgrs 2024-03-22T05:54:00.485 DEBUG:teuthology.orchestra.run.smithi117:> sudo /bin/ls /boot/loader/entries || true 2024-03-22T05:54:00.553 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-22T05:54:00.553 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-22T05:54:00.553 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-22T05:54:00.553 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-22T05:54:00.555 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-22T05:54:00.706 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-03-22T05:54:08.362 INFO:teuthology.orchestra.run.smithi006.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 671 kB/s | 27 MB 00:41 2024-03-22T05:54:08.837 INFO:teuthology.orchestra.run.smithi006.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 795 kB/s | 33 MB 00:43 2024-03-22T05:54:08.838 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T05:54:08.838 INFO:teuthology.orchestra.run.smithi006.stdout:Total 1.8 MB/s | 76 MB 00:43 2024-03-22T05:54:09.528 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T05:54:09.965 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T05:54:09.965 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T05:54:10.978 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T05:54:10.979 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T05:54:13.580 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T05:54:14.225 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-22T05:54:14.314 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T05:54:17.207 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T05:54:17.348 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T05:54:22.426 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T05:54:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:54:27.560 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:54:30.709 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-22T05:54:30.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-22T05:54:30.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T05:54:49.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-03-22T05:54:58.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-22T05:54:58.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T05:54:58.479 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:54:58.479 INFO:teuthology.orchestra.run.smithi006.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-22T05:54:58.479 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T05:54:58.525 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:54:58.886 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:54:58.886 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-22T05:54:58.886 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T05:54:58.886 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T05:54:59.322 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T05:54:59.322 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T05:54:59.322 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T05:54:59.322 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-22T05:54:59.323 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-22T05:54:59.323 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-22T05:54:59.323 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-22T05:54:59.323 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T05:54:59.323 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T05:54:59.646 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T05:54:59.770 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T05:54:59.770 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T05:54:59.770 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-22T05:54:59.770 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-22T05:54:59.770 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-22T05:54:59.770 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-03-22T05:54:59.770 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi006.front.sepia.ceph.com, path=None, version=distro) 2024-03-22T05:54:59.770 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T05:54:59.853 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T05:54:59.853 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T05:54:59.853 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -qi grub2-tools 2024-03-22T05:54:59.898 INFO:teuthology.orchestra.run.smithi006.stdout:Name : grub2-tools 2024-03-22T05:54:59.898 INFO:teuthology.orchestra.run.smithi006.stdout:Epoch : 1 2024-03-22T05:54:59.898 INFO:teuthology.orchestra.run.smithi006.stdout:Version : 2.06 2024-03-22T05:54:59.898 INFO:teuthology.orchestra.run.smithi006.stdout:Release : 68.el9 2024-03-22T05:54:59.898 INFO:teuthology.orchestra.run.smithi006.stdout:Architecture: x86_64 2024-03-22T05:54:59.898 INFO:teuthology.orchestra.run.smithi006.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-22T05:54:59.898 INFO:teuthology.orchestra.run.smithi006.stdout:Group : Unspecified 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:Size : 8264254 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:License : GPLv3+ 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:Packager : builder@centos.org 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:Vendor : CentOS 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:Summary : Support tools for GRUB. 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:Description : 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-22T05:54:59.899 INFO:teuthology.orchestra.run.smithi006.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-22T05:54:59.900 INFO:teuthology.orchestra.run.smithi006.stdout:hardware devices. 2024-03-22T05:54:59.900 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T05:54:59.900 INFO:teuthology.orchestra.run.smithi006.stdout:This subpackage provides tools for support of all platforms. 2024-03-22T05:54:59.900 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-22T05:54:59.901 INFO:teuthology.task.kernel:Updating grub on smithi006 to boot 5.14.0-430.el9.x86_64 2024-03-22T05:54:59.901 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-22T05:55:00.542 INFO:teuthology.orchestra.run.smithi006.stderr:Generating grub configuration file ... 2024-03-22T05:55:01.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-03-22T05:55:01.965 INFO:teuthology.orchestra.run.smithi006.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-22T05:55:02.010 INFO:teuthology.orchestra.run.smithi006.stderr:done 2024-03-22T05:55:02.012 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2024-03-22T05:55:02.067 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.M32T8fSoU3 2024-03-22T05:55:02.067 DEBUG:teuthology.orchestra.run.smithi006:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.M32T8fSoU3 2024-03-22T05:55:02.138 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.M32T8fSoU3 2024-03-22T05:55:02.284 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.M32T8fSoU3 is 6KB 2024-03-22T05:55:02.333 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.M32T8fSoU3 2024-03-22T05:55:02.347 DEBUG:teuthology.orchestra.run.smithi006:> sudo /bin/ls /boot/loader/entries || true 2024-03-22T05:55:02.416 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-22T05:55:02.416 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-22T05:55:02.416 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-22T05:55:02.416 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-22T05:55:02.417 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-22T05:55:02.550 DEBUG:teuthology.orchestra.run.smithi006:> sudo shutdown -r now 2024-03-22T05:55:13.322 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-22T05:55:13.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T05:55:32.570 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-22T05:55:32.571 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-22T05:55:32.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T05:55:45.117 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-03-22T05:55:45.699 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-22T05:55:45.699 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-430.el9.x86_64"... 2024-03-22T05:55:45.700 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-03-22T05:55:45.713 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-430.el9.x86_64 2024-03-22T05:55:45.713 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-22T05:55:45.713 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-22T05:55:45.713 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-22T05:55:46.714 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-22T05:55:46.714 DEBUG:teuthology.orchestra.run.smithi117:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-22T05:55:46.795 INFO:teuthology.orchestra.run.smithi117.stdout:ttyS1 2024-03-22T05:55:46.829 DEBUG:teuthology.parallel:result is None 2024-03-22T05:55:50.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.6 2024-03-22T05:55:59.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-22T05:55:59.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T05:56:59.946 DEBUG:teuthology.orchestra.remote:timed out 2024-03-22T05:57:11.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-22T05:57:11.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T05:57:12.225 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-03-22T05:57:12.634 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-22T05:57:12.634 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-430.el9.x86_64"... 2024-03-22T05:57:12.634 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-03-22T05:57:12.649 INFO:teuthology.orchestra.run.smithi006.stdout:5.14.0-430.el9.x86_64 2024-03-22T05:57:12.649 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-22T05:57:12.649 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-22T05:57:12.649 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-22T05:57:13.650 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-22T05:57:13.651 DEBUG:teuthology.orchestra.run.smithi006:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-22T05:57:13.734 INFO:teuthology.orchestra.run.smithi006.stdout:ttyS1 2024-03-22T05:57:13.776 DEBUG:teuthology.parallel:result is None 2024-03-22T05:57:13.776 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-22T05:57:13.784 INFO:teuthology.task.internal:Creating test directory... 2024-03-22T05:57:13.784 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T05:57:13.786 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T05:57:13.804 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-22T05:57:13.822 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-22T05:57:13.856 INFO:teuthology.task.internal:Creating archive directory... 2024-03-22T05:57:13.857 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T05:57:13.859 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T05:57:14.053 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-22T05:57:14.060 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-22T05:57:14.060 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T05:57:14.063 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T05:57:14.101 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T05:57:14.101 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T05:57:14.115 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T05:57:14.116 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T05:57:14.117 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-22T05:57:14.124 INFO:teuthology.task.internal:Configuring sudo... 2024-03-22T05:57:14.125 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T05:57:14.160 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T05:57:14.190 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-22T05:57:14.199 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-22T05:57:14.199 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T05:57:14.230 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T05:57:14.246 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T05:57:14.326 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T05:57:14.372 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T05:57:14.456 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T05:57:14.496 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T05:57:14.497 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T05:57:14.564 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T05:57:14.609 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T05:57:14.670 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T05:57:14.710 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T05:57:14.758 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T05:57:14.758 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T05:57:14.827 DEBUG:teuthology.orchestra.run.smithi006:> sudo service rsyslog restart 2024-03-22T05:57:14.829 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-03-22T05:57:14.871 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T05:57:14.917 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T05:57:15.315 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-22T05:57:15.322 INFO:teuthology.task.internal:Starting timer... 2024-03-22T05:57:15.322 INFO:teuthology.run_tasks:Running task pcp... 2024-03-22T05:57:15.360 INFO:teuthology.run_tasks:Running task selinux... 2024-03-22T05:57:15.392 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-22T05:57:15.393 DEBUG:teuthology.orchestra.run.smithi006:> sudo service auditd rotate 2024-03-22T05:57:15.440 INFO:teuthology.orchestra.run.smithi006.stdout:Rotating logs: 2024-03-22T05:57:15.442 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-03-22T05:57:15.505 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-03-22T05:57:15.506 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-22T05:57:15.507 DEBUG:teuthology.orchestra.run.smithi006:> /usr/sbin/getenforce 2024-03-22T05:57:15.530 INFO:teuthology.orchestra.run.smithi006.stdout:Permissive 2024-03-22T05:57:15.530 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-03-22T05:57:15.558 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-03-22T05:57:15.558 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi006.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive'} 2024-03-22T05:57:15.558 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-22T05:57:15.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T05:57:15.598 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-22T05:57:15.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T05:57:15.626 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-22T05:57:15.626 DEBUG:teuthology.orchestra.run.smithi006:> sudo /usr/sbin/setenforce permissive 2024-03-22T05:57:15.673 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-03-22T05:57:15.710 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-22T05:57:15.721 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-22T05:57:15.739 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-22T05:57:15.741 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi006.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-22T06:01:33.408 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi006.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2024-03-22T06:01:33.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-22T06:01:33.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:01:33.507 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-03-22T06:01:33.589 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-22T06:01:33.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-22T06:01:33.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:01:33.682 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-03-22T06:01:33.762 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-22T06:01:33.762 INFO:teuthology.run_tasks:Running task clock... 2024-03-22T06:01:33.772 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-22T06:01:33.773 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T06:01:33.773 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T06:01:33.776 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T06:01:33.776 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T06:01:33.804 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-22T06:01:33.821 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-22T06:01:33.835 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-22T06:01:33.851 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: ntpd: command not found 2024-03-22T06:01:33.852 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-22T06:01:33.865 INFO:teuthology.orchestra.run.smithi006.stdout:506 Cannot talk to daemon 2024-03-22T06:01:33.882 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-22T06:01:33.882 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-03-22T06:01:33.895 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-03-22T06:01:33.898 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-22T06:01:33.912 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-22T06:01:33.928 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-22T06:01:33.953 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-03-22T06:01:33.957 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T06:01:33.957 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-03-22T06:01:33.957 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:01:33.957 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:01:33.957 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:01:33.957 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:01:33.983 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-03-22T06:01:33.987 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T06:01:33.987 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-03-22T06:01:33.987 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:01:33.987 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:01:33.987 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:01:33.988 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:01:33.988 INFO:teuthology.run_tasks:Running task install... 2024-03-22T06:01:33.997 DEBUG:teuthology.task.install:project ceph 2024-03-22T06:01:33.997 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ea01217f8a79678e64ef6a5d68a043a4ef21ca37'}} 2024-03-22T06:01:33.997 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'ea01217f8a79678e64ef6a5d68a043a4ef21ca37'} 2024-03-22T06:01:33.998 INFO:teuthology.task.install:Using flavor: default 2024-03-22T06:01:34.003 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-22T06:01:34.003 INFO:teuthology.task.install:extra packages: [] 2024-03-22T06:01:34.004 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ea01217f8a79678e64ef6a5d68a043a4ef21ca37', 'tag': None, 'wait_for_package': False} 2024-03-22T06:01:34.004 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:01:34.004 INFO:teuthology.packaging:ref: None 2024-03-22T06:01:34.004 INFO:teuthology.packaging:tag: None 2024-03-22T06:01:34.004 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:01:34.004 INFO:teuthology.packaging:sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:01:34.004 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T06:01:34.006 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ea01217f8a79678e64ef6a5d68a043a4ef21ca37', 'tag': None, 'wait_for_package': False} 2024-03-22T06:01:34.007 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:01:34.007 INFO:teuthology.packaging:ref: None 2024-03-22T06:01:34.007 INFO:teuthology.packaging:tag: None 2024-03-22T06:01:34.007 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:01:34.007 INFO:teuthology.packaging:sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:01:34.007 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T06:01:34.262 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/ 2024-03-22T06:01:34.262 INFO:teuthology.task.install.rpm:Package version is 17.2.7-726.g9c4e0c34 2024-03-22T06:01:34.271 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/ 2024-03-22T06:01:34.272 INFO:teuthology.task.install.rpm:Package version is 17.2.7-726.g9c4e0c34 2024-03-22T06:01:34.377 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-22T06:01:34.377 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:01:34.377 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-22T06:01:34.405 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-22T06:01:34.405 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:01:34.405 INFO:teuthology.packaging:ref: None 2024-03-22T06:01:34.405 INFO:teuthology.packaging:tag: None 2024-03-22T06:01:34.406 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:01:34.406 INFO:teuthology.packaging:sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:01:34.406 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-22T06:01:34.445 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-22T06:01:34.446 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:01:34.446 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-22T06:01:34.473 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-22T06:01:34.475 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-22T06:01:34.475 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:01:34.475 INFO:teuthology.packaging:ref: None 2024-03-22T06:01:34.475 INFO:teuthology.packaging:tag: None 2024-03-22T06:01:34.475 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:01:34.475 INFO:teuthology.packaging:sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:01:34.475 DEBUG:teuthology.orchestra.run.smithi006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-22T06:01:34.544 DEBUG:teuthology.orchestra.run.smithi006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-22T06:01:34.553 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-22T06:01:34.620 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-03-22T06:01:34.621 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-03-22T06:01:34.625 DEBUG:teuthology.orchestra.run.smithi006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-22T06:01:34.692 INFO:teuthology.orchestra.run.smithi006.stdout:check_obsoletes = 1 2024-03-22T06:01:34.693 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum clean all 2024-03-22T06:01:34.859 INFO:teuthology.orchestra.run.smithi117.stdout:74 files removed 2024-03-22T06:01:34.886 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw 2024-03-22T06:01:34.937 INFO:teuthology.orchestra.run.smithi006.stdout:74 files removed 2024-03-22T06:01:34.964 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-radosgw 2024-03-22T06:01:35.632 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 172 kB/s | 77 kB 00:00 2024-03-22T06:01:35.672 INFO:teuthology.orchestra.run.smithi006.stdout:ceph packages for x86_64 192 kB/s | 77 kB 00:00 2024-03-22T06:01:36.074 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 23 kB/s | 9.2 kB 00:00 2024-03-22T06:01:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:ceph noarch packages 22 kB/s | 9.2 kB 00:00 2024-03-22T06:01:36.465 INFO:teuthology.orchestra.run.smithi006.stdout:ceph source packages 6.1 kB/s | 1.8 kB 00:00 2024-03-22T06:01:36.658 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 3.3 kB/s | 1.8 kB 00:00 2024-03-22T06:01:37.309 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - AppStream 8.3 MB/s | 6.7 MB 00:00 2024-03-22T06:01:37.402 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - AppStream 9.5 MB/s | 6.7 MB 00:00 2024-03-22T06:01:38.886 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2024-03-22T06:01:38.919 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-22T06:01:40.410 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.0 MB 00:01 2024-03-22T06:01:56.582 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 474 kB/s | 8.0 MB 00:17 2024-03-22T06:01:58.820 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-03-22T06:02:02.565 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 914 kB/s | 19 MB 00:21 2024-03-22T06:02:03.452 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.1 MB 00:01 2024-03-22T06:02:04.945 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-03-22T06:02:06.013 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-03-22T06:02:08.449 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.1 MB 00:02 2024-03-22T06:02:10.941 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-03-22T06:02:11.338 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-22T06:02:12.919 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-22T06:02:13.861 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:02:13.879 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:02:13.879 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-22T06:02:13.879 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 14 M 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: librados2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.1 M 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 5.0 M 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 23 M 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 636 k 2024-03-22T06:02:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 457 k 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 4.9 M 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 45 k 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 117 k 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 156 k 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-22T06:02:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 320 k 2024-03-22T06:02:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 293 k 2024-03-22T06:02:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 99 k 2024-03-22T06:02:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-22T06:02:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:13.882 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:02:13.882 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:02:13.882 INFO:teuthology.orchestra.run.smithi006.stdout:Install 22 Packages 2024-03-22T06:02:13.882 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 2 Packages 2024-03-22T06:02:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:13.883 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 58 M 2024-03-22T06:02:13.883 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:02:14.715 INFO:teuthology.orchestra.run.smithi006.stdout:(1/24): ceph-base-17.2.7-726.g9c4e0c34.el9.x86_ 7.7 MB/s | 5.0 MB 00:00 2024-03-22T06:02:14.765 INFO:teuthology.orchestra.run.smithi006.stdout:(2/24): ceph-selinux-17.2.7-726.g9c4e0c34.el9.x 503 kB/s | 25 kB 00:00 2024-03-22T06:02:14.857 INFO:teuthology.orchestra.run.smithi006.stdout:(3/24): libcephfs2-17.2.7-726.g9c4e0c34.el9.x86 6.8 MB/s | 636 kB 00:00 2024-03-22T06:02:14.941 INFO:teuthology.orchestra.run.smithi006.stdout:(4/24): libradosstriper1-17.2.7-726.g9c4e0c34.e 5.4 MB/s | 457 kB 00:00 2024-03-22T06:02:15.341 INFO:teuthology.orchestra.run.smithi006.stdout:(5/24): ceph-radosgw-17.2.7-726.g9c4e0c34.el9.x 11 MB/s | 14 MB 00:01 2024-03-22T06:02:15.416 INFO:teuthology.orchestra.run.smithi006.stdout:(6/24): librgw2-17.2.7-726.g9c4e0c34.el9.x86_64 10 MB/s | 4.9 MB 00:00 2024-03-22T06:02:15.433 INFO:teuthology.orchestra.run.smithi006.stdout:(7/24): python3-ceph-argparse-17.2.7-726.g9c4e0 491 kB/s | 45 kB 00:00 2024-03-22T06:02:15.458 INFO:teuthology.orchestra.run.smithi006.stdout:(8/24): python3-ceph-common-17.2.7-726.g9c4e0c3 2.8 MB/s | 117 kB 00:00 2024-03-22T06:02:15.484 INFO:teuthology.orchestra.run.smithi006.stdout:(9/24): python3-cephfs-17.2.7-726.g9c4e0c34.el9 3.0 MB/s | 156 kB 00:00 2024-03-22T06:02:15.517 INFO:teuthology.orchestra.run.smithi006.stdout:(10/24): python3-rados-17.2.7-726.g9c4e0c34.el9 5.3 MB/s | 320 kB 00:00 2024-03-22T06:02:15.543 INFO:teuthology.orchestra.run.smithi006.stdout:(11/24): python3-rbd-17.2.7-726.g9c4e0c34.el9.x 4.9 MB/s | 293 kB 00:00 2024-03-22T06:02:15.568 INFO:teuthology.orchestra.run.smithi006.stdout:(12/24): python3-rgw-17.2.7-726.g9c4e0c34.el9.x 1.9 MB/s | 99 kB 00:00 2024-03-22T06:02:15.685 INFO:teuthology.orchestra.run.smithi006.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 378 kB/s | 44 kB 00:00 2024-03-22T06:02:15.710 INFO:teuthology.orchestra.run.smithi006.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 952 kB/s | 159 kB 00:00 2024-03-22T06:02:15.769 INFO:teuthology.orchestra.run.smithi006.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-03-22T06:02:15.794 INFO:teuthology.orchestra.run.smithi006.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-03-22T06:02:15.811 INFO:teuthology.orchestra.run.smithi006.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 991 kB/s | 41 kB 00:00 2024-03-22T06:02:15.961 INFO:teuthology.orchestra.run.smithi006.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 323 kB/s | 49 kB 00:00 2024-03-22T06:02:16.061 INFO:teuthology.orchestra.run.smithi006.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.1 MB/s | 309 kB 00:00 2024-03-22T06:02:16.087 INFO:teuthology.orchestra.run.smithi006.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 537 kB/s | 67 kB 00:00 2024-03-22T06:02:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-03-22T06:02:16.337 INFO:teuthology.orchestra.run.smithi006.stdout:(22/24): librados2-17.2.7-726.g9c4e0c34.el9.x86 12 MB/s | 3.1 MB 00:00 2024-03-22T06:02:16.529 INFO:teuthology.orchestra.run.smithi006.stdout:(23/24): librbd1-17.2.7-726.g9c4e0c34.el9.x86_6 9.5 MB/s | 2.9 MB 00:00 2024-03-22T06:02:17.105 INFO:teuthology.orchestra.run.smithi006.stdout:(24/24): ceph-common-17.2.7-726.g9c4e0c34.el9.x 7.4 MB/s | 23 MB 00:03 2024-03-22T06:02:17.109 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:02:17.109 INFO:teuthology.orchestra.run.smithi006.stdout:Total 18 MB/s | 58 MB 00:03 2024-03-22T06:02:17.109 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:02:17.143 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:02:17.143 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:02:17.618 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:02:17.618 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:02:17.875 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-22T06:02:18.884 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:02:19.224 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-22T06:02:19.298 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:02:19.391 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:02:19.480 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T06:02:19.509 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:02:19.597 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:02:19.649 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-22T06:02:19.680 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-22T06:02:20.135 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-22T06:02:20.161 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T06:02:20.215 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T06:02:20.244 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-22T06:02:20.287 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 10/26 2024-03-22T06:02:20.324 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T06:02:20.327 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:02:20.330 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 14 M 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.1 M 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 5.0 M 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 23 M 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-22T06:02:20.331 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 636 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 457 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 4.9 M 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 45 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 117 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 156 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-22T06:02:20.332 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 320 k 2024-03-22T06:02:20.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 293 k 2024-03-22T06:02:20.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 99 k 2024-03-22T06:02:20.333 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-22T06:02:20.333 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:20.333 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:02:20.333 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:02:20.333 INFO:teuthology.orchestra.run.smithi117.stdout:Install 22 Packages 2024-03-22T06:02:20.333 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 2 Packages 2024-03-22T06:02:20.333 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:20.334 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 58 M 2024-03-22T06:02:20.334 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:02:20.389 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T06:02:20.445 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:02:20.496 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:02:20.536 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-22T06:02:20.581 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-22T06:02:20.613 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-22T06:02:20.909 INFO:teuthology.orchestra.run.smithi117.stdout:(1/24): ceph-base-17.2.7-726.g9c4e0c34.el9.x86_ 9.6 MB/s | 5.0 MB 00:00 2024-03-22T06:02:20.914 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-22T06:02:20.943 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T06:02:20.951 INFO:teuthology.orchestra.run.smithi117.stdout:(2/24): ceph-selinux-17.2.7-726.g9c4e0c34.el9.x 605 kB/s | 25 kB 00:00 2024-03-22T06:02:21.025 INFO:teuthology.orchestra.run.smithi117.stdout:(3/24): libcephfs2-17.2.7-726.g9c4e0c34.el9.x86 9.3 MB/s | 636 kB 00:00 2024-03-22T06:02:21.027 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T06:02:21.068 INFO:teuthology.orchestra.run.smithi117.stdout:(4/24): libradosstriper1-17.2.7-726.g9c4e0c34.e 8.9 MB/s | 457 kB 00:00 2024-03-22T06:02:21.101 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 19/26 2024-03-22T06:02:21.129 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 20/26 2024-03-22T06:02:21.418 INFO:teuthology.orchestra.run.smithi117.stdout:(5/24): librgw2-17.2.7-726.g9c4e0c34.el9.x86_64 14 MB/s | 4.9 MB 00:00 2024-03-22T06:02:21.468 INFO:teuthology.orchestra.run.smithi117.stdout:(6/24): python3-ceph-argparse-17.2.7-726.g9c4e0 904 kB/s | 45 kB 00:00 2024-03-22T06:02:21.535 INFO:teuthology.orchestra.run.smithi117.stdout:(7/24): python3-ceph-common-17.2.7-726.g9c4e0c3 1.7 MB/s | 117 kB 00:00 2024-03-22T06:02:21.594 INFO:teuthology.orchestra.run.smithi117.stdout:(8/24): python3-cephfs-17.2.7-726.g9c4e0c34.el9 2.6 MB/s | 156 kB 00:00 2024-03-22T06:02:21.669 INFO:teuthology.orchestra.run.smithi117.stdout:(9/24): python3-rados-17.2.7-726.g9c4e0c34.el9. 4.2 MB/s | 320 kB 00:00 2024-03-22T06:02:21.736 INFO:teuthology.orchestra.run.smithi117.stdout:(10/24): python3-rbd-17.2.7-726.g9c4e0c34.el9.x 4.3 MB/s | 293 kB 00:00 2024-03-22T06:02:21.786 INFO:teuthology.orchestra.run.smithi117.stdout:(11/24): python3-rgw-17.2.7-726.g9c4e0c34.el9.x 1.9 MB/s | 99 kB 00:00 2024-03-22T06:02:21.904 INFO:teuthology.orchestra.run.smithi117.stdout:(12/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-03-22T06:02:21.929 INFO:teuthology.orchestra.run.smithi117.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.7 MB/s | 44 kB 00:00 2024-03-22T06:02:22.004 INFO:teuthology.orchestra.run.smithi117.stdout:(14/24): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-22T06:02:22.046 INFO:teuthology.orchestra.run.smithi117.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-22T06:02:22.071 INFO:teuthology.orchestra.run.smithi117.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.6 MB/s | 41 kB 00:00 2024-03-22T06:02:22.280 INFO:teuthology.orchestra.run.smithi117.stdout:(17/24): ceph-radosgw-17.2.7-726.g9c4e0c34.el9. 7.4 MB/s | 14 MB 00:01 2024-03-22T06:02:22.380 INFO:teuthology.orchestra.run.smithi117.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-03-22T06:02:22.414 INFO:teuthology.orchestra.run.smithi117.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 902 kB/s | 309 kB 00:00 2024-03-22T06:02:22.447 INFO:teuthology.orchestra.run.smithi117.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-22T06:02:22.706 INFO:teuthology.orchestra.run.smithi117.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-03-22T06:02:22.756 INFO:teuthology.orchestra.run.smithi117.stdout:(22/24): librados2-17.2.7-726.g9c4e0c34.el9.x86 10 MB/s | 3.1 MB 00:00 2024-03-22T06:02:23.081 INFO:teuthology.orchestra.run.smithi117.stdout:(23/24): ceph-common-17.2.7-726.g9c4e0c34.el9.x 8.4 MB/s | 23 MB 00:02 2024-03-22T06:02:23.173 INFO:teuthology.orchestra.run.smithi117.stdout:(24/24): librbd1-17.2.7-726.g9c4e0c34.el9.x86_6 6.3 MB/s | 2.9 MB 00:00 2024-03-22T06:02:23.176 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:02:23.176 INFO:teuthology.orchestra.run.smithi117.stdout:Total 20 MB/s | 58 MB 00:02 2024-03-22T06:02:23.176 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:02:23.206 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:02:23.206 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:02:23.663 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:02:23.664 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:02:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:02:23.708 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:02:24.275 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:02:24.320 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T06:02:24.408 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T06:02:24.408 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-22T06:02:24.408 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-22T06:02:24.408 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:24.446 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:02:24.901 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:02:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-22T06:02:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:02:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:02:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T06:02:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:02:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:02:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-22T06:02:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-22T06:02:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-22T06:02:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T06:02:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T06:02:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-22T06:02:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 10/26 2024-03-22T06:02:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T06:02:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T06:02:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:02:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:02:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-22T06:02:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-22T06:02:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-22T06:02:26.849 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-22T06:02:26.877 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T06:02:26.950 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T06:02:27.022 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 19/26 2024-03-22T06:02:27.046 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 20/26 2024-03-22T06:02:29.450 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:02:29.484 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:02:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:02:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T06:02:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T06:02:30.148 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-22T06:02:30.148 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-22T06:02:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:30.188 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:02:34.590 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:02:34.590 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /sys 2024-03-22T06:02:34.590 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /proc 2024-03-22T06:02:34.590 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /mnt 2024-03-22T06:02:34.590 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /var/tmp 2024-03-22T06:02:34.590 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /home 2024-03-22T06:02:34.590 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /root 2024-03-22T06:02:34.590 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /tmp 2024-03-22T06:02:34.590 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:35.911 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T06:02:35.985 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T06:02:35.986 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:02:35.986 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-22T06:02:35.986 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T06:02:35.986 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T06:02:35.986 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:35.987 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T06:02:36.072 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T06:02:36.072 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:02:37.333 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:02:37.333 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/26 2024-03-22T06:02:37.333 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:02:37.333 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T06:02:37.333 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:02:37.333 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/26 2024-03-22T06:02:37.333 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/26 2024-03-22T06:02:37.334 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 7/26 2024-03-22T06:02:37.334 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 8/26 2024-03-22T06:02:37.334 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 9/26 2024-03-22T06:02:37.334 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 10/26 2024-03-22T06:02:37.334 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T06:02:37.334 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T06:02:37.334 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:02:37.334 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-22T06:02:37.334 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-22T06:02:37.334 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-22T06:02:37.335 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-22T06:02:37.335 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-22T06:02:37.335 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-22T06:02:37.335 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-22T06:02:37.335 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-22T06:02:37.335 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-22T06:02:37.335 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:02:37.335 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-22T06:02:37.335 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 25/26 2024-03-22T06:02:38.190 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:02:38.190 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-22T06:02:38.191 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:38.192 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:02:38.497 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-test 2024-03-22T06:02:39.069 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:28 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:02:39.222 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:02:39.223 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:02:39.223 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-22T06:02:39.223 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:02:39.223 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:02:39.223 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 48 M 2024-03-22T06:02:39.223 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-22T06:02:39.223 INFO:teuthology.orchestra.run.smithi006.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-22T06:02:39.223 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 163 k 2024-03-22T06:02:39.224 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-22T06:02:39.224 INFO:teuthology.orchestra.run.smithi006.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-22T06:02:39.224 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-22T06:02:39.224 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:39.224 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:02:39.224 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:02:39.224 INFO:teuthology.orchestra.run.smithi006.stdout:Install 6 Packages 2024-03-22T06:02:39.224 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:39.225 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 49 M 2024-03-22T06:02:39.225 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 196 M 2024-03-22T06:02:39.225 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:02:39.505 INFO:teuthology.orchestra.run.smithi006.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.3 MB/s | 217 kB 00:00 2024-03-22T06:02:39.580 INFO:teuthology.orchestra.run.smithi006.stdout:(2/6): libcephsqlite-17.2.7-726.g9c4e0c34.el9.x 694 kB/s | 163 kB 00:00 2024-03-22T06:02:39.605 INFO:teuthology.orchestra.run.smithi006.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.9 MB/s | 300 kB 00:00 2024-03-22T06:02:39.622 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:02:39.622 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-03-22T06:02:39.622 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-03-22T06:02:39.622 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-03-22T06:02:39.622 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-03-22T06:02:39.622 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-03-22T06:02:39.622 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-03-22T06:02:39.622 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-03-22T06:02:39.622 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:39.630 INFO:teuthology.orchestra.run.smithi006.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-22T06:02:39.831 INFO:teuthology.orchestra.run.smithi006.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 831 kB/s | 187 kB 00:00 2024-03-22T06:02:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T06:02:40.976 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T06:02:40.977 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:02:40.977 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-22T06:02:40.977 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T06:02:40.977 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T06:02:40.977 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:40.978 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T06:02:41.044 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T06:02:41.044 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:02:41.989 INFO:teuthology.orchestra.run.smithi006.stdout:(6/6): ceph-test-17.2.7-726.g9c4e0c34.el9.x86_6 18 MB/s | 48 MB 00:02 2024-03-22T06:02:41.992 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:02:41.993 INFO:teuthology.orchestra.run.smithi006.stdout:Total 18 MB/s | 49 MB 00:02 2024-03-22T06:02:42.007 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:02:42.029 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:02:42.029 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:02:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:02:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/26 2024-03-22T06:02:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:02:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T06:02:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:02:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/26 2024-03-22T06:02:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/26 2024-03-22T06:02:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 7/26 2024-03-22T06:02:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 8/26 2024-03-22T06:02:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 9/26 2024-03-22T06:02:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 10/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-22T06:02:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-22T06:02:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:02:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-22T06:02:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 25/26 2024-03-22T06:02:42.429 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:02:42.430 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:02:43.049 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-22T06:02:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.053 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:43.053 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-22T06:02:43.053 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:43.053 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:02:43.132 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-22T06:02:43.165 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-22T06:02:43.198 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T06:02:43.240 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-22T06:02:43.267 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T06:02:43.386 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-test 2024-03-22T06:02:43.944 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:26 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:02:44.094 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:02:44.095 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:02:44.095 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-22T06:02:44.095 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:02:44.095 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:02:44.095 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 48 M 2024-03-22T06:02:44.095 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-22T06:02:44.095 INFO:teuthology.orchestra.run.smithi117.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-22T06:02:44.095 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 163 k 2024-03-22T06:02:44.095 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-22T06:02:44.096 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-22T06:02:44.096 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-22T06:02:44.096 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:44.096 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:02:44.096 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:02:44.096 INFO:teuthology.orchestra.run.smithi117.stdout:Install 6 Packages 2024-03-22T06:02:44.096 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:44.096 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 49 M 2024-03-22T06:02:44.097 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 196 M 2024-03-22T06:02:44.097 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:02:44.366 INFO:teuthology.orchestra.run.smithi117.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.4 MB/s | 217 kB 00:00 2024-03-22T06:02:44.433 INFO:teuthology.orchestra.run.smithi117.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2024-03-22T06:02:44.458 INFO:teuthology.orchestra.run.smithi117.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.5 MB/s | 63 kB 00:00 2024-03-22T06:02:44.516 INFO:teuthology.orchestra.run.smithi117.stdout:(4/6): libcephsqlite-17.2.7-726.g9c4e0c34.el9.x 534 kB/s | 163 kB 00:00 2024-03-22T06:02:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-03-22T06:02:46.583 INFO:teuthology.orchestra.run.smithi117.stdout:(6/6): ceph-test-17.2.7-726.g9c4e0c34.el9.x86_6 20 MB/s | 48 MB 00:02 2024-03-22T06:02:46.586 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:02:46.587 INFO:teuthology.orchestra.run.smithi117.stdout:Total 20 MB/s | 49 MB 00:02 2024-03-22T06:02:46.601 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:02:46.621 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:02:46.621 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:02:47.001 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:02:47.001 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:02:47.567 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:02:47.650 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-22T06:02:47.684 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-22T06:02:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T06:02:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-22T06:02:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T06:02:47.950 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T06:02:48.027 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T06:02:50.259 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T06:02:50.259 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/6 2024-03-22T06:02:50.259 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/6 2024-03-22T06:02:50.259 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T06:02:50.259 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-22T06:02:50.259 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-22T06:02:50.673 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-22T06:02:50.674 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:50.674 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:02:50.674 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:50.674 INFO:teuthology.orchestra.run.smithi006.stdout: jq-1.6-16.el9.x86_64 2024-03-22T06:02:50.674 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:50.674 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-22T06:02:50.674 INFO:teuthology.orchestra.run.smithi006.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-22T06:02:50.674 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-22T06:02:50.674 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:50.674 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:02:50.951 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph 2024-03-22T06:02:51.533 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:40 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:02:51.702 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:02:51.705 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2024-03-22T06:02:51.705 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-22T06:02:51.705 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2024-03-22T06:02:51.705 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:02:51.705 INFO:teuthology.orchestra.run.smithi006.stdout: ceph x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 6.5 k 2024-03-22T06:02:51.705 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-22T06:02:51.705 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.1 M 2024-03-22T06:02:51.705 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 1.4 M 2024-03-22T06:02:51.705 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 240 k 2024-03-22T06:02:51.705 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.7 M 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 15 M 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-22T06:02:51.706 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-22T06:02:51.707 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-22T06:02:51.708 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-22T06:02:51.708 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:51.708 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:02:51.708 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2024-03-22T06:02:51.708 INFO:teuthology.orchestra.run.smithi006.stdout:Install 33 Packages 2024-03-22T06:02:51.708 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:51.708 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 27 M 2024-03-22T06:02:51.709 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 100 M 2024-03-22T06:02:51.709 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:02:52.159 INFO:teuthology.orchestra.run.smithi006.stdout:(1/33): ceph-17.2.7-726.g9c4e0c34.el9.x86_64.rp 38 kB/s | 6.5 kB 00:00 2024-03-22T06:02:52.173 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T06:02:52.242 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T06:02:52.334 INFO:teuthology.orchestra.run.smithi006.stdout:(2/33): ceph-mgr-17.2.7-726.g9c4e0c34.el9.x86_6 4.1 MB/s | 1.4 MB 00:00 2024-03-22T06:02:52.609 INFO:teuthology.orchestra.run.smithi006.stdout:(3/33): ceph-mon-17.2.7-726.g9c4e0c34.el9.x86_6 8.2 MB/s | 3.7 MB 00:00 2024-03-22T06:02:52.718 INFO:teuthology.orchestra.run.smithi006.stdout:(4/33): ceph-mgr-modules-core-17.2.7-726.g9c4e0 2.2 MB/s | 240 kB 00:00 2024-03-22T06:02:52.835 INFO:teuthology.orchestra.run.smithi006.stdout:(5/33): ceph-mds-17.2.7-726.g9c4e0c34.el9.x86_6 2.4 MB/s | 2.1 MB 00:00 2024-03-22T06:02:52.860 INFO:teuthology.orchestra.run.smithi006.stdout:(6/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 144 kB 00:00 2024-03-22T06:02:52.878 INFO:teuthology.orchestra.run.smithi006.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 757 kB/s | 32 kB 00:00 2024-03-22T06:02:52.894 INFO:teuthology.orchestra.run.smithi006.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-22T06:02:52.911 INFO:teuthology.orchestra.run.smithi006.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-22T06:02:52.945 INFO:teuthology.orchestra.run.smithi006.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-22T06:02:53.004 INFO:teuthology.orchestra.run.smithi006.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2024-03-22T06:02:53.028 INFO:teuthology.orchestra.run.smithi006.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-22T06:02:53.054 INFO:teuthology.orchestra.run.smithi006.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-22T06:02:53.079 INFO:teuthology.orchestra.run.smithi006.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-22T06:02:53.263 INFO:teuthology.orchestra.run.smithi006.stdout:(15/33): ceph-osd-17.2.7-726.g9c4e0c34.el9.x86_ 17 MB/s | 15 MB 00:00 2024-03-22T06:02:53.280 INFO:teuthology.orchestra.run.smithi006.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 217 kB/s | 43 kB 00:00 2024-03-22T06:02:53.305 INFO:teuthology.orchestra.run.smithi006.stdout:(17/33): python3-requests-2.25.1-8.el9.noarch.r 498 kB/s | 125 kB 00:00 2024-03-22T06:02:53.330 INFO:teuthology.orchestra.run.smithi006.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-03-22T06:02:53.355 INFO:teuthology.orchestra.run.smithi006.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-03-22T06:02:53.381 INFO:teuthology.orchestra.run.smithi006.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-03-22T06:02:53.406 INFO:teuthology.orchestra.run.smithi006.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-03-22T06:02:53.432 INFO:teuthology.orchestra.run.smithi006.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-03-22T06:02:53.457 INFO:teuthology.orchestra.run.smithi006.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-03-22T06:02:53.482 INFO:teuthology.orchestra.run.smithi006.stdout:(24/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-03-22T06:02:53.507 INFO:teuthology.orchestra.run.smithi006.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-03-22T06:02:53.532 INFO:teuthology.orchestra.run.smithi006.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-03-22T06:02:53.558 INFO:teuthology.orchestra.run.smithi006.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-03-22T06:02:53.583 INFO:teuthology.orchestra.run.smithi006.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-03-22T06:02:53.609 INFO:teuthology.orchestra.run.smithi006.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 709 kB/s | 36 kB 00:00 2024-03-22T06:02:53.634 INFO:teuthology.orchestra.run.smithi006.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-03-22T06:02:53.659 INFO:teuthology.orchestra.run.smithi006.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-22T06:02:53.684 INFO:teuthology.orchestra.run.smithi006.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-03-22T06:02:53.709 INFO:teuthology.orchestra.run.smithi006.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-03-22T06:02:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:02:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:Total 13 MB/s | 27 MB 00:02 2024-03-22T06:02:53.726 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:02:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:02:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:02:54.142 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:02:54.142 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:02:54.552 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T06:02:54.553 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/6 2024-03-22T06:02:54.553 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/6 2024-03-22T06:02:54.553 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T06:02:54.553 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-22T06:02:54.553 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-22T06:02:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-22T06:02:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:54.947 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:02:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: jq-1.6-16.el9.x86_64 2024-03-22T06:02:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:02:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-22T06:02:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-22T06:02:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-22T06:02:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:54.948 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:02:55.234 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph 2024-03-22T06:02:55.370 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:02:55.439 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-22T06:02:55.472 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-22T06:02:55.505 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-22T06:02:55.599 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-22T06:02:55.806 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:38 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:02:55.819 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-22T06:02:55.883 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-22T06:02:55.957 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-22T06:02:55.962 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:02:55.964 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================== 2024-03-22T06:02:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-22T06:02:55.964 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================== 2024-03-22T06:02:55.964 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:02:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 6.5 k 2024-03-22T06:02:55.964 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-22T06:02:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.1 M 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 1.4 M 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 240 k 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.7 M 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 15 M 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-22T06:02:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-22T06:02:55.966 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-22T06:02:55.967 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-22T06:02:55.967 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-22T06:02:55.967 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-22T06:02:55.967 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-22T06:02:55.967 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:55.967 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:02:55.967 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================== 2024-03-22T06:02:55.967 INFO:teuthology.orchestra.run.smithi117.stdout:Install 33 Packages 2024-03-22T06:02:55.967 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:02:55.968 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 27 M 2024-03-22T06:02:55.968 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 100 M 2024-03-22T06:02:55.968 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:02:56.009 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-22T06:02:56.046 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-22T06:02:56.096 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-22T06:02:56.158 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-22T06:02:56.357 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-22T06:02:56.409 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-22T06:02:56.448 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-22T06:02:56.482 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-22T06:02:56.512 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-22T06:02:56.543 INFO:teuthology.orchestra.run.smithi117.stdout:(1/33): ceph-17.2.7-726.g9c4e0c34.el9.x86_64.rp 25 kB/s | 6.5 kB 00:00 2024-03-22T06:02:56.591 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-22T06:02:56.622 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T06:02:56.656 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-22T06:02:56.668 INFO:teuthology.orchestra.run.smithi117.stdout:(2/33): ceph-mds-17.2.7-726.g9c4e0c34.el9.x86_6 5.4 MB/s | 2.1 MB 00:00 2024-03-22T06:02:56.688 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-22T06:02:56.731 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-22T06:02:56.765 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-22T06:02:56.858 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-22T06:02:56.897 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-22T06:02:56.971 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-22T06:02:57.112 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-22T06:02:57.160 INFO:teuthology.orchestra.run.smithi117.stdout:(3/33): ceph-mgr-17.2.7-726.g9c4e0c34.el9.x86_6 1.6 MB/s | 1.4 MB 00:00 2024-03-22T06:02:57.244 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T06:02:57.285 INFO:teuthology.orchestra.run.smithi117.stdout:(4/33): ceph-mgr-modules-core-17.2.7-726.g9c4e0 1.9 MB/s | 240 kB 00:00 2024-03-22T06:02:57.401 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 28/33 2024-03-22T06:02:57.426 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T06:02:57.451 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T06:02:57.451 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:02:57.452 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-22T06:02:57.452 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T06:02:57.452 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T06:02:57.452 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:57.469 INFO:teuthology.orchestra.run.smithi117.stdout:(5/33): ceph-osd-17.2.7-726.g9c4e0c34.el9.x86_6 19 MB/s | 15 MB 00:00 2024-03-22T06:02:57.511 INFO:teuthology.orchestra.run.smithi117.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 786 kB/s | 32 kB 00:00 2024-03-22T06:02:57.536 INFO:teuthology.orchestra.run.smithi117.stdout:(7/33): python3-mako-1.1.4-6.el9.noarch.rpm 575 kB/s | 144 kB 00:00 2024-03-22T06:02:57.561 INFO:teuthology.orchestra.run.smithi117.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 944 kB/s | 47 kB 00:00 2024-03-22T06:02:57.586 INFO:teuthology.orchestra.run.smithi117.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 772 kB/s | 39 kB 00:00 2024-03-22T06:02:57.612 INFO:teuthology.orchestra.run.smithi117.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-22T06:02:57.654 INFO:teuthology.orchestra.run.smithi117.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-03-22T06:02:57.679 INFO:teuthology.orchestra.run.smithi117.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-22T06:02:57.713 INFO:teuthology.orchestra.run.smithi117.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.1 MB/s | 124 kB 00:00 2024-03-22T06:02:57.738 INFO:teuthology.orchestra.run.smithi117.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 187 kB 00:00 2024-03-22T06:02:57.855 INFO:teuthology.orchestra.run.smithi117.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 882 kB/s | 125 kB 00:00 2024-03-22T06:02:58.063 INFO:teuthology.orchestra.run.smithi117.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 134 kB/s | 43 kB 00:00 2024-03-22T06:02:58.330 INFO:teuthology.orchestra.run.smithi117.stdout:(17/33): ceph-mon-17.2.7-726.g9c4e0c34.el9.x86_ 2.1 MB/s | 3.7 MB 00:01 2024-03-22T06:02:58.422 INFO:teuthology.orchestra.run.smithi117.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 303 kB/s | 172 kB 00:00 2024-03-22T06:02:58.447 INFO:teuthology.orchestra.run.smithi117.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 934 kB/s | 358 kB 00:00 2024-03-22T06:02:58.514 INFO:teuthology.orchestra.run.smithi117.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 58 kB/s | 11 kB 00:00 2024-03-22T06:02:58.531 INFO:teuthology.orchestra.run.smithi117.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 162 kB/s | 18 kB 00:00 2024-03-22T06:02:58.548 INFO:teuthology.orchestra.run.smithi117.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 230 kB/s | 23 kB 00:00 2024-03-22T06:02:58.657 INFO:teuthology.orchestra.run.smithi117.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 157 kB/s | 20 kB 00:00 2024-03-22T06:02:58.674 INFO:teuthology.orchestra.run.smithi117.stdout:(24/33): python3-jaraco-functools-3.5.0-2.el9.n 122 kB/s | 19 kB 00:00 2024-03-22T06:02:58.691 INFO:teuthology.orchestra.run.smithi117.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 326 kB/s | 46 kB 00:00 2024-03-22T06:02:58.783 INFO:teuthology.orchestra.run.smithi117.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 628 kB/s | 79 kB 00:00 2024-03-22T06:02:58.799 INFO:teuthology.orchestra.run.smithi117.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 151 kB/s | 16 kB 00:00 2024-03-22T06:02:58.860 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T06:02:58.886 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T06:02:58.886 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:02:58.886 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-22T06:02:58.886 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T06:02:58.886 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T06:02:58.886 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:58.900 INFO:teuthology.orchestra.run.smithi117.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 357 kB/s | 36 kB 00:00 2024-03-22T06:02:58.925 INFO:teuthology.orchestra.run.smithi117.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 632 kB/s | 90 kB 00:00 2024-03-22T06:02:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 756 kB/s | 272 kB 00:00 2024-03-22T06:02:59.059 INFO:teuthology.orchestra.run.smithi117.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-03-22T06:02:59.126 INFO:teuthology.orchestra.run.smithi117.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 218 kB/s | 20 kB 00:00 2024-03-22T06:02:59.176 INFO:teuthology.orchestra.run.smithi117.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.7 MB/s | 427 kB 00:00 2024-03-22T06:02:59.180 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:02:59.180 INFO:teuthology.orchestra.run.smithi117.stdout:Total 8.3 MB/s | 27 MB 00:03 2024-03-22T06:02:59.194 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:02:59.233 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:02:59.233 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:02:59.286 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T06:02:59.309 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T06:02:59.309 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:02:59.309 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-22T06:02:59.309 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T06:02:59.309 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T06:02:59.309 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:59.520 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T06:02:59.542 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T06:02:59.542 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:02:59.543 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-22T06:02:59.543 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T06:02:59.543 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T06:02:59.543 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:02:59.566 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:02:59.567 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:02:59.620 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T06:03:00.837 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:00.896 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-22T06:03:00.928 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-22T06:03:00.961 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-22T06:03:01.053 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-22T06:03:01.130 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-22T06:03:01.180 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-22T06:03:01.199 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T06:03:01.199 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/33 2024-03-22T06:03:01.199 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/33 2024-03-22T06:03:01.199 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/33 2024-03-22T06:03:01.199 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/33 2024-03-22T06:03:01.199 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/33 2024-03-22T06:03:01.199 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 6/33 2024-03-22T06:03:01.199 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-22T06:03:01.199 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-22T06:03:01.199 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-22T06:03:01.199 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-22T06:03:01.200 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-22T06:03:01.201 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-22T06:03:01.201 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T06:03:01.201 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-22T06:03:01.201 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-22T06:03:01.201 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-22T06:03:01.201 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-22T06:03:01.201 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-22T06:03:01.260 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-22T06:03:01.314 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-22T06:03:01.352 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-22T06:03:01.401 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-22T06:03:01.469 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-22T06:03:01.656 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-22T06:03:01.698 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-22T06:03:01.744 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-22T06:03:01.780 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-22T06:03:01.818 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-22T06:03:01.894 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-22T06:03:01.928 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T06:03:01.962 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-22T06:03:02.002 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-22T06:03:02.046 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-22T06:03:02.081 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-22T06:03:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-22T06:03:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:02.156 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:02.156 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-22T06:03:02.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply-3.11-14.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-22T06:03:02.158 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-22T06:03:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-22T06:03:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-22T06:03:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-22T06:03:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-22T06:03:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-22T06:03:02.159 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:02.159 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:02.170 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-22T06:03:02.211 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-22T06:03:02.284 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-22T06:03:02.428 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-22T06:03:02.484 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-base 2024-03-22T06:03:02.555 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T06:03:02.718 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 28/33 2024-03-22T06:03:02.755 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T06:03:02.774 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T06:03:02.774 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:03:02.774 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-22T06:03:02.774 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T06:03:02.774 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T06:03:02.774 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:03.063 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:52 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:03.162 INFO:teuthology.orchestra.run.smithi006.stdout:Package ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:03.216 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:03.217 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-22T06:03:03.218 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:03.277 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install cephadm 2024-03-22T06:03:03.848 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:52 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:04.000 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:04.001 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:04.002 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-22T06:03:04.002 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:04.002 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:04.002 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 82 k 2024-03-22T06:03:04.002 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:04.002 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:04.002 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:04.002 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-22T06:03:04.002 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:04.002 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 82 k 2024-03-22T06:03:04.003 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 372 k 2024-03-22T06:03:04.003 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:04.143 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T06:03:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T06:03:04.171 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:03:04.171 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-22T06:03:04.171 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T06:03:04.171 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T06:03:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:04.277 INFO:teuthology.orchestra.run.smithi006.stdout:cephadm-17.2.7-726.g9c4e0c34.el9.noarch.rpm 299 kB/s | 82 kB 00:00 2024-03-22T06:03:04.278 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:04.278 INFO:teuthology.orchestra.run.smithi006.stdout:Total 297 kB/s | 82 kB 00:00 2024-03-22T06:03:04.278 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:04.283 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:04.283 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:04.285 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:04.285 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:04.355 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:04.544 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T06:03:04.566 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T06:03:04.566 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:03:04.566 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-22T06:03:04.566 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T06:03:04.566 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T06:03:04.566 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:04.761 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T06:03:04.782 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:03:04.783 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T06:03:04.783 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:03:04.783 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-22T06:03:04.783 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T06:03:04.783 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T06:03:04.784 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:04.849 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T06:03:04.884 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:03:05.220 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:03:05.514 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:03:05.514 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:05.514 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:05.515 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:05.515 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:05.515 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:05.724 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-immutable-object-cache 2024-03-22T06:03:06.300 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:55 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:06.455 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 137 k 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 137 k 2024-03-22T06:03:06.456 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 379 k 2024-03-22T06:03:06.457 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:06.467 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T06:03:06.467 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/33 2024-03-22T06:03:06.467 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/33 2024-03-22T06:03:06.467 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/33 2024-03-22T06:03:06.467 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 6/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-22T06:03:06.468 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-22T06:03:06.469 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-22T06:03:06.819 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-immutable-object-cache-17.2.7-726.g9c4e0c3 379 kB/s | 137 kB 00:00 2024-03-22T06:03:06.820 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:06.820 INFO:teuthology.orchestra.run.smithi006.stdout:Total 378 kB/s | 137 kB 00:00 2024-03-22T06:03:06.820 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:06.828 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:06.828 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:06.881 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:06.881 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:07.024 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:07.084 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:03:07.107 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:03:07.108 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:03:07.108 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-22T06:03:07.108 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:07.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-22T06:03:07.639 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:07.639 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:07.639 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:07.639 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:07.639 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:07.639 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:07.639 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:07.639 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:07.639 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-03-22T06:03:07.640 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:07.641 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:07.767 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-base 2024-03-22T06:03:08.056 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:03:08.056 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:08.057 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:08.057 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:08.057 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:08.323 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr 2024-03-22T06:03:08.328 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:51 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:08.425 INFO:teuthology.orchestra.run.smithi117.stdout:Package ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:08.478 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:08.479 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-03-22T06:03:08.479 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:08.544 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install cephadm 2024-03-22T06:03:08.900 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:57 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:08.998 INFO:teuthology.orchestra.run.smithi006.stdout:Package ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:09.055 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:09.056 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-22T06:03:09.056 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:09.104 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:52 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:09.113 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-dashboard 2024-03-22T06:03:09.251 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:09.252 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:09.252 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-22T06:03:09.252 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:09.252 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:09.252 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 82 k 2024-03-22T06:03:09.252 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:09.252 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:09.252 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:09.252 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-03-22T06:03:09.252 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:09.252 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 82 k 2024-03-22T06:03:09.253 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 372 k 2024-03-22T06:03:09.253 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:09.438 INFO:teuthology.orchestra.run.smithi117.stdout:cephadm-17.2.7-726.g9c4e0c34.el9.noarch.rpm 442 kB/s | 82 kB 00:00 2024-03-22T06:03:09.439 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:09.439 INFO:teuthology.orchestra.run.smithi117.stdout:Total 439 kB/s | 82 kB 00:00 2024-03-22T06:03:09.439 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:09.528 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:09.681 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:58 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:09.837 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:09.838 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:09.838 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-22T06:03:09.838 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:09.838 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:09.838 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 1.7 M 2024-03-22T06:03:09.838 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-22T06:03:09.838 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 23 k 2024-03-22T06:03:09.839 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 15 k 2024-03-22T06:03:09.839 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-22T06:03:09.839 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-22T06:03:09.839 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-22T06:03:09.839 INFO:teuthology.orchestra.run.smithi006.stdout:Installing weak dependencies: 2024-03-22T06:03:09.839 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-22T06:03:09.839 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:09.839 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:09.839 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:09.839 INFO:teuthology.orchestra.run.smithi006.stdout:Install 7 Packages 2024-03-22T06:03:09.839 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:09.840 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 2.0 M 2024-03-22T06:03:09.840 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 8.1 M 2024-03-22T06:03:09.840 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:09.932 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:03:10.023 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:03:10.170 INFO:teuthology.orchestra.run.smithi006.stdout:(1/7): ceph-prometheus-alerts-17.2.7-726.g9c4e0 73 kB/s | 15 kB 00:00 2024-03-22T06:03:10.212 INFO:teuthology.orchestra.run.smithi006.stdout:(2/7): ceph-grafana-dashboards-17.2.7-726.g9c4e 97 kB/s | 23 kB 00:00 2024-03-22T06:03:10.237 INFO:teuthology.orchestra.run.smithi006.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-03-22T06:03:10.262 INFO:teuthology.orchestra.run.smithi006.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 813 kB/s | 41 kB 00:00 2024-03-22T06:03:10.304 INFO:teuthology.orchestra.run.smithi006.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 459 kB/s | 31 kB 00:00 2024-03-22T06:03:10.329 INFO:teuthology.orchestra.run.smithi006.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-03-22T06:03:10.372 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:03:10.546 INFO:teuthology.orchestra.run.smithi006.stdout:(7/7): ceph-mgr-dashboard-17.2.7-726.g9c4e0c34. 2.9 MB/s | 1.7 MB 00:00 2024-03-22T06:03:10.547 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:10.548 INFO:teuthology.orchestra.run.smithi006.stdout:Total 2.8 MB/s | 2.0 MB 00:00 2024-03-22T06:03:10.548 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:10.556 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:10.556 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:10.623 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:10.624 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:03:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:10.694 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:10.694 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:10.892 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-immutable-object-cache 2024-03-22T06:03:10.925 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:10.989 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-22T06:03:11.023 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-22T06:03:11.079 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-22T06:03:11.118 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-22T06:03:11.147 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 5/7 2024-03-22T06:03:11.386 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 6/7 2024-03-22T06:03:11.425 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T06:03:11.460 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:54 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:11.612 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:11.613 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:11.613 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repo Size 2024-03-22T06:03:11.614 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:11.614 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:11.614 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 137 k 2024-03-22T06:03:11.614 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:11.614 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:11.614 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:11.614 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-03-22T06:03:11.614 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:11.614 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 137 k 2024-03-22T06:03:11.614 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 379 k 2024-03-22T06:03:11.614 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:11.842 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-immutable-object-cache-17.2.7-726.g9c4e0c3 605 kB/s | 137 kB 00:00 2024-03-22T06:03:11.842 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:11.843 INFO:teuthology.orchestra.run.smithi117.stdout:Total 601 kB/s | 137 kB 00:00 2024-03-22T06:03:11.843 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:11.850 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:11.850 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:11.901 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:11.901 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:11.952 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T06:03:11.952 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 1/7 2024-03-22T06:03:11.952 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2/7 2024-03-22T06:03:11.953 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 3/7 2024-03-22T06:03:11.953 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-22T06:03:11.953 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-22T06:03:11.953 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-22T06:03:12.044 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:12.101 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:03:12.125 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:03:12.125 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:03:12.125 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-22T06:03:12.125 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:12.373 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-22T06:03:12.374 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:12.374 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:12.374 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:12.374 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:12.374 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:12.374 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-22T06:03:12.374 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-22T06:03:12.374 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-22T06:03:12.374 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-22T06:03:12.374 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:12.374 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:12.704 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-22T06:03:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:03:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:13.026 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:13.026 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:13.245 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr 2024-03-22T06:03:13.276 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:02 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:13.432 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:13.433 INFO:teuthology.orchestra.run.smithi006.stdout:=========================================================================================== 2024-03-22T06:03:13.433 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-22T06:03:13.433 INFO:teuthology.orchestra.run.smithi006.stdout:=========================================================================================== 2024-03-22T06:03:13.433 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:13.433 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 7.4 M 2024-03-22T06:03:13.433 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-22T06:03:13.433 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout:=========================================================================================== 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout:Install 12 Packages 2024-03-22T06:03:13.434 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:13.435 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 37 M 2024-03-22T06:03:13.435 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 209 M 2024-03-22T06:03:13.435 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:13.720 INFO:teuthology.orchestra.run.smithi006.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 225 kB/s | 32 kB 00:00 2024-03-22T06:03:13.753 INFO:teuthology.orchestra.run.smithi006.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 537 kB/s | 17 kB 00:00 2024-03-22T06:03:13.795 INFO:teuthology.orchestra.run.smithi006.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 849 kB/s | 34 kB 00:00 2024-03-22T06:03:13.816 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:56 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:13.862 INFO:teuthology.orchestra.run.smithi006.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-03-22T06:03:13.915 INFO:teuthology.orchestra.run.smithi117.stdout:Package ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:13.966 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:13.967 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-03-22T06:03:13.968 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:14.034 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-dashboard 2024-03-22T06:03:14.046 INFO:teuthology.orchestra.run.smithi006.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2024-03-22T06:03:14.114 INFO:teuthology.orchestra.run.smithi006.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2024-03-22T06:03:14.138 INFO:teuthology.orchestra.run.smithi006.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-03-22T06:03:14.339 INFO:teuthology.orchestra.run.smithi006.stdout:(8/12): ceph-mgr-diskprediction-local-17.2.7-72 9.8 MB/s | 7.4 MB 00:00 2024-03-22T06:03:14.381 INFO:teuthology.orchestra.run.smithi006.stdout:(9/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 803 kB 00:00 2024-03-22T06:03:14.539 INFO:teuthology.orchestra.run.smithi006.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-03-22T06:03:14.597 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:57 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:14.615 INFO:teuthology.orchestra.run.smithi006.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 715 kB/s | 195 kB 00:00 2024-03-22T06:03:14.744 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:14.745 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:14.745 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-22T06:03:14.745 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:14.745 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:14.745 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 1.7 M 2024-03-22T06:03:14.745 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 23 k 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 15 k 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout:Install 7 Packages 2024-03-22T06:03:14.746 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:14.747 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 2.0 M 2024-03-22T06:03:14.747 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 8.1 M 2024-03-22T06:03:14.747 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:15.031 INFO:teuthology.orchestra.run.smithi006.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 378 kB/s | 245 kB 00:00 2024-03-22T06:03:15.033 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:15.034 INFO:teuthology.orchestra.run.smithi006.stdout:Total 23 MB/s | 37 MB 00:01 2024-03-22T06:03:15.076 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:15.087 INFO:teuthology.orchestra.run.smithi117.stdout:(1/7): ceph-prometheus-alerts-17.2.7-726.g9c4e0 56 kB/s | 15 kB 00:00 2024-03-22T06:03:15.102 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:15.102 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:15.112 INFO:teuthology.orchestra.run.smithi117.stdout:(2/7): ceph-grafana-dashboards-17.2.7-726.g9c4e 81 kB/s | 23 kB 00:00 2024-03-22T06:03:15.246 INFO:teuthology.orchestra.run.smithi117.stdout:(3/7): ceph-mgr-dashboard-17.2.7-726.g9c4e0c34. 4.0 MB/s | 1.7 MB 00:00 2024-03-22T06:03:15.355 INFO:teuthology.orchestra.run.smithi117.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 34 kB/s | 9.0 kB 00:00 2024-03-22T06:03:15.413 INFO:teuthology.orchestra.run.smithi117.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 136 kB/s | 41 kB 00:00 2024-03-22T06:03:15.513 INFO:teuthology.orchestra.run.smithi117.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 116 kB/s | 31 kB 00:00 2024-03-22T06:03:15.557 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:15.557 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 753 kB/s | 188 kB 00:00 2024-03-22T06:03:15.609 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:15.610 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.3 MB/s | 2.0 MB 00:00 2024-03-22T06:03:15.610 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:15.617 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:15.617 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:15.675 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:15.675 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:15.963 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:16.032 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-22T06:03:16.065 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-22T06:03:16.120 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-22T06:03:16.154 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-22T06:03:16.181 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 5/7 2024-03-22T06:03:16.409 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 6/7 2024-03-22T06:03:16.409 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:16.434 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T06:03:16.507 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-22T06:03:16.535 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-22T06:03:16.642 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-22T06:03:16.681 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-22T06:03:16.864 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T06:03:16.896 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T06:03:16.948 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T06:03:16.949 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 1/7 2024-03-22T06:03:16.949 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2/7 2024-03-22T06:03:16.949 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 3/7 2024-03-22T06:03:16.949 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-22T06:03:16.949 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-22T06:03:16.949 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-22T06:03:16.975 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-22T06:03:17.251 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-22T06:03:17.251 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:17.251 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:17.251 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:17.251 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:17.251 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:17.251 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-22T06:03:17.251 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-22T06:03:17.251 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-22T06:03:17.252 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-22T06:03:17.252 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:17.252 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:17.488 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-22T06:03:17.677 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-22T06:03:17.788 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-22T06:03:18.050 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:01 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:18.199 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout:=========================================================================================== 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout:=========================================================================================== 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 7.4 M 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-22T06:03:18.200 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-22T06:03:18.201 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-22T06:03:18.201 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-22T06:03:18.201 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-22T06:03:18.201 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-22T06:03:18.201 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-22T06:03:18.201 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:18.201 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:18.201 INFO:teuthology.orchestra.run.smithi117.stdout:=========================================================================================== 2024-03-22T06:03:18.201 INFO:teuthology.orchestra.run.smithi117.stdout:Install 12 Packages 2024-03-22T06:03:18.201 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:18.202 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 37 M 2024-03-22T06:03:18.202 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 209 M 2024-03-22T06:03:18.202 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:18.529 INFO:teuthology.orchestra.run.smithi117.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 219 kB/s | 32 kB 00:00 2024-03-22T06:03:18.562 INFO:teuthology.orchestra.run.smithi117.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 522 kB/s | 17 kB 00:00 2024-03-22T06:03:18.596 INFO:teuthology.orchestra.run.smithi117.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-03-22T06:03:18.655 INFO:teuthology.orchestra.run.smithi117.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-03-22T06:03:18.822 INFO:teuthology.orchestra.run.smithi117.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2024-03-22T06:03:18.889 INFO:teuthology.orchestra.run.smithi117.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-03-22T06:03:18.914 INFO:teuthology.orchestra.run.smithi117.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-03-22T06:03:19.039 INFO:teuthology.orchestra.run.smithi117.stdout:(8/12): ceph-mgr-diskprediction-local-17.2.7-72 11 MB/s | 7.4 MB 00:00 2024-03-22T06:03:19.231 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-22T06:03:19.281 INFO:teuthology.orchestra.run.smithi117.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-03-22T06:03:19.365 INFO:teuthology.orchestra.run.smithi117.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 605 kB/s | 195 kB 00:00 2024-03-22T06:03:19.448 INFO:teuthology.orchestra.run.smithi117.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-03-22T06:03:19.482 INFO:teuthology.orchestra.run.smithi117.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-03-22T06:03:19.483 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:19.484 INFO:teuthology.orchestra.run.smithi117.stdout:Total 29 MB/s | 37 MB 00:01 2024-03-22T06:03:19.525 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:19.548 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:19.548 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:19.991 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:19.991 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:20.303 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-22T06:03:20.328 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T06:03:20.688 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:20.778 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-22T06:03:20.811 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-22T06:03:20.902 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-22T06:03:20.961 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-22T06:03:21.132 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T06:03:21.163 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T06:03:21.239 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-22T06:03:21.891 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-22T06:03:21.999 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-22T06:03:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T06:03:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 1/12 2024-03-22T06:03:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-22T06:03:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-22T06:03:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-22T06:03:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T06:03:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T06:03:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-22T06:03:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-22T06:03:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-22T06:03:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-22T06:03:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-22T06:03:23.231 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-22T06:03:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-22T06:03:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:23.349 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-22T06:03:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-22T06:03:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-22T06:03:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-22T06:03:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-22T06:03:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-22T06:03:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-22T06:03:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-22T06:03:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-22T06:03:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-22T06:03:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-22T06:03:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:23.350 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:23.683 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-rook 2024-03-22T06:03:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-22T06:03:24.268 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:13 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T06:03:24.431 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:24.432 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-03-22T06:03:24.432 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-22T06:03:24.432 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-03-22T06:03:24.432 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:24.432 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 47 k 2024-03-22T06:03:24.432 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-22T06:03:24.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-03-22T06:03:24.433 INFO:teuthology.orchestra.run.smithi006.stdout:Install 13 Packages 2024-03-22T06:03:24.434 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 2.0 M 2024-03-22T06:03:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 26 M 2024-03-22T06:03:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:24.645 INFO:teuthology.orchestra.run.smithi006.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 191 kB/s | 24 kB 00:00 2024-03-22T06:03:24.670 INFO:teuthology.orchestra.run.smithi006.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 118 kB/s | 18 kB 00:00 2024-03-22T06:03:24.712 INFO:teuthology.orchestra.run.smithi006.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2024-03-22T06:03:24.746 INFO:teuthology.orchestra.run.smithi006.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-22T06:03:24.771 INFO:teuthology.orchestra.run.smithi006.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.5 MB/s | 211 kB 00:00 2024-03-22T06:03:24.797 INFO:teuthology.orchestra.run.smithi006.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 852 kB/s | 43 kB 00:00 2024-03-22T06:03:24.822 INFO:teuthology.orchestra.run.smithi006.stdout:(7/13): ceph-mgr-rook-17.2.7-726.g9c4e0c34.el9. 156 kB/s | 47 kB 00:00 2024-03-22T06:03:25.047 INFO:teuthology.orchestra.run.smithi006.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 60 kB/s | 15 kB 00:00 2024-03-22T06:03:25.072 INFO:teuthology.orchestra.run.smithi006.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 107 kB/s | 32 kB 00:00 2024-03-22T06:03:25.315 INFO:teuthology.orchestra.run.smithi006.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 244 kB/s | 59 kB 00:00 2024-03-22T06:03:25.541 INFO:teuthology.orchestra.run.smithi006.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 282 kB/s | 203 kB 00:00 2024-03-22T06:03:25.565 INFO:teuthology.orchestra.run.smithi006.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 359 kB/s | 90 kB 00:00 2024-03-22T06:03:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T06:03:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 1/12 2024-03-22T06:03:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-22T06:03:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-22T06:03:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-22T06:03:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T06:03:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T06:03:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-22T06:03:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-22T06:03:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-22T06:03:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-22T06:03:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-22T06:03:26.890 INFO:teuthology.orchestra.run.smithi006.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 567 kB/s | 1.0 MB 00:01 2024-03-22T06:03:26.892 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:26.893 INFO:teuthology.orchestra.run.smithi006.stdout:Total 851 kB/s | 2.0 MB 00:02 2024-03-22T06:03:26.893 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:26.909 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:26.909 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:27.012 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:27.012 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:27.229 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-22T06:03:27.229 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:27.229 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:27.230 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:27.230 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-22T06:03:27.230 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-22T06:03:27.230 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-22T06:03:27.230 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-22T06:03:27.230 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-22T06:03:27.230 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-22T06:03:27.230 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-22T06:03:27.230 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-22T06:03:27.231 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-22T06:03:27.231 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-22T06:03:27.231 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-22T06:03:27.231 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:27.231 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:27.588 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-rook 2024-03-22T06:03:27.671 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:27.843 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-22T06:03:27.892 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-22T06:03:27.948 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-22T06:03:27.994 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-22T06:03:28.034 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-22T06:03:28.130 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-22T06:03:28.157 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:11 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:28.242 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-22T06:03:28.294 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-22T06:03:28.307 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:28.310 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-03-22T06:03:28.310 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-22T06:03:28.310 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-03-22T06:03:28.310 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:28.310 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 47 k 2024-03-22T06:03:28.310 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-22T06:03:28.310 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-22T06:03:28.310 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-22T06:03:28.310 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout:Install 13 Packages 2024-03-22T06:03:28.311 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:28.312 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 2.0 M 2024-03-22T06:03:28.312 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 26 M 2024-03-22T06:03:28.312 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:28.554 INFO:teuthology.orchestra.run.smithi117.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 206 kB/s | 24 kB 00:00 2024-03-22T06:03:28.579 INFO:teuthology.orchestra.run.smithi117.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 126 kB/s | 18 kB 00:00 2024-03-22T06:03:28.621 INFO:teuthology.orchestra.run.smithi117.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2024-03-22T06:03:28.646 INFO:teuthology.orchestra.run.smithi117.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-22T06:03:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-22T06:03:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 856 kB/s | 43 kB 00:00 2024-03-22T06:03:28.722 INFO:teuthology.orchestra.run.smithi117.stdout:(7/13): ceph-mgr-rook-17.2.7-726.g9c4e0c34.el9. 165 kB/s | 47 kB 00:00 2024-03-22T06:03:28.814 INFO:teuthology.orchestra.run.smithi117.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 128 kB/s | 15 kB 00:00 2024-03-22T06:03:28.839 INFO:teuthology.orchestra.run.smithi117.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 192 kB/s | 32 kB 00:00 2024-03-22T06:03:28.956 INFO:teuthology.orchestra.run.smithi117.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 867 kB/s | 203 kB 00:00 2024-03-22T06:03:28.981 INFO:teuthology.orchestra.run.smithi117.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 416 kB/s | 59 kB 00:00 2024-03-22T06:03:29.040 INFO:teuthology.orchestra.run.smithi117.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-03-22T06:03:29.058 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-22T06:03:29.098 INFO:teuthology.orchestra.run.smithi117.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.6 MB/s | 1.0 MB 00:00 2024-03-22T06:03:29.100 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:29.100 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.6 MB/s | 2.0 MB 00:00 2024-03-22T06:03:29.100 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:29.107 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-22T06:03:29.113 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:29.113 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:29.140 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-22T06:03:29.187 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-22T06:03:29.201 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:29.201 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:29.210 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T06:03:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-22T06:03:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-22T06:03:30.001 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T06:03:30.001 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 1/13 2024-03-22T06:03:30.001 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-22T06:03:30.001 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-22T06:03:30.001 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-22T06:03:30.001 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-22T06:03:30.001 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-22T06:03:30.001 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-22T06:03:30.001 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-22T06:03:30.001 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-22T06:03:30.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-22T06:03:30.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-22T06:03:30.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-22T06:03:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-22T06:03:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-22T06:03:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-22T06:03:30.177 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-22T06:03:30.286 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-22T06:03:30.335 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-22T06:03:30.363 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-22T06:03:30.363 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:30.363 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:30.363 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:30.363 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-22T06:03:30.363 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-22T06:03:30.363 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-22T06:03:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-22T06:03:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-22T06:03:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-22T06:03:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-22T06:03:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-22T06:03:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-22T06:03:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-22T06:03:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-22T06:03:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-22T06:03:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:30.364 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:30.570 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-cephadm 2024-03-22T06:03:31.052 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-22T06:03:31.098 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-22T06:03:31.132 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-22T06:03:31.145 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:20 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:31.178 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-22T06:03:31.202 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T06:03:31.301 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:31.301 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 123 k 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:31.302 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:31.303 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-03-22T06:03:31.303 INFO:teuthology.orchestra.run.smithi006.stdout:Install 6 Packages 2024-03-22T06:03:31.303 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:31.303 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 6.8 M 2024-03-22T06:03:31.303 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 33 M 2024-03-22T06:03:31.304 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:31.700 INFO:teuthology.orchestra.run.smithi006.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-03-22T06:03:31.833 INFO:teuthology.orchestra.run.smithi006.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-03-22T06:03:31.925 INFO:teuthology.orchestra.run.smithi006.stdout:(3/6): ceph-mgr-cephadm-17.2.7-726.g9c4e0c34.el 309 kB/s | 123 kB 00:00 2024-03-22T06:03:31.974 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T06:03:31.974 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 1/13 2024-03-22T06:03:31.974 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-22T06:03:31.974 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-22T06:03:31.974 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-22T06:03:31.974 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-22T06:03:31.974 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-22T06:03:31.974 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-22T06:03:31.974 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-22T06:03:31.975 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-22T06:03:31.975 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-22T06:03:31.975 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-22T06:03:31.975 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-22T06:03:32.201 INFO:teuthology.orchestra.run.smithi006.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 157 kB/s | 58 kB 00:00 2024-03-22T06:03:32.250 INFO:teuthology.orchestra.run.smithi006.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 157 kB/s | 51 kB 00:00 2024-03-22T06:03:32.332 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-22T06:03:32.332 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:32.332 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:32.332 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:32.332 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-22T06:03:32.332 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:32.333 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:32.368 INFO:teuthology.orchestra.run.smithi006.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 823 kB/s | 548 kB 00:00 2024-03-22T06:03:32.370 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:32.370 INFO:teuthology.orchestra.run.smithi006.stdout:Total 6.4 MB/s | 6.8 MB 00:01 2024-03-22T06:03:32.385 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:32.397 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:32.397 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:32.492 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:32.492 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:32.586 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-cephadm 2024-03-22T06:03:32.790 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:32.966 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-22T06:03:33.012 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-22T06:03:33.157 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:16 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:33.316 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:33.316 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 123 k 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-03-22T06:03:33.317 INFO:teuthology.orchestra.run.smithi117.stdout:Install 6 Packages 2024-03-22T06:03:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:33.318 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 6.8 M 2024-03-22T06:03:33.318 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 33 M 2024-03-22T06:03:33.318 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:33.560 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-22T06:03:33.650 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-22T06:03:33.700 INFO:teuthology.orchestra.run.smithi117.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 2.1 MB/s | 248 kB 00:00 2024-03-22T06:03:33.712 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-22T06:03:33.740 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T06:03:33.875 INFO:teuthology.orchestra.run.smithi117.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-03-22T06:03:33.967 INFO:teuthology.orchestra.run.smithi117.stdout:(3/6): ceph-mgr-cephadm-17.2.7-726.g9c4e0c34.el 323 kB/s | 123 kB 00:00 2024-03-22T06:03:34.573 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T06:03:34.573 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/6 2024-03-22T06:03:34.573 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-22T06:03:34.573 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-22T06:03:34.573 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-22T06:03:34.573 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-22T06:03:34.880 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-22T06:03:34.880 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:34.880 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:34.880 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:34.880 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-22T06:03:34.881 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-22T06:03:34.881 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-22T06:03:34.881 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-22T06:03:34.881 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-22T06:03:34.881 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:34.881 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:35.122 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-fuse 2024-03-22T06:03:35.701 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:24 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:35.768 INFO:teuthology.orchestra.run.smithi117.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 265 kB/s | 548 kB 00:02 2024-03-22T06:03:35.856 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:35.856 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:35.857 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-22T06:03:35.857 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:35.857 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:35.857 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 812 k 2024-03-22T06:03:35.857 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-22T06:03:35.857 INFO:teuthology.orchestra.run.smithi006.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-22T06:03:35.857 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:35.857 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:35.857 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:35.857 INFO:teuthology.orchestra.run.smithi006.stdout:Install 2 Packages 2024-03-22T06:03:35.857 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:35.858 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 891 k 2024-03-22T06:03:35.858 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 2.6 M 2024-03-22T06:03:35.858 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:36.002 INFO:teuthology.orchestra.run.smithi006.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 546 kB/s | 79 kB 00:00 2024-03-22T06:03:36.034 INFO:teuthology.orchestra.run.smithi117.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 27 kB/s | 58 kB 00:02 2024-03-22T06:03:36.186 INFO:teuthology.orchestra.run.smithi006.stdout:(2/2): ceph-fuse-17.2.7-726.g9c4e0c34.el9.x86_6 2.4 MB/s | 812 kB 00:00 2024-03-22T06:03:36.187 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:36.187 INFO:teuthology.orchestra.run.smithi006.stdout:Total 2.6 MB/s | 891 kB 00:00 2024-03-22T06:03:36.187 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:36.208 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:36.208 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:36.282 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:36.282 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:36.368 INFO:teuthology.orchestra.run.smithi117.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 21 kB/s | 51 kB 00:02 2024-03-22T06:03:36.371 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:36.371 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.2 MB/s | 6.8 MB 00:03 2024-03-22T06:03:36.386 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:36.398 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:36.398 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:36.469 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:36.491 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:36.493 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:36.601 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-22T06:03:36.671 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T06:03:36.790 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:36.951 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-22T06:03:37.003 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-22T06:03:37.350 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T06:03:37.350 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/2 2024-03-22T06:03:37.545 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-22T06:03:37.628 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-22T06:03:37.628 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:37.628 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:37.628 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-22T06:03:37.628 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:37.628 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:37.639 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-22T06:03:37.693 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-22T06:03:37.723 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T06:03:37.875 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install librados-devel 2024-03-22T06:03:38.458 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:27 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:38.502 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T06:03:38.502 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/6 2024-03-22T06:03:38.502 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-22T06:03:38.502 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-22T06:03:38.502 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-22T06:03:38.502 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-22T06:03:38.613 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:38.613 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:38.613 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2024-03-22T06:03:38.613 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:38.614 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:38.614 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 126 k 2024-03-22T06:03:38.614 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:38.614 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:38.614 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:38.614 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-22T06:03:38.614 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:38.614 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 126 k 2024-03-22T06:03:38.614 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 448 k 2024-03-22T06:03:38.614 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:38.797 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-22T06:03:38.797 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:38.797 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:38.797 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:03:38.797 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-22T06:03:38.797 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-22T06:03:38.797 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-22T06:03:38.797 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-22T06:03:38.798 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-22T06:03:38.798 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:38.798 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:38.932 INFO:teuthology.orchestra.run.smithi006.stdout:librados-devel-17.2.7-726.g9c4e0c34.el9.x86_64. 397 kB/s | 126 kB 00:00 2024-03-22T06:03:38.933 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:38.933 INFO:teuthology.orchestra.run.smithi006.stdout:Total 395 kB/s | 126 kB 00:00 2024-03-22T06:03:38.933 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:38.939 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:38.939 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:38.997 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:38.998 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:39.046 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-fuse 2024-03-22T06:03:39.144 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:39.250 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:39.608 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:22 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:39.668 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:39.757 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:39.758 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:39.758 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-22T06:03:39.758 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:39.758 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:39.758 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 812 k 2024-03-22T06:03:39.758 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-22T06:03:39.758 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-22T06:03:39.759 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:39.759 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:39.759 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:39.759 INFO:teuthology.orchestra.run.smithi117.stdout:Install 2 Packages 2024-03-22T06:03:39.759 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:39.759 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 891 k 2024-03-22T06:03:39.759 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 2.6 M 2024-03-22T06:03:39.759 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:39.886 INFO:teuthology.orchestra.run.smithi117.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 620 kB/s | 79 kB 00:00 2024-03-22T06:03:39.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:39.962 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:39.962 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:39.962 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:39.962 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:39.962 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:40.203 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install libcephfs2 2024-03-22T06:03:40.211 INFO:teuthology.orchestra.run.smithi117.stdout:(2/2): ceph-fuse-17.2.7-726.g9c4e0c34.el9.x86_6 1.8 MB/s | 812 kB 00:00 2024-03-22T06:03:40.212 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:40.212 INFO:teuthology.orchestra.run.smithi117.stdout:Total 1.9 MB/s | 891 kB 00:00 2024-03-22T06:03:40.212 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:40.232 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:40.232 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:40.303 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:40.303 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:40.479 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:40.615 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-22T06:03:40.702 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T06:03:40.775 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:29 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:40.875 INFO:teuthology.orchestra.run.smithi006.stdout:Package libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:40.930 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:40.931 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-22T06:03:40.931 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:40.980 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install libcephfs-devel 2024-03-22T06:03:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T06:03:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/2 2024-03-22T06:03:41.557 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:30 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:41.707 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-22T06:03:41.707 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:41.707 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:41.707 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-22T06:03:41.707 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:41.707 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:41.712 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 25 k 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 110 k 2024-03-22T06:03:41.713 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:41.886 INFO:teuthology.orchestra.run.smithi006.stdout:libcephfs-devel-17.2.7-726.g9c4e0c34.el9.x86_64 145 kB/s | 25 kB 00:00 2024-03-22T06:03:41.886 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:41.887 INFO:teuthology.orchestra.run.smithi006.stdout:Total 144 kB/s | 25 kB 00:00 2024-03-22T06:03:41.887 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:41.890 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:41.890 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:41.892 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:41.892 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:41.960 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librados-devel 2024-03-22T06:03:41.978 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:42.103 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:42.455 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:42.537 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:25 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:42.688 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repo Size 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 126 k 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 126 k 2024-03-22T06:03:42.689 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 448 k 2024-03-22T06:03:42.690 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:42.711 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:42.711 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:42.711 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:42.711 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:42.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:42.712 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:42.889 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install librados2 2024-03-22T06:03:42.915 INFO:teuthology.orchestra.run.smithi117.stdout:librados-devel-17.2.7-726.g9c4e0c34.el9.x86_64. 561 kB/s | 126 kB 00:00 2024-03-22T06:03:42.915 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:42.916 INFO:teuthology.orchestra.run.smithi117.stdout:Total 557 kB/s | 126 kB 00:00 2024-03-22T06:03:42.916 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:42.921 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:42.921 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:42.976 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:42.976 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:43.462 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:32 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:43.563 INFO:teuthology.orchestra.run.smithi006.stdout:Package librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:43.618 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:43.619 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-22T06:03:43.619 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:43.680 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install librbd1 2024-03-22T06:03:43.914 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:43.914 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:43.915 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:43.915 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:43.915 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:43.915 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:44.159 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install libcephfs2 2024-03-22T06:03:44.252 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:33 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:44.353 INFO:teuthology.orchestra.run.smithi006.stdout:Package librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:44.408 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:44.410 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-22T06:03:44.410 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:44.475 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-rados 2024-03-22T06:03:44.738 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:27 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:44.839 INFO:teuthology.orchestra.run.smithi117.stdout:Package libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:44.893 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:44.894 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-03-22T06:03:44.894 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:44.962 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install libcephfs-devel 2024-03-22T06:03:45.070 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:34 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:45.174 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:45.229 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:45.230 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-22T06:03:45.230 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:45.283 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-rgw 2024-03-22T06:03:45.525 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:28 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:45.673 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:45.674 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:45.674 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repo Size 2024-03-22T06:03:45.674 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:45.674 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:45.674 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T06:03:45.674 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:45.674 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:45.675 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:45.675 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-03-22T06:03:45.675 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:45.675 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 25 k 2024-03-22T06:03:45.675 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 110 k 2024-03-22T06:03:45.675 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:45.845 INFO:teuthology.orchestra.run.smithi117.stdout:libcephfs-devel-17.2.7-726.g9c4e0c34.el9.x86_64 147 kB/s | 25 kB 00:00 2024-03-22T06:03:45.846 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:45.846 INFO:teuthology.orchestra.run.smithi117.stdout:Total 145 kB/s | 25 kB 00:00 2024-03-22T06:03:45.846 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:45.849 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:45.849 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:45.851 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:45.851 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:45.852 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:34 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:45.939 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:45.952 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:46.010 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:46.011 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-22T06:03:46.011 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:46.059 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:46.072 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-cephfs 2024-03-22T06:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:46.645 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:35 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:46.687 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:46.687 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:46.687 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:46.687 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:46.687 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:46.687 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:46.745 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:46.800 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:46.801 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-22T06:03:46.801 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:46.861 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librados2 2024-03-22T06:03:46.925 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-rbd 2024-03-22T06:03:47.415 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:30 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:47.502 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:36 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:47.514 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:47.565 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:47.566 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-03-22T06:03:47.567 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:47.601 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:47.638 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librbd1 2024-03-22T06:03:47.656 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:47.657 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-22T06:03:47.657 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:47.714 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install rbd-fuse 2024-03-22T06:03:48.200 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:31 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:48.288 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:37 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:48.296 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:48.348 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:48.349 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-03-22T06:03:48.349 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:48.407 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rados 2024-03-22T06:03:48.444 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:48.444 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:48.444 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2024-03-22T06:03:48.444 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:48.444 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:48.444 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 82 k 2024-03-22T06:03:48.445 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:48.445 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:48.445 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:48.445 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-22T06:03:48.445 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:48.445 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 82 k 2024-03-22T06:03:48.445 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 222 k 2024-03-22T06:03:48.446 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:48.756 INFO:teuthology.orchestra.run.smithi006.stdout:rbd-fuse-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 263 kB/s | 82 kB 00:00 2024-03-22T06:03:48.756 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:48.757 INFO:teuthology.orchestra.run.smithi006.stdout:Total 262 kB/s | 82 kB 00:00 2024-03-22T06:03:48.757 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:48.764 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:48.764 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:48.821 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:48.822 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:48.962 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:48.964 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:31 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:49.061 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:49.074 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:49.114 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:49.115 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-03-22T06:03:49.115 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:49.182 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rgw 2024-03-22T06:03:49.442 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:49.719 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:49.719 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:49.719 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:49.719 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:49.719 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:49.719 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:49.737 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:32 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:49.834 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:49.886 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:49.887 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-03-22T06:03:49.887 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:49.959 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-cephfs 2024-03-22T06:03:49.978 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install rbd-mirror 2024-03-22T06:03:50.522 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:33 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:50.558 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:39 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:50.619 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:50.671 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:50.673 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-03-22T06:03:50.673 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:50.715 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:50.716 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:50.716 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2024-03-22T06:03:50.716 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:50.717 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:50.717 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T06:03:50.717 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:50.717 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:50.717 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:50.717 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-22T06:03:50.717 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:50.717 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 2.9 M 2024-03-22T06:03:50.717 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 12 M 2024-03-22T06:03:50.717 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:50.740 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rbd 2024-03-22T06:03:51.267 INFO:teuthology.orchestra.run.smithi006.stdout:rbd-mirror-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 5.3 MB/s | 2.9 MB 00:00 2024-03-22T06:03:51.268 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:51.268 INFO:teuthology.orchestra.run.smithi006.stdout:Total 5.3 MB/s | 2.9 MB 00:00 2024-03-22T06:03:51.268 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:51.278 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:51.278 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:51.298 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:34 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:51.353 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:51.353 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:51.394 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:03:51.448 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:51.449 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-03-22T06:03:51.449 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:51.525 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-fuse 2024-03-22T06:03:51.778 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:51.826 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:51.850 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:51.850 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:03:51.850 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-22T06:03:51.851 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T06:03:51.851 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T06:03:51.851 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:52.082 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:35 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:52.232 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 82 k 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 82 k 2024-03-22T06:03:52.233 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 222 k 2024-03-22T06:03:52.234 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:52.439 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-fuse-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 398 kB/s | 82 kB 00:00 2024-03-22T06:03:52.439 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:52.440 INFO:teuthology.orchestra.run.smithi117.stdout:Total 395 kB/s | 82 kB 00:00 2024-03-22T06:03:52.440 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:52.446 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:52.447 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:52.501 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:52.502 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:52.645 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:52.754 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:53.069 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:53.069 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:53.070 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:53.070 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:53.070 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:53.070 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:53.122 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:53.308 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install rbd-nbd 2024-03-22T06:03:53.362 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:53.362 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:53.362 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:53.362 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:53.362 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:53.362 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:53.614 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-mirror 2024-03-22T06:03:53.891 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:42 ago on Fri 22 Mar 2024 06:02:11 AM UTC. 2024-03-22T06:03:54.046 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 165 k 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 165 k 2024-03-22T06:03:54.047 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 477 k 2024-03-22T06:03:54.048 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-22T06:03:54.179 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:37 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:54.332 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:54.333 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:54.334 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repo Size 2024-03-22T06:03:54.334 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:54.334 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:54.334 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T06:03:54.334 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:54.334 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:54.334 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:54.334 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-03-22T06:03:54.334 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:54.335 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 2.9 M 2024-03-22T06:03:54.335 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 12 M 2024-03-22T06:03:54.335 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:54.396 INFO:teuthology.orchestra.run.smithi006.stdout:rbd-nbd-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 474 kB/s | 165 kB 00:00 2024-03-22T06:03:54.397 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:54.398 INFO:teuthology.orchestra.run.smithi006.stdout:Total 471 kB/s | 165 kB 00:00 2024-03-22T06:03:54.398 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-22T06:03:54.406 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-22T06:03:54.406 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-22T06:03:54.465 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-22T06:03:54.465 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-22T06:03:54.624 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-22T06:03:54.700 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-mirror-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 8.0 MB/s | 2.9 MB 00:00 2024-03-22T06:03:54.701 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:54.701 INFO:teuthology.orchestra.run.smithi117.stdout:Total 8.0 MB/s | 2.9 MB 00:00 2024-03-22T06:03:54.701 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:54.731 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:54.781 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:54.781 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:55.170 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:55.187 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:55.268 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:55.291 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:55.291 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:03:55.292 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-22T06:03:55.292 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T06:03:55.292 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T06:03:55.292 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:55.459 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:55.459 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:55.459 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-22T06:03:55.459 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:55.459 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:03:55.459 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-22T06:03:55.745 DEBUG:teuthology.parallel:result is None 2024-03-22T06:03:56.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:56.353 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:56.353 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:56.354 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:56.354 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:56.354 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:56.569 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-nbd 2024-03-22T06:03:57.144 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:40 ago on Fri 22 Mar 2024 06:02:17 AM UTC. 2024-03-22T06:03:57.294 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-22T06:03:57.295 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:57.295 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-03-22T06:03:57.295 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:57.295 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-22T06:03:57.295 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 165 k 2024-03-22T06:03:57.295 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:57.295 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-22T06:03:57.296 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-22T06:03:57.296 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-03-22T06:03:57.296 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:57.296 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 165 k 2024-03-22T06:03:57.296 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 477 k 2024-03-22T06:03:57.296 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-22T06:03:57.618 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-nbd-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 513 kB/s | 165 kB 00:00 2024-03-22T06:03:57.619 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:03:57.619 INFO:teuthology.orchestra.run.smithi117.stdout:Total 510 kB/s | 165 kB 00:00 2024-03-22T06:03:57.619 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-22T06:03:57.626 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-22T06:03:57.626 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-22T06:03:57.681 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-22T06:03:57.681 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-22T06:03:57.826 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-22T06:03:57.927 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:58.337 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:58.609 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:03:58.609 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:58.609 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-22T06:03:58.610 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:03:58.610 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:03:58.610 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-22T06:03:58.832 DEBUG:teuthology.parallel:result is None 2024-03-22T06:03:58.833 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:03:58.833 INFO:teuthology.packaging:ref: None 2024-03-22T06:03:58.833 INFO:teuthology.packaging:tag: None 2024-03-22T06:03:58.833 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:03:58.833 INFO:teuthology.packaging:sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:03:58.833 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T06:03:59.088 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-22T06:03:59.112 INFO:teuthology.orchestra.run.smithi006.stdout:17.2.7-726.g9c4e0c34.el9 2024-03-22T06:03:59.113 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-726.g9c4e0c34.el9 2024-03-22T06:03:59.113 INFO:teuthology.task.install:The correct ceph version 17.2.7-726.g9c4e0c34 is installed. 2024-03-22T06:03:59.115 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:03:59.115 INFO:teuthology.packaging:ref: None 2024-03-22T06:03:59.115 INFO:teuthology.packaging:tag: None 2024-03-22T06:03:59.115 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:03:59.115 INFO:teuthology.packaging:sha1: ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:03:59.115 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T06:03:59.365 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-22T06:03:59.386 INFO:teuthology.orchestra.run.smithi117.stdout:17.2.7-726.g9c4e0c34.el9 2024-03-22T06:03:59.386 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-726.g9c4e0c34.el9 2024-03-22T06:03:59.387 INFO:teuthology.task.install:The correct ceph version 17.2.7-726.g9c4e0c34 is installed. 2024-03-22T06:03:59.388 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-22T06:03:59.388 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:03:59.389 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T06:03:59.422 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:03:59.422 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T06:03:59.454 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-22T06:03:59.454 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:03:59.454 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T06:03:59.492 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T06:03:59.562 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:03:59.562 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T06:03:59.590 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T06:03:59.659 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-22T06:03:59.659 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:03:59.659 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T06:03:59.688 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T06:03:59.757 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:03:59.757 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T06:03:59.784 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T06:03:59.850 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-22T06:03:59.850 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:03:59.850 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T06:03:59.879 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T06:03:59.948 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:03:59.948 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T06:03:59.974 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T06:04:00.041 INFO:teuthology.run_tasks:Running task print... 2024-03-22T06:04:00.118 INFO:teuthology.task.print:**** done install task... 2024-03-22T06:04:00.118 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-22T06:04:00.237 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': 'ea01217f8a79678e64ef6a5d68a043a4ef21ca37'} 2024-03-22T06:04:00.237 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-03-22T06:04:00.237 INFO:tasks.cephadm:Cluster fsid is fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:04:00.238 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-22T06:04:00.238 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-22T06:04:00.238 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi006': '172.21.15.6', 'mon.smithi117': '172.21.15.117'} 2024-03-22T06:04:00.238 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-22T06:04:00.238 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname $(hostname -s) 2024-03-22T06:04:00.267 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-03-22T06:04:00.295 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-03-22T06:04:00.295 DEBUG:teuthology.orchestra.run.smithi006:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-22T06:04:00.526 INFO:teuthology.orchestra.run.smithi006.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 22 06:04 /home/ubuntu/cephtest/cephadm 2024-03-22T06:04:00.526 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-22T06:04:00.693 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 22 06:04 /home/ubuntu/cephtest/cephadm 2024-03-22T06:04:00.693 DEBUG:teuthology.orchestra.run.smithi006:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-22T06:04:00.711 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-22T06:04:00.736 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-03-22T06:04:00.736 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-22T06:04:00.753 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-22T06:04:01.567 INFO:teuthology.orchestra.run.smithi006.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T06:04:01.622 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T06:04:37.445 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:04:37.445 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-22T06:04:37.445 INFO:teuthology.orchestra.run.smithi006.stdout: "image_id": "837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de", 2024-03-22T06:04:37.445 INFO:teuthology.orchestra.run.smithi006.stdout: "repo_digests": [ 2024-03-22T06:04:37.445 INFO:teuthology.orchestra.run.smithi006.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232" 2024-03-22T06:04:37.445 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-03-22T06:04:37.445 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:05:18.393 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-03-22T06:05:18.393 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-22T06:05:18.393 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de", 2024-03-22T06:05:18.393 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-03-22T06:05:18.394 INFO:teuthology.orchestra.run.smithi117.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232" 2024-03-22T06:05:18.394 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-03-22T06:05:18.394 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-03-22T06:05:18.412 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph 2024-03-22T06:05:18.448 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-03-22T06:05:18.478 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 777 /etc/ceph 2024-03-22T06:05:18.517 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-03-22T06:05:18.545 INFO:tasks.cephadm:Writing seed config... 2024-03-22T06:05:18.546 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-22T06:05:18.546 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-22T06:05:18.546 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-22T06:05:18.546 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-22T06:05:18.546 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-22T06:05:18.547 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-22T06:05:18.548 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-22T06:05:18.549 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-22T06:05:18.549 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-22T06:05:18.549 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-22T06:05:18.549 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:05:18.549 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-22T06:05:18.574 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = fa0e13c2-e811-11ee-95cd-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-22T06:05:18.574 DEBUG:teuthology.orchestra.run.smithi006:mon.smithi006> sudo journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006.service 2024-03-22T06:05:18.616 INFO:tasks.cephadm:Bootstrapping... 2024-03-22T06:05:18.617 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.6 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-22T06:05:18.831 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:05:18.831 INFO:teuthology.orchestra.run.smithi006.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'fa0e13c2-e811-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.6', '--skip-admin-label'] 2024-03-22T06:05:18.856 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T06:05:18.856 INFO:teuthology.orchestra.run.smithi006.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-22T06:05:18.857 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying podman|docker is present... 2024-03-22T06:05:18.881 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T06:05:18.881 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying lvm2 is present... 2024-03-22T06:05:18.881 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying time synchronization is in place... 2024-03-22T06:05:18.889 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-22T06:05:18.890 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-22T06:05:18.895 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-22T06:05:18.895 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-22T06:05:18.902 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2024-03-22T06:05:18.909 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2024-03-22T06:05:18.909 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2024-03-22T06:05:18.909 INFO:teuthology.orchestra.run.smithi006.stdout:Repeating the final host check... 2024-03-22T06:05:18.938 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T06:05:18.938 INFO:teuthology.orchestra.run.smithi006.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-22T06:05:18.938 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl is present 2024-03-22T06:05:18.938 INFO:teuthology.orchestra.run.smithi006.stdout:lvcreate is present 2024-03-22T06:05:18.945 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-22T06:05:18.945 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-22T06:05:18.951 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-22T06:05:18.951 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-22T06:05:18.958 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2024-03-22T06:05:18.965 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2024-03-22T06:05:18.965 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2024-03-22T06:05:18.965 INFO:teuthology.orchestra.run.smithi006.stdout:Host looks OK 2024-03-22T06:05:18.965 INFO:teuthology.orchestra.run.smithi006.stdout:Cluster fsid: fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:05:18.965 INFO:teuthology.orchestra.run.smithi006.stdout:Acquiring lock 139743351649376 on /run/cephadm/fa0e13c2-e811-11ee-95cd-87774f69a715.lock 2024-03-22T06:05:18.966 INFO:teuthology.orchestra.run.smithi006.stdout:Lock 139743351649376 acquired on /run/cephadm/fa0e13c2-e811-11ee-95cd-87774f69a715.lock 2024-03-22T06:05:18.966 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 3300 ... 2024-03-22T06:05:18.966 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 6789 ... 2024-03-22T06:05:18.966 INFO:teuthology.orchestra.run.smithi006.stdout:Base mon IP(s) is [172.21.15.6:3300, 172.21.15.6:6789], mon addrv is [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2024-03-22T06:05:18.970 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.6 metric 100 2024-03-22T06:05:18.970 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.6 metric 100 2024-03-22T06:05:18.973 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-22T06:05:18.973 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-22T06:05:18.973 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-22T06:05:18.976 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-22T06:05:18.976 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-22T06:05:18.976 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-22T06:05:18.976 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-22T06:05:18.976 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2024-03-22T06:05:18.976 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-22T06:05:18.977 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2024-03-22T06:05:18.977 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2024-03-22T06:05:18.977 INFO:teuthology.orchestra.run.smithi006.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-22T06:05:18.977 INFO:teuthology.orchestra.run.smithi006.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-22T06:05:18.978 INFO:teuthology.orchestra.run.smithi006.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T06:05:19.644 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de 2024-03-22T06:05:19.644 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T06:05:19.645 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Getting image source signatures 2024-03-22T06:05:19.645 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:601ec453fe24458667695dda21742a4f3ea719c31ad8aa8ec48bf585cc42a7d7 2024-03-22T06:05:19.645 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-22T06:05:19.645 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying config sha256:837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de 2024-03-22T06:05:19.645 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-22T06:05:20.574 INFO:teuthology.orchestra.run.smithi006.stdout:ceph: stdout ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable) 2024-03-22T06:05:20.574 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph version: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable) 2024-03-22T06:05:20.574 INFO:teuthology.orchestra.run.smithi006.stdout:Extracting ceph user uid/gid from container image... 2024-03-22T06:05:21.554 INFO:teuthology.orchestra.run.smithi006.stdout:stat: stdout 167 167 2024-03-22T06:05:21.554 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial keys... 2024-03-22T06:05:22.443 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQChH/1l3VbhNRAAomn14lto+aI6so8MDRx8Kw== 2024-03-22T06:05:23.291 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCiH/1lqWwrLBAAwpfHiKLvmbcLq3FU9cnlsw== 2024-03-22T06:05:24.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCjH/1lx64WIxAAVJ180ogLbA9jbzEh/Y0OGw== 2024-03-22T06:05:24.322 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial monmap... 2024-03-22T06:05:25.227 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-22T06:05:25.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-03-22T06:05:25.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:05:25.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-22T06:05:25.228 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool for smithi006 [v2:172.21.15.6:3300,v1:172.21.15.6:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-22T06:05:25.228 INFO:teuthology.orchestra.run.smithi006.stdout:setting min_mon_release = octopus 2024-03-22T06:05:25.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: set fsid to fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:05:25.228 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-22T06:05:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:05:25.228 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mon... 2024-03-22T06:05:26.050 INFO:teuthology.orchestra.run.smithi006.stdout:create mon.smithi006 on 2024-03-22T06:05:26.248 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-22T06:05:26.387 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-22T06:05:26.531 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-fa0e13c2-e811-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-fa0e13c2-e811-11ee-95cd-87774f69a715.target. 2024-03-22T06:05:26.531 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-fa0e13c2-e811-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-fa0e13c2-e811-11ee-95cd-87774f69a715.target. 2024-03-22T06:05:26.690 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006 2024-03-22T06:05:26.690 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006.service: Unit ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006.service not loaded. 2024-03-22T06:05:26.860 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fa0e13c2-e811-11ee-95cd-87774f69a715.target.wants/ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006.service → /etc/systemd/system/ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@.service. 2024-03-22T06:05:27.431 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T06:05:27.431 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-03-22T06:05:27.438 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T06:05:27.438 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-22T06:05:27.438 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-03-22T06:05:27.438 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2024-03-22T06:05:27.438 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon to start... 2024-03-22T06:05:27.438 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon... 2024-03-22T06:05:28.507 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:28 smithi006 ceph-mon[28929]: from='client.? 172.21.15.6:0/4245237584' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-22T06:05:28.515 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout cluster: 2024-03-22T06:05:28.515 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout id: fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:05:28.515 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-22T06:05:28.515 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:28.515 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout services: 2024-03-22T06:05:28.515 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi006 (age 0.560649s) 2024-03-22T06:05:28.515 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-22T06:05:28.515 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-22T06:05:28.515 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:28.515 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout data: 2024-03-22T06:05:28.515 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pgs: 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.857+0000 7f5edefb6700 1 Processor -- start 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.857+0000 7f5edefb6700 1 -- start start 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.857+0000 7f5edefb6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed81052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.857+0000 7f5edefb6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed8105820 con 0x7f5ed8071360 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.858+0000 7f5edcd52700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed81052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.858+0000 7f5edcd52700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed81052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54912/0 (socket says 172.21.15.6:54912) 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.858+0000 7f5edcd52700 1 -- 172.21.15.6:0/890244286 learned_addr learned my addr 172.21.15.6:0/890244286 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.858+0000 7f5edcd52700 1 -- 172.21.15.6:0/890244286 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed8105960 con 0x7f5ed8071360 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.858+0000 7f5edcd52700 1 --2- 172.21.15.6:0/890244286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed81052e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009e50 tx=0x7f5ed0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7e9ba1f03fa1fb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.858+0000 7f5ed77fe700 1 -- 172.21.15.6:0/890244286 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed00092e0 con 0x7f5ed8071360 2024-03-22T06:05:28.516 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.859+0000 7f5ed77fe700 1 -- 172.21.15.6:0/890244286 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed000ba20 con 0x7f5ed8071360 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.859+0000 7f5ed77fe700 1 -- 172.21.15.6:0/890244286 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed000bd30 con 0x7f5ed8071360 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.859+0000 7f5edefb6700 1 -- 172.21.15.6:0/890244286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 msgr2=0x7f5ed81052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.859+0000 7f5edefb6700 1 --2- 172.21.15.6:0/890244286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed81052e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009e50 tx=0x7f5ed0009840 comp rx=0 tx=0).stop 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.859+0000 7f5edefb6700 1 -- 172.21.15.6:0/890244286 shutdown_connections 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.859+0000 7f5edefb6700 1 --2- 172.21.15.6:0/890244286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed81052e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.859+0000 7f5edefb6700 1 -- 172.21.15.6:0/890244286 >> 172.21.15.6:0/890244286 conn(0x7f5ed80fcd50 msgr2=0x7f5ed80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.859+0000 7f5edefb6700 1 -- 172.21.15.6:0/890244286 shutdown_connections 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.859+0000 7f5edefb6700 1 -- 172.21.15.6:0/890244286 wait complete. 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.860+0000 7f5edefb6700 1 Processor -- start 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.860+0000 7f5edefb6700 1 -- start start 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.860+0000 7f5edefb6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed8110eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.860+0000 7f5edefb6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed81113f0 con 0x7f5ed8071360 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.860+0000 7f5edcd52700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed8110eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.860+0000 7f5edcd52700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed8110eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54920/0 (socket says 172.21.15.6:54920) 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.860+0000 7f5edcd52700 1 -- 172.21.15.6:0/4245237584 learned_addr learned my addr 172.21.15.6:0/4245237584 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:28.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.860+0000 7f5edcd52700 1 -- 172.21.15.6:0/4245237584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed0009ae0 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.860+0000 7f5edcd52700 1 --2- 172.21.15.6:0/4245237584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed8110eb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0003000 tx=0x7f5ed0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.861+0000 7f5ed57fa700 1 -- 172.21.15.6:0/4245237584 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed0012b90 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.861+0000 7f5ed57fa700 1 -- 172.21.15.6:0/4245237584 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed0012d10 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.861+0000 7f5edefb6700 1 -- 172.21.15.6:0/4245237584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed81115f0 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.861+0000 7f5ed57fa700 1 -- 172.21.15.6:0/4245237584 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed001e400 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.861+0000 7f5edefb6700 1 -- 172.21.15.6:0/4245237584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed8111fd0 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.861+0000 7f5ed57fa700 1 -- 172.21.15.6:0/4245237584 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5ed0019030 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.861+0000 7f5ed57fa700 1 -- 172.21.15.6:0/4245237584 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5ed001ed00 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.862+0000 7f5edefb6700 1 -- 172.21.15.6:0/4245237584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed804bed0 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.864+0000 7f5ed57fa700 1 -- 172.21.15.6:0/4245237584 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5ed0017070 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.937+0000 7f5edefb6700 1 -- 172.21.15.6:0/4245237584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5ed8112ab0 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.938+0000 7f5ed57fa700 1 -- 172.21.15.6:0/4245237584 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f5ed001e620 con 0x7f5ed8071360 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.938+0000 7f5edefb6700 1 -- 172.21.15.6:0/4245237584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 msgr2=0x7f5ed8110eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:28.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.938+0000 7f5edefb6700 1 --2- 172.21.15.6:0/4245237584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed8110eb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0003000 tx=0x7f5ed0017000 comp rx=0 tx=0).stop 2024-03-22T06:05:28.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.939+0000 7f5edefb6700 1 -- 172.21.15.6:0/4245237584 shutdown_connections 2024-03-22T06:05:28.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.939+0000 7f5edefb6700 1 --2- 172.21.15.6:0/4245237584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ed8071360 0x7f5ed8110eb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:28.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.939+0000 7f5edefb6700 1 -- 172.21.15.6:0/4245237584 >> 172.21.15.6:0/4245237584 conn(0x7f5ed80fcd50 msgr2=0x7f5ed80fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:28.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.939+0000 7f5edefb6700 1 -- 172.21.15.6:0/4245237584 shutdown_connections 2024-03-22T06:05:28.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:27.939+0000 7f5edefb6700 1 -- 172.21.15.6:0/4245237584 wait complete. 2024-03-22T06:05:28.519 INFO:teuthology.orchestra.run.smithi006.stdout:mon is available 2024-03-22T06:05:28.519 INFO:teuthology.orchestra.run.smithi006.stdout:Assimilating anything we can from ceph.conf... 2024-03-22T06:05:29.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:29.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2024-03-22T06:05:29.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:05:29.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2024-03-22T06:05:29.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-22T06:05:29.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-22T06:05:29.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-22T06:05:29.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-22T06:05:29.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:29.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.958+0000 7feca873b700 1 Processor -- start 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.958+0000 7feca873b700 1 -- start start 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.958+0000 7feca873b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0104190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.958+0000 7feca873b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca01046d0 con 0x7feca0103d80 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.958+0000 7feca64d7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0104190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:29.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.958+0000 7feca64d7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0104190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54932/0 (socket says 172.21.15.6:54932) 2024-03-22T06:05:29.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.958+0000 7feca64d7700 1 -- 172.21.15.6:0/3286472578 learned_addr learned my addr 172.21.15.6:0/3286472578 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:29.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.959+0000 7feca64d7700 1 -- 172.21.15.6:0/3286472578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca0104810 con 0x7feca0103d80 2024-03-22T06:05:29.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.959+0000 7feca64d7700 1 --2- 172.21.15.6:0/3286472578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0104190 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0098e0 tx=0x7fec9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6378cfe71350fc0d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:29.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.959+0000 7feca54d5700 1 -- 172.21.15.6:0/3286472578 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec9c00b860 con 0x7feca0103d80 2024-03-22T06:05:29.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.959+0000 7feca54d5700 1 -- 172.21.15.6:0/3286472578 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c00b9c0 con 0x7feca0103d80 2024-03-22T06:05:29.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.959+0000 7feca54d5700 1 -- 172.21.15.6:0/3286472578 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec9c00bc90 con 0x7feca0103d80 2024-03-22T06:05:29.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.959+0000 7feca873b700 1 -- 172.21.15.6:0/3286472578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 msgr2=0x7feca0104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:29.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.959+0000 7feca873b700 1 --2- 172.21.15.6:0/3286472578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0104190 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0098e0 tx=0x7fec9c009250 comp rx=0 tx=0).stop 2024-03-22T06:05:29.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca873b700 1 -- 172.21.15.6:0/3286472578 shutdown_connections 2024-03-22T06:05:29.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca873b700 1 --2- 172.21.15.6:0/3286472578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0104190 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:29.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca873b700 1 -- 172.21.15.6:0/3286472578 >> 172.21.15.6:0/3286472578 conn(0x7feca00fc560 msgr2=0x7feca00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:29.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca873b700 1 -- 172.21.15.6:0/3286472578 shutdown_connections 2024-03-22T06:05:29.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca873b700 1 -- 172.21.15.6:0/3286472578 wait complete. 2024-03-22T06:05:29.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca873b700 1 Processor -- start 2024-03-22T06:05:29.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca873b700 1 -- start start 2024-03-22T06:05:29.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca873b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0110d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:29.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca873b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca0111250 con 0x7feca0103d80 2024-03-22T06:05:29.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca64d7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0110d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:29.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca64d7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0110d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54944/0 (socket says 172.21.15.6:54944) 2024-03-22T06:05:29.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.960+0000 7feca64d7700 1 -- 172.21.15.6:0/2448107513 learned_addr learned my addr 172.21.15.6:0/2448107513 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:29.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.961+0000 7feca64d7700 1 -- 172.21.15.6:0/2448107513 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0094f0 con 0x7feca0103d80 2024-03-22T06:05:29.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.961+0000 7feca64d7700 1 --2- 172.21.15.6:0/2448107513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0110d10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fec9c003080 tx=0x7fec9c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:29.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.961+0000 7fec96ffd700 1 -- 172.21.15.6:0/2448107513 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec9c01c5c0 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.961+0000 7fec96ffd700 1 -- 172.21.15.6:0/2448107513 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c01c720 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.961+0000 7feca873b700 1 -- 172.21.15.6:0/2448107513 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca0111450 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.961+0000 7fec96ffd700 1 -- 172.21.15.6:0/2448107513 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec9c01ca30 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.961+0000 7feca873b700 1 -- 172.21.15.6:0/2448107513 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca0111fc0 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.961+0000 7fec96ffd700 1 -- 172.21.15.6:0/2448107513 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fec9c01a450 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.961+0000 7fec96ffd700 1 -- 172.21.15.6:0/2448107513 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fec9c02e790 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.962+0000 7feca873b700 1 -- 172.21.15.6:0/2448107513 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca01118e0 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:28.964+0000 7fec96ffd700 1 -- 172.21.15.6:0/2448107513 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fec9c01a890 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.033+0000 7feca873b700 1 -- 172.21.15.6:0/2448107513 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7feca0100810 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.036+0000 7fec96ffd700 1 -- 172.21.15.6:0/2448107513 <== mon.0 v2:172.21.15.6:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fec9c02ee80 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.036+0000 7fec96ffd700 1 -- 172.21.15.6:0/2448107513 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7fec9c024890 con 0x7feca0103d80 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.038+0000 7feca873b700 1 -- 172.21.15.6:0/2448107513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 msgr2=0x7feca0110d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.038+0000 7feca873b700 1 --2- 172.21.15.6:0/2448107513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0110d10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fec9c003080 tx=0x7fec9c016000 comp rx=0 tx=0).stop 2024-03-22T06:05:29.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.038+0000 7feca873b700 1 -- 172.21.15.6:0/2448107513 shutdown_connections 2024-03-22T06:05:29.595 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.038+0000 7feca873b700 1 --2- 172.21.15.6:0/2448107513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca0103d80 0x7feca0110d10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:29.595 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.038+0000 7feca873b700 1 -- 172.21.15.6:0/2448107513 >> 172.21.15.6:0/2448107513 conn(0x7feca00fc560 msgr2=0x7feca0105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:29.595 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.038+0000 7feca873b700 1 -- 172.21.15.6:0/2448107513 shutdown_connections 2024-03-22T06:05:29.595 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.038+0000 7feca873b700 1 -- 172.21.15.6:0/2448107513 wait complete. 2024-03-22T06:05:29.595 INFO:teuthology.orchestra.run.smithi006.stdout:Generating new minimal ceph.conf... 2024-03-22T06:05:29.689 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:29 smithi006 ceph-mon[28929]: from='client.? 172.21.15.6:0/2448107513' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-22T06:05:29.689 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:29 smithi006 ceph-mon[28929]: from='client.? 172.21.15.6:0/2448107513' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-22T06:05:30.620 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.998+0000 7f78998e7700 1 Processor -- start 2024-03-22T06:05:30.620 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.998+0000 7f78998e7700 1 -- start start 2024-03-22T06:05:30.620 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.999+0000 7f78998e7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.999+0000 7f78998e7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7894103080 con 0x7f7894102730 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.999+0000 7f7892ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.999+0000 7f7892ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54956/0 (socket says 172.21.15.6:54956) 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.999+0000 7f7892ffd700 1 -- 172.21.15.6:0/4036791035 learned_addr learned my addr 172.21.15.6:0/4036791035 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.999+0000 7f7892ffd700 1 -- 172.21.15.6:0/4036791035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78941031c0 con 0x7f7894102730 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.999+0000 7f7892ffd700 1 --2- 172.21.15.6:0/4036791035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894102b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f787c0098e0 tx=0x7f787c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271d7c0264fbcba5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.999+0000 7f7891ffb700 1 -- 172.21.15.6:0/4036791035 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f787c00b860 con 0x7f7894102730 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.999+0000 7f7891ffb700 1 -- 172.21.15.6:0/4036791035 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f787c00be50 con 0x7f7894102730 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:29.999+0000 7f7891ffb700 1 -- 172.21.15.6:0/4036791035 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f787c010380 con 0x7f7894102730 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.000+0000 7f78998e7700 1 -- 172.21.15.6:0/4036791035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 msgr2=0x7f7894102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.000+0000 7f78998e7700 1 --2- 172.21.15.6:0/4036791035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894102b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f787c0098e0 tx=0x7f787c009250 comp rx=0 tx=0).stop 2024-03-22T06:05:30.621 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.000+0000 7f78998e7700 1 -- 172.21.15.6:0/4036791035 shutdown_connections 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.000+0000 7f78998e7700 1 --2- 172.21.15.6:0/4036791035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894102b40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.000+0000 7f78998e7700 1 -- 172.21.15.6:0/4036791035 >> 172.21.15.6:0/4036791035 conn(0x7f78940fdf10 msgr2=0x7f7894100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.000+0000 7f78998e7700 1 -- 172.21.15.6:0/4036791035 shutdown_connections 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.000+0000 7f78998e7700 1 -- 172.21.15.6:0/4036791035 wait complete. 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.001+0000 7f78998e7700 1 Processor -- start 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.001+0000 7f78998e7700 1 -- start start 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.001+0000 7f78998e7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.001+0000 7f78998e7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7894079110 con 0x7f7894102730 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.001+0000 7f7892ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.001+0000 7f7892ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54972/0 (socket says 172.21.15.6:54972) 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.001+0000 7f7892ffd700 1 -- 172.21.15.6:0/1219626386 learned_addr learned my addr 172.21.15.6:0/1219626386 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.001+0000 7f7892ffd700 1 -- 172.21.15.6:0/1219626386 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0094f0 con 0x7f7894102730 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.001+0000 7f7892ffd700 1 --2- 172.21.15.6:0/1219626386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894078bd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f787c00b440 tx=0x7f787c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:30.622 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.002+0000 7f788b7fe700 1 -- 172.21.15.6:0/1219626386 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f787c010760 con 0x7f7894102730 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.002+0000 7f788b7fe700 1 -- 172.21.15.6:0/1219626386 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f787c010d50 con 0x7f7894102730 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.002+0000 7f78998e7700 1 -- 172.21.15.6:0/1219626386 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7894077180 con 0x7f7894102730 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.002+0000 7f788b7fe700 1 -- 172.21.15.6:0/1219626386 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f787c01adc0 con 0x7f7894102730 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.002+0000 7f78998e7700 1 -- 172.21.15.6:0/1219626386 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7894077670 con 0x7f7894102730 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.002+0000 7f788b7fe700 1 -- 172.21.15.6:0/1219626386 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f787c024670 con 0x7f7894102730 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.002+0000 7f788b7fe700 1 -- 172.21.15.6:0/1219626386 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f787c019880 con 0x7f7894102730 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.003+0000 7f78998e7700 1 -- 172.21.15.6:0/1219626386 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789405f5f0 con 0x7f7894102730 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.005+0000 7f788b7fe700 1 -- 172.21.15.6:0/1219626386 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f787c029060 con 0x7f7894102730 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.074+0000 7f78998e7700 1 -- 172.21.15.6:0/1219626386 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7894077af0 con 0x7f7894102730 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.074+0000 7f788b7fe700 1 -- 172.21.15.6:0/1219626386 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f787c01a5f0 con 0x7f7894102730 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.075+0000 7f78998e7700 1 -- 172.21.15.6:0/1219626386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 msgr2=0x7f7894078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.075+0000 7f78998e7700 1 --2- 172.21.15.6:0/1219626386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894078bd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f787c00b440 tx=0x7f787c009250 comp rx=0 tx=0).stop 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.075+0000 7f78998e7700 1 -- 172.21.15.6:0/1219626386 shutdown_connections 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.075+0000 7f78998e7700 1 --2- 172.21.15.6:0/1219626386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7894102730 0x7f7894078bd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:30.623 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.075+0000 7f78998e7700 1 -- 172.21.15.6:0/1219626386 >> 172.21.15.6:0/1219626386 conn(0x7f78940fdf10 msgr2=0x7f78940feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:30.624 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.075+0000 7f78998e7700 1 -- 172.21.15.6:0/1219626386 shutdown_connections 2024-03-22T06:05:30.624 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:30.075+0000 7f78998e7700 1 -- 172.21.15.6:0/1219626386 wait complete. 2024-03-22T06:05:30.624 INFO:teuthology.orchestra.run.smithi006.stdout:Restarting the monitor... 2024-03-22T06:05:30.638 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:30 smithi006 ceph-mon[28929]: from='client.? 172.21.15.6:0/1219626386' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:05:30.947 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:30 smithi006 systemd[1]: Stopping Ceph mon.smithi006 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:05:30.948 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:30 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006[28925]: 2024-03-22T06:05:30.874+0000 7fdd97792700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi006 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:05:30.948 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:30 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006[28925]: 2024-03-22T06:05:30.874+0000 7fdd97792700 -1 mon.smithi006@0(leader) e1 *** Got Signal Terminated *** 2024-03-22T06:05:30.948 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:30 smithi006 podman[29174]: 2024-03-22 06:05:30.947147701 +0000 UTC m=+0.163291122 container died 43afd02fa7e434d5015035360e3de2fdcab45978064288ca6313edd45d064936 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T06:05:31.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:31 smithi006 podman[29174]: 2024-03-22 06:05:31.316933523 +0000 UTC m=+0.533076993 container cleanup 43afd02fa7e434d5015035360e3de2fdcab45978064288ca6313edd45d064936 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-9c4e0c3, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:05:31.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:31 smithi006 bash[29174]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006 2024-03-22T06:05:31.943 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:31 smithi006 podman[29185]: 2024-03-22 06:05:31.567712633 +0000 UTC m=+0.623963074 container remove 43afd02fa7e434d5015035360e3de2fdcab45978064288ca6313edd45d064936 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:05:31.943 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:31 smithi006 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006.service: Deactivated successfully. 2024-03-22T06:05:31.943 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:31 smithi006 systemd[1]: Stopped Ceph mon.smithi006 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:05:31.943 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:31 smithi006 systemd[1]: Starting Ceph mon.smithi006 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:05:32.219 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:31 smithi006 podman[29282]: 2024-03-22 06:05:31.846387916 +0000 UTC m=+0.017573380 image pull 837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de quay.ceph.io/ceph-ci/ceph:quincy 2024-03-22T06:05:32.219 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 podman[29282]: 2024-03-22 06:05:32.110534152 +0000 UTC m=+0.281719616 container create b2fe36c6b00594a1110ecb7ecfc952a6e7f98479fe63b4c71cf0fb610bd77eb8 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-22T06:05:32.458 INFO:teuthology.orchestra.run.smithi006.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-22T06:05:32.666 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 podman[29282]: 2024-03-22 06:05:32.363759646 +0000 UTC m=+0.534945120 container init b2fe36c6b00594a1110ecb7ecfc952a6e7f98479fe63b4c71cf0fb610bd77eb8 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, maintainer=Guillaume Abrioux , RELEASE=quincy-9c4e0c3, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-22T06:05:32.666 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 podman[29282]: 2024-03-22 06:05:32.366566475 +0000 UTC m=+0.537751934 container start b2fe36c6b00594a1110ecb7ecfc952a6e7f98479fe63b4c71cf0fb610bd77eb8 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-22T06:05:32.666 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T06:05:32.666 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable), process ceph-mon, pid 2 2024-03-22T06:05:32.666 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: pidfile_write: ignore empty --pid-file 2024-03-22T06:05:32.666 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: load: jerasure load: lrc 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: RocksDB version: 6.15.5 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Compile date Mar 21 2024 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: DB SUMMARY 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: DB Session ID: UWLXI9IHBL036289R2Q1 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: CURRENT file: CURRENT 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi006/store.db dir, Total Num: 1, files: 000008.sst 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi006/store.db: 000010.log size: 97254 ; 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.error_if_exists: 0 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.create_if_missing: 0 2024-03-22T06:05:32.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.paranoid_checks: 1 2024-03-22T06:05:32.668 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T06:05:32.668 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.env: 0x55607873ca80 2024-03-22T06:05:32.668 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.fs: Posix File System 2024-03-22T06:05:32.668 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.info_log: 0x556079571c00 2024-03-22T06:05:32.668 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T06:05:32.668 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.statistics: (nil) 2024-03-22T06:05:32.668 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.use_fsync: 0 2024-03-22T06:05:32.668 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_log_file_size: 0 2024-03-22T06:05:32.669 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T06:05:32.669 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T06:05:32.669 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.allow_fallocate: 1 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.use_direct_reads: 0 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.db_log_dir: 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi006/store.db 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T06:05:32.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.write_buffer_manager: 0x55607a2e6ab0 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T06:05:32.671 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.unordered_write: 0 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.row_cache: None 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.wal_filter: None 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.preserve_deletes: 0 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.two_write_queues: 0 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.atomic_flush: 0 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T06:05:32.672 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.log_readahead_size: 0 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_background_jobs: 2 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_background_compactions: -1 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_subcompactions: 1 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T06:05:32.673 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_open_files: -1 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_background_flushes: -1 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Compression algorithms supported: 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: kZSTD supported: 0 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: kXpressCompression supported: 0 2024-03-22T06:05:32.674 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: kZlibCompression supported: 1 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: kSnappyCompression supported: 1 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: kLZ4Compression supported: 1 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: kBZip2Compression supported: 0 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi006/store.db/MANIFEST-000009 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.merge_operator: 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_filter: None 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T06:05:32.675 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5560795c6f58) 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: cache_index_and_filter_blocks: 1 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: pin_top_level_index_and_filter: 1 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: index_type: 0 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: data_block_index_type: 0 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: index_shortening: 1 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: hash_index_allow_collision: 1 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: checksum: 1 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: no_block_cache: 0 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_cache: 0x5560795f9090 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_cache_name: BinnedLRUCache 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_cache_options: 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: capacity : 536870912 2024-03-22T06:05:32.676 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: num_shard_bits : 4 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: strict_capacity_limit : 0 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: high_pri_pool_ratio: 0.000 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_cache_compressed: (nil) 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: persistent_cache: (nil) 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_size: 4096 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_size_deviation: 10 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_restart_interval: 16 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: index_block_restart_interval: 1 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: metadata_block_size: 4096 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: partition_filters: 0 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: use_delta_encoding: 1 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: whole_key_filtering: 1 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: verify_compression: 0 2024-03-22T06:05:32.677 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: read_amp_bytes_per_bit: 0 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: format_version: 4 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: enable_index_compression: 1 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_align: 0 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compression: NoCompression 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.num_levels: 7 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T06:05:32.678 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T06:05:32.679 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T06:05:32.680 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.arena_block_size: 4194304 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T06:05:32.681 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.table_properties_collectors: 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.inplace_update_support: 0 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.bloom_locality: 0 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.max_successive_merges: 0 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T06:05:32.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.ttl: 2592000 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.enable_blob_files: false 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.min_blob_size: 0 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi006/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711087532392567, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-22T06:05:32.683 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711087532393366, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92526, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90621, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83336, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1711087532, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "80009ed1-8913-4099-a8af-470819289974", "db_session_id": "UWLXI9IHBL036289R2Q1"}} 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711087532393720, "job": 1, "event": "recovery_finished"} 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi006/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55607958d180 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: DB pointer 0x556079662000 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ** DB Stats ** 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:05:32.684 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ** Compaction Stats [default] ** 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: L0 2/0 92.16 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.7 0.00 0.00 1 0.001 0 0 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Sum 2/0 92.16 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.7 0.00 0.00 1 0.001 0 0 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.7 0.00 0.00 1 0.001 0 0 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ** Compaction Stats [default] ** 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 158.7 0.00 0.00 1 0.001 0 0 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T06:05:32.685 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Cumulative compaction: 0.00 GB write, 28.76 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Interval compaction: 0.00 GB write, 28.76 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ** Compaction Stats [default] ** 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: L0 2/0 92.16 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.7 0.00 0.00 1 0.001 0 0 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Sum 2/0 92.16 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.7 0.00 0.00 1 0.001 0 0 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-22T06:05:32.686 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ** Compaction Stats [default] ** 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 158.7 0.00 0.00 1 0.001 0 0 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Cumulative compaction: 0.00 GB write, 28.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: starting mon.smithi006 rank 0 at public addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] at bind addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi006 fsid fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:05:32.687 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: mon.smithi006@-1(???) e1 preinit fsid fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: mon.smithi006@-1(???).mds e1 new map 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: mon.smithi006@-1(???).mds e1 print_map 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: e1 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: legacy client fscid: -1 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: No filesystems configured 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: mon.smithi006@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: mon.smithi006@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: mon.smithi006@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: mon.smithi006@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: mon.smithi006@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 bash[29282]: b2fe36c6b00594a1110ecb7ecfc952a6e7f98479fe63b4c71cf0fb610bd77eb8 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 systemd[1]: Started Ceph mon.smithi006 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: mon.smithi006 is new leader, mons smithi006 in quorum (ranks 0) 2024-03-22T06:05:32.688 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: monmap e1: 1 mons at {smithi006=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T06:05:32.689 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: fsmap 2024-03-22T06:05:32.689 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: osdmap e1: 0 total, 0 up, 0 in 2024-03-22T06:05:32.689 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:32 smithi006 ceph-mon[29301]: mgrmap e1: no daemons active 2024-03-22T06:05:33.485 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.865+0000 7f13b7ca3700 1 Processor -- start 2024-03-22T06:05:33.485 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.865+0000 7f13b7ca3700 1 -- start start 2024-03-22T06:05:33.485 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.865+0000 7f13b7ca3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:33.485 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.865+0000 7f13b7ca3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0103000 con 0x7f13b01026b0 2024-03-22T06:05:33.485 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.866+0000 7f13b5a3f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:33.485 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.866+0000 7f13b5a3f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0102ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54986/0 (socket says 172.21.15.6:54986) 2024-03-22T06:05:33.485 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.866+0000 7f13b5a3f700 1 -- 172.21.15.6:0/1248351404 learned_addr learned my addr 172.21.15.6:0/1248351404 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:33.485 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.866+0000 7f13b5a3f700 1 -- 172.21.15.6:0/1248351404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b0103140 con 0x7f13b01026b0 2024-03-22T06:05:33.485 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.866+0000 7f13b5a3f700 1 --2- 172.21.15.6:0/1248351404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0102ac0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0098e0 tx=0x7f13ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e6c6582437b4d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:33.485 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.867+0000 7f13b4a3d700 1 -- 172.21.15.6:0/1248351404 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13ac00b860 con 0x7f13b01026b0 2024-03-22T06:05:33.485 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.867+0000 7f13b4a3d700 1 -- 172.21.15.6:0/1248351404 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f13ac00be50 con 0x7f13b01026b0 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.867+0000 7f13b4a3d700 1 -- 172.21.15.6:0/1248351404 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13ac010380 con 0x7f13b01026b0 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.867+0000 7f13b7ca3700 1 -- 172.21.15.6:0/1248351404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 msgr2=0x7f13b0102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.867+0000 7f13b7ca3700 1 --2- 172.21.15.6:0/1248351404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0102ac0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0098e0 tx=0x7f13ac009250 comp rx=0 tx=0).stop 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.867+0000 7f13b7ca3700 1 -- 172.21.15.6:0/1248351404 shutdown_connections 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.867+0000 7f13b7ca3700 1 --2- 172.21.15.6:0/1248351404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0102ac0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.867+0000 7f13b7ca3700 1 -- 172.21.15.6:0/1248351404 >> 172.21.15.6:0/1248351404 conn(0x7f13b00fde70 msgr2=0x7f13b0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.867+0000 7f13b7ca3700 1 -- 172.21.15.6:0/1248351404 shutdown_connections 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.867+0000 7f13b7ca3700 1 -- 172.21.15.6:0/1248351404 wait complete. 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.868+0000 7f13b7ca3700 1 Processor -- start 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.868+0000 7f13b7ca3700 1 -- start start 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.868+0000 7f13b7ca3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.868+0000 7f13b7ca3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0111ad0 con 0x7f13b01026b0 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.868+0000 7f13b5a3f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:33.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.868+0000 7f13b5a3f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54990/0 (socket says 172.21.15.6:54990) 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.868+0000 7f13b5a3f700 1 -- 172.21.15.6:0/2836463947 learned_addr learned my addr 172.21.15.6:0/2836463947 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.868+0000 7f13b5a3f700 1 -- 172.21.15.6:0/2836463947 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac0094f0 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.869+0000 7f13b5a3f700 1 --2- 172.21.15.6:0/2836463947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0111590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f13ac000c00 tx=0x7f13ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.869+0000 7f13a67fc700 1 -- 172.21.15.6:0/2836463947 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13ac0107c0 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.869+0000 7f13a67fc700 1 -- 172.21.15.6:0/2836463947 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f13ac010db0 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.869+0000 7f13b7ca3700 1 -- 172.21.15.6:0/2836463947 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b0111cd0 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.869+0000 7f13a67fc700 1 -- 172.21.15.6:0/2836463947 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13ac01ad00 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.869+0000 7f13b7ca3700 1 -- 172.21.15.6:0/2836463947 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b0112840 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.869+0000 7f13a67fc700 1 -- 172.21.15.6:0/2836463947 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f13ac024440 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.869+0000 7f13a67fc700 1 -- 172.21.15.6:0/2836463947 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f13ac019710 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.870+0000 7f13b7ca3700 1 -- 172.21.15.6:0/2836463947 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b005f5f0 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.872+0000 7f13a67fc700 1 -- 172.21.15.6:0/2836463947 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f13ac01a530 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.940+0000 7f13b7ca3700 1 -- 172.21.15.6:0/2836463947 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f13b0104520 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.943+0000 7f13a67fc700 1 -- 172.21.15.6:0/2836463947 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f13ac010920 con 0x7f13b01026b0 2024-03-22T06:05:33.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.944+0000 7f13b7ca3700 1 -- 172.21.15.6:0/2836463947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 msgr2=0x7f13b0111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:33.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.944+0000 7f13b7ca3700 1 --2- 172.21.15.6:0/2836463947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0111590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f13ac000c00 tx=0x7f13ac009250 comp rx=0 tx=0).stop 2024-03-22T06:05:33.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.944+0000 7f13b7ca3700 1 -- 172.21.15.6:0/2836463947 shutdown_connections 2024-03-22T06:05:33.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.944+0000 7f13b7ca3700 1 --2- 172.21.15.6:0/2836463947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13b01026b0 0x7f13b0111590 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:33.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.944+0000 7f13b7ca3700 1 -- 172.21.15.6:0/2836463947 >> 172.21.15.6:0/2836463947 conn(0x7f13b00fde70 msgr2=0x7f13b00fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:33.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.944+0000 7f13b7ca3700 1 -- 172.21.15.6:0/2836463947 shutdown_connections 2024-03-22T06:05:33.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:32.944+0000 7f13b7ca3700 1 -- 172.21.15.6:0/2836463947 wait complete. 2024-03-22T06:05:33.488 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-22T06:05:33.488 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-22T06:05:33.488 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mgr... 2024-03-22T06:05:33.488 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying port 9283 ... 2024-03-22T06:05:33.669 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mgr.smithi006.sfpxdy 2024-03-22T06:05:33.670 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mgr.smithi006.sfpxdy.service: Unit ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mgr.smithi006.sfpxdy.service not loaded. 2024-03-22T06:05:33.815 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fa0e13c2-e811-11ee-95cd-87774f69a715.target.wants/ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mgr.smithi006.sfpxdy.service → /etc/systemd/system/ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@.service. 2024-03-22T06:05:34.157 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:33 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2836463947' entity='client.admin' 2024-03-22T06:05:34.385 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T06:05:34.386 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-03-22T06:05:34.393 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T06:05:34.393 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-22T06:05:34.393 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-03-22T06:05:34.393 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2024-03-22T06:05:34.401 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T06:05:34.402 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-03-22T06:05:34.409 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T06:05:34.409 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-22T06:05:34.409 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-03-22T06:05:34.409 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-03-22T06:05:34.409 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr to start... 2024-03-22T06:05:34.409 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr... 2024-03-22T06:05:35.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:35 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3549009240' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T06:05:35.599 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:35.602 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:05:35.602 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "fa0e13c2-e811-11ee-95cd-87774f69a715", 2024-03-22T06:05:35.602 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T06:05:35.602 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T06:05:35.602 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T06:05:35.602 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T06:05:35.602 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:35.602 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "smithi006" 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:35.603 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T06:05:35.604 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:35.604 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T06:05:35.604 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T06:05:35.604 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T06:05:35.612 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T06:05:35.612 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T06:05:35.612 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T06:05:35.613 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:35.614 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T06:05:27.379768+0000", 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.909+0000 7fa9eba62700 1 Processor -- start 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.909+0000 7fa9eba62700 1 -- start start 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.909+0000 7fa9eba62700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4077200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:35.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.909+0000 7fa9eba62700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e4079110 con 0x7fa9e4078d00 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.909+0000 7fa9e97fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4077200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.909+0000 7fa9e97fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4077200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49672/0 (socket says 172.21.15.6:49672) 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.910+0000 7fa9e97fe700 1 -- 172.21.15.6:0/261165747 learned_addr learned my addr 172.21.15.6:0/261165747 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.910+0000 7fa9e97fe700 1 -- 172.21.15.6:0/261165747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e4077740 con 0x7fa9e4078d00 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.910+0000 7fa9e97fe700 1 --2- 172.21.15.6:0/261165747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4077200 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00098e0 tx=0x7fa9e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12d66e572cd474 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.910+0000 7fa9dbfff700 1 -- 172.21.15.6:0/261165747 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9e000b860 con 0x7fa9e4078d00 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.910+0000 7fa9dbfff700 1 -- 172.21.15.6:0/261165747 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa9e000be50 con 0x7fa9e4078d00 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.910+0000 7fa9dbfff700 1 -- 172.21.15.6:0/261165747 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9e0010380 con 0x7fa9e4078d00 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.911+0000 7fa9eba62700 1 -- 172.21.15.6:0/261165747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 msgr2=0x7fa9e4077200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.911+0000 7fa9eba62700 1 --2- 172.21.15.6:0/261165747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4077200 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00098e0 tx=0x7fa9e0009250 comp rx=0 tx=0).stop 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.911+0000 7fa9eba62700 1 -- 172.21.15.6:0/261165747 shutdown_connections 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.911+0000 7fa9eba62700 1 --2- 172.21.15.6:0/261165747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4077200 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.911+0000 7fa9eba62700 1 -- 172.21.15.6:0/261165747 >> 172.21.15.6:0/261165747 conn(0x7fa9e40fdf10 msgr2=0x7fa9e4100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:35.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.911+0000 7fa9eba62700 1 -- 172.21.15.6:0/261165747 shutdown_connections 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.911+0000 7fa9eba62700 1 -- 172.21.15.6:0/261165747 wait complete. 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.912+0000 7fa9eba62700 1 Processor -- start 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.912+0000 7fa9eba62700 1 -- start start 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.912+0000 7fa9eba62700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.912+0000 7fa9eba62700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e4113cd0 con 0x7fa9e4078d00 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.912+0000 7fa9e97fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.912+0000 7fa9e97fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4113790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49684/0 (socket says 172.21.15.6:49684) 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.912+0000 7fa9e97fe700 1 -- 172.21.15.6:0/3549009240 learned_addr learned my addr 172.21.15.6:0/3549009240 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.912+0000 7fa9e97fe700 1 -- 172.21.15.6:0/3549009240 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e00094f0 con 0x7fa9e4078d00 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.912+0000 7fa9e97fe700 1 --2- 172.21.15.6:0/3549009240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4113790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0006b00 tx=0x7fa9e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.913+0000 7fa9d9ffb700 1 -- 172.21.15.6:0/3549009240 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9e00107c0 con 0x7fa9e4078d00 2024-03-22T06:05:35.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.913+0000 7fa9d9ffb700 1 -- 172.21.15.6:0/3549009240 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa9e0010db0 con 0x7fa9e4078d00 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.913+0000 7fa9d9ffb700 1 -- 172.21.15.6:0/3549009240 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9e001acf0 con 0x7fa9e4078d00 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.913+0000 7fa9eba62700 1 -- 172.21.15.6:0/3549009240 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e4113ed0 con 0x7fa9e4078d00 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.913+0000 7fa9eba62700 1 -- 172.21.15.6:0/3549009240 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e4114a40 con 0x7fa9e4078d00 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.913+0000 7fa9d9ffb700 1 -- 172.21.15.6:0/3549009240 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa9e0024440 con 0x7fa9e4078d00 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.913+0000 7fa9d9ffb700 1 -- 172.21.15.6:0/3549009240 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa9e0019750 con 0x7fa9e4078d00 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.914+0000 7fa9eba62700 1 -- 172.21.15.6:0/3549009240 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e4114360 con 0x7fa9e4078d00 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.916+0000 7fa9d9ffb700 1 -- 172.21.15.6:0/3549009240 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa9e00426d0 con 0x7fa9e4078d00 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.989+0000 7fa9eba62700 1 -- 172.21.15.6:0/3549009240 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa9e404bed0 con 0x7fa9e4078d00 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.989+0000 7fa9d9ffb700 1 -- 172.21.15.6:0/3549009240 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fa9e0031030 con 0x7fa9e4078d00 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.990+0000 7fa9eba62700 1 -- 172.21.15.6:0/3549009240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 msgr2=0x7fa9e4113790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.990+0000 7fa9eba62700 1 --2- 172.21.15.6:0/3549009240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4113790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0006b00 tx=0x7fa9e0009250 comp rx=0 tx=0).stop 2024-03-22T06:05:35.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.991+0000 7fa9eba62700 1 -- 172.21.15.6:0/3549009240 shutdown_connections 2024-03-22T06:05:35.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.991+0000 7fa9eba62700 1 --2- 172.21.15.6:0/3549009240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9e4078d00 0x7fa9e4113790 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:35.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.991+0000 7fa9eba62700 1 -- 172.21.15.6:0/3549009240 >> 172.21.15.6:0/3549009240 conn(0x7fa9e40fdf10 msgr2=0x7fa9e40fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:35.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.991+0000 7fa9eba62700 1 -- 172.21.15.6:0/3549009240 shutdown_connections 2024-03-22T06:05:35.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:34.991+0000 7fa9eba62700 1 -- 172.21.15.6:0/3549009240 wait complete. 2024-03-22T06:05:35.619 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (1/15)... 2024-03-22T06:05:38.237 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:38 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2024586719' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T06:05:38.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:38.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:05:38.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "fa0e13c2-e811-11ee-95cd-87774f69a715", 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "smithi006" 2024-03-22T06:05:38.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:38.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-22T06:05:38.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T06:05:38.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:38.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T06:05:38.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T06:05:38.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:38.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T06:05:38.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:38.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T06:05:38.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T06:05:38.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T06:05:38.892 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T06:05:38.892 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T06:05:27.379768+0000", 2024-03-22T06:05:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.028+0000 7fab90f6d700 1 Processor -- start 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.028+0000 7fab90f6d700 1 -- start start 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.029+0000 7fab90f6d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c105310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.029+0000 7fab90f6d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c105850 con 0x7fab8c076bc0 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.029+0000 7fab8a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c105310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.029+0000 7fab8a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c105310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49698/0 (socket says 172.21.15.6:49698) 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.029+0000 7fab8a59c700 1 -- 172.21.15.6:0/1845347941 learned_addr learned my addr 172.21.15.6:0/1845347941 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.029+0000 7fab8a59c700 1 -- 172.21.15.6:0/1845347941 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c105990 con 0x7fab8c076bc0 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.029+0000 7fab8a59c700 1 --2- 172.21.15.6:0/1845347941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c105310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0098e0 tx=0x7fab7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612a9672614d7edd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.029+0000 7fab8959a700 1 -- 172.21.15.6:0/1845347941 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab7c00b860 con 0x7fab8c076bc0 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.029+0000 7fab8959a700 1 -- 172.21.15.6:0/1845347941 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fab7c00be50 con 0x7fab8c076bc0 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.029+0000 7fab8959a700 1 -- 172.21.15.6:0/1845347941 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab7c010380 con 0x7fab8c076bc0 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.030+0000 7fab90f6d700 1 -- 172.21.15.6:0/1845347941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 msgr2=0x7fab8c105310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.030+0000 7fab90f6d700 1 --2- 172.21.15.6:0/1845347941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c105310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0098e0 tx=0x7fab7c009250 comp rx=0 tx=0).stop 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.030+0000 7fab90f6d700 1 -- 172.21.15.6:0/1845347941 shutdown_connections 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.030+0000 7fab90f6d700 1 --2- 172.21.15.6:0/1845347941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c105310 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.030+0000 7fab90f6d700 1 -- 172.21.15.6:0/1845347941 >> 172.21.15.6:0/1845347941 conn(0x7fab8c0fcd80 msgr2=0x7fab8c0ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.030+0000 7fab90f6d700 1 -- 172.21.15.6:0/1845347941 shutdown_connections 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.030+0000 7fab90f6d700 1 -- 172.21.15.6:0/1845347941 wait complete. 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.031+0000 7fab90f6d700 1 Processor -- start 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.031+0000 7fab90f6d700 1 -- start start 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.031+0000 7fab90f6d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.031+0000 7fab90f6d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c1031a0 con 0x7fab8c076bc0 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.031+0000 7fab8a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.031+0000 7fab8a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49712/0 (socket says 172.21.15.6:49712) 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.031+0000 7fab8a59c700 1 -- 172.21.15.6:0/2024586719 learned_addr learned my addr 172.21.15.6:0/2024586719 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.031+0000 7fab8a59c700 1 -- 172.21.15.6:0/2024586719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c0094f0 con 0x7fab8c076bc0 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.031+0000 7fab8a59c700 1 --2- 172.21.15.6:0/2024586719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c102c60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fab7c003050 tx=0x7fab7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.032+0000 7fab82ffd700 1 -- 172.21.15.6:0/2024586719 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab7c010760 con 0x7fab8c076bc0 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.032+0000 7fab82ffd700 1 -- 172.21.15.6:0/2024586719 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fab7c010d50 con 0x7fab8c076bc0 2024-03-22T06:05:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.032+0000 7fab90f6d700 1 -- 172.21.15.6:0/2024586719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8c101270 con 0x7fab8c076bc0 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.032+0000 7fab82ffd700 1 -- 172.21.15.6:0/2024586719 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab7c01ae20 con 0x7fab8c076bc0 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.032+0000 7fab90f6d700 1 -- 172.21.15.6:0/2024586719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8c101760 con 0x7fab8c076bc0 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.032+0000 7fab82ffd700 1 -- 172.21.15.6:0/2024586719 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fab7c02e6c0 con 0x7fab8c076bc0 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.032+0000 7fab82ffd700 1 -- 172.21.15.6:0/2024586719 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fab7c024870 con 0x7fab8c076bc0 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.033+0000 7fab90f6d700 1 -- 172.21.15.6:0/2024586719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8c04bed0 con 0x7fab8c076bc0 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.035+0000 7fab82ffd700 1 -- 172.21.15.6:0/2024586719 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fab7c029070 con 0x7fab8c076bc0 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.107+0000 7fab90f6d700 1 -- 172.21.15.6:0/2024586719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fab8c101a00 con 0x7fab8c076bc0 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.107+0000 7fab82ffd700 1 -- 172.21.15.6:0/2024586719 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fab7c01a6a0 con 0x7fab8c076bc0 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.108+0000 7fab90f6d700 1 -- 172.21.15.6:0/2024586719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 msgr2=0x7fab8c102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.108+0000 7fab90f6d700 1 --2- 172.21.15.6:0/2024586719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c102c60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fab7c003050 tx=0x7fab7c009250 comp rx=0 tx=0).stop 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.108+0000 7fab90f6d700 1 -- 172.21.15.6:0/2024586719 shutdown_connections 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.108+0000 7fab90f6d700 1 --2- 172.21.15.6:0/2024586719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab8c076bc0 0x7fab8c102c60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.108+0000 7fab90f6d700 1 -- 172.21.15.6:0/2024586719 >> 172.21.15.6:0/2024586719 conn(0x7fab8c0fcd80 msgr2=0x7fab8c0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.108+0000 7fab90f6d700 1 -- 172.21.15.6:0/2024586719 shutdown_connections 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:38.108+0000 7fab90f6d700 1 -- 172.21.15.6:0/2024586719 wait complete. 2024-03-22T06:05:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (2/15)... 2024-03-22T06:05:41.331 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: Activating manager daemon smithi006.sfpxdy 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: mgrmap e2: smithi006.sfpxdy(active, starting, since 0.00262568s) 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: from='mgr.14100 172.21.15.6:0/4128914868' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: from='mgr.14100 172.21.15.6:0/4128914868' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: from='mgr.14100 172.21.15.6:0/4128914868' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: from='mgr.14100 172.21.15.6:0/4128914868' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: from='mgr.14100 172.21.15.6:0/4128914868' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi006.sfpxdy", "id": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: Manager daemon smithi006.sfpxdy is now available 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: from='mgr.14100 172.21.15.6:0/4128914868' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: from='mgr.14100 172.21.15.6:0/4128914868' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: from='mgr.14100 172.21.15.6:0/4128914868' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/trash_purge_schedule"}]: dispatch 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: from='mgr.14100 172.21.15.6:0/4128914868' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:05:41.332 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:41 smithi006 ceph-mon[29301]: from='mgr.14100 172.21.15.6:0/4128914868' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:05:42.099 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:42.099 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:05:42.099 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "fa0e13c2-e811-11ee-95cd-87774f69a715", 2024-03-22T06:05:42.099 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T06:05:42.099 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T06:05:42.099 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T06:05:42.099 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "smithi006" 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T06:05:42.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T06:05:42.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:42.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T06:05:42.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:42.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T06:05:42.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T06:05:42.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T06:05:42.108 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:42.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:42.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T06:05:27.379768+0000", 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.425+0000 7fe8604b6700 1 Processor -- start 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.425+0000 7fe8604b6700 1 -- start start 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.426+0000 7fe8604b6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.426+0000 7fe8604b6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858103070 con 0x7fe858102720 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.426+0000 7fe85e252700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.426+0000 7fe85e252700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49776/0 (socket says 172.21.15.6:49776) 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.426+0000 7fe85e252700 1 -- 172.21.15.6:0/2418948526 learned_addr learned my addr 172.21.15.6:0/2418948526 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.426+0000 7fe85e252700 1 -- 172.21.15.6:0/2418948526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8581031b0 con 0x7fe858102720 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.426+0000 7fe85e252700 1 --2- 172.21.15.6:0/2418948526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858102b30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe8480098e0 tx=0x7fe848009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3487461ee7fcdb24 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.426+0000 7fe85d250700 1 -- 172.21.15.6:0/2418948526 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe84800b860 con 0x7fe858102720 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.427+0000 7fe85d250700 1 -- 172.21.15.6:0/2418948526 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe84800be50 con 0x7fe858102720 2024-03-22T06:05:42.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.427+0000 7fe85d250700 1 -- 172.21.15.6:0/2418948526 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe848010380 con 0x7fe858102720 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.427+0000 7fe8604b6700 1 -- 172.21.15.6:0/2418948526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 msgr2=0x7fe858102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.427+0000 7fe8604b6700 1 --2- 172.21.15.6:0/2418948526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858102b30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe8480098e0 tx=0x7fe848009250 comp rx=0 tx=0).stop 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.427+0000 7fe8604b6700 1 -- 172.21.15.6:0/2418948526 shutdown_connections 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.427+0000 7fe8604b6700 1 --2- 172.21.15.6:0/2418948526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858102b30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.427+0000 7fe8604b6700 1 -- 172.21.15.6:0/2418948526 >> 172.21.15.6:0/2418948526 conn(0x7fe8580fdee0 msgr2=0x7fe8581002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.427+0000 7fe8604b6700 1 -- 172.21.15.6:0/2418948526 shutdown_connections 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.427+0000 7fe8604b6700 1 -- 172.21.15.6:0/2418948526 wait complete. 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.428+0000 7fe8604b6700 1 Processor -- start 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.428+0000 7fe8604b6700 1 -- start start 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.428+0000 7fe8604b6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858111560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.428+0000 7fe8604b6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858111aa0 con 0x7fe858102720 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.428+0000 7fe85e252700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858111560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.428+0000 7fe85e252700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858111560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49782/0 (socket says 172.21.15.6:49782) 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.428+0000 7fe85e252700 1 -- 172.21.15.6:0/58735570 learned_addr learned my addr 172.21.15.6:0/58735570 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.428+0000 7fe85e252700 1 -- 172.21.15.6:0/58735570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480094f0 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.428+0000 7fe85e252700 1 --2- 172.21.15.6:0/58735570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858111560 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe84800b440 tx=0x7fe848009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.429+0000 7fe84effd700 1 -- 172.21.15.6:0/58735570 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe848010760 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.429+0000 7fe84effd700 1 -- 172.21.15.6:0/58735570 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe848010d50 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.429+0000 7fe8604b6700 1 -- 172.21.15.6:0/58735570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe858111ca0 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.429+0000 7fe84effd700 1 -- 172.21.15.6:0/58735570 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe84801ae20 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.429+0000 7fe8604b6700 1 -- 172.21.15.6:0/58735570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe858112810 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.429+0000 7fe84effd700 1 -- 172.21.15.6:0/58735570 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 2) v1 ==== 43124+0+0 (secure 0 0 0) 0x7fe84802e440 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.430+0000 7fe8604b6700 1 -- 172.21.15.6:0/58735570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85805f5f0 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.430+0000 7fe84effd700 1 -- 172.21.15.6:0/58735570 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe84804a0f0 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.432+0000 7fe84effd700 1 -- 172.21.15.6:0/58735570 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe848018940 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.503+0000 7fe8604b6700 1 -- 172.21.15.6:0/58735570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe85804bed0 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.504+0000 7fe84effd700 1 -- 172.21.15.6:0/58735570 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fe848018350 con 0x7fe858102720 2024-03-22T06:05:42.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.504+0000 7fe8604b6700 1 -- 172.21.15.6:0/58735570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 msgr2=0x7fe858111560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:42.114 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.504+0000 7fe8604b6700 1 --2- 172.21.15.6:0/58735570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858111560 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe84800b440 tx=0x7fe848009250 comp rx=0 tx=0).stop 2024-03-22T06:05:42.114 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.504+0000 7fe8604b6700 1 -- 172.21.15.6:0/58735570 shutdown_connections 2024-03-22T06:05:42.114 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.504+0000 7fe8604b6700 1 --2- 172.21.15.6:0/58735570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe858102720 0x7fe858111560 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:42.114 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.504+0000 7fe8604b6700 1 -- 172.21.15.6:0/58735570 >> 172.21.15.6:0/58735570 conn(0x7fe8580fdee0 msgr2=0x7fe8580fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:42.114 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.505+0000 7fe8604b6700 1 -- 172.21.15.6:0/58735570 shutdown_connections 2024-03-22T06:05:42.114 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:41.505+0000 7fe8604b6700 1 -- 172.21.15.6:0/58735570 wait complete. 2024-03-22T06:05:42.114 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (3/15)... 2024-03-22T06:05:42.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:42 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/58735570' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T06:05:42.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:42 smithi006 ceph-mon[29301]: mgrmap e3: smithi006.sfpxdy(active, since 1.00613s) 2024-03-22T06:05:44.317 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:44 smithi006 ceph-mon[29301]: mgrmap e4: smithi006.sfpxdy(active, since 2s) 2024-03-22T06:05:45.124 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:45 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3089629141' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T06:05:45.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:45.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:05:45.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "fa0e13c2-e811-11ee-95cd-87774f69a715", 2024-03-22T06:05:45.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T06:05:45.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T06:05:45.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T06:05:45.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T06:05:45.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "smithi006" 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T06:05:45.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:45.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T06:05:45.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T06:05:45.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T06:05:45.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T06:05:45.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T06:05:45.449 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T06:05:27.379768+0000", 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:05:45.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.525+0000 7f3e21712700 1 Processor -- start 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.525+0000 7f3e21712700 1 -- start start 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.525+0000 7f3e21712700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.525+0000 7f3e21712700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c103080 con 0x7f3e1c102730 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.525+0000 7f3e1affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.525+0000 7f3e1affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46124/0 (socket says 172.21.15.6:46124) 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.525+0000 7f3e1affd700 1 -- 172.21.15.6:0/1625729286 learned_addr learned my addr 172.21.15.6:0/1625729286 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.525+0000 7f3e1affd700 1 -- 172.21.15.6:0/1625729286 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c1031c0 con 0x7f3e1c102730 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.526+0000 7f3e1affd700 1 --2- 172.21.15.6:0/1625729286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c102b40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0098e0 tx=0x7f3e0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6becdf3ee8b77c29 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.526+0000 7f3e19ffb700 1 -- 172.21.15.6:0/1625729286 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e0c00b860 con 0x7f3e1c102730 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.526+0000 7f3e19ffb700 1 -- 172.21.15.6:0/1625729286 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3e0c00be50 con 0x7f3e1c102730 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.526+0000 7f3e19ffb700 1 -- 172.21.15.6:0/1625729286 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e0c010380 con 0x7f3e1c102730 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.526+0000 7f3e21712700 1 -- 172.21.15.6:0/1625729286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 msgr2=0x7f3e1c102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:45.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.526+0000 7f3e21712700 1 --2- 172.21.15.6:0/1625729286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c102b40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0098e0 tx=0x7f3e0c009250 comp rx=0 tx=0).stop 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.526+0000 7f3e21712700 1 -- 172.21.15.6:0/1625729286 shutdown_connections 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.526+0000 7f3e21712700 1 --2- 172.21.15.6:0/1625729286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c102b40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.526+0000 7f3e21712700 1 -- 172.21.15.6:0/1625729286 >> 172.21.15.6:0/1625729286 conn(0x7f3e1c0fdf10 msgr2=0x7f3e1c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.527+0000 7f3e21712700 1 -- 172.21.15.6:0/1625729286 shutdown_connections 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.527+0000 7f3e21712700 1 -- 172.21.15.6:0/1625729286 wait complete. 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.527+0000 7f3e21712700 1 Processor -- start 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.527+0000 7f3e21712700 1 -- start start 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.527+0000 7f3e21712700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c1115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.527+0000 7f3e21712700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c111b30 con 0x7f3e1c102730 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.528+0000 7f3e1affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c1115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.528+0000 7f3e1affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c1115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46140/0 (socket says 172.21.15.6:46140) 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.528+0000 7f3e1affd700 1 -- 172.21.15.6:0/3089629141 learned_addr learned my addr 172.21.15.6:0/3089629141 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.528+0000 7f3e1affd700 1 -- 172.21.15.6:0/3089629141 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c0094f0 con 0x7f3e1c102730 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.528+0000 7f3e1affd700 1 --2- 172.21.15.6:0/3089629141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c1115f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00b440 tx=0x7f3e0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.528+0000 7f3e037fe700 1 -- 172.21.15.6:0/3089629141 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e0c010760 con 0x7f3e1c102730 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.528+0000 7f3e21712700 1 -- 172.21.15.6:0/3089629141 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1c111d30 con 0x7f3e1c102730 2024-03-22T06:05:45.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.528+0000 7f3e037fe700 1 -- 172.21.15.6:0/3089629141 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3e0c010d50 con 0x7f3e1c102730 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.528+0000 7f3e037fe700 1 -- 172.21.15.6:0/3089629141 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e0c01adc0 con 0x7f3e1c102730 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.528+0000 7f3e21712700 1 -- 172.21.15.6:0/3089629141 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1c1128a0 con 0x7f3e1c102730 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.529+0000 7f3e037fe700 1 -- 172.21.15.6:0/3089629141 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f3e0c019410 con 0x7f3e1c102730 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.529+0000 7f3e037fe700 1 --2- 172.21.15.6:0/3089629141 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f3e04036720 0x7f3e04038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.529+0000 7f3e21712700 1 -- 172.21.15.6:0/3089629141 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1c1121c0 con 0x7f3e1c102730 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.529+0000 7f3e037fe700 1 -- 172.21.15.6:0/3089629141 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3e0c04a140 con 0x7f3e1c102730 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.529+0000 7f3e1a7fc700 1 --2- 172.21.15.6:0/3089629141 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f3e04036720 0x7f3e04038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.530+0000 7f3e1a7fc700 1 --2- 172.21.15.6:0/3089629141 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f3e04036720 0x7f3e04038bd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009990 tx=0x7f3e10006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.534+0000 7f3e037fe700 1 -- 172.21.15.6:0/3089629141 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3e0c023710 con 0x7f3e1c102730 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.810+0000 7f3e21712700 1 -- 172.21.15.6:0/3089629141 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3e1c05f5f0 con 0x7f3e1c102730 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.811+0000 7f3e037fe700 1 -- 172.21.15.6:0/3089629141 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f3e0c00e6e0 con 0x7f3e1c102730 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.814+0000 7f3e21712700 1 -- 172.21.15.6:0/3089629141 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f3e04036720 msgr2=0x7f3e04038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.814+0000 7f3e21712700 1 --2- 172.21.15.6:0/3089629141 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f3e04036720 0x7f3e04038bd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009990 tx=0x7f3e10006e40 comp rx=0 tx=0).stop 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.814+0000 7f3e21712700 1 -- 172.21.15.6:0/3089629141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 msgr2=0x7f3e1c1115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.814+0000 7f3e21712700 1 --2- 172.21.15.6:0/3089629141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c1115f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00b440 tx=0x7f3e0c009250 comp rx=0 tx=0).stop 2024-03-22T06:05:45.453 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.814+0000 7f3e21712700 1 -- 172.21.15.6:0/3089629141 shutdown_connections 2024-03-22T06:05:45.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.814+0000 7f3e21712700 1 --2- 172.21.15.6:0/3089629141 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f3e04036720 0x7f3e04038bd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:45.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.814+0000 7f3e21712700 1 --2- 172.21.15.6:0/3089629141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e1c102730 0x7f3e1c1115f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:45.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.814+0000 7f3e21712700 1 -- 172.21.15.6:0/3089629141 >> 172.21.15.6:0/3089629141 conn(0x7f3e1c0fdf10 msgr2=0x7f3e1c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:45.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.814+0000 7f3e21712700 1 -- 172.21.15.6:0/3089629141 shutdown_connections 2024-03-22T06:05:45.454 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:44.814+0000 7f3e21712700 1 -- 172.21.15.6:0/3089629141 wait complete. 2024-03-22T06:05:45.454 INFO:teuthology.orchestra.run.smithi006.stdout:mgr is available 2024-03-22T06:05:46.414 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:46 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3731085581' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-22T06:05:46.675 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-22T06:05:46.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.863+0000 7f0589775700 1 Processor -- start 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.863+0000 7f0589775700 1 -- start start 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.863+0000 7f0589775700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f0584102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.863+0000 7f0589775700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0584103080 con 0x7f0584102730 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.863+0000 7f0582ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f0584102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.863+0000 7f0582ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f0584102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46142/0 (socket says 172.21.15.6:46142) 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.863+0000 7f0582ffd700 1 -- 172.21.15.6:0/2063612735 learned_addr learned my addr 172.21.15.6:0/2063612735 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.864+0000 7f0582ffd700 1 -- 172.21.15.6:0/2063612735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05841031c0 con 0x7f0584102730 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.864+0000 7f0582ffd700 1 --2- 172.21.15.6:0/2063612735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f0584102b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f05740098e0 tx=0x7f0574009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ae44c01b917bc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.864+0000 7f0581ffb700 1 -- 172.21.15.6:0/2063612735 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f057400b860 con 0x7f0584102730 2024-03-22T06:05:46.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.864+0000 7f0581ffb700 1 -- 172.21.15.6:0/2063612735 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f057400be50 con 0x7f0584102730 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.864+0000 7f0581ffb700 1 -- 172.21.15.6:0/2063612735 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0574010380 con 0x7f0584102730 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.865+0000 7f0589775700 1 -- 172.21.15.6:0/2063612735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 msgr2=0x7f0584102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.865+0000 7f0589775700 1 --2- 172.21.15.6:0/2063612735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f0584102b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f05740098e0 tx=0x7f0574009250 comp rx=0 tx=0).stop 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.865+0000 7f0589775700 1 -- 172.21.15.6:0/2063612735 shutdown_connections 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.865+0000 7f0589775700 1 --2- 172.21.15.6:0/2063612735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f0584102b40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.865+0000 7f0589775700 1 -- 172.21.15.6:0/2063612735 >> 172.21.15.6:0/2063612735 conn(0x7f05840fdf10 msgr2=0x7f0584100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.865+0000 7f0589775700 1 -- 172.21.15.6:0/2063612735 shutdown_connections 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.865+0000 7f0589775700 1 -- 172.21.15.6:0/2063612735 wait complete. 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.865+0000 7f0589775700 1 Processor -- start 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f0589775700 1 -- start start 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f0589775700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f05841115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f0589775700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0584111b30 con 0x7f0584102730 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f0582ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f05841115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f0582ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f05841115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46148/0 (socket says 172.21.15.6:46148) 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f0582ffd700 1 -- 172.21.15.6:0/3731085581 learned_addr learned my addr 172.21.15.6:0/3731085581 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f0582ffd700 1 -- 172.21.15.6:0/3731085581 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05740094f0 con 0x7f0584102730 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f0582ffd700 1 --2- 172.21.15.6:0/3731085581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f05841115f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f057400b440 tx=0x7f0574009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f057b7fe700 1 -- 172.21.15.6:0/3731085581 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0574010760 con 0x7f0584102730 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f057b7fe700 1 -- 172.21.15.6:0/3731085581 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0574010d50 con 0x7f0584102730 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f0589775700 1 -- 172.21.15.6:0/3731085581 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0584111d30 con 0x7f0584102730 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.866+0000 7f057b7fe700 1 -- 172.21.15.6:0/3731085581 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f057401adc0 con 0x7f0584102730 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.867+0000 7f0589775700 1 -- 172.21.15.6:0/3731085581 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05841128a0 con 0x7f0584102730 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.867+0000 7f057b7fe700 1 -- 172.21.15.6:0/3731085581 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f05740108c0 con 0x7f0584102730 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.868+0000 7f057b7fe700 1 --2- 172.21.15.6:0/3731085581 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f0570036790 0x7f0570038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.868+0000 7f0589775700 1 -- 172.21.15.6:0/3731085581 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05841121c0 con 0x7f0584102730 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.868+0000 7f057b7fe700 1 -- 172.21.15.6:0/3731085581 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0574049f60 con 0x7f0584102730 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.868+0000 7f05827fc700 1 --2- 172.21.15.6:0/3731085581 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f0570036790 0x7f0570038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.868+0000 7f05827fc700 1 --2- 172.21.15.6:0/3731085581 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f0570036790 0x7f0570038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f056c009990 tx=0x7f056c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:45.872+0000 7f057b7fe700 1 -- 172.21.15.6:0/3731085581 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0574022300 con 0x7f0584102730 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.067+0000 7f0589775700 1 -- 172.21.15.6:0/3731085581 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f058405f5f0 con 0x7f0584102730 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.068+0000 7f057b7fe700 1 -- 172.21.15.6:0/3731085581 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f0574022300 con 0x7f0584102730 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.070+0000 7f0589775700 1 -- 172.21.15.6:0/3731085581 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f0570036790 msgr2=0x7f0570038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.070+0000 7f0589775700 1 --2- 172.21.15.6:0/3731085581 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f0570036790 0x7f0570038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f056c009990 tx=0x7f056c006e40 comp rx=0 tx=0).stop 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.070+0000 7f0589775700 1 -- 172.21.15.6:0/3731085581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 msgr2=0x7f05841115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.070+0000 7f0589775700 1 --2- 172.21.15.6:0/3731085581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f05841115f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f057400b440 tx=0x7f0574009250 comp rx=0 tx=0).stop 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.070+0000 7f0589775700 1 -- 172.21.15.6:0/3731085581 shutdown_connections 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.070+0000 7f0589775700 1 --2- 172.21.15.6:0/3731085581 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f0570036790 0x7f0570038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.070+0000 7f0589775700 1 --2- 172.21.15.6:0/3731085581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0584102730 0x7f05841115f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.070+0000 7f0589775700 1 -- 172.21.15.6:0/3731085581 >> 172.21.15.6:0/3731085581 conn(0x7f05840fdf10 msgr2=0x7f05840feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.070+0000 7f0589775700 1 -- 172.21.15.6:0/3731085581 shutdown_connections 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:46.070+0000 7f0589775700 1 -- 172.21.15.6:0/3731085581 wait complete. 2024-03-22T06:05:46.680 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling cephadm module... 2024-03-22T06:05:47.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:47 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/4104056182' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-22T06:05:47.980 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.100+0000 7f55417f7700 1 Processor -- start 2024-03-22T06:05:47.980 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.100+0000 7f55417f7700 1 -- start start 2024-03-22T06:05:47.980 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.100+0000 7f55417f7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.100+0000 7f55417f7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1034e0 con 0x7f553c102b90 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.101+0000 7f553affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.101+0000 7f553affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c102fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46158/0 (socket says 172.21.15.6:46158) 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.101+0000 7f553affd700 1 -- 172.21.15.6:0/263897131 learned_addr learned my addr 172.21.15.6:0/263897131 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.101+0000 7f553affd700 1 -- 172.21.15.6:0/263897131 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c103620 con 0x7f553c102b90 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.101+0000 7f553affd700 1 --2- 172.21.15.6:0/263897131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c102fa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55340098e0 tx=0x7f5534009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8713c88e2b5e4ea server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.101+0000 7f5539ffb700 1 -- 172.21.15.6:0/263897131 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f553400b860 con 0x7f553c102b90 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.101+0000 7f5539ffb700 1 -- 172.21.15.6:0/263897131 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f553400be50 con 0x7f553c102b90 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.101+0000 7f5539ffb700 1 -- 172.21.15.6:0/263897131 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5534010380 con 0x7f553c102b90 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.102+0000 7f55417f7700 1 -- 172.21.15.6:0/263897131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 msgr2=0x7f553c102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.102+0000 7f55417f7700 1 --2- 172.21.15.6:0/263897131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c102fa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55340098e0 tx=0x7f5534009250 comp rx=0 tx=0).stop 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.102+0000 7f55417f7700 1 -- 172.21.15.6:0/263897131 shutdown_connections 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.102+0000 7f55417f7700 1 --2- 172.21.15.6:0/263897131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c102fa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:47.981 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.102+0000 7f55417f7700 1 -- 172.21.15.6:0/263897131 >> 172.21.15.6:0/263897131 conn(0x7f553c0fc4d0 msgr2=0x7f553c0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.102+0000 7f55417f7700 1 -- 172.21.15.6:0/263897131 shutdown_connections 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.102+0000 7f55417f7700 1 -- 172.21.15.6:0/263897131 wait complete. 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.103+0000 7f55417f7700 1 Processor -- start 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.103+0000 7f55417f7700 1 -- start start 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.103+0000 7f55417f7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c110460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.103+0000 7f55417f7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1109a0 con 0x7f553c102b90 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.103+0000 7f553affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c110460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.103+0000 7f553affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c110460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46166/0 (socket says 172.21.15.6:46166) 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.103+0000 7f553affd700 1 -- 172.21.15.6:0/4104056182 learned_addr learned my addr 172.21.15.6:0/4104056182 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.103+0000 7f553affd700 1 -- 172.21.15.6:0/4104056182 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55340094f0 con 0x7f553c102b90 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.103+0000 7f553affd700 1 --2- 172.21.15.6:0/4104056182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c110460 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5534000c00 tx=0x7f5534009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.103+0000 7f55337fe700 1 -- 172.21.15.6:0/4104056182 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55340107c0 con 0x7f553c102b90 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.104+0000 7f55337fe700 1 -- 172.21.15.6:0/4104056182 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5534010db0 con 0x7f553c102b90 2024-03-22T06:05:47.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.104+0000 7f55337fe700 1 -- 172.21.15.6:0/4104056182 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f553401ad00 con 0x7f553c102b90 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.104+0000 7f55417f7700 1 -- 172.21.15.6:0/4104056182 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553c110ba0 con 0x7f553c102b90 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.104+0000 7f55417f7700 1 -- 172.21.15.6:0/4104056182 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553c111710 con 0x7f553c102b90 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.104+0000 7f55337fe700 1 -- 172.21.15.6:0/4104056182 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f553401a530 con 0x7f553c102b90 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.105+0000 7f55337fe700 1 --2- 172.21.15.6:0/4104056182 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f5528036790 0x7f5528038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.105+0000 7f55417f7700 1 -- 172.21.15.6:0/4104056182 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f553c05f5f0 con 0x7f553c102b90 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.105+0000 7f55337fe700 1 -- 172.21.15.6:0/4104056182 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f553404a570 con 0x7f553c102b90 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.105+0000 7f553a7fc700 1 --2- 172.21.15.6:0/4104056182 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f5528036790 0x7f5528038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.105+0000 7f553a7fc700 1 --2- 172.21.15.6:0/4104056182 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f5528036790 0x7f5528038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5524009990 tx=0x7f5524006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.110+0000 7f55337fe700 1 -- 172.21.15.6:0/4104056182 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5534010920 con 0x7f553c102b90 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.364+0000 7f55417f7700 1 -- 172.21.15.6:0/4104056182 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f553c04bed0 con 0x7f553c102b90 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.417+0000 7f55337fe700 1 -- 172.21.15.6:0/4104056182 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f5534016d00 con 0x7f553c102b90 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.418+0000 7f55337fe700 1 -- 172.21.15.6:0/4104056182 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f5534044e30 con 0x7f553c102b90 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.421+0000 7f55417f7700 1 -- 172.21.15.6:0/4104056182 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f5528036790 msgr2=0x7f5528038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.421+0000 7f55417f7700 1 --2- 172.21.15.6:0/4104056182 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f5528036790 0x7f5528038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5524009990 tx=0x7f5524006e40 comp rx=0 tx=0).stop 2024-03-22T06:05:47.983 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.421+0000 7f55417f7700 1 -- 172.21.15.6:0/4104056182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 msgr2=0x7f553c110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.421+0000 7f55417f7700 1 --2- 172.21.15.6:0/4104056182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c110460 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5534000c00 tx=0x7f5534009250 comp rx=0 tx=0).stop 2024-03-22T06:05:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.421+0000 7f55417f7700 1 -- 172.21.15.6:0/4104056182 shutdown_connections 2024-03-22T06:05:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.421+0000 7f55417f7700 1 --2- 172.21.15.6:0/4104056182 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f5528036790 0x7f5528038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.421+0000 7f55417f7700 1 --2- 172.21.15.6:0/4104056182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f553c102b90 0x7f553c110460 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.421+0000 7f55417f7700 1 -- 172.21.15.6:0/4104056182 >> 172.21.15.6:0/4104056182 conn(0x7f553c0fc4d0 msgr2=0x7f553c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.421+0000 7f55417f7700 1 -- 172.21.15.6:0/4104056182 shutdown_connections 2024-03-22T06:05:47.984 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:47.421+0000 7f55417f7700 1 -- 172.21.15.6:0/4104056182 wait complete. 2024-03-22T06:05:48.629 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:48 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/4104056182' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-22T06:05:48.630 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:48 smithi006 ceph-mon[29301]: mgrmap e5: smithi006.sfpxdy(active, since 6s) 2024-03-22T06:05:49.229 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:05:49.229 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-22T06:05:49.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T06:05:49.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "smithi006.sfpxdy", 2024-03-22T06:05:49.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-22T06:05:49.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:05:49.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.371+0000 7f9285322700 1 Processor -- start 2024-03-22T06:05:49.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.371+0000 7f9285322700 1 -- start start 2024-03-22T06:05:49.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.371+0000 7f9285322700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f9280102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:49.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.371+0000 7f9285322700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9280103080 con 0x7f9280102730 2024-03-22T06:05:49.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.371+0000 7f927effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f9280102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:49.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.371+0000 7f927effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f9280102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46196/0 (socket says 172.21.15.6:46196) 2024-03-22T06:05:49.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.371+0000 7f927effd700 1 -- 172.21.15.6:0/1964205296 learned_addr learned my addr 172.21.15.6:0/1964205296 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.372+0000 7f927effd700 1 -- 172.21.15.6:0/1964205296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92801031c0 con 0x7f9280102730 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.372+0000 7f927effd700 1 --2- 172.21.15.6:0/1964205296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f9280102b40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92740098e0 tx=0x7f9274009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fdcccc432cf6a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.372+0000 7f927dffb700 1 -- 172.21.15.6:0/1964205296 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f927400b860 con 0x7f9280102730 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.372+0000 7f927dffb700 1 -- 172.21.15.6:0/1964205296 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f927400be50 con 0x7f9280102730 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.372+0000 7f927dffb700 1 -- 172.21.15.6:0/1964205296 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9274010380 con 0x7f9280102730 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.373+0000 7f9285322700 1 -- 172.21.15.6:0/1964205296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 msgr2=0x7f9280102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.373+0000 7f9285322700 1 --2- 172.21.15.6:0/1964205296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f9280102b40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92740098e0 tx=0x7f9274009250 comp rx=0 tx=0).stop 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.373+0000 7f9285322700 1 -- 172.21.15.6:0/1964205296 shutdown_connections 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.373+0000 7f9285322700 1 --2- 172.21.15.6:0/1964205296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f9280102b40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.373+0000 7f9285322700 1 -- 172.21.15.6:0/1964205296 >> 172.21.15.6:0/1964205296 conn(0x7f92800fdf10 msgr2=0x7f9280100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.373+0000 7f9285322700 1 -- 172.21.15.6:0/1964205296 shutdown_connections 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.373+0000 7f9285322700 1 -- 172.21.15.6:0/1964205296 wait complete. 2024-03-22T06:05:49.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.373+0000 7f9285322700 1 Processor -- start 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.373+0000 7f9285322700 1 -- start start 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f9285322700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f92801115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f9285322700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9280111b30 con 0x7f9280102730 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f927effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f92801115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f927effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f92801115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46204/0 (socket says 172.21.15.6:46204) 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f927effd700 1 -- 172.21.15.6:0/589386860 learned_addr learned my addr 172.21.15.6:0/589386860 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f927effd700 1 -- 172.21.15.6:0/589386860 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92740094f0 con 0x7f9280102730 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f927effd700 1 --2- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f92801115f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9274006fd0 tx=0x7f9274009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f92677fe700 1 -- 172.21.15.6:0/589386860 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92740107c0 con 0x7f9280102730 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f9285322700 1 -- 172.21.15.6:0/589386860 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9280111d30 con 0x7f9280102730 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f92677fe700 1 -- 172.21.15.6:0/589386860 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9274010db0 con 0x7f9280102730 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f92677fe700 1 -- 172.21.15.6:0/589386860 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f927401adc0 con 0x7f9280102730 2024-03-22T06:05:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.374+0000 7f9285322700 1 -- 172.21.15.6:0/589386860 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92801128a0 con 0x7f9280102730 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.375+0000 7f92677fe700 1 -- 172.21.15.6:0/589386860 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f92740193c0 con 0x7f9280102730 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.375+0000 7f9285322700 1 -- 172.21.15.6:0/589386860 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92801121c0 con 0x7f9280102730 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.376+0000 7f92677fe700 1 --2- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f92680367e0 0x7f9268038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.376+0000 7f92677fe700 1 -- 172.21.15.6:0/589386860 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9274049ce0 con 0x7f9280102730 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.376+0000 7f927e7fc700 1 -- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f92680367e0 msgr2=0x7f9268038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3099962970 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.376+0000 7f927e7fc700 1 --2- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f92680367e0 0x7f9268038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.380+0000 7f92677fe700 1 -- 172.21.15.6:0/589386860 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f927402bc90 con 0x7f9280102730 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.577+0000 7f927e7fc700 1 -- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f92680367e0 msgr2=0x7f9268038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3099962970 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.577+0000 7f927e7fc700 1 --2- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f92680367e0 0x7f9268038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.629+0000 7f9285322700 1 -- 172.21.15.6:0/589386860 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9280112be0 con 0x7f9280102730 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.629+0000 7f92677fe700 1 -- 172.21.15.6:0/589386860 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f9274020030 con 0x7f9280102730 2024-03-22T06:05:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.631+0000 7f9285322700 1 -- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f92680367e0 msgr2=0x7f9268038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T06:05:49.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.631+0000 7f9285322700 1 --2- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f92680367e0 0x7f9268038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:49.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.631+0000 7f9285322700 1 -- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 msgr2=0x7f92801115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:49.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.631+0000 7f9285322700 1 --2- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f92801115f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9274006fd0 tx=0x7f9274009250 comp rx=0 tx=0).stop 2024-03-22T06:05:49.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.631+0000 7f9285322700 1 -- 172.21.15.6:0/589386860 shutdown_connections 2024-03-22T06:05:49.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.631+0000 7f9285322700 1 --2- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7f92680367e0 0x7f9268038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:49.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.631+0000 7f9285322700 1 --2- 172.21.15.6:0/589386860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9280102730 0x7f92801115f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:49.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.631+0000 7f9285322700 1 -- 172.21.15.6:0/589386860 >> 172.21.15.6:0/589386860 conn(0x7f92800fdf10 msgr2=0x7f92800feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:49.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.631+0000 7f9285322700 1 -- 172.21.15.6:0/589386860 shutdown_connections 2024-03-22T06:05:49.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:48.632+0000 7f9285322700 1 -- 172.21.15.6:0/589386860 wait complete. 2024-03-22T06:05:49.234 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2024-03-22T06:05:49.234 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 5... 2024-03-22T06:05:49.484 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:49 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/589386860' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-22T06:05:54.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: Active manager daemon smithi006.sfpxdy restarted 2024-03-22T06:05:54.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: Activating manager daemon smithi006.sfpxdy 2024-03-22T06:05:54.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: osdmap e2: 0 total, 0 up, 0 in 2024-03-22T06:05:54.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: mgrmap e6: smithi006.sfpxdy(active, starting, since 0.00306232s) 2024-03-22T06:05:54.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:05:54.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi006.sfpxdy", "id": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:05:54.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T06:05:54.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T06:05:54.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T06:05:54.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: Manager daemon smithi006.sfpxdy is now available 2024-03-22T06:05:54.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:05:54.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:05:54.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:05:54.534 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:05:54.534 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:54 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:05:55.409 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:55 smithi006 ceph-mon[29301]: Found migration_current of "None". Setting to last migration. 2024-03-22T06:05:55.409 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:55 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:05:55.409 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:55 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/trash_purge_schedule"}]: dispatch 2024-03-22T06:05:55.409 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:55 smithi006 ceph-mon[29301]: mgrmap e7: smithi006.sfpxdy(active, since 1.00531s) 2024-03-22T06:05:55.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:05:55.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-22T06:05:55.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-22T06:05:55.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:05:55.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.640+0000 7efd51e7f700 1 Processor -- start 2024-03-22T06:05:55.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.641+0000 7efd51e7f700 1 -- start start 2024-03-22T06:05:55.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.641+0000 7efd51e7f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:55.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.641+0000 7efd51e7f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c103510 con 0x7efd4c102bc0 2024-03-22T06:05:55.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.641+0000 7efd4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.641+0000 7efd4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c102fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46212/0 (socket says 172.21.15.6:46212) 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.641+0000 7efd4b7fe700 1 -- 172.21.15.6:0/2911670412 learned_addr learned my addr 172.21.15.6:0/2911670412 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.641+0000 7efd4b7fe700 1 -- 172.21.15.6:0/2911670412 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c103650 con 0x7efd4c102bc0 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.641+0000 7efd4b7fe700 1 --2- 172.21.15.6:0/2911670412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c102fd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0098e0 tx=0x7efd3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cdb3dc247fb2372 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.642+0000 7efd4a7fc700 1 -- 172.21.15.6:0/2911670412 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd3c00b860 con 0x7efd4c102bc0 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.642+0000 7efd4a7fc700 1 -- 172.21.15.6:0/2911670412 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efd3c00be50 con 0x7efd4c102bc0 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.642+0000 7efd4a7fc700 1 -- 172.21.15.6:0/2911670412 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd3c010380 con 0x7efd4c102bc0 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.642+0000 7efd51e7f700 1 -- 172.21.15.6:0/2911670412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 msgr2=0x7efd4c102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.642+0000 7efd51e7f700 1 --2- 172.21.15.6:0/2911670412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c102fd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0098e0 tx=0x7efd3c009250 comp rx=0 tx=0).stop 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.643+0000 7efd51e7f700 1 -- 172.21.15.6:0/2911670412 shutdown_connections 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.643+0000 7efd51e7f700 1 --2- 172.21.15.6:0/2911670412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c102fd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:55.714 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.643+0000 7efd51e7f700 1 -- 172.21.15.6:0/2911670412 >> 172.21.15.6:0/2911670412 conn(0x7efd4c0fc4a0 msgr2=0x7efd4c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.643+0000 7efd51e7f700 1 -- 172.21.15.6:0/2911670412 shutdown_connections 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.643+0000 7efd51e7f700 1 -- 172.21.15.6:0/2911670412 wait complete. 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.643+0000 7efd51e7f700 1 Processor -- start 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.644+0000 7efd51e7f700 1 -- start start 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.644+0000 7efd51e7f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.644+0000 7efd51e7f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c1142a0 con 0x7efd4c102bc0 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.644+0000 7efd4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.644+0000 7efd4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46226/0 (socket says 172.21.15.6:46226) 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.644+0000 7efd4b7fe700 1 -- 172.21.15.6:0/2687303503 learned_addr learned my addr 172.21.15.6:0/2687303503 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.644+0000 7efd4b7fe700 1 -- 172.21.15.6:0/2687303503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0094f0 con 0x7efd4c102bc0 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.644+0000 7efd4b7fe700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c101390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efd3c006b00 tx=0x7efd3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.644+0000 7efd33fff700 1 -- 172.21.15.6:0/2687303503 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd3c010810 con 0x7efd4c102bc0 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.645+0000 7efd33fff700 1 -- 172.21.15.6:0/2687303503 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efd3c010e00 con 0x7efd4c102bc0 2024-03-22T06:05:55.715 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.645+0000 7efd51e7f700 1 -- 172.21.15.6:0/2687303503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4c1143e0 con 0x7efd4c102bc0 2024-03-22T06:05:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.645+0000 7efd51e7f700 1 -- 172.21.15.6:0/2687303503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c114870 con 0x7efd4c102bc0 2024-03-22T06:05:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.645+0000 7efd33fff700 1 -- 172.21.15.6:0/2687303503 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd3c01adb0 con 0x7efd4c102bc0 2024-03-22T06:05:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.645+0000 7efd33fff700 1 -- 172.21.15.6:0/2687303503 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7efd3c0193c0 con 0x7efd4c102bc0 2024-03-22T06:05:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.646+0000 7efd33fff700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 0x7efd34038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.646+0000 7efd51e7f700 1 -- 172.21.15.6:0/2687303503 --> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd38000d40 con 0x7efd340367e0 2024-03-22T06:05:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.646+0000 7efd33fff700 1 -- 172.21.15.6:0/2687303503 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7efd3c04afd0 con 0x7efd4c102bc0 2024-03-22T06:05:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.646+0000 7efd4affd700 1 -- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 msgr2=0x7efd34038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3099962970 2024-03-22T06:05:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.646+0000 7efd4affd700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 0x7efd34038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T06:05:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.847+0000 7efd4affd700 1 -- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 msgr2=0x7efd34038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3099962970 2024-03-22T06:05:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:49.847+0000 7efd4affd700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 0x7efd34038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T06:05:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:50.248+0000 7efd4affd700 1 -- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 msgr2=0x7efd34038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3099962970 2024-03-22T06:05:55.717 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:50.248+0000 7efd4affd700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 0x7efd34038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T06:05:55.717 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:51.049+0000 7efd4affd700 1 -- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 msgr2=0x7efd34038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3099962970 2024-03-22T06:05:55.717 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:51.049+0000 7efd4affd700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 0x7efd34038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T06:05:55.717 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:52.650+0000 7efd4affd700 1 -- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 msgr2=0x7efd34038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3099962970 2024-03-22T06:05:55.717 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:52.650+0000 7efd4affd700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 0x7efd34038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-22T06:05:55.717 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:54.101+0000 7efd33fff700 1 -- 172.21.15.6:0/2687303503 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7efd3c021d40 con 0x7efd4c102bc0 2024-03-22T06:05:55.717 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:54.102+0000 7efd33fff700 1 -- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 msgr2=0x7efd34038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:54.102+0000 7efd33fff700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 0x7efd34038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.103+0000 7efd33fff700 1 -- 172.21.15.6:0/2687303503 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7efd3c00eb20 con 0x7efd4c102bc0 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.103+0000 7efd33fff700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efd34039f50 0x7efd3403c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.103+0000 7efd33fff700 1 -- 172.21.15.6:0/2687303503 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd38000d40 con 0x7efd34039f50 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.103+0000 7efd4affd700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efd34039f50 0x7efd3403c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.104+0000 7efd4affd700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efd34039f50 0x7efd3403c330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efd400039d0 tx=0x7efd40009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.104+0000 7efd33fff700 1 -- 172.21.15.6:0/2687303503 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7efd38000d40 con 0x7efd34039f50 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.110+0000 7efd51e7f700 1 -- 172.21.15.6:0/2687303503 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7efd380027c0 con 0x7efd34039f50 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd33fff700 1 -- 172.21.15.6:0/2687303503 <== mgr.14120 v2:172.21.15.6:6800/3903206979 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7efd380027c0 con 0x7efd34039f50 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd51e7f700 1 -- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efd34039f50 msgr2=0x7efd3403c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd51e7f700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efd34039f50 0x7efd3403c330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efd400039d0 tx=0x7efd40009350 comp rx=0 tx=0).stop 2024-03-22T06:05:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd51e7f700 1 -- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 msgr2=0x7efd4c101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:55.719 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd51e7f700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c101390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efd3c006b00 tx=0x7efd3c009250 comp rx=0 tx=0).stop 2024-03-22T06:05:55.719 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd51e7f700 1 -- 172.21.15.6:0/2687303503 shutdown_connections 2024-03-22T06:05:55.719 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd51e7f700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efd34039f50 0x7efd3403c330 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:55.719 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd51e7f700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4c102bc0 0x7efd4c101390 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:55.719 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd51e7f700 1 --2- 172.21.15.6:0/2687303503 >> [v2:172.21.15.6:6800/3099962970,v1:172.21.15.6:6801/3099962970] conn(0x7efd340367e0 0x7efd34038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:55.719 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd51e7f700 1 -- 172.21.15.6:0/2687303503 >> 172.21.15.6:0/2687303503 conn(0x7efd4c0fc4a0 msgr2=0x7efd4c0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:55.719 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd51e7f700 1 -- 172.21.15.6:0/2687303503 shutdown_connections 2024-03-22T06:05:55.719 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:55.111+0000 7efd51e7f700 1 -- 172.21.15.6:0/2687303503 wait complete. 2024-03-22T06:05:55.719 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 5 is available 2024-03-22T06:05:55.719 INFO:teuthology.orchestra.run.smithi006.stdout:Setting orchestrator backend to cephadm... 2024-03-22T06:05:56.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:56 smithi006 ceph-mon[29301]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-22T06:05:56.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:56 smithi006 ceph-mon[29301]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-22T06:05:56.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:56 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:05:56.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.077+0000 7f4c88128700 1 Processor -- start 2024-03-22T06:05:56.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.077+0000 7f4c88128700 1 -- start start 2024-03-22T06:05:56.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.077+0000 7f4c88128700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:56.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.077+0000 7f4c88128700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c801034e0 con 0x7f4c80102b90 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.078+0000 7f4c85ec4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.078+0000 7f4c85ec4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80102fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39440/0 (socket says 172.21.15.6:39440) 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.078+0000 7f4c85ec4700 1 -- 172.21.15.6:0/990954377 learned_addr learned my addr 172.21.15.6:0/990954377 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.078+0000 7f4c85ec4700 1 -- 172.21.15.6:0/990954377 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c80103620 con 0x7f4c80102b90 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.078+0000 7f4c85ec4700 1 --2- 172.21.15.6:0/990954377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80102fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c740098e0 tx=0x7f4c74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbbc02fe19f8398 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.078+0000 7f4c84ec2700 1 -- 172.21.15.6:0/990954377 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c7400b860 con 0x7f4c80102b90 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.078+0000 7f4c84ec2700 1 -- 172.21.15.6:0/990954377 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4c7400be50 con 0x7f4c80102b90 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.078+0000 7f4c84ec2700 1 -- 172.21.15.6:0/990954377 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c74010380 con 0x7f4c80102b90 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.079+0000 7f4c88128700 1 -- 172.21.15.6:0/990954377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 msgr2=0x7f4c80102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.079+0000 7f4c88128700 1 --2- 172.21.15.6:0/990954377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80102fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c740098e0 tx=0x7f4c74009250 comp rx=0 tx=0).stop 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.079+0000 7f4c88128700 1 -- 172.21.15.6:0/990954377 shutdown_connections 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.079+0000 7f4c88128700 1 --2- 172.21.15.6:0/990954377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80102fa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.079+0000 7f4c88128700 1 -- 172.21.15.6:0/990954377 >> 172.21.15.6:0/990954377 conn(0x7f4c800fc4d0 msgr2=0x7f4c800fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:56.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.079+0000 7f4c88128700 1 -- 172.21.15.6:0/990954377 shutdown_connections 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.079+0000 7f4c88128700 1 -- 172.21.15.6:0/990954377 wait complete. 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.079+0000 7f4c88128700 1 Processor -- start 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c88128700 1 -- start start 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c88128700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80110460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c88128700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c801109a0 con 0x7f4c80102b90 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c85ec4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80110460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c85ec4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80110460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39450/0 (socket says 172.21.15.6:39450) 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c85ec4700 1 -- 172.21.15.6:0/4305461 learned_addr learned my addr 172.21.15.6:0/4305461 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c85ec4700 1 -- 172.21.15.6:0/4305461 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740094f0 con 0x7f4c80102b90 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c85ec4700 1 --2- 172.21.15.6:0/4305461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80110460 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400b440 tx=0x7f4c74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c727fc700 1 -- 172.21.15.6:0/4305461 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c74010760 con 0x7f4c80102b90 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c727fc700 1 -- 172.21.15.6:0/4305461 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4c74010d50 con 0x7f4c80102b90 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c88128700 1 -- 172.21.15.6:0/4305461 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c80110ba0 con 0x7f4c80102b90 2024-03-22T06:05:56.885 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c727fc700 1 -- 172.21.15.6:0/4305461 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c7401ae20 con 0x7f4c80102b90 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.080+0000 7f4c88128700 1 -- 172.21.15.6:0/4305461 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c80111710 con 0x7f4c80102b90 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.081+0000 7f4c727fc700 1 -- 172.21.15.6:0/4305461 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f4c7402e440 con 0x7f4c80102b90 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.081+0000 7f4c88128700 1 -- 172.21.15.6:0/4305461 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8005f5f0 con 0x7f4c80102b90 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.081+0000 7f4c727fc700 1 --2- 172.21.15.6:0/4305461 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4c6c036660 0x7f4c6c038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.081+0000 7f4c727fc700 1 -- 172.21.15.6:0/4305461 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4c7404a590 con 0x7f4c80102b90 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.081+0000 7f4c856c3700 1 --2- 172.21.15.6:0/4305461 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4c6c036660 0x7f4c6c038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.082+0000 7f4c856c3700 1 --2- 172.21.15.6:0/4305461 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4c6c036660 0x7f4c6c038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009990 tx=0x7f4c7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.086+0000 7f4c727fc700 1 -- 172.21.15.6:0/4305461 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4c74018940 con 0x7f4c80102b90 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.154+0000 7f4c727fc700 1 -- 172.21.15.6:0/4305461 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f4c740243f0 con 0x7f4c80102b90 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.298+0000 7f4c88128700 1 -- 172.21.15.6:0/4305461 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c800fd780 con 0x7f4c6c036660 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.308+0000 7f4c727fc700 1 -- 172.21.15.6:0/4305461 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4c800fd780 con 0x7f4c6c036660 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.310+0000 7f4c88128700 1 -- 172.21.15.6:0/4305461 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4c6c036660 msgr2=0x7f4c6c038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.310+0000 7f4c88128700 1 --2- 172.21.15.6:0/4305461 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4c6c036660 0x7f4c6c038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009990 tx=0x7f4c7c006e40 comp rx=0 tx=0).stop 2024-03-22T06:05:56.886 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.310+0000 7f4c88128700 1 -- 172.21.15.6:0/4305461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 msgr2=0x7f4c80110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:56.887 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.310+0000 7f4c88128700 1 --2- 172.21.15.6:0/4305461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80110460 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400b440 tx=0x7f4c74009250 comp rx=0 tx=0).stop 2024-03-22T06:05:56.887 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.310+0000 7f4c88128700 1 -- 172.21.15.6:0/4305461 shutdown_connections 2024-03-22T06:05:56.887 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.310+0000 7f4c88128700 1 --2- 172.21.15.6:0/4305461 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4c6c036660 0x7f4c6c038b10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:56.887 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.310+0000 7f4c88128700 1 --2- 172.21.15.6:0/4305461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c80102b90 0x7f4c80110460 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:56.887 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.310+0000 7f4c88128700 1 -- 172.21.15.6:0/4305461 >> 172.21.15.6:0/4305461 conn(0x7f4c800fc4d0 msgr2=0x7f4c800fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:56.887 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.310+0000 7f4c88128700 1 -- 172.21.15.6:0/4305461 shutdown_connections 2024-03-22T06:05:56.887 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:56.310+0000 7f4c88128700 1 -- 172.21.15.6:0/4305461 wait complete. 2024-03-22T06:05:57.520 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:57 smithi006 ceph-mon[29301]: [22/Mar/2024:06:05:56] ENGINE Bus STARTING 2024-03-22T06:05:57.520 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:57 smithi006 ceph-mon[29301]: [22/Mar/2024:06:05:56] ENGINE Serving on https://172.21.15.6:7150 2024-03-22T06:05:57.520 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:57 smithi006 ceph-mon[29301]: [22/Mar/2024:06:05:56] ENGINE Bus STARTED 2024-03-22T06:05:57.520 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:57 smithi006 ceph-mon[29301]: mgrmap e8: smithi006.sfpxdy(active, since 2s) 2024-03-22T06:05:57.520 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:57 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:05:57.520 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:57 smithi006 ceph-mon[29301]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:05:57.520 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:57 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:05:57.520 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:57 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:05:58.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.312+0000 7eff2d19e700 1 Processor -- start 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.312+0000 7eff2d19e700 1 -- start start 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.312+0000 7eff2d19e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.312+0000 7eff2d19e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff28103070 con 0x7eff28102720 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.312+0000 7eff26d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.312+0000 7eff26d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39462/0 (socket says 172.21.15.6:39462) 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.312+0000 7eff26d9d700 1 -- 172.21.15.6:0/3001618288 learned_addr learned my addr 172.21.15.6:0/3001618288 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.312+0000 7eff26d9d700 1 -- 172.21.15.6:0/3001618288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff281031b0 con 0x7eff28102720 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.313+0000 7eff26d9d700 1 --2- 172.21.15.6:0/3001618288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28102b30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff180098e0 tx=0x7eff18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d681ec567d820106 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.313+0000 7eff25d9b700 1 -- 172.21.15.6:0/3001618288 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff1800b860 con 0x7eff28102720 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.313+0000 7eff25d9b700 1 -- 172.21.15.6:0/3001618288 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7eff1800be50 con 0x7eff28102720 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.313+0000 7eff25d9b700 1 -- 172.21.15.6:0/3001618288 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff18010380 con 0x7eff28102720 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.313+0000 7eff2d19e700 1 -- 172.21.15.6:0/3001618288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 msgr2=0x7eff28102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.313+0000 7eff2d19e700 1 --2- 172.21.15.6:0/3001618288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28102b30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff180098e0 tx=0x7eff18009250 comp rx=0 tx=0).stop 2024-03-22T06:05:58.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.313+0000 7eff2d19e700 1 -- 172.21.15.6:0/3001618288 shutdown_connections 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.313+0000 7eff2d19e700 1 --2- 172.21.15.6:0/3001618288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28102b30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.313+0000 7eff2d19e700 1 -- 172.21.15.6:0/3001618288 >> 172.21.15.6:0/3001618288 conn(0x7eff280fdee0 msgr2=0x7eff281002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.313+0000 7eff2d19e700 1 -- 172.21.15.6:0/3001618288 shutdown_connections 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.313+0000 7eff2d19e700 1 -- 172.21.15.6:0/3001618288 wait complete. 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.314+0000 7eff2d19e700 1 Processor -- start 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.314+0000 7eff2d19e700 1 -- start start 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.314+0000 7eff2d19e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.314+0000 7eff2d19e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff28079110 con 0x7eff28102720 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.314+0000 7eff26d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.314+0000 7eff26d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39476/0 (socket says 172.21.15.6:39476) 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.314+0000 7eff26d9d700 1 -- 172.21.15.6:0/2302309426 learned_addr learned my addr 172.21.15.6:0/2302309426 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.315+0000 7eff26d9d700 1 -- 172.21.15.6:0/2302309426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff180094f0 con 0x7eff28102720 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.315+0000 7eff26d9d700 1 --2- 172.21.15.6:0/2302309426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28078bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff1800b440 tx=0x7eff18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:58.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.315+0000 7eff0f7fe700 1 -- 172.21.15.6:0/2302309426 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff18010760 con 0x7eff28102720 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.315+0000 7eff0f7fe700 1 -- 172.21.15.6:0/2302309426 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7eff18010d50 con 0x7eff28102720 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.315+0000 7eff2d19e700 1 -- 172.21.15.6:0/2302309426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff28077180 con 0x7eff28102720 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.315+0000 7eff0f7fe700 1 -- 172.21.15.6:0/2302309426 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff1801ae20 con 0x7eff28102720 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.315+0000 7eff2d19e700 1 -- 172.21.15.6:0/2302309426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff28077670 con 0x7eff28102720 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.316+0000 7eff0f7fe700 1 -- 172.21.15.6:0/2302309426 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7eff1802e440 con 0x7eff28102720 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.316+0000 7eff2d19e700 1 -- 172.21.15.6:0/2302309426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2804bed0 con 0x7eff28102720 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.316+0000 7eff0f7fe700 1 --2- 172.21.15.6:0/2302309426 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff10036770 0x7eff10038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.316+0000 7eff0f7fe700 1 -- 172.21.15.6:0/2302309426 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7eff1804a5d0 con 0x7eff28102720 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.316+0000 7eff2659c700 1 --2- 172.21.15.6:0/2302309426 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff10036770 0x7eff10038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.317+0000 7eff2659c700 1 --2- 172.21.15.6:0/2302309426 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff10036770 0x7eff10038c20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009990 tx=0x7eff1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.321+0000 7eff0f7fe700 1 -- 172.21.15.6:0/2302309426 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7eff18018530 con 0x7eff28102720 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.519+0000 7eff2d19e700 1 -- 172.21.15.6:0/2302309426 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7eff28077d80 con 0x7eff10036770 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.520+0000 7eff0f7fe700 1 -- 172.21.15.6:0/2302309426 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7eff28077d80 con 0x7eff10036770 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.521+0000 7eff2d19e700 1 -- 172.21.15.6:0/2302309426 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff10036770 msgr2=0x7eff10038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.521+0000 7eff2d19e700 1 --2- 172.21.15.6:0/2302309426 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff10036770 0x7eff10038c20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009990 tx=0x7eff1c006e40 comp rx=0 tx=0).stop 2024-03-22T06:05:58.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.521+0000 7eff2d19e700 1 -- 172.21.15.6:0/2302309426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 msgr2=0x7eff28078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:58.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.521+0000 7eff2d19e700 1 --2- 172.21.15.6:0/2302309426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28078bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff1800b440 tx=0x7eff18009250 comp rx=0 tx=0).stop 2024-03-22T06:05:58.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.521+0000 7eff2d19e700 1 -- 172.21.15.6:0/2302309426 shutdown_connections 2024-03-22T06:05:58.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.521+0000 7eff2d19e700 1 --2- 172.21.15.6:0/2302309426 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff10036770 0x7eff10038c20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:58.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.521+0000 7eff2d19e700 1 --2- 172.21.15.6:0/2302309426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff28102720 0x7eff28078bd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:58.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.521+0000 7eff2d19e700 1 -- 172.21.15.6:0/2302309426 >> 172.21.15.6:0/2302309426 conn(0x7eff280fdee0 msgr2=0x7eff280fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:58.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.522+0000 7eff2d19e700 1 -- 172.21.15.6:0/2302309426 shutdown_connections 2024-03-22T06:05:58.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:57.522+0000 7eff2d19e700 1 -- 172.21.15.6:0/2302309426 wait complete. 2024-03-22T06:05:58.060 INFO:teuthology.orchestra.run.smithi006.stdout:Generating ssh key... 2024-03-22T06:05:58.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:05:58 smithi006 ceph-mon[29301]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:05:59.829 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.513+0000 7f029a810700 1 Processor -- start 2024-03-22T06:05:59.829 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.513+0000 7f029a810700 1 -- start start 2024-03-22T06:05:59.829 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.513+0000 7f029a810700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f0294101a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.513+0000 7f029a810700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0294101f50 con 0x7f0294101600 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.513+0000 7f0293fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f0294101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.513+0000 7f0293fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f0294101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39486/0 (socket says 172.21.15.6:39486) 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.513+0000 7f0293fff700 1 -- 172.21.15.6:0/3108475262 learned_addr learned my addr 172.21.15.6:0/3108475262 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.514+0000 7f0293fff700 1 -- 172.21.15.6:0/3108475262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294102090 con 0x7f0294101600 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.514+0000 7f0293fff700 1 --2- 172.21.15.6:0/3108475262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f0294101a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f02840098e0 tx=0x7f0284009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2d11d3092a387f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.515+0000 7f0292ffd700 1 -- 172.21.15.6:0/3108475262 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f028400b860 con 0x7f0294101600 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.515+0000 7f0292ffd700 1 -- 172.21.15.6:0/3108475262 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f028400be50 con 0x7f0294101600 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.515+0000 7f0292ffd700 1 -- 172.21.15.6:0/3108475262 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0284010380 con 0x7f0294101600 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.516+0000 7f029a810700 1 -- 172.21.15.6:0/3108475262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 msgr2=0x7f0294101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.516+0000 7f029a810700 1 --2- 172.21.15.6:0/3108475262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f0294101a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f02840098e0 tx=0x7f0284009250 comp rx=0 tx=0).stop 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.516+0000 7f029a810700 1 -- 172.21.15.6:0/3108475262 shutdown_connections 2024-03-22T06:05:59.830 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.516+0000 7f029a810700 1 --2- 172.21.15.6:0/3108475262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f0294101a10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.516+0000 7f029a810700 1 -- 172.21.15.6:0/3108475262 >> 172.21.15.6:0/3108475262 conn(0x7f02940fcd80 msgr2=0x7f02940ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.516+0000 7f029a810700 1 -- 172.21.15.6:0/3108475262 shutdown_connections 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.516+0000 7f029a810700 1 -- 172.21.15.6:0/3108475262 wait complete. 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.517+0000 7f029a810700 1 Processor -- start 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.517+0000 7f029a810700 1 -- start start 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.517+0000 7f029a810700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f02941078a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.517+0000 7f029a810700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0294107de0 con 0x7f0294101600 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.517+0000 7f0293fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f02941078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.517+0000 7f0293fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f02941078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39496/0 (socket says 172.21.15.6:39496) 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.517+0000 7f0293fff700 1 -- 172.21.15.6:0/3986543640 learned_addr learned my addr 172.21.15.6:0/3986543640 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.517+0000 7f0293fff700 1 -- 172.21.15.6:0/3986543640 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02840094f0 con 0x7f0294101600 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.518+0000 7f0293fff700 1 --2- 172.21.15.6:0/3986543640 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f02941078a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0284000c00 tx=0x7f0284009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.518+0000 7f0290ff9700 1 -- 172.21.15.6:0/3986543640 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02840107c0 con 0x7f0294101600 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.518+0000 7f0290ff9700 1 -- 172.21.15.6:0/3986543640 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0284010db0 con 0x7f0294101600 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.518+0000 7f029a810700 1 -- 172.21.15.6:0/3986543640 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0294105e50 con 0x7f0294101600 2024-03-22T06:05:59.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.518+0000 7f0290ff9700 1 -- 172.21.15.6:0/3986543640 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f028401ad00 con 0x7f0294101600 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.518+0000 7f029a810700 1 -- 172.21.15.6:0/3986543640 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0294106340 con 0x7f0294101600 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.519+0000 7f0290ff9700 1 -- 172.21.15.6:0/3986543640 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f02840193c0 con 0x7f0294101600 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.519+0000 7f029a810700 1 -- 172.21.15.6:0/3986543640 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029404bed0 con 0x7f0294101600 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.519+0000 7f0290ff9700 1 --2- 172.21.15.6:0/3986543640 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f027c0367e0 0x7f027c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.519+0000 7f0290ff9700 1 -- 172.21.15.6:0/3986543640 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f028404a650 con 0x7f0294101600 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.519+0000 7f02937fe700 1 --2- 172.21.15.6:0/3986543640 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f027c0367e0 0x7f027c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.519+0000 7f02937fe700 1 --2- 172.21.15.6:0/3986543640 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f027c0367e0 0x7f027c038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0288009990 tx=0x7f0288006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.524+0000 7f0290ff9700 1 -- 172.21.15.6:0/3986543640 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0284022930 con 0x7f0294101600 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:58.718+0000 7f029a810700 1 -- 172.21.15.6:0/3986543640 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0294106b00 con 0x7f027c0367e0 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:59.275+0000 7f0290ff9700 1 -- 172.21.15.6:0/3986543640 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0294106b00 con 0x7f027c0367e0 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:59.277+0000 7f029a810700 1 -- 172.21.15.6:0/3986543640 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f027c0367e0 msgr2=0x7f027c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:59.277+0000 7f029a810700 1 --2- 172.21.15.6:0/3986543640 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f027c0367e0 0x7f027c038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0288009990 tx=0x7f0288006e40 comp rx=0 tx=0).stop 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:59.277+0000 7f029a810700 1 -- 172.21.15.6:0/3986543640 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 msgr2=0x7f02941078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:05:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:59.277+0000 7f029a810700 1 --2- 172.21.15.6:0/3986543640 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f02941078a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0284000c00 tx=0x7f0284009250 comp rx=0 tx=0).stop 2024-03-22T06:05:59.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:59.277+0000 7f029a810700 1 -- 172.21.15.6:0/3986543640 shutdown_connections 2024-03-22T06:05:59.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:59.277+0000 7f029a810700 1 --2- 172.21.15.6:0/3986543640 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f027c0367e0 0x7f027c038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:59.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:59.277+0000 7f029a810700 1 --2- 172.21.15.6:0/3986543640 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0294101600 0x7f02941078a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:05:59.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:59.277+0000 7f029a810700 1 -- 172.21.15.6:0/3986543640 >> 172.21.15.6:0/3986543640 conn(0x7f02940fcd80 msgr2=0x7f02940fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:05:59.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:59.278+0000 7f029a810700 1 -- 172.21.15.6:0/3986543640 shutdown_connections 2024-03-22T06:05:59.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:05:59.278+0000 7f029a810700 1 -- 172.21.15.6:0/3986543640 wait complete. 2024-03-22T06:06:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:00 smithi006 ceph-mon[29301]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:00 smithi006 ceph-mon[29301]: Generating ssh key... 2024-03-22T06:06:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:00 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:00 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:01.053 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0j9Tiiux0asll9l2rlt8oDlx3SU70B5W7rn83trBtd195dUCkCff9FYOTvRB6ckYiXJVJaIuRCc13G8AJ/3OaK4Vo9FOKwzMsNVn9FKIet1OG/TYGbEQ+t3097uRJGImZmpXJsrbkgCqPaWipHp58UombAv8XLCJlh8TFXZsw9pmZ25ooiL0eXqyhYVYNRSaa2j1rxk5of1sANL8uH5OkUw7DLKFfXpm/edsiYcXXczCOg25UMb5KsLa2WEbNhr54RRTY2j4g3lS7VgBDrjLM1oZA7jRIdHuxevCqmUHU6K9kCPOnMqkb+/whv3QmZE69UPSFgGHYY9+LRwBuLW8vhlgI4ZuzAKUgC0qz6zmm5dRksnyZ2+hwu9keMjE664XbnfEM05WilCHr1rXylSSmvD6ehq1wXC9ISRFymfoYRj7WA04Vte1NBGQuPb00293nH8jrfPMjGwBUepDO/11xkpqQEAgBt3Y5MNnkzHbjSZ10mrAKLoP4/U0lNGCJRTM= ceph-fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.212+0000 7fe36a80a700 1 Processor -- start 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.213+0000 7fe36a80a700 1 -- start start 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.213+0000 7fe36a80a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe364103070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.213+0000 7fe36a80a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641035b0 con 0x7fe364102c60 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.213+0000 7fe363fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe364103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.213+0000 7fe363fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe364103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39510/0 (socket says 172.21.15.6:39510) 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.213+0000 7fe363fff700 1 -- 172.21.15.6:0/371146331 learned_addr learned my addr 172.21.15.6:0/371146331 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.213+0000 7fe363fff700 1 -- 172.21.15.6:0/371146331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3641036f0 con 0x7fe364102c60 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.213+0000 7fe363fff700 1 --2- 172.21.15.6:0/371146331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe364103070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe3540098e0 tx=0x7fe354009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3e43322665c6ea server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.213+0000 7fe362ffd700 1 -- 172.21.15.6:0/371146331 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe35400b860 con 0x7fe364102c60 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.214+0000 7fe362ffd700 1 -- 172.21.15.6:0/371146331 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe35400be50 con 0x7fe364102c60 2024-03-22T06:06:01.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.214+0000 7fe362ffd700 1 -- 172.21.15.6:0/371146331 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe354010380 con 0x7fe364102c60 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.214+0000 7fe36a80a700 1 -- 172.21.15.6:0/371146331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 msgr2=0x7fe364103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.214+0000 7fe36a80a700 1 --2- 172.21.15.6:0/371146331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe364103070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe3540098e0 tx=0x7fe354009250 comp rx=0 tx=0).stop 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.214+0000 7fe36a80a700 1 -- 172.21.15.6:0/371146331 shutdown_connections 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.214+0000 7fe36a80a700 1 --2- 172.21.15.6:0/371146331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe364103070 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.214+0000 7fe36a80a700 1 -- 172.21.15.6:0/371146331 >> 172.21.15.6:0/371146331 conn(0x7fe3640fc560 msgr2=0x7fe3640fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.214+0000 7fe36a80a700 1 -- 172.21.15.6:0/371146331 shutdown_connections 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.214+0000 7fe36a80a700 1 -- 172.21.15.6:0/371146331 wait complete. 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.215+0000 7fe36a80a700 1 Processor -- start 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.215+0000 7fe36a80a700 1 -- start start 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.215+0000 7fe36a80a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe3641104b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.215+0000 7fe36a80a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641109f0 con 0x7fe364102c60 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.215+0000 7fe363fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe3641104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.215+0000 7fe363fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe3641104b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39526/0 (socket says 172.21.15.6:39526) 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.215+0000 7fe363fff700 1 -- 172.21.15.6:0/2660158526 learned_addr learned my addr 172.21.15.6:0/2660158526 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:01.055 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.215+0000 7fe363fff700 1 -- 172.21.15.6:0/2660158526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3540094f0 con 0x7fe364102c60 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.216+0000 7fe363fff700 1 --2- 172.21.15.6:0/2660158526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe3641104b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe354006b00 tx=0x7fe354009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.216+0000 7fe360ff9700 1 -- 172.21.15.6:0/2660158526 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe354010760 con 0x7fe364102c60 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.216+0000 7fe360ff9700 1 -- 172.21.15.6:0/2660158526 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe354010d50 con 0x7fe364102c60 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.216+0000 7fe36a80a700 1 -- 172.21.15.6:0/2660158526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe364110bf0 con 0x7fe364102c60 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.216+0000 7fe360ff9700 1 -- 172.21.15.6:0/2660158526 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe35401ad40 con 0x7fe364102c60 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.216+0000 7fe36a80a700 1 -- 172.21.15.6:0/2660158526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe364111760 con 0x7fe364102c60 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.217+0000 7fe360ff9700 1 -- 172.21.15.6:0/2660158526 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fe35402e440 con 0x7fe364102c60 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.217+0000 7fe360ff9700 1 --2- 172.21.15.6:0/2660158526 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fe350036790 0x7fe350038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.217+0000 7fe360ff9700 1 -- 172.21.15.6:0/2660158526 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe35404a5b0 con 0x7fe364102c60 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.217+0000 7fe3637fe700 1 --2- 172.21.15.6:0/2660158526 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fe350036790 0x7fe350038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.217+0000 7fe36a80a700 1 -- 172.21.15.6:0/2660158526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36405f5f0 con 0x7fe364102c60 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.217+0000 7fe3637fe700 1 --2- 172.21.15.6:0/2660158526 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fe350036790 0x7fe350038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009990 tx=0x7fe34c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.222+0000 7fe360ff9700 1 -- 172.21.15.6:0/2660158526 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe354018940 con 0x7fe364102c60 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.419+0000 7fe36a80a700 1 -- 172.21.15.6:0/2660158526 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3640fd7f0 con 0x7fe350036790 2024-03-22T06:06:01.056 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.420+0000 7fe360ff9700 1 -- 172.21.15.6:0/2660158526 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fe3640fd7f0 con 0x7fe350036790 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.422+0000 7fe36a80a700 1 -- 172.21.15.6:0/2660158526 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fe350036790 msgr2=0x7fe350038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.422+0000 7fe36a80a700 1 --2- 172.21.15.6:0/2660158526 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fe350036790 0x7fe350038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009990 tx=0x7fe34c006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.422+0000 7fe36a80a700 1 -- 172.21.15.6:0/2660158526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 msgr2=0x7fe3641104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.422+0000 7fe36a80a700 1 --2- 172.21.15.6:0/2660158526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe3641104b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe354006b00 tx=0x7fe354009250 comp rx=0 tx=0).stop 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.422+0000 7fe36a80a700 1 -- 172.21.15.6:0/2660158526 shutdown_connections 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.422+0000 7fe36a80a700 1 --2- 172.21.15.6:0/2660158526 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fe350036790 0x7fe350038c40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.422+0000 7fe36a80a700 1 --2- 172.21.15.6:0/2660158526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe364102c60 0x7fe3641104b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.422+0000 7fe36a80a700 1 -- 172.21.15.6:0/2660158526 >> 172.21.15.6:0/2660158526 conn(0x7fe3640fc560 msgr2=0x7fe3640fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.422+0000 7fe36a80a700 1 -- 172.21.15.6:0/2660158526 shutdown_connections 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:00.422+0000 7fe36a80a700 1 -- 172.21.15.6:0/2660158526 wait complete. 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:Adding key to root@localhost authorized_keys... 2024-03-22T06:06:01.057 INFO:teuthology.orchestra.run.smithi006.stdout:Adding host smithi006... 2024-03-22T06:06:01.340 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:01 smithi006 ceph-mon[29301]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:02.642 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:02 smithi006 ceph-mon[29301]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:02.642 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:02 smithi006 ceph-mon[29301]: Deploying cephadm binary to smithi006 2024-03-22T06:06:04.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Added host 'smithi006' with addr '172.21.15.6' 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.441+0000 7f86a922a700 1 Processor -- start 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.441+0000 7f86a922a700 1 -- start start 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.442+0000 7f86a922a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4101a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.442+0000 7f86a922a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a4101f70 con 0x7f86a4101620 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.442+0000 7f86a2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.442+0000 7f86a2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39532/0 (socket says 172.21.15.6:39532) 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.442+0000 7f86a2d9d700 1 -- 172.21.15.6:0/1655082336 learned_addr learned my addr 172.21.15.6:0/1655082336 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.442+0000 7f86a2d9d700 1 -- 172.21.15.6:0/1655082336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a41020b0 con 0x7f86a4101620 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.442+0000 7f86a2d9d700 1 --2- 172.21.15.6:0/1655082336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4101a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f86940098e0 tx=0x7f8694009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22796392950b6682 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.442+0000 7f86a1d9b700 1 -- 172.21.15.6:0/1655082336 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f869400b860 con 0x7f86a4101620 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.442+0000 7f86a1d9b700 1 -- 172.21.15.6:0/1655082336 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f869400be50 con 0x7f86a4101620 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.442+0000 7f86a1d9b700 1 -- 172.21.15.6:0/1655082336 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8694010380 con 0x7f86a4101620 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.443+0000 7f86a922a700 1 -- 172.21.15.6:0/1655082336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 msgr2=0x7f86a4101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.443+0000 7f86a922a700 1 --2- 172.21.15.6:0/1655082336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4101a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f86940098e0 tx=0x7f8694009250 comp rx=0 tx=0).stop 2024-03-22T06:06:04.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.443+0000 7f86a922a700 1 -- 172.21.15.6:0/1655082336 shutdown_connections 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.443+0000 7f86a922a700 1 --2- 172.21.15.6:0/1655082336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4101a30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.443+0000 7f86a922a700 1 -- 172.21.15.6:0/1655082336 >> 172.21.15.6:0/1655082336 conn(0x7f86a40fcdc0 msgr2=0x7f86a40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.443+0000 7f86a922a700 1 -- 172.21.15.6:0/1655082336 shutdown_connections 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.443+0000 7f86a922a700 1 -- 172.21.15.6:0/1655082336 wait complete. 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.444+0000 7f86a922a700 1 Processor -- start 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.444+0000 7f86a922a700 1 -- start start 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.444+0000 7f86a922a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4076b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.444+0000 7f86a922a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a4114220 con 0x7f86a4101620 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.444+0000 7f86a2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4076b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.444+0000 7f86a2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4076b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39534/0 (socket says 172.21.15.6:39534) 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.444+0000 7f86a2d9d700 1 -- 172.21.15.6:0/518354452 learned_addr learned my addr 172.21.15.6:0/518354452 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.444+0000 7f86a2d9d700 1 -- 172.21.15.6:0/518354452 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940094f0 con 0x7f86a4101620 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.444+0000 7f86a2d9d700 1 --2- 172.21.15.6:0/518354452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4076b10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f869400b440 tx=0x7f8694009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:04.205 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.445+0000 7f868b7fe700 1 -- 172.21.15.6:0/518354452 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8694010760 con 0x7f86a4101620 2024-03-22T06:06:04.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.445+0000 7f868b7fe700 1 -- 172.21.15.6:0/518354452 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8694010d50 con 0x7f86a4101620 2024-03-22T06:06:04.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.445+0000 7f86a922a700 1 -- 172.21.15.6:0/518354452 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a4114360 con 0x7f86a4101620 2024-03-22T06:06:04.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.445+0000 7f868b7fe700 1 -- 172.21.15.6:0/518354452 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f869401ae20 con 0x7f86a4101620 2024-03-22T06:06:04.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.445+0000 7f86a922a700 1 -- 172.21.15.6:0/518354452 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a4114850 con 0x7f86a4101620 2024-03-22T06:06:04.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.445+0000 7f868b7fe700 1 -- 172.21.15.6:0/518354452 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f869402e440 con 0x7f86a4101620 2024-03-22T06:06:04.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.446+0000 7f868b7fe700 1 --2- 172.21.15.6:0/518354452 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f868c0367e0 0x7f868c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:04.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.446+0000 7f86a922a700 1 -- 172.21.15.6:0/518354452 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a404bed0 con 0x7f86a4101620 2024-03-22T06:06:04.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.446+0000 7f868b7fe700 1 -- 172.21.15.6:0/518354452 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f869404a710 con 0x7f86a4101620 2024-03-22T06:06:04.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.446+0000 7f86a259c700 1 --2- 172.21.15.6:0/518354452 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f868c0367e0 0x7f868c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:04.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.446+0000 7f86a259c700 1 --2- 172.21.15.6:0/518354452 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f868c0367e0 0x7f868c038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8698009990 tx=0x7f8698006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:04.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.450+0000 7f868b7fe700 1 -- 172.21.15.6:0/518354452 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8694018710 con 0x7f86a4101620 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:01.649+0000 7f86a922a700 1 -- 172.21.15.6:0/518354452 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}) v1 -- 0x7f86a40fe050 con 0x7f868c0367e0 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:03.681+0000 7f868b7fe700 1 -- 172.21.15.6:0/518354452 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f86a40fe050 con 0x7f868c0367e0 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:03.683+0000 7f86a922a700 1 -- 172.21.15.6:0/518354452 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f868c0367e0 msgr2=0x7f868c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:03.683+0000 7f86a922a700 1 --2- 172.21.15.6:0/518354452 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f868c0367e0 0x7f868c038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8698009990 tx=0x7f8698006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:03.683+0000 7f86a922a700 1 -- 172.21.15.6:0/518354452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 msgr2=0x7f86a4076b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:03.683+0000 7f86a922a700 1 --2- 172.21.15.6:0/518354452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4076b10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f869400b440 tx=0x7f8694009250 comp rx=0 tx=0).stop 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:03.683+0000 7f86a922a700 1 -- 172.21.15.6:0/518354452 shutdown_connections 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:03.683+0000 7f86a922a700 1 --2- 172.21.15.6:0/518354452 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f868c0367e0 0x7f868c038c90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:03.683+0000 7f86a922a700 1 --2- 172.21.15.6:0/518354452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a4101620 0x7f86a4076b10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:03.683+0000 7f86a922a700 1 -- 172.21.15.6:0/518354452 >> 172.21.15.6:0/518354452 conn(0x7f86a40fcdc0 msgr2=0x7f86a40fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:03.683+0000 7f86a922a700 1 -- 172.21.15.6:0/518354452 shutdown_connections 2024-03-22T06:06:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:03.683+0000 7f86a922a700 1 -- 172.21.15.6:0/518354452 wait complete. 2024-03-22T06:06:04.208 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying mon service with default placement... 2024-03-22T06:06:04.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:04 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:04.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:04 smithi006 ceph-mon[29301]: Added host smithi006 2024-03-22T06:06:04.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:04 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:06:05.707 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-22T06:06:05.707 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.600+0000 7f0e8fe5c700 1 Processor -- start 2024-03-22T06:06:05.707 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.600+0000 7f0e8fe5c700 1 -- start start 2024-03-22T06:06:05.707 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.600+0000 7f0e8fe5c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e88102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:05.707 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.600+0000 7f0e8fe5c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88103080 con 0x7f0e88102730 2024-03-22T06:06:05.707 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.600+0000 7f0e8dbf8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e88102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:05.707 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.601+0000 7f0e8dbf8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e88102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57064/0 (socket says 172.21.15.6:57064) 2024-03-22T06:06:05.707 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.601+0000 7f0e8dbf8700 1 -- 172.21.15.6:0/3220426957 learned_addr learned my addr 172.21.15.6:0/3220426957 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:05.707 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.601+0000 7f0e8dbf8700 1 -- 172.21.15.6:0/3220426957 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e881031c0 con 0x7f0e88102730 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.601+0000 7f0e8dbf8700 1 --2- 172.21.15.6:0/3220426957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e88102b40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e800098e0 tx=0x7f0e80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddb8de25b144807 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.601+0000 7f0e8cbf6700 1 -- 172.21.15.6:0/3220426957 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e8000b860 con 0x7f0e88102730 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.601+0000 7f0e8cbf6700 1 -- 172.21.15.6:0/3220426957 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0e8000be50 con 0x7f0e88102730 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.601+0000 7f0e8cbf6700 1 -- 172.21.15.6:0/3220426957 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e80010380 con 0x7f0e88102730 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.602+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/3220426957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 msgr2=0x7f0e88102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.602+0000 7f0e8fe5c700 1 --2- 172.21.15.6:0/3220426957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e88102b40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e800098e0 tx=0x7f0e80009250 comp rx=0 tx=0).stop 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.602+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/3220426957 shutdown_connections 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.602+0000 7f0e8fe5c700 1 --2- 172.21.15.6:0/3220426957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e88102b40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.602+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/3220426957 >> 172.21.15.6:0/3220426957 conn(0x7f0e880fdf10 msgr2=0x7f0e88100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.602+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/3220426957 shutdown_connections 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.602+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/3220426957 wait complete. 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.602+0000 7f0e8fe5c700 1 Processor -- start 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.602+0000 7f0e8fe5c700 1 -- start start 2024-03-22T06:06:05.708 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.602+0000 7f0e8fe5c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e881115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.602+0000 7f0e8fe5c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88111b30 con 0x7f0e88102730 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.603+0000 7f0e8dbf8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e881115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.603+0000 7f0e8dbf8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e881115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57076/0 (socket says 172.21.15.6:57076) 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.603+0000 7f0e8dbf8700 1 -- 172.21.15.6:0/2491764282 learned_addr learned my addr 172.21.15.6:0/2491764282 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.603+0000 7f0e8dbf8700 1 -- 172.21.15.6:0/2491764282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800094f0 con 0x7f0e88102730 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.603+0000 7f0e8dbf8700 1 --2- 172.21.15.6:0/2491764282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e881115f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e80006fd0 tx=0x7f0e80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.603+0000 7f0e7e7fc700 1 -- 172.21.15.6:0/2491764282 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e800107c0 con 0x7f0e88102730 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.603+0000 7f0e7e7fc700 1 -- 172.21.15.6:0/2491764282 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0e80010db0 con 0x7f0e88102730 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.603+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/2491764282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e88111d30 con 0x7f0e88102730 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.603+0000 7f0e7e7fc700 1 -- 172.21.15.6:0/2491764282 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e8001ada0 con 0x7f0e88102730 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.603+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/2491764282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e881128a0 con 0x7f0e88102730 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.604+0000 7f0e7e7fc700 1 -- 172.21.15.6:0/2491764282 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f0e80010920 con 0x7f0e88102730 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.604+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/2491764282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e881121c0 con 0x7f0e88102730 2024-03-22T06:06:05.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.604+0000 7f0e7e7fc700 1 --2- 172.21.15.6:0/2491764282 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0e740367e0 0x7f0e74038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.604+0000 7f0e7e7fc700 1 -- 172.21.15.6:0/2491764282 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0e80049d00 con 0x7f0e88102730 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.604+0000 7f0e8d3f7700 1 --2- 172.21.15.6:0/2491764282 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0e740367e0 0x7f0e74038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.605+0000 7f0e8d3f7700 1 --2- 172.21.15.6:0/2491764282 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0e740367e0 0x7f0e74038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009990 tx=0x7f0e78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.609+0000 7f0e7e7fc700 1 -- 172.21.15.6:0/2491764282 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0e80023360 con 0x7f0e88102730 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.838+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/2491764282 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e880ff2d0 con 0x7f0e740367e0 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.846+0000 7f0e7e7fc700 1 -- 172.21.15.6:0/2491764282 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0e880ff2d0 con 0x7f0e740367e0 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.848+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/2491764282 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0e740367e0 msgr2=0x7f0e74038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.848+0000 7f0e8fe5c700 1 --2- 172.21.15.6:0/2491764282 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0e740367e0 0x7f0e74038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009990 tx=0x7f0e78006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.848+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/2491764282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 msgr2=0x7f0e881115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.848+0000 7f0e8fe5c700 1 --2- 172.21.15.6:0/2491764282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e881115f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e80006fd0 tx=0x7f0e80009250 comp rx=0 tx=0).stop 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.848+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/2491764282 shutdown_connections 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.848+0000 7f0e8fe5c700 1 --2- 172.21.15.6:0/2491764282 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0e740367e0 0x7f0e74038c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:05.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.848+0000 7f0e8fe5c700 1 --2- 172.21.15.6:0/2491764282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e88102730 0x7f0e881115f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:05.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.848+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/2491764282 >> 172.21.15.6:0/2491764282 conn(0x7f0e880fdf10 msgr2=0x7f0e880feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:05.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.848+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/2491764282 shutdown_connections 2024-03-22T06:06:05.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:04.848+0000 7f0e8fe5c700 1 -- 172.21.15.6:0/2491764282 wait complete. 2024-03-22T06:06:05.711 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying mgr service with default placement... 2024-03-22T06:06:05.975 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:05 smithi006 ceph-mon[29301]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:05.975 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:05 smithi006 ceph-mon[29301]: Saving service mon spec with placement count:5 2024-03-22T06:06:05.976 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:05 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:07.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-22T06:06:07.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.213+0000 7f6c245cd700 1 Processor -- start 2024-03-22T06:06:07.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.213+0000 7f6c245cd700 1 -- start start 2024-03-22T06:06:07.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.214+0000 7f6c245cd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c105300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:07.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.214+0000 7f6c245cd700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c105840 con 0x7f6c1c076bf0 2024-03-22T06:06:07.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.214+0000 7f6c22369700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c105300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:07.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.214+0000 7f6c22369700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c105300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57092/0 (socket says 172.21.15.6:57092) 2024-03-22T06:06:07.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.214+0000 7f6c22369700 1 -- 172.21.15.6:0/3777143681 learned_addr learned my addr 172.21.15.6:0/3777143681 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.214+0000 7f6c22369700 1 -- 172.21.15.6:0/3777143681 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c105980 con 0x7f6c1c076bf0 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.215+0000 7f6c22369700 1 --2- 172.21.15.6:0/3777143681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c105300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6c180098e0 tx=0x7f6c18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dced4bc54a616ab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.215+0000 7f6c21367700 1 -- 172.21.15.6:0/3777143681 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c1800b860 con 0x7f6c1c076bf0 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.215+0000 7f6c21367700 1 -- 172.21.15.6:0/3777143681 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6c1800be50 con 0x7f6c1c076bf0 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.215+0000 7f6c21367700 1 -- 172.21.15.6:0/3777143681 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c18010380 con 0x7f6c1c076bf0 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.215+0000 7f6c245cd700 1 -- 172.21.15.6:0/3777143681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 msgr2=0x7f6c1c105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.215+0000 7f6c245cd700 1 --2- 172.21.15.6:0/3777143681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c105300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6c180098e0 tx=0x7f6c18009250 comp rx=0 tx=0).stop 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.215+0000 7f6c245cd700 1 -- 172.21.15.6:0/3777143681 shutdown_connections 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.215+0000 7f6c245cd700 1 --2- 172.21.15.6:0/3777143681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c105300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.215+0000 7f6c245cd700 1 -- 172.21.15.6:0/3777143681 >> 172.21.15.6:0/3777143681 conn(0x7f6c1c0fcd50 msgr2=0x7f6c1c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.216+0000 7f6c245cd700 1 -- 172.21.15.6:0/3777143681 shutdown_connections 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.216+0000 7f6c245cd700 1 -- 172.21.15.6:0/3777143681 wait complete. 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.216+0000 7f6c245cd700 1 Processor -- start 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.216+0000 7f6c245cd700 1 -- start start 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.216+0000 7f6c245cd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:07.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.216+0000 7f6c245cd700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c103190 con 0x7f6c1c076bf0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.216+0000 7f6c22369700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.216+0000 7f6c22369700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57102/0 (socket says 172.21.15.6:57102) 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.216+0000 7f6c22369700 1 -- 172.21.15.6:0/3752255799 learned_addr learned my addr 172.21.15.6:0/3752255799 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.217+0000 7f6c22369700 1 -- 172.21.15.6:0/3752255799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c180094f0 con 0x7f6c1c076bf0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.217+0000 7f6c22369700 1 --2- 172.21.15.6:0/3752255799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c102c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6c18006b00 tx=0x7f6c18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.217+0000 7f6c0effd700 1 -- 172.21.15.6:0/3752255799 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c180107c0 con 0x7f6c1c076bf0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.217+0000 7f6c0effd700 1 -- 172.21.15.6:0/3752255799 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6c18010db0 con 0x7f6c1c076bf0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.217+0000 7f6c0effd700 1 -- 172.21.15.6:0/3752255799 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c1801acf0 con 0x7f6c1c076bf0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.217+0000 7f6c245cd700 1 -- 172.21.15.6:0/3752255799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c101200 con 0x7f6c1c076bf0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.217+0000 7f6c245cd700 1 -- 172.21.15.6:0/3752255799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c1016f0 con 0x7f6c1c076bf0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.218+0000 7f6c0effd700 1 -- 172.21.15.6:0/3752255799 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f6c180193c0 con 0x7f6c1c076bf0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.218+0000 7f6c0effd700 1 --2- 172.21.15.6:0/3752255799 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f6c08036790 0x7f6c08038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.218+0000 7f6c0effd700 1 -- 172.21.15.6:0/3752255799 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6c1804a640 con 0x7f6c1c076bf0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.218+0000 7f6c245cd700 1 -- 172.21.15.6:0/3752255799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c04bed0 con 0x7f6c1c076bf0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.218+0000 7f6c21b68700 1 --2- 172.21.15.6:0/3752255799 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f6c08036790 0x7f6c08038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.218+0000 7f6c21b68700 1 --2- 172.21.15.6:0/3752255799 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f6c08036790 0x7f6c08038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009990 tx=0x7f6c10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.223+0000 7f6c0effd700 1 -- 172.21.15.6:0/3752255799 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6c180196c0 con 0x7f6c1c076bf0 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.431+0000 7f6c245cd700 1 -- 172.21.15.6:0/3752255799 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c1c101b80 con 0x7f6c08036790 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.435+0000 7f6c0effd700 1 -- 172.21.15.6:0/3752255799 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6c1c101b80 con 0x7f6c08036790 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.437+0000 7f6c245cd700 1 -- 172.21.15.6:0/3752255799 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f6c08036790 msgr2=0x7f6c08038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.437+0000 7f6c245cd700 1 --2- 172.21.15.6:0/3752255799 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f6c08036790 0x7f6c08038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009990 tx=0x7f6c10006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.437+0000 7f6c245cd700 1 -- 172.21.15.6:0/3752255799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 msgr2=0x7f6c1c102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.437+0000 7f6c245cd700 1 --2- 172.21.15.6:0/3752255799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c102c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6c18006b00 tx=0x7f6c18009250 comp rx=0 tx=0).stop 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.437+0000 7f6c245cd700 1 -- 172.21.15.6:0/3752255799 shutdown_connections 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.437+0000 7f6c245cd700 1 --2- 172.21.15.6:0/3752255799 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f6c08036790 0x7f6c08038c40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.437+0000 7f6c245cd700 1 --2- 172.21.15.6:0/3752255799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c1c076bf0 0x7f6c1c102c50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.437+0000 7f6c245cd700 1 -- 172.21.15.6:0/3752255799 >> 172.21.15.6:0/3752255799 conn(0x7f6c1c0fcd50 msgr2=0x7f6c1c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.437+0000 7f6c245cd700 1 -- 172.21.15.6:0/3752255799 shutdown_connections 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:06.438+0000 7f6c245cd700 1 -- 172.21.15.6:0/3752255799 wait complete. 2024-03-22T06:06:07.368 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying crash service with default placement... 2024-03-22T06:06:07.743 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:07 smithi006 ceph-mon[29301]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:07.743 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:07 smithi006 ceph-mon[29301]: Saving service mgr spec with placement count:2 2024-03-22T06:06:07.743 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:07 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:09.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.209+0000 7efe24cb4700 1 Processor -- start 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.209+0000 7efe24cb4700 1 -- start start 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.209+0000 7efe24cb4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe20105280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.209+0000 7efe24cb4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe201057c0 con 0x7efe20076c70 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.209+0000 7efe1e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe20105280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.209+0000 7efe1e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe20105280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57110/0 (socket says 172.21.15.6:57110) 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.209+0000 7efe1e59c700 1 -- 172.21.15.6:0/4021485675 learned_addr learned my addr 172.21.15.6:0/4021485675 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.210+0000 7efe1e59c700 1 -- 172.21.15.6:0/4021485675 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe20105900 con 0x7efe20076c70 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.210+0000 7efe1e59c700 1 --2- 172.21.15.6:0/4021485675 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe20105280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe100098e0 tx=0x7efe10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125c2b33f8c06d84 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.210+0000 7efe1d59a700 1 -- 172.21.15.6:0/4021485675 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe1000b860 con 0x7efe20076c70 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.210+0000 7efe1d59a700 1 -- 172.21.15.6:0/4021485675 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe1000be90 con 0x7efe20076c70 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.210+0000 7efe1d59a700 1 -- 172.21.15.6:0/4021485675 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe100103c0 con 0x7efe20076c70 2024-03-22T06:06:09.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.210+0000 7efe24cb4700 1 -- 172.21.15.6:0/4021485675 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 msgr2=0x7efe20105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.210+0000 7efe24cb4700 1 --2- 172.21.15.6:0/4021485675 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe20105280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe100098e0 tx=0x7efe10009250 comp rx=0 tx=0).stop 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.210+0000 7efe24cb4700 1 -- 172.21.15.6:0/4021485675 shutdown_connections 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.210+0000 7efe24cb4700 1 --2- 172.21.15.6:0/4021485675 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe20105280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.210+0000 7efe24cb4700 1 -- 172.21.15.6:0/4021485675 >> 172.21.15.6:0/4021485675 conn(0x7efe200fcd50 msgr2=0x7efe200ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.211+0000 7efe24cb4700 1 -- 172.21.15.6:0/4021485675 shutdown_connections 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.211+0000 7efe24cb4700 1 -- 172.21.15.6:0/4021485675 wait complete. 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.211+0000 7efe24cb4700 1 Processor -- start 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.211+0000 7efe24cb4700 1 -- start start 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.211+0000 7efe24cb4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe201115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.211+0000 7efe24cb4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe20111b10 con 0x7efe20076c70 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.211+0000 7efe1e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe201115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.211+0000 7efe1e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe201115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57126/0 (socket says 172.21.15.6:57126) 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.211+0000 7efe1e59c700 1 -- 172.21.15.6:0/1824130889 learned_addr learned my addr 172.21.15.6:0/1824130889 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.211+0000 7efe1e59c700 1 -- 172.21.15.6:0/1824130889 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100094f0 con 0x7efe20076c70 2024-03-22T06:06:09.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.212+0000 7efe1e59c700 1 --2- 172.21.15.6:0/1824130889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe201115d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe10006d40 tx=0x7efe10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.212+0000 7efe0effd700 1 -- 172.21.15.6:0/1824130889 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe10010840 con 0x7efe20076c70 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.212+0000 7efe0effd700 1 -- 172.21.15.6:0/1824130889 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe10010e70 con 0x7efe20076c70 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.212+0000 7efe24cb4700 1 -- 172.21.15.6:0/1824130889 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe20111d10 con 0x7efe20076c70 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.212+0000 7efe0effd700 1 -- 172.21.15.6:0/1824130889 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe10024ea0 con 0x7efe20076c70 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.212+0000 7efe24cb4700 1 -- 172.21.15.6:0/1824130889 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe20112880 con 0x7efe20076c70 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.213+0000 7efe0effd700 1 -- 172.21.15.6:0/1824130889 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7efe1001a4c0 con 0x7efe20076c70 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.213+0000 7efe0effd700 1 --2- 172.21.15.6:0/1824130889 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efe08036790 0x7efe08038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.213+0000 7efe0effd700 1 -- 172.21.15.6:0/1824130889 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7efe1004a700 con 0x7efe20076c70 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.213+0000 7efe24cb4700 1 -- 172.21.15.6:0/1824130889 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2005f5f0 con 0x7efe20076c70 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.213+0000 7efe1dd9b700 1 --2- 172.21.15.6:0/1824130889 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efe08036790 0x7efe08038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.213+0000 7efe1dd9b700 1 --2- 172.21.15.6:0/1824130889 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efe08036790 0x7efe08038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efe14009990 tx=0x7efe14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.218+0000 7efe0effd700 1 -- 172.21.15.6:0/1824130889 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7efe1002e440 con 0x7efe20076c70 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.438+0000 7efe24cb4700 1 -- 172.21.15.6:0/1824130889 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7efe200fe0a0 con 0x7efe08036790 2024-03-22T06:06:09.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.443+0000 7efe0effd700 1 -- 172.21.15.6:0/1824130889 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7efe200fe0a0 con 0x7efe08036790 2024-03-22T06:06:09.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.445+0000 7efe24cb4700 1 -- 172.21.15.6:0/1824130889 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efe08036790 msgr2=0x7efe08038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:09.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.445+0000 7efe24cb4700 1 --2- 172.21.15.6:0/1824130889 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efe08036790 0x7efe08038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efe14009990 tx=0x7efe14006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:09.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.445+0000 7efe24cb4700 1 -- 172.21.15.6:0/1824130889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 msgr2=0x7efe201115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:09.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.445+0000 7efe24cb4700 1 --2- 172.21.15.6:0/1824130889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe201115d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe10006d40 tx=0x7efe10009250 comp rx=0 tx=0).stop 2024-03-22T06:06:09.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.445+0000 7efe24cb4700 1 -- 172.21.15.6:0/1824130889 shutdown_connections 2024-03-22T06:06:09.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.445+0000 7efe24cb4700 1 --2- 172.21.15.6:0/1824130889 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7efe08036790 0x7efe08038c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:09.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.445+0000 7efe24cb4700 1 --2- 172.21.15.6:0/1824130889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe20076c70 0x7efe201115d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:09.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.445+0000 7efe24cb4700 1 -- 172.21.15.6:0/1824130889 >> 172.21.15.6:0/1824130889 conn(0x7efe200fcd50 msgr2=0x7efe200fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:09.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.446+0000 7efe24cb4700 1 -- 172.21.15.6:0/1824130889 shutdown_connections 2024-03-22T06:06:09.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:08.446+0000 7efe24cb4700 1 -- 172.21.15.6:0/1824130889 wait complete. 2024-03-22T06:06:09.029 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying prometheus service with default placement... 2024-03-22T06:06:09.134 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:08 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:09.135 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:08 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:09.135 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:08 smithi006 ceph-mon[29301]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:09.135 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:08 smithi006 ceph-mon[29301]: Saving service crash spec with placement * 2024-03-22T06:06:09.135 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:08 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:10.517 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.459+0000 7f2f43822700 1 Processor -- start 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.459+0000 7f2f43822700 1 -- start start 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.459+0000 7f2f43822700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.459+0000 7f2f43822700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c103070 con 0x7f2f3c102720 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.459+0000 7f2f415be700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.459+0000 7f2f415be700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57134/0 (socket says 172.21.15.6:57134) 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.459+0000 7f2f415be700 1 -- 172.21.15.6:0/1918375358 learned_addr learned my addr 172.21.15.6:0/1918375358 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.459+0000 7f2f415be700 1 -- 172.21.15.6:0/1918375358 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c1031b0 con 0x7f2f3c102720 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.459+0000 7f2f415be700 1 --2- 172.21.15.6:0/1918375358 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c102b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009e50 tx=0x7f2f2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315dfdaf5dfcd92b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.460+0000 7f2f3bfff700 1 -- 172.21.15.6:0/1918375358 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f2c0092e0 con 0x7f2f3c102720 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.460+0000 7f2f3bfff700 1 -- 172.21.15.6:0/1918375358 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f2c012430 con 0x7f2f3c102720 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.460+0000 7f2f3bfff700 1 -- 172.21.15.6:0/1918375358 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f2c0075a0 con 0x7f2f3c102720 2024-03-22T06:06:10.518 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.460+0000 7f2f43822700 1 -- 172.21.15.6:0/1918375358 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 msgr2=0x7f2f3c102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.460+0000 7f2f43822700 1 --2- 172.21.15.6:0/1918375358 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c102b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009e50 tx=0x7f2f2c009840 comp rx=0 tx=0).stop 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.460+0000 7f2f43822700 1 -- 172.21.15.6:0/1918375358 shutdown_connections 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.460+0000 7f2f43822700 1 --2- 172.21.15.6:0/1918375358 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c102b30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.460+0000 7f2f43822700 1 -- 172.21.15.6:0/1918375358 >> 172.21.15.6:0/1918375358 conn(0x7f2f3c0fdee0 msgr2=0x7f2f3c1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.460+0000 7f2f43822700 1 -- 172.21.15.6:0/1918375358 shutdown_connections 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.460+0000 7f2f43822700 1 -- 172.21.15.6:0/1918375358 wait complete. 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.461+0000 7f2f43822700 1 Processor -- start 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.461+0000 7f2f43822700 1 -- start start 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.461+0000 7f2f43822700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.461+0000 7f2f43822700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c079110 con 0x7f2f3c102720 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.461+0000 7f2f415be700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.461+0000 7f2f415be700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57138/0 (socket says 172.21.15.6:57138) 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.461+0000 7f2f415be700 1 -- 172.21.15.6:0/1920124629 learned_addr learned my addr 172.21.15.6:0/1920124629 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.461+0000 7f2f415be700 1 -- 172.21.15.6:0/1920124629 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c009ae0 con 0x7f2f3c102720 2024-03-22T06:06:10.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.462+0000 7f2f415be700 1 --2- 172.21.15.6:0/1920124629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c078bd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0127d0 tx=0x7f2f2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.462+0000 7f2f39ffb700 1 -- 172.21.15.6:0/1920124629 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f2c0092e0 con 0x7f2f3c102720 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.462+0000 7f2f39ffb700 1 -- 172.21.15.6:0/1920124629 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f2c007d30 con 0x7f2f3c102720 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.462+0000 7f2f43822700 1 -- 172.21.15.6:0/1920124629 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c077180 con 0x7f2f3c102720 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.462+0000 7f2f39ffb700 1 -- 172.21.15.6:0/1920124629 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f2c025960 con 0x7f2f3c102720 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.462+0000 7f2f43822700 1 -- 172.21.15.6:0/1920124629 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c077670 con 0x7f2f3c102720 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.463+0000 7f2f39ffb700 1 -- 172.21.15.6:0/1920124629 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f2f2c019050 con 0x7f2f3c102720 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.463+0000 7f2f39ffb700 1 --2- 172.21.15.6:0/1920124629 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f2f240367e0 0x7f2f24038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.463+0000 7f2f43822700 1 -- 172.21.15.6:0/1920124629 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c04bed0 con 0x7f2f3c102720 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.463+0000 7f2f39ffb700 1 -- 172.21.15.6:0/1920124629 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f2f2c04a150 con 0x7f2f3c102720 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.463+0000 7f2f40dbd700 1 --2- 172.21.15.6:0/1920124629 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f2f240367e0 0x7f2f24038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.464+0000 7f2f40dbd700 1 --2- 172.21.15.6:0/1920124629 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f2f240367e0 0x7f2f24038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009990 tx=0x7f2f30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.468+0000 7f2f39ffb700 1 -- 172.21.15.6:0/1920124629 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2f2c02d9a0 con 0x7f2f3c102720 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.674+0000 7f2f43822700 1 -- 172.21.15.6:0/1920124629 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f3c077d80 con 0x7f2f240367e0 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.678+0000 7f2f39ffb700 1 -- 172.21.15.6:0/1920124629 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f2f3c077d80 con 0x7f2f240367e0 2024-03-22T06:06:10.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.680+0000 7f2f43822700 1 -- 172.21.15.6:0/1920124629 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f2f240367e0 msgr2=0x7f2f24038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:10.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.680+0000 7f2f43822700 1 --2- 172.21.15.6:0/1920124629 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f2f240367e0 0x7f2f24038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009990 tx=0x7f2f30006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:10.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.680+0000 7f2f43822700 1 -- 172.21.15.6:0/1920124629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 msgr2=0x7f2f3c078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:10.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.680+0000 7f2f43822700 1 --2- 172.21.15.6:0/1920124629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c078bd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0127d0 tx=0x7f2f2c009840 comp rx=0 tx=0).stop 2024-03-22T06:06:10.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.680+0000 7f2f43822700 1 -- 172.21.15.6:0/1920124629 shutdown_connections 2024-03-22T06:06:10.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.680+0000 7f2f43822700 1 --2- 172.21.15.6:0/1920124629 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f2f240367e0 0x7f2f24038c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:10.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.680+0000 7f2f43822700 1 --2- 172.21.15.6:0/1920124629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f3c102720 0x7f2f3c078bd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:10.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.680+0000 7f2f43822700 1 -- 172.21.15.6:0/1920124629 >> 172.21.15.6:0/1920124629 conn(0x7f2f3c0fdee0 msgr2=0x7f2f3c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:10.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.680+0000 7f2f43822700 1 -- 172.21.15.6:0/1920124629 shutdown_connections 2024-03-22T06:06:10.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:09.680+0000 7f2f43822700 1 -- 172.21.15.6:0/1920124629 wait complete. 2024-03-22T06:06:10.521 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying grafana service with default placement... 2024-03-22T06:06:10.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:10 smithi006 ceph-mon[29301]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:10.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:10 smithi006 ceph-mon[29301]: Saving service prometheus spec with placement count:1 2024-03-22T06:06:10.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:10 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:10.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:10 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:11.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-22T06:06:11.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.965+0000 7f424aa00700 1 Processor -- start 2024-03-22T06:06:11.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.965+0000 7f424aa00700 1 -- start start 2024-03-22T06:06:11.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.966+0000 7f424aa00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244105310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:11.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.966+0000 7f424aa00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4244105850 con 0x7f4244076bc0 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.966+0000 7f4243fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244105310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.966+0000 7f4243fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244105310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57142/0 (socket says 172.21.15.6:57142) 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.966+0000 7f4243fff700 1 -- 172.21.15.6:0/2950755727 learned_addr learned my addr 172.21.15.6:0/2950755727 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.966+0000 7f4243fff700 1 -- 172.21.15.6:0/2950755727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4244105990 con 0x7f4244076bc0 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.966+0000 7f4243fff700 1 --2- 172.21.15.6:0/2950755727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244105310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42340098e0 tx=0x7f4234009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ae31e4c5e2f62a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.966+0000 7f4242ffd700 1 -- 172.21.15.6:0/2950755727 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f423400b860 con 0x7f4244076bc0 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.966+0000 7f4242ffd700 1 -- 172.21.15.6:0/2950755727 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f423400be90 con 0x7f4244076bc0 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.967+0000 7f4242ffd700 1 -- 172.21.15.6:0/2950755727 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42340103c0 con 0x7f4244076bc0 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.967+0000 7f424aa00700 1 -- 172.21.15.6:0/2950755727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 msgr2=0x7f4244105310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.967+0000 7f424aa00700 1 --2- 172.21.15.6:0/2950755727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244105310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42340098e0 tx=0x7f4234009250 comp rx=0 tx=0).stop 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.967+0000 7f424aa00700 1 -- 172.21.15.6:0/2950755727 shutdown_connections 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.967+0000 7f424aa00700 1 --2- 172.21.15.6:0/2950755727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244105310 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.967+0000 7f424aa00700 1 -- 172.21.15.6:0/2950755727 >> 172.21.15.6:0/2950755727 conn(0x7f42440fcd80 msgr2=0x7f42440ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:11.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.967+0000 7f424aa00700 1 -- 172.21.15.6:0/2950755727 shutdown_connections 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.967+0000 7f424aa00700 1 -- 172.21.15.6:0/2950755727 wait complete. 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.968+0000 7f424aa00700 1 Processor -- start 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.968+0000 7f424aa00700 1 -- start start 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.968+0000 7f424aa00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244111560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.968+0000 7f424aa00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4244111aa0 con 0x7f4244076bc0 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.968+0000 7f4243fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244111560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.968+0000 7f4243fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244111560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57146/0 (socket says 172.21.15.6:57146) 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.968+0000 7f4243fff700 1 -- 172.21.15.6:0/1040659142 learned_addr learned my addr 172.21.15.6:0/1040659142 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.969+0000 7f4243fff700 1 -- 172.21.15.6:0/1040659142 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42340094f0 con 0x7f4244076bc0 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.969+0000 7f4243fff700 1 --2- 172.21.15.6:0/1040659142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244111560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4234006d40 tx=0x7f4234009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.969+0000 7f4240ff9700 1 -- 172.21.15.6:0/1040659142 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4234010840 con 0x7f4244076bc0 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.969+0000 7f4240ff9700 1 -- 172.21.15.6:0/1040659142 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4234010e70 con 0x7f4244076bc0 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.969+0000 7f424aa00700 1 -- 172.21.15.6:0/1040659142 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4244111d00 con 0x7f4244076bc0 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.969+0000 7f424aa00700 1 -- 172.21.15.6:0/1040659142 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4244112810 con 0x7f4244076bc0 2024-03-22T06:06:11.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.970+0000 7f4240ff9700 1 -- 172.21.15.6:0/1040659142 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4234024ea0 con 0x7f4244076bc0 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.970+0000 7f4240ff9700 1 -- 172.21.15.6:0/1040659142 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f423401a500 con 0x7f4244076bc0 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.970+0000 7f424aa00700 1 -- 172.21.15.6:0/1040659142 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424405f5f0 con 0x7f4244076bc0 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.970+0000 7f4240ff9700 1 --2- 172.21.15.6:0/1040659142 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4230036830 0x7f4230038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.970+0000 7f4240ff9700 1 -- 172.21.15.6:0/1040659142 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f423404b890 con 0x7f4244076bc0 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.970+0000 7f42437fe700 1 --2- 172.21.15.6:0/1040659142 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4230036830 0x7f4230038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.971+0000 7f42437fe700 1 --2- 172.21.15.6:0/1040659142 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4230036830 0x7f4230038ce0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f422c009990 tx=0x7f422c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:10.975+0000 7f4240ff9700 1 -- 172.21.15.6:0/1040659142 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4234018940 con 0x7f4244076bc0 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.182+0000 7f424aa00700 1 -- 172.21.15.6:0/1040659142 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f42440fe0b0 con 0x7f4230036830 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.188+0000 7f4240ff9700 1 -- 172.21.15.6:0/1040659142 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f42440fe0b0 con 0x7f4230036830 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.190+0000 7f424aa00700 1 -- 172.21.15.6:0/1040659142 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4230036830 msgr2=0x7f4230038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.190+0000 7f424aa00700 1 --2- 172.21.15.6:0/1040659142 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4230036830 0x7f4230038ce0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f422c009990 tx=0x7f422c006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.190+0000 7f424aa00700 1 -- 172.21.15.6:0/1040659142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 msgr2=0x7f4244111560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:11.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.190+0000 7f424aa00700 1 --2- 172.21.15.6:0/1040659142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244111560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4234006d40 tx=0x7f4234009250 comp rx=0 tx=0).stop 2024-03-22T06:06:11.795 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.190+0000 7f424aa00700 1 -- 172.21.15.6:0/1040659142 shutdown_connections 2024-03-22T06:06:11.795 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.190+0000 7f424aa00700 1 --2- 172.21.15.6:0/1040659142 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f4230036830 0x7f4230038ce0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:11.795 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.190+0000 7f424aa00700 1 --2- 172.21.15.6:0/1040659142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244076bc0 0x7f4244111560 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:11.795 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.190+0000 7f424aa00700 1 -- 172.21.15.6:0/1040659142 >> 172.21.15.6:0/1040659142 conn(0x7f42440fcd80 msgr2=0x7f42440fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:11.795 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.190+0000 7f424aa00700 1 -- 172.21.15.6:0/1040659142 shutdown_connections 2024-03-22T06:06:11.795 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:11.190+0000 7f424aa00700 1 -- 172.21.15.6:0/1040659142 wait complete. 2024-03-22T06:06:11.795 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying node-exporter service with default placement... 2024-03-22T06:06:12.396 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:12 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:12.396 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:12 smithi006 ceph-mon[29301]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:12.396 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:12 smithi006 ceph-mon[29301]: Saving service grafana spec with placement count:1 2024-03-22T06:06:12.396 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:12 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.226+0000 7fec6da19700 1 Processor -- start 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.226+0000 7fec6da19700 1 -- start start 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.226+0000 7fec6da19700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.226+0000 7fec6da19700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec68103080 con 0x7fec68102730 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.226+0000 7fec66ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.226+0000 7fec66ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57158/0 (socket says 172.21.15.6:57158) 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.226+0000 7fec66ffd700 1 -- 172.21.15.6:0/452364156 learned_addr learned my addr 172.21.15.6:0/452364156 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.227+0000 7fec66ffd700 1 -- 172.21.15.6:0/452364156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec681031c0 con 0x7fec68102730 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.227+0000 7fec66ffd700 1 --2- 172.21.15.6:0/452364156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68102b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec580098e0 tx=0x7fec58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ad05e6c20c056f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.227+0000 7fec65ffb700 1 -- 172.21.15.6:0/452364156 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec5800b860 con 0x7fec68102730 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.227+0000 7fec65ffb700 1 -- 172.21.15.6:0/452364156 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec5800be90 con 0x7fec68102730 2024-03-22T06:06:13.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.227+0000 7fec65ffb700 1 -- 172.21.15.6:0/452364156 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec580103c0 con 0x7fec68102730 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.228+0000 7fec6da19700 1 -- 172.21.15.6:0/452364156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 msgr2=0x7fec68102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.228+0000 7fec6da19700 1 --2- 172.21.15.6:0/452364156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68102b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec580098e0 tx=0x7fec58009250 comp rx=0 tx=0).stop 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.228+0000 7fec6da19700 1 -- 172.21.15.6:0/452364156 shutdown_connections 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.228+0000 7fec6da19700 1 --2- 172.21.15.6:0/452364156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68102b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.228+0000 7fec6da19700 1 -- 172.21.15.6:0/452364156 >> 172.21.15.6:0/452364156 conn(0x7fec680fdf10 msgr2=0x7fec68100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.228+0000 7fec6da19700 1 -- 172.21.15.6:0/452364156 shutdown_connections 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.228+0000 7fec6da19700 1 -- 172.21.15.6:0/452364156 wait complete. 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.228+0000 7fec6da19700 1 Processor -- start 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.228+0000 7fec6da19700 1 -- start start 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.228+0000 7fec6da19700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68111630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.228+0000 7fec6da19700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec68111b70 con 0x7fec68102730 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.229+0000 7fec66ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68111630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.229+0000 7fec66ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68111630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57162/0 (socket says 172.21.15.6:57162) 2024-03-22T06:06:13.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.229+0000 7fec66ffd700 1 -- 172.21.15.6:0/2861851890 learned_addr learned my addr 172.21.15.6:0/2861851890 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.229+0000 7fec66ffd700 1 -- 172.21.15.6:0/2861851890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec580094f0 con 0x7fec68102730 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.229+0000 7fec66ffd700 1 --2- 172.21.15.6:0/2861851890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68111630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fec58006fd0 tx=0x7fec58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.229+0000 7fec4f7fe700 1 -- 172.21.15.6:0/2861851890 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec58010630 con 0x7fec68102730 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.229+0000 7fec4f7fe700 1 -- 172.21.15.6:0/2861851890 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec58010c60 con 0x7fec68102730 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.229+0000 7fec6da19700 1 -- 172.21.15.6:0/2861851890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec68111d70 con 0x7fec68102730 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.229+0000 7fec4f7fe700 1 -- 172.21.15.6:0/2861851890 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec5801ad40 con 0x7fec68102730 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.229+0000 7fec6da19700 1 -- 172.21.15.6:0/2861851890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec681128e0 con 0x7fec68102730 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.230+0000 7fec4f7fe700 1 -- 172.21.15.6:0/2861851890 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fec580193c0 con 0x7fec68102730 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.230+0000 7fec4f7fe700 1 --2- 172.21.15.6:0/2861851890 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fec500367e0 0x7fec50038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.230+0000 7fec6da19700 1 -- 172.21.15.6:0/2861851890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6805f5f0 con 0x7fec68102730 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.230+0000 7fec4f7fe700 1 -- 172.21.15.6:0/2861851890 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fec5804a1c0 con 0x7fec68102730 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.230+0000 7fec667fc700 1 --2- 172.21.15.6:0/2861851890 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fec500367e0 0x7fec50038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.231+0000 7fec667fc700 1 --2- 172.21.15.6:0/2861851890 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fec500367e0 0x7fec50038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009990 tx=0x7fec5c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.235+0000 7fec4f7fe700 1 -- 172.21.15.6:0/2861851890 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fec5802c300 con 0x7fec68102730 2024-03-22T06:06:13.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.454+0000 7fec6da19700 1 -- 172.21.15.6:0/2861851890 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fec680ff2d0 con 0x7fec500367e0 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.458+0000 7fec4f7fe700 1 -- 172.21.15.6:0/2861851890 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fec680ff2d0 con 0x7fec500367e0 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.460+0000 7fec6da19700 1 -- 172.21.15.6:0/2861851890 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fec500367e0 msgr2=0x7fec50038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.460+0000 7fec6da19700 1 --2- 172.21.15.6:0/2861851890 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fec500367e0 0x7fec50038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009990 tx=0x7fec5c006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.460+0000 7fec6da19700 1 -- 172.21.15.6:0/2861851890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 msgr2=0x7fec68111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.460+0000 7fec6da19700 1 --2- 172.21.15.6:0/2861851890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68111630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fec58006fd0 tx=0x7fec58009250 comp rx=0 tx=0).stop 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.460+0000 7fec6da19700 1 -- 172.21.15.6:0/2861851890 shutdown_connections 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.460+0000 7fec6da19700 1 --2- 172.21.15.6:0/2861851890 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fec500367e0 0x7fec50038c90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.460+0000 7fec6da19700 1 --2- 172.21.15.6:0/2861851890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec68102730 0x7fec68111630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.460+0000 7fec6da19700 1 -- 172.21.15.6:0/2861851890 >> 172.21.15.6:0/2861851890 conn(0x7fec680fdf10 msgr2=0x7fec680feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.460+0000 7fec6da19700 1 -- 172.21.15.6:0/2861851890 shutdown_connections 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:12.460+0000 7fec6da19700 1 -- 172.21.15.6:0/2861851890 wait complete. 2024-03-22T06:06:13.088 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying alertmanager service with default placement... 2024-03-22T06:06:13.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:13 smithi006 ceph-mon[29301]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:13.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:13 smithi006 ceph-mon[29301]: Saving service node-exporter spec with placement * 2024-03-22T06:06:13.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:13 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:15.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-22T06:06:15.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.292+0000 7f9d0c218700 1 Processor -- start 2024-03-22T06:06:15.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.292+0000 7f9d0c218700 1 -- start start 2024-03-22T06:06:15.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.292+0000 7f9d0c218700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d04104130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:15.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.292+0000 7f9d0c218700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04104670 con 0x7f9d04103d20 2024-03-22T06:06:15.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.293+0000 7f9d09fb4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d04104130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:15.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.293+0000 7f9d09fb4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d04104130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37286/0 (socket says 172.21.15.6:37286) 2024-03-22T06:06:15.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.293+0000 7f9d09fb4700 1 -- 172.21.15.6:0/322814976 learned_addr learned my addr 172.21.15.6:0/322814976 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:15.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.293+0000 7f9d09fb4700 1 -- 172.21.15.6:0/322814976 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d041047b0 con 0x7f9d04103d20 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.293+0000 7f9d09fb4700 1 --2- 172.21.15.6:0/322814976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d04104130 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009e50 tx=0x7f9cfc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa49fc6f15c7549 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.293+0000 7f9d08fb2700 1 -- 172.21.15.6:0/322814976 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cfc0092e0 con 0x7f9d04103d20 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.293+0000 7f9d08fb2700 1 -- 172.21.15.6:0/322814976 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cfc012430 con 0x7f9d04103d20 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.293+0000 7f9d08fb2700 1 -- 172.21.15.6:0/322814976 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cfc0075a0 con 0x7f9d04103d20 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.294+0000 7f9d0c218700 1 -- 172.21.15.6:0/322814976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 msgr2=0x7f9d04104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.294+0000 7f9d0c218700 1 --2- 172.21.15.6:0/322814976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d04104130 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009e50 tx=0x7f9cfc009840 comp rx=0 tx=0).stop 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.294+0000 7f9d0c218700 1 -- 172.21.15.6:0/322814976 shutdown_connections 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.294+0000 7f9d0c218700 1 --2- 172.21.15.6:0/322814976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d04104130 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.294+0000 7f9d0c218700 1 -- 172.21.15.6:0/322814976 >> 172.21.15.6:0/322814976 conn(0x7f9d040fc500 msgr2=0x7f9d040fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.294+0000 7f9d0c218700 1 -- 172.21.15.6:0/322814976 shutdown_connections 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.294+0000 7f9d0c218700 1 -- 172.21.15.6:0/322814976 wait complete. 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.295+0000 7f9d0c218700 1 Processor -- start 2024-03-22T06:06:15.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.295+0000 7f9d0c218700 1 -- start start 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.295+0000 7f9d0c218700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d041115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.295+0000 7f9d0c218700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04111b10 con 0x7f9d04103d20 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.295+0000 7f9d09fb4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d041115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.295+0000 7f9d09fb4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d041115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37288/0 (socket says 172.21.15.6:37288) 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.295+0000 7f9d09fb4700 1 -- 172.21.15.6:0/2564646525 learned_addr learned my addr 172.21.15.6:0/2564646525 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.295+0000 7f9d09fb4700 1 -- 172.21.15.6:0/2564646525 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc009ae0 con 0x7f9d04103d20 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.295+0000 7f9d09fb4700 1 --2- 172.21.15.6:0/2564646525 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d041115d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0127d0 tx=0x7f9cfc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.296+0000 7f9d027fc700 1 -- 172.21.15.6:0/2564646525 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cfc0092e0 con 0x7f9d04103d20 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.296+0000 7f9d0c218700 1 -- 172.21.15.6:0/2564646525 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d04111d10 con 0x7f9d04103d20 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.296+0000 7f9d027fc700 1 -- 172.21.15.6:0/2564646525 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cfc007d30 con 0x7f9d04103d20 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.296+0000 7f9d027fc700 1 -- 172.21.15.6:0/2564646525 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cfc025960 con 0x7f9d04103d20 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.296+0000 7f9d0c218700 1 -- 172.21.15.6:0/2564646525 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d04112880 con 0x7f9d04103d20 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.297+0000 7f9d027fc700 1 -- 172.21.15.6:0/2564646525 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f9cfc019050 con 0x7f9d04103d20 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.297+0000 7f9d0c218700 1 -- 172.21.15.6:0/2564646525 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0405f5f0 con 0x7f9d04103d20 2024-03-22T06:06:15.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.297+0000 7f9d027fc700 1 --2- 172.21.15.6:0/2564646525 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f9cec0367e0 0x7f9cec038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.297+0000 7f9d027fc700 1 -- 172.21.15.6:0/2564646525 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9cfc04a140 con 0x7f9d04103d20 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.297+0000 7f9d097b3700 1 --2- 172.21.15.6:0/2564646525 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f9cec0367e0 0x7f9cec038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.298+0000 7f9d097b3700 1 --2- 172.21.15.6:0/2564646525 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f9cec0367e0 0x7f9cec038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009990 tx=0x7f9cf8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.302+0000 7f9d027fc700 1 -- 172.21.15.6:0/2564646525 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9cfc02d9a0 con 0x7f9d04103d20 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.506+0000 7f9d0c218700 1 -- 172.21.15.6:0/2564646525 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d040fd7d0 con 0x7f9cec0367e0 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.513+0000 7f9d027fc700 1 -- 172.21.15.6:0/2564646525 <== mgr.14120 v2:172.21.15.6:6800/3903206979 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f9d040fd7d0 con 0x7f9cec0367e0 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.515+0000 7f9d0c218700 1 -- 172.21.15.6:0/2564646525 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f9cec0367e0 msgr2=0x7f9cec038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.515+0000 7f9d0c218700 1 --2- 172.21.15.6:0/2564646525 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f9cec0367e0 0x7f9cec038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009990 tx=0x7f9cf8006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.515+0000 7f9d0c218700 1 -- 172.21.15.6:0/2564646525 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 msgr2=0x7f9d041115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.515+0000 7f9d0c218700 1 --2- 172.21.15.6:0/2564646525 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d041115d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0127d0 tx=0x7f9cfc009840 comp rx=0 tx=0).stop 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.515+0000 7f9d0c218700 1 -- 172.21.15.6:0/2564646525 shutdown_connections 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.515+0000 7f9d0c218700 1 --2- 172.21.15.6:0/2564646525 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f9cec0367e0 0x7f9cec038c90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.515+0000 7f9d0c218700 1 --2- 172.21.15.6:0/2564646525 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d04103d20 0x7f9d041115d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:15.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.515+0000 7f9d0c218700 1 -- 172.21.15.6:0/2564646525 >> 172.21.15.6:0/2564646525 conn(0x7f9d040fc500 msgr2=0x7f9d040fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:15.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.515+0000 7f9d0c218700 1 -- 172.21.15.6:0/2564646525 shutdown_connections 2024-03-22T06:06:15.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:14.515+0000 7f9d0c218700 1 -- 172.21.15.6:0/2564646525 wait complete. 2024-03-22T06:06:15.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:15 smithi006 ceph-mon[29301]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:06:15.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:15 smithi006 ceph-mon[29301]: from='mgr.14120 172.21.15.6:0/2261480220' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.432+0000 7eff51ef1700 1 Processor -- start 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.432+0000 7eff51ef1700 1 -- start start 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.432+0000 7eff51ef1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c103080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.432+0000 7eff51ef1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c1035c0 con 0x7eff4c102c70 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.432+0000 7eff4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.432+0000 7eff4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37300/0 (socket says 172.21.15.6:37300) 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.432+0000 7eff4b7fe700 1 -- 172.21.15.6:0/1863364843 learned_addr learned my addr 172.21.15.6:0/1863364843 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.432+0000 7eff4b7fe700 1 -- 172.21.15.6:0/1863364843 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c103700 con 0x7eff4c102c70 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.432+0000 7eff4b7fe700 1 --2- 172.21.15.6:0/1863364843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c103080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7eff340098e0 tx=0x7eff34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8f16716fd951b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.433+0000 7eff4a7fc700 1 -- 172.21.15.6:0/1863364843 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff3400b860 con 0x7eff4c102c70 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.433+0000 7eff4a7fc700 1 -- 172.21.15.6:0/1863364843 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff3400be90 con 0x7eff4c102c70 2024-03-22T06:06:16.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.433+0000 7eff4a7fc700 1 -- 172.21.15.6:0/1863364843 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff340103c0 con 0x7eff4c102c70 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.433+0000 7eff51ef1700 1 -- 172.21.15.6:0/1863364843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 msgr2=0x7eff4c103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.433+0000 7eff51ef1700 1 --2- 172.21.15.6:0/1863364843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c103080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7eff340098e0 tx=0x7eff34009250 comp rx=0 tx=0).stop 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.433+0000 7eff51ef1700 1 -- 172.21.15.6:0/1863364843 shutdown_connections 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.433+0000 7eff51ef1700 1 --2- 172.21.15.6:0/1863364843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c103080 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.433+0000 7eff51ef1700 1 -- 172.21.15.6:0/1863364843 >> 172.21.15.6:0/1863364843 conn(0x7eff4c0fc4a0 msgr2=0x7eff4c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.433+0000 7eff51ef1700 1 -- 172.21.15.6:0/1863364843 shutdown_connections 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.433+0000 7eff51ef1700 1 -- 172.21.15.6:0/1863364843 wait complete. 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.434+0000 7eff51ef1700 1 Processor -- start 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.434+0000 7eff51ef1700 1 -- start start 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.434+0000 7eff51ef1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.434+0000 7eff51ef1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c113cd0 con 0x7eff4c102c70 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.434+0000 7eff4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.434+0000 7eff4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c113790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37308/0 (socket says 172.21.15.6:37308) 2024-03-22T06:06:16.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.434+0000 7eff4b7fe700 1 -- 172.21.15.6:0/2781198653 learned_addr learned my addr 172.21.15.6:0/2781198653 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.434+0000 7eff4b7fe700 1 -- 172.21.15.6:0/2781198653 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff340094f0 con 0x7eff4c102c70 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.435+0000 7eff4b7fe700 1 --2- 172.21.15.6:0/2781198653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c113790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7eff340090b0 tx=0x7eff34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.435+0000 7eff43fff700 1 -- 172.21.15.6:0/2781198653 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff34010840 con 0x7eff4c102c70 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.435+0000 7eff51ef1700 1 -- 172.21.15.6:0/2781198653 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4c113ed0 con 0x7eff4c102c70 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.435+0000 7eff43fff700 1 -- 172.21.15.6:0/2781198653 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff34010e70 con 0x7eff4c102c70 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.435+0000 7eff43fff700 1 -- 172.21.15.6:0/2781198653 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff34024e40 con 0x7eff4c102c70 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.435+0000 7eff51ef1700 1 -- 172.21.15.6:0/2781198653 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff4c114a40 con 0x7eff4c102c70 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.436+0000 7eff43fff700 1 -- 172.21.15.6:0/2781198653 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7eff3401a510 con 0x7eff4c102c70 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.436+0000 7eff51ef1700 1 -- 172.21.15.6:0/2781198653 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4c05f5f0 con 0x7eff4c102c70 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.436+0000 7eff43fff700 1 --2- 172.21.15.6:0/2781198653 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff38036740 0x7eff38038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.436+0000 7eff43fff700 1 -- 172.21.15.6:0/2781198653 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7eff3404aa20 con 0x7eff4c102c70 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.436+0000 7eff4affd700 1 --2- 172.21.15.6:0/2781198653 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff38036740 0x7eff38038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:16.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.436+0000 7eff4affd700 1 --2- 172.21.15.6:0/2781198653 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff38036740 0x7eff38038bf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009990 tx=0x7eff3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.440+0000 7eff43fff700 1 -- 172.21.15.6:0/2781198653 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7eff3402e440 con 0x7eff4c102c70 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.633+0000 7eff51ef1700 1 -- 172.21.15.6:0/2781198653 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7eff4c04bed0 con 0x7eff4c102c70 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.636+0000 7eff43fff700 1 -- 172.21.15.6:0/2781198653 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7eff3402d3a0 con 0x7eff4c102c70 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.638+0000 7eff51ef1700 1 -- 172.21.15.6:0/2781198653 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff38036740 msgr2=0x7eff38038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.638+0000 7eff51ef1700 1 --2- 172.21.15.6:0/2781198653 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff38036740 0x7eff38038bf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009990 tx=0x7eff3c006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.638+0000 7eff51ef1700 1 -- 172.21.15.6:0/2781198653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 msgr2=0x7eff4c113790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.638+0000 7eff51ef1700 1 --2- 172.21.15.6:0/2781198653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c113790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7eff340090b0 tx=0x7eff34009250 comp rx=0 tx=0).stop 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.638+0000 7eff51ef1700 1 -- 172.21.15.6:0/2781198653 shutdown_connections 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.638+0000 7eff51ef1700 1 --2- 172.21.15.6:0/2781198653 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7eff38036740 0x7eff38038bf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.638+0000 7eff51ef1700 1 --2- 172.21.15.6:0/2781198653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4c102c70 0x7eff4c113790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.638+0000 7eff51ef1700 1 -- 172.21.15.6:0/2781198653 >> 172.21.15.6:0/2781198653 conn(0x7eff4c0fc4a0 msgr2=0x7eff4c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.638+0000 7eff51ef1700 1 -- 172.21.15.6:0/2781198653 shutdown_connections 2024-03-22T06:06:16.223 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:15.638+0000 7eff51ef1700 1 -- 172.21.15.6:0/2781198653 wait complete. 2024-03-22T06:06:16.336 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:16 smithi006 ceph-mon[29301]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:16.336 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:16 smithi006 ceph-mon[29301]: Saving service alertmanager spec with placement count:1 2024-03-22T06:06:16.336 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:16 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2781198653' entity='client.admin' 2024-03-22T06:06:17.411 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:17 smithi006 ceph-mon[29301]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:06:17.411 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:17 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1073815730' entity='client.admin' 2024-03-22T06:06:17.419 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.615+0000 7fd2dbafc700 1 Processor -- start 2024-03-22T06:06:17.419 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.616+0000 7fd2dbafc700 1 -- start start 2024-03-22T06:06:17.419 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.616+0000 7fd2dbafc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4102b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:17.419 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.616+0000 7fd2dbafc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d4103090 con 0x7fd2d4102740 2024-03-22T06:06:17.419 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.616+0000 7fd2d9898700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:17.419 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.616+0000 7fd2d9898700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37322/0 (socket says 172.21.15.6:37322) 2024-03-22T06:06:17.419 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.616+0000 7fd2d9898700 1 -- 172.21.15.6:0/426789028 learned_addr learned my addr 172.21.15.6:0/426789028 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:17.419 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.616+0000 7fd2d9898700 1 -- 172.21.15.6:0/426789028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d41031d0 con 0x7fd2d4102740 2024-03-22T06:06:17.419 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.617+0000 7fd2d9898700 1 --2- 172.21.15.6:0/426789028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4102b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00098e0 tx=0x7fd2d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be0f4f0ec5fe7fa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.617+0000 7fd2d8896700 1 -- 172.21.15.6:0/426789028 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2d000b860 con 0x7fd2d4102740 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.617+0000 7fd2d8896700 1 -- 172.21.15.6:0/426789028 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2d000be90 con 0x7fd2d4102740 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.617+0000 7fd2d8896700 1 -- 172.21.15.6:0/426789028 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2d00103c0 con 0x7fd2d4102740 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.617+0000 7fd2dbafc700 1 -- 172.21.15.6:0/426789028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 msgr2=0x7fd2d4102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.617+0000 7fd2dbafc700 1 --2- 172.21.15.6:0/426789028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4102b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00098e0 tx=0x7fd2d0009250 comp rx=0 tx=0).stop 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.617+0000 7fd2dbafc700 1 -- 172.21.15.6:0/426789028 shutdown_connections 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.617+0000 7fd2dbafc700 1 --2- 172.21.15.6:0/426789028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4102b50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.617+0000 7fd2dbafc700 1 -- 172.21.15.6:0/426789028 >> 172.21.15.6:0/426789028 conn(0x7fd2d40fdea0 msgr2=0x7fd2d41002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.617+0000 7fd2dbafc700 1 -- 172.21.15.6:0/426789028 shutdown_connections 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.618+0000 7fd2dbafc700 1 -- 172.21.15.6:0/426789028 wait complete. 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.618+0000 7fd2dbafc700 1 Processor -- start 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.618+0000 7fd2dbafc700 1 -- start start 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.618+0000 7fd2dbafc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.618+0000 7fd2dbafc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d4079110 con 0x7fd2d4102740 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.618+0000 7fd2d9898700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.618+0000 7fd2d9898700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37324/0 (socket says 172.21.15.6:37324) 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.618+0000 7fd2d9898700 1 -- 172.21.15.6:0/1073815730 learned_addr learned my addr 172.21.15.6:0/1073815730 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.619+0000 7fd2d9898700 1 -- 172.21.15.6:0/1073815730 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d00094f0 con 0x7fd2d4102740 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.619+0000 7fd2d9898700 1 --2- 172.21.15.6:0/1073815730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4078bd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0006b00 tx=0x7fd2d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.619+0000 7fd2ca7fc700 1 -- 172.21.15.6:0/1073815730 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2d0010840 con 0x7fd2d4102740 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.619+0000 7fd2ca7fc700 1 -- 172.21.15.6:0/1073815730 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2d0010e70 con 0x7fd2d4102740 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.619+0000 7fd2dbafc700 1 -- 172.21.15.6:0/1073815730 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d4077180 con 0x7fd2d4102740 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.619+0000 7fd2ca7fc700 1 -- 172.21.15.6:0/1073815730 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2d001ae70 con 0x7fd2d4102740 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.619+0000 7fd2dbafc700 1 -- 172.21.15.6:0/1073815730 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d4077670 con 0x7fd2d4102740 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.620+0000 7fd2ca7fc700 1 -- 172.21.15.6:0/1073815730 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd2d00244d0 con 0x7fd2d4102740 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.620+0000 7fd2ca7fc700 1 --2- 172.21.15.6:0/1073815730 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fd2c0036790 0x7fd2c0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.620+0000 7fd2dbafc700 1 -- 172.21.15.6:0/1073815730 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d404bed0 con 0x7fd2d4102740 2024-03-22T06:06:17.421 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.620+0000 7fd2ca7fc700 1 -- 172.21.15.6:0/1073815730 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd2d004ab30 con 0x7fd2d4102740 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.620+0000 7fd2d9097700 1 --2- 172.21.15.6:0/1073815730 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fd2c0036790 0x7fd2c0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.620+0000 7fd2d9097700 1 --2- 172.21.15.6:0/1073815730 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fd2c0036790 0x7fd2c0038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009990 tx=0x7fd2c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.625+0000 7fd2ca7fc700 1 -- 172.21.15.6:0/1073815730 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd2d002e440 con 0x7fd2d4102740 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.817+0000 7fd2dbafc700 1 -- 172.21.15.6:0/1073815730 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fd2d4104530 con 0x7fd2d4102740 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.823+0000 7fd2ca7fc700 1 -- 172.21.15.6:0/1073815730 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fd2d002cb80 con 0x7fd2d4102740 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.825+0000 7fd2dbafc700 1 -- 172.21.15.6:0/1073815730 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fd2c0036790 msgr2=0x7fd2c0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.825+0000 7fd2dbafc700 1 --2- 172.21.15.6:0/1073815730 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fd2c0036790 0x7fd2c0038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009990 tx=0x7fd2c4006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.825+0000 7fd2dbafc700 1 -- 172.21.15.6:0/1073815730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 msgr2=0x7fd2d4078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.825+0000 7fd2dbafc700 1 --2- 172.21.15.6:0/1073815730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4078bd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0006b00 tx=0x7fd2d0009250 comp rx=0 tx=0).stop 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.825+0000 7fd2dbafc700 1 -- 172.21.15.6:0/1073815730 shutdown_connections 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.825+0000 7fd2dbafc700 1 --2- 172.21.15.6:0/1073815730 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7fd2c0036790 0x7fd2c0038c40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.825+0000 7fd2dbafc700 1 --2- 172.21.15.6:0/1073815730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2d4102740 0x7fd2d4078bd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:17.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.825+0000 7fd2dbafc700 1 -- 172.21.15.6:0/1073815730 >> 172.21.15.6:0/1073815730 conn(0x7fd2d40fdea0 msgr2=0x7fd2d40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:17.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.825+0000 7fd2dbafc700 1 -- 172.21.15.6:0/1073815730 shutdown_connections 2024-03-22T06:06:17.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:16.825+0000 7fd2dbafc700 1 -- 172.21.15.6:0/1073815730 wait complete. 2024-03-22T06:06:17.423 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling the dashboard module... 2024-03-22T06:06:18.322 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:18 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/184803814' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-22T06:06:18.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.788+0000 7f0d65533700 1 Processor -- start 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.788+0000 7f0d65533700 1 -- start start 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.788+0000 7f0d65533700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60077200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.788+0000 7f0d65533700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d600790b0 con 0x7f0d60078ca0 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.788+0000 7f0d5effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60077200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.788+0000 7f0d5effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60077200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37326/0 (socket says 172.21.15.6:37326) 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.788+0000 7f0d5effd700 1 -- 172.21.15.6:0/2730298392 learned_addr learned my addr 172.21.15.6:0/2730298392 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.788+0000 7f0d5effd700 1 -- 172.21.15.6:0/2730298392 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d60077740 con 0x7f0d60078ca0 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.788+0000 7f0d5effd700 1 --2- 172.21.15.6:0/2730298392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60077200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d500098e0 tx=0x7f0d50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7819787c40064ced server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.788+0000 7f0d5dffb700 1 -- 172.21.15.6:0/2730298392 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d5000b860 con 0x7f0d60078ca0 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.789+0000 7f0d5dffb700 1 -- 172.21.15.6:0/2730298392 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d5000be90 con 0x7f0d60078ca0 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.789+0000 7f0d5dffb700 1 -- 172.21.15.6:0/2730298392 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d500103c0 con 0x7f0d60078ca0 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.789+0000 7f0d65533700 1 -- 172.21.15.6:0/2730298392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 msgr2=0x7f0d60077200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:18.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.789+0000 7f0d65533700 1 --2- 172.21.15.6:0/2730298392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60077200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d500098e0 tx=0x7f0d50009250 comp rx=0 tx=0).stop 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.789+0000 7f0d65533700 1 -- 172.21.15.6:0/2730298392 shutdown_connections 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.789+0000 7f0d65533700 1 --2- 172.21.15.6:0/2730298392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60077200 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.789+0000 7f0d65533700 1 -- 172.21.15.6:0/2730298392 >> 172.21.15.6:0/2730298392 conn(0x7f0d600fdea0 msgr2=0x7f0d601002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.789+0000 7f0d65533700 1 -- 172.21.15.6:0/2730298392 shutdown_connections 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.789+0000 7f0d65533700 1 -- 172.21.15.6:0/2730298392 wait complete. 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.790+0000 7f0d65533700 1 Processor -- start 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.790+0000 7f0d65533700 1 -- start start 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.790+0000 7f0d65533700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.790+0000 7f0d65533700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d60113d80 con 0x7f0d60078ca0 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.790+0000 7f0d5effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.790+0000 7f0d5effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37328/0 (socket says 172.21.15.6:37328) 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.790+0000 7f0d5effd700 1 -- 172.21.15.6:0/184803814 learned_addr learned my addr 172.21.15.6:0/184803814 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.790+0000 7f0d5effd700 1 -- 172.21.15.6:0/184803814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500094f0 con 0x7f0d60078ca0 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.790+0000 7f0d5effd700 1 --2- 172.21.15.6:0/184803814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60113840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d500094c0 tx=0x7f0d50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.791+0000 7f0d477fe700 1 -- 172.21.15.6:0/184803814 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d500118f0 con 0x7f0d60078ca0 2024-03-22T06:06:18.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.791+0000 7f0d477fe700 1 -- 172.21.15.6:0/184803814 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d50010d60 con 0x7f0d60078ca0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.791+0000 7f0d65533700 1 -- 172.21.15.6:0/184803814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d60113f80 con 0x7f0d60078ca0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.791+0000 7f0d477fe700 1 -- 172.21.15.6:0/184803814 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d5001a430 con 0x7f0d60078ca0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.791+0000 7f0d65533700 1 -- 172.21.15.6:0/184803814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d60114af0 con 0x7f0d60078ca0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.791+0000 7f0d477fe700 1 -- 172.21.15.6:0/184803814 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f0d5001a590 con 0x7f0d60078ca0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.792+0000 7f0d477fe700 1 --2- 172.21.15.6:0/184803814 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0d480367e0 0x7f0d48038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.792+0000 7f0d65533700 1 -- 172.21.15.6:0/184803814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6005f5f0 con 0x7f0d60078ca0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.792+0000 7f0d477fe700 1 -- 172.21.15.6:0/184803814 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0d5004a5c0 con 0x7f0d60078ca0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.792+0000 7f0d5e7fc700 1 --2- 172.21.15.6:0/184803814 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0d480367e0 0x7f0d48038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.792+0000 7f0d5e7fc700 1 --2- 172.21.15.6:0/184803814 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0d480367e0 0x7f0d48038c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009990 tx=0x7f0d54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:17.797+0000 7f0d477fe700 1 -- 172.21.15.6:0/184803814 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0d50010890 con 0x7f0d60078ca0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.047+0000 7f0d65533700 1 -- 172.21.15.6:0/184803814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f0d6004bed0 con 0x7f0d60078ca0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.121+0000 7f0d477fe700 1 -- 172.21.15.6:0/184803814 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f0d50024760 con 0x7f0d60078ca0 2024-03-22T06:06:18.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.122+0000 7f0d477fe700 1 -- 172.21.15.6:0/184803814 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f0d50018350 con 0x7f0d60078ca0 2024-03-22T06:06:18.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.124+0000 7f0d65533700 1 -- 172.21.15.6:0/184803814 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0d480367e0 msgr2=0x7f0d48038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:18.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.124+0000 7f0d65533700 1 --2- 172.21.15.6:0/184803814 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0d480367e0 0x7f0d48038c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009990 tx=0x7f0d54006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:18.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.124+0000 7f0d65533700 1 -- 172.21.15.6:0/184803814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 msgr2=0x7f0d60113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:18.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.124+0000 7f0d65533700 1 --2- 172.21.15.6:0/184803814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60113840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d500094c0 tx=0x7f0d50009250 comp rx=0 tx=0).stop 2024-03-22T06:06:18.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.124+0000 7f0d65533700 1 -- 172.21.15.6:0/184803814 shutdown_connections 2024-03-22T06:06:18.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.124+0000 7f0d65533700 1 --2- 172.21.15.6:0/184803814 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f0d480367e0 0x7f0d48038c90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:18.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.124+0000 7f0d65533700 1 --2- 172.21.15.6:0/184803814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d60078ca0 0x7f0d60113840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:18.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.124+0000 7f0d65533700 1 -- 172.21.15.6:0/184803814 >> 172.21.15.6:0/184803814 conn(0x7f0d600fdea0 msgr2=0x7f0d600fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:18.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.124+0000 7f0d65533700 1 -- 172.21.15.6:0/184803814 shutdown_connections 2024-03-22T06:06:18.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:18.124+0000 7f0d65533700 1 -- 172.21.15.6:0/184803814 wait complete. 2024-03-22T06:06:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:19 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/184803814' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-22T06:06:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:19 smithi006 ceph-mon[29301]: mgrmap e9: smithi006.sfpxdy(active, since 24s) 2024-03-22T06:06:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:06:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-22T06:06:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T06:06:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "smithi006.sfpxdy", 2024-03-22T06:06:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-22T06:06:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:06:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.117+0000 7f26ec79d700 1 Processor -- start 2024-03-22T06:06:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.117+0000 7f26ec79d700 1 -- start start 2024-03-22T06:06:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.118+0000 7f26ec79d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4105340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.118+0000 7f26ec79d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4105880 con 0x7f26e4071300 2024-03-22T06:06:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.118+0000 7f26ea539700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4105340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.118+0000 7f26ea539700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4105340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37356/0 (socket says 172.21.15.6:37356) 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.118+0000 7f26ea539700 1 -- 172.21.15.6:0/833522349 learned_addr learned my addr 172.21.15.6:0/833522349 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.118+0000 7f26ea539700 1 -- 172.21.15.6:0/833522349 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e41059c0 con 0x7f26e4071300 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.119+0000 7f26ea539700 1 --2- 172.21.15.6:0/833522349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4105340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26e00098e0 tx=0x7f26e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cc06248c21dbbd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.119+0000 7f26e9537700 1 -- 172.21.15.6:0/833522349 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26e000b860 con 0x7f26e4071300 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.119+0000 7f26e9537700 1 -- 172.21.15.6:0/833522349 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26e000be90 con 0x7f26e4071300 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.119+0000 7f26e9537700 1 -- 172.21.15.6:0/833522349 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26e00103c0 con 0x7f26e4071300 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.120+0000 7f26ec79d700 1 -- 172.21.15.6:0/833522349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 msgr2=0x7f26e4105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.120+0000 7f26ec79d700 1 --2- 172.21.15.6:0/833522349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4105340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26e00098e0 tx=0x7f26e0009250 comp rx=0 tx=0).stop 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.120+0000 7f26ec79d700 1 -- 172.21.15.6:0/833522349 shutdown_connections 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.120+0000 7f26ec79d700 1 --2- 172.21.15.6:0/833522349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4105340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.120+0000 7f26ec79d700 1 -- 172.21.15.6:0/833522349 >> 172.21.15.6:0/833522349 conn(0x7f26e40fcd50 msgr2=0x7f26e40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.120+0000 7f26ec79d700 1 -- 172.21.15.6:0/833522349 shutdown_connections 2024-03-22T06:06:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.120+0000 7f26ec79d700 1 -- 172.21.15.6:0/833522349 wait complete. 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.120+0000 7f26ec79d700 1 Processor -- start 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.120+0000 7f26ec79d700 1 -- start start 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.121+0000 7f26ec79d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.121+0000 7f26ec79d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4111b50 con 0x7f26e4071300 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.121+0000 7f26ea539700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.121+0000 7f26ea539700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37368/0 (socket says 172.21.15.6:37368) 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.121+0000 7f26ea539700 1 -- 172.21.15.6:0/1634766327 learned_addr learned my addr 172.21.15.6:0/1634766327 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.121+0000 7f26ea539700 1 -- 172.21.15.6:0/1634766327 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e00094f0 con 0x7f26e4071300 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.121+0000 7f26ea539700 1 --2- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4111610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f26e00094c0 tx=0x7f26e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.123+0000 7f26daffd700 1 -- 172.21.15.6:0/1634766327 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26e00118f0 con 0x7f26e4071300 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.123+0000 7f26daffd700 1 -- 172.21.15.6:0/1634766327 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26e0010d60 con 0x7f26e4071300 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.123+0000 7f26ec79d700 1 -- 172.21.15.6:0/1634766327 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e4111d50 con 0x7f26e4071300 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.123+0000 7f26daffd700 1 -- 172.21.15.6:0/1634766327 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26e001a430 con 0x7f26e4071300 2024-03-22T06:06:19.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.124+0000 7f26ec79d700 1 -- 172.21.15.6:0/1634766327 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e41128c0 con 0x7f26e4071300 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.125+0000 7f26ec79d700 1 -- 172.21.15.6:0/1634766327 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e405f5f0 con 0x7f26e4071300 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.125+0000 7f26daffd700 1 -- 172.21.15.6:0/1634766327 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f26e0024760 con 0x7f26e4071300 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.125+0000 7f26daffd700 1 --2- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f26d0036830 0x7f26d0038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.125+0000 7f26e9d38700 1 -- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f26d0036830 msgr2=0x7f26d0038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3903206979 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.125+0000 7f26e9d38700 1 --2- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f26d0036830 0x7f26d0038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.125+0000 7f26daffd700 1 -- 172.21.15.6:0/1634766327 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f26e004a510 con 0x7f26e4071300 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.130+0000 7f26daffd700 1 -- 172.21.15.6:0/1634766327 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f26e0022b30 con 0x7f26e4071300 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.326+0000 7f26e9d38700 1 -- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f26d0036830 msgr2=0x7f26d0038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3903206979 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.326+0000 7f26e9d38700 1 --2- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f26d0036830 0x7f26d0038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.379+0000 7f26ec79d700 1 -- 172.21.15.6:0/1634766327 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f26e4112c60 con 0x7f26e4071300 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.380+0000 7f26daffd700 1 -- 172.21.15.6:0/1634766327 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f26e0010ed0 con 0x7f26e4071300 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.382+0000 7f26ec79d700 1 -- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f26d0036830 msgr2=0x7f26d0038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.382+0000 7f26ec79d700 1 --2- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f26d0036830 0x7f26d0038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:19.934 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.382+0000 7f26ec79d700 1 -- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 msgr2=0x7f26e4111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:19.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.382+0000 7f26ec79d700 1 --2- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4111610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f26e00094c0 tx=0x7f26e0009250 comp rx=0 tx=0).stop 2024-03-22T06:06:19.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.382+0000 7f26ec79d700 1 -- 172.21.15.6:0/1634766327 shutdown_connections 2024-03-22T06:06:19.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.382+0000 7f26ec79d700 1 --2- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f26d0036830 0x7f26d0038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:19.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.382+0000 7f26ec79d700 1 --2- 172.21.15.6:0/1634766327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4071300 0x7f26e4111610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:19.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.382+0000 7f26ec79d700 1 -- 172.21.15.6:0/1634766327 >> 172.21.15.6:0/1634766327 conn(0x7f26e40fcd50 msgr2=0x7f26e40fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:19.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.382+0000 7f26ec79d700 1 -- 172.21.15.6:0/1634766327 shutdown_connections 2024-03-22T06:06:19.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:19.382+0000 7f26ec79d700 1 -- 172.21.15.6:0/1634766327 wait complete. 2024-03-22T06:06:19.935 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2024-03-22T06:06:19.935 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 9... 2024-03-22T06:06:20.176 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:20 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1634766327' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-22T06:06:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: Active manager daemon smithi006.sfpxdy restarted 2024-03-22T06:06:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: Activating manager daemon smithi006.sfpxdy 2024-03-22T06:06:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: osdmap e3: 0 total, 0 up, 0 in 2024-03-22T06:06:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: mgrmap e10: smithi006.sfpxdy(active, starting, since 0.00311614s) 2024-03-22T06:06:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:06:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi006.sfpxdy", "id": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:06:25.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T06:06:25.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T06:06:25.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T06:06:25.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: Manager daemon smithi006.sfpxdy is now available 2024-03-22T06:06:25.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:06:25.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:24 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:06:26.181 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:25 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:06:26.181 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:25 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/trash_purge_schedule"}]: dispatch 2024-03-22T06:06:26.181 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:25 smithi006 ceph-mon[29301]: mgrmap e11: smithi006.sfpxdy(active, since 1.00541s) 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.382+0000 7f524d298700 1 Processor -- start 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.382+0000 7f524d298700 1 -- start start 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.382+0000 7f524d298700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248105330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.382+0000 7f524d298700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5248105870 con 0x7f5248076bb0 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.382+0000 7f5246ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248105330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.382+0000 7f5246ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248105330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37370/0 (socket says 172.21.15.6:37370) 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.382+0000 7f5246ffd700 1 -- 172.21.15.6:0/2223563384 learned_addr learned my addr 172.21.15.6:0/2223563384 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.383+0000 7f5246ffd700 1 -- 172.21.15.6:0/2223563384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52481059b0 con 0x7f5248076bb0 2024-03-22T06:06:26.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.383+0000 7f5246ffd700 1 --2- 172.21.15.6:0/2223563384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248105330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52380098e0 tx=0x7f5238009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb3bc2e5b418531 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.383+0000 7f5245ffb700 1 -- 172.21.15.6:0/2223563384 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f523800b860 con 0x7f5248076bb0 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.383+0000 7f5245ffb700 1 -- 172.21.15.6:0/2223563384 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f523800be90 con 0x7f5248076bb0 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.383+0000 7f5245ffb700 1 -- 172.21.15.6:0/2223563384 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52380103c0 con 0x7f5248076bb0 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.384+0000 7f524d298700 1 -- 172.21.15.6:0/2223563384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 msgr2=0x7f5248105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.384+0000 7f524d298700 1 --2- 172.21.15.6:0/2223563384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248105330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52380098e0 tx=0x7f5238009250 comp rx=0 tx=0).stop 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.384+0000 7f524d298700 1 -- 172.21.15.6:0/2223563384 shutdown_connections 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.384+0000 7f524d298700 1 --2- 172.21.15.6:0/2223563384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248105330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.384+0000 7f524d298700 1 -- 172.21.15.6:0/2223563384 >> 172.21.15.6:0/2223563384 conn(0x7f52480fcde0 msgr2=0x7f52480ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.384+0000 7f524d298700 1 -- 172.21.15.6:0/2223563384 shutdown_connections 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.384+0000 7f524d298700 1 -- 172.21.15.6:0/2223563384 wait complete. 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.385+0000 7f524d298700 1 Processor -- start 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.385+0000 7f524d298700 1 -- start start 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.385+0000 7f524d298700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.385+0000 7f524d298700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5248111ac0 con 0x7f5248076bb0 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.385+0000 7f5246ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:26.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.385+0000 7f5246ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248111580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37380/0 (socket says 172.21.15.6:37380) 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.385+0000 7f5246ffd700 1 -- 172.21.15.6:0/1406100262 learned_addr learned my addr 172.21.15.6:0/1406100262 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.385+0000 7f5246ffd700 1 -- 172.21.15.6:0/1406100262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380094f0 con 0x7f5248076bb0 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.386+0000 7f5246ffd700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248111580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f52380094c0 tx=0x7f5238009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.386+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5238010890 con 0x7f5248076bb0 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.386+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5238010ec0 con 0x7f5248076bb0 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.386+0000 7f524d298700 1 -- 172.21.15.6:0/1406100262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5248111cc0 con 0x7f5248076bb0 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.386+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52380243f0 con 0x7f5248076bb0 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.386+0000 7f524d298700 1 -- 172.21.15.6:0/1406100262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5248112830 con 0x7f5248076bb0 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.387+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f523802d3f0 con 0x7f5248076bb0 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.387+0000 7f522f7fe700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 0x7f5230038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.387+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 --> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52300393a0 con 0x7f52300367e0 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.387+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f523804a330 con 0x7f5248076bb0 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.387+0000 7f52467fc700 1 -- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 msgr2=0x7f5230038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3903206979 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.387+0000 7f52467fc700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 0x7f5230038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T06:06:26.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.588+0000 7f52467fc700 1 -- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 msgr2=0x7f5230038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3903206979 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.588+0000 7f52467fc700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 0x7f5230038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.989+0000 7f52467fc700 1 -- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 msgr2=0x7f5230038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3903206979 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:20.989+0000 7f52467fc700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 0x7f5230038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:21.790+0000 7f52467fc700 1 -- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 msgr2=0x7f5230038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3903206979 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:21.790+0000 7f52467fc700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 0x7f5230038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:23.391+0000 7f52467fc700 1 -- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 msgr2=0x7f5230038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3903206979 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:23.391+0000 7f52467fc700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 0x7f5230038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:24.833+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7f523802d740 con 0x7f5248076bb0 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:24.833+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 msgr2=0x7f5230038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:24.833+0000 7f522f7fe700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 0x7f5230038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.834+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f523801a7e0 con 0x7f5248076bb0 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.834+0000 7f522f7fe700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f523003a070 0x7f523003c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.834+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 --> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52300393a0 con 0x7f523003a070 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.834+0000 7f52467fc700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f523003a070 0x7f523003c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.835+0000 7f52467fc700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f523003a070 0x7f523003c450 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f523c0039d0 tx=0x7f523c0092f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:26.494 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.836+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 <== mgr.14162 v2:172.21.15.6:6800/358732355 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f52300393a0 con 0x7f523003a070 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.846+0000 7f524d298700 1 -- 172.21.15.6:0/1406100262 --> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f5248112150 con 0x7f523003a070 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.846+0000 7f522f7fe700 1 -- 172.21.15.6:0/1406100262 <== mgr.14162 v2:172.21.15.6:6800/358732355 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f5248112150 con 0x7f523003a070 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.847+0000 7f524d298700 1 -- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f523003a070 msgr2=0x7f523003c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.847+0000 7f524d298700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f523003a070 0x7f523003c450 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f523c0039d0 tx=0x7f523c0092f0 comp rx=0 tx=0).stop 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.847+0000 7f524d298700 1 -- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 msgr2=0x7f5248111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.847+0000 7f524d298700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248111580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f52380094c0 tx=0x7f5238009250 comp rx=0 tx=0).stop 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.847+0000 7f524d298700 1 -- 172.21.15.6:0/1406100262 shutdown_connections 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.847+0000 7f524d298700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f523003a070 0x7f523003c450 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.847+0000 7f524d298700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5248076bb0 0x7f5248111580 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.847+0000 7f524d298700 1 --2- 172.21.15.6:0/1406100262 >> [v2:172.21.15.6:6800/3903206979,v1:172.21.15.6:6801/3903206979] conn(0x7f52300367e0 0x7f5230038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.847+0000 7f524d298700 1 -- 172.21.15.6:0/1406100262 >> 172.21.15.6:0/1406100262 conn(0x7f52480fcde0 msgr2=0x7f52480fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.847+0000 7f524d298700 1 -- 172.21.15.6:0/1406100262 shutdown_connections 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:25.847+0000 7f524d298700 1 -- 172.21.15.6:0/1406100262 wait complete. 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 9 is available 2024-03-22T06:06:26.495 INFO:teuthology.orchestra.run.smithi006.stdout:Generating a dashboard self-signed certificate... 2024-03-22T06:06:27.125 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:26 smithi006 ceph-mon[29301]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-22T06:06:27.125 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:26 smithi006 ceph-mon[29301]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-22T06:06:27.125 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:26 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.909+0000 7fda2320f700 1 Processor -- start 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.909+0000 7fda2320f700 1 -- start start 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.909+0000 7fda2320f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.909+0000 7fda2320f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c102030 con 0x7fda1c1016e0 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.909+0000 7fda20fab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.909+0000 7fda20fab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41330/0 (socket says 172.21.15.6:41330) 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.909+0000 7fda20fab700 1 -- 172.21.15.6:0/549074573 learned_addr learned my addr 172.21.15.6:0/549074573 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.910+0000 7fda20fab700 1 -- 172.21.15.6:0/549074573 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c102170 con 0x7fda1c1016e0 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.910+0000 7fda20fab700 1 --2- 172.21.15.6:0/549074573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c101af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fda1000bb80 tx=0x7fda1000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3577f927d96932e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.910+0000 7fda1b7fe700 1 -- 172.21.15.6:0/549074573 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda1000dd00 con 0x7fda1c1016e0 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.910+0000 7fda1b7fe700 1 -- 172.21.15.6:0/549074573 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda1000de60 con 0x7fda1c1016e0 2024-03-22T06:06:27.789 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.910+0000 7fda1b7fe700 1 -- 172.21.15.6:0/549074573 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda10018920 con 0x7fda1c1016e0 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.911+0000 7fda2320f700 1 -- 172.21.15.6:0/549074573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 msgr2=0x7fda1c101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.911+0000 7fda2320f700 1 --2- 172.21.15.6:0/549074573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c101af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fda1000bb80 tx=0x7fda1000b4f0 comp rx=0 tx=0).stop 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.911+0000 7fda2320f700 1 -- 172.21.15.6:0/549074573 shutdown_connections 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.911+0000 7fda2320f700 1 --2- 172.21.15.6:0/549074573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c101af0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.911+0000 7fda2320f700 1 -- 172.21.15.6:0/549074573 >> 172.21.15.6:0/549074573 conn(0x7fda1c0fce40 msgr2=0x7fda1c0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.911+0000 7fda2320f700 1 -- 172.21.15.6:0/549074573 shutdown_connections 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.911+0000 7fda2320f700 1 -- 172.21.15.6:0/549074573 wait complete. 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.911+0000 7fda2320f700 1 Processor -- start 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.912+0000 7fda2320f700 1 -- start start 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.912+0000 7fda2320f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c10f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.912+0000 7fda2320f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c10f890 con 0x7fda1c1016e0 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.912+0000 7fda20fab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c10f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.912+0000 7fda20fab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c10f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41342/0 (socket says 172.21.15.6:41342) 2024-03-22T06:06:27.790 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.912+0000 7fda20fab700 1 -- 172.21.15.6:0/706751369 learned_addr learned my addr 172.21.15.6:0/706751369 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.912+0000 7fda20fab700 1 -- 172.21.15.6:0/706751369 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1000b790 con 0x7fda1c1016e0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.912+0000 7fda20fab700 1 --2- 172.21.15.6:0/706751369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c10f350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda100106a0 tx=0x7fda1000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.912+0000 7fda197fa700 1 -- 172.21.15.6:0/706751369 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda10010c30 con 0x7fda1c1016e0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.913+0000 7fda197fa700 1 -- 172.21.15.6:0/706751369 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda1001d050 con 0x7fda1c1016e0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.913+0000 7fda2320f700 1 -- 172.21.15.6:0/706751369 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1c10fa90 con 0x7fda1c1016e0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.913+0000 7fda197fa700 1 -- 172.21.15.6:0/706751369 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda10022590 con 0x7fda1c1016e0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.913+0000 7fda2320f700 1 -- 172.21.15.6:0/706751369 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1c110600 con 0x7fda1c1016e0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.914+0000 7fda2320f700 1 -- 172.21.15.6:0/706751369 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1c05f5f0 con 0x7fda1c1016e0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.914+0000 7fda197fa700 1 -- 172.21.15.6:0/706751369 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fda10023a40 con 0x7fda1c1016e0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.914+0000 7fda197fa700 1 --2- 172.21.15.6:0/706751369 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fda0c0366b0 0x7fda0c038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.914+0000 7fda197fa700 1 -- 172.21.15.6:0/706751369 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fda10014070 con 0x7fda1c1016e0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.914+0000 7fda1bfff700 1 --2- 172.21.15.6:0/706751369 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fda0c0366b0 0x7fda0c038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.915+0000 7fda1bfff700 1 --2- 172.21.15.6:0/706751369 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fda0c0366b0 0x7fda0c038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fda08009990 tx=0x7fda08006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:26.923+0000 7fda197fa700 1 -- 172.21.15.6:0/706751369 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fda1000e360 con 0x7fda1c1016e0 2024-03-22T06:06:27.791 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.124+0000 7fda2320f700 1 -- 172.21.15.6:0/706751369 --> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fda1c1109b0 con 0x7fda0c0366b0 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.182+0000 7fda197fa700 1 -- 172.21.15.6:0/706751369 <== mgr.14162 v2:172.21.15.6:6800/358732355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fda1c1109b0 con 0x7fda0c0366b0 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.185+0000 7fda2320f700 1 -- 172.21.15.6:0/706751369 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fda0c0366b0 msgr2=0x7fda0c038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.185+0000 7fda2320f700 1 --2- 172.21.15.6:0/706751369 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fda0c0366b0 0x7fda0c038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fda08009990 tx=0x7fda08006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.185+0000 7fda2320f700 1 -- 172.21.15.6:0/706751369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 msgr2=0x7fda1c10f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.185+0000 7fda2320f700 1 --2- 172.21.15.6:0/706751369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c10f350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda100106a0 tx=0x7fda1000b4f0 comp rx=0 tx=0).stop 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.185+0000 7fda2320f700 1 -- 172.21.15.6:0/706751369 shutdown_connections 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.185+0000 7fda2320f700 1 --2- 172.21.15.6:0/706751369 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fda0c0366b0 0x7fda0c038b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.185+0000 7fda2320f700 1 --2- 172.21.15.6:0/706751369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda1c1016e0 0x7fda1c10f350 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.185+0000 7fda2320f700 1 -- 172.21.15.6:0/706751369 >> 172.21.15.6:0/706751369 conn(0x7fda1c0fce40 msgr2=0x7fda1c105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.185+0000 7fda2320f700 1 -- 172.21.15.6:0/706751369 shutdown_connections 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:27.185+0000 7fda2320f700 1 -- 172.21.15.6:0/706751369 wait complete. 2024-03-22T06:06:27.792 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial admin user... 2024-03-22T06:06:28.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:28 smithi006 ceph-mon[29301]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:28.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:28 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:28.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:28 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:28.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:28 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:28.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:28 smithi006 ceph-mon[29301]: mgrmap e12: smithi006.sfpxdy(active, since 2s) 2024-03-22T06:06:28.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:28 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:29.366 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:29 smithi006 ceph-mon[29301]: [22/Mar/2024:06:06:27] ENGINE Bus STARTING 2024-03-22T06:06:29.366 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:29 smithi006 ceph-mon[29301]: [22/Mar/2024:06:06:27] ENGINE Serving on https://172.21.15.6:7150 2024-03-22T06:06:29.366 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:29 smithi006 ceph-mon[29301]: [22/Mar/2024:06:06:27] ENGINE Bus STARTED 2024-03-22T06:06:29.366 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:29 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:29.893 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$3vuX.fbA/8i8aE.woJiS2uDFGAKM6UzrghU0MQNSoHhMnMbNOwt32", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711087588, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.218+0000 7f3cda1a5700 1 Processor -- start 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.218+0000 7f3cda1a5700 1 -- start start 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.218+0000 7f3cda1a5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.218+0000 7f3cda1a5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4103080 con 0x7f3cd4102730 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.218+0000 7f3cd37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.218+0000 7f3cd37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41350/0 (socket says 172.21.15.6:41350) 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.218+0000 7f3cd37fe700 1 -- 172.21.15.6:0/803181428 learned_addr learned my addr 172.21.15.6:0/803181428 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.218+0000 7f3cd37fe700 1 -- 172.21.15.6:0/803181428 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd41031c0 con 0x7f3cd4102730 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.219+0000 7f3cd37fe700 1 --2- 172.21.15.6:0/803181428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4102b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80098e0 tx=0x7f3cc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d12e866f465386e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.219+0000 7f3cd27fc700 1 -- 172.21.15.6:0/803181428 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cc800b860 con 0x7f3cd4102730 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.219+0000 7f3cd27fc700 1 -- 172.21.15.6:0/803181428 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cc800be90 con 0x7f3cd4102730 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.219+0000 7f3cd27fc700 1 -- 172.21.15.6:0/803181428 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cc80103c0 con 0x7f3cd4102730 2024-03-22T06:06:29.894 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.219+0000 7f3cda1a5700 1 -- 172.21.15.6:0/803181428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 msgr2=0x7f3cd4102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.219+0000 7f3cda1a5700 1 --2- 172.21.15.6:0/803181428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4102b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80098e0 tx=0x7f3cc8009250 comp rx=0 tx=0).stop 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.219+0000 7f3cda1a5700 1 -- 172.21.15.6:0/803181428 shutdown_connections 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.219+0000 7f3cda1a5700 1 --2- 172.21.15.6:0/803181428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4102b40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.219+0000 7f3cda1a5700 1 -- 172.21.15.6:0/803181428 >> 172.21.15.6:0/803181428 conn(0x7f3cd40fdf10 msgr2=0x7f3cd4100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.219+0000 7f3cda1a5700 1 -- 172.21.15.6:0/803181428 shutdown_connections 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.219+0000 7f3cda1a5700 1 -- 172.21.15.6:0/803181428 wait complete. 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.220+0000 7f3cda1a5700 1 Processor -- start 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.220+0000 7f3cda1a5700 1 -- start start 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.220+0000 7f3cda1a5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.220+0000 7f3cda1a5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4079110 con 0x7f3cd4102730 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.220+0000 7f3cd37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.220+0000 7f3cd37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41360/0 (socket says 172.21.15.6:41360) 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.220+0000 7f3cd37fe700 1 -- 172.21.15.6:0/7631874 learned_addr learned my addr 172.21.15.6:0/7631874 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.220+0000 7f3cd37fe700 1 -- 172.21.15.6:0/7631874 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80094f0 con 0x7f3cd4102730 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.221+0000 7f3cd37fe700 1 --2- 172.21.15.6:0/7631874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4078bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80094c0 tx=0x7f3cc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:29.895 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.221+0000 7f3cbbfff700 1 -- 172.21.15.6:0/7631874 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cc8010680 con 0x7f3cd4102730 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.221+0000 7f3cbbfff700 1 -- 172.21.15.6:0/7631874 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cc8010cb0 con 0x7f3cd4102730 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.221+0000 7f3cda1a5700 1 -- 172.21.15.6:0/7631874 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd4077180 con 0x7f3cd4102730 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.221+0000 7f3cbbfff700 1 -- 172.21.15.6:0/7631874 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cc801ad40 con 0x7f3cd4102730 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.221+0000 7f3cda1a5700 1 -- 172.21.15.6:0/7631874 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd4077670 con 0x7f3cd4102730 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.222+0000 7f3cbbfff700 1 -- 172.21.15.6:0/7631874 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f3cc8025400 con 0x7f3cd4102730 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.222+0000 7f3cda1a5700 1 -- 172.21.15.6:0/7631874 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd404bed0 con 0x7f3cd4102730 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.222+0000 7f3cbbfff700 1 --2- 172.21.15.6:0/7631874 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3cbc0367e0 0x7f3cbc038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.222+0000 7f3cbbfff700 1 -- 172.21.15.6:0/7631874 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3cc804a5f0 con 0x7f3cd4102730 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.222+0000 7f3cd2ffd700 1 --2- 172.21.15.6:0/7631874 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3cbc0367e0 0x7f3cbc038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.222+0000 7f3cd2ffd700 1 --2- 172.21.15.6:0/7631874 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3cbc0367e0 0x7f3cbc038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009990 tx=0x7f3cc4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.227+0000 7f3cbbfff700 1 -- 172.21.15.6:0/7631874 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3cc80193c0 con 0x7f3cd4102730 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.446+0000 7f3cda1a5700 1 -- 172.21.15.6:0/7631874 --> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3cd405f410 con 0x7f3cbc0367e0 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.693+0000 7f3cbbfff700 1 -- 172.21.15.6:0/7631874 <== mgr.14162 v2:172.21.15.6:6800/358732355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f3cd405f410 con 0x7f3cbc0367e0 2024-03-22T06:06:29.896 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.695+0000 7f3cda1a5700 1 -- 172.21.15.6:0/7631874 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3cbc0367e0 msgr2=0x7f3cbc038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:29.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.695+0000 7f3cda1a5700 1 --2- 172.21.15.6:0/7631874 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3cbc0367e0 0x7f3cbc038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009990 tx=0x7f3cc4006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:29.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.695+0000 7f3cda1a5700 1 -- 172.21.15.6:0/7631874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 msgr2=0x7f3cd4078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:29.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.695+0000 7f3cda1a5700 1 --2- 172.21.15.6:0/7631874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4078bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80094c0 tx=0x7f3cc8009250 comp rx=0 tx=0).stop 2024-03-22T06:06:29.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.695+0000 7f3cda1a5700 1 -- 172.21.15.6:0/7631874 shutdown_connections 2024-03-22T06:06:29.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.695+0000 7f3cda1a5700 1 --2- 172.21.15.6:0/7631874 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3cbc0367e0 0x7f3cbc038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:29.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.695+0000 7f3cda1a5700 1 --2- 172.21.15.6:0/7631874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cd4102730 0x7f3cd4078bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:29.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.695+0000 7f3cda1a5700 1 -- 172.21.15.6:0/7631874 >> 172.21.15.6:0/7631874 conn(0x7f3cd40fdf10 msgr2=0x7f3cd40feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:29.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.695+0000 7f3cda1a5700 1 -- 172.21.15.6:0/7631874 shutdown_connections 2024-03-22T06:06:29.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:28.695+0000 7f3cda1a5700 1 -- 172.21.15.6:0/7631874 wait complete. 2024-03-22T06:06:29.897 INFO:teuthology.orchestra.run.smithi006.stdout:Fetching dashboard port number... 2024-03-22T06:06:30.272 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:30 smithi006 ceph-mon[29301]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:31.167 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 8443 2024-03-22T06:06:31.167 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.370+0000 7f2508aa3700 1 Processor -- start 2024-03-22T06:06:31.167 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.370+0000 7f2508aa3700 1 -- start start 2024-03-22T06:06:31.167 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.370+0000 7f2508aa3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f2504102b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:31.167 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.370+0000 7f2508aa3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2504103090 con 0x7f2504102740 2024-03-22T06:06:31.167 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.370+0000 7f250259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f2504102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:31.167 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.370+0000 7f250259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f2504102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41374/0 (socket says 172.21.15.6:41374) 2024-03-22T06:06:31.167 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.370+0000 7f250259c700 1 -- 172.21.15.6:0/4291777213 learned_addr learned my addr 172.21.15.6:0/4291777213 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:31.167 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.370+0000 7f250259c700 1 -- 172.21.15.6:0/4291777213 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25041031d0 con 0x7f2504102740 2024-03-22T06:06:31.167 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.371+0000 7f250259c700 1 --2- 172.21.15.6:0/4291777213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f2504102b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f24f40098e0 tx=0x7f24f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82240d5b9143fffc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.371+0000 7f250159a700 1 -- 172.21.15.6:0/4291777213 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24f400b860 con 0x7f2504102740 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.371+0000 7f250159a700 1 -- 172.21.15.6:0/4291777213 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24f400be90 con 0x7f2504102740 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.371+0000 7f250159a700 1 -- 172.21.15.6:0/4291777213 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24f40103c0 con 0x7f2504102740 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.371+0000 7f2508aa3700 1 -- 172.21.15.6:0/4291777213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 msgr2=0x7f2504102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.371+0000 7f2508aa3700 1 --2- 172.21.15.6:0/4291777213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f2504102b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f24f40098e0 tx=0x7f24f4009250 comp rx=0 tx=0).stop 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.371+0000 7f2508aa3700 1 -- 172.21.15.6:0/4291777213 shutdown_connections 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.371+0000 7f2508aa3700 1 --2- 172.21.15.6:0/4291777213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f2504102b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.371+0000 7f2508aa3700 1 -- 172.21.15.6:0/4291777213 >> 172.21.15.6:0/4291777213 conn(0x7f25040fdea0 msgr2=0x7f25041002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.371+0000 7f2508aa3700 1 -- 172.21.15.6:0/4291777213 shutdown_connections 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.371+0000 7f2508aa3700 1 -- 172.21.15.6:0/4291777213 wait complete. 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.372+0000 7f2508aa3700 1 Processor -- start 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.372+0000 7f2508aa3700 1 -- start start 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.372+0000 7f2508aa3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f25041115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.372+0000 7f2508aa3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2504111b20 con 0x7f2504102740 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.372+0000 7f250259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f25041115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:31.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.372+0000 7f250259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f25041115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41384/0 (socket says 172.21.15.6:41384) 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.372+0000 7f250259c700 1 -- 172.21.15.6:0/3808184668 learned_addr learned my addr 172.21.15.6:0/3808184668 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.372+0000 7f250259c700 1 -- 172.21.15.6:0/3808184668 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f40094f0 con 0x7f2504102740 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.373+0000 7f250259c700 1 --2- 172.21.15.6:0/3808184668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f25041115e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f24f4006b00 tx=0x7f24f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.373+0000 7f24f2ffd700 1 -- 172.21.15.6:0/3808184668 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24f4010840 con 0x7f2504102740 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.373+0000 7f24f2ffd700 1 -- 172.21.15.6:0/3808184668 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24f4010e70 con 0x7f2504102740 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.373+0000 7f24f2ffd700 1 -- 172.21.15.6:0/3808184668 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24f401ae70 con 0x7f2504102740 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.373+0000 7f2508aa3700 1 -- 172.21.15.6:0/3808184668 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2504111d20 con 0x7f2504102740 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.373+0000 7f2508aa3700 1 -- 172.21.15.6:0/3808184668 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2504112890 con 0x7f2504102740 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.374+0000 7f24f2ffd700 1 -- 172.21.15.6:0/3808184668 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f24f401a780 con 0x7f2504102740 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.374+0000 7f24f2ffd700 1 --2- 172.21.15.6:0/3808184668 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f24ec036830 0x7f24ec038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.374+0000 7f2508aa3700 1 -- 172.21.15.6:0/3808184668 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f250405f5f0 con 0x7f2504102740 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.374+0000 7f24f2ffd700 1 -- 172.21.15.6:0/3808184668 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f24f404ac30 con 0x7f2504102740 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.374+0000 7f2501d9b700 1 --2- 172.21.15.6:0/3808184668 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f24ec036830 0x7f24ec038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.374+0000 7f2501d9b700 1 --2- 172.21.15.6:0/3808184668 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f24ec036830 0x7f24ec038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009990 tx=0x7f24f8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.379+0000 7f24f2ffd700 1 -- 172.21.15.6:0/3808184668 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f24f401a430 con 0x7f2504102740 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.574+0000 7f2508aa3700 1 -- 172.21.15.6:0/3808184668 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f250404bed0 con 0x7f2504102740 2024-03-22T06:06:31.169 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.574+0000 7f24f2ffd700 1 -- 172.21.15.6:0/3808184668 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f24f4048020 con 0x7f2504102740 2024-03-22T06:06:31.170 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.577+0000 7f2508aa3700 1 -- 172.21.15.6:0/3808184668 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f24ec036830 msgr2=0x7f24ec038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:31.170 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.577+0000 7f2508aa3700 1 --2- 172.21.15.6:0/3808184668 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f24ec036830 0x7f24ec038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009990 tx=0x7f24f8006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:31.170 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.577+0000 7f2508aa3700 1 -- 172.21.15.6:0/3808184668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 msgr2=0x7f25041115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:31.170 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.577+0000 7f2508aa3700 1 --2- 172.21.15.6:0/3808184668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f25041115e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f24f4006b00 tx=0x7f24f4009250 comp rx=0 tx=0).stop 2024-03-22T06:06:31.170 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.577+0000 7f2508aa3700 1 -- 172.21.15.6:0/3808184668 shutdown_connections 2024-03-22T06:06:31.170 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.577+0000 7f2508aa3700 1 --2- 172.21.15.6:0/3808184668 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f24ec036830 0x7f24ec038ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:31.170 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.577+0000 7f2508aa3700 1 --2- 172.21.15.6:0/3808184668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504102740 0x7f25041115e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:31.170 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.577+0000 7f2508aa3700 1 -- 172.21.15.6:0/3808184668 >> 172.21.15.6:0/3808184668 conn(0x7f25040fdea0 msgr2=0x7f25040fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:31.170 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.577+0000 7f2508aa3700 1 -- 172.21.15.6:0/3808184668 shutdown_connections 2024-03-22T06:06:31.170 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:30.577+0000 7f2508aa3700 1 -- 172.21.15.6:0/3808184668 wait complete. 2024-03-22T06:06:31.175 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T06:06:31.175 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-03-22T06:06:31.184 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T06:06:31.184 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-22T06:06:31.184 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-03-22T06:06:31.185 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-22T06:06:31.187 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph Dashboard is now available at: 2024-03-22T06:06:31.187 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:06:31.187 INFO:teuthology.orchestra.run.smithi006.stdout: URL: https://smithi006.front.sepia.ceph.com:8443/ 2024-03-22T06:06:31.187 INFO:teuthology.orchestra.run.smithi006.stdout: User: admin 2024-03-22T06:06:31.187 INFO:teuthology.orchestra.run.smithi006.stdout: Password: 4cmw2hpxel 2024-03-22T06:06:31.187 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:06:31.188 INFO:teuthology.orchestra.run.smithi006.stdout:Saving cluster configuration to /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config directory 2024-03-22T06:06:31.188 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling autotune for osd_memory_target 2024-03-22T06:06:31.415 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:31 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3808184668' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-22T06:06:32.399 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:32 smithi006 ceph-mon[29301]: mgrmap e13: smithi006.sfpxdy(active, since 6s) 2024-03-22T06:06:32.406 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.618+0000 7fc5b31a7700 1 Processor -- start 2024-03-22T06:06:32.406 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.618+0000 7fc5b31a7700 1 -- start start 2024-03-22T06:06:32.406 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.618+0000 7fc5b31a7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac101a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:32.406 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.618+0000 7fc5b31a7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac101f70 con 0x7fc5ac101620 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.618+0000 7fc5b0f43700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.618+0000 7fc5b0f43700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41394/0 (socket says 172.21.15.6:41394) 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.618+0000 7fc5b0f43700 1 -- 172.21.15.6:0/230406426 learned_addr learned my addr 172.21.15.6:0/230406426 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.619+0000 7fc5b0f43700 1 -- 172.21.15.6:0/230406426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac1020b0 con 0x7fc5ac101620 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.619+0000 7fc5b0f43700 1 --2- 172.21.15.6:0/230406426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac101a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00098e0 tx=0x7fc5a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eeff4101b421431 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.619+0000 7fc5ab7fe700 1 -- 172.21.15.6:0/230406426 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5a000b860 con 0x7fc5ac101620 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.619+0000 7fc5ab7fe700 1 -- 172.21.15.6:0/230406426 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5a000be90 con 0x7fc5ac101620 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.619+0000 7fc5ab7fe700 1 -- 172.21.15.6:0/230406426 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5a00103c0 con 0x7fc5ac101620 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.620+0000 7fc5b31a7700 1 -- 172.21.15.6:0/230406426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 msgr2=0x7fc5ac101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.620+0000 7fc5b31a7700 1 --2- 172.21.15.6:0/230406426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac101a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00098e0 tx=0x7fc5a0009250 comp rx=0 tx=0).stop 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.620+0000 7fc5b31a7700 1 -- 172.21.15.6:0/230406426 shutdown_connections 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.620+0000 7fc5b31a7700 1 --2- 172.21.15.6:0/230406426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac101a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.620+0000 7fc5b31a7700 1 -- 172.21.15.6:0/230406426 >> 172.21.15.6:0/230406426 conn(0x7fc5ac0fcdc0 msgr2=0x7fc5ac0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.620+0000 7fc5b31a7700 1 -- 172.21.15.6:0/230406426 shutdown_connections 2024-03-22T06:06:32.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.620+0000 7fc5b31a7700 1 -- 172.21.15.6:0/230406426 wait complete. 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.620+0000 7fc5b31a7700 1 Processor -- start 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.620+0000 7fc5b31a7700 1 -- start start 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5b31a7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac076ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5b31a7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac112090 con 0x7fc5ac101620 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5b0f43700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac076ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5b0f43700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac076ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41404/0 (socket says 172.21.15.6:41404) 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5b0f43700 1 -- 172.21.15.6:0/424091215 learned_addr learned my addr 172.21.15.6:0/424091215 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5b0f43700 1 -- 172.21.15.6:0/424091215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a00094f0 con 0x7fc5ac101620 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5b0f43700 1 --2- 172.21.15.6:0/424091215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac076ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0011680 tx=0x7fc5a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5a97fa700 1 -- 172.21.15.6:0/424091215 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5a0010890 con 0x7fc5ac101620 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5a97fa700 1 -- 172.21.15.6:0/424091215 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5a0010ec0 con 0x7fc5ac101620 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5a97fa700 1 -- 172.21.15.6:0/424091215 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5a00243f0 con 0x7fc5ac101620 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5b31a7700 1 -- 172.21.15.6:0/424091215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac1166a0 con 0x7fc5ac101620 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.621+0000 7fc5b31a7700 1 -- 172.21.15.6:0/424091215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac116a70 con 0x7fc5ac101620 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.622+0000 7fc5a97fa700 1 -- 172.21.15.6:0/424091215 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fc5a00109f0 con 0x7fc5ac101620 2024-03-22T06:06:32.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.622+0000 7fc5b31a7700 1 -- 172.21.15.6:0/424091215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac04bed0 con 0x7fc5ac101620 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.623+0000 7fc5a97fa700 1 --2- 172.21.15.6:0/424091215 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fc59c036880 0x7fc59c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.623+0000 7fc5a97fa700 1 -- 172.21.15.6:0/424091215 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc5a004a440 con 0x7fc5ac101620 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.623+0000 7fc5abfff700 1 --2- 172.21.15.6:0/424091215 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fc59c036880 0x7fc59c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.623+0000 7fc5abfff700 1 --2- 172.21.15.6:0/424091215 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fc59c036880 0x7fc59c038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc598009990 tx=0x7fc598006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.627+0000 7fc5a97fa700 1 -- 172.21.15.6:0/424091215 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc5a002b940 con 0x7fc5ac101620 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.822+0000 7fc5b31a7700 1 -- 172.21.15.6:0/424091215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fc5ac116c60 con 0x7fc5ac101620 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.822+0000 7fc5a97fa700 1 -- 172.21.15.6:0/424091215 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc5a0006d70 con 0x7fc5ac101620 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.824+0000 7fc5b31a7700 1 -- 172.21.15.6:0/424091215 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fc59c036880 msgr2=0x7fc59c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.824+0000 7fc5b31a7700 1 --2- 172.21.15.6:0/424091215 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fc59c036880 0x7fc59c038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc598009990 tx=0x7fc598006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.824+0000 7fc5b31a7700 1 -- 172.21.15.6:0/424091215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 msgr2=0x7fc5ac076ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.824+0000 7fc5b31a7700 1 --2- 172.21.15.6:0/424091215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac076ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0011680 tx=0x7fc5a0009250 comp rx=0 tx=0).stop 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.824+0000 7fc5b31a7700 1 -- 172.21.15.6:0/424091215 shutdown_connections 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.824+0000 7fc5b31a7700 1 --2- 172.21.15.6:0/424091215 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fc59c036880 0x7fc59c038d30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.824+0000 7fc5b31a7700 1 --2- 172.21.15.6:0/424091215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5ac101620 0x7fc5ac076ab0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.824+0000 7fc5b31a7700 1 -- 172.21.15.6:0/424091215 >> 172.21.15.6:0/424091215 conn(0x7fc5ac0fcdc0 msgr2=0x7fc5ac0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:32.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.824+0000 7fc5b31a7700 1 -- 172.21.15.6:0/424091215 shutdown_connections 2024-03-22T06:06:32.410 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:31.824+0000 7fc5b31a7700 1 -- 172.21.15.6:0/424091215 wait complete. 2024-03-22T06:06:33.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.824+0000 7fcb12384700 1 Processor -- start 2024-03-22T06:06:33.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.824+0000 7fcb12384700 1 -- start start 2024-03-22T06:06:33.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.824+0000 7fcb12384700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c1052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.825+0000 7fcb12384700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c105820 con 0x7fcb0c071360 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.825+0000 7fcb0bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c1052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.825+0000 7fcb0bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c1052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41414/0 (socket says 172.21.15.6:41414) 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.825+0000 7fcb0bfff700 1 -- 172.21.15.6:0/1263462552 learned_addr learned my addr 172.21.15.6:0/1263462552 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.825+0000 7fcb0bfff700 1 -- 172.21.15.6:0/1263462552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c105960 con 0x7fcb0c071360 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.825+0000 7fcb0bfff700 1 --2- 172.21.15.6:0/1263462552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c1052e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0098e0 tx=0x7fcafc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2166a0962bd7d2eb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.825+0000 7fcb0affd700 1 -- 172.21.15.6:0/1263462552 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcafc00b860 con 0x7fcb0c071360 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.825+0000 7fcb0affd700 1 -- 172.21.15.6:0/1263462552 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcafc00be90 con 0x7fcb0c071360 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.825+0000 7fcb0affd700 1 -- 172.21.15.6:0/1263462552 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcafc0103c0 con 0x7fcb0c071360 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.826+0000 7fcb12384700 1 -- 172.21.15.6:0/1263462552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 msgr2=0x7fcb0c1052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.826+0000 7fcb12384700 1 --2- 172.21.15.6:0/1263462552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c1052e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0098e0 tx=0x7fcafc009250 comp rx=0 tx=0).stop 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.826+0000 7fcb12384700 1 -- 172.21.15.6:0/1263462552 shutdown_connections 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.826+0000 7fcb12384700 1 --2- 172.21.15.6:0/1263462552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c1052e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.826+0000 7fcb12384700 1 -- 172.21.15.6:0/1263462552 >> 172.21.15.6:0/1263462552 conn(0x7fcb0c0fcd50 msgr2=0x7fcb0c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.826+0000 7fcb12384700 1 -- 172.21.15.6:0/1263462552 shutdown_connections 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.826+0000 7fcb12384700 1 -- 172.21.15.6:0/1263462552 wait complete. 2024-03-22T06:06:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.827+0000 7fcb12384700 1 Processor -- start 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.827+0000 7fcb12384700 1 -- start start 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.827+0000 7fcb12384700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.827+0000 7fcb12384700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c103170 con 0x7fcb0c071360 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.827+0000 7fcb0bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.827+0000 7fcb0bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41430/0 (socket says 172.21.15.6:41430) 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.827+0000 7fcb0bfff700 1 -- 172.21.15.6:0/373894682 learned_addr learned my addr 172.21.15.6:0/373894682 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.827+0000 7fcb0bfff700 1 -- 172.21.15.6:0/373894682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc0094f0 con 0x7fcb0c071360 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.827+0000 7fcb0bfff700 1 --2- 172.21.15.6:0/373894682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c102c30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0094c0 tx=0x7fcafc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.828+0000 7fcb08ff9700 1 -- 172.21.15.6:0/373894682 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcafc0118f0 con 0x7fcb0c071360 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.828+0000 7fcb08ff9700 1 -- 172.21.15.6:0/373894682 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcafc010d60 con 0x7fcb0c071360 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.828+0000 7fcb12384700 1 -- 172.21.15.6:0/373894682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c1011e0 con 0x7fcb0c071360 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.828+0000 7fcb08ff9700 1 -- 172.21.15.6:0/373894682 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcafc01a430 con 0x7fcb0c071360 2024-03-22T06:06:33.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.828+0000 7fcb12384700 1 -- 172.21.15.6:0/373894682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c1016d0 con 0x7fcb0c071360 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.829+0000 7fcb08ff9700 1 -- 172.21.15.6:0/373894682 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fcafc01a590 con 0x7fcb0c071360 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.829+0000 7fcb08ff9700 1 --2- 172.21.15.6:0/373894682 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcaf40368d0 0x7fcaf4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.829+0000 7fcb08ff9700 1 -- 172.21.15.6:0/373894682 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcafc04a8e0 con 0x7fcb0c071360 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.829+0000 7fcb12384700 1 -- 172.21.15.6:0/373894682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0c04bed0 con 0x7fcb0c071360 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.829+0000 7fcb0b7fe700 1 --2- 172.21.15.6:0/373894682 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcaf40368d0 0x7fcaf4038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.829+0000 7fcb0b7fe700 1 --2- 172.21.15.6:0/373894682 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcaf40368d0 0x7fcaf4038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009990 tx=0x7fcb00006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:32.834+0000 7fcb08ff9700 1 -- 172.21.15.6:0/373894682 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcafc02d3f0 con 0x7fcb0c071360 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.142+0000 7fcb12384700 1 -- 172.21.15.6:0/373894682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fcb0c101b50 con 0x7fcb0c071360 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.145+0000 7fcb08ff9700 1 -- 172.21.15.6:0/373894682 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7fcafc018350 con 0x7fcb0c071360 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.146+0000 7fcb12384700 1 -- 172.21.15.6:0/373894682 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcaf40368d0 msgr2=0x7fcaf4038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.146+0000 7fcb12384700 1 --2- 172.21.15.6:0/373894682 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcaf40368d0 0x7fcaf4038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009990 tx=0x7fcb00006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.146+0000 7fcb12384700 1 -- 172.21.15.6:0/373894682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 msgr2=0x7fcb0c102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.146+0000 7fcb12384700 1 --2- 172.21.15.6:0/373894682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c102c30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0094c0 tx=0x7fcafc009250 comp rx=0 tx=0).stop 2024-03-22T06:06:33.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.147+0000 7fcb12384700 1 -- 172.21.15.6:0/373894682 shutdown_connections 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.147+0000 7fcb12384700 1 --2- 172.21.15.6:0/373894682 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcaf40368d0 0x7fcaf4038d80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.147+0000 7fcb12384700 1 --2- 172.21.15.6:0/373894682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0c071360 0x7fcb0c102c30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.147+0000 7fcb12384700 1 -- 172.21.15.6:0/373894682 >> 172.21.15.6:0/373894682 conn(0x7fcb0c0fcd50 msgr2=0x7fcb0c0fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.147+0000 7fcb12384700 1 -- 172.21.15.6:0/373894682 shutdown_connections 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-22T06:06:33.147+0000 7fcb12384700 1 -- 172.21.15.6:0/373894682 wait complete. 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout:Or, if you are only running a single cluster on this host: 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:06:33.768 INFO:teuthology.orchestra.run.smithi006.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-22T06:06:33.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:06:33.769 INFO:teuthology.orchestra.run.smithi006.stdout: ceph telemetry on 2024-03-22T06:06:33.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:06:33.769 INFO:teuthology.orchestra.run.smithi006.stdout:For more information see: 2024-03-22T06:06:33.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:06:33.769 INFO:teuthology.orchestra.run.smithi006.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-03-22T06:06:33.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:06:33.769 INFO:teuthology.orchestra.run.smithi006.stdout:Bootstrap complete. 2024-03-22T06:06:33.801 INFO:tasks.cephadm:Fetching config... 2024-03-22T06:06:33.802 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:06:33.802 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-22T06:06:33.857 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-22T06:06:33.857 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:06:33.857 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-22T06:06:33.912 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-22T06:06:33.912 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:06:33.912 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/keyring of=/dev/stdout 2024-03-22T06:06:33.980 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-22T06:06:33.980 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:06:33.980 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-22T06:06:34.034 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-22T06:06:34.035 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0j9Tiiux0asll9l2rlt8oDlx3SU70B5W7rn83trBtd195dUCkCff9FYOTvRB6ckYiXJVJaIuRCc13G8AJ/3OaK4Vo9FOKwzMsNVn9FKIet1OG/TYGbEQ+t3097uRJGImZmpXJsrbkgCqPaWipHp58UombAv8XLCJlh8TFXZsw9pmZ25ooiL0eXqyhYVYNRSaa2j1rxk5of1sANL8uH5OkUw7DLKFfXpm/edsiYcXXczCOg25UMb5KsLa2WEbNhr54RRTY2j4g3lS7VgBDrjLM1oZA7jRIdHuxevCqmUHU6K9kCPOnMqkb+/whv3QmZE69UPSFgGHYY9+LRwBuLW8vhlgI4ZuzAKUgC0qz6zmm5dRksnyZ2+hwu9keMjE664XbnfEM05WilCHr1rXylSSmvD6ehq1wXC9ISRFymfoYRj7WA04Vte1NBGQuPb00293nH8jrfPMjGwBUepDO/11xkpqQEAgBt3Y5MNnkzHbjSZ10mrAKLoP4/U0lNGCJRTM= ceph-fa0e13c2-e811-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-22T06:06:34.116 INFO:teuthology.orchestra.run.smithi006.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0j9Tiiux0asll9l2rlt8oDlx3SU70B5W7rn83trBtd195dUCkCff9FYOTvRB6ckYiXJVJaIuRCc13G8AJ/3OaK4Vo9FOKwzMsNVn9FKIet1OG/TYGbEQ+t3097uRJGImZmpXJsrbkgCqPaWipHp58UombAv8XLCJlh8TFXZsw9pmZ25ooiL0eXqyhYVYNRSaa2j1rxk5of1sANL8uH5OkUw7DLKFfXpm/edsiYcXXczCOg25UMb5KsLa2WEbNhr54RRTY2j4g3lS7VgBDrjLM1oZA7jRIdHuxevCqmUHU6K9kCPOnMqkb+/whv3QmZE69UPSFgGHYY9+LRwBuLW8vhlgI4ZuzAKUgC0qz6zmm5dRksnyZ2+hwu9keMjE664XbnfEM05WilCHr1rXylSSmvD6ehq1wXC9ISRFymfoYRj7WA04Vte1NBGQuPb00293nH8jrfPMjGwBUepDO/11xkpqQEAgBt3Y5MNnkzHbjSZ10mrAKLoP4/U0lNGCJRTM= ceph-fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:06:34.129 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0j9Tiiux0asll9l2rlt8oDlx3SU70B5W7rn83trBtd195dUCkCff9FYOTvRB6ckYiXJVJaIuRCc13G8AJ/3OaK4Vo9FOKwzMsNVn9FKIet1OG/TYGbEQ+t3097uRJGImZmpXJsrbkgCqPaWipHp58UombAv8XLCJlh8TFXZsw9pmZ25ooiL0eXqyhYVYNRSaa2j1rxk5of1sANL8uH5OkUw7DLKFfXpm/edsiYcXXczCOg25UMb5KsLa2WEbNhr54RRTY2j4g3lS7VgBDrjLM1oZA7jRIdHuxevCqmUHU6K9kCPOnMqkb+/whv3QmZE69UPSFgGHYY9+LRwBuLW8vhlgI4ZuzAKUgC0qz6zmm5dRksnyZ2+hwu9keMjE664XbnfEM05WilCHr1rXylSSmvD6ehq1wXC9ISRFymfoYRj7WA04Vte1NBGQuPb00293nH8jrfPMjGwBUepDO/11xkpqQEAgBt3Y5MNnkzHbjSZ10mrAKLoP4/U0lNGCJRTM= ceph-fa0e13c2-e811-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-22T06:06:34.175 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0j9Tiiux0asll9l2rlt8oDlx3SU70B5W7rn83trBtd195dUCkCff9FYOTvRB6ckYiXJVJaIuRCc13G8AJ/3OaK4Vo9FOKwzMsNVn9FKIet1OG/TYGbEQ+t3097uRJGImZmpXJsrbkgCqPaWipHp58UombAv8XLCJlh8TFXZsw9pmZ25ooiL0eXqyhYVYNRSaa2j1rxk5of1sANL8uH5OkUw7DLKFfXpm/edsiYcXXczCOg25UMb5KsLa2WEbNhr54RRTY2j4g3lS7VgBDrjLM1oZA7jRIdHuxevCqmUHU6K9kCPOnMqkb+/whv3QmZE69UPSFgGHYY9+LRwBuLW8vhlgI4ZuzAKUgC0qz6zmm5dRksnyZ2+hwu9keMjE664XbnfEM05WilCHr1rXylSSmvD6ehq1wXC9ISRFymfoYRj7WA04Vte1NBGQuPb00293nH8jrfPMjGwBUepDO/11xkpqQEAgBt3Y5MNnkzHbjSZ10mrAKLoP4/U0lNGCJRTM= ceph-fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:06:34.190 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-22T06:06:34.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:34 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/373894682' entity='client.admin' 2024-03-22T06:06:34.445 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.916+0000 7f002754d700 1 -- 172.21.15.6:0/929540229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00200fd050 msgr2=0x7f00200fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.916+0000 7f002754d700 1 --2- 172.21.15.6:0/929540229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00200fd050 0x7f00200fd460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f001c0048c0 tx=0x7f001c009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.916+0000 7f002754d700 1 -- 172.21.15.6:0/929540229 shutdown_connections 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.916+0000 7f002754d700 1 --2- 172.21.15.6:0/929540229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00200fd050 0x7f00200fd460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.916+0000 7f002754d700 1 -- 172.21.15.6:0/929540229 >> 172.21.15.6:0/929540229 conn(0x7f00200f70d0 msgr2=0x7f00200f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.916+0000 7f002754d700 1 -- 172.21.15.6:0/929540229 shutdown_connections 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.917+0000 7f002754d700 1 -- 172.21.15.6:0/929540229 wait complete. 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.917+0000 7f002754d700 1 Processor -- start 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.917+0000 7f002754d700 1 -- start start 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.917+0000 7f002754d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00200fd050 0x7f0020065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.917+0000 7f002754d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002010fde0 con 0x7f00200fd050 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.917+0000 7f00252e9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00200fd050 0x7f0020065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.918+0000 7f00252e9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00200fd050 0x7f0020065bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54166/0 (socket says 172.21.15.6:54166) 2024-03-22T06:06:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.918+0000 7f00252e9700 1 -- 172.21.15.6:0/2013379134 learned_addr learned my addr 172.21.15.6:0/2013379134 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.918+0000 7f00252e9700 1 -- 172.21.15.6:0/2013379134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c005040 con 0x7f00200fd050 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.918+0000 7f00252e9700 1 --2- 172.21.15.6:0/2013379134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00200fd050 0x7f0020065bd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f001c0096b0 tx=0x7f001c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.918+0000 7f0015ffb700 1 -- 172.21.15.6:0/2013379134 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f001c00bed0 con 0x7f00200fd050 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.918+0000 7f0015ffb700 1 -- 172.21.15.6:0/2013379134 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f001c00b5c0 con 0x7f00200fd050 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.918+0000 7f002754d700 1 -- 172.21.15.6:0/2013379134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002010ff20 con 0x7f00200fd050 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.918+0000 7f0015ffb700 1 -- 172.21.15.6:0/2013379134 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f001c01c5b0 con 0x7f00200fd050 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.918+0000 7f002754d700 1 -- 172.21.15.6:0/2013379134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0020110410 con 0x7f00200fd050 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.919+0000 7f0015ffb700 1 -- 172.21.15.6:0/2013379134 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f001c01c820 con 0x7f00200fd050 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.919+0000 7f0015ffb700 1 --2- 172.21.15.6:0/2013379134 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f000c036880 0x7f000c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.919+0000 7f0015ffb700 1 -- 172.21.15.6:0/2013379134 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f001c049a40 con 0x7f00200fd050 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.919+0000 7f002754d700 1 -- 172.21.15.6:0/2013379134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0020105920 con 0x7f00200fd050 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.919+0000 7f0024ae8700 1 --2- 172.21.15.6:0/2013379134 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f000c036880 0x7f000c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.920+0000 7f0024ae8700 1 --2- 172.21.15.6:0/2013379134 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f000c036880 0x7f000c038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0010009990 tx=0x7f0010006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:36.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:36.924+0000 7f0015ffb700 1 -- 172.21.15.6:0/2013379134 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f001c023300 con 0x7f00200fd050 2024-03-22T06:06:37.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.133+0000 7f002754d700 1 -- 172.21.15.6:0/2013379134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f002004bf40 con 0x7f00200fd050 2024-03-22T06:06:37.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.137+0000 7f0015ffb700 1 -- 172.21.15.6:0/2013379134 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f001c011b60 con 0x7f00200fd050 2024-03-22T06:06:37.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.140+0000 7f002754d700 1 -- 172.21.15.6:0/2013379134 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f000c036880 msgr2=0x7f000c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:37.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.140+0000 7f002754d700 1 --2- 172.21.15.6:0/2013379134 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f000c036880 0x7f000c038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0010009990 tx=0x7f0010006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:37.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.140+0000 7f002754d700 1 -- 172.21.15.6:0/2013379134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00200fd050 msgr2=0x7f0020065bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:37.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.140+0000 7f002754d700 1 --2- 172.21.15.6:0/2013379134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00200fd050 0x7f0020065bd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f001c0096b0 tx=0x7f001c009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:37.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.140+0000 7f002754d700 1 -- 172.21.15.6:0/2013379134 shutdown_connections 2024-03-22T06:06:37.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.140+0000 7f002754d700 1 --2- 172.21.15.6:0/2013379134 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f000c036880 0x7f000c038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:37.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.140+0000 7f002754d700 1 --2- 172.21.15.6:0/2013379134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00200fd050 0x7f0020065bd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:37.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.140+0000 7f002754d700 1 -- 172.21.15.6:0/2013379134 >> 172.21.15.6:0/2013379134 conn(0x7f00200f70d0 msgr2=0x7f0020102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:37.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.140+0000 7f002754d700 1 -- 172.21.15.6:0/2013379134 shutdown_connections 2024-03-22T06:06:37.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:37.140+0000 7f002754d700 1 -- 172.21.15.6:0/2013379134 wait complete. 2024-03-22T06:06:37.758 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-22T06:06:37.758 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-22T06:06:38.278 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:38 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2013379134' entity='client.admin' 2024-03-22T06:06:38.394 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:06:39.452 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:39 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:39.452 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:39 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:39.452 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:39 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd/host:smithi006", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:06:39.452 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:39 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:39.452 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:39 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi006", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:06:39.452 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:39 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi006", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-22T06:06:39.452 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:39 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:06:39.452 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:39 smithi006 ceph-mon[29301]: Deploying daemon crash.smithi006 on smithi006 2024-03-22T06:06:40.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.984+0000 7ff8089b3700 1 -- 172.21.15.6:0/4147686965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8040fd050 msgr2=0x7ff8040fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:40.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.984+0000 7ff8089b3700 1 --2- 172.21.15.6:0/4147686965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8040fd050 0x7ff8040fd460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40048c0 tx=0x7ff7f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:40.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.984+0000 7ff8089b3700 1 -- 172.21.15.6:0/4147686965 shutdown_connections 2024-03-22T06:06:40.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.984+0000 7ff8089b3700 1 --2- 172.21.15.6:0/4147686965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8040fd050 0x7ff8040fd460 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:40.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.984+0000 7ff8089b3700 1 -- 172.21.15.6:0/4147686965 >> 172.21.15.6:0/4147686965 conn(0x7ff8040f70d0 msgr2=0x7ff8040f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:40.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.984+0000 7ff8089b3700 1 -- 172.21.15.6:0/4147686965 shutdown_connections 2024-03-22T06:06:40.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.984+0000 7ff8089b3700 1 -- 172.21.15.6:0/4147686965 wait complete. 2024-03-22T06:06:40.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.984+0000 7ff8089b3700 1 Processor -- start 2024-03-22T06:06:40.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.984+0000 7ff8089b3700 1 -- start start 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff8089b3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8040fd050 0x7ff8040f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff8089b3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040f95a0 con 0x7ff8040fd050 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff80259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8040fd050 0x7ff8040f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff80259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8040fd050 0x7ff8040f9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54178/0 (socket says 172.21.15.6:54178) 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff80259c700 1 -- 172.21.15.6:0/518200609 learned_addr learned my addr 172.21.15.6:0/518200609 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff80259c700 1 -- 172.21.15.6:0/518200609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f4005040 con 0x7ff8040fd050 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff80259c700 1 --2- 172.21.15.6:0/518200609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8040fd050 0x7ff8040f9060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40096b0 tx=0x7ff7f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff7f2ffd700 1 -- 172.21.15.6:0/518200609 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7f400bed0 con 0x7ff8040fd050 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff8089b3700 1 -- 172.21.15.6:0/518200609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8040f7670 con 0x7ff8040fd050 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff7f2ffd700 1 -- 172.21.15.6:0/518200609 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7f400b5c0 con 0x7ff8040fd050 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff7f2ffd700 1 -- 172.21.15.6:0/518200609 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7f401c5b0 con 0x7ff8040fd050 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.985+0000 7ff8089b3700 1 -- 172.21.15.6:0/518200609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8040f7b60 con 0x7ff8040fd050 2024-03-22T06:06:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.986+0000 7ff7f2ffd700 1 -- 172.21.15.6:0/518200609 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7ff7f401c820 con 0x7ff8040fd050 2024-03-22T06:06:40.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.986+0000 7ff7f2ffd700 1 --2- 172.21.15.6:0/518200609 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7ff7ec036880 0x7ff7ec038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:40.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.986+0000 7ff8089b3700 1 -- 172.21.15.6:0/518200609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80405f5f0 con 0x7ff8040fd050 2024-03-22T06:06:40.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.986+0000 7ff7f2ffd700 1 -- 172.21.15.6:0/518200609 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff7f4049a80 con 0x7ff8040fd050 2024-03-22T06:06:40.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.986+0000 7ff801d9b700 1 --2- 172.21.15.6:0/518200609 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7ff7ec036880 0x7ff7ec038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:40.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.987+0000 7ff801d9b700 1 --2- 172.21.15.6:0/518200609 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7ff7ec036880 0x7ff7ec038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009990 tx=0x7ff7f8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:40.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:40.991+0000 7ff7f2ffd700 1 -- 172.21.15.6:0/518200609 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff7f4023300 con 0x7ff8040fd050 2024-03-22T06:06:41.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.193+0000 7ff8089b3700 1 -- 172.21.15.6:0/518200609 --> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8040f7ee0 con 0x7ff7ec036880 2024-03-22T06:06:41.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.197+0000 7ff7f2ffd700 1 -- 172.21.15.6:0/518200609 <== mgr.14162 v2:172.21.15.6:6800/358732355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff8040f7ee0 con 0x7ff7ec036880 2024-03-22T06:06:41.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.198+0000 7ff8089b3700 1 -- 172.21.15.6:0/518200609 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7ff7ec036880 msgr2=0x7ff7ec038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:41.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.198+0000 7ff8089b3700 1 --2- 172.21.15.6:0/518200609 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7ff7ec036880 0x7ff7ec038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009990 tx=0x7ff7f8006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:41.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.198+0000 7ff8089b3700 1 -- 172.21.15.6:0/518200609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8040fd050 msgr2=0x7ff8040f9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:41.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.198+0000 7ff8089b3700 1 --2- 172.21.15.6:0/518200609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8040fd050 0x7ff8040f9060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40096b0 tx=0x7ff7f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:41.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.198+0000 7ff8089b3700 1 -- 172.21.15.6:0/518200609 shutdown_connections 2024-03-22T06:06:41.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.198+0000 7ff8089b3700 1 --2- 172.21.15.6:0/518200609 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7ff7ec036880 0x7ff7ec038d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:41.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.198+0000 7ff8089b3700 1 --2- 172.21.15.6:0/518200609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8040fd050 0x7ff8040f9060 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:41.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.198+0000 7ff8089b3700 1 -- 172.21.15.6:0/518200609 >> 172.21.15.6:0/518200609 conn(0x7ff8040f70d0 msgr2=0x7ff804065070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:41.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.199+0000 7ff8089b3700 1 -- 172.21.15.6:0/518200609 shutdown_connections 2024-03-22T06:06:41.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:41.199+0000 7ff8089b3700 1 -- 172.21.15.6:0/518200609 wait complete. 2024-03-22T06:06:41.867 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi117 2024-03-22T06:06:41.867 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:06:41.868 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.conf 2024-03-22T06:06:41.887 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:06:41.887 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:06:41.946 INFO:tasks.cephadm:Adding host smithi117 to orchestrator... 2024-03-22T06:06:41.946 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph orch host add smithi117 2024-03-22T06:06:42.215 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:06:42.513 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:42 smithi006 ceph-mon[29301]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:42.513 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:42 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:42.513 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:42 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:42.513 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:42 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:42.513 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:42 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:42.513 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:42 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:42.513 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:42 smithi006 ceph-mon[29301]: Deploying daemon node-exporter.smithi006 on smithi006 2024-03-22T06:06:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.676+0000 7fcb946de700 1 -- 172.21.15.6:0/3372975398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb8c071e20 msgr2=0x7fcb8c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.676+0000 7fcb946de700 1 --2- 172.21.15.6:0/3372975398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb8c071e20 0x7fcb8c070330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcb880048c0 tx=0x7fcb88009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.676+0000 7fcb946de700 1 -- 172.21.15.6:0/3372975398 shutdown_connections 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.676+0000 7fcb946de700 1 --2- 172.21.15.6:0/3372975398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb8c071e20 0x7fcb8c070330 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.676+0000 7fcb946de700 1 -- 172.21.15.6:0/3372975398 >> 172.21.15.6:0/3372975398 conn(0x7fcb8c0f81c0 msgr2=0x7fcb8c0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.676+0000 7fcb946de700 1 -- 172.21.15.6:0/3372975398 shutdown_connections 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.676+0000 7fcb946de700 1 -- 172.21.15.6:0/3372975398 wait complete. 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.677+0000 7fcb946de700 1 Processor -- start 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.677+0000 7fcb946de700 1 -- start start 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.677+0000 7fcb946de700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb8c071e20 0x7fcb8c10f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.677+0000 7fcb946de700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c10f7f0 con 0x7fcb8c071e20 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.677+0000 7fcb9247a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb8c071e20 0x7fcb8c10f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.677+0000 7fcb9247a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb8c071e20 0x7fcb8c10f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46738/0 (socket says 172.21.15.6:46738) 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.677+0000 7fcb9247a700 1 -- 172.21.15.6:0/2351267964 learned_addr learned my addr 172.21.15.6:0/2351267964 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.677+0000 7fcb9247a700 1 -- 172.21.15.6:0/2351267964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb88005040 con 0x7fcb8c071e20 2024-03-22T06:06:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.677+0000 7fcb9247a700 1 --2- 172.21.15.6:0/2351267964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb8c071e20 0x7fcb8c10f2b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800b900 tx=0x7fcb88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.678+0000 7fcb82ffd700 1 -- 172.21.15.6:0/2351267964 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb8800b5c0 con 0x7fcb8c071e20 2024-03-22T06:06:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.678+0000 7fcb82ffd700 1 -- 172.21.15.6:0/2351267964 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb8800be70 con 0x7fcb8c071e20 2024-03-22T06:06:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.678+0000 7fcb946de700 1 -- 172.21.15.6:0/2351267964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c10f9f0 con 0x7fcb8c071e20 2024-03-22T06:06:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.678+0000 7fcb82ffd700 1 -- 172.21.15.6:0/2351267964 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb8801c590 con 0x7fcb8c071e20 2024-03-22T06:06:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.678+0000 7fcb946de700 1 -- 172.21.15.6:0/2351267964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c110560 con 0x7fcb8c071e20 2024-03-22T06:06:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.679+0000 7fcb82ffd700 1 -- 172.21.15.6:0/2351267964 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fcb8801c850 con 0x7fcb8c071e20 2024-03-22T06:06:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.679+0000 7fcb82ffd700 1 --2- 172.21.15.6:0/2351267964 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcb78036880 0x7fcb78038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.679+0000 7fcb946de700 1 -- 172.21.15.6:0/2351267964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c105160 con 0x7fcb8c071e20 2024-03-22T06:06:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.679+0000 7fcb91c79700 1 --2- 172.21.15.6:0/2351267964 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcb78036880 0x7fcb78038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.679+0000 7fcb82ffd700 1 -- 172.21.15.6:0/2351267964 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcb88049390 con 0x7fcb8c071e20 2024-03-22T06:06:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.679+0000 7fcb91c79700 1 --2- 172.21.15.6:0/2351267964 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcb78036880 0x7fcb78038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009990 tx=0x7fcb7c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:43.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.684+0000 7fcb82ffd700 1 -- 172.21.15.6:0/2351267964 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcb88011960 con 0x7fcb8c071e20 2024-03-22T06:06:43.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:43.904+0000 7fcb946de700 1 -- 172.21.15.6:0/2351267964 --> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb8c1107a0 con 0x7fcb78036880 2024-03-22T06:06:44.422 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:44 smithi006 ceph-mon[29301]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:46.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:46.051+0000 7fcb82ffd700 1 -- 172.21.15.6:0/2351267964 <== mgr.14162 v2:172.21.15.6:6800/358732355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fcb8c1107a0 con 0x7fcb78036880 2024-03-22T06:06:46.053 INFO:teuthology.orchestra.run.smithi006.stdout:Added host 'smithi117' with addr '172.21.15.117' 2024-03-22T06:06:46.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:46.054+0000 7fcb946de700 1 -- 172.21.15.6:0/2351267964 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcb78036880 msgr2=0x7fcb78038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:46.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:46.054+0000 7fcb946de700 1 --2- 172.21.15.6:0/2351267964 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcb78036880 0x7fcb78038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009990 tx=0x7fcb7c006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:46.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:46.054+0000 7fcb946de700 1 -- 172.21.15.6:0/2351267964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb8c071e20 msgr2=0x7fcb8c10f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:46.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:46.054+0000 7fcb946de700 1 --2- 172.21.15.6:0/2351267964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb8c071e20 0x7fcb8c10f2b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800b900 tx=0x7fcb88009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:46.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:46.054+0000 7fcb946de700 1 -- 172.21.15.6:0/2351267964 shutdown_connections 2024-03-22T06:06:46.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:46.054+0000 7fcb946de700 1 --2- 172.21.15.6:0/2351267964 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fcb78036880 0x7fcb78038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:46.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:46.054+0000 7fcb946de700 1 --2- 172.21.15.6:0/2351267964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb8c071e20 0x7fcb8c10f2b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:46.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:46.054+0000 7fcb946de700 1 -- 172.21.15.6:0/2351267964 >> 172.21.15.6:0/2351267964 conn(0x7fcb8c0f81c0 msgr2=0x7fcb8c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:46.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:46.054+0000 7fcb946de700 1 -- 172.21.15.6:0/2351267964 shutdown_connections 2024-03-22T06:06:46.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:46.054+0000 7fcb946de700 1 -- 172.21.15.6:0/2351267964 wait complete. 2024-03-22T06:06:46.226 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:45 smithi006 ceph-mon[29301]: Deploying cephadm binary to smithi117 2024-03-22T06:06:46.226 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:45 smithi006 ceph-mon[29301]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:06:46.226 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:45 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:46.957 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-22T06:06:47.203 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:47 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:47.203 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:47 smithi006 ceph-mon[29301]: Added host smithi117 2024-03-22T06:06:47.431 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:06:48.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:48 smithi006 ceph-mon[29301]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:06:48.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:48 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:48.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:48 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:48.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:48 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:48.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:48 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:48.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.989+0000 7f3dfd976700 1 -- 172.21.15.6:0/1485663047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3df8065d50 msgr2=0x7f3df80ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:48.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.989+0000 7f3dfd976700 1 --2- 172.21.15.6:0/1485663047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3df8065d50 0x7f3df80ff5f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009c60 tx=0x7f3de8004d10 comp rx=0 tx=0).stop 2024-03-22T06:06:48.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.989+0000 7f3dfd976700 1 -- 172.21.15.6:0/1485663047 shutdown_connections 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.989+0000 7f3dfd976700 1 --2- 172.21.15.6:0/1485663047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3df8065d50 0x7f3df80ff5f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.989+0000 7f3dfd976700 1 -- 172.21.15.6:0/1485663047 >> 172.21.15.6:0/1485663047 conn(0x7f3df80f7060 msgr2=0x7f3df80f9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.989+0000 7f3dfd976700 1 -- 172.21.15.6:0/1485663047 shutdown_connections 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.989+0000 7f3dfd976700 1 -- 172.21.15.6:0/1485663047 wait complete. 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3dfd976700 1 Processor -- start 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3dfd976700 1 -- start start 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3dfd976700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3df8065d50 0x7f3df810d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3dfd976700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df810d580 con 0x7f3df8065d50 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3df6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3df8065d50 0x7f3df810d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3df6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3df8065d50 0x7f3df810d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46758/0 (socket says 172.21.15.6:46758) 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3df6ffd700 1 -- 172.21.15.6:0/3320364179 learned_addr learned my addr 172.21.15.6:0/3320364179 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3df6ffd700 1 -- 172.21.15.6:0/3320364179 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8009940 con 0x7f3df8065d50 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3df6ffd700 1 --2- 172.21.15.6:0/3320364179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3df8065d50 0x7f3df810d040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3de80097b0 tx=0x7f3de8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:48.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3def7fe700 1 -- 172.21.15.6:0/3320364179 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3de8012410 con 0x7f3df8065d50 2024-03-22T06:06:48.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3def7fe700 1 -- 172.21.15.6:0/3320364179 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3de8012a40 con 0x7f3df8065d50 2024-03-22T06:06:48.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3dfd976700 1 -- 172.21.15.6:0/3320364179 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df810d780 con 0x7f3df8065d50 2024-03-22T06:06:48.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.990+0000 7f3def7fe700 1 -- 172.21.15.6:0/3320364179 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3de801b980 con 0x7f3df8065d50 2024-03-22T06:06:48.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.991+0000 7f3dfd976700 1 -- 172.21.15.6:0/3320364179 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df810e2f0 con 0x7f3df8065d50 2024-03-22T06:06:48.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.991+0000 7f3def7fe700 1 -- 172.21.15.6:0/3320364179 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f3de8019040 con 0x7f3df8065d50 2024-03-22T06:06:48.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.991+0000 7f3dfd976700 1 -- 172.21.15.6:0/3320364179 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df8102f00 con 0x7f3df8065d50 2024-03-22T06:06:48.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.992+0000 7f3def7fe700 1 --2- 172.21.15.6:0/3320364179 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3de4036880 0x7f3de4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:48.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.992+0000 7f3def7fe700 1 -- 172.21.15.6:0/3320364179 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3de8049c90 con 0x7f3df8065d50 2024-03-22T06:06:48.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.992+0000 7f3df67fc700 1 --2- 172.21.15.6:0/3320364179 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3de4036880 0x7f3de4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:48.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.992+0000 7f3df67fc700 1 --2- 172.21.15.6:0/3320364179 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3de4036880 0x7f3de4038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009990 tx=0x7f3de0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:48.996+0000 7f3def7fe700 1 -- 172.21.15.6:0/3320364179 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3de8022b50 con 0x7f3df8065d50 2024-03-22T06:06:49.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.210+0000 7f3dfd976700 1 -- 172.21.15.6:0/3320364179 --> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3df805f690 con 0x7f3de4036880 2024-03-22T06:06:49.210 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:49 smithi006 ceph-mon[29301]: Deploying daemon alertmanager.smithi006 on smithi006 2024-03-22T06:06:49.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.211+0000 7f3def7fe700 1 -- 172.21.15.6:0/3320364179 <== mgr.14162 v2:172.21.15.6:6800/358732355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f3df805f690 con 0x7f3de4036880 2024-03-22T06:06:49.211 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:06:49.212 INFO:teuthology.orchestra.run.smithi006.stdout:[{"addr": "172.21.15.6", "hostname": "smithi006", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}] 2024-03-22T06:06:49.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.213+0000 7f3dfd976700 1 -- 172.21.15.6:0/3320364179 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3de4036880 msgr2=0x7f3de4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:49.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.213+0000 7f3dfd976700 1 --2- 172.21.15.6:0/3320364179 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3de4036880 0x7f3de4038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009990 tx=0x7f3de0006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:49.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.213+0000 7f3dfd976700 1 -- 172.21.15.6:0/3320364179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3df8065d50 msgr2=0x7f3df810d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:49.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.213+0000 7f3dfd976700 1 --2- 172.21.15.6:0/3320364179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3df8065d50 0x7f3df810d040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3de80097b0 tx=0x7f3de8004d10 comp rx=0 tx=0).stop 2024-03-22T06:06:49.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.213+0000 7f3dfd976700 1 -- 172.21.15.6:0/3320364179 shutdown_connections 2024-03-22T06:06:49.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.213+0000 7f3dfd976700 1 --2- 172.21.15.6:0/3320364179 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f3de4036880 0x7f3de4038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:49.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.213+0000 7f3dfd976700 1 --2- 172.21.15.6:0/3320364179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3df8065d50 0x7f3df810d040 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:49.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.213+0000 7f3dfd976700 1 -- 172.21.15.6:0/3320364179 >> 172.21.15.6:0/3320364179 conn(0x7f3df80f7060 msgr2=0x7f3df8103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:49.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.213+0000 7f3dfd976700 1 -- 172.21.15.6:0/3320364179 shutdown_connections 2024-03-22T06:06:49.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:49.213+0000 7f3dfd976700 1 -- 172.21.15.6:0/3320364179 wait complete. 2024-03-22T06:06:50.171 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-22T06:06:50.171 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-22T06:06:50.440 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:06:50.484 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:50 smithi006 ceph-mon[29301]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:06:50.484 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:50 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:51.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:51 smithi006 ceph-mon[29301]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T06:06:52.494 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:52 smithi006 ceph-mon[29301]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:06:53.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.678+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/118437893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab00fc9e0 msgr2=0x7f9ab00fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.678+0000 7f9ab5ad1700 1 --2- 172.21.15.6:0/118437893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab00fc9e0 0x7f9ab00fcdf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00048c0 tx=0x7f9aa0009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.678+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/118437893 shutdown_connections 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.678+0000 7f9ab5ad1700 1 --2- 172.21.15.6:0/118437893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab00fc9e0 0x7f9ab00fcdf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.678+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/118437893 >> 172.21.15.6:0/118437893 conn(0x7f9ab00f8160 msgr2=0x7f9ab00fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.678+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/118437893 shutdown_connections 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.678+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/118437893 wait complete. 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.679+0000 7f9ab5ad1700 1 Processor -- start 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.679+0000 7f9ab5ad1700 1 -- start start 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.679+0000 7f9ab5ad1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab00fc9e0 0x7f9ab0071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.679+0000 7f9ab5ad1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab00722a0 con 0x7f9ab00fc9e0 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.679+0000 7f9aaf7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab00fc9e0 0x7f9ab0071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.679+0000 7f9aaf7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab00fc9e0 0x7f9ab0071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46596/0 (socket says 172.21.15.6:46596) 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.679+0000 7f9aaf7fe700 1 -- 172.21.15.6:0/1850969146 learned_addr learned my addr 172.21.15.6:0/1850969146 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.680+0000 7f9aaf7fe700 1 -- 172.21.15.6:0/1850969146 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa0005040 con 0x7f9ab00fc9e0 2024-03-22T06:06:53.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.680+0000 7f9aaf7fe700 1 --2- 172.21.15.6:0/1850969146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab00fc9e0 0x7f9ab0071d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009680 tx=0x7f9aa0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.680+0000 7f9a97fff700 1 -- 172.21.15.6:0/1850969146 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9aa000b5c0 con 0x7f9ab00fc9e0 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.680+0000 7f9a97fff700 1 -- 172.21.15.6:0/1850969146 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9aa000be70 con 0x7f9ab00fc9e0 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.680+0000 7f9a97fff700 1 -- 172.21.15.6:0/1850969146 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9aa00135a0 con 0x7f9ab00fc9e0 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.680+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/1850969146 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab0070370 con 0x7f9ab00fc9e0 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.680+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/1850969146 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab0070860 con 0x7f9ab00fc9e0 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.681+0000 7f9a97fff700 1 -- 172.21.15.6:0/1850969146 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f9aa0013860 con 0x7f9ab00fc9e0 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.681+0000 7f9a97fff700 1 --2- 172.21.15.6:0/1850969146 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f9a98036880 0x7f9a98038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.681+0000 7f9a97fff700 1 -- 172.21.15.6:0/1850969146 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9aa0049770 con 0x7f9ab00fc9e0 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.681+0000 7f9aaeffd700 1 --2- 172.21.15.6:0/1850969146 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f9a98036880 0x7f9a98038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.681+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/1850969146 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab005f5f0 con 0x7f9ab00fc9e0 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.681+0000 7f9aaeffd700 1 --2- 172.21.15.6:0/1850969146 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f9a98036880 0x7f9a98038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009990 tx=0x7f9aa4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:53.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.686+0000 7f9a97fff700 1 -- 172.21.15.6:0/1850969146 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9aa0021020 con 0x7f9ab00fc9e0 2024-03-22T06:06:53.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:53.885+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/1850969146 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f9ab01063c0 con 0x7f9ab00fc9e0 2024-03-22T06:06:54.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:54.075+0000 7f9a97fff700 1 -- 172.21.15.6:0/1850969146 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9aa0013bb0 con 0x7f9ab00fc9e0 2024-03-22T06:06:54.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:54.077+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/1850969146 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f9a98036880 msgr2=0x7f9a98038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:54.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:54.077+0000 7f9ab5ad1700 1 --2- 172.21.15.6:0/1850969146 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f9a98036880 0x7f9a98038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009990 tx=0x7f9aa4006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:54.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:54.077+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/1850969146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab00fc9e0 msgr2=0x7f9ab0071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:54.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:54.077+0000 7f9ab5ad1700 1 --2- 172.21.15.6:0/1850969146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab00fc9e0 0x7f9ab0071d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009680 tx=0x7f9aa0009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:54.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:54.077+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/1850969146 shutdown_connections 2024-03-22T06:06:54.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:54.077+0000 7f9ab5ad1700 1 --2- 172.21.15.6:0/1850969146 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f9a98036880 0x7f9a98038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:54.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:54.077+0000 7f9ab5ad1700 1 --2- 172.21.15.6:0/1850969146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab00fc9e0 0x7f9ab0071d60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:54.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:54.077+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/1850969146 >> 172.21.15.6:0/1850969146 conn(0x7f9ab00f8160 msgr2=0x7f9ab0103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:54.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:54.078+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/1850969146 shutdown_connections 2024-03-22T06:06:54.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:06:54.078+0000 7f9ab5ad1700 1 -- 172.21.15.6:0/1850969146 wait complete. 2024-03-22T06:06:54.079 INFO:teuthology.orchestra.run.smithi006.stderr:adjusted tunables profile to default 2024-03-22T06:06:54.210 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:54 smithi006 ceph-mon[29301]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:06:54.210 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:54 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1850969146' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-22T06:06:54.994 INFO:tasks.cephadm:Adding mon.smithi006 on smithi006 2024-03-22T06:06:54.994 INFO:tasks.cephadm:Adding mon.smithi117 on smithi117 2024-03-22T06:06:54.994 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph orch apply mon '2;smithi006:172.21.15.6=smithi006;smithi117:172.21.15.117=smithi117' 2024-03-22T06:06:55.239 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:06:55.240 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:55 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1850969146' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-22T06:06:55.241 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:55 smithi006 ceph-mon[29301]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T06:06:55.285 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:06:56.171 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:56 smithi006 ceph-mon[29301]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:06:57.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.019+0000 7f00f9c33700 1 -- 172.21.15.117:0/1475162740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f40faea0 msgr2=0x7f00f40fb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.019+0000 7f00f9c33700 1 --2- 172.21.15.117:0/1475162740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f40faea0 0x7f00f40fb2b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f00e40048c0 tx=0x7f00e4009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.019+0000 7f00f9c33700 1 -- 172.21.15.117:0/1475162740 shutdown_connections 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.019+0000 7f00f9c33700 1 --2- 172.21.15.117:0/1475162740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f40faea0 0x7f00f40fb2b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.019+0000 7f00f9c33700 1 -- 172.21.15.117:0/1475162740 >> 172.21.15.117:0/1475162740 conn(0x7f00f40f6840 msgr2=0x7f00f40f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.019+0000 7f00f9c33700 1 -- 172.21.15.117:0/1475162740 shutdown_connections 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.019+0000 7f00f9c33700 1 -- 172.21.15.117:0/1475162740 wait complete. 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.020+0000 7f00f9c33700 1 Processor -- start 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.020+0000 7f00f9c33700 1 -- start start 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.020+0000 7f00f9c33700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f40faea0 0x7f00f410d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.020+0000 7f00f9c33700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f410d600 con 0x7f00f40faea0 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.020+0000 7f00f37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f40faea0 0x7f00f410d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.021+0000 7f00f37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f40faea0 0x7f00f410d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:60720/0 (socket says 172.21.15.117:60720) 2024-03-22T06:06:57.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.021+0000 7f00f37fe700 1 -- 172.21.15.117:0/575954249 learned_addr learned my addr 172.21.15.117:0/575954249 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:06:57.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.021+0000 7f00f37fe700 1 -- 172.21.15.117:0/575954249 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e4005040 con 0x7f00f40faea0 2024-03-22T06:06:57.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.021+0000 7f00f37fe700 1 --2- 172.21.15.117:0/575954249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f40faea0 0x7f00f410d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00e40096b0 tx=0x7f00e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:57.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.021+0000 7f00dbfff700 1 -- 172.21.15.117:0/575954249 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00e400bed0 con 0x7f00f40faea0 2024-03-22T06:06:57.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.021+0000 7f00f9c33700 1 -- 172.21.15.117:0/575954249 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f410d800 con 0x7f00f40faea0 2024-03-22T06:06:57.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.021+0000 7f00dbfff700 1 -- 172.21.15.117:0/575954249 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00e400b5c0 con 0x7f00f40faea0 2024-03-22T06:06:57.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.021+0000 7f00dbfff700 1 -- 172.21.15.117:0/575954249 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00e401c5b0 con 0x7f00f40faea0 2024-03-22T06:06:57.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.021+0000 7f00f9c33700 1 -- 172.21.15.117:0/575954249 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f410e370 con 0x7f00f40faea0 2024-03-22T06:06:57.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.022+0000 7f00dbfff700 1 -- 172.21.15.117:0/575954249 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f00e4013420 con 0x7f00f40faea0 2024-03-22T06:06:57.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.022+0000 7f00f9c33700 1 -- 172.21.15.117:0/575954249 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f4102f80 con 0x7f00f40faea0 2024-03-22T06:06:57.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.022+0000 7f00dbfff700 1 --2- 172.21.15.117:0/575954249 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00dc036880 0x7f00dc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:57.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.022+0000 7f00dbfff700 1 -- 172.21.15.117:0/575954249 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f00e4049a10 con 0x7f00f40faea0 2024-03-22T06:06:57.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.023+0000 7f00f2ffd700 1 --2- 172.21.15.117:0/575954249 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00dc036880 0x7f00dc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:57.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.023+0000 7f00f2ffd700 1 --2- 172.21.15.117:0/575954249 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00dc036880 0x7f00dc038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009990 tx=0x7f00e8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:57.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.027+0000 7f00dbfff700 1 -- 172.21.15.117:0/575954249 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f00e4025400 con 0x7f00f40faea0 2024-03-22T06:06:57.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.220+0000 7f00f9c33700 1 -- 172.21.15.117:0/575954249 --> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi006:172.21.15.6=smithi006;smithi117:172.21.15.117=smithi117", "target": ["mon-mgr", ""]}) v1 -- 0x7f00f410e670 con 0x7f00dc036880 2024-03-22T06:06:57.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.229+0000 7f00dbfff700 1 -- 172.21.15.117:0/575954249 <== mgr.14162 v2:172.21.15.6:6800/358732355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f00f410e670 con 0x7f00dc036880 2024-03-22T06:06:57.229 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled mon update... 2024-03-22T06:06:57.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.231+0000 7f00f9c33700 1 -- 172.21.15.117:0/575954249 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00dc036880 msgr2=0x7f00dc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:57.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.231+0000 7f00f9c33700 1 --2- 172.21.15.117:0/575954249 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00dc036880 0x7f00dc038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009990 tx=0x7f00e8006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:57.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.231+0000 7f00f9c33700 1 -- 172.21.15.117:0/575954249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f40faea0 msgr2=0x7f00f410d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:57.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.231+0000 7f00f9c33700 1 --2- 172.21.15.117:0/575954249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f40faea0 0x7f00f410d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00e40096b0 tx=0x7f00e4009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:57.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.231+0000 7f00f9c33700 1 -- 172.21.15.117:0/575954249 shutdown_connections 2024-03-22T06:06:57.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.231+0000 7f00f9c33700 1 --2- 172.21.15.117:0/575954249 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00dc036880 0x7f00dc038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:57.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.232+0000 7f00f9c33700 1 --2- 172.21.15.117:0/575954249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f40faea0 0x7f00f410d0c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:57.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.232+0000 7f00f9c33700 1 -- 172.21.15.117:0/575954249 >> 172.21.15.117:0/575954249 conn(0x7f00f40f6840 msgr2=0x7f00f4103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:57.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.232+0000 7f00f9c33700 1 -- 172.21.15.117:0/575954249 shutdown_connections 2024-03-22T06:06:57.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:57.232+0000 7f00f9c33700 1 -- 172.21.15.117:0/575954249 wait complete. 2024-03-22T06:06:57.791 DEBUG:teuthology.orchestra.run.smithi117:mon.smithi117> sudo journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi117.service 2024-03-22T06:06:57.794 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:06:57.794 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:06:58.068 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:06:58.115 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:06:58.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:58 smithi006 ceph-mon[29301]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:06:58.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:58 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:59 smithi006 ceph-mon[29301]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi006:172.21.15.6=smithi006;smithi117:172.21.15.117=smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:06:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:59 smithi006 ceph-mon[29301]: Saving service mon spec with placement smithi006:172.21.15.6=smithi006;smithi117:172.21.15.117=smithi117;count:2 2024-03-22T06:06:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:59 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:59 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:59 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:59 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:59.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:59 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:59.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:59 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:59.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:59 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-22T06:06:59.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:06:59 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:06:59.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.604+0000 7f46f7159700 1 -- 172.21.15.117:0/3495660277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46f00fe0b0 msgr2=0x7f46f00fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:59.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.604+0000 7f46f7159700 1 --2- 172.21.15.117:0/3495660277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46f00fe0b0 0x7f46f00fe4c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f46e40048c0 tx=0x7f46e4009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:59.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.604+0000 7f46f7159700 1 -- 172.21.15.117:0/3495660277 shutdown_connections 2024-03-22T06:06:59.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.604+0000 7f46f7159700 1 --2- 172.21.15.117:0/3495660277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46f00fe0b0 0x7f46f00fe4c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:59.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.604+0000 7f46f7159700 1 -- 172.21.15.117:0/3495660277 >> 172.21.15.117:0/3495660277 conn(0x7f46f00f6850 msgr2=0x7f46f00f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:59.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.604+0000 7f46f7159700 1 -- 172.21.15.117:0/3495660277 shutdown_connections 2024-03-22T06:06:59.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.604+0000 7f46f7159700 1 -- 172.21.15.117:0/3495660277 wait complete. 2024-03-22T06:06:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.605+0000 7f46f7159700 1 Processor -- start 2024-03-22T06:06:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.605+0000 7f46f7159700 1 -- start start 2024-03-22T06:06:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.605+0000 7f46f7159700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46f00fe0b0 0x7f46f00fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.605+0000 7f46f7159700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f00fdd00 con 0x7f46f00fe0b0 2024-03-22T06:06:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.605+0000 7f46f4ef5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46f00fe0b0 0x7f46f00fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.605+0000 7f46f4ef5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46f00fe0b0 0x7f46f00fd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:60736/0 (socket says 172.21.15.117:60736) 2024-03-22T06:06:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.605+0000 7f46f4ef5700 1 -- 172.21.15.117:0/3247306233 learned_addr learned my addr 172.21.15.117:0/3247306233 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:06:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.605+0000 7f46f4ef5700 1 -- 172.21.15.117:0/3247306233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4005040 con 0x7f46f00fe0b0 2024-03-22T06:06:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.606+0000 7f46f4ef5700 1 --2- 172.21.15.117:0/3247306233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46f00fe0b0 0x7f46f00fd7c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f46e40096b0 tx=0x7f46e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.606+0000 7f46ed7fa700 1 -- 172.21.15.117:0/3247306233 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46e400bed0 con 0x7f46f00fe0b0 2024-03-22T06:06:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.606+0000 7f46f7159700 1 -- 172.21.15.117:0/3247306233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f00fbdd0 con 0x7f46f00fe0b0 2024-03-22T06:06:59.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.606+0000 7f46ed7fa700 1 -- 172.21.15.117:0/3247306233 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46e400b5c0 con 0x7f46f00fe0b0 2024-03-22T06:06:59.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.606+0000 7f46ed7fa700 1 -- 172.21.15.117:0/3247306233 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46e401c5b0 con 0x7f46f00fe0b0 2024-03-22T06:06:59.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.606+0000 7f46f7159700 1 -- 172.21.15.117:0/3247306233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f00fc2c0 con 0x7f46f00fe0b0 2024-03-22T06:06:59.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.607+0000 7f46f7159700 1 -- 172.21.15.117:0/3247306233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f005f5f0 con 0x7f46f00fe0b0 2024-03-22T06:06:59.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.607+0000 7f46ed7fa700 1 -- 172.21.15.117:0/3247306233 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f46e4013a80 con 0x7f46f00fe0b0 2024-03-22T06:06:59.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.607+0000 7f46ed7fa700 1 --2- 172.21.15.117:0/3247306233 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f46e0036880 0x7f46e0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:06:59.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.607+0000 7f46ed7fa700 1 -- 172.21.15.117:0/3247306233 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f46e40499f0 con 0x7f46f00fe0b0 2024-03-22T06:06:59.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.607+0000 7f46effff700 1 --2- 172.21.15.117:0/3247306233 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f46e0036880 0x7f46e0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:06:59.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.608+0000 7f46effff700 1 --2- 172.21.15.117:0/3247306233 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f46e0036880 0x7f46e0038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009990 tx=0x7f46dc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:06:59.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.612+0000 7f46ed7fa700 1 -- 172.21.15.117:0/3247306233 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f46e4011960 con 0x7f46f00fe0b0 2024-03-22T06:06:59.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.875+0000 7f46f7159700 1 -- 172.21.15.117:0/3247306233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f46f0106450 con 0x7f46f00fe0b0 2024-03-22T06:06:59.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.875+0000 7f46ed7fa700 1 -- 172.21.15.117:0/3247306233 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f46e4011370 con 0x7f46f00fe0b0 2024-03-22T06:06:59.875 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:06:59.876 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:06:59.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.877+0000 7f46f7159700 1 -- 172.21.15.117:0/3247306233 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f46e0036880 msgr2=0x7f46e0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:59.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.877+0000 7f46f7159700 1 --2- 172.21.15.117:0/3247306233 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f46e0036880 0x7f46e0038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009990 tx=0x7f46dc006e40 comp rx=0 tx=0).stop 2024-03-22T06:06:59.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.877+0000 7f46f7159700 1 -- 172.21.15.117:0/3247306233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46f00fe0b0 msgr2=0x7f46f00fd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:06:59.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.877+0000 7f46f7159700 1 --2- 172.21.15.117:0/3247306233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46f00fe0b0 0x7f46f00fd7c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f46e40096b0 tx=0x7f46e4009e30 comp rx=0 tx=0).stop 2024-03-22T06:06:59.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.877+0000 7f46f7159700 1 -- 172.21.15.117:0/3247306233 shutdown_connections 2024-03-22T06:06:59.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.877+0000 7f46f7159700 1 --2- 172.21.15.117:0/3247306233 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f46e0036880 0x7f46e0038d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:59.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.877+0000 7f46f7159700 1 --2- 172.21.15.117:0/3247306233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46f00fe0b0 0x7f46f00fd7c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:06:59.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.877+0000 7f46f7159700 1 -- 172.21.15.117:0/3247306233 >> 172.21.15.117:0/3247306233 conn(0x7f46f00f6850 msgr2=0x7f46f0103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:06:59.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.878+0000 7f46f7159700 1 -- 172.21.15.117:0/3247306233 shutdown_connections 2024-03-22T06:06:59.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:06:59.878+0000 7f46f7159700 1 -- 172.21.15.117:0/3247306233 wait complete. 2024-03-22T06:06:59.878 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:00 smithi006 ceph-mon[29301]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-22T06:07:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:00 smithi006 ceph-mon[29301]: Deploying daemon grafana.smithi006 on smithi006 2024-03-22T06:07:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:00 smithi006 ceph-mon[29301]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:00 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/3247306233' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:00.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:00 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:01.455 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:01.455 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:01.702 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:01.749 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:02.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:02 smithi006 ceph-mon[29301]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:03.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.076+0000 7f97539b3700 1 -- 172.21.15.117:0/1846836995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fe0a0 msgr2=0x7f974c0fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.076+0000 7f97539b3700 1 --2- 172.21.15.117:0/1846836995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fe0a0 0x7f974c0fe4b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f97480048c0 tx=0x7f9748009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.076+0000 7f97539b3700 1 -- 172.21.15.117:0/1846836995 shutdown_connections 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.076+0000 7f97539b3700 1 --2- 172.21.15.117:0/1846836995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fe0a0 0x7f974c0fe4b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.076+0000 7f97539b3700 1 -- 172.21.15.117:0/1846836995 >> 172.21.15.117:0/1846836995 conn(0x7f974c0f6840 msgr2=0x7f974c0f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.076+0000 7f97539b3700 1 -- 172.21.15.117:0/1846836995 shutdown_connections 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.076+0000 7f97539b3700 1 -- 172.21.15.117:0/1846836995 wait complete. 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.076+0000 7f97539b3700 1 Processor -- start 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.076+0000 7f97539b3700 1 -- start start 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.077+0000 7f97539b3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fe0a0 0x7f974c0fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.077+0000 7f97539b3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c0fdcf0 con 0x7f974c0fe0a0 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.077+0000 7f975174f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fe0a0 0x7f974c0fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.077+0000 7f975174f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fe0a0 0x7f974c0fd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:60746/0 (socket says 172.21.15.117:60746) 2024-03-22T06:07:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.077+0000 7f975174f700 1 -- 172.21.15.117:0/247068084 learned_addr learned my addr 172.21.15.117:0/247068084 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.077+0000 7f975174f700 1 -- 172.21.15.117:0/247068084 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9748005040 con 0x7f974c0fe0a0 2024-03-22T06:07:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.077+0000 7f975174f700 1 --2- 172.21.15.117:0/247068084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fe0a0 0x7f974c0fd7b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9748009680 tx=0x7f9748009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.077+0000 7f9741ffb700 1 -- 172.21.15.117:0/247068084 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f974800b5c0 con 0x7f974c0fe0a0 2024-03-22T06:07:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.077+0000 7f97539b3700 1 -- 172.21.15.117:0/247068084 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c0fbdc0 con 0x7f974c0fe0a0 2024-03-22T06:07:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.077+0000 7f9741ffb700 1 -- 172.21.15.117:0/247068084 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f974800be70 con 0x7f974c0fe0a0 2024-03-22T06:07:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.077+0000 7f9741ffb700 1 -- 172.21.15.117:0/247068084 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97480135a0 con 0x7f974c0fe0a0 2024-03-22T06:07:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.078+0000 7f97539b3700 1 -- 172.21.15.117:0/247068084 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c0fc2b0 con 0x7f974c0fe0a0 2024-03-22T06:07:03.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.078+0000 7f9741ffb700 1 -- 172.21.15.117:0/247068084 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f974801c430 con 0x7f974c0fe0a0 2024-03-22T06:07:03.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.078+0000 7f97539b3700 1 -- 172.21.15.117:0/247068084 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c05f5f0 con 0x7f974c0fe0a0 2024-03-22T06:07:03.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.079+0000 7f9741ffb700 1 --2- 172.21.15.117:0/247068084 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f97380368d0 0x7f9738038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:03.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.079+0000 7f9741ffb700 1 -- 172.21.15.117:0/247068084 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f97480493a0 con 0x7f974c0fe0a0 2024-03-22T06:07:03.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.079+0000 7f9750f4e700 1 --2- 172.21.15.117:0/247068084 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f97380368d0 0x7f9738038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:03.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.079+0000 7f9750f4e700 1 --2- 172.21.15.117:0/247068084 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f97380368d0 0x7f9738038d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f973c009990 tx=0x7f973c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:03.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.084+0000 7f9741ffb700 1 -- 172.21.15.117:0/247068084 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f97480117e0 con 0x7f974c0fe0a0 2024-03-22T06:07:03.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.342+0000 7f97539b3700 1 -- 172.21.15.117:0/247068084 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f974c106440 con 0x7f974c0fe0a0 2024-03-22T06:07:03.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.343+0000 7f9741ffb700 1 -- 172.21.15.117:0/247068084 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f9748021070 con 0x7f974c0fe0a0 2024-03-22T06:07:03.343 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:03.343 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:03.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.345+0000 7f97539b3700 1 -- 172.21.15.117:0/247068084 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f97380368d0 msgr2=0x7f9738038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:03.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.345+0000 7f97539b3700 1 --2- 172.21.15.117:0/247068084 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f97380368d0 0x7f9738038d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f973c009990 tx=0x7f973c006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:03.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.345+0000 7f97539b3700 1 -- 172.21.15.117:0/247068084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fe0a0 msgr2=0x7f974c0fd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:03.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.345+0000 7f97539b3700 1 --2- 172.21.15.117:0/247068084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fe0a0 0x7f974c0fd7b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9748009680 tx=0x7f9748009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:03.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.345+0000 7f97539b3700 1 -- 172.21.15.117:0/247068084 shutdown_connections 2024-03-22T06:07:03.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.345+0000 7f97539b3700 1 --2- 172.21.15.117:0/247068084 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f97380368d0 0x7f9738038d80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:03.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.345+0000 7f97539b3700 1 --2- 172.21.15.117:0/247068084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fe0a0 0x7f974c0fd7b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:03.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.345+0000 7f97539b3700 1 -- 172.21.15.117:0/247068084 >> 172.21.15.117:0/247068084 conn(0x7f974c0f6840 msgr2=0x7f974c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:03.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.345+0000 7f97539b3700 1 -- 172.21.15.117:0/247068084 shutdown_connections 2024-03-22T06:07:03.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:03.345+0000 7f97539b3700 1 -- 172.21.15.117:0/247068084 wait complete. 2024-03-22T06:07:03.346 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:04.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:04 smithi006 ceph-mon[29301]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:04.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:04 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/247068084' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:04.910 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:04.910 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:05.157 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:05.205 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:06.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:06 smithi006 ceph-mon[29301]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.747+0000 7fa958e96700 1 -- 172.21.15.117:0/2931928582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9540fd050 msgr2=0x7fa9540fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.747+0000 7fa958e96700 1 --2- 172.21.15.117:0/2931928582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9540fd050 0x7fa9540fd460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0048c0 tx=0x7fa93c009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.747+0000 7fa958e96700 1 -- 172.21.15.117:0/2931928582 shutdown_connections 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.747+0000 7fa958e96700 1 --2- 172.21.15.117:0/2931928582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9540fd050 0x7fa9540fd460 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.747+0000 7fa958e96700 1 -- 172.21.15.117:0/2931928582 >> 172.21.15.117:0/2931928582 conn(0x7fa9540f70d0 msgr2=0x7fa9540f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.747+0000 7fa958e96700 1 -- 172.21.15.117:0/2931928582 shutdown_connections 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.747+0000 7fa958e96700 1 -- 172.21.15.117:0/2931928582 wait complete. 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.748+0000 7fa958e96700 1 Processor -- start 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.748+0000 7fa958e96700 1 -- start start 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.748+0000 7fa958e96700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9540fd050 0x7fa95410d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.748+0000 7fa958e96700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95410d620 con 0x7fa9540fd050 2024-03-22T06:07:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.748+0000 7fa95259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9540fd050 0x7fa95410d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.748+0000 7fa95259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9540fd050 0x7fa95410d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:33250/0 (socket says 172.21.15.117:33250) 2024-03-22T06:07:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.748+0000 7fa95259c700 1 -- 172.21.15.117:0/3314165041 learned_addr learned my addr 172.21.15.117:0/3314165041 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.748+0000 7fa95259c700 1 -- 172.21.15.117:0/3314165041 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c005040 con 0x7fa9540fd050 2024-03-22T06:07:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.749+0000 7fa95259c700 1 --2- 172.21.15.117:0/3314165041 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9540fd050 0x7fa95410d0e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0096b0 tx=0x7fa93c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.749+0000 7fa94affd700 1 -- 172.21.15.117:0/3314165041 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa93c00bed0 con 0x7fa9540fd050 2024-03-22T06:07:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.749+0000 7fa958e96700 1 -- 172.21.15.117:0/3314165041 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95410d820 con 0x7fa9540fd050 2024-03-22T06:07:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.749+0000 7fa94affd700 1 -- 172.21.15.117:0/3314165041 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa93c00b5c0 con 0x7fa9540fd050 2024-03-22T06:07:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.749+0000 7fa94affd700 1 -- 172.21.15.117:0/3314165041 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa93c01c5b0 con 0x7fa9540fd050 2024-03-22T06:07:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.749+0000 7fa958e96700 1 -- 172.21.15.117:0/3314165041 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95410e390 con 0x7fa9540fd050 2024-03-22T06:07:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.750+0000 7fa958e96700 1 -- 172.21.15.117:0/3314165041 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa954101e70 con 0x7fa9540fd050 2024-03-22T06:07:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.750+0000 7fa94affd700 1 -- 172.21.15.117:0/3314165041 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fa93c004d30 con 0x7fa9540fd050 2024-03-22T06:07:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.750+0000 7fa94affd700 1 --2- 172.21.15.117:0/3314165041 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fa940036880 0x7fa940038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.750+0000 7fa94affd700 1 -- 172.21.15.117:0/3314165041 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa93c049a10 con 0x7fa9540fd050 2024-03-22T06:07:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.750+0000 7fa951d9b700 1 --2- 172.21.15.117:0/3314165041 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fa940036880 0x7fa940038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.750+0000 7fa951d9b700 1 --2- 172.21.15.117:0/3314165041 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fa940036880 0x7fa940038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa944009990 tx=0x7fa944006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:06.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:06.759+0000 7fa94affd700 1 -- 172.21.15.117:0/3314165041 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa93c011960 con 0x7fa9540fd050 2024-03-22T06:07:07.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.022+0000 7fa958e96700 1 -- 172.21.15.117:0/3314165041 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa95410e690 con 0x7fa9540fd050 2024-03-22T06:07:07.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.022+0000 7fa94affd700 1 -- 172.21.15.117:0/3314165041 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fa93c011370 con 0x7fa9540fd050 2024-03-22T06:07:07.022 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:07.023 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:07.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.024+0000 7fa958e96700 1 -- 172.21.15.117:0/3314165041 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fa940036880 msgr2=0x7fa940038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:07.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.024+0000 7fa958e96700 1 --2- 172.21.15.117:0/3314165041 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fa940036880 0x7fa940038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa944009990 tx=0x7fa944006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:07.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.024+0000 7fa958e96700 1 -- 172.21.15.117:0/3314165041 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9540fd050 msgr2=0x7fa95410d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:07.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.024+0000 7fa958e96700 1 --2- 172.21.15.117:0/3314165041 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9540fd050 0x7fa95410d0e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0096b0 tx=0x7fa93c009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:07.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.024+0000 7fa958e96700 1 -- 172.21.15.117:0/3314165041 shutdown_connections 2024-03-22T06:07:07.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.024+0000 7fa958e96700 1 --2- 172.21.15.117:0/3314165041 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fa940036880 0x7fa940038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:07.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.024+0000 7fa958e96700 1 --2- 172.21.15.117:0/3314165041 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9540fd050 0x7fa95410d0e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:07.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.024+0000 7fa958e96700 1 -- 172.21.15.117:0/3314165041 >> 172.21.15.117:0/3314165041 conn(0x7fa9540f70d0 msgr2=0x7fa954065070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:07.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.024+0000 7fa958e96700 1 -- 172.21.15.117:0/3314165041 shutdown_connections 2024-03-22T06:07:07.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:07.024+0000 7fa958e96700 1 -- 172.21.15.117:0/3314165041 wait complete. 2024-03-22T06:07:07.025 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:07.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:07 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/3314165041' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:08.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:08 smithi006 ceph-mon[29301]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:08.599 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:08.599 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:08.848 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:08.895 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:10.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.315+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/3012311958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8cec0fd650 msgr2=0x7f8cec0ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:10.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.315+0000 7f8cf3fe4700 1 --2- 172.21.15.117:0/3012311958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8cec0fd650 0x7f8cec0ffa30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40048c0 tx=0x7f8ce4009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:10.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.315+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/3012311958 shutdown_connections 2024-03-22T06:07:10.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.315+0000 7f8cf3fe4700 1 --2- 172.21.15.117:0/3012311958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8cec0fd650 0x7f8cec0ffa30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:10.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.315+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/3012311958 >> 172.21.15.117:0/3012311958 conn(0x7f8cec0f70d0 msgr2=0x7f8cec0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:10.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.315+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/3012311958 shutdown_connections 2024-03-22T06:07:10.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.315+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/3012311958 wait complete. 2024-03-22T06:07:10.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.316+0000 7f8cf3fe4700 1 Processor -- start 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.316+0000 7f8cf3fe4700 1 -- start start 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.316+0000 7f8cf3fe4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8cec0fd650 0x7f8cec10d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.316+0000 7f8cf3fe4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec10d5c0 con 0x7f8cec0fd650 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.316+0000 7f8cf1d80700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8cec0fd650 0x7f8cec10d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.316+0000 7f8cf1d80700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8cec0fd650 0x7f8cec10d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:33274/0 (socket says 172.21.15.117:33274) 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.316+0000 7f8cf1d80700 1 -- 172.21.15.117:0/2716315248 learned_addr learned my addr 172.21.15.117:0/2716315248 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.317+0000 7f8cf1d80700 1 -- 172.21.15.117:0/2716315248 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce4005040 con 0x7f8cec0fd650 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.317+0000 7f8cf1d80700 1 --2- 172.21.15.117:0/2716315248 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8cec0fd650 0x7f8cec10d080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009680 tx=0x7f8ce4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.317+0000 7f8cea7fc700 1 -- 172.21.15.117:0/2716315248 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ce400bed0 con 0x7f8cec0fd650 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.317+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/2716315248 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cec10d7c0 con 0x7f8cec0fd650 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.317+0000 7f8cea7fc700 1 -- 172.21.15.117:0/2716315248 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ce400b5c0 con 0x7f8cec0fd650 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.317+0000 7f8cea7fc700 1 -- 172.21.15.117:0/2716315248 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ce401c5b0 con 0x7f8cec0fd650 2024-03-22T06:07:10.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.317+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/2716315248 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cec10e330 con 0x7f8cec0fd650 2024-03-22T06:07:10.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.318+0000 7f8cea7fc700 1 -- 172.21.15.117:0/2716315248 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f8ce401c710 con 0x7f8cec0fd650 2024-03-22T06:07:10.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.318+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/2716315248 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cec102f60 con 0x7f8cec0fd650 2024-03-22T06:07:10.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.318+0000 7f8cea7fc700 1 --2- 172.21.15.117:0/2716315248 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f8cd4036880 0x7f8cd4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:10.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.318+0000 7f8cea7fc700 1 -- 172.21.15.117:0/2716315248 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8ce40499f0 con 0x7f8cec0fd650 2024-03-22T06:07:10.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.318+0000 7f8cf157f700 1 --2- 172.21.15.117:0/2716315248 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f8cd4036880 0x7f8cd4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:10.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.319+0000 7f8cf157f700 1 --2- 172.21.15.117:0/2716315248 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f8cd4036880 0x7f8cd4038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009990 tx=0x7f8ce0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:10.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.323+0000 7f8cea7fc700 1 -- 172.21.15.117:0/2716315248 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8ce4025400 con 0x7f8cec0fd650 2024-03-22T06:07:10.399 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:10 smithi006 ceph-mon[29301]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:10.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.585+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/2716315248 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8cec10ed40 con 0x7f8cec0fd650 2024-03-22T06:07:10.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.585+0000 7f8cea7fc700 1 -- 172.21.15.117:0/2716315248 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f8ce4011b60 con 0x7f8cec0fd650 2024-03-22T06:07:10.586 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:10.586 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:10.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.587+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/2716315248 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f8cd4036880 msgr2=0x7f8cd4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:10.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.587+0000 7f8cf3fe4700 1 --2- 172.21.15.117:0/2716315248 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f8cd4036880 0x7f8cd4038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009990 tx=0x7f8ce0006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:10.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.587+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/2716315248 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8cec0fd650 msgr2=0x7f8cec10d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:10.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.587+0000 7f8cf3fe4700 1 --2- 172.21.15.117:0/2716315248 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8cec0fd650 0x7f8cec10d080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009680 tx=0x7f8ce4009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:10.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.587+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/2716315248 shutdown_connections 2024-03-22T06:07:10.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.587+0000 7f8cf3fe4700 1 --2- 172.21.15.117:0/2716315248 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f8cd4036880 0x7f8cd4038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:10.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.587+0000 7f8cf3fe4700 1 --2- 172.21.15.117:0/2716315248 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8cec0fd650 0x7f8cec10d080 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:10.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.587+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/2716315248 >> 172.21.15.117:0/2716315248 conn(0x7f8cec0f70d0 msgr2=0x7f8cec103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:10.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.587+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/2716315248 shutdown_connections 2024-03-22T06:07:10.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:10.587+0000 7f8cf3fe4700 1 -- 172.21.15.117:0/2716315248 wait complete. 2024-03-22T06:07:10.588 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:11.434 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:11 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/2716315248' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:12.190 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:12.190 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:12.370 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:12 smithi006 ceph-mon[29301]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:12.435 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:12.481 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.779+0000 7fca7806b700 1 -- 172.21.15.117:0/3559171051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca700fe0b0 msgr2=0x7fca700fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.779+0000 7fca7806b700 1 --2- 172.21.15.117:0/3559171051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca700fe0b0 0x7fca700fe4c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009c60 tx=0x7fca6c004d10 comp rx=0 tx=0).stop 2024-03-22T06:07:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.779+0000 7fca7806b700 1 -- 172.21.15.117:0/3559171051 shutdown_connections 2024-03-22T06:07:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.779+0000 7fca7806b700 1 --2- 172.21.15.117:0/3559171051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca700fe0b0 0x7fca700fe4c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.779+0000 7fca7806b700 1 -- 172.21.15.117:0/3559171051 >> 172.21.15.117:0/3559171051 conn(0x7fca700f6850 msgr2=0x7fca700f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.779+0000 7fca7806b700 1 -- 172.21.15.117:0/3559171051 shutdown_connections 2024-03-22T06:07:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.779+0000 7fca7806b700 1 -- 172.21.15.117:0/3559171051 wait complete. 2024-03-22T06:07:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.780+0000 7fca7806b700 1 Processor -- start 2024-03-22T06:07:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.780+0000 7fca7806b700 1 -- start start 2024-03-22T06:07:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.780+0000 7fca7806b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca700fe0b0 0x7fca700fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.780+0000 7fca7806b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca700fdd00 con 0x7fca700fe0b0 2024-03-22T06:07:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.780+0000 7fca75e07700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca700fe0b0 0x7fca700fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.780+0000 7fca75e07700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca700fe0b0 0x7fca700fd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:46992/0 (socket says 172.21.15.117:46992) 2024-03-22T06:07:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.780+0000 7fca75e07700 1 -- 172.21.15.117:0/487219790 learned_addr learned my addr 172.21.15.117:0/487219790 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:13.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.780+0000 7fca75e07700 1 -- 172.21.15.117:0/487219790 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c009940 con 0x7fca700fe0b0 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.781+0000 7fca75e07700 1 --2- 172.21.15.117:0/487219790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca700fe0b0 0x7fca700fd7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fca6c004e60 tx=0x7fca6c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.781+0000 7fca667fc700 1 -- 172.21.15.117:0/487219790 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca6c012410 con 0x7fca700fe0b0 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.781+0000 7fca667fc700 1 -- 172.21.15.117:0/487219790 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca6c012a40 con 0x7fca700fe0b0 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.781+0000 7fca7806b700 1 -- 172.21.15.117:0/487219790 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca700fbdd0 con 0x7fca700fe0b0 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.781+0000 7fca667fc700 1 -- 172.21.15.117:0/487219790 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca6c01ba40 con 0x7fca700fe0b0 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.781+0000 7fca7806b700 1 -- 172.21.15.117:0/487219790 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca700fc2c0 con 0x7fca700fe0b0 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.782+0000 7fca7806b700 1 -- 172.21.15.117:0/487219790 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7005f5f0 con 0x7fca700fe0b0 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.782+0000 7fca667fc700 1 -- 172.21.15.117:0/487219790 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fca6c019040 con 0x7fca700fe0b0 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.782+0000 7fca667fc700 1 --2- 172.21.15.117:0/487219790 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fca5c036880 0x7fca5c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.782+0000 7fca667fc700 1 -- 172.21.15.117:0/487219790 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fca6c049c40 con 0x7fca700fe0b0 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.782+0000 7fca75606700 1 --2- 172.21.15.117:0/487219790 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fca5c036880 0x7fca5c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.783+0000 7fca75606700 1 --2- 172.21.15.117:0/487219790 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fca5c036880 0x7fca5c038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fca60009990 tx=0x7fca60006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:13.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:13.787+0000 7fca667fc700 1 -- 172.21.15.117:0/487219790 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fca6c01b3f0 con 0x7fca700fe0b0 2024-03-22T06:07:14.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.049+0000 7fca7806b700 1 -- 172.21.15.117:0/487219790 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fca7004bed0 con 0x7fca700fe0b0 2024-03-22T06:07:14.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.049+0000 7fca667fc700 1 -- 172.21.15.117:0/487219790 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fca6c017090 con 0x7fca700fe0b0 2024-03-22T06:07:14.051 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:14.051 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:14.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.051+0000 7fca7806b700 1 -- 172.21.15.117:0/487219790 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fca5c036880 msgr2=0x7fca5c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:14.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.051+0000 7fca7806b700 1 --2- 172.21.15.117:0/487219790 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fca5c036880 0x7fca5c038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fca60009990 tx=0x7fca60006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:14.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.051+0000 7fca7806b700 1 -- 172.21.15.117:0/487219790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca700fe0b0 msgr2=0x7fca700fd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:14.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.051+0000 7fca7806b700 1 --2- 172.21.15.117:0/487219790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca700fe0b0 0x7fca700fd7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fca6c004e60 tx=0x7fca6c004d10 comp rx=0 tx=0).stop 2024-03-22T06:07:14.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.051+0000 7fca7806b700 1 -- 172.21.15.117:0/487219790 shutdown_connections 2024-03-22T06:07:14.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.051+0000 7fca7806b700 1 --2- 172.21.15.117:0/487219790 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fca5c036880 0x7fca5c038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:14.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.051+0000 7fca7806b700 1 --2- 172.21.15.117:0/487219790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca700fe0b0 0x7fca700fd7c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:14.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.051+0000 7fca7806b700 1 -- 172.21.15.117:0/487219790 >> 172.21.15.117:0/487219790 conn(0x7fca700f6850 msgr2=0x7fca70103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:14.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.051+0000 7fca7806b700 1 -- 172.21.15.117:0/487219790 shutdown_connections 2024-03-22T06:07:14.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:14.051+0000 7fca7806b700 1 -- 172.21.15.117:0/487219790 wait complete. 2024-03-22T06:07:14.053 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:14.149 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:14 smithi006 ceph-mon[29301]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:14.149 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:14 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:14.149 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:14 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:14.149 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:14 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:14.150 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:14 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:14.150 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:14 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:14.150 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:14 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:14.150 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:14 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/487219790' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:15.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:15 smithi006 ceph-mon[29301]: Deploying daemon prometheus.smithi006 on smithi006 2024-03-22T06:07:15.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:15 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:15.585 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:15.586 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:15.831 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:15.880 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:16.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:16 smithi006 ceph-mon[29301]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:17.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.197+0000 7fed42610700 1 -- 172.21.15.117:0/380970877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0fe0a0 msgr2=0x7fed3c0fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.197+0000 7fed42610700 1 --2- 172.21.15.117:0/380970877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0fe0a0 0x7fed3c0fe4b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0048c0 tx=0x7fed2c009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.197+0000 7fed42610700 1 -- 172.21.15.117:0/380970877 shutdown_connections 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.197+0000 7fed42610700 1 --2- 172.21.15.117:0/380970877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0fe0a0 0x7fed3c0fe4b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.197+0000 7fed42610700 1 -- 172.21.15.117:0/380970877 >> 172.21.15.117:0/380970877 conn(0x7fed3c0f6840 msgr2=0x7fed3c0f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.197+0000 7fed42610700 1 -- 172.21.15.117:0/380970877 shutdown_connections 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.197+0000 7fed42610700 1 -- 172.21.15.117:0/380970877 wait complete. 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.198+0000 7fed42610700 1 Processor -- start 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.198+0000 7fed42610700 1 -- start start 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.198+0000 7fed42610700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0fe0a0 0x7fed3c10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.198+0000 7fed42610700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c10d5f0 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.199+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0fe0a0 0x7fed3c10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.199+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0fe0a0 0x7fed3c10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:47020/0 (socket says 172.21.15.117:47020) 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.199+0000 7fed3bfff700 1 -- 172.21.15.117:0/944120109 learned_addr learned my addr 172.21.15.117:0/944120109 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.199+0000 7fed3bfff700 1 -- 172.21.15.117:0/944120109 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c005040 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.199+0000 7fed3bfff700 1 --2- 172.21.15.117:0/944120109 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0fe0a0 0x7fed3c10d0b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0096b0 tx=0x7fed2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.199+0000 7fed38ff9700 1 -- 172.21.15.117:0/944120109 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed2c00bed0 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.199+0000 7fed42610700 1 -- 172.21.15.117:0/944120109 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c10d7f0 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.199+0000 7fed38ff9700 1 -- 172.21.15.117:0/944120109 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed2c00b5c0 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.199+0000 7fed38ff9700 1 -- 172.21.15.117:0/944120109 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed2c01c5b0 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.199+0000 7fed42610700 1 -- 172.21.15.117:0/944120109 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c10e360 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.200+0000 7fed38ff9700 1 -- 172.21.15.117:0/944120109 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fed2c013a80 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.200+0000 7fed42610700 1 -- 172.21.15.117:0/944120109 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3c102f70 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.200+0000 7fed38ff9700 1 --2- 172.21.15.117:0/944120109 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fed24036880 0x7fed24038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.200+0000 7fed38ff9700 1 -- 172.21.15.117:0/944120109 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fed2c049ad0 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.200+0000 7fed3b7fe700 1 --2- 172.21.15.117:0/944120109 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fed24036880 0x7fed24038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:17.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.201+0000 7fed3b7fe700 1 --2- 172.21.15.117:0/944120109 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fed24036880 0x7fed24038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed30009990 tx=0x7fed30006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:17.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.205+0000 7fed38ff9700 1 -- 172.21.15.117:0/944120109 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fed2c011940 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.466+0000 7fed42610700 1 -- 172.21.15.117:0/944120109 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fed3c10ed70 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.467+0000 7fed38ff9700 1 -- 172.21.15.117:0/944120109 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fed2c011370 con 0x7fed3c0fe0a0 2024-03-22T06:07:17.468 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:17.468 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:17.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.468+0000 7fed42610700 1 -- 172.21.15.117:0/944120109 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fed24036880 msgr2=0x7fed24038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:17.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.468+0000 7fed42610700 1 --2- 172.21.15.117:0/944120109 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fed24036880 0x7fed24038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed30009990 tx=0x7fed30006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:17.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.468+0000 7fed42610700 1 -- 172.21.15.117:0/944120109 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0fe0a0 msgr2=0x7fed3c10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:17.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.468+0000 7fed42610700 1 --2- 172.21.15.117:0/944120109 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0fe0a0 0x7fed3c10d0b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0096b0 tx=0x7fed2c009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:17.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.469+0000 7fed42610700 1 -- 172.21.15.117:0/944120109 shutdown_connections 2024-03-22T06:07:17.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.469+0000 7fed42610700 1 --2- 172.21.15.117:0/944120109 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fed24036880 0x7fed24038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:17.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.469+0000 7fed42610700 1 --2- 172.21.15.117:0/944120109 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0fe0a0 0x7fed3c10d0b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:17.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.469+0000 7fed42610700 1 -- 172.21.15.117:0/944120109 >> 172.21.15.117:0/944120109 conn(0x7fed3c0f6840 msgr2=0x7fed3c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:17.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.469+0000 7fed42610700 1 -- 172.21.15.117:0/944120109 shutdown_connections 2024-03-22T06:07:17.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:17.469+0000 7fed42610700 1 -- 172.21.15.117:0/944120109 wait complete. 2024-03-22T06:07:17.470 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:18.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:18 smithi006 ceph-mon[29301]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:18.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:18 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/944120109' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:19.029 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:19.029 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:19.279 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:19.326 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:20.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:20 smithi006 ceph-mon[29301]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:20.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.808+0000 7fadfdb08700 1 -- 172.21.15.117:0/3687347663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fadf8071ea0 msgr2=0x7fadf80703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:20.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.808+0000 7fadfdb08700 1 --2- 172.21.15.117:0/3687347663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fadf8071ea0 0x7fadf80703b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fade80048c0 tx=0x7fade8009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:20.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.808+0000 7fadfdb08700 1 -- 172.21.15.117:0/3687347663 shutdown_connections 2024-03-22T06:07:20.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.808+0000 7fadfdb08700 1 --2- 172.21.15.117:0/3687347663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fadf8071ea0 0x7fadf80703b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:20.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.808+0000 7fadfdb08700 1 -- 172.21.15.117:0/3687347663 >> 172.21.15.117:0/3687347663 conn(0x7fadf80f81e0 msgr2=0x7fadf80fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:20.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.808+0000 7fadfdb08700 1 -- 172.21.15.117:0/3687347663 shutdown_connections 2024-03-22T06:07:20.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.808+0000 7fadfdb08700 1 -- 172.21.15.117:0/3687347663 wait complete. 2024-03-22T06:07:20.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.808+0000 7fadfdb08700 1 Processor -- start 2024-03-22T06:07:20.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.809+0000 7fadfdb08700 1 -- start start 2024-03-22T06:07:20.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.809+0000 7fadfdb08700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fadf8071ea0 0x7fadf810f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:20.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.809+0000 7fadfdb08700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf810f850 con 0x7fadf8071ea0 2024-03-22T06:07:20.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.809+0000 7fadf77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fadf8071ea0 0x7fadf810f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:20.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.809+0000 7fadf77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fadf8071ea0 0x7fadf810f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:47030/0 (socket says 172.21.15.117:47030) 2024-03-22T06:07:20.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.809+0000 7fadf77fe700 1 -- 172.21.15.117:0/1314556285 learned_addr learned my addr 172.21.15.117:0/1314556285 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:20.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.809+0000 7fadf77fe700 1 -- 172.21.15.117:0/1314556285 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade8005040 con 0x7fadf8071ea0 2024-03-22T06:07:20.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.810+0000 7fadf77fe700 1 --2- 172.21.15.117:0/1314556285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fadf8071ea0 0x7fadf810f310 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fade800b900 tx=0x7fade8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:20.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.810+0000 7faddffff700 1 -- 172.21.15.117:0/1314556285 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fade800b5c0 con 0x7fadf8071ea0 2024-03-22T06:07:20.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.810+0000 7fadfdb08700 1 -- 172.21.15.117:0/1314556285 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf810fa50 con 0x7fadf8071ea0 2024-03-22T06:07:20.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.810+0000 7faddffff700 1 -- 172.21.15.117:0/1314556285 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fade800be70 con 0x7fadf8071ea0 2024-03-22T06:07:20.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.810+0000 7faddffff700 1 -- 172.21.15.117:0/1314556285 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fade801c590 con 0x7fadf8071ea0 2024-03-22T06:07:20.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.810+0000 7fadfdb08700 1 -- 172.21.15.117:0/1314556285 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf81105c0 con 0x7fadf8071ea0 2024-03-22T06:07:20.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.811+0000 7fadfdb08700 1 -- 172.21.15.117:0/1314556285 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf81051a0 con 0x7fadf8071ea0 2024-03-22T06:07:20.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.811+0000 7faddffff700 1 -- 172.21.15.117:0/1314556285 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fade8004d30 con 0x7fadf8071ea0 2024-03-22T06:07:20.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.811+0000 7faddffff700 1 --2- 172.21.15.117:0/1314556285 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fade0036880 0x7fade0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:20.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.811+0000 7faddffff700 1 -- 172.21.15.117:0/1314556285 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fade8049370 con 0x7fadf8071ea0 2024-03-22T06:07:20.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.811+0000 7fadf6ffd700 1 --2- 172.21.15.117:0/1314556285 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fade0036880 0x7fade0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:20.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.812+0000 7fadf6ffd700 1 --2- 172.21.15.117:0/1314556285 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fade0036880 0x7fade0038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fadec009990 tx=0x7fadec006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:20.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:20.816+0000 7faddffff700 1 -- 172.21.15.117:0/1314556285 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fade801b350 con 0x7fadf8071ea0 2024-03-22T06:07:21.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.076+0000 7fadfdb08700 1 -- 172.21.15.117:0/1314556285 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fadf805f5f0 con 0x7fadf8071ea0 2024-03-22T06:07:21.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.077+0000 7faddffff700 1 -- 172.21.15.117:0/1314556285 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fade8021020 con 0x7fadf8071ea0 2024-03-22T06:07:21.078 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:21.078 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:21.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.079+0000 7fadfdb08700 1 -- 172.21.15.117:0/1314556285 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fade0036880 msgr2=0x7fade0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:21.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.079+0000 7fadfdb08700 1 --2- 172.21.15.117:0/1314556285 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fade0036880 0x7fade0038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fadec009990 tx=0x7fadec006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:21.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.079+0000 7fadfdb08700 1 -- 172.21.15.117:0/1314556285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fadf8071ea0 msgr2=0x7fadf810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:21.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.079+0000 7fadfdb08700 1 --2- 172.21.15.117:0/1314556285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fadf8071ea0 0x7fadf810f310 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fade800b900 tx=0x7fade8009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:21.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.079+0000 7fadfdb08700 1 -- 172.21.15.117:0/1314556285 shutdown_connections 2024-03-22T06:07:21.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.079+0000 7fadfdb08700 1 --2- 172.21.15.117:0/1314556285 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fade0036880 0x7fade0038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:21.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.079+0000 7fadfdb08700 1 --2- 172.21.15.117:0/1314556285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fadf8071ea0 0x7fadf810f310 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:21.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.079+0000 7fadfdb08700 1 -- 172.21.15.117:0/1314556285 >> 172.21.15.117:0/1314556285 conn(0x7fadf80f81e0 msgr2=0x7fadf8105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:21.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.079+0000 7fadfdb08700 1 -- 172.21.15.117:0/1314556285 shutdown_connections 2024-03-22T06:07:21.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:21.079+0000 7fadfdb08700 1 -- 172.21.15.117:0/1314556285 wait complete. 2024-03-22T06:07:21.081 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:21.253 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:21 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/1314556285' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:22.410 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:22 smithi006 ceph-mon[29301]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:22.574 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:22.574 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:22.822 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:24.304 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:24 smithi006 ceph-mon[29301]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:24.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.319+0000 7fd1cf74b700 1 -- 172.21.15.117:0/3190810560 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1c80fb8e0 msgr2=0x7fd1c80fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:24.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.319+0000 7fd1cf74b700 1 --2- 172.21.15.117:0/3190810560 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1c80fb8e0 0x7fd1c80fbcf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40048c0 tx=0x7fd1c4009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:24.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.319+0000 7fd1cf74b700 1 -- 172.21.15.117:0/3190810560 shutdown_connections 2024-03-22T06:07:24.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.319+0000 7fd1cf74b700 1 --2- 172.21.15.117:0/3190810560 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1c80fb8e0 0x7fd1c80fbcf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:24.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.319+0000 7fd1cf74b700 1 -- 172.21.15.117:0/3190810560 >> 172.21.15.117:0/3190810560 conn(0x7fd1c80f7040 msgr2=0x7fd1c80f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:24.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.319+0000 7fd1cf74b700 1 -- 172.21.15.117:0/3190810560 shutdown_connections 2024-03-22T06:07:24.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.319+0000 7fd1cf74b700 1 -- 172.21.15.117:0/3190810560 wait complete. 2024-03-22T06:07:24.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.320+0000 7fd1cf74b700 1 Processor -- start 2024-03-22T06:07:24.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.320+0000 7fd1cf74b700 1 -- start start 2024-03-22T06:07:24.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.320+0000 7fd1cf74b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1c80fb8e0 0x7fd1c8066460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.320+0000 7fd1cf74b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c810fd30 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.320+0000 7fd1cd4e7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1c80fb8e0 0x7fd1c8066460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.320+0000 7fd1cd4e7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1c80fb8e0 0x7fd1c8066460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:40870/0 (socket says 172.21.15.117:40870) 2024-03-22T06:07:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.320+0000 7fd1cd4e7700 1 -- 172.21.15.117:0/4263395934 learned_addr learned my addr 172.21.15.117:0/4263395934 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.320+0000 7fd1cd4e7700 1 -- 172.21.15.117:0/4263395934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c4005040 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.321+0000 7fd1cd4e7700 1 --2- 172.21.15.117:0/4263395934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1c80fb8e0 0x7fd1c8066460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40096b0 tx=0x7fd1c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.321+0000 7fd1bdffb700 1 -- 172.21.15.117:0/4263395934 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1c400bed0 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.321+0000 7fd1bdffb700 1 -- 172.21.15.117:0/4263395934 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1c400b5c0 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.321+0000 7fd1cf74b700 1 -- 172.21.15.117:0/4263395934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c810fe70 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.321+0000 7fd1bdffb700 1 -- 172.21.15.117:0/4263395934 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1c401c5b0 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.321+0000 7fd1cf74b700 1 -- 172.21.15.117:0/4263395934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c8110360 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.322+0000 7fd1bdffb700 1 -- 172.21.15.117:0/4263395934 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fd1c4013420 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.322+0000 7fd1cf74b700 1 -- 172.21.15.117:0/4263395934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c81058a0 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.322+0000 7fd1bdffb700 1 --2- 172.21.15.117:0/4263395934 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fd1b4036880 0x7fd1b4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:24.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.322+0000 7fd1bdffb700 1 -- 172.21.15.117:0/4263395934 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd1c4049a40 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.322+0000 7fd1ccce6700 1 --2- 172.21.15.117:0/4263395934 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fd1b4036880 0x7fd1b4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:24.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.323+0000 7fd1ccce6700 1 --2- 172.21.15.117:0/4263395934 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fd1b4036880 0x7fd1b4038d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009990 tx=0x7fd1b8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:24.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.327+0000 7fd1bdffb700 1 -- 172.21.15.117:0/4263395934 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd1c4025400 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.584+0000 7fd1cf74b700 1 -- 172.21.15.117:0/4263395934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd1c805f5f0 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.585+0000 7fd1bdffb700 1 -- 172.21.15.117:0/4263395934 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fd1c4004d30 con 0x7fd1c80fb8e0 2024-03-22T06:07:24.586 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:24.586 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:24.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.587+0000 7fd1cf74b700 1 -- 172.21.15.117:0/4263395934 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fd1b4036880 msgr2=0x7fd1b4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:24.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.587+0000 7fd1cf74b700 1 --2- 172.21.15.117:0/4263395934 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fd1b4036880 0x7fd1b4038d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009990 tx=0x7fd1b8006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:24.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.587+0000 7fd1cf74b700 1 -- 172.21.15.117:0/4263395934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1c80fb8e0 msgr2=0x7fd1c8066460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:24.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.587+0000 7fd1cf74b700 1 --2- 172.21.15.117:0/4263395934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1c80fb8e0 0x7fd1c8066460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40096b0 tx=0x7fd1c4009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:24.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.587+0000 7fd1cf74b700 1 -- 172.21.15.117:0/4263395934 shutdown_connections 2024-03-22T06:07:24.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.587+0000 7fd1cf74b700 1 --2- 172.21.15.117:0/4263395934 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fd1b4036880 0x7fd1b4038d30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:24.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.587+0000 7fd1cf74b700 1 --2- 172.21.15.117:0/4263395934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1c80fb8e0 0x7fd1c8066460 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:24.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.587+0000 7fd1cf74b700 1 -- 172.21.15.117:0/4263395934 >> 172.21.15.117:0/4263395934 conn(0x7fd1c80f7040 msgr2=0x7fd1c8102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:24.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.587+0000 7fd1cf74b700 1 -- 172.21.15.117:0/4263395934 shutdown_connections 2024-03-22T06:07:24.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:24.587+0000 7fd1cf74b700 1 -- 172.21.15.117:0/4263395934 wait complete. 2024-03-22T06:07:24.589 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:25.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:25 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:25.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:25 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:25.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:25 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:25.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:25 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-22T06:07:25.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:25 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/4263395934' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:25.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:25 smithi006 ceph-mon[29301]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:25.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:25 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:26.203 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:26.204 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:26.452 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:26.499 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:26.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:26 smithi006 ceph-mon[29301]: from='mgr.14162 172.21.15.6:0/158535407' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-22T06:07:26.682 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:26 smithi006 ceph-mon[29301]: mgrmap e14: smithi006.sfpxdy(active, since 60s) 2024-03-22T06:07:27.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.939+0000 7fdc78ca2700 1 -- 172.21.15.117:0/3287520614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc740fb960 msgr2=0x7fdc740fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:27.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.939+0000 7fdc78ca2700 1 --2- 172.21.15.117:0/3287520614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc740fb960 0x7fdc740fbd70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdc680048c0 tx=0x7fdc68009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:27.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.939+0000 7fdc78ca2700 1 -- 172.21.15.117:0/3287520614 shutdown_connections 2024-03-22T06:07:27.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.939+0000 7fdc78ca2700 1 --2- 172.21.15.117:0/3287520614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc740fb960 0x7fdc740fbd70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:27.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.939+0000 7fdc78ca2700 1 -- 172.21.15.117:0/3287520614 >> 172.21.15.117:0/3287520614 conn(0x7fdc740f70c0 msgr2=0x7fdc740f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:27.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.939+0000 7fdc78ca2700 1 -- 172.21.15.117:0/3287520614 shutdown_connections 2024-03-22T06:07:27.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.939+0000 7fdc78ca2700 1 -- 172.21.15.117:0/3287520614 wait complete. 2024-03-22T06:07:27.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.940+0000 7fdc78ca2700 1 Processor -- start 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.940+0000 7fdc78ca2700 1 -- start start 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.940+0000 7fdc78ca2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc740fb960 0x7fdc7410bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.940+0000 7fdc78ca2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7410c4d0 con 0x7fdc740fb960 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.940+0000 7fdc7259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc740fb960 0x7fdc7410bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.940+0000 7fdc7259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc740fb960 0x7fdc7410bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:40880/0 (socket says 172.21.15.117:40880) 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.940+0000 7fdc7259c700 1 -- 172.21.15.117:0/447171038 learned_addr learned my addr 172.21.15.117:0/447171038 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.941+0000 7fdc7259c700 1 -- 172.21.15.117:0/447171038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc68005040 con 0x7fdc740fb960 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.941+0000 7fdc7259c700 1 --2- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc740fb960 0x7fdc7410bf90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc680096b0 tx=0x7fdc68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.941+0000 7fdc62ffd700 1 -- 172.21.15.117:0/447171038 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc6800bed0 con 0x7fdc740fb960 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.941+0000 7fdc78ca2700 1 -- 172.21.15.117:0/447171038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7410c6d0 con 0x7fdc740fb960 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.941+0000 7fdc62ffd700 1 -- 172.21.15.117:0/447171038 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc6800b5c0 con 0x7fdc740fb960 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.941+0000 7fdc62ffd700 1 -- 172.21.15.117:0/447171038 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc6801c5b0 con 0x7fdc740fb960 2024-03-22T06:07:27.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.941+0000 7fdc78ca2700 1 -- 172.21.15.117:0/447171038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7410d240 con 0x7fdc740fb960 2024-03-22T06:07:27.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.942+0000 7fdc62ffd700 1 -- 172.21.15.117:0/447171038 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 14) v1 ==== 43437+0+0 (secure 0 0 0) 0x7fdc6801c820 con 0x7fdc740fb960 2024-03-22T06:07:27.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.942+0000 7fdc78ca2700 1 -- 172.21.15.117:0/447171038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc74101e70 con 0x7fdc740fb960 2024-03-22T06:07:27.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.942+0000 7fdc62ffd700 1 --2- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fdc5c0368d0 0x7fdc5c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:27.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.942+0000 7fdc62ffd700 1 -- 172.21.15.117:0/447171038 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdc68049ba0 con 0x7fdc740fb960 2024-03-22T06:07:27.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.942+0000 7fdc71d9b700 1 -- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fdc5c0368d0 msgr2=0x7fdc5c038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/358732355 2024-03-22T06:07:27.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.942+0000 7fdc71d9b700 1 --2- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fdc5c0368d0 0x7fdc5c038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T06:07:27.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:27.947+0000 7fdc62ffd700 1 -- 172.21.15.117:0/447171038 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdc68025400 con 0x7fdc740fb960 2024-03-22T06:07:28.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.143+0000 7fdc71d9b700 1 -- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fdc5c0368d0 msgr2=0x7fdc5c038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/358732355 2024-03-22T06:07:28.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.143+0000 7fdc71d9b700 1 --2- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fdc5c0368d0 0x7fdc5c038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T06:07:28.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.204+0000 7fdc78ca2700 1 -- 172.21.15.117:0/447171038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdc7410dd10 con 0x7fdc740fb960 2024-03-22T06:07:28.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.205+0000 7fdc62ffd700 1 -- 172.21.15.117:0/447171038 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fdc68011370 con 0x7fdc740fb960 2024-03-22T06:07:28.206 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:28.206 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:28.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.206+0000 7fdc78ca2700 1 -- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fdc5c0368d0 msgr2=0x7fdc5c038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T06:07:28.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.206+0000 7fdc78ca2700 1 --2- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fdc5c0368d0 0x7fdc5c038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:28.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.206+0000 7fdc78ca2700 1 -- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc740fb960 msgr2=0x7fdc7410bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:28.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.206+0000 7fdc78ca2700 1 --2- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc740fb960 0x7fdc7410bf90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc680096b0 tx=0x7fdc68009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:28.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.207+0000 7fdc78ca2700 1 -- 172.21.15.117:0/447171038 shutdown_connections 2024-03-22T06:07:28.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.207+0000 7fdc78ca2700 1 --2- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7fdc5c0368d0 0x7fdc5c038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:28.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.207+0000 7fdc78ca2700 1 --2- 172.21.15.117:0/447171038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc740fb960 0x7fdc7410bf90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:28.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.207+0000 7fdc78ca2700 1 -- 172.21.15.117:0/447171038 >> 172.21.15.117:0/447171038 conn(0x7fdc740f70c0 msgr2=0x7fdc74102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:28.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.207+0000 7fdc78ca2700 1 -- 172.21.15.117:0/447171038 shutdown_connections 2024-03-22T06:07:28.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:28.207+0000 7fdc78ca2700 1 -- 172.21.15.117:0/447171038 wait complete. 2024-03-22T06:07:28.209 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:28.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:28 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/447171038' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:29.765 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:29.765 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:30.016 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:30.062 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:31.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.515+0000 7f010c6f0700 1 -- 172.21.15.117:0/2389110714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01040fb960 msgr2=0x7f01040fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:31.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.515+0000 7f010c6f0700 1 --2- 172.21.15.117:0/2389110714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01040fb960 0x7f01040fbd70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f00f40048c0 tx=0x7f00f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:31.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.515+0000 7f010c6f0700 1 -- 172.21.15.117:0/2389110714 shutdown_connections 2024-03-22T06:07:31.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.515+0000 7f010c6f0700 1 --2- 172.21.15.117:0/2389110714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01040fb960 0x7f01040fbd70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:31.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.515+0000 7f010c6f0700 1 -- 172.21.15.117:0/2389110714 >> 172.21.15.117:0/2389110714 conn(0x7f01040f70c0 msgr2=0x7f01040f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:31.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.515+0000 7f010c6f0700 1 -- 172.21.15.117:0/2389110714 shutdown_connections 2024-03-22T06:07:31.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.515+0000 7f010c6f0700 1 -- 172.21.15.117:0/2389110714 wait complete. 2024-03-22T06:07:31.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.516+0000 7f010c6f0700 1 Processor -- start 2024-03-22T06:07:31.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.516+0000 7f010c6f0700 1 -- start start 2024-03-22T06:07:31.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.516+0000 7f010c6f0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01040fb960 0x7f010410bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:31.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.516+0000 7f010c6f0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010410c4d0 con 0x7f01040fb960 2024-03-22T06:07:31.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.516+0000 7f010a48c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01040fb960 0x7f010410bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:31.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.516+0000 7f010a48c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01040fb960 0x7f010410bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:40890/0 (socket says 172.21.15.117:40890) 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.516+0000 7f010a48c700 1 -- 172.21.15.117:0/2744363744 learned_addr learned my addr 172.21.15.117:0/2744363744 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.517+0000 7f010a48c700 1 -- 172.21.15.117:0/2744363744 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4005040 con 0x7f01040fb960 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.517+0000 7f010a48c700 1 --2- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01040fb960 0x7f010410bf90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f00f40096b0 tx=0x7f00f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.517+0000 7f00faffd700 1 -- 172.21.15.117:0/2744363744 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00f400bed0 con 0x7f01040fb960 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.517+0000 7f00faffd700 1 -- 172.21.15.117:0/2744363744 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00f400b5c0 con 0x7f01040fb960 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.517+0000 7f010c6f0700 1 -- 172.21.15.117:0/2744363744 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010410c6d0 con 0x7f01040fb960 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.517+0000 7f00faffd700 1 -- 172.21.15.117:0/2744363744 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00f401c5b0 con 0x7f01040fb960 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.517+0000 7f010c6f0700 1 -- 172.21.15.117:0/2744363744 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010410d240 con 0x7f01040fb960 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.518+0000 7f00faffd700 1 -- 172.21.15.117:0/2744363744 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 14) v1 ==== 43437+0+0 (secure 0 0 0) 0x7f00f4013420 con 0x7f01040fb960 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.518+0000 7f010c6f0700 1 -- 172.21.15.117:0/2744363744 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0104101e70 con 0x7f01040fb960 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.518+0000 7f00faffd700 1 --2- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00f0036920 0x7f00f0038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.518+0000 7f00faffd700 1 -- 172.21.15.117:0/2744363744 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f00f4049b10 con 0x7f01040fb960 2024-03-22T06:07:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.518+0000 7f0109c8b700 1 -- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00f0036920 msgr2=0x7f00f0038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/358732355 2024-03-22T06:07:31.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.518+0000 7f0109c8b700 1 --2- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00f0036920 0x7f00f0038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T06:07:31.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.523+0000 7f00faffd700 1 -- 172.21.15.117:0/2744363744 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f00f4004d30 con 0x7f01040fb960 2024-03-22T06:07:31.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.719+0000 7f0109c8b700 1 -- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00f0036920 msgr2=0x7f00f0038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/358732355 2024-03-22T06:07:31.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.719+0000 7f0109c8b700 1 --2- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00f0036920 0x7f00f0038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T06:07:31.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.782+0000 7f010c6f0700 1 -- 172.21.15.117:0/2744363744 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f010410dc50 con 0x7f01040fb960 2024-03-22T06:07:31.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.782+0000 7f00faffd700 1 -- 172.21.15.117:0/2744363744 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f00f4011370 con 0x7f01040fb960 2024-03-22T06:07:31.783 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:31.783 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:31.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.784+0000 7f010c6f0700 1 -- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00f0036920 msgr2=0x7f00f0038dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T06:07:31.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.784+0000 7f010c6f0700 1 --2- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00f0036920 0x7f00f0038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:31.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.784+0000 7f010c6f0700 1 -- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01040fb960 msgr2=0x7f010410bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:31.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.784+0000 7f010c6f0700 1 --2- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01040fb960 0x7f010410bf90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f00f40096b0 tx=0x7f00f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:31.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.784+0000 7f010c6f0700 1 -- 172.21.15.117:0/2744363744 shutdown_connections 2024-03-22T06:07:31.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.784+0000 7f010c6f0700 1 --2- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:6800/358732355,v1:172.21.15.6:6801/358732355] conn(0x7f00f0036920 0x7f00f0038dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:31.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.784+0000 7f010c6f0700 1 --2- 172.21.15.117:0/2744363744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01040fb960 0x7f010410bf90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:31.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.784+0000 7f010c6f0700 1 -- 172.21.15.117:0/2744363744 >> 172.21.15.117:0/2744363744 conn(0x7f01040f70c0 msgr2=0x7f0104102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:31.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.784+0000 7f010c6f0700 1 -- 172.21.15.117:0/2744363744 shutdown_connections 2024-03-22T06:07:31.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:31.784+0000 7f010c6f0700 1 -- 172.21.15.117:0/2744363744 wait complete. 2024-03-22T06:07:31.786 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:32.094 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:31 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/2744363744' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:33.171 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: Active manager daemon smithi006.sfpxdy restarted 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: Activating manager daemon smithi006.sfpxdy 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: osdmap e5: 0 total, 0 up, 0 in 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: mgrmap e15: smithi006.sfpxdy(active, starting, since 0.00577164s) 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi006.sfpxdy", "id": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: Manager daemon smithi006.sfpxdy is now available 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:07:33.172 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/trash_purge_schedule"}]: dispatch 2024-03-22T06:07:33.173 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:33.173 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:33.448 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:33.449 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:33.693 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:33.739 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T06:07:34.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:34 smithi006 ceph-mon[29301]: [22/Mar/2024:06:07:32] ENGINE Bus STARTING 2024-03-22T06:07:34.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:34 smithi006 ceph-mon[29301]: [22/Mar/2024:06:07:32] ENGINE Serving on https://172.21.15.6:7150 2024-03-22T06:07:34.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:34 smithi006 ceph-mon[29301]: [22/Mar/2024:06:07:32] ENGINE Bus STARTED 2024-03-22T06:07:34.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:34 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:34.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:34 smithi006 ceph-mon[29301]: mgrmap e16: smithi006.sfpxdy(active, since 1.00733s) 2024-03-22T06:07:34.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:34 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:36.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.170+0000 7f3dc9f34700 1 -- 172.21.15.117:0/3896354079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dc40fb960 msgr2=0x7f3dc40fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:36.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.170+0000 7f3dc9f34700 1 --2- 172.21.15.117:0/3896354079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dc40fb960 0x7f3dc40fbd70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3db40048c0 tx=0x7f3db4009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:36.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.170+0000 7f3dc9f34700 1 -- 172.21.15.117:0/3896354079 shutdown_connections 2024-03-22T06:07:36.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.170+0000 7f3dc9f34700 1 --2- 172.21.15.117:0/3896354079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dc40fb960 0x7f3dc40fbd70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:36.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.170+0000 7f3dc9f34700 1 -- 172.21.15.117:0/3896354079 >> 172.21.15.117:0/3896354079 conn(0x7f3dc40f70c0 msgr2=0x7f3dc40f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.170+0000 7f3dc9f34700 1 -- 172.21.15.117:0/3896354079 shutdown_connections 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.170+0000 7f3dc9f34700 1 -- 172.21.15.117:0/3896354079 wait complete. 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.170+0000 7f3dc9f34700 1 Processor -- start 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.170+0000 7f3dc9f34700 1 -- start start 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.171+0000 7f3dc9f34700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dc40fb960 0x7f3dc4066860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.171+0000 7f3dc9f34700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc410fdb0 con 0x7f3dc40fb960 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.171+0000 7f3dc37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dc40fb960 0x7f3dc4066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.171+0000 7f3dc37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dc40fb960 0x7f3dc4066860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:57376/0 (socket says 172.21.15.117:57376) 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.171+0000 7f3dc37fe700 1 -- 172.21.15.117:0/2255212682 learned_addr learned my addr 172.21.15.117:0/2255212682 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.171+0000 7f3dc37fe700 1 -- 172.21.15.117:0/2255212682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db4005040 con 0x7f3dc40fb960 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.171+0000 7f3dc37fe700 1 --2- 172.21.15.117:0/2255212682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dc40fb960 0x7f3dc4066860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009680 tx=0x7f3db4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.172+0000 7f3dabfff700 1 -- 172.21.15.117:0/2255212682 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3db400b5c0 con 0x7f3dc40fb960 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.172+0000 7f3dabfff700 1 -- 172.21.15.117:0/2255212682 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3db400be70 con 0x7f3dc40fb960 2024-03-22T06:07:36.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.172+0000 7f3dc9f34700 1 -- 172.21.15.117:0/2255212682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc410fef0 con 0x7f3dc40fb960 2024-03-22T06:07:36.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.172+0000 7f3dabfff700 1 -- 172.21.15.117:0/2255212682 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3db40135a0 con 0x7f3dc40fb960 2024-03-22T06:07:36.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.172+0000 7f3dc9f34700 1 -- 172.21.15.117:0/2255212682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc41103e0 con 0x7f3dc40fb960 2024-03-22T06:07:36.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.172+0000 7f3dabfff700 1 -- 172.21.15.117:0/2255212682 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 43479+0+0 (secure 0 0 0) 0x7f3db4013860 con 0x7f3dc40fb960 2024-03-22T06:07:36.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.173+0000 7f3dabfff700 1 --2- 172.21.15.117:0/2255212682 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3dac036970 0x7f3dac038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:36.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.172+0000 7f3dc9f34700 1 -- 172.21.15.117:0/2255212682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc4105920 con 0x7f3dc40fb960 2024-03-22T06:07:36.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.173+0000 7f3dabfff700 1 -- 172.21.15.117:0/2255212682 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f3db40496e0 con 0x7f3dc40fb960 2024-03-22T06:07:36.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.173+0000 7f3dc2ffd700 1 --2- 172.21.15.117:0/2255212682 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3dac036970 0x7f3dac038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:36.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.173+0000 7f3dc2ffd700 1 --2- 172.21.15.117:0/2255212682 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3dac036970 0x7f3dac038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009990 tx=0x7f3db8006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:36.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.178+0000 7f3dabfff700 1 -- 172.21.15.117:0/2255212682 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3db4024b10 con 0x7f3dc40fb960 2024-03-22T06:07:36.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:36 smithi006 ceph-mon[29301]: mgrmap e17: smithi006.sfpxdy(active, since 2s) 2024-03-22T06:07:36.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.444+0000 7f3dc9f34700 1 -- 172.21.15.117:0/2255212682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3dc4101830 con 0x7f3dc40fb960 2024-03-22T06:07:36.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.445+0000 7f3dabfff700 1 -- 172.21.15.117:0/2255212682 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f3dc4101830 con 0x7f3dc40fb960 2024-03-22T06:07:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:36.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.447+0000 7f3dc9f34700 1 -- 172.21.15.117:0/2255212682 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3dac036970 msgr2=0x7f3dac038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:36.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.447+0000 7f3dc9f34700 1 --2- 172.21.15.117:0/2255212682 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3dac036970 0x7f3dac038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009990 tx=0x7f3db8006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:36.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.447+0000 7f3dc9f34700 1 -- 172.21.15.117:0/2255212682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dc40fb960 msgr2=0x7f3dc4066860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:36.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.447+0000 7f3dc9f34700 1 --2- 172.21.15.117:0/2255212682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dc40fb960 0x7f3dc4066860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009680 tx=0x7f3db4009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:36.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.447+0000 7f3dc9f34700 1 -- 172.21.15.117:0/2255212682 shutdown_connections 2024-03-22T06:07:36.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.447+0000 7f3dc9f34700 1 --2- 172.21.15.117:0/2255212682 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3dac036970 0x7f3dac038e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:36.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.447+0000 7f3dc9f34700 1 --2- 172.21.15.117:0/2255212682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dc40fb960 0x7f3dc4066860 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:36.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.447+0000 7f3dc9f34700 1 -- 172.21.15.117:0/2255212682 >> 172.21.15.117:0/2255212682 conn(0x7f3dc40f70c0 msgr2=0x7f3dc4102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:36.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.448+0000 7f3dc9f34700 1 -- 172.21.15.117:0/2255212682 shutdown_connections 2024-03-22T06:07:36.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:36.448+0000 7f3dc9f34700 1 -- 172.21.15.117:0/2255212682 wait complete. 2024-03-22T06:07:36.449 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:37.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:37 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:37.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:37 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:37.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:37 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/2255212682' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:37.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:37 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:37.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:37 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:37.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:37 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd/host:smithi006", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:07:38.147 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:38.147 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:38.394 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:07:39.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:39.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:39.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:39.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:39.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:07:39.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:07:39.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:07:39.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-22T06:07:39.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: Updating smithi117:/etc/ceph/ceph.conf 2024-03-22T06:07:39.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:07:39.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:07:39.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:07:39.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:38 smithi006 ceph-mon[29301]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:07:40.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:39 smithi006 ceph-mon[29301]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:07:40.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:39 smithi006 ceph-mon[29301]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:07:40.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:40.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:40.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:40.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:40.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:40.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:07:40.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-22T06:07:40.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:07:40.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:39 smithi006 ceph-mon[29301]: Deploying daemon crash.smithi117 on smithi117 2024-03-22T06:07:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.127+0000 7f0e99cd8700 1 -- 172.21.15.117:0/2064870470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e94102bd0 msgr2=0x7f0e94102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.127+0000 7f0e99cd8700 1 --2- 172.21.15.117:0/2064870470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e94102bd0 0x7f0e94102fa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0e840048c0 tx=0x7f0e84009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.127+0000 7f0e99cd8700 1 -- 172.21.15.117:0/2064870470 shutdown_connections 2024-03-22T06:07:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.127+0000 7f0e99cd8700 1 --2- 172.21.15.117:0/2064870470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e94102bd0 0x7f0e94102fa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.127+0000 7f0e99cd8700 1 -- 172.21.15.117:0/2064870470 >> 172.21.15.117:0/2064870470 conn(0x7f0e940f85e0 msgr2=0x7f0e940fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.127+0000 7f0e99cd8700 1 -- 172.21.15.117:0/2064870470 shutdown_connections 2024-03-22T06:07:41.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.127+0000 7f0e99cd8700 1 -- 172.21.15.117:0/2064870470 wait complete. 2024-03-22T06:07:41.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.127+0000 7f0e99cd8700 1 Processor -- start 2024-03-22T06:07:41.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.127+0000 7f0e99cd8700 1 -- start start 2024-03-22T06:07:41.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.128+0000 7f0e99cd8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e94102bd0 0x7f0e94115420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:41.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.128+0000 7f0e99cd8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94115960 con 0x7f0e94102bd0 2024-03-22T06:07:41.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.128+0000 7f0e937fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e94102bd0 0x7f0e94115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:41.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.128+0000 7f0e937fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e94102bd0 0x7f0e94115420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:57392/0 (socket says 172.21.15.117:57392) 2024-03-22T06:07:41.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.128+0000 7f0e937fe700 1 -- 172.21.15.117:0/470957068 learned_addr learned my addr 172.21.15.117:0/470957068 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:41.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.129+0000 7f0e937fe700 1 -- 172.21.15.117:0/470957068 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84005040 con 0x7f0e94102bd0 2024-03-22T06:07:41.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.129+0000 7f0e937fe700 1 --2- 172.21.15.117:0/470957068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e94102bd0 0x7f0e94115420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400b820 tx=0x7f0e84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:41.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.129+0000 7f0e7bfff700 1 -- 172.21.15.117:0/470957068 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e840093c0 con 0x7f0e94102bd0 2024-03-22T06:07:41.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.129+0000 7f0e99cd8700 1 -- 172.21.15.117:0/470957068 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e94115b60 con 0x7f0e94102bd0 2024-03-22T06:07:41.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.129+0000 7f0e99cd8700 1 -- 172.21.15.117:0/470957068 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e94116050 con 0x7f0e94102bd0 2024-03-22T06:07:41.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.129+0000 7f0e7bfff700 1 -- 172.21.15.117:0/470957068 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e84013420 con 0x7f0e94102bd0 2024-03-22T06:07:41.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.129+0000 7f0e7bfff700 1 -- 172.21.15.117:0/470957068 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e840255c0 con 0x7f0e94102bd0 2024-03-22T06:07:41.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.130+0000 7f0e99cd8700 1 -- 172.21.15.117:0/470957068 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9405f5f0 con 0x7f0e94102bd0 2024-03-22T06:07:41.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.130+0000 7f0e7bfff700 1 -- 172.21.15.117:0/470957068 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 43479+0+0 (secure 0 0 0) 0x7f0e8401c3f0 con 0x7f0e94102bd0 2024-03-22T06:07:41.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.131+0000 7f0e7bfff700 1 --2- 172.21.15.117:0/470957068 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e7c036970 0x7f0e7c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:41.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.131+0000 7f0e7bfff700 1 -- 172.21.15.117:0/470957068 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f0e84049de0 con 0x7f0e94102bd0 2024-03-22T06:07:41.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.131+0000 7f0e92ffd700 1 --2- 172.21.15.117:0/470957068 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e7c036970 0x7f0e7c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:41.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.131+0000 7f0e92ffd700 1 --2- 172.21.15.117:0/470957068 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e7c036970 0x7f0e7c038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009990 tx=0x7f0e88006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:41.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.136+0000 7f0e7bfff700 1 -- 172.21.15.117:0/470957068 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0e8401a8e0 con 0x7f0e94102bd0 2024-03-22T06:07:41.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.422+0000 7f0e99cd8700 1 -- 172.21.15.117:0/470957068 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0e94110ab0 con 0x7f0e94102bd0 2024-03-22T06:07:41.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.422+0000 7f0e7bfff700 1 -- 172.21.15.117:0/470957068 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f0e84011370 con 0x7f0e94102bd0 2024-03-22T06:07:41.422 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:41.423 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:41.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.424+0000 7f0e99cd8700 1 -- 172.21.15.117:0/470957068 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e7c036970 msgr2=0x7f0e7c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:41.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.424+0000 7f0e99cd8700 1 --2- 172.21.15.117:0/470957068 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e7c036970 0x7f0e7c038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009990 tx=0x7f0e88006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:41.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.424+0000 7f0e99cd8700 1 -- 172.21.15.117:0/470957068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e94102bd0 msgr2=0x7f0e94115420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:41.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.424+0000 7f0e99cd8700 1 --2- 172.21.15.117:0/470957068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e94102bd0 0x7f0e94115420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400b820 tx=0x7f0e84009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:41.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.424+0000 7f0e99cd8700 1 -- 172.21.15.117:0/470957068 shutdown_connections 2024-03-22T06:07:41.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.424+0000 7f0e99cd8700 1 --2- 172.21.15.117:0/470957068 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e7c036970 0x7f0e7c038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:41.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.424+0000 7f0e99cd8700 1 --2- 172.21.15.117:0/470957068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e94102bd0 0x7f0e94115420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:41.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.424+0000 7f0e99cd8700 1 -- 172.21.15.117:0/470957068 >> 172.21.15.117:0/470957068 conn(0x7f0e940f85e0 msgr2=0x7f0e940f89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:41.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.424+0000 7f0e99cd8700 1 -- 172.21.15.117:0/470957068 shutdown_connections 2024-03-22T06:07:41.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:41.424+0000 7f0e99cd8700 1 -- 172.21.15.117:0/470957068 wait complete. 2024-03-22T06:07:41.425 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:41 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/470957068' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:42.940 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:42.940 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:43.217 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:07:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:43 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:43 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:43 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:43 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:43 smithi006 ceph-mon[29301]: Deploying daemon node-exporter.smithi117 on smithi117 2024-03-22T06:07:45.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.977+0000 7f5ff72f9700 1 -- 172.21.15.117:0/3573498209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ff01019b0 msgr2=0x7f5ff0101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.977+0000 7f5ff72f9700 1 --2- 172.21.15.117:0/3573498209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ff01019b0 0x7f5ff0101d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0048c0 tx=0x7f5fec009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.977+0000 7f5ff72f9700 1 -- 172.21.15.117:0/3573498209 shutdown_connections 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.977+0000 7f5ff72f9700 1 --2- 172.21.15.117:0/3573498209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ff01019b0 0x7f5ff0101d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.977+0000 7f5ff72f9700 1 -- 172.21.15.117:0/3573498209 >> 172.21.15.117:0/3573498209 conn(0x7f5ff00f7510 msgr2=0x7f5ff00f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.977+0000 7f5ff72f9700 1 -- 172.21.15.117:0/3573498209 shutdown_connections 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.977+0000 7f5ff72f9700 1 -- 172.21.15.117:0/3573498209 wait complete. 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.978+0000 7f5ff72f9700 1 Processor -- start 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.978+0000 7f5ff72f9700 1 -- start start 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.978+0000 7f5ff72f9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ff01019b0 0x7f5ff0112010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.978+0000 7f5ff72f9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0112550 con 0x7f5ff01019b0 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.978+0000 7f5ff5095700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ff01019b0 0x7f5ff0112010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.978+0000 7f5ff5095700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ff01019b0 0x7f5ff0112010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:38886/0 (socket says 172.21.15.117:38886) 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.978+0000 7f5ff5095700 1 -- 172.21.15.117:0/661431883 learned_addr learned my addr 172.21.15.117:0/661431883 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.978+0000 7f5ff5095700 1 -- 172.21.15.117:0/661431883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec005040 con 0x7f5ff01019b0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.979+0000 7f5ff5095700 1 --2- 172.21.15.117:0/661431883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ff01019b0 0x7f5ff0112010 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00b820 tx=0x7f5fec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.979+0000 7f5fe5ffb700 1 -- 172.21.15.117:0/661431883 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fec0093c0 con 0x7f5ff01019b0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.979+0000 7f5ff72f9700 1 -- 172.21.15.117:0/661431883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff0112750 con 0x7f5ff01019b0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.979+0000 7f5fe5ffb700 1 -- 172.21.15.117:0/661431883 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fec013420 con 0x7f5ff01019b0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.979+0000 7f5fe5ffb700 1 -- 172.21.15.117:0/661431883 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fec01c5b0 con 0x7f5ff01019b0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.979+0000 7f5ff72f9700 1 -- 172.21.15.117:0/661431883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff0112c40 con 0x7f5ff01019b0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.980+0000 7f5ff72f9700 1 -- 172.21.15.117:0/661431883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff00fd0b0 con 0x7f5ff01019b0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.980+0000 7f5fe5ffb700 1 -- 172.21.15.117:0/661431883 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 43479+0+0 (secure 0 0 0) 0x7f5fec013a90 con 0x7f5ff01019b0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.980+0000 7f5fe5ffb700 1 --2- 172.21.15.117:0/661431883 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5fdc036970 0x7f5fdc038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.980+0000 7f5fe5ffb700 1 -- 172.21.15.117:0/661431883 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f5fec049b20 con 0x7f5ff01019b0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.980+0000 7f5ff4894700 1 --2- 172.21.15.117:0/661431883 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5fdc036970 0x7f5fdc038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.981+0000 7f5ff4894700 1 --2- 172.21.15.117:0/661431883 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5fdc036970 0x7f5fdc038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009990 tx=0x7f5fe0006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:44.985+0000 7f5fe5ffb700 1 -- 172.21.15.117:0/661431883 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5fec0098a0 con 0x7f5ff01019b0 2024-03-22T06:07:45.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.289+0000 7f5ff72f9700 1 -- 172.21.15.117:0/661431883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5ff005f5f0 con 0x7f5ff01019b0 2024-03-22T06:07:45.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.289+0000 7f5fe5ffb700 1 -- 172.21.15.117:0/661431883 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f5fec023300 con 0x7f5ff01019b0 2024-03-22T06:07:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:45.290 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:45.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.291+0000 7f5ff72f9700 1 -- 172.21.15.117:0/661431883 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5fdc036970 msgr2=0x7f5fdc038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:45.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.291+0000 7f5ff72f9700 1 --2- 172.21.15.117:0/661431883 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5fdc036970 0x7f5fdc038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009990 tx=0x7f5fe0006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:45.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.292+0000 7f5ff72f9700 1 -- 172.21.15.117:0/661431883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ff01019b0 msgr2=0x7f5ff0112010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:45.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.292+0000 7f5ff72f9700 1 --2- 172.21.15.117:0/661431883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ff01019b0 0x7f5ff0112010 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00b820 tx=0x7f5fec009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:45.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.292+0000 7f5ff72f9700 1 -- 172.21.15.117:0/661431883 shutdown_connections 2024-03-22T06:07:45.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.292+0000 7f5ff72f9700 1 --2- 172.21.15.117:0/661431883 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5fdc036970 0x7f5fdc038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:45.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.292+0000 7f5ff72f9700 1 --2- 172.21.15.117:0/661431883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ff01019b0 0x7f5ff0112010 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:45.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.292+0000 7f5ff72f9700 1 -- 172.21.15.117:0/661431883 >> 172.21.15.117:0/661431883 conn(0x7f5ff00f7510 msgr2=0x7f5ff00f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:45.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.292+0000 7f5ff72f9700 1 -- 172.21.15.117:0/661431883 shutdown_connections 2024-03-22T06:07:45.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:45.292+0000 7f5ff72f9700 1 -- 172.21.15.117:0/661431883 wait complete. 2024-03-22T06:07:45.293 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:45.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:45 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/661431883' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:47.308 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:47.308 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:47.565 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:07:48.485 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:48.485 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:07:48.486 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:48.486 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:48.486 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:48.486 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:48.486 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:07:48.486 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-22T06:07:48.486 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:07:48.486 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:07:48.486 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:48 smithi006 ceph-mon[29301]: Deploying daemon mgr.smithi117.bqeskk on smithi117 2024-03-22T06:07:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.153+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/3024105899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e640fdad0 msgr2=0x7f5e640fdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.153+0000 7f5e6a7cd700 1 --2- 172.21.15.117:0/3024105899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e640fdad0 0x7f5e640fdea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5e580048c0 tx=0x7f5e58009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.153+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/3024105899 shutdown_connections 2024-03-22T06:07:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.153+0000 7f5e6a7cd700 1 --2- 172.21.15.117:0/3024105899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e640fdad0 0x7f5e640fdea0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.153+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/3024105899 >> 172.21.15.117:0/3024105899 conn(0x7f5e640f74e0 msgr2=0x7f5e640f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.153+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/3024105899 shutdown_connections 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.153+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/3024105899 wait complete. 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.154+0000 7f5e6a7cd700 1 Processor -- start 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.154+0000 7f5e6a7cd700 1 -- start start 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.154+0000 7f5e6a7cd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e640fdad0 0x7f5e64113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.154+0000 7f5e6a7cd700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e641136c0 con 0x7f5e640fdad0 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.154+0000 7f5e63fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e640fdad0 0x7f5e64113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.154+0000 7f5e63fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e640fdad0 0x7f5e64113180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:38900/0 (socket says 172.21.15.117:38900) 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.154+0000 7f5e63fff700 1 -- 172.21.15.117:0/1786096046 learned_addr learned my addr 172.21.15.117:0/1786096046 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.155+0000 7f5e63fff700 1 -- 172.21.15.117:0/1786096046 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e58005040 con 0x7f5e640fdad0 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.155+0000 7f5e63fff700 1 --2- 172.21.15.117:0/1786096046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e640fdad0 0x7f5e64113180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800b820 tx=0x7f5e58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.155+0000 7f5e60ff9700 1 -- 172.21.15.117:0/1786096046 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e580093c0 con 0x7f5e640fdad0 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.155+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/1786096046 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e641138c0 con 0x7f5e640fdad0 2024-03-22T06:07:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.155+0000 7f5e60ff9700 1 -- 172.21.15.117:0/1786096046 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e58013420 con 0x7f5e640fdad0 2024-03-22T06:07:50.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.155+0000 7f5e60ff9700 1 -- 172.21.15.117:0/1786096046 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e5801c5b0 con 0x7f5e640fdad0 2024-03-22T06:07:50.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.155+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/1786096046 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e64113d30 con 0x7f5e640fdad0 2024-03-22T06:07:50.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.156+0000 7f5e60ff9700 1 -- 172.21.15.117:0/1786096046 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 43479+0+0 (secure 0 0 0) 0x7f5e5801c7f0 con 0x7f5e640fdad0 2024-03-22T06:07:50.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.156+0000 7f5e60ff9700 1 --2- 172.21.15.117:0/1786096046 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e4c036970 0x7f5e4c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:50.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.156+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/1786096046 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e640665e0 con 0x7f5e640fdad0 2024-03-22T06:07:50.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.156+0000 7f5e60ff9700 1 -- 172.21.15.117:0/1786096046 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f5e58049b70 con 0x7f5e640fdad0 2024-03-22T06:07:50.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.156+0000 7f5e637fe700 1 --2- 172.21.15.117:0/1786096046 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e4c036970 0x7f5e4c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:50.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.157+0000 7f5e637fe700 1 --2- 172.21.15.117:0/1786096046 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e4c036970 0x7f5e4c038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009990 tx=0x7f5e54006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:50.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.161+0000 7f5e60ff9700 1 -- 172.21.15.117:0/1786096046 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5e58023300 con 0x7f5e640fdad0 2024-03-22T06:07:50.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.431+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/1786096046 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5e6405f5f0 con 0x7f5e640fdad0 2024-03-22T06:07:50.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.432+0000 7f5e60ff9700 1 -- 172.21.15.117:0/1786096046 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f5e58011b60 con 0x7f5e640fdad0 2024-03-22T06:07:50.433 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:50.433 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.434+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/1786096046 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e4c036970 msgr2=0x7f5e4c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.434+0000 7f5e6a7cd700 1 --2- 172.21.15.117:0/1786096046 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e4c036970 0x7f5e4c038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009990 tx=0x7f5e54006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.434+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/1786096046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e640fdad0 msgr2=0x7f5e64113180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.434+0000 7f5e6a7cd700 1 --2- 172.21.15.117:0/1786096046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e640fdad0 0x7f5e64113180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800b820 tx=0x7f5e58009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.434+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/1786096046 shutdown_connections 2024-03-22T06:07:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.434+0000 7f5e6a7cd700 1 --2- 172.21.15.117:0/1786096046 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e4c036970 0x7f5e4c038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:50.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.434+0000 7f5e6a7cd700 1 --2- 172.21.15.117:0/1786096046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e640fdad0 0x7f5e64113180 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:50.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.434+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/1786096046 >> 172.21.15.117:0/1786096046 conn(0x7f5e640f74e0 msgr2=0x7f5e64103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:50.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.435+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/1786096046 shutdown_connections 2024-03-22T06:07:50.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:50.435+0000 7f5e6a7cd700 1 -- 172.21.15.117:0/1786096046 wait complete. 2024-03-22T06:07:50.436 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:50.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:50 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/1786096046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:07:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:51 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:51 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:51 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:51 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:52.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:51 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:07:52.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:51 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:07:52.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:51 smithi006 ceph-mon[29301]: Deploying daemon mon.smithi117 on smithi117 2024-03-22T06:07:52.180 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:52.180 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:53.370 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi117/config 2024-03-22T06:07:53.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:53 smithi006 ceph-mon[29301]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:53.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:53 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:07:55.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.104+0000 7fb46ebb7700 1 -- 172.21.15.117:0/3113623361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb468101ae0 msgr2=0x7fb468101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:55.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.104+0000 7fb46ebb7700 1 --2- 172.21.15.117:0/3113623361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb468101ae0 0x7fb468101eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0048c0 tx=0x7fb45c009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:55.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.104+0000 7fb46ebb7700 1 -- 172.21.15.117:0/3113623361 shutdown_connections 2024-03-22T06:07:55.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.104+0000 7fb46ebb7700 1 --2- 172.21.15.117:0/3113623361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb468101ae0 0x7fb468101eb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:55.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.104+0000 7fb46ebb7700 1 -- 172.21.15.117:0/3113623361 >> 172.21.15.117:0/3113623361 conn(0x7fb4680f75e0 msgr2=0x7fb4680f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:55.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.104+0000 7fb46ebb7700 1 -- 172.21.15.117:0/3113623361 shutdown_connections 2024-03-22T06:07:55.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.104+0000 7fb46ebb7700 1 -- 172.21.15.117:0/3113623361 wait complete. 2024-03-22T06:07:55.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.105+0000 7fb46ebb7700 1 Processor -- start 2024-03-22T06:07:55.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.105+0000 7fb46ebb7700 1 -- start start 2024-03-22T06:07:55.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.105+0000 7fb46ebb7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb468101ae0 0x7fb4680f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:55.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.105+0000 7fb46ebb7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680f7fc0 con 0x7fb468101ae0 2024-03-22T06:07:55.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.105+0000 7fb46c953700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb468101ae0 0x7fb4680f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:55.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.105+0000 7fb46c953700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb468101ae0 0x7fb4680f7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.117:53694/0 (socket says 172.21.15.117:53694) 2024-03-22T06:07:55.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.105+0000 7fb46c953700 1 -- 172.21.15.117:0/2404806676 learned_addr learned my addr 172.21.15.117:0/2404806676 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:07:55.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.106+0000 7fb46c953700 1 -- 172.21.15.117:0/2404806676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c005040 con 0x7fb468101ae0 2024-03-22T06:07:55.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.106+0000 7fb46c953700 1 --2- 172.21.15.117:0/2404806676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb468101ae0 0x7fb4680f7a80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00b820 tx=0x7fb45c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:55.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.106+0000 7fb4657fa700 1 -- 172.21.15.117:0/2404806676 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb45c0093c0 con 0x7fb468101ae0 2024-03-22T06:07:55.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.106+0000 7fb46ebb7700 1 -- 172.21.15.117:0/2404806676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4680f81c0 con 0x7fb468101ae0 2024-03-22T06:07:55.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.106+0000 7fb4657fa700 1 -- 172.21.15.117:0/2404806676 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb45c013420 con 0x7fb468101ae0 2024-03-22T06:07:55.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.106+0000 7fb4657fa700 1 -- 172.21.15.117:0/2404806676 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb45c01c5b0 con 0x7fb468101ae0 2024-03-22T06:07:55.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.106+0000 7fb46ebb7700 1 -- 172.21.15.117:0/2404806676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4680f86b0 con 0x7fb468101ae0 2024-03-22T06:07:55.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.107+0000 7fb4657fa700 1 -- 172.21.15.117:0/2404806676 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 43479+0+0 (secure 0 0 0) 0x7fb45c01c7f0 con 0x7fb468101ae0 2024-03-22T06:07:55.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.107+0000 7fb46ebb7700 1 -- 172.21.15.117:0/2404806676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4680f88f0 con 0x7fb468101ae0 2024-03-22T06:07:55.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.107+0000 7fb4657fa700 1 --2- 172.21.15.117:0/2404806676 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb458036970 0x7fb458038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:07:55.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.107+0000 7fb4657fa700 1 -- 172.21.15.117:0/2404806676 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fb45c049b60 con 0x7fb468101ae0 2024-03-22T06:07:55.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.107+0000 7fb467fff700 1 --2- 172.21.15.117:0/2404806676 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb458036970 0x7fb458038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:07:55.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.108+0000 7fb467fff700 1 --2- 172.21.15.117:0/2404806676 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb458036970 0x7fb458038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb454009990 tx=0x7fb454006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:07:55.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.112+0000 7fb4657fa700 1 -- 172.21.15.117:0/2404806676 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb4680f88f0 con 0x7fb468101ae0 2024-03-22T06:07:55.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.398+0000 7fb46ebb7700 1 -- 172.21.15.117:0/2404806676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb4680f88f0 con 0x7fb468101ae0 2024-03-22T06:07:55.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.399+0000 7fb4657fa700 1 -- 172.21.15.117:0/2404806676 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fb4680f88f0 con 0x7fb468101ae0 2024-03-22T06:07:55.399 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:07:55.399 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:05:24.616649Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T06:07:55.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.401+0000 7fb46ebb7700 1 -- 172.21.15.117:0/2404806676 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb458036970 msgr2=0x7fb458038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:55.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.401+0000 7fb46ebb7700 1 --2- 172.21.15.117:0/2404806676 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb458036970 0x7fb458038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb454009990 tx=0x7fb454006e40 comp rx=0 tx=0).stop 2024-03-22T06:07:55.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.401+0000 7fb46ebb7700 1 -- 172.21.15.117:0/2404806676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb468101ae0 msgr2=0x7fb4680f7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:07:55.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.401+0000 7fb46ebb7700 1 --2- 172.21.15.117:0/2404806676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb468101ae0 0x7fb4680f7a80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00b820 tx=0x7fb45c009e30 comp rx=0 tx=0).stop 2024-03-22T06:07:55.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.401+0000 7fb46ebb7700 1 -- 172.21.15.117:0/2404806676 shutdown_connections 2024-03-22T06:07:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.401+0000 7fb46ebb7700 1 --2- 172.21.15.117:0/2404806676 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb458036970 0x7fb458038e20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.401+0000 7fb46ebb7700 1 --2- 172.21.15.117:0/2404806676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb468101ae0 0x7fb4680f7a80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:07:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.401+0000 7fb46ebb7700 1 -- 172.21.15.117:0/2404806676 >> 172.21.15.117:0/2404806676 conn(0x7fb4680f75e0 msgr2=0x7fb468066040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:07:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.401+0000 7fb46ebb7700 1 -- 172.21.15.117:0/2404806676 shutdown_connections 2024-03-22T06:07:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:07:55.401+0000 7fb46ebb7700 1 -- 172.21.15.117:0/2404806676 wait complete. 2024-03-22T06:07:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-03-22T06:07:55.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:07:55 smithi006 ceph-mon[29301]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:07:55.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:07:55 smithi117 systemd[1]: Started Ceph mon.smithi117 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:07:57.298 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T06:07:57.298 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T06:07:57.547 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi117/config 2024-03-22T06:08:01.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:08:01.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: mon.smithi006 calling monitor election 2024-03-22T06:08:01.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:01.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/3273197791' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/crt"}]: dispatch 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: mon.smithi117 calling monitor election 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: mon.smithi006 is new leader, mons smithi006,smithi117 in quorum (ranks 0,1) 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: monmap e2: 2 mons at {smithi006=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: fsmap 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: osdmap e5: 0 total, 0 up, 0 in 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: mgrmap e17: smithi006.sfpxdy(active, since 28s) 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: overall HEALTH_OK 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: Standby manager daemon smithi117.bqeskk started 2024-03-22T06:08:01.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/3273197791' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:08:01.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:01.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/3273197791' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/key"}]: dispatch 2024-03-22T06:08:01.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/3273197791' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:08:01.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:01.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:01.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:01.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:01.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:08:01.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: mon.smithi006 calling monitor election 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/3273197791' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/crt"}]: dispatch 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: mon.smithi117 calling monitor election 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: mon.smithi006 is new leader, mons smithi006,smithi117 in quorum (ranks 0,1) 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: monmap e2: 2 mons at {smithi006=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T06:08:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: fsmap 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: osdmap e5: 0 total, 0 up, 0 in 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: mgrmap e17: smithi006.sfpxdy(active, since 28s) 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: overall HEALTH_OK 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: Standby manager daemon smithi117.bqeskk started 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/3273197791' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/3273197791' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/key"}]: dispatch 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/3273197791' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:01.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:02.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:01 smithi117 ceph-mon[36514]: mgrmap e18: smithi006.sfpxdy(active, since 28s), standbys: smithi117.bqeskk 2024-03-22T06:08:02.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi117.bqeskk", "id": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:08:02.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:08:02.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:02.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.019+0000 7fe0169cb700 1 -- 172.21.15.117:0/700978817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff8003580 msgr2=0x7fdff8005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:02.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.019+0000 7fe0169cb700 1 --2- 172.21.15.117:0/700978817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff8003580 0x7fdff8005a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe010070210 tx=0x7fe00000b040 comp rx=0 tx=0).stop 2024-03-22T06:08:02.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.019+0000 7fe0169cb700 1 -- 172.21.15.117:0/700978817 shutdown_connections 2024-03-22T06:08:02.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.019+0000 7fe0169cb700 1 --2- 172.21.15.117:0/700978817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff8003580 0x7fdff8005a00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:02.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.019+0000 7fe0169cb700 1 -- 172.21.15.117:0/700978817 >> 172.21.15.117:0/700978817 conn(0x7fe0100f8730 msgr2=0x7fe0100fab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:08:02.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.019+0000 7fe0169cb700 1 -- 172.21.15.117:0/700978817 shutdown_connections 2024-03-22T06:08:02.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.019+0000 7fe0169cb700 1 -- 172.21.15.117:0/700978817 wait complete. 2024-03-22T06:08:02.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.020+0000 7fe0169cb700 1 Processor -- start 2024-03-22T06:08:02.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.020+0000 7fe0169cb700 1 -- start start 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.020+0000 7fe0169cb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff8003580 0x7fe010115550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.020+0000 7fe0169cb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe010115a90 0x7fe01010f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.020+0000 7fe0169cb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe010116050 con 0x7fdff8003580 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.020+0000 7fe0169cb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01010fa10 con 0x7fe010115a90 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe010115a90 0x7fe01010f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe010115a90 0x7fe01010f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52654/0 (socket says 172.21.15.117:52654) 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00f7fe700 1 -- 172.21.15.117:0/151892251 learned_addr learned my addr 172.21.15.117:0/151892251 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00ffff700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff8003580 0x7fe010115550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00f7fe700 1 -- 172.21.15.117:0/151892251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe010115a90 msgr2=0x7fe01010f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00f7fe700 1 -- 172.21.15.117:0/151892251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe010115a90 msgr2=0x7fe01010f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00f7fe700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe010115a90 0x7fe01010f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00f7fe700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe010115a90 0x7fe01010f4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00ffff700 1 -- 172.21.15.117:0/151892251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe010115a90 msgr2=0x7fe01010f4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T06:08:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00ffff700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe010115a90 0x7fe01010f4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:02.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00ffff700 1 -- 172.21.15.117:0/151892251 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe000005040 con 0x7fdff8003580 2024-03-22T06:08:02.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00ffff700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff8003580 0x7fe010115550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe000009ce0 tx=0x7fe00000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:08:02.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00cff9700 1 -- 172.21.15.117:0/151892251 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe00001b070 con 0x7fdff8003580 2024-03-22T06:08:02.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe0169cb700 1 -- 172.21.15.117:0/151892251 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01010fc90 con 0x7fdff8003580 2024-03-22T06:08:02.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00cff9700 1 -- 172.21.15.117:0/151892251 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe000016d10 con 0x7fdff8003580 2024-03-22T06:08:02.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe00cff9700 1 -- 172.21.15.117:0/151892251 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe00001f620 con 0x7fdff8003580 2024-03-22T06:08:02.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.021+0000 7fe0169cb700 1 -- 172.21.15.117:0/151892251 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0101101d0 con 0x7fdff8003580 2024-03-22T06:08:02.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.022+0000 7fe0169cb700 1 -- 172.21.15.117:0/151892251 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe010103de0 con 0x7fdff8003580 2024-03-22T06:08:02.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.023+0000 7fe00cff9700 1 -- 172.21.15.117:0/151892251 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7fe000020440 con 0x7fdff8003580 2024-03-22T06:08:02.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.023+0000 7fe00cff9700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdffc069320 0x7fdffc06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:02.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.023+0000 7fe00cff9700 1 -- 172.21.15.117:0/151892251 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fe000087f40 con 0x7fdff8003580 2024-03-22T06:08:02.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.023+0000 7fe00f7fe700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdffc069320 0x7fdffc06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:02.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.024+0000 7fe00f7fe700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdffc069320 0x7fdffc06b7d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe004006fd0 tx=0x7fe004008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:08:02.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.028+0000 7fe00cff9700 1 -- 172.21.15.117:0/151892251 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe000058cc0 con 0x7fdff8003580 2024-03-22T06:08:02.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:01 smithi006 ceph-mon[29301]: mgrmap e18: smithi006.sfpxdy(active, since 28s), standbys: smithi117.bqeskk 2024-03-22T06:08:02.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi117.bqeskk", "id": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:08:02.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:08:02.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:08:02.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.295+0000 7fe0169cb700 1 -- 172.21.15.117:0/151892251 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe010106ae0 con 0x7fdff8003580 2024-03-22T06:08:02.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.295+0000 7fe00cff9700 1 -- 172.21.15.117:0/151892251 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1013 (secure 0 0 0) 0x7fe000058850 con 0x7fdff8003580 2024-03-22T06:08:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-22T06:08:02.296 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":2,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","modified":"2024-03-22T06:07:55.556663Z","created":"2024-03-22T06:05:24.616649Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi006","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-22T06:08:02.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.298+0000 7fe0169cb700 1 -- 172.21.15.117:0/151892251 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdffc069320 msgr2=0x7fdffc06b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:02.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.298+0000 7fe0169cb700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdffc069320 0x7fdffc06b7d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe004006fd0 tx=0x7fe004008040 comp rx=0 tx=0).stop 2024-03-22T06:08:02.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.298+0000 7fe0169cb700 1 -- 172.21.15.117:0/151892251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff8003580 msgr2=0x7fe010115550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:02.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.298+0000 7fe0169cb700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff8003580 0x7fe010115550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe000009ce0 tx=0x7fe00000b040 comp rx=0 tx=0).stop 2024-03-22T06:08:02.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.298+0000 7fe0169cb700 1 -- 172.21.15.117:0/151892251 shutdown_connections 2024-03-22T06:08:02.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.298+0000 7fe0169cb700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdffc069320 0x7fdffc06b7d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:02.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.298+0000 7fe0169cb700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff8003580 0x7fe010115550 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:02.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.298+0000 7fe0169cb700 1 --2- 172.21.15.117:0/151892251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe010115a90 0x7fe01010f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:02.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.299+0000 7fe0169cb700 1 -- 172.21.15.117:0/151892251 >> 172.21.15.117:0/151892251 conn(0x7fe0100f8730 msgr2=0x7fe0100fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:08:02.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.299+0000 7fe0169cb700 1 -- 172.21.15.117:0/151892251 shutdown_connections 2024-03-22T06:08:02.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:08:02.299+0000 7fe0169cb700 1 -- 172.21.15.117:0/151892251 wait complete. 2024-03-22T06:08:02.300 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 2 2024-03-22T06:08:02.842 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:02 smithi117 ceph-mon[36514]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:02.842 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:08:02.842 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:02.842 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:02 smithi117 ceph-mon[36514]: from='client.? 172.21.15.117:0/151892251' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:08:02.865 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-22T06:08:02.865 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-22T06:08:03.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:02 smithi006 ceph-mon[29301]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:03.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:08:03.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:03.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:02 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/151892251' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T06:08:03.145 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:08:04.214 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.214 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.214 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:04.214 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:08:04.214 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-22T06:08:04.214 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: Updating smithi117:/etc/ceph/ceph.conf 2024-03-22T06:08:04.214 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:08:04.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:08:04.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:08:04.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:08:04.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: Updating smithi117:/etc/ceph/ceph.conf 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:08:04.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:08:04.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:05.228 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:04 smithi006 ceph-mon[29301]: Reconfiguring mon.smithi006 (unknown last config time)... 2024-03-22T06:08:05.228 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:04 smithi006 ceph-mon[29301]: Reconfiguring daemon mon.smithi006 on smithi006 2024-03-22T06:08:05.228 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:04 smithi006 ceph-mon[29301]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:05.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:04 smithi117 ceph-mon[36514]: Reconfiguring mon.smithi006 (unknown last config time)... 2024-03-22T06:08:05.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:04 smithi117 ceph-mon[36514]: Reconfiguring daemon mon.smithi006 on smithi006 2024-03-22T06:08:05.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:04 smithi117 ceph-mon[36514]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:05.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.680+0000 7fa848588700 1 -- 172.21.15.6:0/3433112050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8401026c0 msgr2=0x7fa8400fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:05.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.680+0000 7fa848588700 1 --2- 172.21.15.6:0/3433112050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8401026c0 0x7fa8400fbd30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa830004940 tx=0x7fa830009e30 comp rx=0 tx=0).stop 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.680+0000 7fa848588700 1 -- 172.21.15.6:0/3433112050 shutdown_connections 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.680+0000 7fa848588700 1 --2- 172.21.15.6:0/3433112050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8401026c0 0x7fa8400fbd30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.680+0000 7fa848588700 1 --2- 172.21.15.6:0/3433112050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa840101d20 0x7fa8401020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.680+0000 7fa848588700 1 -- 172.21.15.6:0/3433112050 >> 172.21.15.6:0/3433112050 conn(0x7fa8400f78d0 msgr2=0x7fa8400f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.680+0000 7fa848588700 1 -- 172.21.15.6:0/3433112050 shutdown_connections 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.680+0000 7fa848588700 1 -- 172.21.15.6:0/3433112050 wait complete. 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa848588700 1 Processor -- start 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa848588700 1 -- start start 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa848588700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa840101d20 0x7fa8400fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa848588700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8401026c0 0x7fa8400fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa848588700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400fe470 con 0x7fa8401026c0 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa848588700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400fe5b0 con 0x7fa840101d20 2024-03-22T06:08:05.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa845b23700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8401026c0 0x7fa8400fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa845b23700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8401026c0 0x7fa8400fdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42628/0 (socket says 172.21.15.6:42628) 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa845b23700 1 -- 172.21.15.6:0/2348115559 learned_addr learned my addr 172.21.15.6:0/2348115559 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa846324700 1 --2- 172.21.15.6:0/2348115559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa840101d20 0x7fa8400fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa845b23700 1 -- 172.21.15.6:0/2348115559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa840101d20 msgr2=0x7fa8400fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa845b23700 1 --2- 172.21.15.6:0/2348115559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa840101d20 0x7fa8400fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa845b23700 1 -- 172.21.15.6:0/2348115559 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa830005040 con 0x7fa8401026c0 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.681+0000 7fa845b23700 1 --2- 172.21.15.6:0/2348115559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8401026c0 0x7fa8400fdee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa830009100 tx=0x7fa830009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.682+0000 7fa836ffd700 1 -- 172.21.15.6:0/2348115559 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83000e040 con 0x7fa8401026c0 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.682+0000 7fa836ffd700 1 -- 172.21.15.6:0/2348115559 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa830013420 con 0x7fa8401026c0 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.682+0000 7fa836ffd700 1 -- 172.21.15.6:0/2348115559 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83001c630 con 0x7fa8401026c0 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.682+0000 7fa848588700 1 -- 172.21.15.6:0/2348115559 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8400fe800 con 0x7fa8401026c0 2024-03-22T06:08:05.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.682+0000 7fa848588700 1 -- 172.21.15.6:0/2348115559 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84011a320 con 0x7fa8401026c0 2024-03-22T06:08:05.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.683+0000 7fa848588700 1 -- 172.21.15.6:0/2348115559 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8400fd470 con 0x7fa8401026c0 2024-03-22T06:08:05.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.683+0000 7fa836ffd700 1 -- 172.21.15.6:0/2348115559 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7fa830013a00 con 0x7fa8401026c0 2024-03-22T06:08:05.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.684+0000 7fa836ffd700 1 --2- 172.21.15.6:0/2348115559 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa82c069210 0x7fa82c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:05.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.684+0000 7fa836ffd700 1 -- 172.21.15.6:0/2348115559 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fa830087180 con 0x7fa8401026c0 2024-03-22T06:08:05.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.684+0000 7fa846324700 1 --2- 172.21.15.6:0/2348115559 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa82c069210 0x7fa82c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:05.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.684+0000 7fa846324700 1 --2- 172.21.15.6:0/2348115559 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa82c069210 0x7fa82c06b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa83c004f40 tx=0x7fa83c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:08:05.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.688+0000 7fa836ffd700 1 -- 172.21.15.6:0/2348115559 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa830054570 con 0x7fa8401026c0 2024-03-22T06:08:05.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.888+0000 7fa848588700 1 -- 172.21.15.6:0/2348115559 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa840108e30 con 0x7fa8401026c0 2024-03-22T06:08:05.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.888+0000 7fa836ffd700 1 -- 172.21.15.6:0/2348115559 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7fa830057b90 con 0x7fa8401026c0 2024-03-22T06:08:05.889 INFO:teuthology.orchestra.run.smithi006.stdout:# minimal ceph.conf for fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:08:05.889 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2024-03-22T06:08:05.890 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:08:05.890 INFO:teuthology.orchestra.run.smithi006.stdout: mon_host = [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] 2024-03-22T06:08:05.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.890+0000 7fa848588700 1 -- 172.21.15.6:0/2348115559 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa82c069210 msgr2=0x7fa82c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:05.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.890+0000 7fa848588700 1 --2- 172.21.15.6:0/2348115559 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa82c069210 0x7fa82c06b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa83c004f40 tx=0x7fa83c00b040 comp rx=0 tx=0).stop 2024-03-22T06:08:05.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.890+0000 7fa848588700 1 -- 172.21.15.6:0/2348115559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8401026c0 msgr2=0x7fa8400fdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:05.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.890+0000 7fa848588700 1 --2- 172.21.15.6:0/2348115559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8401026c0 0x7fa8400fdee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa830009100 tx=0x7fa830009e30 comp rx=0 tx=0).stop 2024-03-22T06:08:05.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.890+0000 7fa848588700 1 -- 172.21.15.6:0/2348115559 shutdown_connections 2024-03-22T06:08:05.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.890+0000 7fa848588700 1 --2- 172.21.15.6:0/2348115559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa840101d20 0x7fa8400fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:05.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.890+0000 7fa848588700 1 --2- 172.21.15.6:0/2348115559 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa82c069210 0x7fa82c06b6c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:05.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.890+0000 7fa848588700 1 --2- 172.21.15.6:0/2348115559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8401026c0 0x7fa8400fdee0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:05.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.890+0000 7fa848588700 1 -- 172.21.15.6:0/2348115559 >> 172.21.15.6:0/2348115559 conn(0x7fa8400f78d0 msgr2=0x7fa8400f88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:08:05.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.890+0000 7fa848588700 1 -- 172.21.15.6:0/2348115559 shutdown_connections 2024-03-22T06:08:05.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:05.890+0000 7fa848588700 1 -- 172.21.15.6:0/2348115559 wait complete. 2024-03-22T06:08:06.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:06 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:06.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:06 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:06.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:06 smithi006 ceph-mon[29301]: Reconfiguring mgr.smithi006.sfpxdy (unknown last config time)... 2024-03-22T06:08:06.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:06 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi006.sfpxdy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:08:06.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:06 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:08:06.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:06 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:06.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:06 smithi006 ceph-mon[29301]: Reconfiguring daemon mgr.smithi006.sfpxdy on smithi006 2024-03-22T06:08:06.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:06 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2348115559' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:06 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:06 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:06 smithi117 ceph-mon[36514]: Reconfiguring mgr.smithi006.sfpxdy (unknown last config time)... 2024-03-22T06:08:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:06 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi006.sfpxdy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:08:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:06 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:08:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:06 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:06 smithi117 ceph-mon[36514]: Reconfiguring daemon mgr.smithi006.sfpxdy on smithi006 2024-03-22T06:08:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:06 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2348115559' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:07.463 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-22T06:08:07.463 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:08:07.463 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.conf 2024-03-22T06:08:07.494 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:08:07.494 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:08:07.558 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:07 smithi006 ceph-mon[29301]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:07.558 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:07 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:07.558 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:07 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:07.558 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:07 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi006", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:08:07.558 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:07 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:07.562 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:08:07.563 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-03-22T06:08:07.594 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:07 smithi117 ceph-mon[36514]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:07.594 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:07 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:07.594 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:07 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:07.594 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:07 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi006", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:08:07.595 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:07 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:07.599 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:08:07.599 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:08:07.671 INFO:tasks.cephadm:Deploying OSDs... 2024-03-22T06:08:07.671 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:08:07.671 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T06:08:07.686 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T06:08:07.686 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_1 2024-03-22T06:08:07.742 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T06:08:07.743 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T06:08:07.743 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-22T06:08:07.743 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T06:08:07.743 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T06:08:07.743 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-03-22 06:07:23.795241259 +0000 2024-03-22T06:08:07.743 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-03-22 06:01:10.644237659 +0000 2024-03-22T06:08:07.743 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-03-22 06:01:10.644237659 +0000 2024-03-22T06:08:07.743 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-03-22 06:01:10.644237659 +0000 2024-03-22T06:08:07.743 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T06:08:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-03-22T06:08:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-03-22T06:08:07.809 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000333679 s, 1.5 MB/s 2024-03-22T06:08:07.810 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T06:08:07.865 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_2 2024-03-22T06:08:07.920 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T06:08:07.921 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T06:08:07.921 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-22T06:08:07.921 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T06:08:07.921 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T06:08:07.921 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-03-22 06:07:23.795241259 +0000 2024-03-22T06:08:07.921 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-03-22 06:01:11.000234080 +0000 2024-03-22T06:08:07.921 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-03-22 06:01:11.000234080 +0000 2024-03-22T06:08:07.921 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-03-22 06:01:11.000234080 +0000 2024-03-22T06:08:07.921 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T06:08:07.987 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-03-22T06:08:07.987 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-03-22T06:08:07.987 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000195613 s, 2.6 MB/s 2024-03-22T06:08:07.988 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T06:08:08.043 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_3 2024-03-22T06:08:08.097 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T06:08:08.097 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T06:08:08.097 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-22T06:08:08.097 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T06:08:08.098 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T06:08:08.098 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-03-22 06:07:23.796241249 +0000 2024-03-22T06:08:08.098 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-03-22 06:01:11.341230651 +0000 2024-03-22T06:08:08.098 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-03-22 06:01:11.341230651 +0000 2024-03-22T06:08:08.098 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-03-22 06:01:11.341230651 +0000 2024-03-22T06:08:08.098 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T06:08:08.163 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-03-22T06:08:08.163 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-03-22T06:08:08.163 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000181799 s, 2.8 MB/s 2024-03-22T06:08:08.164 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T06:08:08.220 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_4 2024-03-22T06:08:08.276 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T06:08:08.276 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T06:08:08.276 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-22T06:08:08.276 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T06:08:08.276 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T06:08:08.276 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-03-22 06:07:23.796241249 +0000 2024-03-22T06:08:08.276 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-03-22 06:01:11.692227121 +0000 2024-03-22T06:08:08.276 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-03-22 06:01:11.692227121 +0000 2024-03-22T06:08:08.277 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-03-22 06:01:11.692227121 +0000 2024-03-22T06:08:08.277 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T06:08:08.341 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-03-22T06:08:08.342 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-03-22T06:08:08.342 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000206741 s, 2.5 MB/s 2024-03-22T06:08:08.343 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T06:08:08.398 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:08:08.399 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T06:08:08.417 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T06:08:08.418 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-03-22T06:08:08.475 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T06:08:08.475 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T06:08:08.475 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-22T06:08:08.475 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T06:08:08.475 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T06:08:08.475 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-22 06:07:53.597203603 +0000 2024-03-22T06:08:08.475 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-22 06:01:07.274099765 +0000 2024-03-22T06:08:08.476 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-22 06:01:07.274099765 +0000 2024-03-22T06:08:08.476 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-22 06:01:07.274099765 +0000 2024-03-22T06:08:08.476 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T06:08:08.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:08 smithi006 ceph-mon[29301]: Reconfiguring crash.smithi006 (monmap changed)... 2024-03-22T06:08:08.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:08 smithi006 ceph-mon[29301]: Reconfiguring daemon crash.smithi006 on smithi006 2024-03-22T06:08:08.541 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:08 smithi117 ceph-mon[36514]: Reconfiguring crash.smithi006 (monmap changed)... 2024-03-22T06:08:08.541 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:08 smithi117 ceph-mon[36514]: Reconfiguring daemon crash.smithi006 on smithi006 2024-03-22T06:08:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-22T06:08:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-22T06:08:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000229788 s, 2.2 MB/s 2024-03-22T06:08:08.546 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T06:08:08.603 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-03-22T06:08:08.659 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T06:08:08.659 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T06:08:08.659 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-22T06:08:08.659 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T06:08:08.659 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T06:08:08.659 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-22 06:07:53.597203603 +0000 2024-03-22T06:08:08.659 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-22 06:01:07.623092643 +0000 2024-03-22T06:08:08.659 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-22 06:01:07.623092643 +0000 2024-03-22T06:08:08.659 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-22 06:01:07.623092643 +0000 2024-03-22T06:08:08.660 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T06:08:08.726 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-22T06:08:08.727 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-22T06:08:08.727 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000254762 s, 2.0 MB/s 2024-03-22T06:08:08.728 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T06:08:08.785 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-03-22T06:08:08.841 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T06:08:08.841 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T06:08:08.841 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-22T06:08:08.841 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T06:08:08.841 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T06:08:08.841 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-22 06:07:53.597203603 +0000 2024-03-22T06:08:08.841 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-22 06:01:07.955085868 +0000 2024-03-22T06:08:08.841 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-22 06:01:07.955085868 +0000 2024-03-22T06:08:08.841 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-22 06:01:07.955085868 +0000 2024-03-22T06:08:08.842 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T06:08:08.907 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-22T06:08:08.907 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-22T06:08:08.907 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000203803 s, 2.5 MB/s 2024-03-22T06:08:08.908 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T06:08:08.965 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-03-22T06:08:09.020 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T06:08:09.020 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T06:08:09.020 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-22T06:08:09.020 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T06:08:09.020 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T06:08:09.021 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-22 06:07:53.598203583 +0000 2024-03-22T06:08:09.021 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-22 06:01:08.290079032 +0000 2024-03-22T06:08:09.021 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-22 06:01:08.290079032 +0000 2024-03-22T06:08:09.021 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-22 06:01:08.290079032 +0000 2024-03-22T06:08:09.021 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T06:08:09.086 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-22T06:08:09.086 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-22T06:08:09.086 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.00019014 s, 2.7 MB/s 2024-03-22T06:08:09.087 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T06:08:09.143 INFO:tasks.cephadm:Deploying osd.0 on smithi006 with /dev/vg_nvme/lv_4... 2024-03-22T06:08:09.143 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-22T06:08:09.449 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:09 smithi006 ceph-mon[29301]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:09.450 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:09 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:09.450 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:09 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:09.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:09 smithi117 ceph-mon[36514]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:09.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:09 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:09.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:09 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:10.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:10 smithi006 ceph-mon[29301]: Reconfiguring alertmanager.smithi006 (dependencies changed)... 2024-03-22T06:08:10.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:10 smithi006 ceph-mon[29301]: Reconfiguring daemon alertmanager.smithi006 on smithi006 2024-03-22T06:08:10.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:10 smithi117 ceph-mon[36514]: Reconfiguring alertmanager.smithi006 (dependencies changed)... 2024-03-22T06:08:10.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:10 smithi117 ceph-mon[36514]: Reconfiguring daemon alertmanager.smithi006 on smithi006 2024-03-22T06:08:11.100 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:08:11.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:11 smithi006 ceph-mon[29301]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:11.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:11 smithi117 ceph-mon[36514]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:13.432 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:13 smithi006 ceph-mon[29301]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:13.432 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-22T06:08:13.432 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-22T06:08:13.433 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-03-22T06:08:13.433 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-03-22T06:08:13.433 INFO:teuthology.orchestra.run.smithi006.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112856 s, 929 MB/s 2024-03-22T06:08:13.433 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 2024-03-22T06:08:13.433 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-03-22T06:08:13.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:13 smithi117 ceph-mon[36514]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:14.166 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi006:vg_nvme/lv_4 2024-03-22T06:08:14.566 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:08:15.623 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:15 smithi006 ceph-mon[29301]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:15.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:15 smithi117 ceph-mon[36514]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:16.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.683+0000 7f4f6a069700 1 -- 172.21.15.6:0/2774293668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f64070870 msgr2=0x7f4f64070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:16.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.683+0000 7f4f6a069700 1 --2- 172.21.15.6:0/2774293668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f64070870 0x7f4f64070ce0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c004940 tx=0x7f4f4c009e30 comp rx=0 tx=0).stop 2024-03-22T06:08:16.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.683+0000 7f4f6a069700 1 -- 172.21.15.6:0/2774293668 shutdown_connections 2024-03-22T06:08:16.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.683+0000 7f4f6a069700 1 --2- 172.21.15.6:0/2774293668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f64070870 0x7f4f64070ce0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:16.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.683+0000 7f4f6a069700 1 --2- 172.21.15.6:0/2774293668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f64071e20 0x7f4f64070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:16.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.683+0000 7f4f6a069700 1 -- 172.21.15.6:0/2774293668 >> 172.21.15.6:0/2774293668 conn(0x7f4f640f85b0 msgr2=0x7f4f640fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:08:16.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.683+0000 7f4f6a069700 1 -- 172.21.15.6:0/2774293668 shutdown_connections 2024-03-22T06:08:16.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.683+0000 7f4f6a069700 1 -- 172.21.15.6:0/2774293668 wait complete. 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.683+0000 7f4f6a069700 1 Processor -- start 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f6a069700 1 -- start start 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f6a069700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f64070870 0x7f4f6410fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f6a069700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f64071e20 0x7f4f64110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f6a069700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f64110760 con 0x7f4f64070870 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f6a069700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f64110fb0 con 0x7f4f64071e20 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f637fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f64070870 0x7f4f6410fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f637fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f64070870 0x7f4f6410fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32934/0 (socket says 172.21.15.6:32934) 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f637fe700 1 -- 172.21.15.6:0/2827650396 learned_addr learned my addr 172.21.15.6:0/2827650396 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f62ffd700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f64071e20 0x7f4f64110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f637fe700 1 -- 172.21.15.6:0/2827650396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f64071e20 msgr2=0x7f4f64110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:16.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f637fe700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f64071e20 0x7f4f64110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f637fe700 1 -- 172.21.15.6:0/2827650396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c005040 con 0x7f4f64070870 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f62ffd700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f64071e20 0x7f4f64110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f637fe700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f64070870 0x7f4f6410fc20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00cb10 tx=0x7f4f5c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f5bfff700 1 -- 172.21.15.6:0/2827650396 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f5c00a980 con 0x7f4f64070870 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f5bfff700 1 -- 172.21.15.6:0/2827650396 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f5c012420 con 0x7f4f64070870 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f6a069700 1 -- 172.21.15.6:0/2827650396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f64111290 con 0x7f4f64070870 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.684+0000 7f4f5bfff700 1 -- 172.21.15.6:0/2827650396 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f5c0115c0 con 0x7f4f64070870 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.685+0000 7f4f6a069700 1 -- 172.21.15.6:0/2827650396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f641117d0 con 0x7f4f64070870 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.685+0000 7f4f6a069700 1 -- 172.21.15.6:0/2827650396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6405f5f0 con 0x7f4f64070870 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.686+0000 7f4f5bfff700 1 -- 172.21.15.6:0/2827650396 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7f4f5c003020 con 0x7f4f64070870 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.686+0000 7f4f5bfff700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4f50069210 0x7f4f5006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.686+0000 7f4f5bfff700 1 -- 172.21.15.6:0/2827650396 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f4f5c0864c0 con 0x7f4f64070870 2024-03-22T06:08:16.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.686+0000 7f4f62ffd700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4f50069210 0x7f4f5006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.687+0000 7f4f62ffd700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4f50069210 0x7f4f5006b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00b970 tx=0x7f4f4c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:08:16.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.691+0000 7f4f5bfff700 1 -- 172.21.15.6:0/2827650396 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4f5c057360 con 0x7f4f64070870 2024-03-22T06:08:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:16.902+0000 7f4f6a069700 1 -- 172.21.15.6:0/2827650396 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f64111bb0 con 0x7f4f50069210 2024-03-22T06:08:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:17 smithi117 ceph-mon[36514]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:17 smithi117 ceph-mon[36514]: Reconfiguring grafana.smithi006 (dependencies changed)... 2024-03-22T06:08:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:17 smithi117 ceph-mon[36514]: Reconfiguring daemon grafana.smithi006 on smithi006 2024-03-22T06:08:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:08:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:08:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:08:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:17 smithi006 ceph-mon[29301]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:17 smithi006 ceph-mon[29301]: Reconfiguring grafana.smithi006 (dependencies changed)... 2024-03-22T06:08:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:17 smithi006 ceph-mon[29301]: Reconfiguring daemon grafana.smithi006 on smithi006 2024-03-22T06:08:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:08:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:08:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:08:18.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:18 smithi117 ceph-mon[36514]: from='client.14246 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:08:18.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:18 smithi006 ceph-mon[29301]: from='client.14246 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:08:19.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:19 smithi117 ceph-mon[36514]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:19.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:19 smithi006 ceph-mon[29301]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:21.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:21 smithi006 ceph-mon[29301]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:21.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:21 smithi117 ceph-mon[36514]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:22.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:22 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:22.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:22 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:22.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:22 smithi006 ceph-mon[29301]: Reconfiguring prometheus.smithi006 (dependencies changed)... 2024-03-22T06:08:22.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:22 smithi006 ceph-mon[29301]: Reconfiguring daemon prometheus.smithi006 on smithi006 2024-03-22T06:08:22.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:22 smithi006 ceph-mon[29301]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:23.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:22 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:23.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:22 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:23.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:22 smithi117 ceph-mon[36514]: Reconfiguring prometheus.smithi006 (dependencies changed)... 2024-03-22T06:08:23.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:22 smithi117 ceph-mon[36514]: Reconfiguring daemon prometheus.smithi006 on smithi006 2024-03-22T06:08:23.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:22 smithi117 ceph-mon[36514]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:23.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:23 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2355532492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecd88bb1-d643-4505-a098-3728178c89e1"}]: dispatch 2024-03-22T06:08:23.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:23 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2355532492' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecd88bb1-d643-4505-a098-3728178c89e1"}]': finished 2024-03-22T06:08:23.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:23 smithi006 ceph-mon[29301]: osdmap e6: 1 total, 0 up, 1 in 2024-03-22T06:08:23.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:23 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:23.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:23 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1076775281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:08:24.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:23 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2355532492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecd88bb1-d643-4505-a098-3728178c89e1"}]: dispatch 2024-03-22T06:08:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:23 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2355532492' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecd88bb1-d643-4505-a098-3728178c89e1"}]': finished 2024-03-22T06:08:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:23 smithi117 ceph-mon[36514]: osdmap e6: 1 total, 0 up, 1 in 2024-03-22T06:08:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:23 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:23 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1076775281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:08:25.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:24 smithi117 ceph-mon[36514]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:25.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:24 smithi006 ceph-mon[29301]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:27 smithi117 ceph-mon[36514]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:27 smithi006 ceph-mon[29301]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:29.218 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:29 smithi006 ceph-mon[29301]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:29 smithi117 ceph-mon[36514]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:31.434 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:31 smithi006 ceph-mon[29301]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:31.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:31 smithi117 ceph-mon[36514]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:32.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:08:32.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:32 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:08:33.368 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:33 smithi006 ceph-mon[29301]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:33.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:33 smithi117 ceph-mon[36514]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:34.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:34 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:34.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:34 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:34.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:34 smithi117 ceph-mon[36514]: Reconfiguring crash.smithi117 (monmap changed)... 2024-03-22T06:08:34.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:34 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:08:34.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:34 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:34.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:34 smithi117 ceph-mon[36514]: Reconfiguring daemon crash.smithi117 on smithi117 2024-03-22T06:08:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:34 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:34 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:34 smithi006 ceph-mon[29301]: Reconfiguring crash.smithi117 (monmap changed)... 2024-03-22T06:08:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:34 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:08:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:34 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:34 smithi006 ceph-mon[29301]: Reconfiguring daemon crash.smithi117 on smithi117 2024-03-22T06:08:35.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:35 smithi006 ceph-mon[29301]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:35.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:35.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:35.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:08:35.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:08:35.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:35.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T06:08:35.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:35.432 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:35 smithi117 ceph-mon[36514]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:35.432 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:35.433 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:35.433 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:08:35.433 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:08:35.433 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:35.433 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T06:08:35.433 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:36.307 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:36 smithi006 ceph-mon[29301]: Reconfiguring mgr.smithi117.bqeskk (monmap changed)... 2024-03-22T06:08:36.307 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:36 smithi006 ceph-mon[29301]: Reconfiguring daemon mgr.smithi117.bqeskk on smithi117 2024-03-22T06:08:36.307 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:36 smithi006 ceph-mon[29301]: Deploying daemon osd.0 on smithi006 2024-03-22T06:08:36.307 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:36 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:36.307 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:36 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:36.308 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:36 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:08:36.308 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:36 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:08:36.308 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:36 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:36.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:36 smithi117 ceph-mon[36514]: Reconfiguring mgr.smithi117.bqeskk (monmap changed)... 2024-03-22T06:08:36.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:36 smithi117 ceph-mon[36514]: Reconfiguring daemon mgr.smithi117.bqeskk on smithi117 2024-03-22T06:08:36.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:36 smithi117 ceph-mon[36514]: Deploying daemon osd.0 on smithi006 2024-03-22T06:08:36.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:36 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:36.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:36 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:36.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:36 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:08:36.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:36 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:08:36.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:36 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:37.415 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:37 smithi006 ceph-mon[29301]: Reconfiguring mon.smithi117 (monmap changed)... 2024-03-22T06:08:37.415 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:37 smithi006 ceph-mon[29301]: Reconfiguring daemon mon.smithi117 on smithi117 2024-03-22T06:08:37.415 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:37 smithi006 ceph-mon[29301]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:37.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:37 smithi117 ceph-mon[36514]: Reconfiguring mon.smithi117 (monmap changed)... 2024-03-22T06:08:37.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:37 smithi117 ceph-mon[36514]: Reconfiguring daemon mon.smithi117 on smithi117 2024-03-22T06:08:37.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:37 smithi117 ceph-mon[36514]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi006.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi006.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi006.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi006.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi006.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T06:08:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi006.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T06:08:38.534 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:38.534 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:08:38.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:38.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:38.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:08:38.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:08:38.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi006.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T06:08:38.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi006.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi006.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi006.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi006.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi006.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:38.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:08:39.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:39 smithi006 ceph-mon[29301]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:39.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:39.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:39.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:39 smithi117 ceph-mon[36514]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:39.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:39 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:39.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:39 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:41.717 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:41 smithi006 ceph-mon[29301]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:41.717 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:41 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:41.717 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:41 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:41.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:41 smithi117 ceph-mon[36514]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:41.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:41 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:41.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:41 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:43.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:43 smithi117 ceph-mon[36514]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:43 smithi006 ceph-mon[29301]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:44.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.531+0000 7f4f5bfff700 1 -- 172.21.15.6:0/2827650396 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4f64111bb0 con 0x7f4f50069210 2024-03-22T06:08:44.532 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 0 on host 'smithi006' 2024-03-22T06:08:44.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.533+0000 7f4f6a069700 1 -- 172.21.15.6:0/2827650396 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4f50069210 msgr2=0x7f4f5006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:44.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.533+0000 7f4f6a069700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4f50069210 0x7f4f5006b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00b970 tx=0x7f4f4c009e30 comp rx=0 tx=0).stop 2024-03-22T06:08:44.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.533+0000 7f4f6a069700 1 -- 172.21.15.6:0/2827650396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f64070870 msgr2=0x7f4f6410fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:44.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.533+0000 7f4f6a069700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f64070870 0x7f4f6410fc20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00cb10 tx=0x7f4f5c00ced0 comp rx=0 tx=0).stop 2024-03-22T06:08:44.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.533+0000 7f4f6a069700 1 -- 172.21.15.6:0/2827650396 shutdown_connections 2024-03-22T06:08:44.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.533+0000 7f4f6a069700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4f50069210 0x7f4f5006b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:44.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.533+0000 7f4f6a069700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f64070870 0x7f4f6410fc20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:44.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.533+0000 7f4f6a069700 1 --2- 172.21.15.6:0/2827650396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f64071e20 0x7f4f64110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:44.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.533+0000 7f4f6a069700 1 -- 172.21.15.6:0/2827650396 >> 172.21.15.6:0/2827650396 conn(0x7f4f640f85b0 msgr2=0x7f4f640f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:08:44.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.533+0000 7f4f6a069700 1 -- 172.21.15.6:0/2827650396 shutdown_connections 2024-03-22T06:08:44.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:44.533+0000 7f4f6a069700 1 -- 172.21.15.6:0/2827650396 wait complete. 2024-03-22T06:08:45.429 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:45 smithi006 ceph-mon[29301]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:45.429 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:45 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:45.429 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:45 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:45.429 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:45 smithi006 ceph-mon[29301]: from='osd.0 [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T06:08:45.452 DEBUG:teuthology.orchestra.run.smithi006:osd.0> sudo journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.0.service 2024-03-22T06:08:45.454 INFO:tasks.cephadm:Deploying osd.1 on smithi006 with /dev/vg_nvme/lv_3... 2024-03-22T06:08:45.455 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-22T06:08:45.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:45 smithi117 ceph-mon[36514]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:45.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:45 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:45.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:45 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:45.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:45 smithi117 ceph-mon[36514]: from='osd.0 [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T06:08:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:08:46.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:46 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:46.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:46 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:46.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:46 smithi117 ceph-mon[36514]: from='osd.0 [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T06:08:46.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:46 smithi117 ceph-mon[36514]: osdmap e7: 1 total, 0 up, 1 in 2024-03-22T06:08:46.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:46 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:46.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:46 smithi117 ceph-mon[36514]: from='osd.0 [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:08:46.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:46 smithi117 ceph-mon[36514]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:46.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:46 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:46.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:46 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:46.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:46 smithi006 ceph-mon[29301]: from='osd.0 [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T06:08:46.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:46 smithi006 ceph-mon[29301]: osdmap e7: 1 total, 0 up, 1 in 2024-03-22T06:08:46.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:46 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:46.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:46 smithi006 ceph-mon[29301]: from='osd.0 [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:08:46.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:46 smithi006 ceph-mon[29301]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:46.783 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:08:46 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[47983]: 2024-03-22T06:08:46.539+0000 7fe0bd6be700 -1 osd.0 0 waiting for initial osdmap 2024-03-22T06:08:46.783 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:08:46 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[47983]: 2024-03-22T06:08:46.542+0000 7fe0b64b5700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:08:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-22T06:08:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-22T06:08:47.726 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-03-22T06:08:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-03-22T06:08:47.726 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120162 s, 873 MB/s 2024-03-22T06:08:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-03-22T06:08:47.816 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:47 smithi006 ceph-mon[29301]: from='osd.0 [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-22T06:08:47.816 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:47 smithi006 ceph-mon[29301]: osdmap e8: 1 total, 0 up, 1 in 2024-03-22T06:08:47.817 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:47.817 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:47.817 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:08:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:47 smithi117 ceph-mon[36514]: from='osd.0 [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-22T06:08:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:47 smithi117 ceph-mon[36514]: osdmap e8: 1 total, 0 up, 1 in 2024-03-22T06:08:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:08:48.762 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:48 smithi006 ceph-mon[29301]: purged_snaps scrub starts 2024-03-22T06:08:48.762 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:48 smithi006 ceph-mon[29301]: purged_snaps scrub ok 2024-03-22T06:08:48.763 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:48 smithi006 ceph-mon[29301]: osd.0 [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022] boot 2024-03-22T06:08:48.763 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:48 smithi006 ceph-mon[29301]: osdmap e9: 1 total, 1 up, 1 in 2024-03-22T06:08:48.763 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:48.763 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:48.763 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:48 smithi006 ceph-mon[29301]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:49.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:48 smithi117 ceph-mon[36514]: purged_snaps scrub starts 2024-03-22T06:08:49.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:48 smithi117 ceph-mon[36514]: purged_snaps scrub ok 2024-03-22T06:08:49.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:48 smithi117 ceph-mon[36514]: osd.0 [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022] boot 2024-03-22T06:08:49.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:48 smithi117 ceph-mon[36514]: osdmap e9: 1 total, 1 up, 1 in 2024-03-22T06:08:49.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:48 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:49.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:48 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:08:49.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:48 smithi117 ceph-mon[36514]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T06:08:49.231 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi006:vg_nvme/lv_3 2024-03-22T06:08:49.748 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:08:49.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:49 smithi006 ceph-mon[29301]: osdmap e10: 1 total, 1 up, 1 in 2024-03-22T06:08:50.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:49 smithi117 ceph-mon[36514]: osdmap e10: 1 total, 1 up, 1 in 2024-03-22T06:08:50.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:50 smithi006 ceph-mon[29301]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:08:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:50 smithi117 ceph-mon[36514]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:08:51.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.960+0000 7f5eb9cff700 1 -- 172.21.15.6:0/1722288668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb40fe440 msgr2=0x7f5eb40fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:51.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.960+0000 7f5eb9cff700 1 --2- 172.21.15.6:0/1722288668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb40fe440 0x7f5eb40fe850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4004940 tx=0x7f5ea4009e30 comp rx=0 tx=0).stop 2024-03-22T06:08:51.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.960+0000 7f5eb9cff700 1 -- 172.21.15.6:0/1722288668 shutdown_connections 2024-03-22T06:08:51.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.960+0000 7f5eb9cff700 1 --2- 172.21.15.6:0/1722288668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eb40fed90 0x7f5eb40faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:51.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.960+0000 7f5eb9cff700 1 --2- 172.21.15.6:0/1722288668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb40fe440 0x7f5eb40fe850 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:51.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.960+0000 7f5eb9cff700 1 -- 172.21.15.6:0/1722288668 >> 172.21.15.6:0/1722288668 conn(0x7f5eb40f6be0 msgr2=0x7f5eb40f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:08:51.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.960+0000 7f5eb9cff700 1 -- 172.21.15.6:0/1722288668 shutdown_connections 2024-03-22T06:08:51.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.960+0000 7f5eb9cff700 1 -- 172.21.15.6:0/1722288668 wait complete. 2024-03-22T06:08:51.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.960+0000 7f5eb9cff700 1 Processor -- start 2024-03-22T06:08:51.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.960+0000 7f5eb9cff700 1 -- start start 2024-03-22T06:08:51.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.960+0000 7f5eb9cff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb40fe440 0x7f5eb410d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:51.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb9cff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eb40fed90 0x7f5eb410ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:51.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb9cff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb410e3f0 con 0x7f5eb40fe440 2024-03-22T06:08:51.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb9cff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb410e560 con 0x7f5eb40fed90 2024-03-22T06:08:51.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb40fe440 0x7f5eb410d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:51.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb40fe440 0x7f5eb410d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33394/0 (socket says 172.21.15.6:33394) 2024-03-22T06:08:51.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb37fe700 1 -- 172.21.15.6:0/2831500373 learned_addr learned my addr 172.21.15.6:0/2831500373 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:08:51.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb2ffd700 1 --2- 172.21.15.6:0/2831500373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eb40fed90 0x7f5eb410ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:51.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb37fe700 1 -- 172.21.15.6:0/2831500373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eb40fed90 msgr2=0x7f5eb410ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:08:51.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb37fe700 1 --2- 172.21.15.6:0/2831500373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eb40fed90 0x7f5eb410ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:08:51.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb37fe700 1 -- 172.21.15.6:0/2831500373 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea4005040 con 0x7f5eb40fe440 2024-03-22T06:08:51.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb37fe700 1 --2- 172.21.15.6:0/2831500373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb40fe440 0x7f5eb410d8b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4013870 tx=0x7f5ea4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:08:51.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5e9bfff700 1 -- 172.21.15.6:0/2831500373 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ea4021070 con 0x7f5eb40fe440 2024-03-22T06:08:51.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5e9bfff700 1 -- 172.21.15.6:0/2831500373 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ea401c400 con 0x7f5eb40fe440 2024-03-22T06:08:51.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb9cff700 1 -- 172.21.15.6:0/2831500373 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb4112f60 con 0x7f5eb40fe440 2024-03-22T06:08:51.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5e9bfff700 1 -- 172.21.15.6:0/2831500373 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ea400e070 con 0x7f5eb40fe440 2024-03-22T06:08:51.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.961+0000 7f5eb9cff700 1 -- 172.21.15.6:0/2831500373 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb4113410 con 0x7f5eb40fe440 2024-03-22T06:08:51.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.962+0000 7f5eb9cff700 1 -- 172.21.15.6:0/2831500373 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb404c710 con 0x7f5eb40fe440 2024-03-22T06:08:51.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.963+0000 7f5e9bfff700 1 -- 172.21.15.6:0/2831500373 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7f5ea401ca20 con 0x7f5eb40fe440 2024-03-22T06:08:51.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.963+0000 7f5e9bfff700 1 --2- 172.21.15.6:0/2831500373 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e9c069210 0x7f5e9c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:08:51.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.963+0000 7f5e9bfff700 1 -- 172.21.15.6:0/2831500373 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f5ea40875c0 con 0x7f5eb40fe440 2024-03-22T06:08:51.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.963+0000 7f5eb2ffd700 1 --2- 172.21.15.6:0/2831500373 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e9c069210 0x7f5e9c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:08:51.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.964+0000 7f5eb2ffd700 1 --2- 172.21.15.6:0/2831500373 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e9c069210 0x7f5e9c06b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8004600 tx=0x7f5ea800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:08:51.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:51.968+0000 7f5e9bfff700 1 -- 172.21.15.6:0/2831500373 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5ea40582a0 con 0x7f5eb40fe440 2024-03-22T06:08:52.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:08:52.178+0000 7f5eb9cff700 1 -- 172.21.15.6:0/2831500373 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5eb4113680 con 0x7f5e9c069210 2024-03-22T06:08:52.351 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:52 smithi006 ceph-mon[29301]: Detected new or changed devices on smithi006 2024-03-22T06:08:52.351 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:52 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:52.351 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:52 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:52.351 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:52 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:08:52.351 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:52 smithi006 ceph-mon[29301]: Adjusting osd_memory_target on smithi006 to 12869M 2024-03-22T06:08:52.351 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:52 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:52.351 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:52 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:52.351 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:52 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:08:52.351 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:52 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:52.351 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:52 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:08:52.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:52 smithi117 ceph-mon[36514]: Detected new or changed devices on smithi006 2024-03-22T06:08:52.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:52 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:52.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:52 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:52.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:52 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:08:52.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:52 smithi117 ceph-mon[36514]: Adjusting osd_memory_target on smithi006 to 12869M 2024-03-22T06:08:52.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:52 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:52.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:52 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:52.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:52 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:08:52.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:52 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:52.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:52 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:08:53.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:53 smithi006 ceph-mon[29301]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:08:53.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:53 smithi006 ceph-mon[29301]: from='client.14264 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:08:53.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:53 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:08:53.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:53 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:08:53.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:53 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:53.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:53 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:53.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:53 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:53.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:53 smithi117 ceph-mon[36514]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:08:53.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:53 smithi117 ceph-mon[36514]: from='client.14264 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:08:53.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:53 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:08:53.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:53 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:08:53.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:53 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:53.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:53 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:53.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:53 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:55.390 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:55 smithi006 ceph-mon[29301]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:08:55.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:55 smithi117 ceph-mon[36514]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:08:56.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:56 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3864404835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b55307-2c3b-4ab8-a893-af8072f333ea"}]: dispatch 2024-03-22T06:08:56.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:56 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3864404835' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1b55307-2c3b-4ab8-a893-af8072f333ea"}]': finished 2024-03-22T06:08:56.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:56 smithi117 ceph-mon[36514]: osdmap e11: 2 total, 1 up, 2 in 2024-03-22T06:08:56.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:56 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:08:56.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:56 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3864404835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b55307-2c3b-4ab8-a893-af8072f333ea"}]: dispatch 2024-03-22T06:08:56.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:56 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3864404835' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1b55307-2c3b-4ab8-a893-af8072f333ea"}]': finished 2024-03-22T06:08:56.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:56 smithi006 ceph-mon[29301]: osdmap e11: 2 total, 1 up, 2 in 2024-03-22T06:08:56.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:56 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:08:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:57 smithi117 ceph-mon[36514]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:08:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:57 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2066602178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:08:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:57 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:57 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:57 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:57 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:08:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:57 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:57 smithi006 ceph-mon[29301]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:08:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:57 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2066602178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:08:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:57 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:57 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:57 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:08:57.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:57 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:08:57.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:57 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:08:59.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:08:59 smithi117 ceph-mon[36514]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:08:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:08:59 smithi006 ceph-mon[29301]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:01.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:01 smithi117 ceph-mon[36514]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:01.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:01 smithi006 ceph-mon[29301]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:02.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:09:02.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:09:03.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:03 smithi006 ceph-mon[29301]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:03.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T06:09:03.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:03 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:03.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:03 smithi117 ceph-mon[36514]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:03.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T06:09:03.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:03 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:04.460 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:04 smithi006 ceph-mon[29301]: Deploying daemon osd.1 on smithi006 2024-03-22T06:09:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:04 smithi117 ceph-mon[36514]: Deploying daemon osd.1 on smithi006 2024-03-22T06:09:05.442 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:05 smithi006 ceph-mon[29301]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:05.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:05 smithi117 ceph-mon[36514]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:07.397 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:07 smithi006 ceph-mon[29301]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:07.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:07 smithi117 ceph-mon[36514]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:09.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:09 smithi006 ceph-mon[29301]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:09.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:09 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:09 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:09 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:09:09.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:09 smithi117 ceph-mon[36514]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:09.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:09 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:09.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:09 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:09.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:09 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:09:11.131 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:10 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:11.131 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:10 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:11.131 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:10 smithi006 ceph-mon[29301]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:11.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:10 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:11.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:10 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:11.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:10 smithi117 ceph-mon[36514]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:11.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.883+0000 7f5e9bfff700 1 -- 172.21.15.6:0/2831500373 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5eb4113680 con 0x7f5e9c069210 2024-03-22T06:09:11.884 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 1 on host 'smithi006' 2024-03-22T06:09:11.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.885+0000 7f5eb9cff700 1 -- 172.21.15.6:0/2831500373 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e9c069210 msgr2=0x7f5e9c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:09:11.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.885+0000 7f5eb9cff700 1 --2- 172.21.15.6:0/2831500373 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e9c069210 0x7f5e9c06b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8004600 tx=0x7f5ea800b040 comp rx=0 tx=0).stop 2024-03-22T06:09:11.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.886+0000 7f5eb9cff700 1 -- 172.21.15.6:0/2831500373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb40fe440 msgr2=0x7f5eb410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:09:11.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.886+0000 7f5eb9cff700 1 --2- 172.21.15.6:0/2831500373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb40fe440 0x7f5eb410d8b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4013870 tx=0x7f5ea4009e30 comp rx=0 tx=0).stop 2024-03-22T06:09:11.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.886+0000 7f5eb9cff700 1 -- 172.21.15.6:0/2831500373 shutdown_connections 2024-03-22T06:09:11.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.886+0000 7f5eb9cff700 1 --2- 172.21.15.6:0/2831500373 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5e9c069210 0x7f5e9c06b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:11.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.886+0000 7f5eb9cff700 1 --2- 172.21.15.6:0/2831500373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb40fe440 0x7f5eb410d8b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:11.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.886+0000 7f5eb9cff700 1 --2- 172.21.15.6:0/2831500373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5eb40fed90 0x7f5eb410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:11.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.886+0000 7f5eb9cff700 1 -- 172.21.15.6:0/2831500373 >> 172.21.15.6:0/2831500373 conn(0x7f5eb40f6be0 msgr2=0x7f5eb40f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:09:11.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.886+0000 7f5eb9cff700 1 -- 172.21.15.6:0/2831500373 shutdown_connections 2024-03-22T06:09:11.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:11.886+0000 7f5eb9cff700 1 -- 172.21.15.6:0/2831500373 wait complete. 2024-03-22T06:09:12.825 DEBUG:teuthology.orchestra.run.smithi006:osd.1> sudo journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.1.service 2024-03-22T06:09:12.827 INFO:tasks.cephadm:Deploying osd.2 on smithi006 with /dev/vg_nvme/lv_2... 2024-03-22T06:09:12.827 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-22T06:09:13.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:12 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:13.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:12 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:13.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:12 smithi006 ceph-mon[29301]: from='osd.1 [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T06:09:13.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:12 smithi006 ceph-mon[29301]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:12 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:12 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:12 smithi117 ceph-mon[36514]: from='osd.1 [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T06:09:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:12 smithi117 ceph-mon[36514]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:13.526 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:09:13.971 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:13 smithi006 ceph-mon[29301]: from='osd.1 [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T06:09:13.971 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:13 smithi006 ceph-mon[29301]: osdmap e12: 2 total, 1 up, 2 in 2024-03-22T06:09:13.971 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:13 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:09:13.971 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:13 smithi006 ceph-mon[29301]: from='osd.1 [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:09:13.971 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:09:13 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[55399]: 2024-03-22T06:09:13.849+0000 7f90f4784700 -1 osd.1 0 waiting for initial osdmap 2024-03-22T06:09:13.972 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:09:13 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[55399]: 2024-03-22T06:09:13.853+0000 7f90ef57f700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:09:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:13 smithi117 ceph-mon[36514]: from='osd.1 [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T06:09:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:13 smithi117 ceph-mon[36514]: osdmap e12: 2 total, 1 up, 2 in 2024-03-22T06:09:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:13 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:09:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:13 smithi117 ceph-mon[36514]: from='osd.1 [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:09:15.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:14 smithi006 ceph-mon[29301]: from='osd.1 [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-22T06:09:15.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:14 smithi006 ceph-mon[29301]: osdmap e13: 2 total, 1 up, 2 in 2024-03-22T06:09:15.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:14 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:09:15.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:14 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:09:15.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:14 smithi006 ceph-mon[29301]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:14 smithi117 ceph-mon[36514]: from='osd.1 [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-22T06:09:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:14 smithi117 ceph-mon[36514]: osdmap e13: 2 total, 1 up, 2 in 2024-03-22T06:09:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:14 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:09:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:14 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:09:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:14 smithi117 ceph-mon[36514]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T06:09:15.915 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:15 smithi117 ceph-mon[36514]: purged_snaps scrub starts 2024-03-22T06:09:15.915 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:15 smithi117 ceph-mon[36514]: purged_snaps scrub ok 2024-03-22T06:09:15.915 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:15 smithi117 ceph-mon[36514]: osd.1 [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829] boot 2024-03-22T06:09:15.916 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:15 smithi117 ceph-mon[36514]: osdmap e14: 2 total, 2 up, 2 in 2024-03-22T06:09:15.916 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:15 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:09:15.934 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:15 smithi006 ceph-mon[29301]: purged_snaps scrub starts 2024-03-22T06:09:15.934 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:15 smithi006 ceph-mon[29301]: purged_snaps scrub ok 2024-03-22T06:09:15.934 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:15 smithi006 ceph-mon[29301]: osd.1 [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829] boot 2024-03-22T06:09:15.934 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:15 smithi006 ceph-mon[29301]: osdmap e14: 2 total, 2 up, 2 in 2024-03-22T06:09:15.934 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:15 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:09:16.068 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-22T06:09:16.069 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-22T06:09:16.069 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-03-22T06:09:16.069 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-03-22T06:09:16.069 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0173636 s, 604 MB/s 2024-03-22T06:09:16.069 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-03-22T06:09:17.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:16 smithi006 ceph-mon[29301]: osdmap e15: 2 total, 2 up, 2 in 2024-03-22T06:09:17.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:16 smithi006 ceph-mon[29301]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:17.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:16 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:17.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:16 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:17.054 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi006:vg_nvme/lv_2 2024-03-22T06:09:17.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:16 smithi117 ceph-mon[36514]: osdmap e15: 2 total, 2 up, 2 in 2024-03-22T06:09:17.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:16 smithi117 ceph-mon[36514]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:17.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:16 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:17.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:16 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:18.238 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:09:18.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:09:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:09:19.170 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:18 smithi006 ceph-mon[29301]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:19.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:18 smithi117 ceph-mon[36514]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:21.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.002+0000 7f6a79cad700 1 -- 172.21.15.6:0/3070799369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a740fff70 msgr2=0x7f6a74102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:09:21.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.002+0000 7f6a79cad700 1 --2- 172.21.15.6:0/3070799369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a740fff70 0x7f6a74102350 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6a64004940 tx=0x7f6a64009e30 comp rx=0 tx=0).stop 2024-03-22T06:09:21.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.002+0000 7f6a79cad700 1 -- 172.21.15.6:0/3070799369 shutdown_connections 2024-03-22T06:09:21.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.002+0000 7f6a79cad700 1 --2- 172.21.15.6:0/3070799369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a740fff70 0x7f6a74102350 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:21.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.002+0000 7f6a79cad700 1 --2- 172.21.15.6:0/3070799369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a74066b10 0x7f6a740ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:21.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.002+0000 7f6a79cad700 1 -- 172.21.15.6:0/3070799369 >> 172.21.15.6:0/3070799369 conn(0x7f6a740f7480 msgr2=0x7f6a740f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.002+0000 7f6a79cad700 1 -- 172.21.15.6:0/3070799369 shutdown_connections 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.002+0000 7f6a79cad700 1 -- 172.21.15.6:0/3070799369 wait complete. 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.003+0000 7f6a79cad700 1 Processor -- start 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.003+0000 7f6a79cad700 1 -- start start 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.003+0000 7f6a79cad700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a74066b10 0x7f6a7410d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.003+0000 7f6a79cad700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a740fff70 0x7f6a7410df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.003+0000 7f6a79cad700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7410e520 con 0x7f6a740fff70 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.003+0000 7f6a79cad700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7410ed20 con 0x7f6a74066b10 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.003+0000 7f6a72ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a740fff70 0x7f6a7410df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.003+0000 7f6a72ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a740fff70 0x7f6a7410df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34872/0 (socket says 172.21.15.6:34872) 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.003+0000 7f6a72ffd700 1 -- 172.21.15.6:0/3229865812 learned_addr learned my addr 172.21.15.6:0/3229865812 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.004+0000 7f6a737fe700 1 --2- 172.21.15.6:0/3229865812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a74066b10 0x7f6a7410d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.004+0000 7f6a72ffd700 1 -- 172.21.15.6:0/3229865812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a74066b10 msgr2=0x7f6a7410d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:09:21.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.004+0000 7f6a72ffd700 1 --2- 172.21.15.6:0/3229865812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a74066b10 0x7f6a7410d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:21.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.004+0000 7f6a72ffd700 1 -- 172.21.15.6:0/3229865812 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64005040 con 0x7f6a740fff70 2024-03-22T06:09:21.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.004+0000 7f6a72ffd700 1 --2- 172.21.15.6:0/3229865812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a740fff70 0x7f6a7410df20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a64013870 tx=0x7f6a64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:09:21.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.004+0000 7f6a5bfff700 1 -- 172.21.15.6:0/3229865812 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a64021070 con 0x7f6a740fff70 2024-03-22T06:09:21.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.004+0000 7f6a5bfff700 1 -- 172.21.15.6:0/3229865812 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a64009430 con 0x7f6a740fff70 2024-03-22T06:09:21.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.004+0000 7f6a79cad700 1 -- 172.21.15.6:0/3229865812 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7410efa0 con 0x7f6a740fff70 2024-03-22T06:09:21.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.004+0000 7f6a5bfff700 1 -- 172.21.15.6:0/3229865812 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a6400e070 con 0x7f6a740fff70 2024-03-22T06:09:21.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.004+0000 7f6a79cad700 1 -- 172.21.15.6:0/3229865812 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7410f4e0 con 0x7f6a740fff70 2024-03-22T06:09:21.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.005+0000 7f6a79cad700 1 -- 172.21.15.6:0/3229865812 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7405f5f0 con 0x7f6a740fff70 2024-03-22T06:09:21.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.006+0000 7f6a5bfff700 1 -- 172.21.15.6:0/3229865812 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7f6a6401c750 con 0x7f6a740fff70 2024-03-22T06:09:21.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.007+0000 7f6a5bfff700 1 --2- 172.21.15.6:0/3229865812 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6a5c069210 0x7f6a5c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:09:21.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.007+0000 7f6a5bfff700 1 -- 172.21.15.6:0/3229865812 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f6a64087660 con 0x7f6a740fff70 2024-03-22T06:09:21.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.007+0000 7f6a737fe700 1 --2- 172.21.15.6:0/3229865812 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6a5c069210 0x7f6a5c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:09:21.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.007+0000 7f6a737fe700 1 --2- 172.21.15.6:0/3229865812 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6a5c069210 0x7f6a5c06b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009b80 tx=0x7f6a6800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:09:21.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.012+0000 7f6a5bfff700 1 -- 172.21.15.6:0/3229865812 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6a64058190 con 0x7f6a740fff70 2024-03-22T06:09:21.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:21.224+0000 7f6a79cad700 1 -- 172.21.15.6:0/3229865812 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a7410f7c0 con 0x7f6a5c069210 2024-03-22T06:09:21.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:21 smithi006 ceph-mon[29301]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:21.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:21.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:21.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:09:21.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:09:21.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:21.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:21.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:09:21.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:21.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:09:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:21 smithi117 ceph-mon[36514]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:09:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:09:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:09:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:09:22.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:22 smithi006 ceph-mon[29301]: Detected new or changed devices on smithi006 2024-03-22T06:09:22.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:22 smithi006 ceph-mon[29301]: Adjusting osd_memory_target on smithi006 to 6434M 2024-03-22T06:09:22.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:22 smithi006 ceph-mon[29301]: from='client.14282 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:09:22.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:22 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:09:22.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:22 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:09:22.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:22 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:22.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:22 smithi117 ceph-mon[36514]: Detected new or changed devices on smithi006 2024-03-22T06:09:22.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:22 smithi117 ceph-mon[36514]: Adjusting osd_memory_target on smithi006 to 6434M 2024-03-22T06:09:22.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:22 smithi117 ceph-mon[36514]: from='client.14282 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:09:22.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:22 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:09:22.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:22 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:09:22.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:22 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:23.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:23 smithi006 ceph-mon[29301]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:23.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:23 smithi117 ceph-mon[36514]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:25 smithi006 ceph-mon[29301]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:25 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2938041128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bacfe42-3c78-4d86-bac0-0142cde76ff5"}]: dispatch 2024-03-22T06:09:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:25 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2938041128' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bacfe42-3c78-4d86-bac0-0142cde76ff5"}]': finished 2024-03-22T06:09:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:25 smithi006 ceph-mon[29301]: osdmap e16: 3 total, 2 up, 3 in 2024-03-22T06:09:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:25 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:25 smithi117 ceph-mon[36514]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:25 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2938041128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bacfe42-3c78-4d86-bac0-0142cde76ff5"}]: dispatch 2024-03-22T06:09:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:25 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2938041128' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bacfe42-3c78-4d86-bac0-0142cde76ff5"}]': finished 2024-03-22T06:09:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:25 smithi117 ceph-mon[36514]: osdmap e16: 3 total, 2 up, 3 in 2024-03-22T06:09:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:25 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:26.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:26 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2907519750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:09:26.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:26 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2907519750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:09:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:27 smithi117 ceph-mon[36514]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:09:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:27 smithi006 ceph-mon[29301]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:09:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:29.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:29 smithi006 ceph-mon[29301]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:29 smithi117 ceph-mon[36514]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:31.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:31 smithi117 ceph-mon[36514]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:31.515 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:31 smithi006 ceph-mon[29301]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:32.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T06:09:32.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:32.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:09:32.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:32 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T06:09:32.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:32 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:32.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:32 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:09:33.460 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:33 smithi006 ceph-mon[29301]: Deploying daemon osd.2 on smithi006 2024-03-22T06:09:33.461 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:33 smithi006 ceph-mon[29301]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:33 smithi117 ceph-mon[36514]: Deploying daemon osd.2 on smithi006 2024-03-22T06:09:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:33 smithi117 ceph-mon[36514]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:35.455 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:35 smithi006 ceph-mon[29301]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:35.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:35 smithi117 ceph-mon[36514]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:37.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:37 smithi117 ceph-mon[36514]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:37.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:37 smithi006 ceph-mon[29301]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:38.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:38.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:38.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:09:38.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:38 smithi117 ceph-mon[36514]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:38.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:38.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:38.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:09:38.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:38 smithi006 ceph-mon[29301]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:41.416 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:41 smithi006 ceph-mon[29301]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:41.480 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 2 on host 'smithi006' 2024-03-22T06:09:41.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.479+0000 7f6a5bfff700 1 -- 172.21.15.6:0/3229865812 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6a7410f7c0 con 0x7f6a5c069210 2024-03-22T06:09:41.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.481+0000 7f6a79cad700 1 -- 172.21.15.6:0/3229865812 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6a5c069210 msgr2=0x7f6a5c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:09:41.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.481+0000 7f6a79cad700 1 --2- 172.21.15.6:0/3229865812 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6a5c069210 0x7f6a5c06b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009b80 tx=0x7f6a6800b040 comp rx=0 tx=0).stop 2024-03-22T06:09:41.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.482+0000 7f6a79cad700 1 -- 172.21.15.6:0/3229865812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a740fff70 msgr2=0x7f6a7410df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:09:41.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.482+0000 7f6a79cad700 1 --2- 172.21.15.6:0/3229865812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a740fff70 0x7f6a7410df20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a64013870 tx=0x7f6a64009e30 comp rx=0 tx=0).stop 2024-03-22T06:09:41.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.482+0000 7f6a79cad700 1 -- 172.21.15.6:0/3229865812 shutdown_connections 2024-03-22T06:09:41.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.482+0000 7f6a79cad700 1 --2- 172.21.15.6:0/3229865812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a74066b10 0x7f6a7410d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:41.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.482+0000 7f6a79cad700 1 --2- 172.21.15.6:0/3229865812 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6a5c069210 0x7f6a5c06b6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:41.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.482+0000 7f6a79cad700 1 --2- 172.21.15.6:0/3229865812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a740fff70 0x7f6a7410df20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:41.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.482+0000 7f6a79cad700 1 -- 172.21.15.6:0/3229865812 >> 172.21.15.6:0/3229865812 conn(0x7f6a740f7480 msgr2=0x7f6a740f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:09:41.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.482+0000 7f6a79cad700 1 -- 172.21.15.6:0/3229865812 shutdown_connections 2024-03-22T06:09:41.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:09:41.482+0000 7f6a79cad700 1 -- 172.21.15.6:0/3229865812 wait complete. 2024-03-22T06:09:41.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:41 smithi117 ceph-mon[36514]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:42.120 DEBUG:teuthology.orchestra.run.smithi006:osd.2> sudo journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.2.service 2024-03-22T06:09:42.122 INFO:tasks.cephadm:Deploying osd.3 on smithi117 with /dev/vg_nvme/lv_4... 2024-03-22T06:09:42.123 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-22T06:09:42.382 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi117/config 2024-03-22T06:09:42.459 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:42 smithi006 ceph-mon[29301]: from='osd.2 [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T06:09:42.460 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:42 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:42.460 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:42 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:42.504 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:42 smithi117 ceph-mon[36514]: from='osd.2 [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T06:09:42.504 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:42 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:42.504 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:42 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:43.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:43 smithi006 ceph-mon[29301]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:43.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:43 smithi006 ceph-mon[29301]: from='osd.2 [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T06:09:43.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:43 smithi006 ceph-mon[29301]: osdmap e17: 3 total, 2 up, 3 in 2024-03-22T06:09:43.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:43 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:43.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:43 smithi006 ceph-mon[29301]: from='osd.2 [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:09:43.283 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:09:43 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[63182]: 2024-03-22T06:09:43.198+0000 7fce17d33700 -1 osd.2 0 waiting for initial osdmap 2024-03-22T06:09:43.283 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:09:43 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[63182]: 2024-03-22T06:09:43.202+0000 7fce11b2c700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:09:43.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:43 smithi117 ceph-mon[36514]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:43.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:43 smithi117 ceph-mon[36514]: from='osd.2 [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T06:09:43.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:43 smithi117 ceph-mon[36514]: osdmap e17: 3 total, 2 up, 3 in 2024-03-22T06:09:43.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:43 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:43.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:43 smithi117 ceph-mon[36514]: from='osd.2 [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:09:44.125 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-22T06:09:44.126 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-22T06:09:44.126 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-03-22T06:09:44.126 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-03-22T06:09:44.126 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113913 s, 921 MB/s 2024-03-22T06:09:44.126 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 2024-03-22T06:09:44.126 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-03-22T06:09:44.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:44 smithi117 ceph-mon[36514]: from='osd.2 [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-22T06:09:44.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:44 smithi117 ceph-mon[36514]: osdmap e18: 3 total, 2 up, 3 in 2024-03-22T06:09:44.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:44 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:44.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:44 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:44.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:44 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:44.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:44 smithi006 ceph-mon[29301]: from='osd.2 [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-22T06:09:44.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:44 smithi006 ceph-mon[29301]: osdmap e18: 3 total, 2 up, 3 in 2024-03-22T06:09:44.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:44 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:44.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:44 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:44.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:44 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:44.720 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-03-22T06:09:44.969 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi117/config 2024-03-22T06:09:45.365 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:45 smithi117 ceph-mon[36514]: purged_snaps scrub starts 2024-03-22T06:09:45.365 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:45 smithi117 ceph-mon[36514]: purged_snaps scrub ok 2024-03-22T06:09:45.365 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:45 smithi117 ceph-mon[36514]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:45.366 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:45 smithi117 ceph-mon[36514]: osd.2 [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022] boot 2024-03-22T06:09:45.366 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:45 smithi117 ceph-mon[36514]: osdmap e19: 3 total, 3 up, 3 in 2024-03-22T06:09:45.366 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:45 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:45.366 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:45 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:45.366 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:45 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:45.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:45 smithi006 ceph-mon[29301]: purged_snaps scrub starts 2024-03-22T06:09:45.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:45 smithi006 ceph-mon[29301]: purged_snaps scrub ok 2024-03-22T06:09:45.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:45 smithi006 ceph-mon[29301]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T06:09:45.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:45 smithi006 ceph-mon[29301]: osd.2 [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022] boot 2024-03-22T06:09:45.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:45 smithi006 ceph-mon[29301]: osdmap e19: 3 total, 3 up, 3 in 2024-03-22T06:09:45.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:45 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:09:45.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:45 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:45.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:45 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:46.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.531+0000 7f5c54b13700 1 -- 172.21.15.117:0/3833015113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c500fff80 msgr2=0x7f5c50102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.531+0000 7f5c54b13700 1 --2- 172.21.15.117:0/3833015113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c500fff80 0x7f5c50102360 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5c44004940 tx=0x7f5c44009e30 comp rx=0 tx=0).stop 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.531+0000 7f5c54b13700 1 -- 172.21.15.117:0/3833015113 shutdown_connections 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.531+0000 7f5c54b13700 1 --2- 172.21.15.117:0/3833015113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c500fff80 0x7f5c50102360 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.531+0000 7f5c54b13700 1 --2- 172.21.15.117:0/3833015113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c50066260 0x7f5c500ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.532+0000 7f5c54b13700 1 -- 172.21.15.117:0/3833015113 >> 172.21.15.117:0/3833015113 conn(0x7f5c500f7430 msgr2=0x7f5c500f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.532+0000 7f5c54b13700 1 -- 172.21.15.117:0/3833015113 shutdown_connections 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.532+0000 7f5c54b13700 1 -- 172.21.15.117:0/3833015113 wait complete. 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.532+0000 7f5c54b13700 1 Processor -- start 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.532+0000 7f5c54b13700 1 -- start start 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.532+0000 7f5c54b13700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c50066260 0x7f5c500fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.532+0000 7f5c54b13700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c500fff80 0x7f5c500fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.532+0000 7f5c54b13700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500fbe80 con 0x7f5c50066260 2024-03-22T06:09:46.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.532+0000 7f5c54b13700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500fbfc0 con 0x7f5c500fff80 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.532+0000 7f5c4dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c500fff80 0x7f5c500fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c4dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c500fff80 0x7f5c500fb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55104/0 (socket says 172.21.15.117:55104) 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c4dd9b700 1 -- 172.21.15.117:0/2464365141 learned_addr learned my addr 172.21.15.117:0/2464365141 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c4e59c700 1 --2- 172.21.15.117:0/2464365141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c50066260 0x7f5c500fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c4dd9b700 1 -- 172.21.15.117:0/2464365141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c50066260 msgr2=0x7f5c500fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c4dd9b700 1 --2- 172.21.15.117:0/2464365141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c50066260 0x7f5c500fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c4dd9b700 1 -- 172.21.15.117:0/2464365141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44005040 con 0x7f5c500fff80 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c4dd9b700 1 --2- 172.21.15.117:0/2464365141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c500fff80 0x7f5c500fb940 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5c44013870 tx=0x7f5c44009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c3effd700 1 -- 172.21.15.117:0/2464365141 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c44021070 con 0x7f5c500fff80 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c3effd700 1 -- 172.21.15.117:0/2464365141 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c44009430 con 0x7f5c500fff80 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c3effd700 1 -- 172.21.15.117:0/2464365141 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c4400e070 con 0x7f5c500fff80 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c54b13700 1 -- 172.21.15.117:0/2464365141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c500fc240 con 0x7f5c500fff80 2024-03-22T06:09:46.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.533+0000 7f5c54b13700 1 -- 172.21.15.117:0/2464365141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c500fc780 con 0x7f5c500fff80 2024-03-22T06:09:46.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.534+0000 7f5c54b13700 1 -- 172.21.15.117:0/2464365141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5005f5f0 con 0x7f5c500fff80 2024-03-22T06:09:46.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.534+0000 7f5c3effd700 1 -- 172.21.15.117:0/2464365141 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7f5c4401c750 con 0x7f5c500fff80 2024-03-22T06:09:46.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.535+0000 7f5c3effd700 1 --2- 172.21.15.117:0/2464365141 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5c38069210 0x7f5c3806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:09:46.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.535+0000 7f5c3effd700 1 -- 172.21.15.117:0/2464365141 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3217+0+0 (secure 0 0 0) 0x7f5c440879d0 con 0x7f5c500fff80 2024-03-22T06:09:46.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.535+0000 7f5c4e59c700 1 --2- 172.21.15.117:0/2464365141 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5c38069210 0x7f5c3806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:09:46.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.535+0000 7f5c4e59c700 1 --2- 172.21.15.117:0/2464365141 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5c38069210 0x7f5c3806b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c500fd160 tx=0x7f5c4000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:09:46.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.539+0000 7f5c3effd700 1 -- 172.21.15.117:0/2464365141 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5c44054740 con 0x7f5c500fff80 2024-03-22T06:09:46.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:46.732+0000 7f5c54b13700 1 -- 172.21.15.117:0/2464365141 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c500f8f70 con 0x7f5c38069210 2024-03-22T06:09:46.733 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:46 smithi117 ceph-mon[36514]: osdmap e20: 3 total, 3 up, 3 in 2024-03-22T06:09:46.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:46 smithi006 ceph-mon[29301]: osdmap e20: 3 total, 3 up, 3 in 2024-03-22T06:09:47.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: pgmap v75: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:47.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: osdmap e21: 3 total, 3 up, 3 in 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:09:47.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:47.534 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:47.534 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:09:47.534 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:47.534 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:09:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: pgmap v75: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-22T06:09:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-22T06:09:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: osdmap e21: 3 total, 3 up, 3 in 2024-03-22T06:09:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T06:09:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:09:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:09:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:09:47.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:47.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:47.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:09:47.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:09:47.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:09:47.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:47.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:47.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:09:47.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:47.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:09:48.457 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:48 smithi117 ceph-mon[36514]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:09:48.457 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:48 smithi117 ceph-mon[36514]: Detected new or changed devices on smithi006 2024-03-22T06:09:48.457 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:48 smithi117 ceph-mon[36514]: Adjusting osd_memory_target on smithi006 to 4289M 2024-03-22T06:09:48.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:48 smithi006 ceph-mon[29301]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:09:48.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:48 smithi006 ceph-mon[29301]: Detected new or changed devices on smithi006 2024-03-22T06:09:48.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:48 smithi006 ceph-mon[29301]: Adjusting osd_memory_target on smithi006 to 4289M 2024-03-22T06:09:48.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-22T06:09:48.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:48 smithi006 ceph-mon[29301]: osdmap e22: 3 total, 3 up, 3 in 2024-03-22T06:09:48.533 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67512]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T06:09:48.533 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67512]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T06:09:48.533 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67512]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T06:09:48.533 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67512]: pam_unix(sudo:session): session closed for user root 2024-03-22T06:09:48.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:48 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-22T06:09:48.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:48 smithi117 ceph-mon[36514]: osdmap e22: 3 total, 3 up, 3 in 2024-03-22T06:09:48.814 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67602]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T06:09:48.814 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67602]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T06:09:48.814 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67602]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T06:09:48.814 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67602]: pam_unix(sudo:session): session closed for user root 2024-03-22T06:09:49.083 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67625]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T06:09:49.083 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67625]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T06:09:49.083 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67625]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T06:09:49.083 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:09:48 smithi006 sudo[67625]: pam_unix(sudo:session): session closed for user root 2024-03-22T06:09:49.342 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67630]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T06:09:49.342 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67630]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T06:09:49.342 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67630]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T06:09:49.343 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67630]: pam_unix(sudo:session): session closed for user root 2024-03-22T06:09:49.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:49 smithi117 ceph-mon[36514]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:49.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:49 smithi117 ceph-mon[36514]: osdmap e23: 3 total, 3 up, 3 in 2024-03-22T06:09:49.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:49 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:49.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:49 smithi117 ceph-mon[36514]: from='client.? 172.21.15.117:0/3505686143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5a6acd-de41-455d-b6c5-d53bb76197ba"}]: dispatch 2024-03-22T06:09:49.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:49 smithi117 ceph-mon[36514]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5a6acd-de41-455d-b6c5-d53bb76197ba"}]: dispatch 2024-03-22T06:09:49.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:49 smithi117 ceph-mon[36514]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c5a6acd-de41-455d-b6c5-d53bb76197ba"}]': finished 2024-03-22T06:09:49.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:49 smithi117 ceph-mon[36514]: osdmap e24: 4 total, 3 up, 4 in 2024-03-22T06:09:49.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:49 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:09:49.605 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67680]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T06:09:49.605 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67680]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T06:09:49.605 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67680]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T06:09:49.605 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67680]: pam_unix(sudo:session): session closed for user root 2024-03-22T06:09:49.605 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 ceph-mon[29301]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:49.605 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 ceph-mon[29301]: osdmap e23: 3 total, 3 up, 3 in 2024-03-22T06:09:49.605 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:49.605 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/3505686143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5a6acd-de41-455d-b6c5-d53bb76197ba"}]: dispatch 2024-03-22T06:09:49.605 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 ceph-mon[29301]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5a6acd-de41-455d-b6c5-d53bb76197ba"}]: dispatch 2024-03-22T06:09:49.605 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 ceph-mon[29301]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c5a6acd-de41-455d-b6c5-d53bb76197ba"}]': finished 2024-03-22T06:09:49.606 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 ceph-mon[29301]: osdmap e24: 4 total, 3 up, 4 in 2024-03-22T06:09:49.606 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:09:49.877 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67711]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T06:09:49.877 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67711]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T06:09:49.877 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67711]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T06:09:49.877 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67711]: pam_unix(sudo:session): session closed for user root 2024-03-22T06:09:50.150 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67714]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T06:09:50.150 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T06:09:50.150 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67714]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T06:09:50.150 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:49 smithi006 sudo[67714]: pam_unix(sudo:session): session closed for user root 2024-03-22T06:09:50.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 sudo[67764]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T06:09:50.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 sudo[67764]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T06:09:50.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 sudo[67764]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T06:09:50.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 sudo[67764]: pam_unix(sudo:session): session closed for user root 2024-03-22T06:09:50.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 sudo[40270]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T06:09:50.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 sudo[40270]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T06:09:50.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 sudo[40270]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T06:09:50.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 sudo[40270]: pam_unix(sudo:session): session closed for user root 2024-03-22T06:09:50.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:09:50.671+0000 7f5c3effd700 1 -- 172.21.15.117:0/2464365141 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f5c4401c400 con 0x7f5c500fff80 2024-03-22T06:09:51.002 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/233683018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:09:51.003 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 ceph-mon[29301]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T06:09:51.003 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 ceph-mon[29301]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T06:09:51.003 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:09:51.003 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:09:51.003 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:09:51.003 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:09:51.003 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:50 smithi006 ceph-mon[29301]: mgrmap e19: smithi006.sfpxdy(active, since 2m), standbys: smithi117.bqeskk 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 sudo[40457]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 sudo[40457]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 sudo[40457]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 sudo[40457]: pam_unix(sudo:session): session closed for user root 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 ceph-mon[36514]: from='client.? 172.21.15.117:0/233683018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 ceph-mon[36514]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 ceph-mon[36514]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:09:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:50 smithi117 ceph-mon[36514]: mgrmap e19: smithi006.sfpxdy(active, since 2m), standbys: smithi117.bqeskk 2024-03-22T06:09:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:51 smithi117 ceph-mon[36514]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:51 smithi117 ceph-mon[36514]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T06:09:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:51 smithi117 ceph-mon[36514]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T06:09:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:51 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:51 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:51 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:51 smithi006 ceph-mon[29301]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:51 smithi006 ceph-mon[29301]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T06:09:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:51 smithi006 ceph-mon[29301]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T06:09:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:51 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:51 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:51 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:53.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:52 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:53.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:52 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:09:53.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:52 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:53.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:52 smithi117 ceph-mon[36514]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:53.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:52 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:53.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:52 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:09:53.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:52 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:09:53.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:52 smithi006 ceph-mon[29301]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:55.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:55 smithi117 ceph-mon[36514]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:55.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:55 smithi006 ceph-mon[29301]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:56.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:56 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T06:09:56.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:56 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:56.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:56 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T06:09:56.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:56 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:09:57.205 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:57 smithi117 ceph-mon[36514]: Deploying daemon osd.3 on smithi117 2024-03-22T06:09:57.205 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:57 smithi117 ceph-mon[36514]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:57 smithi006 ceph-mon[29301]: Deploying daemon osd.3 on smithi117 2024-03-22T06:09:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:57 smithi006 ceph-mon[29301]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:59.300 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:09:59 smithi117 ceph-mon[36514]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:09:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:09:59 smithi006 ceph-mon[29301]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:10:00.444 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:00 smithi117 ceph-mon[36514]: overall HEALTH_OK 2024-03-22T06:10:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:00 smithi006 ceph-mon[29301]: overall HEALTH_OK 2024-03-22T06:10:01.500 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:01 smithi117 ceph-mon[36514]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:10:01.501 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:01.501 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:01.501 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:10:01.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:01 smithi006 ceph-mon[29301]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:10:01.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:01.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:01.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:10:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:10:02.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:10:03.358 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:03 smithi117 ceph-mon[36514]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:10:03.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:03 smithi006 ceph-mon[29301]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:10:04.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.543+0000 7f5c3effd700 1 -- 172.21.15.117:0/2464365141 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5c500f8f70 con 0x7f5c38069210 2024-03-22T06:10:04.545 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 3 on host 'smithi117' 2024-03-22T06:10:04.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.546+0000 7f5c54b13700 1 -- 172.21.15.117:0/2464365141 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5c38069210 msgr2=0x7f5c3806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:04.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.546+0000 7f5c54b13700 1 --2- 172.21.15.117:0/2464365141 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5c38069210 0x7f5c3806b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c500fd160 tx=0x7f5c4000b040 comp rx=0 tx=0).stop 2024-03-22T06:10:04.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.546+0000 7f5c54b13700 1 -- 172.21.15.117:0/2464365141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c500fff80 msgr2=0x7f5c500fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:04.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.546+0000 7f5c54b13700 1 --2- 172.21.15.117:0/2464365141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c500fff80 0x7f5c500fb940 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5c44013870 tx=0x7f5c44009e30 comp rx=0 tx=0).stop 2024-03-22T06:10:04.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.546+0000 7f5c54b13700 1 -- 172.21.15.117:0/2464365141 shutdown_connections 2024-03-22T06:10:04.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.546+0000 7f5c54b13700 1 --2- 172.21.15.117:0/2464365141 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5c38069210 0x7f5c3806b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:04.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.546+0000 7f5c54b13700 1 --2- 172.21.15.117:0/2464365141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c50066260 0x7f5c500fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:04.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.546+0000 7f5c54b13700 1 --2- 172.21.15.117:0/2464365141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c500fff80 0x7f5c500fb940 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:04.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.546+0000 7f5c54b13700 1 -- 172.21.15.117:0/2464365141 >> 172.21.15.117:0/2464365141 conn(0x7f5c500f7430 msgr2=0x7f5c500f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:10:04.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.546+0000 7f5c54b13700 1 -- 172.21.15.117:0/2464365141 shutdown_connections 2024-03-22T06:10:04.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:04.546+0000 7f5c54b13700 1 -- 172.21.15.117:0/2464365141 wait complete. 2024-03-22T06:10:05.160 DEBUG:teuthology.orchestra.run.smithi117:osd.3> sudo journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.3.service 2024-03-22T06:10:05.162 INFO:tasks.cephadm:Deploying osd.4 on smithi117 with /dev/vg_nvme/lv_3... 2024-03-22T06:10:05.163 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-22T06:10:05.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:05 smithi117 ceph-mon[36514]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:10:05.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:05 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:05.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:05 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:05.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:05 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:05.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:05 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:05.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:05 smithi117 ceph-mon[36514]: from='osd.3 [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T06:10:05.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:05 smithi117 ceph-mon[36514]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T06:10:05.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:05 smithi006 ceph-mon[29301]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:10:05.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:05 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:05.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:05 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:05.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:05 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:05.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:05 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:05.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:05 smithi006 ceph-mon[29301]: from='osd.3 [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T06:10:05.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:05 smithi006 ceph-mon[29301]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T06:10:06.318 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi117/config 2024-03-22T06:10:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:06 smithi117 ceph-mon[36514]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T06:10:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:06 smithi117 ceph-mon[36514]: osdmap e25: 4 total, 3 up, 4 in 2024-03-22T06:10:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:06 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:10:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:06 smithi117 ceph-mon[36514]: from='osd.3 [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:06 smithi117 ceph-mon[36514]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:06 smithi117 ceph-mon[36514]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:10:06.756 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:10:06 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[42097]: 2024-03-22T06:10:06.552+0000 7fb0791d7700 -1 osd.3 0 waiting for initial osdmap 2024-03-22T06:10:06.757 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:10:06 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[42097]: 2024-03-22T06:10:06.559+0000 7fb073fd2700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:10:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:06 smithi006 ceph-mon[29301]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T06:10:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:06 smithi006 ceph-mon[29301]: osdmap e25: 4 total, 3 up, 4 in 2024-03-22T06:10:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:06 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:10:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:06 smithi006 ceph-mon[29301]: from='osd.3 [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:07.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:06 smithi006 ceph-mon[29301]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:07.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:06 smithi006 ceph-mon[29301]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T06:10:07.821 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:07 smithi117 ceph-mon[36514]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-22T06:10:07.821 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:07 smithi117 ceph-mon[36514]: osdmap e26: 4 total, 3 up, 4 in 2024-03-22T06:10:07.822 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:07 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:10:07.822 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:07 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:10:07.822 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:07 smithi117 ceph-mon[36514]: osd.3 [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526] boot 2024-03-22T06:10:07.822 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:07 smithi117 ceph-mon[36514]: osdmap e27: 4 total, 4 up, 4 in 2024-03-22T06:10:07.822 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:07 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:10:07.920 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:07 smithi006 ceph-mon[29301]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-22T06:10:07.920 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:07 smithi006 ceph-mon[29301]: osdmap e26: 4 total, 3 up, 4 in 2024-03-22T06:10:07.920 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:07 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:10:07.920 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:07 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:10:07.920 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:07 smithi006 ceph-mon[29301]: osd.3 [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526] boot 2024-03-22T06:10:07.920 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:07 smithi006 ceph-mon[29301]: osdmap e27: 4 total, 4 up, 4 in 2024-03-22T06:10:07.920 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:07 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:10:08.765 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: purged_snaps scrub starts 2024-03-22T06:10:08.765 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: purged_snaps scrub ok 2024-03-22T06:10:08.765 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:08.765 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: Detected new or changed devices on smithi117 2024-03-22T06:10:08.765 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:08.765 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:08.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:10:08.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: Adjusting osd_memory_target on smithi117 to 15936M 2024-03-22T06:10:08.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:08.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:08.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:10:08.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:08.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:10:08.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:08.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:10:08.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:08.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:08 smithi117 ceph-mon[36514]: osdmap e28: 4 total, 4 up, 4 in 2024-03-22T06:10:09.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: purged_snaps scrub starts 2024-03-22T06:10:09.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: purged_snaps scrub ok 2024-03-22T06:10:09.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:09.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: Detected new or changed devices on smithi117 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: Adjusting osd_memory_target on smithi117 to 15936M 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:09.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:08 smithi006 ceph-mon[29301]: osdmap e28: 4 total, 4 up, 4 in 2024-03-22T06:10:09.200 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-22T06:10:09.200 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-22T06:10:09.200 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-03-22T06:10:09.200 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-03-22T06:10:09.200 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134855 s, 778 MB/s 2024-03-22T06:10:09.200 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-03-22T06:10:09.806 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-03-22T06:10:10.054 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi117/config 2024-03-22T06:10:10.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:10 smithi117 ceph-mon[36514]: osdmap e29: 4 total, 4 up, 4 in 2024-03-22T06:10:10.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:10 smithi006 ceph-mon[29301]: osdmap e29: 4 total, 4 up, 4 in 2024-03-22T06:10:11.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.422+0000 7fdd990e0700 1 -- 172.21.15.117:0/4262147148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd940fdfe0 msgr2=0x7fdd940fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:11.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.422+0000 7fdd990e0700 1 --2- 172.21.15.117:0/4262147148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd940fdfe0 0x7fdd940fe450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdd84004940 tx=0x7fdd84009e30 comp rx=0 tx=0).stop 2024-03-22T06:10:11.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.422+0000 7fdd990e0700 1 -- 172.21.15.117:0/4262147148 shutdown_connections 2024-03-22T06:10:11.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.422+0000 7fdd990e0700 1 --2- 172.21.15.117:0/4262147148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd940fdfe0 0x7fdd940fe450 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:11.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.422+0000 7fdd990e0700 1 --2- 172.21.15.117:0/4262147148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd940fce70 0x7fdd940fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.422+0000 7fdd990e0700 1 -- 172.21.15.117:0/4262147148 >> 172.21.15.117:0/4262147148 conn(0x7fdd940f8550 msgr2=0x7fdd940fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.422+0000 7fdd990e0700 1 -- 172.21.15.117:0/4262147148 shutdown_connections 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.422+0000 7fdd990e0700 1 -- 172.21.15.117:0/4262147148 wait complete. 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd990e0700 1 Processor -- start 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd990e0700 1 -- start start 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd990e0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd940fce70 0x7fdd9410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd990e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd940fdfe0 0x7fdd9410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd990e0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9410e460 con 0x7fdd940fdfe0 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd990e0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9410e5d0 con 0x7fdd940fce70 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd92d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd940fce70 0x7fdd9410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd92d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd940fce70 0x7fdd9410d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50696/0 (socket says 172.21.15.117:50696) 2024-03-22T06:10:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd92d9d700 1 -- 172.21.15.117:0/1445475833 learned_addr learned my addr 172.21.15.117:0/1445475833 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:10:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd9259c700 1 --2- 172.21.15.117:0/1445475833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd940fdfe0 0x7fdd9410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:10:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd92d9d700 1 -- 172.21.15.117:0/1445475833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd940fdfe0 msgr2=0x7fdd9410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd92d9d700 1 --2- 172.21.15.117:0/1445475833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd940fdfe0 0x7fdd9410de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.423+0000 7fdd92d9d700 1 -- 172.21.15.117:0/1445475833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84005040 con 0x7fdd940fce70 2024-03-22T06:10:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.424+0000 7fdd92d9d700 1 --2- 172.21.15.117:0/1445475833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd940fce70 0x7fdd9410d920 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd88002a50 tx=0x7fdd880046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:10:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.424+0000 7fdd7b7fe700 1 -- 172.21.15.117:0/1445475833 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd88007cc0 con 0x7fdd940fce70 2024-03-22T06:10:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.424+0000 7fdd7b7fe700 1 -- 172.21.15.117:0/1445475833 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd8800f040 con 0x7fdd940fce70 2024-03-22T06:10:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.424+0000 7fdd990e0700 1 -- 172.21.15.117:0/1445475833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd94112fd0 con 0x7fdd940fce70 2024-03-22T06:10:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.424+0000 7fdd7b7fe700 1 -- 172.21.15.117:0/1445475833 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd8800da30 con 0x7fdd940fce70 2024-03-22T06:10:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.424+0000 7fdd990e0700 1 -- 172.21.15.117:0/1445475833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd94113560 con 0x7fdd940fce70 2024-03-22T06:10:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.425+0000 7fdd990e0700 1 -- 172.21.15.117:0/1445475833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9404c710 con 0x7fdd940fce70 2024-03-22T06:10:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.425+0000 7fdd7b7fe700 1 -- 172.21.15.117:0/1445475833 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fdd8800db90 con 0x7fdd940fce70 2024-03-22T06:10:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.426+0000 7fdd7b7fe700 1 --2- 172.21.15.117:0/1445475833 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdd7c0692a0 0x7fdd7c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:10:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.426+0000 7fdd7b7fe700 1 -- 172.21.15.117:0/1445475833 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7fdd880868a0 con 0x7fdd940fce70 2024-03-22T06:10:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.426+0000 7fdd9259c700 1 --2- 172.21.15.117:0/1445475833 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdd7c0692a0 0x7fdd7c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:10:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.426+0000 7fdd9259c700 1 --2- 172.21.15.117:0/1445475833 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdd7c0692a0 0x7fdd7c06b750 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400b970 tx=0x7fdd84009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:10:11.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.430+0000 7fdd7b7fe700 1 -- 172.21.15.117:0/1445475833 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdd88056c80 con 0x7fdd940fce70 2024-03-22T06:10:11.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:11.622+0000 7fdd990e0700 1 -- 172.21.15.117:0/1445475833 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd94113750 con 0x7fdd7c0692a0 2024-03-22T06:10:11.623 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:11 smithi117 ceph-mon[36514]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:11.623 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:11 smithi117 ceph-mon[36514]: osdmap e30: 4 total, 4 up, 4 in 2024-03-22T06:10:11.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:11 smithi006 ceph-mon[29301]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:11.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:11 smithi006 ceph-mon[29301]: osdmap e30: 4 total, 4 up, 4 in 2024-03-22T06:10:12.696 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:12 smithi117 ceph-mon[36514]: osdmap e31: 4 total, 4 up, 4 in 2024-03-22T06:10:12.696 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:12 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:10:12.696 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:12 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:10:12.696 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:12 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:12.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:12 smithi006 ceph-mon[29301]: osdmap e31: 4 total, 4 up, 4 in 2024-03-22T06:10:12.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:12 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:10:12.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:12 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:10:12.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:12 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:13.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:13 smithi117 ceph-mon[36514]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:10:13.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:13 smithi117 ceph-mon[36514]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-22T06:10:13.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:13 smithi006 ceph-mon[29301]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:10:13.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:13 smithi006 ceph-mon[29301]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-22T06:10:14.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:14 smithi117 ceph-mon[36514]: from='client.? 172.21.15.117:0/1672255688' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be424041-1e07-41f4-abe0-68fdd21b1eb7"}]: dispatch 2024-03-22T06:10:14.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:14 smithi117 ceph-mon[36514]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be424041-1e07-41f4-abe0-68fdd21b1eb7"}]: dispatch 2024-03-22T06:10:14.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:14 smithi117 ceph-mon[36514]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be424041-1e07-41f4-abe0-68fdd21b1eb7"}]': finished 2024-03-22T06:10:14.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:14 smithi117 ceph-mon[36514]: osdmap e32: 5 total, 4 up, 5 in 2024-03-22T06:10:14.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:14 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:10:14.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:14 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/1672255688' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be424041-1e07-41f4-abe0-68fdd21b1eb7"}]: dispatch 2024-03-22T06:10:14.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:14 smithi006 ceph-mon[29301]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be424041-1e07-41f4-abe0-68fdd21b1eb7"}]: dispatch 2024-03-22T06:10:14.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:14 smithi006 ceph-mon[29301]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be424041-1e07-41f4-abe0-68fdd21b1eb7"}]': finished 2024-03-22T06:10:14.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:14 smithi006 ceph-mon[29301]: osdmap e32: 5 total, 4 up, 5 in 2024-03-22T06:10:14.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:14 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:10:15.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:15 smithi117 ceph-mon[36514]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-22T06:10:15.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:15 smithi117 ceph-mon[36514]: from='client.? 172.21.15.117:0/3221192603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:10:15.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:15 smithi006 ceph-mon[29301]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-22T06:10:15.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:15 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/3221192603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:10:17.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:17 smithi117 ceph-mon[36514]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-22T06:10:17.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:10:17.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:17 smithi006 ceph-mon[29301]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-22T06:10:17.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:10:19.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:19 smithi117 ceph-mon[36514]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-22T06:10:19.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:19 smithi006 ceph-mon[29301]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-22T06:10:21.733 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:21 smithi117 ceph-mon[36514]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:21.733 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T06:10:21.733 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:21.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:21 smithi006 ceph-mon[29301]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:21.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T06:10:21.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:22.716 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:22 smithi117 ceph-mon[36514]: Deploying daemon osd.4 on smithi117 2024-03-22T06:10:22.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:22 smithi006 ceph-mon[29301]: Deploying daemon osd.4 on smithi117 2024-03-22T06:10:23.689 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:23 smithi117 ceph-mon[36514]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:23.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:23 smithi006 ceph-mon[29301]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:25.579 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:25 smithi117 ceph-mon[36514]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:25.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:25 smithi006 ceph-mon[29301]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:27.683 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:27 smithi117 ceph-mon[36514]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:27.684 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:27.684 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:27.684 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:10:27.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:27 smithi006 ceph-mon[29301]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:27.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:27.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:27.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:10:29.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:29 smithi117 ceph-mon[36514]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:29.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:29 smithi006 ceph-mon[29301]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:30.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.232+0000 7fdd7b7fe700 1 -- 172.21.15.117:0/1445475833 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdd94113750 con 0x7fdd7c0692a0 2024-03-22T06:10:30.233 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 4 on host 'smithi117' 2024-03-22T06:10:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.234+0000 7fdd990e0700 1 -- 172.21.15.117:0/1445475833 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdd7c0692a0 msgr2=0x7fdd7c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.234+0000 7fdd990e0700 1 --2- 172.21.15.117:0/1445475833 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdd7c0692a0 0x7fdd7c06b750 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400b970 tx=0x7fdd84009e30 comp rx=0 tx=0).stop 2024-03-22T06:10:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.235+0000 7fdd990e0700 1 -- 172.21.15.117:0/1445475833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd940fce70 msgr2=0x7fdd9410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.235+0000 7fdd990e0700 1 --2- 172.21.15.117:0/1445475833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd940fce70 0x7fdd9410d920 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd88002a50 tx=0x7fdd880046b0 comp rx=0 tx=0).stop 2024-03-22T06:10:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.235+0000 7fdd990e0700 1 -- 172.21.15.117:0/1445475833 shutdown_connections 2024-03-22T06:10:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.235+0000 7fdd990e0700 1 --2- 172.21.15.117:0/1445475833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd940fce70 0x7fdd9410d920 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.235+0000 7fdd990e0700 1 --2- 172.21.15.117:0/1445475833 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fdd7c0692a0 0x7fdd7c06b750 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.235+0000 7fdd990e0700 1 --2- 172.21.15.117:0/1445475833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd940fdfe0 0x7fdd9410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.235+0000 7fdd990e0700 1 -- 172.21.15.117:0/1445475833 >> 172.21.15.117:0/1445475833 conn(0x7fdd940f8550 msgr2=0x7fdd940f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:10:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.235+0000 7fdd990e0700 1 -- 172.21.15.117:0/1445475833 shutdown_connections 2024-03-22T06:10:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:30.235+0000 7fdd990e0700 1 -- 172.21.15.117:0/1445475833 wait complete. 2024-03-22T06:10:30.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:30 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:30.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:30 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:30.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:30 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:30.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:30 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:10:30.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:30 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:30.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:30 smithi006 ceph-mon[29301]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:30.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:30 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:30.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:30 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:30.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:30 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:10:30.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:30 smithi006 ceph-mon[29301]: from='osd.4 [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T06:10:30.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:30 smithi006 ceph-mon[29301]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T06:10:30.859 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:30 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:30.859 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:30 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:30.859 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:30 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:30.859 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:30 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:10:30.859 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:30 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:30.859 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:30 smithi117 ceph-mon[36514]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:30.859 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:30 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:30.859 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:30 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:30.859 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:30 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:10:30.859 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:30 smithi117 ceph-mon[36514]: from='osd.4 [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T06:10:30.860 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:30 smithi117 ceph-mon[36514]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T06:10:30.882 DEBUG:teuthology.orchestra.run.smithi117:osd.4> sudo journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.4.service 2024-03-22T06:10:30.885 INFO:tasks.cephadm:Deploying osd.5 on smithi117 with /dev/vg_nvme/lv_2... 2024-03-22T06:10:30.885 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-22T06:10:31.996 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi117/config 2024-03-22T06:10:32.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:32 smithi117 ceph-mon[36514]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T06:10:32.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:32 smithi117 ceph-mon[36514]: osdmap e33: 5 total, 4 up, 5 in 2024-03-22T06:10:32.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:32 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:10:32.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:32 smithi117 ceph-mon[36514]: from='osd.4 [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:32.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:32 smithi117 ceph-mon[36514]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:32.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:32 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:10:32.256 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:10:32 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[48526]: 2024-03-22T06:10:32.204+0000 7f766e3e9700 -1 osd.4 0 waiting for initial osdmap 2024-03-22T06:10:32.257 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:10:32 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[48526]: 2024-03-22T06:10:32.211+0000 7f76689e3700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:10:32.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:32 smithi006 ceph-mon[29301]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T06:10:32.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:32 smithi006 ceph-mon[29301]: osdmap e33: 5 total, 4 up, 5 in 2024-03-22T06:10:32.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:10:32.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:32 smithi006 ceph-mon[29301]: from='osd.4 [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:32.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:32 smithi006 ceph-mon[29301]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:32.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:10:33.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:33 smithi006 ceph-mon[29301]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:33.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:33 smithi006 ceph-mon[29301]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-22T06:10:33.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:33 smithi006 ceph-mon[29301]: osdmap e34: 5 total, 4 up, 5 in 2024-03-22T06:10:33.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:33 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:10:33.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:33 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:10:33.499 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:33 smithi117 ceph-mon[36514]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T06:10:33.499 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:33 smithi117 ceph-mon[36514]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-22T06:10:33.499 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:33 smithi117 ceph-mon[36514]: osdmap e34: 5 total, 4 up, 5 in 2024-03-22T06:10:33.499 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:33 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:10:33.500 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:33 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:10:34.472 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:34 smithi117 ceph-mon[36514]: purged_snaps scrub starts 2024-03-22T06:10:34.472 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:34 smithi117 ceph-mon[36514]: purged_snaps scrub ok 2024-03-22T06:10:34.472 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:34 smithi117 ceph-mon[36514]: osd.4 [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586] boot 2024-03-22T06:10:34.473 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:34 smithi117 ceph-mon[36514]: osdmap e35: 5 total, 5 up, 5 in 2024-03-22T06:10:34.473 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:34 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:10:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:34 smithi006 ceph-mon[29301]: purged_snaps scrub starts 2024-03-22T06:10:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:34 smithi006 ceph-mon[29301]: purged_snaps scrub ok 2024-03-22T06:10:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:34 smithi006 ceph-mon[29301]: osd.4 [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586] boot 2024-03-22T06:10:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:34 smithi006 ceph-mon[29301]: osdmap e35: 5 total, 5 up, 5 in 2024-03-22T06:10:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:34 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:10:34.902 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-22T06:10:34.903 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-22T06:10:34.903 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-03-22T06:10:34.903 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-03-22T06:10:34.903 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127593 s, 822 MB/s 2024-03-22T06:10:34.903 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-03-22T06:10:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: osdmap e36: 5 total, 5 up, 5 in 2024-03-22T06:10:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: Detected new or changed devices on smithi117 2024-03-22T06:10:35.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:35.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:35.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:10:35.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:10:35.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: Adjusting osd_memory_target on smithi117 to 7968M 2024-03-22T06:10:35.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:35.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:35.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:10:35.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:35.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:35.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: osdmap e36: 5 total, 5 up, 5 in 2024-03-22T06:10:35.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: Detected new or changed devices on smithi117 2024-03-22T06:10:35.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:35.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:35.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:10:35.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:10:35.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: Adjusting osd_memory_target on smithi117 to 7968M 2024-03-22T06:10:35.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:35.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:35.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:10:35.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:35.546 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-03-22T06:10:35.800 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi117/config 2024-03-22T06:10:37.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.370+0000 7ff605aa8700 1 -- 172.21.15.117:0/3884186699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6000fe070 msgr2=0x7ff6000fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:37.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.370+0000 7ff605aa8700 1 --2- 172.21.15.117:0/3884186699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6000fe070 0x7ff6000fe4e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4004940 tx=0x7ff5f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:10:37.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.370+0000 7ff605aa8700 1 -- 172.21.15.117:0/3884186699 shutdown_connections 2024-03-22T06:10:37.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.370+0000 7ff605aa8700 1 --2- 172.21.15.117:0/3884186699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6000fe070 0x7ff6000fe4e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:37.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.370+0000 7ff605aa8700 1 --2- 172.21.15.117:0/3884186699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6000fcf00 0x7ff6000fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:37.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.370+0000 7ff605aa8700 1 -- 172.21.15.117:0/3884186699 >> 172.21.15.117:0/3884186699 conn(0x7ff6000f85a0 msgr2=0x7ff6000fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:10:37.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.370+0000 7ff605aa8700 1 -- 172.21.15.117:0/3884186699 shutdown_connections 2024-03-22T06:10:37.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.370+0000 7ff605aa8700 1 -- 172.21.15.117:0/3884186699 wait complete. 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff605aa8700 1 Processor -- start 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff605aa8700 1 -- start start 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff605aa8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6000fcf00 0x7ff600071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff605aa8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6000fe070 0x7ff600070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff605aa8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff600070950 con 0x7ff6000fe070 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff605aa8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff600070ac0 con 0x7ff6000fcf00 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff5ff7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6000fcf00 0x7ff600071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff5ff7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6000fcf00 0x7ff600071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48342/0 (socket says 172.21.15.117:48342) 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff5ff7fe700 1 -- 172.21.15.117:0/1925737876 learned_addr learned my addr 172.21.15.117:0/1925737876 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff5feffd700 1 --2- 172.21.15.117:0/1925737876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6000fe070 0x7ff600070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff5ff7fe700 1 -- 172.21.15.117:0/1925737876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6000fe070 msgr2=0x7ff600070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff5ff7fe700 1 --2- 172.21.15.117:0/1925737876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6000fe070 0x7ff600070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff5ff7fe700 1 -- 172.21.15.117:0/1925737876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4005040 con 0x7ff6000fcf00 2024-03-22T06:10:37.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.371+0000 7ff5ff7fe700 1 --2- 172.21.15.117:0/1925737876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6000fcf00 0x7ff600071e40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000bbb0 tx=0x7ff5f0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:10:37.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.372+0000 7ff5e7fff700 1 -- 172.21.15.117:0/1925737876 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5f00092e0 con 0x7ff6000fcf00 2024-03-22T06:10:37.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.372+0000 7ff605aa8700 1 -- 172.21.15.117:0/1925737876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff600070da0 con 0x7ff6000fcf00 2024-03-22T06:10:37.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.372+0000 7ff5e7fff700 1 -- 172.21.15.117:0/1925737876 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5f000cbd0 con 0x7ff6000fcf00 2024-03-22T06:10:37.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.372+0000 7ff5e7fff700 1 -- 172.21.15.117:0/1925737876 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5f0007400 con 0x7ff6000fcf00 2024-03-22T06:10:37.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.372+0000 7ff605aa8700 1 -- 172.21.15.117:0/1925737876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff600115ac0 con 0x7ff6000fcf00 2024-03-22T06:10:37.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.373+0000 7ff605aa8700 1 -- 172.21.15.117:0/1925737876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60005f5f0 con 0x7ff6000fcf00 2024-03-22T06:10:37.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.373+0000 7ff5e7fff700 1 -- 172.21.15.117:0/1925737876 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7ff5f0012070 con 0x7ff6000fcf00 2024-03-22T06:10:37.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.373+0000 7ff5e7fff700 1 --2- 172.21.15.117:0/1925737876 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff5e80692a0 0x7ff5e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:10:37.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.374+0000 7ff5e7fff700 1 -- 172.21.15.117:0/1925737876 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4202+0+0 (secure 0 0 0) 0x7ff5f0086aa0 con 0x7ff6000fcf00 2024-03-22T06:10:37.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.374+0000 7ff5feffd700 1 --2- 172.21.15.117:0/1925737876 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff5e80692a0 0x7ff5e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:10:37.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.374+0000 7ff5feffd700 1 --2- 172.21.15.117:0/1925737876 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff5e80692a0 0x7ff5e806b750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400b3b0 tx=0x7ff5f4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:10:37.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.378+0000 7ff5e7fff700 1 -- 172.21.15.117:0/1925737876 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff5f0053240 con 0x7ff6000fcf00 2024-03-22T06:10:37.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:37 smithi117 ceph-mon[36514]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:37.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:37 smithi006 ceph-mon[29301]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:37.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:37.574+0000 7ff605aa8700 1 -- 172.21.15.117:0/1925737876 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff600102340 con 0x7ff5e80692a0 2024-03-22T06:10:38.248 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:10:38.248 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:10:38.248 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:38 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T06:10:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T06:10:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:38 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:39 smithi117 ceph-mon[36514]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:10:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:39 smithi117 ceph-mon[36514]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:39.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:39 smithi006 ceph-mon[29301]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:10:39.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:39 smithi006 ceph-mon[29301]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:40.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:40 smithi117 ceph-mon[36514]: from='client.? 172.21.15.117:0/3225451715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf61ff67-1f07-47ed-ad58-e6c946369d70"}]: dispatch 2024-03-22T06:10:40.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:40 smithi117 ceph-mon[36514]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf61ff67-1f07-47ed-ad58-e6c946369d70"}]: dispatch 2024-03-22T06:10:40.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:40 smithi117 ceph-mon[36514]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf61ff67-1f07-47ed-ad58-e6c946369d70"}]': finished 2024-03-22T06:10:40.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:40 smithi117 ceph-mon[36514]: osdmap e37: 6 total, 5 up, 6 in 2024-03-22T06:10:40.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:40 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:10:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:40 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/3225451715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf61ff67-1f07-47ed-ad58-e6c946369d70"}]: dispatch 2024-03-22T06:10:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:40 smithi006 ceph-mon[29301]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf61ff67-1f07-47ed-ad58-e6c946369d70"}]: dispatch 2024-03-22T06:10:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:40 smithi006 ceph-mon[29301]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf61ff67-1f07-47ed-ad58-e6c946369d70"}]': finished 2024-03-22T06:10:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:40 smithi006 ceph-mon[29301]: osdmap e37: 6 total, 5 up, 6 in 2024-03-22T06:10:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:40 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:10:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:41 smithi117 ceph-mon[36514]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:41 smithi117 ceph-mon[36514]: from='client.? 172.21.15.117:0/3373558260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:10:41.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:41 smithi006 ceph-mon[29301]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:41.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:41 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/3373558260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T06:10:43.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:43 smithi006 ceph-mon[29301]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:43.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:43 smithi117 ceph-mon[36514]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:45.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:45 smithi117 ceph-mon[36514]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:45.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:45 smithi006 ceph-mon[29301]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:47.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:47 smithi117 ceph-mon[36514]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:47.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:10:47.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:47 smithi006 ceph-mon[29301]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:47.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:10:48.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:48 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T06:10:48.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:48 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:48.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:48 smithi117 ceph-mon[36514]: Deploying daemon osd.5 on smithi117 2024-03-22T06:10:48.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T06:10:48.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:48 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:10:48.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:48 smithi006 ceph-mon[29301]: Deploying daemon osd.5 on smithi117 2024-03-22T06:10:49.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:49 smithi117 ceph-mon[36514]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:49.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:49 smithi006 ceph-mon[29301]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:51.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:51 smithi117 ceph-mon[36514]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:51.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:51 smithi006 ceph-mon[29301]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:53.694 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:53 smithi117 ceph-mon[36514]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:53.694 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:53 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:53.694 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:53 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:10:53.694 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:53 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:53 smithi006 ceph-mon[29301]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:53 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:53 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:10:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:53 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:55.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:55 smithi117 ceph-mon[36514]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:55.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:55 smithi006 ceph-mon[29301]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:56.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.091+0000 7ff5e7fff700 1 -- 172.21.15.117:0/1925737876 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff600102340 con 0x7ff5e80692a0 2024-03-22T06:10:56.092 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 5 on host 'smithi117' 2024-03-22T06:10:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.092+0000 7ff605aa8700 1 -- 172.21.15.117:0/1925737876 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff5e80692a0 msgr2=0x7ff5e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.092+0000 7ff605aa8700 1 --2- 172.21.15.117:0/1925737876 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff5e80692a0 0x7ff5e806b750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400b3b0 tx=0x7ff5f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:10:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.092+0000 7ff605aa8700 1 -- 172.21.15.117:0/1925737876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6000fcf00 msgr2=0x7ff600071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.092+0000 7ff605aa8700 1 --2- 172.21.15.117:0/1925737876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6000fcf00 0x7ff600071e40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000bbb0 tx=0x7ff5f0009840 comp rx=0 tx=0).stop 2024-03-22T06:10:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.092+0000 7ff605aa8700 1 -- 172.21.15.117:0/1925737876 shutdown_connections 2024-03-22T06:10:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.092+0000 7ff605aa8700 1 --2- 172.21.15.117:0/1925737876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6000fcf00 0x7ff600071e40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.092+0000 7ff605aa8700 1 --2- 172.21.15.117:0/1925737876 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff5e80692a0 0x7ff5e806b750 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.092+0000 7ff605aa8700 1 --2- 172.21.15.117:0/1925737876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6000fe070 0x7ff600070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.092+0000 7ff605aa8700 1 -- 172.21.15.117:0/1925737876 >> 172.21.15.117:0/1925737876 conn(0x7ff6000f85a0 msgr2=0x7ff6000f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:10:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.093+0000 7ff605aa8700 1 -- 172.21.15.117:0/1925737876 shutdown_connections 2024-03-22T06:10:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:10:56.093+0000 7ff605aa8700 1 -- 172.21.15.117:0/1925737876 wait complete. 2024-03-22T06:10:56.643 DEBUG:teuthology.orchestra.run.smithi117:osd.5> sudo journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.5.service 2024-03-22T06:10:56.645 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-22T06:10:56.645 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-22T06:10:56.915 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:10:57.288 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:57 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:57.288 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:57 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:57.288 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:57 smithi006 ceph-mon[29301]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:57.288 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:57 smithi006 ceph-mon[29301]: from='osd.5 [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T06:10:57.288 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:57 smithi006 ceph-mon[29301]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T06:10:57.289 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:57 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:57.289 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:57 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:57.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:57 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:57 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:57 smithi117 ceph-mon[36514]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:57 smithi117 ceph-mon[36514]: from='osd.5 [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T06:10:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:57 smithi117 ceph-mon[36514]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T06:10:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:57 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:57 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:10:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:58 smithi117 ceph-mon[36514]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T06:10:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:58 smithi117 ceph-mon[36514]: osdmap e38: 6 total, 5 up, 6 in 2024-03-22T06:10:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:58 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:10:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:58 smithi117 ceph-mon[36514]: from='osd.5 [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:58 smithi117 ceph-mon[36514]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:58.256 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:10:58 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[55451]: 2024-03-22T06:10:58.077+0000 7fda0bf12700 -1 osd.5 0 waiting for initial osdmap 2024-03-22T06:10:58.256 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:10:58 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[55451]: 2024-03-22T06:10:58.084+0000 7fda08d11700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:10:58.422 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:58 smithi006 ceph-mon[29301]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T06:10:58.422 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:58 smithi006 ceph-mon[29301]: osdmap e38: 6 total, 5 up, 6 in 2024-03-22T06:10:58.422 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:58 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:10:58.422 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:58 smithi006 ceph-mon[29301]: from='osd.5 [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:58.422 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:58 smithi006 ceph-mon[29301]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:10:58.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.641+0000 7f0669ddc700 1 -- 172.21.15.6:0/579796466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06640fce40 msgr2=0x7f06640fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:58.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.641+0000 7f0669ddc700 1 --2- 172.21.15.6:0/579796466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06640fce40 0x7f06640fd250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f065c009ce0 tx=0x7f065c004d10 comp rx=0 tx=0).stop 2024-03-22T06:10:58.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.641+0000 7f0669ddc700 1 -- 172.21.15.6:0/579796466 shutdown_connections 2024-03-22T06:10:58.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.641+0000 7f0669ddc700 1 --2- 172.21.15.6:0/579796466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06640fe040 0x7f06640fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:58.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.641+0000 7f0669ddc700 1 --2- 172.21.15.6:0/579796466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06640fce40 0x7f06640fd250 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:58.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.641+0000 7f0669ddc700 1 -- 172.21.15.6:0/579796466 >> 172.21.15.6:0/579796466 conn(0x7f06640f8550 msgr2=0x7f06640fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.641+0000 7f0669ddc700 1 -- 172.21.15.6:0/579796466 shutdown_connections 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.641+0000 7f0669ddc700 1 -- 172.21.15.6:0/579796466 wait complete. 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.641+0000 7f0669ddc700 1 Processor -- start 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f0669ddc700 1 -- start start 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f0669ddc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06640fce40 0x7f066410dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f0669ddc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06640fe040 0x7f066410e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f0669ddc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066410e6f0 con 0x7f06640fce40 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f0669ddc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066410ef40 con 0x7f06640fe040 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f06637fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06640fce40 0x7f066410dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f06637fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06640fce40 0x7f066410dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45044/0 (socket says 172.21.15.6:45044) 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f06637fe700 1 -- 172.21.15.6:0/3667890431 learned_addr learned my addr 172.21.15.6:0/3667890431 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f0662ffd700 1 --2- 172.21.15.6:0/3667890431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06640fe040 0x7f066410e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f06637fe700 1 -- 172.21.15.6:0/3667890431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06640fe040 msgr2=0x7f066410e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f06637fe700 1 --2- 172.21.15.6:0/3667890431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06640fe040 0x7f066410e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f06637fe700 1 -- 172.21.15.6:0/3667890431 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c009940 con 0x7f06640fce40 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f06637fe700 1 --2- 172.21.15.6:0/3667890431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06640fce40 0x7f066410dbb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f065c009380 tx=0x7f065c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.642+0000 7f065bfff700 1 -- 172.21.15.6:0/3667890431 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f065c00e040 con 0x7f06640fce40 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.643+0000 7f0669ddc700 1 -- 172.21.15.6:0/3667890431 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066410f1c0 con 0x7f06640fce40 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.643+0000 7f065bfff700 1 -- 172.21.15.6:0/3667890431 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f065c008ea0 con 0x7f06640fce40 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.643+0000 7f0669ddc700 1 -- 172.21.15.6:0/3667890431 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066410f700 con 0x7f06640fce40 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.643+0000 7f065bfff700 1 -- 172.21.15.6:0/3667890431 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f065c01c870 con 0x7f06640fce40 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.643+0000 7f0669ddc700 1 -- 172.21.15.6:0/3667890431 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066405f5f0 con 0x7f06640fce40 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.644+0000 7f065bfff700 1 -- 172.21.15.6:0/3667890431 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f065c012410 con 0x7f06640fce40 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.644+0000 7f065bfff700 1 --2- 172.21.15.6:0/3667890431 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06500692a0 0x7f065006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.644+0000 7f065bfff700 1 -- 172.21.15.6:0/3667890431 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4354+0+0 (secure 0 0 0) 0x7f065c088800 con 0x7f06640fce40 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.644+0000 7f0662ffd700 1 --2- 172.21.15.6:0/3667890431 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06500692a0 0x7f065006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:10:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.645+0000 7f0662ffd700 1 --2- 172.21.15.6:0/3667890431 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06500692a0 0x7f065006b750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f064c004f40 tx=0x7f064c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:10:58.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.649+0000 7f065bfff700 1 -- 172.21.15.6:0/3667890431 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f065c058a10 con 0x7f06640fce40 2024-03-22T06:10:58.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.843+0000 7f0669ddc700 1 -- 172.21.15.6:0/3667890431 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f066404bed0 con 0x7f06640fce40 2024-03-22T06:10:58.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.843+0000 7f065bfff700 1 -- 172.21.15.6:0/3667890431 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f065c0585a0 con 0x7f06640fce40 2024-03-22T06:10:58.844 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:10:58.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.845+0000 7f0669ddc700 1 -- 172.21.15.6:0/3667890431 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06500692a0 msgr2=0x7f065006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:58.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.845+0000 7f0669ddc700 1 --2- 172.21.15.6:0/3667890431 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06500692a0 0x7f065006b750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f064c004f40 tx=0x7f064c00b040 comp rx=0 tx=0).stop 2024-03-22T06:10:58.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.845+0000 7f0669ddc700 1 -- 172.21.15.6:0/3667890431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06640fce40 msgr2=0x7f066410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:10:58.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.845+0000 7f0669ddc700 1 --2- 172.21.15.6:0/3667890431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06640fce40 0x7f066410dbb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f065c009380 tx=0x7f065c004d10 comp rx=0 tx=0).stop 2024-03-22T06:10:58.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.845+0000 7f0669ddc700 1 -- 172.21.15.6:0/3667890431 shutdown_connections 2024-03-22T06:10:58.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.845+0000 7f0669ddc700 1 --2- 172.21.15.6:0/3667890431 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06500692a0 0x7f065006b750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:58.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.845+0000 7f0669ddc700 1 --2- 172.21.15.6:0/3667890431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06640fce40 0x7f066410dbb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:58.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.845+0000 7f0669ddc700 1 --2- 172.21.15.6:0/3667890431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06640fe040 0x7f066410e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:10:58.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.845+0000 7f0669ddc700 1 -- 172.21.15.6:0/3667890431 >> 172.21.15.6:0/3667890431 conn(0x7f06640f8550 msgr2=0x7f06640f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:10:58.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.845+0000 7f0669ddc700 1 -- 172.21.15.6:0/3667890431 shutdown_connections 2024-03-22T06:10:58.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:10:58.845+0000 7f0669ddc700 1 -- 172.21.15.6:0/3667890431 wait complete. 2024-03-22T06:10:59.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:59 smithi117 ceph-mon[36514]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-22T06:10:59.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:59 smithi117 ceph-mon[36514]: osdmap e39: 6 total, 5 up, 6 in 2024-03-22T06:10:59.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:59 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:10:59.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:59 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:10:59.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:59 smithi117 ceph-mon[36514]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:59.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:10:59 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3667890431' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T06:10:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:59 smithi006 ceph-mon[29301]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-22T06:10:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:59 smithi006 ceph-mon[29301]: osdmap e39: 6 total, 5 up, 6 in 2024-03-22T06:10:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:59 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:10:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:59 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:10:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:59 smithi006 ceph-mon[29301]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T06:10:59.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:10:59 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3667890431' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T06:10:59.596 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1711087833,"num_in_osds":6,"osd_in_since":1711087839,"num_remapped_pgs":0} 2024-03-22T06:11:00.240 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:00 smithi117 ceph-mon[36514]: purged_snaps scrub starts 2024-03-22T06:11:00.240 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:00 smithi117 ceph-mon[36514]: purged_snaps scrub ok 2024-03-22T06:11:00.240 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:11:00.240 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:00 smithi117 ceph-mon[36514]: osd.5 [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486] boot 2024-03-22T06:11:00.240 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:00 smithi117 ceph-mon[36514]: osdmap e40: 6 total, 6 up, 6 in 2024-03-22T06:11:00.240 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:00 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:11:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:00 smithi006 ceph-mon[29301]: purged_snaps scrub starts 2024-03-22T06:11:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:00 smithi006 ceph-mon[29301]: purged_snaps scrub ok 2024-03-22T06:11:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:11:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:00 smithi006 ceph-mon[29301]: osd.5 [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486] boot 2024-03-22T06:11:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:00 smithi006 ceph-mon[29301]: osdmap e40: 6 total, 6 up, 6 in 2024-03-22T06:11:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:00 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:11:00.597 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-22T06:11:00.866 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:01.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: osdmap e41: 6 total, 6 up, 6 in 2024-03-22T06:11:01.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:01.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: Detected new or changed devices on smithi117 2024-03-22T06:11:01.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:11:01.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:11:01.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:11:01.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:11:01.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:11:01.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: Adjusting osd_memory_target on smithi117 to 5312M 2024-03-22T06:11:01.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:11:01.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:11:01.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:11:01.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:11:01.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:11:01.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:11:01.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:11:01.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:11:01.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: osdmap e41: 6 total, 6 up, 6 in 2024-03-22T06:11:01.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:01.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: Detected new or changed devices on smithi117 2024-03-22T06:11:01.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:11:01.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:11:01.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:11:01.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:11:01.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:11:01.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: Adjusting osd_memory_target on smithi117 to 5312M 2024-03-22T06:11:01.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:11:01.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:11:01.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:11:01.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:11:01.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:11:01.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:11:01.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:11:01.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:11:02.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:11:02.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:11:02.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.532+0000 7fd43526f700 1 -- 172.21.15.6:0/3225618273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4300fce70 msgr2=0x7fd4300fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.532+0000 7fd43526f700 1 --2- 172.21.15.6:0/3225618273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4300fce70 0x7fd4300fd280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd420004940 tx=0x7fd420009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.533+0000 7fd43526f700 1 -- 172.21.15.6:0/3225618273 shutdown_connections 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.533+0000 7fd43526f700 1 --2- 172.21.15.6:0/3225618273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4300fdfe0 0x7fd4300fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.533+0000 7fd43526f700 1 --2- 172.21.15.6:0/3225618273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4300fce70 0x7fd4300fd280 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.533+0000 7fd43526f700 1 -- 172.21.15.6:0/3225618273 >> 172.21.15.6:0/3225618273 conn(0x7fd4300f8550 msgr2=0x7fd4300fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.533+0000 7fd43526f700 1 -- 172.21.15.6:0/3225618273 shutdown_connections 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.533+0000 7fd43526f700 1 -- 172.21.15.6:0/3225618273 wait complete. 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.533+0000 7fd43526f700 1 Processor -- start 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.533+0000 7fd43526f700 1 -- start start 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd43526f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4300fce70 0x7fd43010d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd43526f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4300fdfe0 0x7fd43010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd43526f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43010e520 con 0x7fd4300fdfe0 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd43526f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd430112ec0 con 0x7fd4300fce70 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd42e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4300fdfe0 0x7fd43010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd42e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4300fdfe0 0x7fd43010de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45062/0 (socket says 172.21.15.6:45062) 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd42e7fc700 1 -- 172.21.15.6:0/389844911 learned_addr learned my addr 172.21.15.6:0/389844911 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd42effd700 1 --2- 172.21.15.6:0/389844911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4300fce70 0x7fd43010d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd42e7fc700 1 -- 172.21.15.6:0/389844911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4300fce70 msgr2=0x7fd43010d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd42e7fc700 1 --2- 172.21.15.6:0/389844911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4300fce70 0x7fd43010d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd42e7fc700 1 -- 172.21.15.6:0/389844911 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd420005040 con 0x7fd4300fdfe0 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd42e7fc700 1 --2- 172.21.15.6:0/389844911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4300fdfe0 0x7fd43010de90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd42400b790 tx=0x7fd424009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd4177fe700 1 -- 172.21.15.6:0/389844911 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd42400f770 con 0x7fd4300fdfe0 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd4177fe700 1 -- 172.21.15.6:0/389844911 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd42400fda0 con 0x7fd4300fdfe0 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd43526f700 1 -- 172.21.15.6:0/389844911 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd430113120 con 0x7fd4300fdfe0 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd4177fe700 1 -- 172.21.15.6:0/389844911 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd42400d310 con 0x7fd4300fdfe0 2024-03-22T06:11:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.534+0000 7fd43526f700 1 -- 172.21.15.6:0/389844911 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd430113690 con 0x7fd4300fdfe0 2024-03-22T06:11:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.535+0000 7fd43526f700 1 -- 172.21.15.6:0/389844911 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43004c710 con 0x7fd4300fdfe0 2024-03-22T06:11:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.536+0000 7fd4177fe700 1 -- 172.21.15.6:0/389844911 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fd424010420 con 0x7fd4300fdfe0 2024-03-22T06:11:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.536+0000 7fd4177fe700 1 --2- 172.21.15.6:0/389844911 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd4180692a0 0x7fd41806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.536+0000 7fd4177fe700 1 -- 172.21.15.6:0/389844911 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd424086d60 con 0x7fd4300fdfe0 2024-03-22T06:11:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.536+0000 7fd42effd700 1 --2- 172.21.15.6:0/389844911 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd4180692a0 0x7fd41806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.537+0000 7fd42effd700 1 --2- 172.21.15.6:0/389844911 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd4180692a0 0x7fd41806b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd42000b3b0 tx=0x7fd420009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:02.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.541+0000 7fd4177fe700 1 -- 172.21.15.6:0/389844911 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd424056f00 con 0x7fd4300fdfe0 2024-03-22T06:11:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.734+0000 7fd43526f700 1 -- 172.21.15.6:0/389844911 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fd430105950 con 0x7fd4300fdfe0 2024-03-22T06:11:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.734+0000 7fd4177fe700 1 -- 172.21.15.6:0/389844911 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7fd424056a90 con 0x7fd4300fdfe0 2024-03-22T06:11:02.735 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:11:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.736+0000 7fd43526f700 1 -- 172.21.15.6:0/389844911 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd4180692a0 msgr2=0x7fd41806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.736+0000 7fd43526f700 1 --2- 172.21.15.6:0/389844911 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd4180692a0 0x7fd41806b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd42000b3b0 tx=0x7fd420009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.736+0000 7fd43526f700 1 -- 172.21.15.6:0/389844911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4300fdfe0 msgr2=0x7fd43010de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.736+0000 7fd43526f700 1 --2- 172.21.15.6:0/389844911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4300fdfe0 0x7fd43010de90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd42400b790 tx=0x7fd424009250 comp rx=0 tx=0).stop 2024-03-22T06:11:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.736+0000 7fd43526f700 1 -- 172.21.15.6:0/389844911 shutdown_connections 2024-03-22T06:11:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.736+0000 7fd43526f700 1 --2- 172.21.15.6:0/389844911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4300fce70 0x7fd43010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.736+0000 7fd43526f700 1 --2- 172.21.15.6:0/389844911 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd4180692a0 0x7fd41806b750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.736+0000 7fd43526f700 1 --2- 172.21.15.6:0/389844911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4300fdfe0 0x7fd43010de90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:02.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.736+0000 7fd43526f700 1 -- 172.21.15.6:0/389844911 >> 172.21.15.6:0/389844911 conn(0x7fd4300f8550 msgr2=0x7fd4300f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:02.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.736+0000 7fd43526f700 1 -- 172.21.15.6:0/389844911 shutdown_connections 2024-03-22T06:11:02.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:02.736+0000 7fd43526f700 1 -- 172.21.15.6:0/389844911 wait complete. 2024-03-22T06:11:03.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:03 smithi006 ceph-mon[29301]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:03.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:03 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/389844911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T06:11:03.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:03 smithi117 ceph-mon[36514]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:03.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:03 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/389844911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T06:11:03.528 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1711087859,"num_in_osds":6,"osd_in_since":1711087839,"num_remapped_pgs":0} 2024-03-22T06:11:03.528 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T06:11:03.792 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:05.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.411+0000 7ffb793bc700 1 -- 172.21.15.6:0/1525489656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb740fe830 msgr2=0x7ffb74106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:05.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.411+0000 7ffb793bc700 1 --2- 172.21.15.6:0/1525489656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb740fe830 0x7ffb74106f00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c004940 tx=0x7ffb5c009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:05.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.411+0000 7ffb793bc700 1 -- 172.21.15.6:0/1525489656 shutdown_connections 2024-03-22T06:11:05.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.411+0000 7ffb793bc700 1 --2- 172.21.15.6:0/1525489656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb740fe830 0x7ffb74106f00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:05.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.411+0000 7ffb793bc700 1 --2- 172.21.15.6:0/1525489656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb740fde90 0x7ffb740fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:05.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.411+0000 7ffb793bc700 1 -- 172.21.15.6:0/1525489656 >> 172.21.15.6:0/1525489656 conn(0x7ffb74072bc0 msgr2=0x7ffb74072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.411+0000 7ffb793bc700 1 -- 172.21.15.6:0/1525489656 shutdown_connections 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.411+0000 7ffb793bc700 1 -- 172.21.15.6:0/1525489656 wait complete. 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb793bc700 1 Processor -- start 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb793bc700 1 -- start start 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb793bc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb740fde90 0x7ffb74113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb793bc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb740fe830 0x7ffb741140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb793bc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74114730 con 0x7ffb740fde90 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb793bc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7410db20 con 0x7ffb740fe830 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb72ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb740fde90 0x7ffb74113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb72ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb740fde90 0x7ffb74113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40458/0 (socket says 172.21.15.6:40458) 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb72ffd700 1 -- 172.21.15.6:0/582966830 learned_addr learned my addr 172.21.15.6:0/582966830 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb727fc700 1 --2- 172.21.15.6:0/582966830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb740fe830 0x7ffb741140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb72ffd700 1 -- 172.21.15.6:0/582966830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb740fe830 msgr2=0x7ffb741140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:05.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb72ffd700 1 --2- 172.21.15.6:0/582966830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb740fe830 0x7ffb741140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:05.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.412+0000 7ffb72ffd700 1 -- 172.21.15.6:0/582966830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c005040 con 0x7ffb740fde90 2024-03-22T06:11:05.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.413+0000 7ffb72ffd700 1 --2- 172.21.15.6:0/582966830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb740fde90 0x7ffb74113ba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c001d80 tx=0x7ffb6c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:05.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.413+0000 7ffb6b7fe700 1 -- 172.21.15.6:0/582966830 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb6c007d10 con 0x7ffb740fde90 2024-03-22T06:11:05.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.413+0000 7ffb6b7fe700 1 -- 172.21.15.6:0/582966830 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb6c001e80 con 0x7ffb740fde90 2024-03-22T06:11:05.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.413+0000 7ffb793bc700 1 -- 172.21.15.6:0/582966830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7410de00 con 0x7ffb740fde90 2024-03-22T06:11:05.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.413+0000 7ffb6b7fe700 1 -- 172.21.15.6:0/582966830 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb6c014640 con 0x7ffb740fde90 2024-03-22T06:11:05.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.413+0000 7ffb793bc700 1 -- 172.21.15.6:0/582966830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7410e340 con 0x7ffb740fde90 2024-03-22T06:11:05.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.414+0000 7ffb793bc700 1 -- 172.21.15.6:0/582966830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb74065e30 con 0x7ffb740fde90 2024-03-22T06:11:05.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.414+0000 7ffb6b7fe700 1 -- 172.21.15.6:0/582966830 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7ffb6c00e960 con 0x7ffb740fde90 2024-03-22T06:11:05.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.415+0000 7ffb6b7fe700 1 --2- 172.21.15.6:0/582966830 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb600692a0 0x7ffb6006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:05.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.415+0000 7ffb727fc700 1 --2- 172.21.15.6:0/582966830 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb600692a0 0x7ffb6006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:05.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.415+0000 7ffb6b7fe700 1 -- 172.21.15.6:0/582966830 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffb6c086bc0 con 0x7ffb740fde90 2024-03-22T06:11:05.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.415+0000 7ffb727fc700 1 --2- 172.21.15.6:0/582966830 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb600692a0 0x7ffb6006b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffb7410ece0 tx=0x7ffb5c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:05.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.419+0000 7ffb6b7fe700 1 -- 172.21.15.6:0/582966830 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ffb6c04e190 con 0x7ffb740fde90 2024-03-22T06:11:05.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:05 smithi117 ceph-mon[36514]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:05.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:05 smithi006 ceph-mon[29301]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:05.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.613+0000 7ffb793bc700 1 -- 172.21.15.6:0/582966830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb74109f10 con 0x7ffb740fde90 2024-03-22T06:11:05.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.613+0000 7ffb6b7fe700 1 -- 172.21.15.6:0/582966830 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11018 (secure 0 0 0) 0x7ffb6c056cf0 con 0x7ffb740fde90 2024-03-22T06:11:05.613 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:11:05.613 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":41,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","created":"2024-03-22T06:05:27.379207+0000","modified":"2024-03-22T06:11:00.076870+0000","last_up_change":"2024-03-22T06:10:59.072791+0000","last_in_change":"2024-03-22T06:10:39.982933+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T06:09:46.449392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"ecd88bb1-d643-4505-a098-3728178c89e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6803","nonce":3577742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6805","nonce":3577742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6809","nonce":3577742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6807","nonce":3577742022}]},"public_addr":"172.21.15.6:6803/3577742022","cluster_addr":"172.21.15.6:6805/3577742022","heartbeat_back_addr":"172.21.15.6:6809/3577742022","heartbeat_front_addr":"172.21.15.6:6807/3577742022","state":["exists","up"]},{"osd":1,"uuid":"f1b55307-2c3b-4ab8-a893-af8072f333ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6811","nonce":3758615829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6813","nonce":3758615829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6817","nonce":3758615829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6815","nonce":3758615829}]},"public_addr":"172.21.15.6:6811/3758615829","cluster_addr":"172.21.15.6:6813/3758615829","heartbeat_back_addr":"172.21.15.6:6817/3758615829","heartbeat_front_addr":"172.21.15.6:6815/3758615829","state":["exists","up"]},{"osd":2,"uuid":"9bacfe42-3c78-4d86-bac0-0142cde76ff5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6819","nonce":907023022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6821","nonce":907023022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6825","nonce":907023022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6823","nonce":907023022}]},"public_addr":"172.21.15.6:6819/907023022","cluster_addr":"172.21.15.6:6821/907023022","heartbeat_back_addr":"172.21.15.6:6825/907023022","heartbeat_front_addr":"172.21.15.6:6823/907023022","state":["exists","up"]},{"osd":3,"uuid":"2c5a6acd-de41-455d-b6c5-d53bb76197ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6801","nonce":769002526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6803","nonce":769002526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6807","nonce":769002526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6805","nonce":769002526}]},"public_addr":"172.21.15.117:6801/769002526","cluster_addr":"172.21.15.117:6803/769002526","heartbeat_back_addr":"172.21.15.117:6807/769002526","heartbeat_front_addr":"172.21.15.117:6805/769002526","state":["exists","up"]},{"osd":4,"uuid":"be424041-1e07-41f4-abe0-68fdd21b1eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6809","nonce":1798261586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6811","nonce":1798261586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6815","nonce":1798261586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6813","nonce":1798261586}]},"public_addr":"172.21.15.117:6809/1798261586","cluster_addr":"172.21.15.117:6811/1798261586","heartbeat_back_addr":"172.21.15.117:6815/1798261586","heartbeat_front_addr":"172.21.15.117:6813/1798261586","state":["exists","up"]},{"osd":5,"uuid":"bf61ff67-1f07-47ed-ad58-e6c946369d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6817","nonce":2701911486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6819","nonce":2701911486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6823","nonce":2701911486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6821","nonce":2701911486}]},"public_addr":"172.21.15.117:6817/2701911486","cluster_addr":"172.21.15.117:6819/2701911486","heartbeat_back_addr":"172.21.15.117:6823/2701911486","heartbeat_front_addr":"172.21.15.117:6821/2701911486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:08:45.922485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:13.118065+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:42.409903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:05.760002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:31.339234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:57.286867+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3489834288":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/1533067608":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/791727884":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/3665872066":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6801/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6801/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2826360973":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6800/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2671132128":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/619515244":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2939515329":"2024-03-23T06:05:54.100943+0000","172.21.15.6:6801/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.6:0/2042811963":"2024-03-23T06:06:24.832062+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T06:11:05.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.615+0000 7ffb793bc700 1 -- 172.21.15.6:0/582966830 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb600692a0 msgr2=0x7ffb6006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:05.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.615+0000 7ffb793bc700 1 --2- 172.21.15.6:0/582966830 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb600692a0 0x7ffb6006b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffb7410ece0 tx=0x7ffb5c009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:05.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.615+0000 7ffb793bc700 1 -- 172.21.15.6:0/582966830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb740fde90 msgr2=0x7ffb74113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:05.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.615+0000 7ffb793bc700 1 --2- 172.21.15.6:0/582966830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb740fde90 0x7ffb74113ba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c001d80 tx=0x7ffb6c009ec0 comp rx=0 tx=0).stop 2024-03-22T06:11:05.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.615+0000 7ffb793bc700 1 -- 172.21.15.6:0/582966830 shutdown_connections 2024-03-22T06:11:05.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.615+0000 7ffb793bc700 1 --2- 172.21.15.6:0/582966830 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb600692a0 0x7ffb6006b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:05.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.615+0000 7ffb793bc700 1 --2- 172.21.15.6:0/582966830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb740fde90 0x7ffb74113ba0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:05.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.615+0000 7ffb793bc700 1 --2- 172.21.15.6:0/582966830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb740fe830 0x7ffb741140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:05.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.615+0000 7ffb793bc700 1 -- 172.21.15.6:0/582966830 >> 172.21.15.6:0/582966830 conn(0x7ffb74072bc0 msgr2=0x7ffb741037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:05.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.616+0000 7ffb793bc700 1 -- 172.21.15.6:0/582966830 shutdown_connections 2024-03-22T06:11:05.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:05.616+0000 7ffb793bc700 1 -- 172.21.15.6:0/582966830 wait complete. 2024-03-22T06:11:06.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:06 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/582966830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:11:06.401 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T06:09:46.449392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-03-22T06:11:06.401 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-22T06:11:06.504 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:06 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/582966830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:11:06.667 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:07.438 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:07 smithi006 ceph-mon[29301]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:07.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:07 smithi117 ceph-mon[36514]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:08.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.234+0000 7f09fa46e700 1 -- 172.21.15.6:0/733013133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09f40fb4a0 msgr2=0x7f09f40fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:08.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.234+0000 7f09fa46e700 1 --2- 172.21.15.6:0/733013133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09f40fb4a0 0x7f09f40fb910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f09e8004940 tx=0x7f09e8009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:08.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.234+0000 7f09fa46e700 1 -- 172.21.15.6:0/733013133 shutdown_connections 2024-03-22T06:11:08.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.234+0000 7f09fa46e700 1 --2- 172.21.15.6:0/733013133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09f40fb4a0 0x7f09f40fb910 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.234+0000 7f09fa46e700 1 --2- 172.21.15.6:0/733013133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f4102e70 0x7f09f4103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.234+0000 7f09fa46e700 1 -- 172.21.15.6:0/733013133 >> 172.21.15.6:0/733013133 conn(0x7f09f40714c0 msgr2=0x7f09f40718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.234+0000 7f09fa46e700 1 -- 172.21.15.6:0/733013133 shutdown_connections 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.234+0000 7f09fa46e700 1 -- 172.21.15.6:0/733013133 wait complete. 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.235+0000 7f09fa46e700 1 Processor -- start 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.235+0000 7f09fa46e700 1 -- start start 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.235+0000 7f09fa46e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09f40fb4a0 0x7f09f4113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.235+0000 7f09fa46e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f4102e70 0x7f09f4114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.235+0000 7f09fa46e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f41146a0 con 0x7f09f40fb4a0 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.235+0000 7f09fa46e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f410da90 con 0x7f09f4102e70 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.235+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09f40fb4a0 0x7f09f4113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.235+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09f40fb4a0 0x7f09f4113b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40478/0 (socket says 172.21.15.6:40478) 2024-03-22T06:11:08.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.235+0000 7f09f3fff700 1 -- 172.21.15.6:0/2051236079 learned_addr learned my addr 172.21.15.6:0/2051236079 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:08.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.236+0000 7f09f37fe700 1 --2- 172.21.15.6:0/2051236079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f4102e70 0x7f09f4114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:08.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.236+0000 7f09f3fff700 1 -- 172.21.15.6:0/2051236079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f4102e70 msgr2=0x7f09f4114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:08.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.236+0000 7f09f3fff700 1 --2- 172.21.15.6:0/2051236079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f4102e70 0x7f09f4114050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:08.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.236+0000 7f09f3fff700 1 -- 172.21.15.6:0/2051236079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e8005040 con 0x7f09f40fb4a0 2024-03-22T06:11:08.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.236+0000 7f09f3fff700 1 --2- 172.21.15.6:0/2051236079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09f40fb4a0 0x7f09f4113b10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f09e4002a50 tx=0x7f09e40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:08.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.236+0000 7f09f0ff9700 1 -- 172.21.15.6:0/2051236079 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09e4007cc0 con 0x7f09f40fb4a0 2024-03-22T06:11:08.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.236+0000 7f09f0ff9700 1 -- 172.21.15.6:0/2051236079 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09e400f040 con 0x7f09f40fb4a0 2024-03-22T06:11:08.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.236+0000 7f09f0ff9700 1 -- 172.21.15.6:0/2051236079 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09e400da30 con 0x7f09f40fb4a0 2024-03-22T06:11:08.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.236+0000 7f09fa46e700 1 -- 172.21.15.6:0/2051236079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f410dd70 con 0x7f09f40fb4a0 2024-03-22T06:11:08.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.236+0000 7f09fa46e700 1 -- 172.21.15.6:0/2051236079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f410e330 con 0x7f09f40fb4a0 2024-03-22T06:11:08.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.237+0000 7f09fa46e700 1 -- 172.21.15.6:0/2051236079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f4100110 con 0x7f09f40fb4a0 2024-03-22T06:11:08.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.238+0000 7f09f0ff9700 1 -- 172.21.15.6:0/2051236079 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f09e400db90 con 0x7f09f40fb4a0 2024-03-22T06:11:08.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.238+0000 7f09f0ff9700 1 --2- 172.21.15.6:0/2051236079 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f09dc0692b0 0x7f09dc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:08.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.238+0000 7f09f0ff9700 1 -- 172.21.15.6:0/2051236079 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f09e4086b20 con 0x7f09f40fb4a0 2024-03-22T06:11:08.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.238+0000 7f09f37fe700 1 --2- 172.21.15.6:0/2051236079 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f09dc0692b0 0x7f09dc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:08.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.238+0000 7f09f37fe700 1 --2- 172.21.15.6:0/2051236079 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f09dc0692b0 0x7f09dc06b760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09f410ec50 tx=0x7f09e8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:08.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.242+0000 7f09f0ff9700 1 -- 172.21.15.6:0/2051236079 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f09e4056c20 con 0x7f09f40fb4a0 2024-03-22T06:11:08.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.433+0000 7f09fa46e700 1 -- 172.21.15.6:0/2051236079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09f410f160 con 0x7f09f40fb4a0 2024-03-22T06:11:08.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.434+0000 7f09f0ff9700 1 -- 172.21.15.6:0/2051236079 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09e40567b0 con 0x7f09f40fb4a0 2024-03-22T06:11:08.434 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-03-22T06:11:08.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.435+0000 7f09fa46e700 1 -- 172.21.15.6:0/2051236079 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f09dc0692b0 msgr2=0x7f09dc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:08.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.435+0000 7f09fa46e700 1 --2- 172.21.15.6:0/2051236079 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f09dc0692b0 0x7f09dc06b760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09f410ec50 tx=0x7f09e8009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:08.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.435+0000 7f09fa46e700 1 -- 172.21.15.6:0/2051236079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09f40fb4a0 msgr2=0x7f09f4113b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:08.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.435+0000 7f09fa46e700 1 --2- 172.21.15.6:0/2051236079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09f40fb4a0 0x7f09f4113b10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f09e4002a50 tx=0x7f09e40046b0 comp rx=0 tx=0).stop 2024-03-22T06:11:08.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.435+0000 7f09fa46e700 1 -- 172.21.15.6:0/2051236079 shutdown_connections 2024-03-22T06:11:08.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.435+0000 7f09fa46e700 1 --2- 172.21.15.6:0/2051236079 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f09dc0692b0 0x7f09dc06b760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:08.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.435+0000 7f09fa46e700 1 --2- 172.21.15.6:0/2051236079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09f40fb4a0 0x7f09f4113b10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:08.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.435+0000 7f09fa46e700 1 --2- 172.21.15.6:0/2051236079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09f4102e70 0x7f09f4114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:08.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.435+0000 7f09fa46e700 1 -- 172.21.15.6:0/2051236079 >> 172.21.15.6:0/2051236079 conn(0x7f09f40714c0 msgr2=0x7f09f40fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:08.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.436+0000 7f09fa46e700 1 -- 172.21.15.6:0/2051236079 shutdown_connections 2024-03-22T06:11:08.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:08.436+0000 7f09fa46e700 1 -- 172.21.15.6:0/2051236079 wait complete. 2024-03-22T06:11:09.143 INFO:tasks.cephadm:Setting up client nodes... 2024-03-22T06:11:09.143 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-22T06:11:09.410 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:09.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:09 smithi117 ceph-mon[36514]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:09.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:09 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2051236079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T06:11:09.531 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:09 smithi006 ceph-mon[29301]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:09.531 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:09 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2051236079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T06:11:11.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.235+0000 7f0c4db12700 1 -- 172.21.15.6:0/2677085904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c480fcfd0 msgr2=0x7f0c480fd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:11.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.235+0000 7f0c4db12700 1 --2- 172.21.15.6:0/2677085904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c480fcfd0 0x7f0c480fd440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c004940 tx=0x7f0c3c009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:11.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.235+0000 7f0c4db12700 1 -- 172.21.15.6:0/2677085904 shutdown_connections 2024-03-22T06:11:11.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.235+0000 7f0c4db12700 1 --2- 172.21.15.6:0/2677085904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c480fcfd0 0x7f0c480fd440 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:11.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.235+0000 7f0c4db12700 1 --2- 172.21.15.6:0/2677085904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c480fbd20 0x7f0c480fc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:11.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.235+0000 7f0c4db12700 1 -- 172.21.15.6:0/2677085904 >> 172.21.15.6:0/2677085904 conn(0x7f0c480f7460 msgr2=0x7f0c480f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:11.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.235+0000 7f0c4db12700 1 -- 172.21.15.6:0/2677085904 shutdown_connections 2024-03-22T06:11:11.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.235+0000 7f0c4db12700 1 -- 172.21.15.6:0/2677085904 wait complete. 2024-03-22T06:11:11.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.236+0000 7f0c4db12700 1 Processor -- start 2024-03-22T06:11:11.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.236+0000 7f0c4db12700 1 -- start start 2024-03-22T06:11:11.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.236+0000 7f0c4db12700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c480fbd20 0x7f0c4810c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:11.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.236+0000 7f0c4db12700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c480fcfd0 0x7f0c4810cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:11.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.236+0000 7f0c4db12700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4810d3f0 con 0x7f0c480fcfd0 2024-03-22T06:11:11.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.236+0000 7f0c4db12700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48111d90 con 0x7f0c480fbd20 2024-03-22T06:11:11.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.236+0000 7f0c46ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c480fcfd0 0x7f0c4810cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:11.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.236+0000 7f0c46ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c480fcfd0 0x7f0c4810cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40502/0 (socket says 172.21.15.6:40502) 2024-03-22T06:11:11.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.236+0000 7f0c46ffd700 1 -- 172.21.15.6:0/1030037466 learned_addr learned my addr 172.21.15.6:0/1030037466 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:11.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.236+0000 7f0c477fe700 1 --2- 172.21.15.6:0/1030037466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c480fbd20 0x7f0c4810c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:11.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.237+0000 7f0c46ffd700 1 -- 172.21.15.6:0/1030037466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c480fbd20 msgr2=0x7f0c4810c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:11.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.237+0000 7f0c46ffd700 1 --2- 172.21.15.6:0/1030037466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c480fbd20 0x7f0c4810c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:11.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.237+0000 7f0c46ffd700 1 -- 172.21.15.6:0/1030037466 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c005040 con 0x7f0c480fcfd0 2024-03-22T06:11:11.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.237+0000 7f0c46ffd700 1 --2- 172.21.15.6:0/1030037466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c480fcfd0 0x7f0c4810cd60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00b3b0 tx=0x7f0c3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:11.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.238+0000 7f0c2ffff700 1 -- 172.21.15.6:0/1030037466 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c3c00e040 con 0x7f0c480fcfd0 2024-03-22T06:11:11.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.238+0000 7f0c2ffff700 1 -- 172.21.15.6:0/1030037466 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c3c00bbf0 con 0x7f0c480fcfd0 2024-03-22T06:11:11.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.238+0000 7f0c4db12700 1 -- 172.21.15.6:0/1030037466 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c48111f90 con 0x7f0c480fcfd0 2024-03-22T06:11:11.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.238+0000 7f0c2ffff700 1 -- 172.21.15.6:0/1030037466 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c3c01c630 con 0x7f0c480fcfd0 2024-03-22T06:11:11.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.238+0000 7f0c4db12700 1 -- 172.21.15.6:0/1030037466 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c481124d0 con 0x7f0c480fcfd0 2024-03-22T06:11:11.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.239+0000 7f0c4db12700 1 -- 172.21.15.6:0/1030037466 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4804c710 con 0x7f0c480fcfd0 2024-03-22T06:11:11.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.240+0000 7f0c2ffff700 1 -- 172.21.15.6:0/1030037466 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f0c3c00bd60 con 0x7f0c480fcfd0 2024-03-22T06:11:11.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.240+0000 7f0c2ffff700 1 --2- 172.21.15.6:0/1030037466 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c300692a0 0x7f0c3006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:11.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.240+0000 7f0c2ffff700 1 -- 172.21.15.6:0/1030037466 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0c3c087b30 con 0x7f0c480fcfd0 2024-03-22T06:11:11.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.240+0000 7f0c477fe700 1 --2- 172.21.15.6:0/1030037466 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c300692a0 0x7f0c3006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:11.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.241+0000 7f0c477fe700 1 --2- 172.21.15.6:0/1030037466 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c300692a0 0x7f0c3006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009bf0 tx=0x7f0c380091a0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:11.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.245+0000 7f0c2ffff700 1 -- 172.21.15.6:0/1030037466 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0c3c057c30 con 0x7f0c480fcfd0 2024-03-22T06:11:11.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:11 smithi006 ceph-mon[29301]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:11.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:11 smithi117 ceph-mon[36514]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:11.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.522+0000 7f0c4db12700 1 -- 172.21.15.6:0/1030037466 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f0c48104890 con 0x7f0c480fcfd0 2024-03-22T06:11:11.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.525+0000 7f0c2ffff700 1 -- 172.21.15.6:0/1030037466 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f0c3c0577c0 con 0x7f0c480fcfd0 2024-03-22T06:11:11.526 INFO:teuthology.orchestra.run.smithi006.stdout:[client.0] 2024-03-22T06:11:11.526 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQD/IP1l1FMzHxAAnqsow22mrhxIkmiboH/jSA== 2024-03-22T06:11:11.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.527+0000 7f0c4db12700 1 -- 172.21.15.6:0/1030037466 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c300692a0 msgr2=0x7f0c3006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:11.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.527+0000 7f0c4db12700 1 --2- 172.21.15.6:0/1030037466 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c300692a0 0x7f0c3006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009bf0 tx=0x7f0c380091a0 comp rx=0 tx=0).stop 2024-03-22T06:11:11.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.527+0000 7f0c4db12700 1 -- 172.21.15.6:0/1030037466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c480fcfd0 msgr2=0x7f0c4810cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:11.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.527+0000 7f0c4db12700 1 --2- 172.21.15.6:0/1030037466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c480fcfd0 0x7f0c4810cd60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00b3b0 tx=0x7f0c3c009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:11.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.527+0000 7f0c4db12700 1 -- 172.21.15.6:0/1030037466 shutdown_connections 2024-03-22T06:11:11.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.527+0000 7f0c4db12700 1 --2- 172.21.15.6:0/1030037466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c480fbd20 0x7f0c4810c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:11.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.527+0000 7f0c4db12700 1 --2- 172.21.15.6:0/1030037466 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c300692a0 0x7f0c3006b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:11.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.527+0000 7f0c4db12700 1 --2- 172.21.15.6:0/1030037466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c480fcfd0 0x7f0c4810cd60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:11.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.527+0000 7f0c4db12700 1 -- 172.21.15.6:0/1030037466 >> 172.21.15.6:0/1030037466 conn(0x7f0c480f7460 msgr2=0x7f0c480f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:11.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.528+0000 7f0c4db12700 1 -- 172.21.15.6:0/1030037466 shutdown_connections 2024-03-22T06:11:11.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:11.528+0000 7f0c4db12700 1 -- 172.21.15.6:0/1030037466 wait complete. 2024-03-22T06:11:12.223 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:11:12.223 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-22T06:11:12.223 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-22T06:11:12.269 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-22T06:11:12.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:12 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1030037466' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T06:11:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:12 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1030037466' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T06:11:12.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:12 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1030037466' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T06:11:12.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:12 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1030037466' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T06:11:12.537 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi117/config 2024-03-22T06:11:13.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:13 smithi006 ceph-mon[29301]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:13.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:13 smithi117 ceph-mon[36514]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:14.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.015+0000 7f48298a7700 1 -- 172.21.15.117:0/398013489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48240fbd10 msgr2=0x7f48240fc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:14.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.015+0000 7f48298a7700 1 --2- 172.21.15.117:0/398013489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48240fbd10 0x7f48240fc120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4814009ce0 tx=0x7f4814004d10 comp rx=0 tx=0).stop 2024-03-22T06:11:14.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.015+0000 7f48298a7700 1 -- 172.21.15.117:0/398013489 shutdown_connections 2024-03-22T06:11:14.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.015+0000 7f48298a7700 1 --2- 172.21.15.117:0/398013489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48240fcfc0 0x7f48240fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:14.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.015+0000 7f48298a7700 1 --2- 172.21.15.117:0/398013489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48240fbd10 0x7f48240fc120 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:14.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.015+0000 7f48298a7700 1 -- 172.21.15.117:0/398013489 >> 172.21.15.117:0/398013489 conn(0x7f48240f7450 msgr2=0x7f48240f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:14.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.015+0000 7f48298a7700 1 -- 172.21.15.117:0/398013489 shutdown_connections 2024-03-22T06:11:14.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.015+0000 7f48298a7700 1 -- 172.21.15.117:0/398013489 wait complete. 2024-03-22T06:11:14.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48298a7700 1 Processor -- start 2024-03-22T06:11:14.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48298a7700 1 -- start start 2024-03-22T06:11:14.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48298a7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48240fbd10 0x7f4824066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:14.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48298a7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48240fcfc0 0x7f48241104c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:14.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48298a7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4824110a00 con 0x7f48240fbd10 2024-03-22T06:11:14.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48298a7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4824110b70 con 0x7f48240fcfc0 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48227fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48240fcfc0 0x7f48241104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48227fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48240fcfc0 0x7f48241104c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34912/0 (socket says 172.21.15.117:34912) 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48227fc700 1 -- 172.21.15.117:0/2180425632 learned_addr learned my addr 172.21.15.117:0/2180425632 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f4822ffd700 1 --2- 172.21.15.117:0/2180425632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48240fbd10 0x7f4824066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48227fc700 1 -- 172.21.15.117:0/2180425632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48240fbd10 msgr2=0x7f4824066cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48227fc700 1 --2- 172.21.15.117:0/2180425632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48240fbd10 0x7f4824066cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48227fc700 1 -- 172.21.15.117:0/2180425632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4814009940 con 0x7f48240fcfc0 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.016+0000 7f48227fc700 1 --2- 172.21.15.117:0/2180425632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48240fcfc0 0x7f48241104c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f480c00eb40 tx=0x7f480c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.017+0000 7f481b7fe700 1 -- 172.21.15.117:0/2180425632 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f480c007cc0 con 0x7f48240fcfc0 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.017+0000 7f481b7fe700 1 -- 172.21.15.117:0/2180425632 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f480c00f040 con 0x7f48240fcfc0 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.017+0000 7f48298a7700 1 -- 172.21.15.117:0/2180425632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4824110da0 con 0x7f48240fcfc0 2024-03-22T06:11:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.017+0000 7f481b7fe700 1 -- 172.21.15.117:0/2180425632 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f480c00ca10 con 0x7f48240fcfc0 2024-03-22T06:11:14.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.017+0000 7f48298a7700 1 -- 172.21.15.117:0/2180425632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4824111360 con 0x7f48240fcfc0 2024-03-22T06:11:14.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.018+0000 7f48298a7700 1 -- 172.21.15.117:0/2180425632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482404c710 con 0x7f48240fcfc0 2024-03-22T06:11:14.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.018+0000 7f481b7fe700 1 -- 172.21.15.117:0/2180425632 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f480c00cb70 con 0x7f48240fcfc0 2024-03-22T06:11:14.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.019+0000 7f481b7fe700 1 --2- 172.21.15.117:0/2180425632 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f48100692a0 0x7f481006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:14.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.019+0000 7f481b7fe700 1 -- 172.21.15.117:0/2180425632 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f480c086960 con 0x7f48240fcfc0 2024-03-22T06:11:14.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.019+0000 7f4822ffd700 1 --2- 172.21.15.117:0/2180425632 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f48100692a0 0x7f481006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:14.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.019+0000 7f4822ffd700 1 --2- 172.21.15.117:0/2180425632 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f48100692a0 0x7f481006b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4814015040 tx=0x7f4814004d10 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:14.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.023+0000 7f481b7fe700 1 -- 172.21.15.117:0/2180425632 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f480c04e190 con 0x7f48240fcfc0 2024-03-22T06:11:14.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.286+0000 7f48298a7700 1 -- 172.21.15.117:0/2180425632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f4824111710 con 0x7f48240fcfc0 2024-03-22T06:11:14.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.293+0000 7f481b7fe700 1 -- 172.21.15.117:0/2180425632 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f480c056880 con 0x7f48240fcfc0 2024-03-22T06:11:14.293 INFO:teuthology.orchestra.run.smithi117.stdout:[client.1] 2024-03-22T06:11:14.293 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQACIf1lT5QqERAAOZ8Q2QDwS9iw/UFzJeXnUg== 2024-03-22T06:11:14.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.295+0000 7f48298a7700 1 -- 172.21.15.117:0/2180425632 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f48100692a0 msgr2=0x7f481006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:14.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.295+0000 7f48298a7700 1 --2- 172.21.15.117:0/2180425632 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f48100692a0 0x7f481006b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4814015040 tx=0x7f4814004d10 comp rx=0 tx=0).stop 2024-03-22T06:11:14.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.295+0000 7f48298a7700 1 -- 172.21.15.117:0/2180425632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48240fcfc0 msgr2=0x7f48241104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:14.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.295+0000 7f48298a7700 1 --2- 172.21.15.117:0/2180425632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48240fcfc0 0x7f48241104c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f480c00eb40 tx=0x7f480c00b040 comp rx=0 tx=0).stop 2024-03-22T06:11:14.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.295+0000 7f48298a7700 1 -- 172.21.15.117:0/2180425632 shutdown_connections 2024-03-22T06:11:14.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.295+0000 7f48298a7700 1 --2- 172.21.15.117:0/2180425632 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f48100692a0 0x7f481006b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:14.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.295+0000 7f48298a7700 1 --2- 172.21.15.117:0/2180425632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48240fbd10 0x7f4824066cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:14.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.295+0000 7f48298a7700 1 --2- 172.21.15.117:0/2180425632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48240fcfc0 0x7f48241104c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:14.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.295+0000 7f48298a7700 1 -- 172.21.15.117:0/2180425632 >> 172.21.15.117:0/2180425632 conn(0x7f48240f7450 msgr2=0x7f48240f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:14.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.296+0000 7f48298a7700 1 -- 172.21.15.117:0/2180425632 shutdown_connections 2024-03-22T06:11:14.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-22T06:11:14.296+0000 7f48298a7700 1 -- 172.21.15.117:0/2180425632 wait complete. 2024-03-22T06:11:14.896 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:11:14.896 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-22T06:11:14.896 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-22T06:11:14.940 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-22T06:11:14.940 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-22T06:11:14.940 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-22T06:11:15.209 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:15.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:15 smithi117 ceph-mon[36514]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:15.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:15 smithi117 ceph-mon[36514]: from='client.? 172.21.15.117:0/2180425632' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T06:11:15.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:15 smithi117 ceph-mon[36514]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T06:11:15.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:15 smithi117 ceph-mon[36514]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T06:11:15.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:15 smithi006 ceph-mon[29301]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:15.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:15 smithi006 ceph-mon[29301]: from='client.? 172.21.15.117:0/2180425632' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T06:11:15.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:15 smithi006 ceph-mon[29301]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T06:11:15.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:15 smithi006 ceph-mon[29301]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T06:11:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.923+0000 7f501c856700 1 -- 172.21.15.6:0/3008628332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5018101d20 msgr2=0x7f50181020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.923+0000 7f501c856700 1 --2- 172.21.15.6:0/3008628332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5018101d20 0x7f50181020f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5010009ce0 tx=0x7f5010004d10 comp rx=0 tx=0).stop 2024-03-22T06:11:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.923+0000 7f501c856700 1 -- 172.21.15.6:0/3008628332 shutdown_connections 2024-03-22T06:11:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.923+0000 7f501c856700 1 --2- 172.21.15.6:0/3008628332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50181026c0 0x7f50180fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.923+0000 7f501c856700 1 --2- 172.21.15.6:0/3008628332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5018101d20 0x7f50181020f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.923+0000 7f501c856700 1 -- 172.21.15.6:0/3008628332 >> 172.21.15.6:0/3008628332 conn(0x7f50180f7870 msgr2=0x7f50180f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.923+0000 7f501c856700 1 -- 172.21.15.6:0/3008628332 shutdown_connections 2024-03-22T06:11:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.923+0000 7f501c856700 1 -- 172.21.15.6:0/3008628332 wait complete. 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.924+0000 7f501c856700 1 Processor -- start 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.924+0000 7f501c856700 1 -- start start 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.924+0000 7f501c856700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5018101d20 0x7f5018102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.924+0000 7f501c856700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50181026c0 0x7f50181030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.924+0000 7f501c856700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5018104ad0 con 0x7f50181026c0 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.924+0000 7f501c856700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5018103600 con 0x7f5018101d20 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.924+0000 7f5015d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50181026c0 0x7f50181030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.924+0000 7f5015d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50181026c0 0x7f50181030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36720/0 (socket says 172.21.15.6:36720) 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.924+0000 7f5015d9b700 1 -- 172.21.15.6:0/3968794849 learned_addr learned my addr 172.21.15.6:0/3968794849 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.925+0000 7f501659c700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5018101d20 0x7f5018102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.925+0000 7f5015d9b700 1 -- 172.21.15.6:0/3968794849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5018101d20 msgr2=0x7f5018102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:16.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.925+0000 7f5015d9b700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5018101d20 0x7f5018102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.925+0000 7f5015d9b700 1 -- 172.21.15.6:0/3968794849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5010009940 con 0x7f50181026c0 2024-03-22T06:11:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.925+0000 7f501659c700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5018101d20 0x7f5018102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:11:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.925+0000 7f5015d9b700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50181026c0 0x7f50181030c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f500000c800 tx=0x7f5000004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.925+0000 7f500effd700 1 -- 172.21.15.6:0/3968794849 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f500000a760 con 0x7f50181026c0 2024-03-22T06:11:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.925+0000 7f501c856700 1 -- 172.21.15.6:0/3968794849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50181038e0 con 0x7f50181026c0 2024-03-22T06:11:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.925+0000 7f500effd700 1 -- 172.21.15.6:0/3968794849 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f500000ad90 con 0x7f50181026c0 2024-03-22T06:11:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.925+0000 7f500effd700 1 -- 172.21.15.6:0/3968794849 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50000113a0 con 0x7f50181026c0 2024-03-22T06:11:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.925+0000 7f501c856700 1 -- 172.21.15.6:0/3968794849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501811a330 con 0x7f50181026c0 2024-03-22T06:11:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.926+0000 7f501c856700 1 -- 172.21.15.6:0/3968794849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50180fd470 con 0x7f50181026c0 2024-03-22T06:11:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.926+0000 7f500effd700 1 -- 172.21.15.6:0/3968794849 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f5000012400 con 0x7f50181026c0 2024-03-22T06:11:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.927+0000 7f500effd700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f50040692a0 0x7f500406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.927+0000 7f500effd700 1 -- 172.21.15.6:0/3968794849 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5000086e90 con 0x7f50181026c0 2024-03-22T06:11:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.927+0000 7f501659c700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f50040692a0 0x7f500406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:16.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.927+0000 7f501659c700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f50040692a0 0x7f500406b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f501000b740 tx=0x7f5010004d10 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:16.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:16.931+0000 7f500effd700 1 -- 172.21.15.6:0/3968794849 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5000053500 con 0x7f50181026c0 2024-03-22T06:11:17.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.189+0000 7f501c856700 1 -- 172.21.15.6:0/3968794849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f501804c710 con 0x7f50181026c0 2024-03-22T06:11:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.194+0000 7f500effd700 1 -- 172.21.15.6:0/3968794849 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167156 (secure 0 0 0) 0x7f5000056b20 con 0x7f50181026c0 2024-03-22T06:11:17.197 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:11:17.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.198+0000 7f501c856700 1 -- 172.21.15.6:0/3968794849 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f50040692a0 msgr2=0x7f500406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:17.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.198+0000 7f501c856700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f50040692a0 0x7f500406b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f501000b740 tx=0x7f5010004d10 comp rx=0 tx=0).stop 2024-03-22T06:11:17.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.198+0000 7f501c856700 1 -- 172.21.15.6:0/3968794849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50181026c0 msgr2=0x7f50181030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:17.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.198+0000 7f501c856700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50181026c0 0x7f50181030c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f500000c800 tx=0x7f5000004470 comp rx=0 tx=0).stop 2024-03-22T06:11:17.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.198+0000 7f501c856700 1 -- 172.21.15.6:0/3968794849 shutdown_connections 2024-03-22T06:11:17.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.198+0000 7f501c856700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5018101d20 0x7f5018102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:17.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.198+0000 7f501c856700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f50040692a0 0x7f500406b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:17.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.198+0000 7f501c856700 1 --2- 172.21.15.6:0/3968794849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50181026c0 0x7f50181030c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:17.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.198+0000 7f501c856700 1 -- 172.21.15.6:0/3968794849 >> 172.21.15.6:0/3968794849 conn(0x7f50180f7870 msgr2=0x7f50180f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:17.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.198+0000 7f501c856700 1 -- 172.21.15.6:0/3968794849 shutdown_connections 2024-03-22T06:11:17.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:17.198+0000 7f501c856700 1 -- 172.21.15.6:0/3968794849 wait complete. 2024-03-22T06:11:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:17 smithi117 ceph-mon[36514]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:11:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:17 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3968794849' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-22T06:11:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:17 smithi006 ceph-mon[29301]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:11:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:17 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3968794849' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-22T06:11:17.876 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":19,"active_gid":14212,"active_name":"smithi006.sfpxdy","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":602685778},{"type":"v1","addr":"172.21.15.6:6801","nonce":602685778}]},"active_addr":"172.21.15.6:6801/602685778","active_change":"2024-03-22T06:07:32.107943+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14232,"name":"smithi117.bqeskk","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.6:8443/","prometheus":"http://172.21.15.6:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":254878217}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":1270793675}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":1354987702}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":2742769039}]}]} 2024-03-22T06:11:17.880 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-22T06:11:17.880 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-22T06:11:17.880 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T06:11:18.142 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:19.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:19 smithi117 ceph-mon[36514]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:19.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:19 smithi006 ceph-mon[29301]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:19.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.810+0000 7fb1c77d8700 1 -- 172.21.15.6:0/3029763872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1c00fe7a0 msgr2=0x7fb1c0106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:19.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.810+0000 7fb1c77d8700 1 --2- 172.21.15.6:0/3029763872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1c00fe7a0 0x7fb1c0106e70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4004940 tx=0x7fb1b4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:19.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.810+0000 7fb1c77d8700 1 -- 172.21.15.6:0/3029763872 shutdown_connections 2024-03-22T06:11:19.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.810+0000 7fb1c77d8700 1 --2- 172.21.15.6:0/3029763872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1c00fe7a0 0x7fb1c0106e70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:19.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.810+0000 7fb1c77d8700 1 --2- 172.21.15.6:0/3029763872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c00fde00 0x7fb1c00fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:19.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.810+0000 7fb1c77d8700 1 -- 172.21.15.6:0/3029763872 >> 172.21.15.6:0/3029763872 conn(0x7fb1c00f7880 msgr2=0x7fb1c00f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:19.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.810+0000 7fb1c77d8700 1 -- 172.21.15.6:0/3029763872 shutdown_connections 2024-03-22T06:11:19.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.810+0000 7fb1c77d8700 1 -- 172.21.15.6:0/3029763872 wait complete. 2024-03-22T06:11:19.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.810+0000 7fb1c77d8700 1 Processor -- start 2024-03-22T06:11:19.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.810+0000 7fb1c77d8700 1 -- start start 2024-03-22T06:11:19.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c77d8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c00fde00 0x7fb1c0113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:19.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c77d8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1c00fe7a0 0x7fb1c01143d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:19.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c77d8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0114ab0 con 0x7fb1c00fe7a0 2024-03-22T06:11:19.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c77d8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c010ddf0 con 0x7fb1c00fde00 2024-03-22T06:11:19.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c4d73700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1c00fe7a0 0x7fb1c01143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:19.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c4d73700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1c00fe7a0 0x7fb1c01143d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36728/0 (socket says 172.21.15.6:36728) 2024-03-22T06:11:19.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c4d73700 1 -- 172.21.15.6:0/881789960 learned_addr learned my addr 172.21.15.6:0/881789960 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:19.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c5574700 1 --2- 172.21.15.6:0/881789960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c00fde00 0x7fb1c0113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:19.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c4d73700 1 -- 172.21.15.6:0/881789960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c00fde00 msgr2=0x7fb1c0113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:19.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c4d73700 1 --2- 172.21.15.6:0/881789960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c00fde00 0x7fb1c0113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:19.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c4d73700 1 -- 172.21.15.6:0/881789960 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b4005040 con 0x7fb1c00fe7a0 2024-03-22T06:11:19.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c4d73700 1 --2- 172.21.15.6:0/881789960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1c00fe7a0 0x7fb1c01143d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009380 tx=0x7fb1b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:19.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1bdffb700 1 -- 172.21.15.6:0/881789960 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b400e040 con 0x7fb1c00fe7a0 2024-03-22T06:11:19.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1bdffb700 1 -- 172.21.15.6:0/881789960 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1b4009af0 con 0x7fb1c00fe7a0 2024-03-22T06:11:19.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1bdffb700 1 -- 172.21.15.6:0/881789960 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b401c630 con 0x7fb1c00fe7a0 2024-03-22T06:11:19.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.811+0000 7fb1c77d8700 1 -- 172.21.15.6:0/881789960 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c010e010 con 0x7fb1c00fe7a0 2024-03-22T06:11:19.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.812+0000 7fb1c77d8700 1 -- 172.21.15.6:0/881789960 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c010e550 con 0x7fb1c00fe7a0 2024-03-22T06:11:19.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.813+0000 7fb1c77d8700 1 -- 172.21.15.6:0/881789960 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c00666f0 con 0x7fb1c00fe7a0 2024-03-22T06:11:19.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.813+0000 7fb1bdffb700 1 -- 172.21.15.6:0/881789960 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fb1b400bbc0 con 0x7fb1c00fe7a0 2024-03-22T06:11:19.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.814+0000 7fb1bdffb700 1 --2- 172.21.15.6:0/881789960 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb1a80692a0 0x7fb1a806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:19.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.814+0000 7fb1c5574700 1 --2- 172.21.15.6:0/881789960 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb1a80692a0 0x7fb1a806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:19.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.814+0000 7fb1bdffb700 1 -- 172.21.15.6:0/881789960 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb1b40881a0 con 0x7fb1c00fe7a0 2024-03-22T06:11:19.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.814+0000 7fb1c5574700 1 --2- 172.21.15.6:0/881789960 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb1a80692a0 0x7fb1a806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8007f30 tx=0x7fb1b8007420 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:19.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:19.818+0000 7fb1bdffb700 1 -- 172.21.15.6:0/881789960 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb1b40582a0 con 0x7fb1c00fe7a0 2024-03-22T06:11:20.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.015+0000 7fb1c77d8700 1 -- 172.21.15.6:0/881789960 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1c005f5f0 con 0x7fb1c00fe7a0 2024-03-22T06:11:20.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.016+0000 7fb1bdffb700 1 -- 172.21.15.6:0/881789960 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11018 (secure 0 0 0) 0x7fb1b4057e30 con 0x7fb1c00fe7a0 2024-03-22T06:11:20.017 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:11:20.017 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":41,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","created":"2024-03-22T06:05:27.379207+0000","modified":"2024-03-22T06:11:00.076870+0000","last_up_change":"2024-03-22T06:10:59.072791+0000","last_in_change":"2024-03-22T06:10:39.982933+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T06:09:46.449392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"ecd88bb1-d643-4505-a098-3728178c89e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6803","nonce":3577742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6805","nonce":3577742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6809","nonce":3577742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6807","nonce":3577742022}]},"public_addr":"172.21.15.6:6803/3577742022","cluster_addr":"172.21.15.6:6805/3577742022","heartbeat_back_addr":"172.21.15.6:6809/3577742022","heartbeat_front_addr":"172.21.15.6:6807/3577742022","state":["exists","up"]},{"osd":1,"uuid":"f1b55307-2c3b-4ab8-a893-af8072f333ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6811","nonce":3758615829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6813","nonce":3758615829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6817","nonce":3758615829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6815","nonce":3758615829}]},"public_addr":"172.21.15.6:6811/3758615829","cluster_addr":"172.21.15.6:6813/3758615829","heartbeat_back_addr":"172.21.15.6:6817/3758615829","heartbeat_front_addr":"172.21.15.6:6815/3758615829","state":["exists","up"]},{"osd":2,"uuid":"9bacfe42-3c78-4d86-bac0-0142cde76ff5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6819","nonce":907023022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6821","nonce":907023022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6825","nonce":907023022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6823","nonce":907023022}]},"public_addr":"172.21.15.6:6819/907023022","cluster_addr":"172.21.15.6:6821/907023022","heartbeat_back_addr":"172.21.15.6:6825/907023022","heartbeat_front_addr":"172.21.15.6:6823/907023022","state":["exists","up"]},{"osd":3,"uuid":"2c5a6acd-de41-455d-b6c5-d53bb76197ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6801","nonce":769002526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6803","nonce":769002526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6807","nonce":769002526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6805","nonce":769002526}]},"public_addr":"172.21.15.117:6801/769002526","cluster_addr":"172.21.15.117:6803/769002526","heartbeat_back_addr":"172.21.15.117:6807/769002526","heartbeat_front_addr":"172.21.15.117:6805/769002526","state":["exists","up"]},{"osd":4,"uuid":"be424041-1e07-41f4-abe0-68fdd21b1eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6809","nonce":1798261586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6811","nonce":1798261586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6815","nonce":1798261586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6813","nonce":1798261586}]},"public_addr":"172.21.15.117:6809/1798261586","cluster_addr":"172.21.15.117:6811/1798261586","heartbeat_back_addr":"172.21.15.117:6815/1798261586","heartbeat_front_addr":"172.21.15.117:6813/1798261586","state":["exists","up"]},{"osd":5,"uuid":"bf61ff67-1f07-47ed-ad58-e6c946369d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6817","nonce":2701911486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6819","nonce":2701911486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6823","nonce":2701911486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6821","nonce":2701911486}]},"public_addr":"172.21.15.117:6817/2701911486","cluster_addr":"172.21.15.117:6819/2701911486","heartbeat_back_addr":"172.21.15.117:6823/2701911486","heartbeat_front_addr":"172.21.15.117:6821/2701911486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:08:45.922485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:13.118065+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:42.409903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:05.760002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:31.339234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:57.286867+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3489834288":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/1533067608":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/791727884":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/3665872066":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6801/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6801/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2826360973":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6800/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2671132128":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/619515244":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2939515329":"2024-03-23T06:05:54.100943+0000","172.21.15.6:6801/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.6:0/2042811963":"2024-03-23T06:06:24.832062+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T06:11:20.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.018+0000 7fb1c77d8700 1 -- 172.21.15.6:0/881789960 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb1a80692a0 msgr2=0x7fb1a806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.018+0000 7fb1c77d8700 1 --2- 172.21.15.6:0/881789960 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb1a80692a0 0x7fb1a806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8007f30 tx=0x7fb1b8007420 comp rx=0 tx=0).stop 2024-03-22T06:11:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.018+0000 7fb1c77d8700 1 -- 172.21.15.6:0/881789960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1c00fe7a0 msgr2=0x7fb1c01143d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.018+0000 7fb1c77d8700 1 --2- 172.21.15.6:0/881789960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1c00fe7a0 0x7fb1c01143d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009380 tx=0x7fb1b4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.018+0000 7fb1c77d8700 1 -- 172.21.15.6:0/881789960 shutdown_connections 2024-03-22T06:11:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.018+0000 7fb1c77d8700 1 --2- 172.21.15.6:0/881789960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c00fde00 0x7fb1c0113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.018+0000 7fb1c77d8700 1 --2- 172.21.15.6:0/881789960 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb1a80692a0 0x7fb1a806b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.018+0000 7fb1c77d8700 1 --2- 172.21.15.6:0/881789960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1c00fe7a0 0x7fb1c01143d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.018+0000 7fb1c77d8700 1 -- 172.21.15.6:0/881789960 >> 172.21.15.6:0/881789960 conn(0x7fb1c00f7880 msgr2=0x7fb1c01039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.018+0000 7fb1c77d8700 1 -- 172.21.15.6:0/881789960 shutdown_connections 2024-03-22T06:11:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:20.018+0000 7fb1c77d8700 1 -- 172.21.15.6:0/881789960 wait complete. 2024-03-22T06:11:20.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:20 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/881789960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:11:20.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:20 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/881789960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:11:20.688 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-22T06:11:20.688 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T06:11:20.981 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:21.356 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:21 smithi006 ceph-mon[29301]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:21 smithi117 ceph-mon[36514]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:22.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.560+0000 7f39c6f88700 1 -- 172.21.15.6:0/2317714388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39c0101d20 msgr2=0x7f39c01020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:22.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.560+0000 7f39c6f88700 1 --2- 172.21.15.6:0/2317714388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39c0101d20 0x7f39c01020f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f39b4004940 tx=0x7f39b4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:22.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.560+0000 7f39c6f88700 1 -- 172.21.15.6:0/2317714388 shutdown_connections 2024-03-22T06:11:22.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.560+0000 7f39c6f88700 1 --2- 172.21.15.6:0/2317714388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c01026c0 0x7f39c00fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.560+0000 7f39c6f88700 1 --2- 172.21.15.6:0/2317714388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39c0101d20 0x7f39c01020f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.560+0000 7f39c6f88700 1 -- 172.21.15.6:0/2317714388 >> 172.21.15.6:0/2317714388 conn(0x7f39c00f7870 msgr2=0x7f39c00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.560+0000 7f39c6f88700 1 -- 172.21.15.6:0/2317714388 shutdown_connections 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.560+0000 7f39c6f88700 1 -- 172.21.15.6:0/2317714388 wait complete. 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.560+0000 7f39c6f88700 1 Processor -- start 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.560+0000 7f39c6f88700 1 -- start start 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39c6f88700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c0101d20 0x7f39c0102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39c6f88700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39c01026c0 0x7f39c01030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39c6f88700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c0104ad0 con 0x7f39c01026c0 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39c6f88700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c0103600 con 0x7f39c0101d20 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39bffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39c01026c0 0x7f39c01030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:22.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39bffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39c01026c0 0x7f39c01030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36746/0 (socket says 172.21.15.6:36746) 2024-03-22T06:11:22.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39bffff700 1 -- 172.21.15.6:0/1361479488 learned_addr learned my addr 172.21.15.6:0/1361479488 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:22.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39c4d24700 1 --2- 172.21.15.6:0/1361479488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c0101d20 0x7f39c0102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:22.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39bffff700 1 -- 172.21.15.6:0/1361479488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c0101d20 msgr2=0x7f39c0102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:22.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39bffff700 1 --2- 172.21.15.6:0/1361479488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c0101d20 0x7f39c0102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:22.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39bffff700 1 -- 172.21.15.6:0/1361479488 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4005040 con 0x7f39c01026c0 2024-03-22T06:11:22.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39bffff700 1 --2- 172.21.15.6:0/1361479488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39c01026c0 0x7f39c01030c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f39ac002a50 tx=0x7f39ac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:22.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39bd7fa700 1 -- 172.21.15.6:0/1361479488 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39ac007cc0 con 0x7f39c01026c0 2024-03-22T06:11:22.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39c6f88700 1 -- 172.21.15.6:0/1361479488 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c01038e0 con 0x7f39c01026c0 2024-03-22T06:11:22.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39bd7fa700 1 -- 172.21.15.6:0/1361479488 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39ac00f040 con 0x7f39c01026c0 2024-03-22T06:11:22.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39bd7fa700 1 -- 172.21.15.6:0/1361479488 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39ac00d9c0 con 0x7f39c01026c0 2024-03-22T06:11:22.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.561+0000 7f39c6f88700 1 -- 172.21.15.6:0/1361479488 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c011a1a0 con 0x7f39c01026c0 2024-03-22T06:11:22.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.562+0000 7f39c6f88700 1 -- 172.21.15.6:0/1361479488 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c00fd470 con 0x7f39c01026c0 2024-03-22T06:11:22.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.563+0000 7f39bd7fa700 1 -- 172.21.15.6:0/1361479488 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f39ac009730 con 0x7f39c01026c0 2024-03-22T06:11:22.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.563+0000 7f39bd7fa700 1 --2- 172.21.15.6:0/1361479488 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f39b00692a0 0x7f39b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:22.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.563+0000 7f39bd7fa700 1 -- 172.21.15.6:0/1361479488 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f39ac086b00 con 0x7f39c01026c0 2024-03-22T06:11:22.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.563+0000 7f39c4d24700 1 --2- 172.21.15.6:0/1361479488 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f39b00692a0 0x7f39b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:22.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.564+0000 7f39c4d24700 1 --2- 172.21.15.6:0/1361479488 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f39b00692a0 0x7f39b006b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f39b4004f90 tx=0x7f39b4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:22.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.568+0000 7f39bd7fa700 1 -- 172.21.15.6:0/1361479488 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f39ac053170 con 0x7f39c01026c0 2024-03-22T06:11:22.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.763+0000 7f39c6f88700 1 -- 172.21.15.6:0/1361479488 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39c011a3e0 con 0x7f39c01026c0 2024-03-22T06:11:22.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.764+0000 7f39bd7fa700 1 -- 172.21.15.6:0/1361479488 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11018 (secure 0 0 0) 0x7f39ac056790 con 0x7f39c01026c0 2024-03-22T06:11:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:11:22.765 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":41,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","created":"2024-03-22T06:05:27.379207+0000","modified":"2024-03-22T06:11:00.076870+0000","last_up_change":"2024-03-22T06:10:59.072791+0000","last_in_change":"2024-03-22T06:10:39.982933+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T06:09:46.449392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"ecd88bb1-d643-4505-a098-3728178c89e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6803","nonce":3577742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6805","nonce":3577742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6809","nonce":3577742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6807","nonce":3577742022}]},"public_addr":"172.21.15.6:6803/3577742022","cluster_addr":"172.21.15.6:6805/3577742022","heartbeat_back_addr":"172.21.15.6:6809/3577742022","heartbeat_front_addr":"172.21.15.6:6807/3577742022","state":["exists","up"]},{"osd":1,"uuid":"f1b55307-2c3b-4ab8-a893-af8072f333ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6811","nonce":3758615829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6813","nonce":3758615829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6817","nonce":3758615829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6815","nonce":3758615829}]},"public_addr":"172.21.15.6:6811/3758615829","cluster_addr":"172.21.15.6:6813/3758615829","heartbeat_back_addr":"172.21.15.6:6817/3758615829","heartbeat_front_addr":"172.21.15.6:6815/3758615829","state":["exists","up"]},{"osd":2,"uuid":"9bacfe42-3c78-4d86-bac0-0142cde76ff5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6819","nonce":907023022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6821","nonce":907023022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6825","nonce":907023022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6823","nonce":907023022}]},"public_addr":"172.21.15.6:6819/907023022","cluster_addr":"172.21.15.6:6821/907023022","heartbeat_back_addr":"172.21.15.6:6825/907023022","heartbeat_front_addr":"172.21.15.6:6823/907023022","state":["exists","up"]},{"osd":3,"uuid":"2c5a6acd-de41-455d-b6c5-d53bb76197ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6801","nonce":769002526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6803","nonce":769002526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6807","nonce":769002526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6805","nonce":769002526}]},"public_addr":"172.21.15.117:6801/769002526","cluster_addr":"172.21.15.117:6803/769002526","heartbeat_back_addr":"172.21.15.117:6807/769002526","heartbeat_front_addr":"172.21.15.117:6805/769002526","state":["exists","up"]},{"osd":4,"uuid":"be424041-1e07-41f4-abe0-68fdd21b1eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6809","nonce":1798261586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6811","nonce":1798261586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6815","nonce":1798261586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6813","nonce":1798261586}]},"public_addr":"172.21.15.117:6809/1798261586","cluster_addr":"172.21.15.117:6811/1798261586","heartbeat_back_addr":"172.21.15.117:6815/1798261586","heartbeat_front_addr":"172.21.15.117:6813/1798261586","state":["exists","up"]},{"osd":5,"uuid":"bf61ff67-1f07-47ed-ad58-e6c946369d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6817","nonce":2701911486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6819","nonce":2701911486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6823","nonce":2701911486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6821","nonce":2701911486}]},"public_addr":"172.21.15.117:6817/2701911486","cluster_addr":"172.21.15.117:6819/2701911486","heartbeat_back_addr":"172.21.15.117:6823/2701911486","heartbeat_front_addr":"172.21.15.117:6821/2701911486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:08:45.922485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:13.118065+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:42.409903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:05.760002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:31.339234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:57.286867+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3489834288":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/1533067608":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/791727884":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/3665872066":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6801/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6801/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2826360973":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6800/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2671132128":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/619515244":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2939515329":"2024-03-23T06:05:54.100943+0000","172.21.15.6:6801/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.6:0/2042811963":"2024-03-23T06:06:24.832062+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T06:11:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.766+0000 7f39c6f88700 1 -- 172.21.15.6:0/1361479488 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f39b00692a0 msgr2=0x7f39b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.766+0000 7f39c6f88700 1 --2- 172.21.15.6:0/1361479488 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f39b00692a0 0x7f39b006b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f39b4004f90 tx=0x7f39b4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.766+0000 7f39c6f88700 1 -- 172.21.15.6:0/1361479488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39c01026c0 msgr2=0x7f39c01030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.766+0000 7f39c6f88700 1 --2- 172.21.15.6:0/1361479488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39c01026c0 0x7f39c01030c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f39ac002a50 tx=0x7f39ac004600 comp rx=0 tx=0).stop 2024-03-22T06:11:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.766+0000 7f39c6f88700 1 -- 172.21.15.6:0/1361479488 shutdown_connections 2024-03-22T06:11:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.766+0000 7f39c6f88700 1 --2- 172.21.15.6:0/1361479488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39c0101d20 0x7f39c0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.766+0000 7f39c6f88700 1 --2- 172.21.15.6:0/1361479488 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f39b00692a0 0x7f39b006b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:22.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.766+0000 7f39c6f88700 1 --2- 172.21.15.6:0/1361479488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39c01026c0 0x7f39c01030c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:22.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.766+0000 7f39c6f88700 1 -- 172.21.15.6:0/1361479488 >> 172.21.15.6:0/1361479488 conn(0x7f39c00f7870 msgr2=0x7f39c00f8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:22.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.766+0000 7f39c6f88700 1 -- 172.21.15.6:0/1361479488 shutdown_connections 2024-03-22T06:11:22.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:22.766+0000 7f39c6f88700 1 -- 172.21.15.6:0/1361479488 wait complete. 2024-03-22T06:11:23.470 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:23 smithi006 ceph-mon[29301]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:23.471 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:23 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1361479488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:11:23.501 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-22T06:11:23.502 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-22T06:11:23.502 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-22T06:11:23.503 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-22T06:11:23.503 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-22T06:11:23.503 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-22T06:11:23.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:23 smithi117 ceph-mon[36514]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:23.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:23 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1361479488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:11:23.828 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:23.833 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:25 smithi006 ceph-mon[29301]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:25.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:25 smithi117 ceph-mon[36514]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:26.138 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:26.139 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:26.147 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.900+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2276565432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfec101d70 msgr2=0x7fbfec102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.900+0000 7fbff1e0e700 1 --2- 172.21.15.6:0/2276565432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfec101d70 0x7fbfec102140 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4004940 tx=0x7fbfd4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.900+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2276565432 shutdown_connections 2024-03-22T06:11:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.900+0000 7fbff1e0e700 1 --2- 172.21.15.6:0/2276565432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfec102710 0x7fbfec0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.900+0000 7fbff1e0e700 1 --2- 172.21.15.6:0/2276565432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfec101d70 0x7fbfec102140 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.900+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2276565432 >> 172.21.15.6:0/2276565432 conn(0x7fbfec072bc0 msgr2=0x7fbfec072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.901+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2276565432 shutdown_connections 2024-03-22T06:11:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.901+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2276565432 wait complete. 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.901+0000 7fbff1e0e700 1 Processor -- start 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.901+0000 7fbff1e0e700 1 -- start start 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.901+0000 7fbff1e0e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfec101d70 0x7fbfec102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.901+0000 7fbff1e0e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfec102710 0x7fbfec103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.901+0000 7fbff1e0e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec103750 con 0x7fbfec101d70 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.901+0000 7fbff1e0e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec103890 con 0x7fbfec102710 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbfeb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfec101d70 0x7fbfec102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbfeb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfec101d70 0x7fbfec102bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45440/0 (socket says 172.21.15.6:45440) 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbfeb7fe700 1 -- 172.21.15.6:0/2856017036 learned_addr learned my addr 172.21.15.6:0/2856017036 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbfeaffd700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfec102710 0x7fbfec103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbfeb7fe700 1 -- 172.21.15.6:0/2856017036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfec102710 msgr2=0x7fbfec103130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbfeb7fe700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfec102710 0x7fbfec103130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:26.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbfeb7fe700 1 -- 172.21.15.6:0/2856017036 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4005040 con 0x7fbfec101d70 2024-03-22T06:11:26.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbfeb7fe700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfec101d70 0x7fbfec102bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400b8b0 tx=0x7fbfd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:26.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbfe3fff700 1 -- 172.21.15.6:0/2856017036 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfd400e040 con 0x7fbfec101d70 2024-03-22T06:11:26.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbfe3fff700 1 -- 172.21.15.6:0/2856017036 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfd4004290 con 0x7fbfec101d70 2024-03-22T06:11:26.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2856017036 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec119e60 con 0x7fbfec101d70 2024-03-22T06:11:26.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbfe3fff700 1 -- 172.21.15.6:0/2856017036 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfd401c3f0 con 0x7fbfec101d70 2024-03-22T06:11:26.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.902+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2856017036 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec11a340 con 0x7fbfec101d70 2024-03-22T06:11:26.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.903+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2856017036 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbfec0fd4c0 con 0x7fbfec101d70 2024-03-22T06:11:26.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.904+0000 7fbfe3fff700 1 -- 172.21.15.6:0/2856017036 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fbfd40137b0 con 0x7fbfec101d70 2024-03-22T06:11:26.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.904+0000 7fbfe3fff700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbfd80692a0 0x7fbfd806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:26.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.904+0000 7fbfe3fff700 1 -- 172.21.15.6:0/2856017036 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbfd4088180 con 0x7fbfec101d70 2024-03-22T06:11:26.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.904+0000 7fbfeaffd700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbfd80692a0 0x7fbfd806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:26.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.904+0000 7fbfe3fff700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586] conn(0x7fbfd806ef70 0x7fbfd8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:26.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.904+0000 7fbfe3fff700 1 -- 172.21.15.6:0/2856017036 --> [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfd8071a30 con 0x7fbfd806ef70 2024-03-22T06:11:26.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.904+0000 7fbfe3fff700 1 -- 172.21.15.6:0/2856017036 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfd408a770 con 0x7fbfec101d70 2024-03-22T06:11:26.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.904+0000 7fbfebfff700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586] conn(0x7fbfd806ef70 0x7fbfd8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:26.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.904+0000 7fbfeaffd700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbfd80692a0 0x7fbfd806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc006fd0 tx=0x7fbfdc008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:26.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.905+0000 7fbfebfff700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586] conn(0x7fbfd806ef70 0x7fbfd8071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:26.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.906+0000 7fbfe3fff700 1 -- 172.21.15.6:0/2856017036 <== osd.4 v2:172.21.15.117:6808/1798261586 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbfd8071a30 con 0x7fbfd806ef70 2024-03-22T06:11:26.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.921+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2856017036 --> [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbfec05e640 con 0x7fbfd806ef70 2024-03-22T06:11:26.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.921+0000 7fbfe3fff700 1 -- 172.21.15.6:0/2856017036 <== osd.4 v2:172.21.15.117:6808/1798261586 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbfec05e640 con 0x7fbfd806ef70 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2856017036 >> [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586] conn(0x7fbfd806ef70 msgr2=0x7fbfd8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586] conn(0x7fbfd806ef70 0x7fbfd8071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2856017036 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbfd80692a0 msgr2=0x7fbfd806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbfd80692a0 0x7fbfd806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc006fd0 tx=0x7fbfdc008040 comp rx=0 tx=0).stop 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2856017036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfec101d70 msgr2=0x7fbfec102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfec101d70 0x7fbfec102bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400b8b0 tx=0x7fbfd4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2856017036 shutdown_connections 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.117:6808/1798261586,v1:172.21.15.117:6809/1798261586] conn(0x7fbfd806ef70 0x7fbfd8071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbfd80692a0 0x7fbfd806b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfec101d70 0x7fbfec102bf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 --2- 172.21.15.6:0/2856017036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfec102710 0x7fbfec103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:26.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2856017036 >> 172.21.15.6:0/2856017036 conn(0x7fbfec072bc0 msgr2=0x7fbfec105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:26.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2856017036 shutdown_connections 2024-03-22T06:11:26.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.922+0000 7fbff1e0e700 1 -- 172.21.15.6:0/2856017036 wait complete. 2024-03-22T06:11:27.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.790+0000 7f6ec2def700 1 -- 172.21.15.6:0/983217570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ebc101d20 msgr2=0x7f6ebc1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:27.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.790+0000 7f6ec2def700 1 --2- 172.21.15.6:0/983217570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ebc101d20 0x7f6ebc1020f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6eac004940 tx=0x7f6eac009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:27.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.790+0000 7f6ec2def700 1 -- 172.21.15.6:0/983217570 shutdown_connections 2024-03-22T06:11:27.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.790+0000 7f6ec2def700 1 --2- 172.21.15.6:0/983217570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc1026c0 0x7f6ebc0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:27.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.790+0000 7f6ec2def700 1 --2- 172.21.15.6:0/983217570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ebc101d20 0x7f6ebc1020f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:27.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.790+0000 7f6ec2def700 1 -- 172.21.15.6:0/983217570 >> 172.21.15.6:0/983217570 conn(0x7f6ebc0f7870 msgr2=0x7f6ebc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:27.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.790+0000 7f6ec2def700 1 -- 172.21.15.6:0/983217570 shutdown_connections 2024-03-22T06:11:27.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.790+0000 7f6ec2def700 1 -- 172.21.15.6:0/983217570 wait complete. 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.791+0000 7f6ec2def700 1 Processor -- start 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.791+0000 7f6ec2def700 1 -- start start 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.791+0000 7f6ec2def700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ebc101d20 0x7f6ebc113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.791+0000 7f6ec2def700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc1026c0 0x7f6ebc113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.791+0000 7f6ec2def700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc114590 con 0x7f6ebc101d20 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.791+0000 7f6ec2def700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc10db20 con 0x7f6ebc1026c0 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.791+0000 7f6ec0b8b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ebc101d20 0x7f6ebc113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.791+0000 7f6ec0b8b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ebc101d20 0x7f6ebc113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45414/0 (socket says 172.21.15.6:45414) 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.791+0000 7f6ec0b8b700 1 -- 172.21.15.6:0/550737204 learned_addr learned my addr 172.21.15.6:0/550737204 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.792+0000 7f6ebbfff700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc1026c0 0x7f6ebc113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.792+0000 7f6ec0b8b700 1 -- 172.21.15.6:0/550737204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc1026c0 msgr2=0x7f6ebc113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.792+0000 7f6ec0b8b700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc1026c0 0x7f6ebc113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:27.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.792+0000 7f6ec0b8b700 1 -- 172.21.15.6:0/550737204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac005040 con 0x7f6ebc101d20 2024-03-22T06:11:27.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.792+0000 7f6ebbfff700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc1026c0 0x7f6ebc113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:11:27.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.792+0000 7f6ec0b8b700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ebc101d20 0x7f6ebc113970 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00b8b0 tx=0x7f6eac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:27.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.792+0000 7f6eb97fa700 1 -- 172.21.15.6:0/550737204 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eac00e040 con 0x7f6ebc101d20 2024-03-22T06:11:27.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.792+0000 7f6ec2def700 1 -- 172.21.15.6:0/550737204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc10dda0 con 0x7f6ebc101d20 2024-03-22T06:11:27.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.792+0000 7f6eb97fa700 1 -- 172.21.15.6:0/550737204 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6eac004290 con 0x7f6ebc101d20 2024-03-22T06:11:27.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.792+0000 7f6eb97fa700 1 -- 172.21.15.6:0/550737204 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eac01c3f0 con 0x7f6ebc101d20 2024-03-22T06:11:27.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.792+0000 7f6ec2def700 1 -- 172.21.15.6:0/550737204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc10e360 con 0x7f6ebc101d20 2024-03-22T06:11:27.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.793+0000 7f6ec2def700 1 -- 172.21.15.6:0/550737204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6ebc10e880 con 0x7f6ebc101d20 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.793+0000 7f6eb97fa700 1 -- 172.21.15.6:0/550737204 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f6eac0137b0 con 0x7f6ebc101d20 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.794+0000 7f6eb97fa700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ea40692a0 0x7f6ea406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.794+0000 7f6ebbfff700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ea40692a0 0x7f6ea406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.794+0000 7f6eb97fa700 1 -- 172.21.15.6:0/550737204 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6eac0881b0 con 0x7f6ebc101d20 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.794+0000 7f6eb97fa700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022] conn(0x7f6ea406ef70 0x7f6ea4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.794+0000 7f6eb97fa700 1 -- 172.21.15.6:0/550737204 --> [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ea4071a30 con 0x7f6ea406ef70 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.794+0000 7f6eb97fa700 1 -- 172.21.15.6:0/550737204 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eac08a7a0 con 0x7f6ebc101d20 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.794+0000 7f6ec138c700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022] conn(0x7f6ea406ef70 0x7f6ea4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.794+0000 7f6ebbfff700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ea40692a0 0x7f6ea406b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc10f210 tx=0x7f6eb0008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.794+0000 7f6ec138c700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022] conn(0x7f6ea406ef70 0x7f6ea4071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.795+0000 7f6eb97fa700 1 -- 172.21.15.6:0/550737204 <== osd.2 v2:172.21.15.6:6818/907023022 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6ea4071a30 con 0x7f6ea406ef70 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.811+0000 7f6ec2def700 1 -- 172.21.15.6:0/550737204 --> [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6ebc0fd470 con 0x7f6ea406ef70 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.811+0000 7f6eb97fa700 1 -- 172.21.15.6:0/550737204 <== osd.2 v2:172.21.15.6:6818/907023022 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6ebc0fd470 con 0x7f6ea406ef70 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 -- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022] conn(0x7f6ea406ef70 msgr2=0x7f6ea4071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022] conn(0x7f6ea406ef70 0x7f6ea4071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 -- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ea40692a0 msgr2=0x7f6ea406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ea40692a0 0x7f6ea406b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc10f210 tx=0x7f6eb0008040 comp rx=0 tx=0).stop 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 -- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ebc101d20 msgr2=0x7f6ebc113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ebc101d20 0x7f6ebc113970 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00b8b0 tx=0x7f6eac009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 -- 172.21.15.6:0/550737204 shutdown_connections 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6818/907023022,v1:172.21.15.6:6819/907023022] conn(0x7f6ea406ef70 0x7f6ea4071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ea40692a0 0x7f6ea406b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ebc101d20 0x7f6ebc113970 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 --2- 172.21.15.6:0/550737204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc1026c0 0x7f6ebc113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 -- 172.21.15.6:0/550737204 >> 172.21.15.6:0/550737204 conn(0x7f6ebc0f7870 msgr2=0x7f6ebc0f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 -- 172.21.15.6:0/550737204 shutdown_connections 2024-03-22T06:11:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:26.812+0000 7f6ec2def700 1 -- 172.21.15.6:0/550737204 wait complete. 2024-03-22T06:11:27.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:27 smithi006 ceph-mon[29301]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:27.317 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:27.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:27 smithi117 ceph-mon[36514]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:29.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:29 smithi006 ceph-mon[29301]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:29.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:29 smithi117 ceph-mon[36514]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:30.040 INFO:teuthology.orchestra.run.smithi006.stdout:150323855372 2024-03-22T06:11:30.041 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-22T06:11:30.998 INFO:teuthology.orchestra.run.smithi006.stdout:81604378648 2024-03-22T06:11:30.999 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-22T06:11:31.351 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:31 smithi006 ceph-mon[29301]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:31.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:31 smithi117 ceph-mon[36514]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:31.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.932+0000 7fafc763e700 1 -- 172.21.15.6:0/3877557422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc00fb4a0 msgr2=0x7fafc00fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:31.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.932+0000 7fafc763e700 1 --2- 172.21.15.6:0/3877557422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc00fb4a0 0x7fafc00fb910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fafb0004940 tx=0x7fafb0009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:31.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.932+0000 7fafc763e700 1 -- 172.21.15.6:0/3877557422 shutdown_connections 2024-03-22T06:11:31.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.932+0000 7fafc763e700 1 --2- 172.21.15.6:0/3877557422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc00fb4a0 0x7fafc00fb910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:31.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.932+0000 7fafc763e700 1 --2- 172.21.15.6:0/3877557422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102e70 0x7fafc0103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:31.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.932+0000 7fafc763e700 1 -- 172.21.15.6:0/3877557422 >> 172.21.15.6:0/3877557422 conn(0x7fafc00714c0 msgr2=0x7fafc00718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:31.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.933+0000 7fafc763e700 1 -- 172.21.15.6:0/3877557422 shutdown_connections 2024-03-22T06:11:31.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.933+0000 7fafc763e700 1 -- 172.21.15.6:0/3877557422 wait complete. 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc763e700 1 Processor -- start 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc763e700 1 -- start start 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc763e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc00fb4a0 0x7fafc0115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc763e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102e70 0x7fafc0116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc763e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0116680 con 0x7fafc00fb4a0 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc763e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc010fa70 con 0x7fafc0102e70 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc53da700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc00fb4a0 0x7fafc0115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc53da700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc00fb4a0 0x7fafc0115af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45462/0 (socket says 172.21.15.6:45462) 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc53da700 1 -- 172.21.15.6:0/4060494656 learned_addr learned my addr 172.21.15.6:0/4060494656 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc4bd9700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102e70 0x7fafc0116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc53da700 1 -- 172.21.15.6:0/4060494656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102e70 msgr2=0x7fafc0116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc53da700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102e70 0x7fafc0116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc53da700 1 -- 172.21.15.6:0/4060494656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0005040 con 0x7fafc00fb4a0 2024-03-22T06:11:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.934+0000 7fafc4bd9700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102e70 0x7fafc0116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:11:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.935+0000 7fafc53da700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc00fb4a0 0x7fafc0115af0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00c830 tx=0x7fafbc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.935+0000 7fafb5ffb700 1 -- 172.21.15.6:0/4060494656 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafbc00a760 con 0x7fafc00fb4a0 2024-03-22T06:11:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.935+0000 7fafc763e700 1 -- 172.21.15.6:0/4060494656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc010fd50 con 0x7fafc00fb4a0 2024-03-22T06:11:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.935+0000 7fafb5ffb700 1 -- 172.21.15.6:0/4060494656 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafbc00ad90 con 0x7fafc00fb4a0 2024-03-22T06:11:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.935+0000 7fafb5ffb700 1 -- 172.21.15.6:0/4060494656 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafbc0113a0 con 0x7fafc00fb4a0 2024-03-22T06:11:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.935+0000 7fafc763e700 1 -- 172.21.15.6:0/4060494656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc0110310 con 0x7fafc00fb4a0 2024-03-22T06:11:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.936+0000 7fafc763e700 1 -- 172.21.15.6:0/4060494656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fafc0110890 con 0x7fafc00fb4a0 2024-03-22T06:11:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.936+0000 7fafb5ffb700 1 -- 172.21.15.6:0/4060494656 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fafbc012400 con 0x7fafc00fb4a0 2024-03-22T06:11:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.937+0000 7fafb5ffb700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fafac0692a0 0x7fafac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.937+0000 7fafb5ffb700 1 -- 172.21.15.6:0/4060494656 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fafbc086ef0 con 0x7fafc00fb4a0 2024-03-22T06:11:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.937+0000 7fafc4bd9700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fafac0692a0 0x7fafac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.937+0000 7fafb5ffb700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486] conn(0x7fafac06ef70 0x7fafac071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.937+0000 7fafb5ffb700 1 -- 172.21.15.6:0/4060494656 --> [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafac071a30 con 0x7fafac06ef70 2024-03-22T06:11:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.937+0000 7fafb5ffb700 1 -- 172.21.15.6:0/4060494656 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafbc089540 con 0x7fafc00fb4a0 2024-03-22T06:11:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.937+0000 7fafc4bd9700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fafac0692a0 0x7fafac06b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fafb000b3b0 tx=0x7fafb0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.937+0000 7fafc5bdb700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486] conn(0x7fafac06ef70 0x7fafac071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.938+0000 7fafc5bdb700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486] conn(0x7fafac06ef70 0x7fafac071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:31.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.939+0000 7fafb5ffb700 1 -- 172.21.15.6:0/4060494656 <== osd.5 v2:172.21.15.117:6816/2701911486 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fafac071a30 con 0x7fafac06ef70 2024-03-22T06:11:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.953+0000 7fafc763e700 1 -- 172.21.15.6:0/4060494656 --> [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fafc00faa80 con 0x7fafac06ef70 2024-03-22T06:11:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.954+0000 7fafb5ffb700 1 -- 172.21.15.6:0/4060494656 <== osd.5 v2:172.21.15.117:6816/2701911486 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fafc00faa80 con 0x7fafac06ef70 2024-03-22T06:11:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.954+0000 7fafc763e700 1 -- 172.21.15.6:0/4060494656 >> [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486] conn(0x7fafac06ef70 msgr2=0x7fafac071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.954+0000 7fafc763e700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486] conn(0x7fafac06ef70 0x7fafac071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.954+0000 7fafc763e700 1 -- 172.21.15.6:0/4060494656 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fafac0692a0 msgr2=0x7fafac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.954+0000 7fafc763e700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fafac0692a0 0x7fafac06b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fafb000b3b0 tx=0x7fafb0009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.954+0000 7fafc763e700 1 -- 172.21.15.6:0/4060494656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc00fb4a0 msgr2=0x7fafc0115af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.954+0000 7fafc763e700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc00fb4a0 0x7fafc0115af0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00c830 tx=0x7fafbc004470 comp rx=0 tx=0).stop 2024-03-22T06:11:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.955+0000 7fafc763e700 1 -- 172.21.15.6:0/4060494656 shutdown_connections 2024-03-22T06:11:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.955+0000 7fafc763e700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.117:6816/2701911486,v1:172.21.15.117:6817/2701911486] conn(0x7fafac06ef70 0x7fafac071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.955+0000 7fafc763e700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fafac0692a0 0x7fafac06b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.955+0000 7fafc763e700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc00fb4a0 0x7fafc0115af0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.955+0000 7fafc763e700 1 --2- 172.21.15.6:0/4060494656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102e70 0x7fafc0116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.955+0000 7fafc763e700 1 -- 172.21.15.6:0/4060494656 >> 172.21.15.6:0/4060494656 conn(0x7fafc00714c0 msgr2=0x7fafc00fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.955+0000 7fafc763e700 1 -- 172.21.15.6:0/4060494656 shutdown_connections 2024-03-22T06:11:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:31.955+0000 7fafc763e700 1 -- 172.21.15.6:0/4060494656 wait complete. 2024-03-22T06:11:32.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:11:32.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:32 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:11:33.466 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:33 smithi006 ceph-mon[29301]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:33.495 INFO:teuthology.orchestra.run.smithi006.stdout:171798691850 2024-03-22T06:11:33.495 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-22T06:11:33.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:33 smithi117 ceph-mon[36514]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:33.953 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:34.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.765+0000 7f44515c1700 1 -- 172.21.15.6:0/1364565147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f444c102710 msgr2=0x7f444c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:34.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.765+0000 7f44515c1700 1 --2- 172.21.15.6:0/1364565147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f444c102710 0x7f444c0fbd80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4434004940 tx=0x7f4434009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:34.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.765+0000 7f44515c1700 1 -- 172.21.15.6:0/1364565147 shutdown_connections 2024-03-22T06:11:34.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.765+0000 7f44515c1700 1 --2- 172.21.15.6:0/1364565147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f444c102710 0x7f444c0fbd80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.765+0000 7f44515c1700 1 --2- 172.21.15.6:0/1364565147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f444c101d70 0x7f444c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.765+0000 7f44515c1700 1 -- 172.21.15.6:0/1364565147 >> 172.21.15.6:0/1364565147 conn(0x7f444c072bc0 msgr2=0x7f444c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.765+0000 7f44515c1700 1 -- 172.21.15.6:0/1364565147 shutdown_connections 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.765+0000 7f44515c1700 1 -- 172.21.15.6:0/1364565147 wait complete. 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.765+0000 7f44515c1700 1 Processor -- start 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f44515c1700 1 -- start start 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f44515c1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f444c101d70 0x7f444c10eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f44515c1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f444c102710 0x7f444c10f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f44515c1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c10f6d0 con 0x7f444c102710 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f44515c1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c10f810 con 0x7f444c101d70 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f444a7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f444c102710 0x7f444c10f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f444a7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f444c102710 0x7f444c10f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58300/0 (socket says 172.21.15.6:58300) 2024-03-22T06:11:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f444a7fc700 1 -- 172.21.15.6:0/618973318 learned_addr learned my addr 172.21.15.6:0/618973318 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:34.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f444a7fc700 1 -- 172.21.15.6:0/618973318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f444c101d70 msgr2=0x7f444c10eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:34.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f444a7fc700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f444c101d70 0x7f444c10eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:34.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f444a7fc700 1 -- 172.21.15.6:0/618973318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4434005040 con 0x7f444c102710 2024-03-22T06:11:34.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f444a7fc700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f444c102710 0x7f444c10f080 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4434009100 tx=0x7f4434009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:34.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f44437fe700 1 -- 172.21.15.6:0/618973318 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f443400e040 con 0x7f444c102710 2024-03-22T06:11:34.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f44437fe700 1 -- 172.21.15.6:0/618973318 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4434013420 con 0x7f444c102710 2024-03-22T06:11:34.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f44515c1700 1 -- 172.21.15.6:0/618973318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c117bb0 con 0x7f444c102710 2024-03-22T06:11:34.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.766+0000 7f44437fe700 1 -- 172.21.15.6:0/618973318 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f443401c610 con 0x7f444c102710 2024-03-22T06:11:34.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.767+0000 7f44515c1700 1 -- 172.21.15.6:0/618973318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c1180c0 con 0x7f444c102710 2024-03-22T06:11:34.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.767+0000 7f44515c1700 1 -- 172.21.15.6:0/618973318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f444c05e6c0 con 0x7f444c102710 2024-03-22T06:11:34.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.768+0000 7f44437fe700 1 -- 172.21.15.6:0/618973318 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f4434013590 con 0x7f444c102710 2024-03-22T06:11:34.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.768+0000 7f44437fe700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f44380692a0 0x7f443806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:34.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.768+0000 7f44437fe700 1 -- 172.21.15.6:0/618973318 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4434088090 con 0x7f444c102710 2024-03-22T06:11:34.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.768+0000 7f444affd700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f44380692a0 0x7f443806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:34.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.768+0000 7f44437fe700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829] conn(0x7f443806ef70 0x7f4438071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:34.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.768+0000 7f44437fe700 1 -- 172.21.15.6:0/618973318 --> [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4438071a30 con 0x7f443806ef70 2024-03-22T06:11:34.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.768+0000 7f44437fe700 1 -- 172.21.15.6:0/618973318 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f443408a6e0 con 0x7f444c102710 2024-03-22T06:11:34.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.769+0000 7f444b7fe700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829] conn(0x7f443806ef70 0x7f4438071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:34.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.769+0000 7f444affd700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f44380692a0 0x7f443806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f443c007fd0 tx=0x7f443c007b20 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:34.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.769+0000 7f444b7fe700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829] conn(0x7f443806ef70 0x7f4438071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:34.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.769+0000 7f44437fe700 1 -- 172.21.15.6:0/618973318 <== osd.1 v2:172.21.15.6:6810/3758615829 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f4438071a30 con 0x7f443806ef70 2024-03-22T06:11:34.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.785+0000 7f44515c1700 1 -- 172.21.15.6:0/618973318 --> [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f444c05eaf0 con 0x7f443806ef70 2024-03-22T06:11:34.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.785+0000 7f44437fe700 1 -- 172.21.15.6:0/618973318 <== osd.1 v2:172.21.15.6:6810/3758615829 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f444c05eaf0 con 0x7f443806ef70 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 -- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829] conn(0x7f443806ef70 msgr2=0x7f4438071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829] conn(0x7f443806ef70 0x7f4438071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 -- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f44380692a0 msgr2=0x7f443806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f44380692a0 0x7f443806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f443c007fd0 tx=0x7f443c007b20 comp rx=0 tx=0).stop 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 -- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f444c102710 msgr2=0x7f444c10f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f444c102710 0x7f444c10f080 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4434009100 tx=0x7f4434009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 -- 172.21.15.6:0/618973318 shutdown_connections 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6810/3758615829,v1:172.21.15.6:6811/3758615829] conn(0x7f443806ef70 0x7f4438071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f444c101d70 0x7f444c10eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f44380692a0 0x7f443806b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 --2- 172.21.15.6:0/618973318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f444c102710 0x7f444c10f080 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4434009100 tx=0x7f4434009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:34.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 -- 172.21.15.6:0/618973318 >> 172.21.15.6:0/618973318 conn(0x7f444c072bc0 msgr2=0x7f444c0fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:34.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 -- 172.21.15.6:0/618973318 shutdown_connections 2024-03-22T06:11:34.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:34.786+0000 7f44515c1700 1 -- 172.21.15.6:0/618973318 wait complete. 2024-03-22T06:11:35.308 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:35 smithi006 ceph-mon[29301]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:35.558 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:35 smithi117 ceph-mon[36514]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:35.811 INFO:teuthology.orchestra.run.smithi006.stdout:60129542175 2024-03-22T06:11:35.812 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-22T06:11:35.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.407+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2242625909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc102e10 msgr2=0x7fa5dc1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:35.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.407+0000 7fa5e3ac8700 1 --2- 172.21.15.6:0/2242625909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc102e10 0x7fa5dc1031e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc004940 tx=0x7fa5cc009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:35.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.407+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2242625909 shutdown_connections 2024-03-22T06:11:35.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.407+0000 7fa5e3ac8700 1 --2- 172.21.15.6:0/2242625909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5dc1037b0 0x7fa5dc0fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:35.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.407+0000 7fa5e3ac8700 1 --2- 172.21.15.6:0/2242625909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc102e10 0x7fa5dc1031e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:35.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.407+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2242625909 >> 172.21.15.6:0/2242625909 conn(0x7fa5dc0f8980 msgr2=0x7fa5dc0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:35.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.407+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2242625909 shutdown_connections 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.407+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2242625909 wait complete. 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e3ac8700 1 Processor -- start 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e3ac8700 1 -- start start 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e3ac8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5dc102e10 0x7fa5dc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e3ac8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc1037b0 0x7fa5dc070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e3ac8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc070d90 con 0x7fa5dc1037b0 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e3ac8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc070ed0 con 0x7fa5dc102e10 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e1063700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc1037b0 0x7fa5dc070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e1063700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc1037b0 0x7fa5dc070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58324/0 (socket says 172.21.15.6:58324) 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e1063700 1 -- 172.21.15.6:0/2622540795 learned_addr learned my addr 172.21.15.6:0/2622540795 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e1864700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5dc102e10 0x7fa5dc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e1063700 1 -- 172.21.15.6:0/2622540795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5dc102e10 msgr2=0x7fa5dc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:35.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e1063700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5dc102e10 0x7fa5dc0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e1063700 1 -- 172.21.15.6:0/2622540795 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc005040 con 0x7fa5dc1037b0 2024-03-22T06:11:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.408+0000 7fa5e1063700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc1037b0 0x7fa5dc070800 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8002a50 tx=0x7fa5d8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.409+0000 7fa5d27fc700 1 -- 172.21.15.6:0/2622540795 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d8007cc0 con 0x7fa5dc1037b0 2024-03-22T06:11:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.409+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2622540795 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5dc119ee0 con 0x7fa5dc1037b0 2024-03-22T06:11:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.409+0000 7fa5d27fc700 1 -- 172.21.15.6:0/2622540795 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5d800f040 con 0x7fa5dc1037b0 2024-03-22T06:11:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.409+0000 7fa5d27fc700 1 -- 172.21.15.6:0/2622540795 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d800d9c0 con 0x7fa5dc1037b0 2024-03-22T06:11:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.409+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2622540795 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5dc11a360 con 0x7fa5dc1037b0 2024-03-22T06:11:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.409+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2622540795 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa5dc05e6c0 con 0x7fa5dc1037b0 2024-03-22T06:11:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.410+0000 7fa5d27fc700 1 -- 172.21.15.6:0/2622540795 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fa5d8009730 con 0x7fa5dc1037b0 2024-03-22T06:11:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.410+0000 7fa5d27fc700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa5c80692a0 0x7fa5c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.410+0000 7fa5d27fc700 1 -- 172.21.15.6:0/2622540795 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa5d8086be0 con 0x7fa5dc1037b0 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.410+0000 7fa5e1864700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa5c80692a0 0x7fa5c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.411+0000 7fa5d27fc700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022] conn(0x7fa5c806ef70 0x7fa5c8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.411+0000 7fa5d27fc700 1 -- 172.21.15.6:0/2622540795 --> [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5c8071a30 con 0x7fa5c806ef70 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.411+0000 7fa5d27fc700 1 -- 172.21.15.6:0/2622540795 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5d808c1e0 con 0x7fa5dc1037b0 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.411+0000 7fa5e2065700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022] conn(0x7fa5c806ef70 0x7fa5c8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.411+0000 7fa5e1864700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa5c80692a0 0x7fa5c806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc004f90 tx=0x7fa5cc009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.411+0000 7fa5e2065700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022] conn(0x7fa5c806ef70 0x7fa5c8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.412+0000 7fa5d27fc700 1 -- 172.21.15.6:0/2622540795 <== osd.0 v2:172.21.15.6:6802/3577742022 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa5c8071a30 con 0x7fa5c806ef70 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.426+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2622540795 --> [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa5dc05eac0 con 0x7fa5c806ef70 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5d27fc700 1 -- 172.21.15.6:0/2622540795 <== osd.0 v2:172.21.15.6:6802/3577742022 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa5dc05eac0 con 0x7fa5c806ef70 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022] conn(0x7fa5c806ef70 msgr2=0x7fa5c8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022] conn(0x7fa5c806ef70 0x7fa5c8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa5c80692a0 msgr2=0x7fa5c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:35.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa5c80692a0 0x7fa5c806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc004f90 tx=0x7fa5cc009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:35.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc1037b0 msgr2=0x7fa5dc070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:35.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc1037b0 0x7fa5dc070800 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8002a50 tx=0x7fa5d8004600 comp rx=0 tx=0).stop 2024-03-22T06:11:35.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2622540795 shutdown_connections 2024-03-22T06:11:35.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6802/3577742022,v1:172.21.15.6:6803/3577742022] conn(0x7fa5c806ef70 0x7fa5c8071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:35.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5dc102e10 0x7fa5dc0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:35.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa5c80692a0 0x7fa5c806b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:35.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 --2- 172.21.15.6:0/2622540795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc1037b0 0x7fa5dc070800 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:35.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2622540795 >> 172.21.15.6:0/2622540795 conn(0x7fa5dc0f8980 msgr2=0x7fa5dc107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:35.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2622540795 shutdown_connections 2024-03-22T06:11:35.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:35.427+0000 7fa5e3ac8700 1 -- 172.21.15.6:0/2622540795 wait complete. 2024-03-22T06:11:36.539 INFO:teuthology.orchestra.run.smithi006.stdout:38654705701 2024-03-22T06:11:36.539 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-22T06:11:36.805 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:37.536 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:37 smithi006 ceph-mon[29301]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:37.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:37 smithi117 ceph-mon[36514]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:37.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.983+0000 7f4e101d0700 1 -- 172.21.15.6:0/1741277127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08102fd0 msgr2=0x7f4e081033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.983+0000 7f4e101d0700 1 --2- 172.21.15.6:0/1741277127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08102fd0 0x7f4e081033a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004940 tx=0x7f4e04009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.983+0000 7f4e101d0700 1 -- 172.21.15.6:0/1741277127 shutdown_connections 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.983+0000 7f4e101d0700 1 --2- 172.21.15.6:0/1741277127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e080fb540 0x7f4e080fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.983+0000 7f4e101d0700 1 --2- 172.21.15.6:0/1741277127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08102fd0 0x7f4e081033a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.983+0000 7f4e101d0700 1 -- 172.21.15.6:0/1741277127 >> 172.21.15.6:0/1741277127 conn(0x7f4e08071530 msgr2=0x7f4e08071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.984+0000 7f4e101d0700 1 -- 172.21.15.6:0/1741277127 shutdown_connections 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.984+0000 7f4e101d0700 1 -- 172.21.15.6:0/1741277127 wait complete. 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.984+0000 7f4e101d0700 1 Processor -- start 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.984+0000 7f4e101d0700 1 -- start start 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.984+0000 7f4e101d0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e080fb540 0x7f4e081139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.984+0000 7f4e101d0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08102fd0 0x7f4e08113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:37.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.984+0000 7f4e101d0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081145c0 con 0x7f4e08102fd0 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.984+0000 7f4e101d0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0810dba0 con 0x7f4e080fb540 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.984+0000 7f4e0d76b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08102fd0 0x7f4e08113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.984+0000 7f4e0d76b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08102fd0 0x7f4e08113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58344/0 (socket says 172.21.15.6:58344) 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.984+0000 7f4e0d76b700 1 -- 172.21.15.6:0/2562198314 learned_addr learned my addr 172.21.15.6:0/2562198314 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.985+0000 7f4e0df6c700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e080fb540 0x7f4e081139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.985+0000 7f4e0d76b700 1 -- 172.21.15.6:0/2562198314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e080fb540 msgr2=0x7f4e081139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.985+0000 7f4e0d76b700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e080fb540 0x7f4e081139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.985+0000 7f4e0d76b700 1 -- 172.21.15.6:0/2562198314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e04005040 con 0x7f4e08102fd0 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.985+0000 7f4e0d76b700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08102fd0 0x7f4e08113ee0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b790 tx=0x7f4df8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.985+0000 7f4dfe7fc700 1 -- 172.21.15.6:0/2562198314 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4df800f770 con 0x7f4e08102fd0 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.985+0000 7f4dfe7fc700 1 -- 172.21.15.6:0/2562198314 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4df800fda0 con 0x7f4e08102fd0 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.985+0000 7f4e101d0700 1 -- 172.21.15.6:0/2562198314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0810de80 con 0x7f4e08102fd0 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.985+0000 7f4dfe7fc700 1 -- 172.21.15.6:0/2562198314 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4df800d310 con 0x7f4e08102fd0 2024-03-22T06:11:37.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.985+0000 7f4e101d0700 1 -- 172.21.15.6:0/2562198314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0810e3c0 con 0x7f4e08102fd0 2024-03-22T06:11:37.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.986+0000 7f4e101d0700 1 -- 172.21.15.6:0/2562198314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4e0810e990 con 0x7f4e08102fd0 2024-03-22T06:11:37.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.986+0000 7f4dfe7fc700 1 -- 172.21.15.6:0/2562198314 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f4df8010420 con 0x7f4e08102fd0 2024-03-22T06:11:37.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.987+0000 7f4dfe7fc700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4df40692a0 0x7f4df406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:37.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.987+0000 7f4dfe7fc700 1 -- 172.21.15.6:0/2562198314 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4df8086da0 con 0x7f4e08102fd0 2024-03-22T06:11:37.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.987+0000 7f4e0df6c700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4df40692a0 0x7f4df406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:37.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.987+0000 7f4dfe7fc700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526] conn(0x7f4df406ef70 0x7f4df4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:37.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.987+0000 7f4dfe7fc700 1 -- 172.21.15.6:0/2562198314 --> [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4df4071a30 con 0x7f4df406ef70 2024-03-22T06:11:37.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.987+0000 7f4dfe7fc700 1 -- 172.21.15.6:0/2562198314 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4df80893b0 con 0x7f4e08102fd0 2024-03-22T06:11:37.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.987+0000 7f4e0df6c700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4df40692a0 0x7f4df406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004f90 tx=0x7f4e04009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:37.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.987+0000 7f4e0e76d700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526] conn(0x7f4df406ef70 0x7f4df4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:37.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.988+0000 7f4e0e76d700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526] conn(0x7f4df406ef70 0x7f4df4071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:37.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:37.989+0000 7f4dfe7fc700 1 -- 172.21.15.6:0/2562198314 <== osd.3 v2:172.21.15.117:6800/769002526 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f4df4071a30 con 0x7f4df406ef70 2024-03-22T06:11:38.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.004+0000 7f4e101d0700 1 -- 172.21.15.6:0/2562198314 --> [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4e0805e640 con 0x7f4df406ef70 2024-03-22T06:11:38.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4dfe7fc700 1 -- 172.21.15.6:0/2562198314 <== osd.3 v2:172.21.15.117:6800/769002526 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4e0805e640 con 0x7f4df406ef70 2024-03-22T06:11:38.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 -- 172.21.15.6:0/2562198314 >> [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526] conn(0x7f4df406ef70 msgr2=0x7f4df4071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:38.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526] conn(0x7f4df406ef70 0x7f4df4071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:38.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 -- 172.21.15.6:0/2562198314 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4df40692a0 msgr2=0x7f4df406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:38.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4df40692a0 0x7f4df406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004f90 tx=0x7f4e04009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:38.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 -- 172.21.15.6:0/2562198314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08102fd0 msgr2=0x7f4e08113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:38.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08102fd0 0x7f4e08113ee0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b790 tx=0x7f4df8009250 comp rx=0 tx=0).stop 2024-03-22T06:11:38.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 -- 172.21.15.6:0/2562198314 shutdown_connections 2024-03-22T06:11:38.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e080fb540 0x7f4e081139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:38.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.117:6800/769002526,v1:172.21.15.117:6801/769002526] conn(0x7f4df406ef70 0x7f4df4071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:38.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4df40692a0 0x7f4df406b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:38.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 --2- 172.21.15.6:0/2562198314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08102fd0 0x7f4e08113ee0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:38.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 -- 172.21.15.6:0/2562198314 >> 172.21.15.6:0/2562198314 conn(0x7f4e08071530 msgr2=0x7f4e080fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:38.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 -- 172.21.15.6:0/2562198314 shutdown_connections 2024-03-22T06:11:38.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:38.005+0000 7f4e101d0700 1 -- 172.21.15.6:0/2562198314 wait complete. 2024-03-22T06:11:38.744 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:38.744 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:38.744 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:39.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:39 smithi006 ceph-mon[29301]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:39.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:39 smithi117 ceph-mon[36514]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:39.850 INFO:teuthology.orchestra.run.smithi006.stdout:115964117012 2024-03-22T06:11:39.850 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-22T06:11:41.605 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:41 smithi006 ceph-mon[29301]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:41.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:41 smithi117 ceph-mon[36514]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:43.427 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:43 smithi006 ceph-mon[29301]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:43.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:43 smithi117 ceph-mon[36514]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:43.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.913+0000 7f0eb9b04700 1 -- 172.21.15.6:0/3581218198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb4102e00 msgr2=0x7f0eb41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:43.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.913+0000 7f0eb9b04700 1 --2- 172.21.15.6:0/3581218198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb4102e00 0x7f0eb41031d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4004940 tx=0x7f0ea4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:43.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.913+0000 7f0eb9b04700 1 -- 172.21.15.6:0/3581218198 shutdown_connections 2024-03-22T06:11:43.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.913+0000 7f0eb9b04700 1 --2- 172.21.15.6:0/3581218198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb41037a0 0x7f0eb40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:43.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.913+0000 7f0eb9b04700 1 --2- 172.21.15.6:0/3581218198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb4102e00 0x7f0eb41031d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:43.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.913+0000 7f0eb9b04700 1 -- 172.21.15.6:0/3581218198 >> 172.21.15.6:0/3581218198 conn(0x7f0eb40f89b0 msgr2=0x7f0eb40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:43.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.913+0000 7f0eb9b04700 1 -- 172.21.15.6:0/3581218198 shutdown_connections 2024-03-22T06:11:43.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.913+0000 7f0eb9b04700 1 -- 172.21.15.6:0/3581218198 wait complete. 2024-03-22T06:11:43.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.913+0000 7f0eb9b04700 1 Processor -- start 2024-03-22T06:11:43.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb9b04700 1 -- start start 2024-03-22T06:11:43.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb9b04700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb4102e00 0x7f0eb40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:43.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb9b04700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb41037a0 0x7f0eb4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:43.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb9b04700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40721c0 con 0x7f0eb4102e00 2024-03-22T06:11:43.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb9b04700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4070d40 con 0x7f0eb41037a0 2024-03-22T06:11:43.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb4102e00 0x7f0eb40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:43.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb4102e00 0x7f0eb40702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45638/0 (socket says 172.21.15.6:45638) 2024-03-22T06:11:43.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb37fe700 1 -- 172.21.15.6:0/704557195 learned_addr learned my addr 172.21.15.6:0/704557195 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:43.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb2ffd700 1 --2- 172.21.15.6:0/704557195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb41037a0 0x7f0eb4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:43.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb37fe700 1 -- 172.21.15.6:0/704557195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb41037a0 msgr2=0x7f0eb4070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:43.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb37fe700 1 --2- 172.21.15.6:0/704557195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb41037a0 0x7f0eb4070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:43.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb37fe700 1 -- 172.21.15.6:0/704557195 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4005040 con 0x7f0eb4102e00 2024-03-22T06:11:43.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.914+0000 7f0eb37fe700 1 --2- 172.21.15.6:0/704557195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb4102e00 0x7f0eb40702c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400b8b0 tx=0x7f0ea4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:43.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.915+0000 7f0e9bfff700 1 -- 172.21.15.6:0/704557195 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ea400e040 con 0x7f0eb4102e00 2024-03-22T06:11:43.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.915+0000 7f0e9bfff700 1 -- 172.21.15.6:0/704557195 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ea4004290 con 0x7f0eb4102e00 2024-03-22T06:11:43.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.915+0000 7f0eb9b04700 1 -- 172.21.15.6:0/704557195 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb4070fc0 con 0x7f0eb4102e00 2024-03-22T06:11:43.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.915+0000 7f0e9bfff700 1 -- 172.21.15.6:0/704557195 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ea401c3f0 con 0x7f0eb4102e00 2024-03-22T06:11:43.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.915+0000 7f0eb9b04700 1 -- 172.21.15.6:0/704557195 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb411a070 con 0x7f0eb4102e00 2024-03-22T06:11:43.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.916+0000 7f0eb9b04700 1 -- 172.21.15.6:0/704557195 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb4100110 con 0x7f0eb4102e00 2024-03-22T06:11:43.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.916+0000 7f0e9bfff700 1 -- 172.21.15.6:0/704557195 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f0ea4013420 con 0x7f0eb4102e00 2024-03-22T06:11:43.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.916+0000 7f0e9bfff700 1 --2- 172.21.15.6:0/704557195 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e9c0692a0 0x7f0e9c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:43.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.916+0000 7f0e9bfff700 1 -- 172.21.15.6:0/704557195 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0ea40880a0 con 0x7f0eb4102e00 2024-03-22T06:11:43.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.917+0000 7f0eb2ffd700 1 --2- 172.21.15.6:0/704557195 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e9c0692a0 0x7f0e9c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:43.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.917+0000 7f0eb2ffd700 1 --2- 172.21.15.6:0/704557195 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e9c0692a0 0x7f0e9c06b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40719a0 tx=0x7f0ea800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:43.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:43.921+0000 7f0e9bfff700 1 -- 172.21.15.6:0/704557195 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0ea4054710 con 0x7f0eb4102e00 2024-03-22T06:11:44.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.088+0000 7f9fea160700 1 -- 172.21.15.6:0/566758286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe41037b0 msgr2=0x7f9fe40fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:44.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.088+0000 7f9fea160700 1 --2- 172.21.15.6:0/566758286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe41037b0 0x7f9fe40fce20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4004940 tx=0x7f9fd4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:44.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.088+0000 7f9fea160700 1 -- 172.21.15.6:0/566758286 shutdown_connections 2024-03-22T06:11:44.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.088+0000 7f9fea160700 1 --2- 172.21.15.6:0/566758286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe41037b0 0x7f9fe40fce20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:44.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.088+0000 7f9fea160700 1 --2- 172.21.15.6:0/566758286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fe4102e10 0x7f9fe41031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:44.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.088+0000 7f9fea160700 1 -- 172.21.15.6:0/566758286 >> 172.21.15.6:0/566758286 conn(0x7f9fe40f89a0 msgr2=0x7f9fe40fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:44.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.088+0000 7f9fea160700 1 -- 172.21.15.6:0/566758286 shutdown_connections 2024-03-22T06:11:44.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.088+0000 7f9fea160700 1 -- 172.21.15.6:0/566758286 wait complete. 2024-03-22T06:11:44.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.089+0000 7f9fea160700 1 Processor -- start 2024-03-22T06:11:44.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.089+0000 7f9fea160700 1 -- start start 2024-03-22T06:11:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.089+0000 7f9fea160700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fe4102e10 0x7f9fe40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.089+0000 7f9fea160700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe41037b0 0x7f9fe4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.089+0000 7f9fea160700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4070d90 con 0x7f9fe41037b0 2024-03-22T06:11:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.089+0000 7f9fea160700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4070ed0 con 0x7f9fe4102e10 2024-03-22T06:11:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.089+0000 7f9fe2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe41037b0 0x7f9fe4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.089+0000 7f9fe2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe41037b0 0x7f9fe4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45654/0 (socket says 172.21.15.6:45654) 2024-03-22T06:11:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.089+0000 7f9fe2ffd700 1 -- 172.21.15.6:0/3744527667 learned_addr learned my addr 172.21.15.6:0/3744527667 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.089+0000 7f9fe37fe700 1 --2- 172.21.15.6:0/3744527667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fe4102e10 0x7f9fe40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.090+0000 7f9fe2ffd700 1 -- 172.21.15.6:0/3744527667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fe4102e10 msgr2=0x7f9fe40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.090+0000 7f9fe2ffd700 1 --2- 172.21.15.6:0/3744527667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fe4102e10 0x7f9fe40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.090+0000 7f9fe2ffd700 1 -- 172.21.15.6:0/3744527667 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4005040 con 0x7f9fe41037b0 2024-03-22T06:11:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.090+0000 7f9fe2ffd700 1 --2- 172.21.15.6:0/3744527667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe41037b0 0x7f9fe4070800 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009100 tx=0x7f9fd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:44.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.090+0000 7f9fdbfff700 1 -- 172.21.15.6:0/3744527667 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fd400e040 con 0x7f9fe41037b0 2024-03-22T06:11:44.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.090+0000 7f9fdbfff700 1 -- 172.21.15.6:0/3744527667 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9fd4009af0 con 0x7f9fe41037b0 2024-03-22T06:11:44.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.090+0000 7f9fea160700 1 -- 172.21.15.6:0/3744527667 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe4071120 con 0x7f9fe41037b0 2024-03-22T06:11:44.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.090+0000 7f9fdbfff700 1 -- 172.21.15.6:0/3744527667 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fd401c3f0 con 0x7f9fe41037b0 2024-03-22T06:11:44.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.090+0000 7f9fea160700 1 -- 172.21.15.6:0/3744527667 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe411a2d0 con 0x7f9fe41037b0 2024-03-22T06:11:44.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.091+0000 7f9fea160700 1 -- 172.21.15.6:0/3744527667 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe41052e0 con 0x7f9fe41037b0 2024-03-22T06:11:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.091+0000 7f9fdbfff700 1 -- 172.21.15.6:0/3744527667 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f9fd400baf0 con 0x7f9fe41037b0 2024-03-22T06:11:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.092+0000 7f9fdbfff700 1 --2- 172.21.15.6:0/3744527667 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fd00692a0 0x7f9fd006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.092+0000 7f9fdbfff700 1 -- 172.21.15.6:0/3744527667 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9fd40880f0 con 0x7f9fe41037b0 2024-03-22T06:11:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.092+0000 7f9fe37fe700 1 --2- 172.21.15.6:0/3744527667 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fd00692a0 0x7f9fd006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.092+0000 7f9fe37fe700 1 --2- 172.21.15.6:0/3744527667 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fd00692a0 0x7f9fd006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc004eb0 tx=0x7f9fcc00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:44.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.096+0000 7f9fdbfff700 1 -- 172.21.15.6:0/3744527667 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9fd4054760 con 0x7f9fe41037b0 2024-03-22T06:11:44.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.121+0000 7f0eb9b04700 1 -- 172.21.15.6:0/704557195 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0eb4109f90 con 0x7f0eb4102e00 2024-03-22T06:11:44.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.121+0000 7f0e9bfff700 1 -- 172.21.15.6:0/704557195 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ea4057d30 con 0x7f0eb4102e00 2024-03-22T06:11:44.122 INFO:teuthology.orchestra.run.smithi006.stdout:171798691852 2024-03-22T06:11:44.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.123+0000 7f0eb9b04700 1 -- 172.21.15.6:0/704557195 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e9c0692a0 msgr2=0x7f0e9c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:44.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.123+0000 7f0eb9b04700 1 --2- 172.21.15.6:0/704557195 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e9c0692a0 0x7f0e9c06b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40719a0 tx=0x7f0ea800b040 comp rx=0 tx=0).stop 2024-03-22T06:11:44.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.123+0000 7f0eb9b04700 1 -- 172.21.15.6:0/704557195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb4102e00 msgr2=0x7f0eb40702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:44.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.123+0000 7f0eb9b04700 1 --2- 172.21.15.6:0/704557195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb4102e00 0x7f0eb40702c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400b8b0 tx=0x7f0ea4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:44.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.123+0000 7f0eb9b04700 1 -- 172.21.15.6:0/704557195 shutdown_connections 2024-03-22T06:11:44.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.123+0000 7f0eb9b04700 1 --2- 172.21.15.6:0/704557195 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0e9c0692a0 0x7f0e9c06b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:44.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.123+0000 7f0eb9b04700 1 --2- 172.21.15.6:0/704557195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb4102e00 0x7f0eb40702c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:44.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.123+0000 7f0eb9b04700 1 --2- 172.21.15.6:0/704557195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb41037a0 0x7f0eb4070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:44.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.123+0000 7f0eb9b04700 1 -- 172.21.15.6:0/704557195 >> 172.21.15.6:0/704557195 conn(0x7f0eb40f89b0 msgr2=0x7f0eb40fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:44.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.124+0000 7f0eb9b04700 1 -- 172.21.15.6:0/704557195 shutdown_connections 2024-03-22T06:11:44.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.124+0000 7f0eb9b04700 1 -- 172.21.15.6:0/704557195 wait complete. 2024-03-22T06:11:44.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.303+0000 7f9fea160700 1 -- 172.21.15.6:0/3744527667 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9fe4071a10 con 0x7f9fe41037b0 2024-03-22T06:11:44.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.303+0000 7f9fdbfff700 1 -- 172.21.15.6:0/3744527667 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9fd4086e60 con 0x7f9fe41037b0 2024-03-22T06:11:44.304 INFO:teuthology.orchestra.run.smithi006.stdout:81604378651 2024-03-22T06:11:44.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.305+0000 7f9fea160700 1 -- 172.21.15.6:0/3744527667 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fd00692a0 msgr2=0x7f9fd006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:44.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.305+0000 7f9fea160700 1 --2- 172.21.15.6:0/3744527667 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fd00692a0 0x7f9fd006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc004eb0 tx=0x7f9fcc00b040 comp rx=0 tx=0).stop 2024-03-22T06:11:44.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.305+0000 7f9fea160700 1 -- 172.21.15.6:0/3744527667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe41037b0 msgr2=0x7f9fe4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:44.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.305+0000 7f9fea160700 1 --2- 172.21.15.6:0/3744527667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe41037b0 0x7f9fe4070800 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009100 tx=0x7f9fd4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:44.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.305+0000 7f9fea160700 1 -- 172.21.15.6:0/3744527667 shutdown_connections 2024-03-22T06:11:44.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.305+0000 7f9fea160700 1 --2- 172.21.15.6:0/3744527667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fe4102e10 0x7f9fe40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:44.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.305+0000 7f9fea160700 1 --2- 172.21.15.6:0/3744527667 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fd00692a0 0x7f9fd006b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:44.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.305+0000 7f9fea160700 1 --2- 172.21.15.6:0/3744527667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe41037b0 0x7f9fe4070800 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:44.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.305+0000 7f9fea160700 1 -- 172.21.15.6:0/3744527667 >> 172.21.15.6:0/3744527667 conn(0x7f9fe40f89a0 msgr2=0x7f9fe40face0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:44.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.306+0000 7f9fea160700 1 -- 172.21.15.6:0/3744527667 shutdown_connections 2024-03-22T06:11:44.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.306+0000 7f9fea160700 1 -- 172.21.15.6:0/3744527667 wait complete. 2024-03-22T06:11:44.551 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:44 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/704557195' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-22T06:11:44.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:44 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/704557195' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-22T06:11:45.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.943+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4247167672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dd0101d20 msgr2=0x7f1dd01020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.943+0000 7f1dd48f3700 1 --2- 172.21.15.6:0/4247167672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dd0101d20 0x7f1dd01020f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0004940 tx=0x7f1dc0009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.943+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4247167672 shutdown_connections 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.943+0000 7f1dd48f3700 1 --2- 172.21.15.6:0/4247167672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dd01026c0 0x7f1dd00fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.943+0000 7f1dd48f3700 1 --2- 172.21.15.6:0/4247167672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dd0101d20 0x7f1dd01020f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.943+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4247167672 >> 172.21.15.6:0/4247167672 conn(0x7f1dd00f7870 msgr2=0x7f1dd00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.943+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4247167672 shutdown_connections 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.943+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4247167672 wait complete. 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dd48f3700 1 Processor -- start 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dd48f3700 1 -- start start 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dd48f3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dd0101d20 0x7f1dd010eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dd48f3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dd01026c0 0x7f1dd010f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dd48f3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd010f5e0 con 0x7f1dd01026c0 2024-03-22T06:11:45.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dd48f3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd010f720 con 0x7f1dd0101d20 2024-03-22T06:11:45.366 INFO:teuthology.orchestra.run.smithi006.stdout:150323855376 2024-03-22T06:11:45.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dcdd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dd01026c0 0x7f1dd010f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:45.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dcdd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dd01026c0 0x7f1dd010f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45658/0 (socket says 172.21.15.6:45658) 2024-03-22T06:11:45.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dcdd9b700 1 -- 172.21.15.6:0/4189897881 learned_addr learned my addr 172.21.15.6:0/4189897881 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:45.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dce59c700 1 --2- 172.21.15.6:0/4189897881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dd0101d20 0x7f1dd010eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:45.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dcdd9b700 1 -- 172.21.15.6:0/4189897881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dd0101d20 msgr2=0x7f1dd010eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:45.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.944+0000 7f1dcdd9b700 1 --2- 172.21.15.6:0/4189897881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dd0101d20 0x7f1dd010eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.945+0000 7f1dcdd9b700 1 -- 172.21.15.6:0/4189897881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc0005040 con 0x7f1dd01026c0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.945+0000 7f1dcdd9b700 1 --2- 172.21.15.6:0/4189897881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dd01026c0 0x7f1dd010f0a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400b790 tx=0x7f1dc4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.945+0000 7f1dbeffd700 1 -- 172.21.15.6:0/4189897881 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dc400f6f0 con 0x7f1dd01026c0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.945+0000 7f1dbeffd700 1 -- 172.21.15.6:0/4189897881 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1dc400fd20 con 0x7f1dd01026c0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.945+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4189897881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd0117bd0 con 0x7f1dd01026c0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.945+0000 7f1dbeffd700 1 -- 172.21.15.6:0/4189897881 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dc40103a0 con 0x7f1dd01026c0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.945+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4189897881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd0117ff0 con 0x7f1dd01026c0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.946+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4189897881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd00fd470 con 0x7f1dd01026c0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.946+0000 7f1dbeffd700 1 -- 172.21.15.6:0/4189897881 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f1dc4011400 con 0x7f1dd01026c0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.947+0000 7f1dbeffd700 1 --2- 172.21.15.6:0/4189897881 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f1db80692a0 0x7f1db806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.947+0000 7f1dce59c700 1 --2- 172.21.15.6:0/4189897881 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f1db80692a0 0x7f1db806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.947+0000 7f1dbeffd700 1 -- 172.21.15.6:0/4189897881 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1dc4086ba0 con 0x7f1dd01026c0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.947+0000 7f1dce59c700 1 --2- 172.21.15.6:0/4189897881 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f1db80692a0 0x7f1db806b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000b3b0 tx=0x7f1dc0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:45.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:44.951+0000 7f1dbeffd700 1 -- 172.21.15.6:0/4189897881 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1dc4053210 con 0x7f1dd01026c0 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.147+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4189897881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1dd004c710 con 0x7f1dd01026c0 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.147+0000 7f1dbeffd700 1 -- 172.21.15.6:0/4189897881 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1dc405ae60 con 0x7f1dd01026c0 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.149+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4189897881 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f1db80692a0 msgr2=0x7f1db806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.149+0000 7f1dd48f3700 1 --2- 172.21.15.6:0/4189897881 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f1db80692a0 0x7f1db806b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000b3b0 tx=0x7f1dc0009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.149+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4189897881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dd01026c0 msgr2=0x7f1dd010f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.149+0000 7f1dd48f3700 1 --2- 172.21.15.6:0/4189897881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dd01026c0 0x7f1dd010f0a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400b790 tx=0x7f1dc4009220 comp rx=0 tx=0).stop 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.149+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4189897881 shutdown_connections 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.149+0000 7f1dd48f3700 1 --2- 172.21.15.6:0/4189897881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dd0101d20 0x7f1dd010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.149+0000 7f1dd48f3700 1 --2- 172.21.15.6:0/4189897881 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f1db80692a0 0x7f1db806b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.149+0000 7f1dd48f3700 1 --2- 172.21.15.6:0/4189897881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1dd01026c0 0x7f1dd010f0a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.149+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4189897881 >> 172.21.15.6:0/4189897881 conn(0x7f1dd00f7870 msgr2=0x7f1dd00f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.149+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4189897881 shutdown_connections 2024-03-22T06:11:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:45.149+0000 7f1dd48f3700 1 -- 172.21.15.6:0/4189897881 wait complete. 2024-03-22T06:11:45.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:45 smithi006 ceph-mon[29301]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:45.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:45 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3744527667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-22T06:11:45.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:45 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/4189897881' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-22T06:11:45.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:45 smithi117 ceph-mon[36514]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:45.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:45 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3744527667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-22T06:11:45.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:45 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/4189897881' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-22T06:11:46.027 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378651 for osd.2 2024-03-22T06:11:46.027 DEBUG:teuthology.parallel:result is None 2024-03-22T06:11:46.522 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691850 got 171798691852 for osd.5 2024-03-22T06:11:46.522 DEBUG:teuthology.parallel:result is None 2024-03-22T06:11:46.910 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855372 got 150323855376 for osd.4 2024-03-22T06:11:46.910 DEBUG:teuthology.parallel:result is None 2024-03-22T06:11:47.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.152+0000 7fb0cb5da700 1 -- 172.21.15.6:0/74755489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0c4102720 msgr2=0x7fb0c40fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.152+0000 7fb0cb5da700 1 --2- 172.21.15.6:0/74755489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0c4102720 0x7fb0c40fbd90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4004940 tx=0x7fb0b4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.152+0000 7fb0cb5da700 1 -- 172.21.15.6:0/74755489 shutdown_connections 2024-03-22T06:11:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.152+0000 7fb0cb5da700 1 --2- 172.21.15.6:0/74755489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0c4102720 0x7fb0c40fbd90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.152+0000 7fb0cb5da700 1 --2- 172.21.15.6:0/74755489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0c4101d80 0x7fb0c4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.152+0000 7fb0cb5da700 1 -- 172.21.15.6:0/74755489 >> 172.21.15.6:0/74755489 conn(0x7fb0c40f78f0 msgr2=0x7fb0c40f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.152+0000 7fb0cb5da700 1 -- 172.21.15.6:0/74755489 shutdown_connections 2024-03-22T06:11:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.152+0000 7fb0cb5da700 1 -- 172.21.15.6:0/74755489 wait complete. 2024-03-22T06:11:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.152+0000 7fb0cb5da700 1 Processor -- start 2024-03-22T06:11:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0cb5da700 1 -- start start 2024-03-22T06:11:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0cb5da700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0c4101d80 0x7fb0c4115b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0cb5da700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0c4102720 0x7fb0c4116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0cb5da700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c41166b0 con 0x7fb0c4101d80 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0cb5da700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c410fc90 con 0x7fb0c4102720 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0c9376700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0c4101d80 0x7fb0c4115b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0c9376700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0c4101d80 0x7fb0c4115b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45678/0 (socket says 172.21.15.6:45678) 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0c9376700 1 -- 172.21.15.6:0/3512565101 learned_addr learned my addr 172.21.15.6:0/3512565101 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0c8b75700 1 --2- 172.21.15.6:0/3512565101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0c4102720 0x7fb0c4116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0c9376700 1 -- 172.21.15.6:0/3512565101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0c4102720 msgr2=0x7fb0c4116060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0c9376700 1 --2- 172.21.15.6:0/3512565101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0c4102720 0x7fb0c4116060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.153+0000 7fb0c9376700 1 -- 172.21.15.6:0/3512565101 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b4005040 con 0x7fb0c4101d80 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.154+0000 7fb0c9376700 1 --2- 172.21.15.6:0/3512565101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0c4101d80 0x7fb0c4115b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000eb40 tx=0x7fb0c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.154+0000 7fb0b9ffb700 1 -- 172.21.15.6:0/3512565101 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0c0007cc0 con 0x7fb0c4101d80 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.154+0000 7fb0cb5da700 1 -- 172.21.15.6:0/3512565101 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c410ff10 con 0x7fb0c4101d80 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.154+0000 7fb0b9ffb700 1 -- 172.21.15.6:0/3512565101 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0c000f040 con 0x7fb0c4101d80 2024-03-22T06:11:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.154+0000 7fb0b9ffb700 1 -- 172.21.15.6:0/3512565101 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0c000ca10 con 0x7fb0c4101d80 2024-03-22T06:11:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.154+0000 7fb0cb5da700 1 -- 172.21.15.6:0/3512565101 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c4110420 con 0x7fb0c4101d80 2024-03-22T06:11:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.155+0000 7fb0cb5da700 1 -- 172.21.15.6:0/3512565101 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c40fd4d0 con 0x7fb0c4101d80 2024-03-22T06:11:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.155+0000 7fb0b9ffb700 1 -- 172.21.15.6:0/3512565101 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fb0c000cc80 con 0x7fb0c4101d80 2024-03-22T06:11:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.156+0000 7fb0b9ffb700 1 --2- 172.21.15.6:0/3512565101 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb0b00692a0 0x7fb0b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.156+0000 7fb0b9ffb700 1 -- 172.21.15.6:0/3512565101 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb0c0086a40 con 0x7fb0c4101d80 2024-03-22T06:11:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.156+0000 7fb0c8b75700 1 --2- 172.21.15.6:0/3512565101 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb0b00692a0 0x7fb0b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.156+0000 7fb0c8b75700 1 --2- 172.21.15.6:0/3512565101 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb0b00692a0 0x7fb0b006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4004890 tx=0x7fb0b4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:47.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.160+0000 7fb0b9ffb700 1 -- 172.21.15.6:0/3512565101 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb0c004e190 con 0x7fb0c4101d80 2024-03-22T06:11:47.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.361+0000 7fb0cb5da700 1 -- 172.21.15.6:0/3512565101 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb0c4111170 con 0x7fb0c4101d80 2024-03-22T06:11:47.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.362+0000 7fb0b9ffb700 1 -- 172.21.15.6:0/3512565101 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb0c0056b70 con 0x7fb0c4101d80 2024-03-22T06:11:47.363 INFO:teuthology.orchestra.run.smithi006.stdout:60129542178 2024-03-22T06:11:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.364+0000 7fb0cb5da700 1 -- 172.21.15.6:0/3512565101 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb0b00692a0 msgr2=0x7fb0b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.364+0000 7fb0cb5da700 1 --2- 172.21.15.6:0/3512565101 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb0b00692a0 0x7fb0b006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4004890 tx=0x7fb0b4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.364+0000 7fb0cb5da700 1 -- 172.21.15.6:0/3512565101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0c4101d80 msgr2=0x7fb0c4115b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.364+0000 7fb0cb5da700 1 --2- 172.21.15.6:0/3512565101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0c4101d80 0x7fb0c4115b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000eb40 tx=0x7fb0c000b040 comp rx=0 tx=0).stop 2024-03-22T06:11:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.364+0000 7fb0cb5da700 1 -- 172.21.15.6:0/3512565101 shutdown_connections 2024-03-22T06:11:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.364+0000 7fb0cb5da700 1 --2- 172.21.15.6:0/3512565101 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb0b00692a0 0x7fb0b006b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.364+0000 7fb0cb5da700 1 --2- 172.21.15.6:0/3512565101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0c4101d80 0x7fb0c4115b20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.364+0000 7fb0cb5da700 1 --2- 172.21.15.6:0/3512565101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0c4102720 0x7fb0c4116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.364+0000 7fb0cb5da700 1 -- 172.21.15.6:0/3512565101 >> 172.21.15.6:0/3512565101 conn(0x7fb0c40f78f0 msgr2=0x7fb0c4066550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.364+0000 7fb0cb5da700 1 -- 172.21.15.6:0/3512565101 shutdown_connections 2024-03-22T06:11:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.364+0000 7fb0cb5da700 1 -- 172.21.15.6:0/3512565101 wait complete. 2024-03-22T06:11:47.410 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:47 smithi006 ceph-mon[29301]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:47.410 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:11:47.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:47 smithi117 ceph-mon[36514]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:11:47.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.837+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2782195851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb0c0fde90 msgr2=0x7ffb0c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:47.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.837+0000 7ffb13d2d700 1 --2- 172.21.15.6:0/2782195851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb0c0fde90 0x7ffb0c0fe260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb04004940 tx=0x7ffb04009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:47.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.837+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2782195851 shutdown_connections 2024-03-22T06:11:47.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.837+0000 7ffb13d2d700 1 --2- 172.21.15.6:0/2782195851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb0c0fe830 0x7ffb0c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:47.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.837+0000 7ffb13d2d700 1 --2- 172.21.15.6:0/2782195851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb0c0fde90 0x7ffb0c0fe260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:47.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.837+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2782195851 >> 172.21.15.6:0/2782195851 conn(0x7ffb0c072bc0 msgr2=0x7ffb0c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:47.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.837+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2782195851 shutdown_connections 2024-03-22T06:11:47.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.837+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2782195851 wait complete. 2024-03-22T06:11:47.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.837+0000 7ffb13d2d700 1 Processor -- start 2024-03-22T06:11:47.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.837+0000 7ffb13d2d700 1 -- start start 2024-03-22T06:11:47.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb13d2d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb0c0fde90 0x7ffb0c113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:47.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb13d2d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb0c0fe830 0x7ffb0c1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:47.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb13d2d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1147a0 con 0x7ffb0c0fde90 2024-03-22T06:11:47.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb13d2d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c10db50 con 0x7ffb0c0fe830 2024-03-22T06:11:47.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb11ac9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb0c0fde90 0x7ffb0c113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:47.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb11ac9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb0c0fde90 0x7ffb0c113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45694/0 (socket says 172.21.15.6:45694) 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb11ac9700 1 -- 172.21.15.6:0/2298794983 learned_addr learned my addr 172.21.15.6:0/2298794983 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb11ac9700 1 -- 172.21.15.6:0/2298794983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb0c0fe830 msgr2=0x7ffb0c1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb11ac9700 1 --2- 172.21.15.6:0/2298794983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb0c0fe830 0x7ffb0c1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb11ac9700 1 -- 172.21.15.6:0/2298794983 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb04005040 con 0x7ffb0c0fde90 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb11ac9700 1 --2- 172.21.15.6:0/2298794983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb0c0fde90 0x7ffb0c113b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400b3b0 tx=0x7ffb04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb0a7fc700 1 -- 172.21.15.6:0/2298794983 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb0400e040 con 0x7ffb0c0fde90 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb0a7fc700 1 -- 172.21.15.6:0/2298794983 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb04013420 con 0x7ffb0c0fde90 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2298794983 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c10ddd0 con 0x7ffb0c0fde90 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb0a7fc700 1 -- 172.21.15.6:0/2298794983 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb0401c610 con 0x7ffb0c0fde90 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.838+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2298794983 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c10e310 con 0x7ffb0c0fde90 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.839+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2298794983 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c065e30 con 0x7ffb0c0fde90 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.840+0000 7ffb0a7fc700 1 -- 172.21.15.6:0/2298794983 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7ffb04013590 con 0x7ffb0c0fde90 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.840+0000 7ffb0a7fc700 1 --2- 172.21.15.6:0/2298794983 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffaf40692a0 0x7ffaf406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:47.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.840+0000 7ffb0a7fc700 1 -- 172.21.15.6:0/2298794983 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffb04088000 con 0x7ffb0c0fde90 2024-03-22T06:11:47.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.840+0000 7ffb112c8700 1 --2- 172.21.15.6:0/2298794983 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffaf40692a0 0x7ffaf406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:47.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.841+0000 7ffb112c8700 1 --2- 172.21.15.6:0/2298794983 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffaf40692a0 0x7ffaf406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb00006fd0 tx=0x7ffb00006db0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:47.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:47.845+0000 7ffb0a7fc700 1 -- 172.21.15.6:0/2298794983 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ffb040581a0 con 0x7ffb0c0fde90 2024-03-22T06:11:47.891 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:48.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.050+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2298794983 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffb0c04bed0 con 0x7ffb0c0fde90 2024-03-22T06:11:48.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.051+0000 7ffb0a7fc700 1 -- 172.21.15.6:0/2298794983 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffb04013840 con 0x7ffb0c0fde90 2024-03-22T06:11:48.051 INFO:teuthology.orchestra.run.smithi006.stdout:38654705703 2024-03-22T06:11:48.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.053+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2298794983 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffaf40692a0 msgr2=0x7ffaf406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:48.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.053+0000 7ffb13d2d700 1 --2- 172.21.15.6:0/2298794983 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffaf40692a0 0x7ffaf406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb00006fd0 tx=0x7ffb00006db0 comp rx=0 tx=0).stop 2024-03-22T06:11:48.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.053+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2298794983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb0c0fde90 msgr2=0x7ffb0c113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:48.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.053+0000 7ffb13d2d700 1 --2- 172.21.15.6:0/2298794983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb0c0fde90 0x7ffb0c113b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400b3b0 tx=0x7ffb04009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:48.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.053+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2298794983 shutdown_connections 2024-03-22T06:11:48.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.053+0000 7ffb13d2d700 1 --2- 172.21.15.6:0/2298794983 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffaf40692a0 0x7ffaf406b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:48.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.053+0000 7ffb13d2d700 1 --2- 172.21.15.6:0/2298794983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb0c0fde90 0x7ffb0c113b80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:48.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.053+0000 7ffb13d2d700 1 --2- 172.21.15.6:0/2298794983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb0c0fe830 0x7ffb0c1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:48.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.053+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2298794983 >> 172.21.15.6:0/2298794983 conn(0x7ffb0c072bc0 msgr2=0x7ffb0c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:48.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.053+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2298794983 shutdown_connections 2024-03-22T06:11:48.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:48.053+0000 7ffb13d2d700 1 -- 172.21.15.6:0/2298794983 wait complete. 2024-03-22T06:11:48.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:48 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3512565101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-22T06:11:48.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:48 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2298794983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-22T06:11:48.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:48 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3512565101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-22T06:11:48.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:48 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2298794983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-22T06:11:49.283 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705701 got 38654705703 for osd.0 2024-03-22T06:11:49.283 DEBUG:teuthology.parallel:result is None 2024-03-22T06:11:49.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:49 smithi006 ceph-mon[29301]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:49.745 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542175 got 60129542178 for osd.1 2024-03-22T06:11:49.745 DEBUG:teuthology.parallel:result is None 2024-03-22T06:11:49.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:49 smithi117 ceph-mon[36514]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:50.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.655+0000 7f2e8d654700 1 -- 172.21.15.6:0/122049950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e880fc5b0 msgr2=0x7f2e880fca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:50.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.655+0000 7f2e8d654700 1 --2- 172.21.15.6:0/122049950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e880fc5b0 0x7f2e880fca20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c004940 tx=0x7f2e7c009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:50.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.655+0000 7f2e8d654700 1 -- 172.21.15.6:0/122049950 shutdown_connections 2024-03-22T06:11:50.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.655+0000 7f2e8d654700 1 --2- 172.21.15.6:0/122049950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e880fc5b0 0x7f2e880fca20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:50.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.655+0000 7f2e8d654700 1 --2- 172.21.15.6:0/122049950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e88101d70 0x7f2e88102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:50.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.655+0000 7f2e8d654700 1 -- 172.21.15.6:0/122049950 >> 172.21.15.6:0/122049950 conn(0x7f2e880714c0 msgr2=0x7f2e880718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:50.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.655+0000 7f2e8d654700 1 -- 172.21.15.6:0/122049950 shutdown_connections 2024-03-22T06:11:50.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.655+0000 7f2e8d654700 1 -- 172.21.15.6:0/122049950 wait complete. 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.656+0000 7f2e8d654700 1 Processor -- start 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.656+0000 7f2e8d654700 1 -- start start 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.656+0000 7f2e8d654700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e880fc5b0 0x7f2e8810ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.656+0000 7f2e8d654700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e88101d70 0x7f2e8810f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.656+0000 7f2e8d654700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8810f780 con 0x7f2e88101d70 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.656+0000 7f2e8d654700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8810f8c0 con 0x7f2e880fc5b0 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.656+0000 7f2e867fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e88101d70 0x7f2e8810f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e867fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e88101d70 0x7f2e8810f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45712/0 (socket says 172.21.15.6:45712) 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e867fc700 1 -- 172.21.15.6:0/33524550 learned_addr learned my addr 172.21.15.6:0/33524550 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e86ffd700 1 --2- 172.21.15.6:0/33524550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e880fc5b0 0x7f2e8810ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e867fc700 1 -- 172.21.15.6:0/33524550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e880fc5b0 msgr2=0x7f2e8810ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e867fc700 1 --2- 172.21.15.6:0/33524550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e880fc5b0 0x7f2e8810ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e867fc700 1 -- 172.21.15.6:0/33524550 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c005040 con 0x7f2e88101d70 2024-03-22T06:11:50.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e867fc700 1 --2- 172.21.15.6:0/33524550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e88101d70 0x7f2e8810f240 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009150 tx=0x7f2e7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:50.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e6f7fe700 1 -- 172.21.15.6:0/33524550 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e7c00e040 con 0x7f2e88101d70 2024-03-22T06:11:50.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e8d654700 1 -- 172.21.15.6:0/33524550 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e88117bb0 con 0x7f2e88101d70 2024-03-22T06:11:50.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e6f7fe700 1 -- 172.21.15.6:0/33524550 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e7c013420 con 0x7f2e88101d70 2024-03-22T06:11:50.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e6f7fe700 1 -- 172.21.15.6:0/33524550 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e7c01c610 con 0x7f2e88101d70 2024-03-22T06:11:50.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.657+0000 7f2e8d654700 1 -- 172.21.15.6:0/33524550 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e88118140 con 0x7f2e88101d70 2024-03-22T06:11:50.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.658+0000 7f2e8d654700 1 -- 172.21.15.6:0/33524550 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e880fd5c0 con 0x7f2e88101d70 2024-03-22T06:11:50.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.659+0000 7f2e6f7fe700 1 -- 172.21.15.6:0/33524550 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f2e7c01c880 con 0x7f2e88101d70 2024-03-22T06:11:50.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.659+0000 7f2e6f7fe700 1 --2- 172.21.15.6:0/33524550 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f2e700692b0 0x7f2e7006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:50.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.659+0000 7f2e6f7fe700 1 -- 172.21.15.6:0/33524550 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2e7c087f90 con 0x7f2e88101d70 2024-03-22T06:11:50.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.659+0000 7f2e86ffd700 1 --2- 172.21.15.6:0/33524550 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f2e700692b0 0x7f2e7006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:50.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.660+0000 7f2e86ffd700 1 --2- 172.21.15.6:0/33524550 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f2e700692b0 0x7f2e7006b760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e88064f70 tx=0x7f2e7800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:50.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.664+0000 7f2e6f7fe700 1 -- 172.21.15.6:0/33524550 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2e7c054710 con 0x7f2e88101d70 2024-03-22T06:11:50.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.879+0000 7f2e8d654700 1 -- 172.21.15.6:0/33524550 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2e8804bed0 con 0x7f2e88101d70 2024-03-22T06:11:50.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.880+0000 7f2e6f7fe700 1 -- 172.21.15.6:0/33524550 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e7c057d30 con 0x7f2e88101d70 2024-03-22T06:11:50.880 INFO:teuthology.orchestra.run.smithi006.stdout:115964117014 2024-03-22T06:11:50.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.881+0000 7f2e8d654700 1 -- 172.21.15.6:0/33524550 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f2e700692b0 msgr2=0x7f2e7006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:50.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.881+0000 7f2e8d654700 1 --2- 172.21.15.6:0/33524550 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f2e700692b0 0x7f2e7006b760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e88064f70 tx=0x7f2e7800b040 comp rx=0 tx=0).stop 2024-03-22T06:11:50.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.881+0000 7f2e8d654700 1 -- 172.21.15.6:0/33524550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e88101d70 msgr2=0x7f2e8810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:50.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.881+0000 7f2e8d654700 1 --2- 172.21.15.6:0/33524550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e88101d70 0x7f2e8810f240 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009150 tx=0x7f2e7c009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:50.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.882+0000 7f2e8d654700 1 -- 172.21.15.6:0/33524550 shutdown_connections 2024-03-22T06:11:50.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.882+0000 7f2e8d654700 1 --2- 172.21.15.6:0/33524550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e880fc5b0 0x7f2e8810ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:50.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.882+0000 7f2e8d654700 1 --2- 172.21.15.6:0/33524550 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f2e700692b0 0x7f2e7006b760 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:50.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.882+0000 7f2e8d654700 1 --2- 172.21.15.6:0/33524550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e88101d70 0x7f2e8810f240 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:50.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.882+0000 7f2e8d654700 1 -- 172.21.15.6:0/33524550 >> 172.21.15.6:0/33524550 conn(0x7f2e880714c0 msgr2=0x7f2e880f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:50.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.882+0000 7f2e8d654700 1 -- 172.21.15.6:0/33524550 shutdown_connections 2024-03-22T06:11:50.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:50.882+0000 7f2e8d654700 1 -- 172.21.15.6:0/33524550 wait complete. 2024-03-22T06:11:51.535 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:51 smithi006 ceph-mon[29301]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:51.535 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:51 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/33524550' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-22T06:11:51.615 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117012 got 115964117014 for osd.3 2024-03-22T06:11:51.615 DEBUG:teuthology.parallel:result is None 2024-03-22T06:11:51.616 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-22T06:11:51.616 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-22T06:11:51.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:51 smithi117 ceph-mon[36514]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:51.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:51 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/33524550' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-22T06:11:51.878 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:53.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.477+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2919514142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e0101d20 msgr2=0x7fa2e01020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:53.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.477+0000 7fa2e6c0d700 1 --2- 172.21.15.6:0/2919514142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e0101d20 0x7fa2e01020f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4004940 tx=0x7fa2d4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:53.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.477+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2919514142 shutdown_connections 2024-03-22T06:11:53.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.477+0000 7fa2e6c0d700 1 --2- 172.21.15.6:0/2919514142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e01026c0 0x7fa2e00fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:53.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.477+0000 7fa2e6c0d700 1 --2- 172.21.15.6:0/2919514142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e0101d20 0x7fa2e01020f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:53.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.477+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2919514142 >> 172.21.15.6:0/2919514142 conn(0x7fa2e00f7870 msgr2=0x7fa2e00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:53.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.477+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2919514142 shutdown_connections 2024-03-22T06:11:53.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.478+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2919514142 wait complete. 2024-03-22T06:11:53.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.478+0000 7fa2e6c0d700 1 Processor -- start 2024-03-22T06:11:53.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.478+0000 7fa2e6c0d700 1 -- start start 2024-03-22T06:11:53.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.478+0000 7fa2e6c0d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e0101d20 0x7fa2e010eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:53.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e6c0d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e01026c0 0x7fa2e010f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:53.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e6c0d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e010f540 con 0x7fa2e0101d20 2024-03-22T06:11:53.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e6c0d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e010f680 con 0x7fa2e01026c0 2024-03-22T06:11:53.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e49a9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e0101d20 0x7fa2e010eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:53.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e49a9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e0101d20 0x7fa2e010eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45724/0 (socket says 172.21.15.6:45724) 2024-03-22T06:11:53.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e49a9700 1 -- 172.21.15.6:0/2177210056 learned_addr learned my addr 172.21.15.6:0/2177210056 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:53.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2dffff700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e01026c0 0x7fa2e010f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:53.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e49a9700 1 -- 172.21.15.6:0/2177210056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e01026c0 msgr2=0x7fa2e010f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:53.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e49a9700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e01026c0 0x7fa2e010f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:53.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e49a9700 1 -- 172.21.15.6:0/2177210056 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d4005040 con 0x7fa2e0101d20 2024-03-22T06:11:53.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2dffff700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e01026c0 0x7fa2e010f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:11:53.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e49a9700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e0101d20 0x7fa2e010eac0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400b8b0 tx=0x7fa2d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:53.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2dd7fa700 1 -- 172.21.15.6:0/2177210056 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d400e040 con 0x7fa2e0101d20 2024-03-22T06:11:53.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2dd7fa700 1 -- 172.21.15.6:0/2177210056 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2d4004290 con 0x7fa2e0101d20 2024-03-22T06:11:53.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2177210056 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e010f900 con 0x7fa2e0101d20 2024-03-22T06:11:53.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2dd7fa700 1 -- 172.21.15.6:0/2177210056 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d401c3f0 con 0x7fa2e0101d20 2024-03-22T06:11:53.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.479+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2177210056 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e0117e80 con 0x7fa2e0101d20 2024-03-22T06:11:53.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.480+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2177210056 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e00fd470 con 0x7fa2e0101d20 2024-03-22T06:11:53.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.481+0000 7fa2dd7fa700 1 -- 172.21.15.6:0/2177210056 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fa2d40137b0 con 0x7fa2e0101d20 2024-03-22T06:11:53.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.481+0000 7fa2dd7fa700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa2d00692a0 0x7fa2d006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:53.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.481+0000 7fa2dd7fa700 1 -- 172.21.15.6:0/2177210056 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa2d4088180 con 0x7fa2e0101d20 2024-03-22T06:11:53.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.481+0000 7fa2dffff700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa2d00692a0 0x7fa2d006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:53.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.482+0000 7fa2dffff700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa2d00692a0 0x7fa2d006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009b80 tx=0x7fa2cc00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:53.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.486+0000 7fa2dd7fa700 1 -- 172.21.15.6:0/2177210056 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa2d40547f0 con 0x7fa2e0101d20 2024-03-22T06:11:53.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:53 smithi006 ceph-mon[29301]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:53.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.675+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2177210056 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2e00668a0 con 0x7fa2d00692a0 2024-03-22T06:11:53.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.676+0000 7fa2dd7fa700 1 -- 172.21.15.6:0/2177210056 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28011 (secure 0 0 0) 0x7fa2e00668a0 con 0x7fa2d00692a0 2024-03-22T06:11:53.677 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:11:53.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.678+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2177210056 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa2d00692a0 msgr2=0x7fa2d006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:53.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.678+0000 7fa2e6c0d700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa2d00692a0 0x7fa2d006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009b80 tx=0x7fa2cc00b040 comp rx=0 tx=0).stop 2024-03-22T06:11:53.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.678+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2177210056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e0101d20 msgr2=0x7fa2e010eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:53.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.678+0000 7fa2e6c0d700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e0101d20 0x7fa2e010eac0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400b8b0 tx=0x7fa2d4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:53.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.679+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2177210056 shutdown_connections 2024-03-22T06:11:53.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.679+0000 7fa2e6c0d700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa2d00692a0 0x7fa2d006b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:53.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.679+0000 7fa2e6c0d700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e0101d20 0x7fa2e010eac0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:53.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.679+0000 7fa2e6c0d700 1 --2- 172.21.15.6:0/2177210056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e01026c0 0x7fa2e010f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:53.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.679+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2177210056 >> 172.21.15.6:0/2177210056 conn(0x7fa2e00f7870 msgr2=0x7fa2e00f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:53.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.679+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2177210056 shutdown_connections 2024-03-22T06:11:53.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:53.679+0000 7fa2e6c0d700 1 -- 172.21.15.6:0/2177210056 wait complete. 2024-03-22T06:11:53.680 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-03-22T06:11:53.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:53 smithi117 ceph-mon[36514]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:54.543 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-03-22T06:11:52.159283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787652,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683260,"statfs":{"total":575970213888,"available":574139658240,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002176"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":189,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T06:10:59.078251+0000","last_change":"2024-03-22T06:10:33.205471+0000","last_active":"2024-03-22T06:10:59.078251+0000","last_peered":"2024-03-22T06:10:59.078251+0000","last_clean":"2024-03-22T06:10:59.078251+0000","last_became_active":"2024-03-22T06:10:33.205033+0000","last_became_peered":"2024-03-22T06:10:33.205033+0000","last_unstale":"2024-03-22T06:10:59.078251+0000","last_undegraded":"2024-03-22T06:10:59.078251+0000","last_fullsized":"2024-03-22T06:10:59.078251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T06:09:46.499521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T06:09:46.499521+0000","last_clean_scrub_stamp":"2024-03-22T06:09:46.499521+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:14:17.229046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]}]},{"osd":4,"up_from":35,"seq":150323855377,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 06:11:37 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.55100000000000005}]},{"osd":1,"last update":"Fri Mar 22 06:11:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Mar 22 06:11:37 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.627},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Mar 22 06:11:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117014,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 06:11:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40799999999999997}]},{"osd":1,"last update":"Fri Mar 22 06:11:08 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.35399999999999998}]},{"osd":2,"last update":"Fri Mar 22 06:11:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.318},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.33400000000000002}]},{"osd":4,"last update":"Fri Mar 22 06:11:36 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.434},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.28899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 06:11:47 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33300000000000002,"15min":0.33700000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46500000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.22},{"interface":"front","average":{"1min":0.318,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.28000000000000003}]},{"osd":1,"last update":"Fri Mar 22 06:11:47 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48199999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.251},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.26600000000000001}]},{"osd":3,"last update":"Fri Mar 22 06:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Mar 22 06:11:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 06:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34100000000000003},"min":{"1min":0.23100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.36099999999999999,"15min":0.35099999999999998},"min":{"1min":0.27000000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.24099999999999999}]},{"osd":2,"last update":"Fri Mar 22 06:11:46 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.53900000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.42199999999999999,"15min":0.42899999999999999},"min":{"1min":0.27200000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56399999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.31}]},{"osd":3,"last update":"Fri Mar 22 06:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.377}]},{"osd":4,"last update":"Fri Mar 22 06:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 06:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.317},"min":{"1min":0.246,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.33100000000000002,"15min":0.32700000000000001},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.32000000000000001}]},{"osd":2,"last update":"Fri Mar 22 06:11:49 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.27900000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Mar 22 06:11:08 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Mar 22 06:11:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T06:11:54.544 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-22T06:11:54.810 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:55.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:55 smithi006 ceph-mon[29301]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T06:11:55.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:55 smithi006 ceph-mon[29301]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:55.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:55 smithi117 ceph-mon[36514]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T06:11:55.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:55 smithi117 ceph-mon[36514]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:56.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.505+0000 7f74e6c1a700 1 -- 172.21.15.6:0/1407187267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74e0101d20 msgr2=0x7f74e01020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:56.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.505+0000 7f74e6c1a700 1 --2- 172.21.15.6:0/1407187267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74e0101d20 0x7f74e01020f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f74d4004940 tx=0x7f74d4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.505+0000 7f74e6c1a700 1 -- 172.21.15.6:0/1407187267 shutdown_connections 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.505+0000 7f74e6c1a700 1 --2- 172.21.15.6:0/1407187267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e01026c0 0x7f74e00fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.505+0000 7f74e6c1a700 1 --2- 172.21.15.6:0/1407187267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74e0101d20 0x7f74e01020f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.505+0000 7f74e6c1a700 1 -- 172.21.15.6:0/1407187267 >> 172.21.15.6:0/1407187267 conn(0x7f74e00f7870 msgr2=0x7f74e00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.505+0000 7f74e6c1a700 1 -- 172.21.15.6:0/1407187267 shutdown_connections 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.505+0000 7f74e6c1a700 1 -- 172.21.15.6:0/1407187267 wait complete. 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.505+0000 7f74e6c1a700 1 Processor -- start 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.505+0000 7f74e6c1a700 1 -- start start 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74e6c1a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74e0101d20 0x7f74e010eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74e6c1a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e01026c0 0x7f74e010f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74e6c1a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e010f600 con 0x7f74e0101d20 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74e6c1a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e010f740 con 0x7f74e01026c0 2024-03-22T06:11:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74e49b6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74e0101d20 0x7f74e010eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74e49b6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74e0101d20 0x7f74e010eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37122/0 (socket says 172.21.15.6:37122) 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74e49b6700 1 -- 172.21.15.6:0/3633131609 learned_addr learned my addr 172.21.15.6:0/3633131609 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74e49b6700 1 -- 172.21.15.6:0/3633131609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e01026c0 msgr2=0x7f74e010f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74dffff700 1 --2- 172.21.15.6:0/3633131609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e01026c0 0x7f74e010f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74e49b6700 1 --2- 172.21.15.6:0/3633131609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e01026c0 0x7f74e010f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74e49b6700 1 -- 172.21.15.6:0/3633131609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d4005040 con 0x7f74e0101d20 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74e49b6700 1 --2- 172.21.15.6:0/3633131609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74e0101d20 0x7f74e010eb80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f74d400b8b0 tx=0x7f74d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.506+0000 7f74dd7fa700 1 -- 172.21.15.6:0/3633131609 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d400e040 con 0x7f74e0101d20 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.507+0000 7f74dd7fa700 1 -- 172.21.15.6:0/3633131609 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74d4004290 con 0x7f74e0101d20 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.507+0000 7f74e6c1a700 1 -- 172.21.15.6:0/3633131609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e010f9c0 con 0x7f74e0101d20 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.507+0000 7f74dd7fa700 1 -- 172.21.15.6:0/3633131609 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d401c3f0 con 0x7f74e0101d20 2024-03-22T06:11:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.507+0000 7f74e6c1a700 1 -- 172.21.15.6:0/3633131609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e0117ef0 con 0x7f74e0101d20 2024-03-22T06:11:56.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.508+0000 7f74e6c1a700 1 -- 172.21.15.6:0/3633131609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e00fd470 con 0x7f74e0101d20 2024-03-22T06:11:56.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.508+0000 7f74dd7fa700 1 -- 172.21.15.6:0/3633131609 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f74d40137b0 con 0x7f74e0101d20 2024-03-22T06:11:56.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.508+0000 7f74dd7fa700 1 --2- 172.21.15.6:0/3633131609 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f74d00692a0 0x7f74d006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:56.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.509+0000 7f74dffff700 1 --2- 172.21.15.6:0/3633131609 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f74d00692a0 0x7f74d006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:56.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.509+0000 7f74dd7fa700 1 -- 172.21.15.6:0/3633131609 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f74d4088180 con 0x7f74e0101d20 2024-03-22T06:11:56.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.509+0000 7f74dffff700 1 --2- 172.21.15.6:0/3633131609 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f74d00692a0 0x7f74d006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f74e0066a60 tx=0x7f74cc009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:56.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.513+0000 7f74dd7fa700 1 -- 172.21.15.6:0/3633131609 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f74d40547f0 con 0x7f74e0101d20 2024-03-22T06:11:56.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.703+0000 7f74e6c1a700 1 -- 172.21.15.6:0/3633131609 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74e0066810 con 0x7f74d00692a0 2024-03-22T06:11:56.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.704+0000 7f74dd7fa700 1 -- 172.21.15.6:0/3633131609 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28012 (secure 0 0 0) 0x7f74e0066810 con 0x7f74d00692a0 2024-03-22T06:11:56.705 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:11:56.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.706+0000 7f74e6c1a700 1 -- 172.21.15.6:0/3633131609 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f74d00692a0 msgr2=0x7f74d006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:56.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.706+0000 7f74e6c1a700 1 --2- 172.21.15.6:0/3633131609 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f74d00692a0 0x7f74d006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f74e0066a60 tx=0x7f74cc009040 comp rx=0 tx=0).stop 2024-03-22T06:11:56.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.706+0000 7f74e6c1a700 1 -- 172.21.15.6:0/3633131609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74e0101d20 msgr2=0x7f74e010eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:56.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.706+0000 7f74e6c1a700 1 --2- 172.21.15.6:0/3633131609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74e0101d20 0x7f74e010eb80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f74d400b8b0 tx=0x7f74d4009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:56.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.707+0000 7f74e6c1a700 1 -- 172.21.15.6:0/3633131609 shutdown_connections 2024-03-22T06:11:56.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.707+0000 7f74e6c1a700 1 --2- 172.21.15.6:0/3633131609 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f74d00692a0 0x7f74d006b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:56.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.707+0000 7f74e6c1a700 1 --2- 172.21.15.6:0/3633131609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74e0101d20 0x7f74e010eb80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:56.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.707+0000 7f74e6c1a700 1 --2- 172.21.15.6:0/3633131609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e01026c0 0x7f74e010f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:56.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.707+0000 7f74e6c1a700 1 -- 172.21.15.6:0/3633131609 >> 172.21.15.6:0/3633131609 conn(0x7f74e00f7870 msgr2=0x7f74e00f8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:56.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.707+0000 7f74e6c1a700 1 -- 172.21.15.6:0/3633131609 shutdown_connections 2024-03-22T06:11:56.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:56.707+0000 7f74e6c1a700 1 -- 172.21.15.6:0/3633131609 wait complete. 2024-03-22T06:11:56.708 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-03-22T06:11:57.483 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-03-22T06:11:56.160075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787652,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683260,"statfs":{"total":575970213888,"available":574139658240,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002231"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":189,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T06:10:59.078251+0000","last_change":"2024-03-22T06:10:33.205471+0000","last_active":"2024-03-22T06:10:59.078251+0000","last_peered":"2024-03-22T06:10:59.078251+0000","last_clean":"2024-03-22T06:10:59.078251+0000","last_became_active":"2024-03-22T06:10:33.205033+0000","last_became_peered":"2024-03-22T06:10:33.205033+0000","last_unstale":"2024-03-22T06:10:59.078251+0000","last_undegraded":"2024-03-22T06:10:59.078251+0000","last_fullsized":"2024-03-22T06:10:59.078251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T06:09:46.499521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T06:09:46.499521+0000","last_clean_scrub_stamp":"2024-03-22T06:09:46.499521+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:14:17.229046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 06:11:37 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Mar 22 06:11:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Mar 22 06:11:37 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.54400000000000004}]},{"osd":3,"last update":"Fri Mar 22 06:11:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]}]},{"osd":3,"up_from":27,"seq":115964117015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 06:11:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Mar 22 06:11:08 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Mar 22 06:11:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.503},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Mar 22 06:11:36 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 06:11:47 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33300000000000002,"15min":0.33700000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46500000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.31},{"interface":"front","average":{"1min":0.318,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.318}]},{"osd":1,"last update":"Fri Mar 22 06:11:47 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48199999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Mar 22 06:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Mar 22 06:11:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 06:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34100000000000003},"min":{"1min":0.23100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.187},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.36099999999999999,"15min":0.35099999999999998},"min":{"1min":0.27000000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.22800000000000001}]},{"osd":2,"last update":"Fri Mar 22 06:11:46 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.53900000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.378,"5min":0.42199999999999999,"15min":0.42899999999999999},"min":{"1min":0.27200000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56399999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.23899999999999999}]},{"osd":3,"last update":"Fri Mar 22 06:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45200000000000001}]},{"osd":4,"last update":"Fri Mar 22 06:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542180,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 06:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.317},"min":{"1min":0.246,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.33100000000000002,"15min":0.32700000000000001},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.27700000000000002}]},{"osd":2,"last update":"Fri Mar 22 06:11:49 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.27900000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.25800000000000001}]},{"osd":3,"last update":"Fri Mar 22 06:11:08 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.441},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Mar 22 06:11:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T06:11:57.484 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-22T06:11:57.484 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-22T06:11:57.484 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-22T06:11:57.485 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-22T06:11:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:11:57.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:57 smithi117 ceph-mon[36514]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:57.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:57 smithi006 ceph-mon[29301]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:58.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:58 smithi006 ceph-mon[29301]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T06:11:58.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:58 smithi117 ceph-mon[36514]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.358+0000 7fde8c21e700 1 -- 172.21.15.6:0/2530484381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde840fb4d0 msgr2=0x7fde840fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.358+0000 7fde8c21e700 1 --2- 172.21.15.6:0/2530484381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde840fb4d0 0x7fde840fb940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fde74004940 tx=0x7fde74009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.358+0000 7fde8c21e700 1 -- 172.21.15.6:0/2530484381 shutdown_connections 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.358+0000 7fde8c21e700 1 --2- 172.21.15.6:0/2530484381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde840fb4d0 0x7fde840fb940 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.358+0000 7fde8c21e700 1 --2- 172.21.15.6:0/2530484381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde84102f50 0x7fde84103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.358+0000 7fde8c21e700 1 -- 172.21.15.6:0/2530484381 >> 172.21.15.6:0/2530484381 conn(0x7fde840714c0 msgr2=0x7fde840718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.359+0000 7fde8c21e700 1 -- 172.21.15.6:0/2530484381 shutdown_connections 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.359+0000 7fde8c21e700 1 -- 172.21.15.6:0/2530484381 wait complete. 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.359+0000 7fde8c21e700 1 Processor -- start 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.359+0000 7fde8c21e700 1 -- start start 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.359+0000 7fde8c21e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde840fb4d0 0x7fde840fc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.359+0000 7fde8c21e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde84102f50 0x7fde840fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.359+0000 7fde8c21e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde840fe600 con 0x7fde840fb4d0 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.359+0000 7fde8c21e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde840fd130 con 0x7fde84102f50 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.359+0000 7fde89fba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde840fb4d0 0x7fde840fc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.359+0000 7fde89fba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde840fb4d0 0x7fde840fc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37146/0 (socket says 172.21.15.6:37146) 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.359+0000 7fde89fba700 1 -- 172.21.15.6:0/3229372116 learned_addr learned my addr 172.21.15.6:0/3229372116 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.360+0000 7fde89fba700 1 -- 172.21.15.6:0/3229372116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde84102f50 msgr2=0x7fde840fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.360+0000 7fde897b9700 1 --2- 172.21.15.6:0/3229372116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde84102f50 0x7fde840fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.360+0000 7fde89fba700 1 --2- 172.21.15.6:0/3229372116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde84102f50 0x7fde840fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.360+0000 7fde89fba700 1 -- 172.21.15.6:0/3229372116 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde74005040 con 0x7fde840fb4d0 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.360+0000 7fde89fba700 1 --2- 172.21.15.6:0/3229372116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde840fb4d0 0x7fde840fc6b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fde80002a80 tx=0x7fde800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.360+0000 7fde7a7fc700 1 -- 172.21.15.6:0/3229372116 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde80007cc0 con 0x7fde840fb4d0 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.360+0000 7fde7a7fc700 1 -- 172.21.15.6:0/3229372116 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde8000f040 con 0x7fde840fb4d0 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.360+0000 7fde8c21e700 1 -- 172.21.15.6:0/3229372116 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde840fd410 con 0x7fde840fb4d0 2024-03-22T06:11:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.360+0000 7fde7a7fc700 1 -- 172.21.15.6:0/3229372116 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde80009630 con 0x7fde840fb4d0 2024-03-22T06:11:59.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.360+0000 7fde8c21e700 1 -- 172.21.15.6:0/3229372116 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8411a110 con 0x7fde840fb4d0 2024-03-22T06:11:59.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.361+0000 7fde8c21e700 1 -- 172.21.15.6:0/3229372116 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde84105330 con 0x7fde840fb4d0 2024-03-22T06:11:59.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.362+0000 7fde7a7fc700 1 -- 172.21.15.6:0/3229372116 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fde8000d690 con 0x7fde840fb4d0 2024-03-22T06:11:59.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.362+0000 7fde7a7fc700 1 --2- 172.21.15.6:0/3229372116 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fde700692a0 0x7fde7006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:11:59.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.362+0000 7fde7a7fc700 1 -- 172.21.15.6:0/3229372116 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fde80086710 con 0x7fde840fb4d0 2024-03-22T06:11:59.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.362+0000 7fde897b9700 1 --2- 172.21.15.6:0/3229372116 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fde700692a0 0x7fde7006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:11:59.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.362+0000 7fde897b9700 1 --2- 172.21.15.6:0/3229372116 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fde700692a0 0x7fde7006b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fde74004f90 tx=0x7fde74009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:11:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.367+0000 7fde7a7fc700 1 -- 172.21.15.6:0/3229372116 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fde8004d190 con 0x7fde840fb4d0 2024-03-22T06:11:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:11:59 smithi006 ceph-mon[29301]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:11:59.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.607+0000 7fde8c21e700 1 -- 172.21.15.6:0/3229372116 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fde840fe7f0 con 0x7fde840fb4d0 2024-03-22T06:11:59.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.607+0000 7fde7a7fc700 1 -- 172.21.15.6:0/3229372116 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fde80052d00 con 0x7fde840fb4d0 2024-03-22T06:11:59.608 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:11:59.608 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-22T06:11:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.609+0000 7fde8c21e700 1 -- 172.21.15.6:0/3229372116 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fde700692a0 msgr2=0x7fde7006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.609+0000 7fde8c21e700 1 --2- 172.21.15.6:0/3229372116 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fde700692a0 0x7fde7006b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fde74004f90 tx=0x7fde74009e30 comp rx=0 tx=0).stop 2024-03-22T06:11:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.609+0000 7fde8c21e700 1 -- 172.21.15.6:0/3229372116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde840fb4d0 msgr2=0x7fde840fc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:11:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.609+0000 7fde8c21e700 1 --2- 172.21.15.6:0/3229372116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde840fb4d0 0x7fde840fc6b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fde80002a80 tx=0x7fde800046b0 comp rx=0 tx=0).stop 2024-03-22T06:11:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.609+0000 7fde8c21e700 1 -- 172.21.15.6:0/3229372116 shutdown_connections 2024-03-22T06:11:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.609+0000 7fde8c21e700 1 --2- 172.21.15.6:0/3229372116 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fde700692a0 0x7fde7006b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.609+0000 7fde8c21e700 1 --2- 172.21.15.6:0/3229372116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde840fb4d0 0x7fde840fc6b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:59.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.609+0000 7fde8c21e700 1 --2- 172.21.15.6:0/3229372116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde84102f50 0x7fde840fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:11:59.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.609+0000 7fde8c21e700 1 -- 172.21.15.6:0/3229372116 >> 172.21.15.6:0/3229372116 conn(0x7fde840714c0 msgr2=0x7fde840fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:11:59.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.610+0000 7fde8c21e700 1 -- 172.21.15.6:0/3229372116 shutdown_connections 2024-03-22T06:11:59.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:11:59.610+0000 7fde8c21e700 1 -- 172.21.15.6:0/3229372116 wait complete. 2024-03-22T06:11:59.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:11:59 smithi117 ceph-mon[36514]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:00.225 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-22T06:12:00.226 INFO:tasks.cephadm:Setup complete, yielding 2024-03-22T06:12:00.226 INFO:teuthology.run_tasks:Running task print... 2024-03-22T06:12:00.236 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-03-22T06:12:00.236 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T06:12:00.291 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:12:00.291 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-22T06:12:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:00 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3229372116' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-22T06:12:00.561 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:00.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:00 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3229372116' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-22T06:12:01.611 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:01 smithi006 ceph-mon[29301]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:01.611 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:12:01.611 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:01.611 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:12:01.611 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:01 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:01.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:01 smithi117 ceph-mon[36514]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:01.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:12:01.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:01.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:12:01.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:01 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:02.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.093+0000 7ff31faa8700 1 -- 172.21.15.6:0/3668254758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3180fee50 msgr2=0x7ff3180faf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:02.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.093+0000 7ff31faa8700 1 --2- 172.21.15.6:0/3668254758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3180fee50 0x7ff3180faf90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff30c004940 tx=0x7ff30c009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:02.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.093+0000 7ff31faa8700 1 -- 172.21.15.6:0/3668254758 shutdown_connections 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.093+0000 7ff31faa8700 1 --2- 172.21.15.6:0/3668254758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3180fee50 0x7ff3180faf90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.093+0000 7ff31faa8700 1 --2- 172.21.15.6:0/3668254758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3180fe500 0x7ff3180fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.093+0000 7ff31faa8700 1 -- 172.21.15.6:0/3668254758 >> 172.21.15.6:0/3668254758 conn(0x7ff3180f6be0 msgr2=0x7ff3180f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.093+0000 7ff31faa8700 1 -- 172.21.15.6:0/3668254758 shutdown_connections 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.093+0000 7ff31faa8700 1 -- 172.21.15.6:0/3668254758 wait complete. 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31faa8700 1 Processor -- start 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31faa8700 1 -- start start 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31faa8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3180fe500 0x7ff3180fdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31faa8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3180fee50 0x7ff3180fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31faa8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3180fe110 con 0x7ff3180fee50 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31faa8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3180fc700 con 0x7ff3180fe500 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31d043700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3180fee50 0x7ff3180fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31d043700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3180fee50 0x7ff3180fc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37164/0 (socket says 172.21.15.6:37164) 2024-03-22T06:12:02.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31d043700 1 -- 172.21.15.6:0/525682045 learned_addr learned my addr 172.21.15.6:0/525682045 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:02.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31d844700 1 --2- 172.21.15.6:0/525682045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3180fe500 0x7ff3180fdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:02.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31d043700 1 -- 172.21.15.6:0/525682045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3180fe500 msgr2=0x7ff3180fdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:02.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.094+0000 7ff31d043700 1 --2- 172.21.15.6:0/525682045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3180fe500 0x7ff3180fdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:02.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.095+0000 7ff31d043700 1 -- 172.21.15.6:0/525682045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c005040 con 0x7ff3180fee50 2024-03-22T06:12:02.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.095+0000 7ff31d043700 1 --2- 172.21.15.6:0/525682045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3180fee50 0x7ff3180fc1c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00b3b0 tx=0x7ff30c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:02.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.095+0000 7ff3167fc700 1 -- 172.21.15.6:0/525682045 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff30c00e040 con 0x7ff3180fee50 2024-03-22T06:12:02.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.095+0000 7ff31faa8700 1 -- 172.21.15.6:0/525682045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3180fc950 con 0x7ff3180fee50 2024-03-22T06:12:02.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.095+0000 7ff3167fc700 1 -- 172.21.15.6:0/525682045 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff30c00bbf0 con 0x7ff3180fee50 2024-03-22T06:12:02.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.095+0000 7ff3167fc700 1 -- 172.21.15.6:0/525682045 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff30c01c630 con 0x7ff3180fee50 2024-03-22T06:12:02.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.095+0000 7ff31faa8700 1 -- 172.21.15.6:0/525682045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3180fcf10 con 0x7ff3180fee50 2024-03-22T06:12:02.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.096+0000 7ff31faa8700 1 -- 172.21.15.6:0/525682045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31804c710 con 0x7ff3180fee50 2024-03-22T06:12:02.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.096+0000 7ff3167fc700 1 -- 172.21.15.6:0/525682045 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7ff30c00bd60 con 0x7ff3180fee50 2024-03-22T06:12:02.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.097+0000 7ff3167fc700 1 --2- 172.21.15.6:0/525682045 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff3000692a0 0x7ff30006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:02.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.097+0000 7ff3167fc700 1 -- 172.21.15.6:0/525682045 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff30c087b00 con 0x7ff3180fee50 2024-03-22T06:12:02.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.097+0000 7ff31d844700 1 --2- 172.21.15.6:0/525682045 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff3000692a0 0x7ff30006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:02.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.097+0000 7ff31d844700 1 --2- 172.21.15.6:0/525682045 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff3000692a0 0x7ff30006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff308004550 tx=0x7ff308007ed0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:02.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.101+0000 7ff3167fc700 1 -- 172.21.15.6:0/525682045 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff30c057c00 con 0x7ff3180fee50 2024-03-22T06:12:02.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.295+0000 7ff31faa8700 1 -- 172.21.15.6:0/525682045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7ff318105910 con 0x7ff3180fee50 2024-03-22T06:12:02.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.299+0000 7ff3167fc700 1 -- 172.21.15.6:0/525682045 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7ff30c057790 con 0x7ff3180fee50 2024-03-22T06:12:02.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.301+0000 7ff31faa8700 1 -- 172.21.15.6:0/525682045 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff3000692a0 msgr2=0x7ff30006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:02.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.302+0000 7ff31faa8700 1 --2- 172.21.15.6:0/525682045 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff3000692a0 0x7ff30006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff308004550 tx=0x7ff308007ed0 comp rx=0 tx=0).stop 2024-03-22T06:12:02.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.302+0000 7ff31faa8700 1 -- 172.21.15.6:0/525682045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3180fee50 msgr2=0x7ff3180fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:02.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.302+0000 7ff31faa8700 1 --2- 172.21.15.6:0/525682045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3180fee50 0x7ff3180fc1c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00b3b0 tx=0x7ff30c009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:02.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.302+0000 7ff31faa8700 1 -- 172.21.15.6:0/525682045 shutdown_connections 2024-03-22T06:12:02.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.302+0000 7ff31faa8700 1 --2- 172.21.15.6:0/525682045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3180fe500 0x7ff3180fdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:02.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.302+0000 7ff31faa8700 1 --2- 172.21.15.6:0/525682045 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff3000692a0 0x7ff30006b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:02.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.302+0000 7ff31faa8700 1 --2- 172.21.15.6:0/525682045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3180fee50 0x7ff3180fc1c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:02.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.302+0000 7ff31faa8700 1 -- 172.21.15.6:0/525682045 >> 172.21.15.6:0/525682045 conn(0x7ff3180f6be0 msgr2=0x7ff3180f82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:02.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.302+0000 7ff31faa8700 1 -- 172.21.15.6:0/525682045 shutdown_connections 2024-03-22T06:12:02.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:02.302+0000 7ff31faa8700 1 -- 172.21.15.6:0/525682045 wait complete. 2024-03-22T06:12:02.593 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:12:02.593 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:02 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/525682045' entity='client.admin' 2024-03-22T06:12:02.593 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:12:02.593 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:02.593 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:12:02.593 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:02.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:12:02.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:02 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/525682045' entity='client.admin' 2024-03-22T06:12:02.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:12:02.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:02.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:12:02.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:02.968 INFO:teuthology.run_tasks:Running task print... 2024-03-22T06:12:02.977 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-22T06:12:02.977 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T06:12:03.012 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:12:03.012 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-22T06:12:03.276 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:03.622 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:03 smithi006 ceph-mon[29301]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:03.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:03 smithi117 ceph-mon[36514]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:04.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.951+0000 7fc4feb39700 1 -- 172.21.15.6:0/1851380926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f80fce70 msgr2=0x7fc4f80fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:04.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.951+0000 7fc4feb39700 1 --2- 172.21.15.6:0/1851380926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f80fce70 0x7fc4f80fd280 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8004940 tx=0x7fc4e8009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:04.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.951+0000 7fc4feb39700 1 -- 172.21.15.6:0/1851380926 shutdown_connections 2024-03-22T06:12:04.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.951+0000 7fc4feb39700 1 --2- 172.21.15.6:0/1851380926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4f80fdfe0 0x7fc4f80fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:04.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.951+0000 7fc4feb39700 1 --2- 172.21.15.6:0/1851380926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f80fce70 0x7fc4f80fd280 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:04.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.951+0000 7fc4feb39700 1 -- 172.21.15.6:0/1851380926 >> 172.21.15.6:0/1851380926 conn(0x7fc4f80f8550 msgr2=0x7fc4f80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:04.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.952+0000 7fc4feb39700 1 -- 172.21.15.6:0/1851380926 shutdown_connections 2024-03-22T06:12:04.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.952+0000 7fc4feb39700 1 -- 172.21.15.6:0/1851380926 wait complete. 2024-03-22T06:12:04.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.952+0000 7fc4feb39700 1 Processor -- start 2024-03-22T06:12:04.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.952+0000 7fc4feb39700 1 -- start start 2024-03-22T06:12:04.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.952+0000 7fc4feb39700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4f80fce70 0x7fc4f810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:04.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.952+0000 7fc4feb39700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f80fdfe0 0x7fc4f810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:04.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4feb39700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f810e460 con 0x7fc4f80fdfe0 2024-03-22T06:12:04.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4feb39700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f810e5d0 con 0x7fc4f80fce70 2024-03-22T06:12:04.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4f7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f80fdfe0 0x7fc4f810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:04.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4f7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f80fdfe0 0x7fc4f810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43956/0 (socket says 172.21.15.6:43956) 2024-03-22T06:12:04.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4f7fff700 1 -- 172.21.15.6:0/3745859995 learned_addr learned my addr 172.21.15.6:0/3745859995 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:04.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4fc8d5700 1 --2- 172.21.15.6:0/3745859995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4f80fce70 0x7fc4f810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:04.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4f7fff700 1 -- 172.21.15.6:0/3745859995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4f80fce70 msgr2=0x7fc4f810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:04.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4f7fff700 1 --2- 172.21.15.6:0/3745859995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4f80fce70 0x7fc4f810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:04.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4f7fff700 1 -- 172.21.15.6:0/3745859995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e8005040 con 0x7fc4f80fdfe0 2024-03-22T06:12:04.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4f7fff700 1 --2- 172.21.15.6:0/3745859995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f80fdfe0 0x7fc4f810de60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c830 tx=0x7fc4e0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:04.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4f57fa700 1 -- 172.21.15.6:0/3745859995 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4e00106f0 con 0x7fc4f80fdfe0 2024-03-22T06:12:04.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4f57fa700 1 -- 172.21.15.6:0/3745859995 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4e0010d20 con 0x7fc4f80fdfe0 2024-03-22T06:12:04.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4feb39700 1 -- 172.21.15.6:0/3745859995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f8113030 con 0x7fc4f80fdfe0 2024-03-22T06:12:04.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4f57fa700 1 -- 172.21.15.6:0/3745859995 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4e00113a0 con 0x7fc4f80fdfe0 2024-03-22T06:12:04.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.953+0000 7fc4feb39700 1 -- 172.21.15.6:0/3745859995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f81135c0 con 0x7fc4f80fdfe0 2024-03-22T06:12:04.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.954+0000 7fc4feb39700 1 -- 172.21.15.6:0/3745859995 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f804c710 con 0x7fc4f80fdfe0 2024-03-22T06:12:04.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.955+0000 7fc4f57fa700 1 -- 172.21.15.6:0/3745859995 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fc4e0012400 con 0x7fc4f80fdfe0 2024-03-22T06:12:04.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.955+0000 7fc4f57fa700 1 --2- 172.21.15.6:0/3745859995 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc4e40692a0 0x7fc4e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:04.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.955+0000 7fc4f57fa700 1 -- 172.21.15.6:0/3745859995 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc4e0086d20 con 0x7fc4f80fdfe0 2024-03-22T06:12:04.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.955+0000 7fc4fc8d5700 1 --2- 172.21.15.6:0/3745859995 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc4e40692a0 0x7fc4e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:04.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.955+0000 7fc4fc8d5700 1 --2- 172.21.15.6:0/3745859995 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc4e40692a0 0x7fc4e406b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800b3b0 tx=0x7fc4e8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:04.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:04.959+0000 7fc4f57fa700 1 -- 172.21.15.6:0/3745859995 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc4e0056ec0 con 0x7fc4f80fdfe0 2024-03-22T06:12:05.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.167+0000 7fc4feb39700 1 -- 172.21.15.6:0/3745859995 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4f81023f0 con 0x7fc4e40692a0 2024-03-22T06:12:05.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.168+0000 7fc4f57fa700 1 -- 172.21.15.6:0/3745859995 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc4f81023f0 con 0x7fc4e40692a0 2024-03-22T06:12:05.168 INFO:teuthology.orchestra.run.smithi006.stdout:Backend: cephadm 2024-03-22T06:12:05.168 INFO:teuthology.orchestra.run.smithi006.stdout:Available: Yes 2024-03-22T06:12:05.168 INFO:teuthology.orchestra.run.smithi006.stdout:Paused: No 2024-03-22T06:12:05.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.170+0000 7fc4feb39700 1 -- 172.21.15.6:0/3745859995 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc4e40692a0 msgr2=0x7fc4e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:05.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.170+0000 7fc4feb39700 1 --2- 172.21.15.6:0/3745859995 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc4e40692a0 0x7fc4e406b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800b3b0 tx=0x7fc4e8009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:05.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.170+0000 7fc4feb39700 1 -- 172.21.15.6:0/3745859995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f80fdfe0 msgr2=0x7fc4f810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:05.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.170+0000 7fc4feb39700 1 --2- 172.21.15.6:0/3745859995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f80fdfe0 0x7fc4f810de60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c830 tx=0x7fc4e0004470 comp rx=0 tx=0).stop 2024-03-22T06:12:05.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.170+0000 7fc4feb39700 1 -- 172.21.15.6:0/3745859995 shutdown_connections 2024-03-22T06:12:05.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.170+0000 7fc4feb39700 1 --2- 172.21.15.6:0/3745859995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4f80fce70 0x7fc4f810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:05.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.170+0000 7fc4feb39700 1 --2- 172.21.15.6:0/3745859995 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc4e40692a0 0x7fc4e406b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:05.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.170+0000 7fc4feb39700 1 --2- 172.21.15.6:0/3745859995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f80fdfe0 0x7fc4f810de60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:05.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.170+0000 7fc4feb39700 1 -- 172.21.15.6:0/3745859995 >> 172.21.15.6:0/3745859995 conn(0x7fc4f80f8550 msgr2=0x7fc4f80f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:05.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.170+0000 7fc4feb39700 1 -- 172.21.15.6:0/3745859995 shutdown_connections 2024-03-22T06:12:05.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:05.170+0000 7fc4feb39700 1 -- 172.21.15.6:0/3745859995 wait complete. 2024-03-22T06:12:05.524 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:05 smithi006 ceph-mon[29301]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:05.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:05 smithi117 ceph-mon[36514]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:05.884 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-22T06:12:06.148 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:06.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:06 smithi006 ceph-mon[29301]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:06.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:06 smithi117 ceph-mon[36514]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:07.620 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:07 smithi006 ceph-mon[29301]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:07.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:07 smithi117 ceph-mon[36514]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:07.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.836+0000 7f3a629b5700 1 -- 172.21.15.6:0/2020221644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a5c0fed80 msgr2=0x7f3a5c0faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:07.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.836+0000 7f3a629b5700 1 --2- 172.21.15.6:0/2020221644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a5c0fed80 0x7f3a5c0faec0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3a50004940 tx=0x7f3a50009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:07.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.836+0000 7f3a629b5700 1 -- 172.21.15.6:0/2020221644 shutdown_connections 2024-03-22T06:12:07.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.836+0000 7f3a629b5700 1 --2- 172.21.15.6:0/2020221644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a5c0fed80 0x7f3a5c0faec0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:07.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.836+0000 7f3a629b5700 1 --2- 172.21.15.6:0/2020221644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a5c0fe430 0x7f3a5c0fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:07.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.836+0000 7f3a629b5700 1 -- 172.21.15.6:0/2020221644 >> 172.21.15.6:0/2020221644 conn(0x7f3a5c0f6bd0 msgr2=0x7f3a5c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:07.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.836+0000 7f3a629b5700 1 -- 172.21.15.6:0/2020221644 shutdown_connections 2024-03-22T06:12:07.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.836+0000 7f3a629b5700 1 -- 172.21.15.6:0/2020221644 wait complete. 2024-03-22T06:12:07.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.836+0000 7f3a629b5700 1 Processor -- start 2024-03-22T06:12:07.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.836+0000 7f3a629b5700 1 -- start start 2024-03-22T06:12:07.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a629b5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a5c0fe430 0x7f3a5c0fda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:07.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a629b5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a5c0fed80 0x7f3a5c0fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:07.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a629b5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0fe040 con 0x7f3a5c0fed80 2024-03-22T06:12:07.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a629b5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0fc630 con 0x7f3a5c0fe430 2024-03-22T06:12:07.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a5b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a5c0fed80 0x7f3a5c0fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:07.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a5b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a5c0fed80 0x7f3a5c0fc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43972/0 (socket says 172.21.15.6:43972) 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a5b7fe700 1 -- 172.21.15.6:0/3525148536 learned_addr learned my addr 172.21.15.6:0/3525148536 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a5bfff700 1 --2- 172.21.15.6:0/3525148536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a5c0fe430 0x7f3a5c0fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a5b7fe700 1 -- 172.21.15.6:0/3525148536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a5c0fe430 msgr2=0x7f3a5c0fda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a5b7fe700 1 --2- 172.21.15.6:0/3525148536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a5c0fe430 0x7f3a5c0fda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a5b7fe700 1 -- 172.21.15.6:0/3525148536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a50005040 con 0x7f3a5c0fed80 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a5b7fe700 1 --2- 172.21.15.6:0/3525148536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a5c0fed80 0x7f3a5c0fc0f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a50013870 tx=0x7f3a50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a58ff9700 1 -- 172.21.15.6:0/3525148536 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a5000e070 con 0x7f3a5c0fed80 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a58ff9700 1 -- 172.21.15.6:0/3525148536 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a50013de0 con 0x7f3a5c0fed80 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a629b5700 1 -- 172.21.15.6:0/3525148536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c0fc880 con 0x7f3a5c0fed80 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a58ff9700 1 -- 172.21.15.6:0/3525148536 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a5001cea0 con 0x7f3a5c0fed80 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.837+0000 7f3a629b5700 1 -- 172.21.15.6:0/3525148536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c0fcd60 con 0x7f3a5c0fed80 2024-03-22T06:12:07.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.838+0000 7f3a629b5700 1 -- 172.21.15.6:0/3525148536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c1060f0 con 0x7f3a5c0fed80 2024-03-22T06:12:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.839+0000 7f3a58ff9700 1 -- 172.21.15.6:0/3525148536 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f3a5001c690 con 0x7f3a5c0fed80 2024-03-22T06:12:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.839+0000 7f3a58ff9700 1 --2- 172.21.15.6:0/3525148536 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3a440692a0 0x7f3a4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.839+0000 7f3a58ff9700 1 -- 172.21.15.6:0/3525148536 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3a50088550 con 0x7f3a5c0fed80 2024-03-22T06:12:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.839+0000 7f3a5bfff700 1 --2- 172.21.15.6:0/3525148536 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3a440692a0 0x7f3a4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.840+0000 7f3a5bfff700 1 --2- 172.21.15.6:0/3525148536 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3a440692a0 0x7f3a4406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009b80 tx=0x7f3a4c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:07.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:07.844+0000 7f3a58ff9700 1 -- 172.21.15.6:0/3525148536 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3a50058650 con 0x7f3a5c0fed80 2024-03-22T06:12:08.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.047+0000 7f3a629b5700 1 -- 172.21.15.6:0/3525148536 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a5c102420 con 0x7f3a440692a0 2024-03-22T06:12:08.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.055+0000 7f3a58ff9700 1 -- 172.21.15.6:0/3525148536 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2718 (secure 0 0 0) 0x7f3a5c102420 con 0x7f3a440692a0 2024-03-22T06:12:08.055 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (3m) 2m ago 5m 24.8M - 0.25.0 c8568f914cd2 bf3cfbba5c2f 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (5m) 2m ago 5m 6656k - 17.2.7-726-g9c4e0c34 837124da804f a49ab95d42e6 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (4m) 71s ago 4m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0a36af1f8ed0 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 running (3m) 2m ago 4m 80.6M - 9.4.7 954c08fa6188 8b365bc3fd8b 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:9283 running (6m) 2m ago 6m 469M - 17.2.7-726-g9c4e0c34 837124da804f 2cea0b1a5c60 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283 running (4m) 71s ago 4m 434M - 17.2.7-726-g9c4e0c34 837124da804f b14c221438cc 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (6m) 2m ago 6m 41.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f b2fe36c6b005 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (4m) 71s ago 4m 39.9M 2048M 17.2.7-726-g9c4e0c34 837124da804f 0559848da9ab 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (5m) 2m ago 5m 16.3M - 1.5.0 0da6a335fe13 0ef65efb379b 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (4m) 71s ago 4m 15.3M - 1.5.0 0da6a335fe13 58a93ac942d3 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (3m) 2m ago 3m 39.0M 4289M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (3m) 2m ago 2m 35.4M 4289M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (2m) 2m ago 2m 39.1M 4289M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (2m) 71s ago 2m 39.6M 5312M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:12:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (102s) 71s ago 101s 39.1M 5312M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:12:08.057 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (76s) 71s ago 75s 9.96M 5312M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:12:08.057 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (3m) 2m ago 4m 40.3M - 2.43.0 a07b618ecd1d 59d1a24d5af8 2024-03-22T06:12:08.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.057+0000 7f3a629b5700 1 -- 172.21.15.6:0/3525148536 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3a440692a0 msgr2=0x7f3a4406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:08.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.057+0000 7f3a629b5700 1 --2- 172.21.15.6:0/3525148536 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3a440692a0 0x7f3a4406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009b80 tx=0x7f3a4c00b040 comp rx=0 tx=0).stop 2024-03-22T06:12:08.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.057+0000 7f3a629b5700 1 -- 172.21.15.6:0/3525148536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a5c0fed80 msgr2=0x7f3a5c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:08.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.057+0000 7f3a629b5700 1 --2- 172.21.15.6:0/3525148536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a5c0fed80 0x7f3a5c0fc0f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a50013870 tx=0x7f3a50009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:08.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.057+0000 7f3a629b5700 1 -- 172.21.15.6:0/3525148536 shutdown_connections 2024-03-22T06:12:08.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.057+0000 7f3a629b5700 1 --2- 172.21.15.6:0/3525148536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a5c0fe430 0x7f3a5c0fda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:08.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.057+0000 7f3a629b5700 1 --2- 172.21.15.6:0/3525148536 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3a440692a0 0x7f3a4406b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:08.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.057+0000 7f3a629b5700 1 --2- 172.21.15.6:0/3525148536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a5c0fed80 0x7f3a5c0fc0f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:08.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.057+0000 7f3a629b5700 1 -- 172.21.15.6:0/3525148536 >> 172.21.15.6:0/3525148536 conn(0x7f3a5c0f6bd0 msgr2=0x7f3a5c0f8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:08.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.057+0000 7f3a629b5700 1 -- 172.21.15.6:0/3525148536 shutdown_connections 2024-03-22T06:12:08.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:08.057+0000 7f3a629b5700 1 -- 172.21.15.6:0/3525148536 wait complete. 2024-03-22T06:12:08.758 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-22T06:12:09.019 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:09.393 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:09 smithi006 ceph-mon[29301]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:09.393 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:09 smithi006 ceph-mon[29301]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:09.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:09 smithi117 ceph-mon[36514]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:09.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:09 smithi117 ceph-mon[36514]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:10.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.542+0000 7f419a583700 1 -- 172.21.15.6:0/2043281875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41940fce80 msgr2=0x7f41940fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.542+0000 7f419a583700 1 --2- 172.21.15.6:0/2043281875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41940fce80 0x7f41940fd290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4188004940 tx=0x7f4188009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.542+0000 7f419a583700 1 -- 172.21.15.6:0/2043281875 shutdown_connections 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.542+0000 7f419a583700 1 --2- 172.21.15.6:0/2043281875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41940fdff0 0x7f41940fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.542+0000 7f419a583700 1 --2- 172.21.15.6:0/2043281875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41940fce80 0x7f41940fd290 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.542+0000 7f419a583700 1 -- 172.21.15.6:0/2043281875 >> 172.21.15.6:0/2043281875 conn(0x7f41940f8580 msgr2=0x7f41940fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.542+0000 7f419a583700 1 -- 172.21.15.6:0/2043281875 shutdown_connections 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.542+0000 7f419a583700 1 -- 172.21.15.6:0/2043281875 wait complete. 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.542+0000 7f419a583700 1 Processor -- start 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f419a583700 1 -- start start 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f419a583700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41940fce80 0x7f419410da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f419a583700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41940fdff0 0x7f419410dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:10.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f419a583700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419410e630 con 0x7f41940fdff0 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f419a583700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419410ee20 con 0x7f41940fce80 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f41937fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41940fdff0 0x7f419410dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f41937fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41940fdff0 0x7f419410dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43982/0 (socket says 172.21.15.6:43982) 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f41937fe700 1 -- 172.21.15.6:0/4259396300 learned_addr learned my addr 172.21.15.6:0/4259396300 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f4193fff700 1 --2- 172.21.15.6:0/4259396300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41940fce80 0x7f419410da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f41937fe700 1 -- 172.21.15.6:0/4259396300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41940fce80 msgr2=0x7f419410da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f41937fe700 1 --2- 172.21.15.6:0/4259396300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41940fce80 0x7f419410da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f41937fe700 1 -- 172.21.15.6:0/4259396300 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4188005040 con 0x7f41940fdff0 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.543+0000 7f41937fe700 1 --2- 172.21.15.6:0/4259396300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41940fdff0 0x7f419410dfa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f418400b790 tx=0x7f4184009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.544+0000 7f4190ff9700 1 -- 172.21.15.6:0/4259396300 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f418400f6f0 con 0x7f41940fdff0 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.544+0000 7f4190ff9700 1 -- 172.21.15.6:0/4259396300 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f418400fd20 con 0x7f41940fdff0 2024-03-22T06:12:10.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.544+0000 7f419a583700 1 -- 172.21.15.6:0/4259396300 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419410f100 con 0x7f41940fdff0 2024-03-22T06:12:10.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.544+0000 7f4190ff9700 1 -- 172.21.15.6:0/4259396300 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41840103a0 con 0x7f41940fdff0 2024-03-22T06:12:10.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.544+0000 7f419a583700 1 -- 172.21.15.6:0/4259396300 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419410f6c0 con 0x7f41940fdff0 2024-03-22T06:12:10.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.545+0000 7f419a583700 1 -- 172.21.15.6:0/4259396300 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419405f5f0 con 0x7f41940fdff0 2024-03-22T06:12:10.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.545+0000 7f4190ff9700 1 -- 172.21.15.6:0/4259396300 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f4184010500 con 0x7f41940fdff0 2024-03-22T06:12:10.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.545+0000 7f4190ff9700 1 --2- 172.21.15.6:0/4259396300 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f417c0692a0 0x7f417c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:10.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.545+0000 7f4190ff9700 1 -- 172.21.15.6:0/4259396300 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4184086c70 con 0x7f41940fdff0 2024-03-22T06:12:10.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.546+0000 7f4193fff700 1 --2- 172.21.15.6:0/4259396300 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f417c0692a0 0x7f417c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:10.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.546+0000 7f4193fff700 1 --2- 172.21.15.6:0/4259396300 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f417c0692a0 0x7f417c06b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f418800b970 tx=0x7f4188009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:10.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.550+0000 7f4190ff9700 1 -- 172.21.15.6:0/4259396300 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4184056de0 con 0x7f41940fdff0 2024-03-22T06:12:10.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.753+0000 7f419a583700 1 -- 172.21.15.6:0/4259396300 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f419410f990 con 0x7f417c0692a0 2024-03-22T06:12:10.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.758+0000 7f4190ff9700 1 -- 172.21.15.6:0/4259396300 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1161 (secure 0 0 0) 0x7f419410f990 con 0x7f417c0692a0 2024-03-22T06:12:10.758 INFO:teuthology.orchestra.run.smithi006.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-22T06:12:10.758 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-03-22T06:12:10.758 INFO:teuthology.orchestra.run.smithi006.stdout:crash 2/2 2m ago 6m * 2024-03-22T06:12:10.758 INFO:teuthology.orchestra.run.smithi006.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-03-22T06:12:10.758 INFO:teuthology.orchestra.run.smithi006.stdout:mgr 2/2 2m ago 6m count:2 2024-03-22T06:12:10.759 INFO:teuthology.orchestra.run.smithi006.stdout:mon 2/2 2m ago 5m smithi006:172.21.15.6=smithi006;smithi117:172.21.15.117=smithi117;count:2 2024-03-22T06:12:10.759 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-03-22T06:12:10.759 INFO:teuthology.orchestra.run.smithi006.stdout:osd 6 2m ago - 2024-03-22T06:12:10.759 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-03-22T06:12:10.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.760+0000 7f419a583700 1 -- 172.21.15.6:0/4259396300 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f417c0692a0 msgr2=0x7f417c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:10.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.760+0000 7f419a583700 1 --2- 172.21.15.6:0/4259396300 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f417c0692a0 0x7f417c06b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f418800b970 tx=0x7f4188009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.760+0000 7f419a583700 1 -- 172.21.15.6:0/4259396300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41940fdff0 msgr2=0x7f419410dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.760+0000 7f419a583700 1 --2- 172.21.15.6:0/4259396300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41940fdff0 0x7f419410dfa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f418400b790 tx=0x7f4184009250 comp rx=0 tx=0).stop 2024-03-22T06:12:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.760+0000 7f419a583700 1 -- 172.21.15.6:0/4259396300 shutdown_connections 2024-03-22T06:12:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.760+0000 7f419a583700 1 --2- 172.21.15.6:0/4259396300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41940fce80 0x7f419410da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.760+0000 7f419a583700 1 --2- 172.21.15.6:0/4259396300 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f417c0692a0 0x7f417c06b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.760+0000 7f419a583700 1 --2- 172.21.15.6:0/4259396300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41940fdff0 0x7f419410dfa0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.760+0000 7f419a583700 1 -- 172.21.15.6:0/4259396300 >> 172.21.15.6:0/4259396300 conn(0x7f41940f8580 msgr2=0x7f41940f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.760+0000 7f419a583700 1 -- 172.21.15.6:0/4259396300 shutdown_connections 2024-03-22T06:12:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:10.760+0000 7f419a583700 1 -- 172.21.15.6:0/4259396300 wait complete. 2024-03-22T06:12:11.448 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-22T06:12:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:11.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:11 smithi117 ceph-mon[36514]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:11.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:11 smithi006 ceph-mon[29301]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:12.494 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:12 smithi006 ceph-mon[29301]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:12.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:12 smithi117 ceph-mon[36514]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:13.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.324+0000 7f126954f700 1 -- 172.21.15.6:0/3513526528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1264071e40 msgr2=0x7f1264070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:13.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.324+0000 7f126954f700 1 --2- 172.21.15.6:0/3513526528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1264071e40 0x7f1264070350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1254004940 tx=0x7f1254009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:13.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 -- 172.21.15.6:0/3513526528 shutdown_connections 2024-03-22T06:12:13.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 --2- 172.21.15.6:0/3513526528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1264070890 0x7f1264070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 --2- 172.21.15.6:0/3513526528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1264071e40 0x7f1264070350 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 -- 172.21.15.6:0/3513526528 >> 172.21.15.6:0/3513526528 conn(0x7f12640f8550 msgr2=0x7f12640fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 -- 172.21.15.6:0/3513526528 shutdown_connections 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 -- 172.21.15.6:0/3513526528 wait complete. 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 Processor -- start 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 -- start start 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1264070890 0x7f126410fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1264071e40 0x7f12641100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1264110760 con 0x7f1264070890 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f126954f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12641150b0 con 0x7f1264071e40 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f1262ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1264070890 0x7f126410fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f1262ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1264070890 0x7f126410fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43996/0 (socket says 172.21.15.6:43996) 2024-03-22T06:12:13.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.325+0000 7f1262ffd700 1 -- 172.21.15.6:0/641300716 learned_addr learned my addr 172.21.15.6:0/641300716 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.326+0000 7f125bfff700 1 --2- 172.21.15.6:0/641300716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1264071e40 0x7f12641100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.326+0000 7f1262ffd700 1 -- 172.21.15.6:0/641300716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1264071e40 msgr2=0x7f12641100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.326+0000 7f1262ffd700 1 --2- 172.21.15.6:0/641300716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1264071e40 0x7f12641100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.326+0000 7f1262ffd700 1 -- 172.21.15.6:0/641300716 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1254005040 con 0x7f1264070890 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.326+0000 7f1262ffd700 1 --2- 172.21.15.6:0/641300716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1264070890 0x7f126410fb90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1254000c00 tx=0x7f1254009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.326+0000 7f125b7fe700 1 -- 172.21.15.6:0/641300716 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125400e040 con 0x7f1264070890 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.326+0000 7f125b7fe700 1 -- 172.21.15.6:0/641300716 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f125400bcc0 con 0x7f1264070890 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.326+0000 7f126954f700 1 -- 172.21.15.6:0/641300716 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12641152b0 con 0x7f1264070890 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.326+0000 7f125b7fe700 1 -- 172.21.15.6:0/641300716 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125401c630 con 0x7f1264070890 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.326+0000 7f126954f700 1 -- 172.21.15.6:0/641300716 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1264115820 con 0x7f1264070890 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.327+0000 7f126954f700 1 -- 172.21.15.6:0/641300716 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126404c710 con 0x7f1264070890 2024-03-22T06:12:13.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.327+0000 7f125b7fe700 1 -- 172.21.15.6:0/641300716 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f1254013980 con 0x7f1264070890 2024-03-22T06:12:13.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.328+0000 7f125b7fe700 1 --2- 172.21.15.6:0/641300716 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f12500692a0 0x7f125006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:13.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.328+0000 7f125b7fe700 1 -- 172.21.15.6:0/641300716 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f12540882a0 con 0x7f1264070890 2024-03-22T06:12:13.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.328+0000 7f125bfff700 1 --2- 172.21.15.6:0/641300716 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f12500692a0 0x7f125006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:13.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.328+0000 7f125bfff700 1 --2- 172.21.15.6:0/641300716 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f12500692a0 0x7f125006b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f124c006fd0 tx=0x7f124c008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:13.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.332+0000 7f125b7fe700 1 -- 172.21.15.6:0/641300716 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1254058450 con 0x7f1264070890 2024-03-22T06:12:13.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.533+0000 7f126954f700 1 -- 172.21.15.6:0/641300716 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1264103e20 con 0x7f12500692a0 2024-03-22T06:12:13.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.534+0000 7f125b7fe700 1 -- 172.21.15.6:0/641300716 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f1264103e20 con 0x7f12500692a0 2024-03-22T06:12:13.534 INFO:teuthology.orchestra.run.smithi006.stdout:HOST ADDR LABELS STATUS 2024-03-22T06:12:13.535 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006 172.21.15.6 2024-03-22T06:12:13.535 INFO:teuthology.orchestra.run.smithi006.stdout:smithi117 172.21.15.117 2024-03-22T06:12:13.535 INFO:teuthology.orchestra.run.smithi006.stdout:2 hosts in cluster 2024-03-22T06:12:13.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.536+0000 7f126954f700 1 -- 172.21.15.6:0/641300716 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f12500692a0 msgr2=0x7f125006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:13.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.536+0000 7f126954f700 1 --2- 172.21.15.6:0/641300716 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f12500692a0 0x7f125006b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f124c006fd0 tx=0x7f124c008040 comp rx=0 tx=0).stop 2024-03-22T06:12:13.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.536+0000 7f126954f700 1 -- 172.21.15.6:0/641300716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1264070890 msgr2=0x7f126410fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:13.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.536+0000 7f126954f700 1 --2- 172.21.15.6:0/641300716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1264070890 0x7f126410fb90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1254000c00 tx=0x7f1254009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:13.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.536+0000 7f126954f700 1 -- 172.21.15.6:0/641300716 shutdown_connections 2024-03-22T06:12:13.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.536+0000 7f126954f700 1 --2- 172.21.15.6:0/641300716 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f12500692a0 0x7f125006b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:13.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.536+0000 7f126954f700 1 --2- 172.21.15.6:0/641300716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1264070890 0x7f126410fb90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:13.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.536+0000 7f126954f700 1 --2- 172.21.15.6:0/641300716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1264071e40 0x7f12641100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:13.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.536+0000 7f126954f700 1 -- 172.21.15.6:0/641300716 >> 172.21.15.6:0/641300716 conn(0x7f12640f8550 msgr2=0x7f12640f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:13.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.536+0000 7f126954f700 1 -- 172.21.15.6:0/641300716 shutdown_connections 2024-03-22T06:12:13.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:13.536+0000 7f126954f700 1 -- 172.21.15.6:0/641300716 wait complete. 2024-03-22T06:12:13.631 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:13 smithi006 ceph-mon[29301]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:13.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:13 smithi117 ceph-mon[36514]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:14.160 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-22T06:12:14.419 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:15.475 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:15 smithi006 ceph-mon[29301]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:15.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:15 smithi006 ceph-mon[29301]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:15.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:15 smithi117 ceph-mon[36514]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:15.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:15 smithi117 ceph-mon[36514]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.990+0000 7f5d7d267700 1 -- 172.21.15.6:0/13623601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d780fe450 msgr2=0x7f5d780fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.990+0000 7f5d7d267700 1 --2- 172.21.15.6:0/13623601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d780fe450 0x7f5d780fe860 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5d68004940 tx=0x7f5d68009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 -- 172.21.15.6:0/13623601 shutdown_connections 2024-03-22T06:12:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 --2- 172.21.15.6:0/13623601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d780feda0 0x7f5d780faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 --2- 172.21.15.6:0/13623601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d780fe450 0x7f5d780fe860 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 -- 172.21.15.6:0/13623601 >> 172.21.15.6:0/13623601 conn(0x7f5d780f6c10 msgr2=0x7f5d780f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 -- 172.21.15.6:0/13623601 shutdown_connections 2024-03-22T06:12:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 -- 172.21.15.6:0/13623601 wait complete. 2024-03-22T06:12:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 Processor -- start 2024-03-22T06:12:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 -- start start 2024-03-22T06:12:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d780fe450 0x7f5d7810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d780feda0 0x7f5d7810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7810e4e0 con 0x7f5d780fe450 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d7d267700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78112e80 con 0x7f5d780feda0 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.991+0000 7f5d76ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d780fe450 0x7f5d7810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.992+0000 7f5d76ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d780fe450 0x7f5d7810d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57124/0 (socket says 172.21.15.6:57124) 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.992+0000 7f5d76ffd700 1 -- 172.21.15.6:0/435598154 learned_addr learned my addr 172.21.15.6:0/435598154 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.992+0000 7f5d76ffd700 1 -- 172.21.15.6:0/435598154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d780feda0 msgr2=0x7f5d7810de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.992+0000 7f5d76ffd700 1 --2- 172.21.15.6:0/435598154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d780feda0 0x7f5d7810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.992+0000 7f5d76ffd700 1 -- 172.21.15.6:0/435598154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d68005040 con 0x7f5d780fe450 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.992+0000 7f5d76ffd700 1 --2- 172.21.15.6:0/435598154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d780fe450 0x7f5d7810d910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800b3b0 tx=0x7f5d68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.992+0000 7f5d5f7fe700 1 -- 172.21.15.6:0/435598154 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d6800e040 con 0x7f5d780fe450 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.992+0000 7f5d5f7fe700 1 -- 172.21.15.6:0/435598154 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d68004290 con 0x7f5d780fe450 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.992+0000 7f5d7d267700 1 -- 172.21.15.6:0/435598154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d78113080 con 0x7f5d780fe450 2024-03-22T06:12:15.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.992+0000 7f5d5f7fe700 1 -- 172.21.15.6:0/435598154 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d6801c630 con 0x7f5d780fe450 2024-03-22T06:12:15.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.992+0000 7f5d7d267700 1 -- 172.21.15.6:0/435598154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d781135f0 con 0x7f5d780fe450 2024-03-22T06:12:15.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.993+0000 7f5d7d267700 1 -- 172.21.15.6:0/435598154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7804c710 con 0x7f5d780fe450 2024-03-22T06:12:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.994+0000 7f5d5f7fe700 1 -- 172.21.15.6:0/435598154 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f5d6800bbf0 con 0x7f5d780fe450 2024-03-22T06:12:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.994+0000 7f5d5f7fe700 1 --2- 172.21.15.6:0/435598154 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5d600692a0 0x7f5d6006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.994+0000 7f5d5f7fe700 1 -- 172.21.15.6:0/435598154 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5d680882a0 con 0x7f5d780fe450 2024-03-22T06:12:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.994+0000 7f5d767fc700 1 --2- 172.21.15.6:0/435598154 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5d600692a0 0x7f5d6006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.994+0000 7f5d767fc700 1 --2- 172.21.15.6:0/435598154 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5d600692a0 0x7f5d6006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c006fd0 tx=0x7f5d6c006db0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:15.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:15.998+0000 7f5d5f7fe700 1 -- 172.21.15.6:0/435598154 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5d680583a0 con 0x7f5d780fe450 2024-03-22T06:12:16.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.199+0000 7f5d7d267700 1 -- 172.21.15.6:0/435598154 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d781024d0 con 0x7f5d600692a0 2024-03-22T06:12:16.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.201+0000 7f5d5f7fe700 1 -- 172.21.15.6:0/435598154 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f5d781024d0 con 0x7f5d600692a0 2024-03-22T06:12:16.201 INFO:teuthology.orchestra.run.smithi006.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-22T06:12:16.201 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330000H400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-22T06:12:16.201 INFO:teuthology.orchestra.run.smithi006.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 75s ago Has a FileSystem, LVM detected 2024-03-22T06:12:16.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.203+0000 7f5d7d267700 1 -- 172.21.15.6:0/435598154 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5d600692a0 msgr2=0x7f5d6006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:16.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.203+0000 7f5d7d267700 1 --2- 172.21.15.6:0/435598154 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5d600692a0 0x7f5d6006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c006fd0 tx=0x7f5d6c006db0 comp rx=0 tx=0).stop 2024-03-22T06:12:16.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.203+0000 7f5d7d267700 1 -- 172.21.15.6:0/435598154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d780fe450 msgr2=0x7f5d7810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:16.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.203+0000 7f5d7d267700 1 --2- 172.21.15.6:0/435598154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d780fe450 0x7f5d7810d910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800b3b0 tx=0x7f5d68009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:16.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.203+0000 7f5d7d267700 1 -- 172.21.15.6:0/435598154 shutdown_connections 2024-03-22T06:12:16.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.203+0000 7f5d7d267700 1 --2- 172.21.15.6:0/435598154 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f5d600692a0 0x7f5d6006b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:16.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.203+0000 7f5d7d267700 1 --2- 172.21.15.6:0/435598154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d780fe450 0x7f5d7810d910 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:16.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.203+0000 7f5d7d267700 1 --2- 172.21.15.6:0/435598154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d780feda0 0x7f5d7810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:16.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.203+0000 7f5d7d267700 1 -- 172.21.15.6:0/435598154 >> 172.21.15.6:0/435598154 conn(0x7f5d780f6c10 msgr2=0x7f5d780f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:16.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.203+0000 7f5d7d267700 1 -- 172.21.15.6:0/435598154 shutdown_connections 2024-03-22T06:12:16.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:16.203+0000 7f5d7d267700 1 -- 172.21.15.6:0/435598154 wait complete. 2024-03-22T06:12:16.938 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T06:12:16.949 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:12:16.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-22T06:12:17.218 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:17.636 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:17 smithi006 ceph-mon[29301]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:17.636 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:17 smithi006 ceph-mon[29301]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:17.636 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:12:17.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:17 smithi117 ceph-mon[36514]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:17.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:17 smithi117 ceph-mon[36514]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:17.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:12:18.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.816+0000 7ff782c0d700 1 -- 172.21.15.6:0/3624639627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff77c0fe430 msgr2=0x7ff77c0fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.816+0000 7ff782c0d700 1 --2- 172.21.15.6:0/3624639627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff77c0fe430 0x7ff77c0fe840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff770004940 tx=0x7ff770009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.816+0000 7ff782c0d700 1 -- 172.21.15.6:0/3624639627 shutdown_connections 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.816+0000 7ff782c0d700 1 --2- 172.21.15.6:0/3624639627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff77c0fed80 0x7ff77c0faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.816+0000 7ff782c0d700 1 --2- 172.21.15.6:0/3624639627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff77c0fe430 0x7ff77c0fe840 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.816+0000 7ff782c0d700 1 -- 172.21.15.6:0/3624639627 >> 172.21.15.6:0/3624639627 conn(0x7ff77c0f6bd0 msgr2=0x7ff77c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.816+0000 7ff782c0d700 1 -- 172.21.15.6:0/3624639627 shutdown_connections 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.816+0000 7ff782c0d700 1 -- 172.21.15.6:0/3624639627 wait complete. 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.817+0000 7ff782c0d700 1 Processor -- start 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.817+0000 7ff782c0d700 1 -- start start 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.817+0000 7ff782c0d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff77c0fe430 0x7ff77c10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.817+0000 7ff782c0d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff77c0fed80 0x7ff77c10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.817+0000 7ff782c0d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c10e460 con 0x7ff77c0fed80 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.817+0000 7ff782c0d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c10e5d0 con 0x7ff77c0fe430 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.817+0000 7ff77bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff77c0fed80 0x7ff77c10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.817+0000 7ff77bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff77c0fed80 0x7ff77c10de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57130/0 (socket says 172.21.15.6:57130) 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.817+0000 7ff77bfff700 1 -- 172.21.15.6:0/2787989202 learned_addr learned my addr 172.21.15.6:0/2787989202 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.818+0000 7ff7809a9700 1 --2- 172.21.15.6:0/2787989202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff77c0fe430 0x7ff77c10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.818+0000 7ff77bfff700 1 -- 172.21.15.6:0/2787989202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff77c0fe430 msgr2=0x7ff77c10d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.818+0000 7ff77bfff700 1 --2- 172.21.15.6:0/2787989202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff77c0fe430 0x7ff77c10d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.818+0000 7ff77bfff700 1 -- 172.21.15.6:0/2787989202 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770005040 con 0x7ff77c0fed80 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.818+0000 7ff77bfff700 1 --2- 172.21.15.6:0/2787989202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff77c0fed80 0x7ff77c10de60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff76800c8c0 tx=0x7ff7680044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.818+0000 7ff7797fa700 1 -- 172.21.15.6:0/2787989202 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff76800a760 con 0x7ff77c0fed80 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.818+0000 7ff7797fa700 1 -- 172.21.15.6:0/2787989202 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff76800ad90 con 0x7ff77c0fed80 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.818+0000 7ff7797fa700 1 -- 172.21.15.6:0/2787989202 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7680113a0 con 0x7ff77c0fed80 2024-03-22T06:12:18.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.818+0000 7ff782c0d700 1 -- 172.21.15.6:0/2787989202 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c113030 con 0x7ff77c0fed80 2024-03-22T06:12:18.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.818+0000 7ff782c0d700 1 -- 172.21.15.6:0/2787989202 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c1135c0 con 0x7ff77c0fed80 2024-03-22T06:12:18.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.819+0000 7ff782c0d700 1 -- 172.21.15.6:0/2787989202 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c04c710 con 0x7ff77c0fed80 2024-03-22T06:12:18.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.820+0000 7ff7797fa700 1 -- 172.21.15.6:0/2787989202 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7ff768012400 con 0x7ff77c0fed80 2024-03-22T06:12:18.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.820+0000 7ff7797fa700 1 --2- 172.21.15.6:0/2787989202 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff76c0692a0 0x7ff76c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:18.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.820+0000 7ff7797fa700 1 -- 172.21.15.6:0/2787989202 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff768086eb0 con 0x7ff77c0fed80 2024-03-22T06:12:18.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.820+0000 7ff7809a9700 1 --2- 172.21.15.6:0/2787989202 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff76c0692a0 0x7ff76c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:18.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.820+0000 7ff7809a9700 1 --2- 172.21.15.6:0/2787989202 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff76c0692a0 0x7ff76c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff77000b3b0 tx=0x7ff770009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:18.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:18.825+0000 7ff7797fa700 1 -- 172.21.15.6:0/2787989202 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff768056fb0 con 0x7ff77c0fed80 2024-03-22T06:12:19.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:19.048+0000 7ff782c0d700 1 -- 172.21.15.6:0/2787989202 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff77c102520 con 0x7ff76c0692a0 2024-03-22T06:12:19.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:19 smithi117 ceph-mon[36514]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:19.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:19 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-22T06:12:19.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:19 smithi006 ceph-mon[29301]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:19.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:19 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-22T06:12:20.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.404+0000 7ff7797fa700 1 -- 172.21.15.6:0/2787989202 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff77c102520 con 0x7ff76c0692a0 2024-03-22T06:12:20.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.407+0000 7ff782c0d700 1 -- 172.21.15.6:0/2787989202 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff76c0692a0 msgr2=0x7ff76c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:20.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.407+0000 7ff782c0d700 1 --2- 172.21.15.6:0/2787989202 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff76c0692a0 0x7ff76c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff77000b3b0 tx=0x7ff770009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:20.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.407+0000 7ff782c0d700 1 -- 172.21.15.6:0/2787989202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff77c0fed80 msgr2=0x7ff77c10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:20.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.407+0000 7ff782c0d700 1 --2- 172.21.15.6:0/2787989202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff77c0fed80 0x7ff77c10de60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff76800c8c0 tx=0x7ff7680044c0 comp rx=0 tx=0).stop 2024-03-22T06:12:20.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.407+0000 7ff782c0d700 1 -- 172.21.15.6:0/2787989202 shutdown_connections 2024-03-22T06:12:20.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.407+0000 7ff782c0d700 1 --2- 172.21.15.6:0/2787989202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff77c0fe430 0x7ff77c10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:20.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.407+0000 7ff782c0d700 1 --2- 172.21.15.6:0/2787989202 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ff76c0692a0 0x7ff76c06b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:20.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.407+0000 7ff782c0d700 1 --2- 172.21.15.6:0/2787989202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff77c0fed80 0x7ff77c10de60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:20.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.407+0000 7ff782c0d700 1 -- 172.21.15.6:0/2787989202 >> 172.21.15.6:0/2787989202 conn(0x7ff77c0f6bd0 msgr2=0x7ff77c0f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:20.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.407+0000 7ff782c0d700 1 -- 172.21.15.6:0/2787989202 shutdown_connections 2024-03-22T06:12:20.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:20.407+0000 7ff782c0d700 1 -- 172.21.15.6:0/2787989202 wait complete. 2024-03-22T06:12:20.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:20 smithi006 ceph-mon[29301]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:20.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:20 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-22T06:12:20.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:20 smithi006 ceph-mon[29301]: osdmap e42: 6 total, 6 up, 6 in 2024-03-22T06:12:20.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:20 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-22T06:12:20.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:20 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006[29297]: 2024-03-22T06:12:20.385+0000 7fa8e9924700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:12:20.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:20 smithi117 ceph-mon[36514]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:12:20.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:20 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-22T06:12:20.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:20 smithi117 ceph-mon[36514]: osdmap e42: 6 total, 6 up, 6 in 2024-03-22T06:12:20.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:20 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-22T06:12:21.082 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T06:12:21.634 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: pgmap v174: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:21.634 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-22T06:12:21.634 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-22T06:12:21.634 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: osdmap e43: 6 total, 6 up, 6 in 2024-03-22T06:12:21.634 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-22T06:12:21.634 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:12:21.634 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T06:12:21.634 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: osdmap e44: 6 total, 6 up, 6 in 2024-03-22T06:12:21.634 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-22T06:12:21.634 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: fsmap cephfs:0 2024-03-22T06:12:21.635 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: Saving service mds.cephfs spec with placement count:4 2024-03-22T06:12:21.635 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:21.635 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:12:21.635 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:21.635 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:12:21.635 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:21.635 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.bztopc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:12:21.635 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.bztopc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T06:12:21.635 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:21.635 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:21 smithi006 ceph-mon[29301]: Deploying daemon mds.cephfs.smithi006.bztopc on smithi006 2024-03-22T06:12:21.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: pgmap v174: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:21.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-22T06:12:21.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-22T06:12:21.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: osdmap e43: 6 total, 6 up, 6 in 2024-03-22T06:12:21.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-22T06:12:21.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:12:21.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T06:12:21.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: osdmap e44: 6 total, 6 up, 6 in 2024-03-22T06:12:21.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-22T06:12:21.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: fsmap cephfs:0 2024-03-22T06:12:21.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: Saving service mds.cephfs spec with placement count:4 2024-03-22T06:12:21.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:21.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:12:21.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:21.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:12:21.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:21.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.bztopc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:12:21.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.bztopc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T06:12:21.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:21.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:21 smithi117 ceph-mon[36514]: Deploying daemon mds.cephfs.smithi006.bztopc on smithi006 2024-03-22T06:12:22.447 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:22.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:22 smithi117 ceph-mon[36514]: osdmap e45: 6 total, 6 up, 6 in 2024-03-22T06:12:22.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:22 smithi006 ceph-mon[29301]: osdmap e45: 6 total, 6 up, 6 in 2024-03-22T06:12:23.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:23 smithi006 ceph-mon[29301]: pgmap v178: 65 pgs: 15 creating+peering, 8 active+clean, 42 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:23.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:23 smithi006 ceph-mon[29301]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-22T06:12:23.670 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:23 smithi006 ceph-mon[29301]: osdmap e46: 6 total, 6 up, 6 in 2024-03-22T06:12:23.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:23 smithi117 ceph-mon[36514]: pgmap v178: 65 pgs: 15 creating+peering, 8 active+clean, 42 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:23.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:23 smithi117 ceph-mon[36514]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-22T06:12:23.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:23 smithi117 ceph-mon[36514]: osdmap e46: 6 total, 6 up, 6 in 2024-03-22T06:12:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.157+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/2325763665 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b240fbcd0 msgr2=0x7f6b240fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.157+0000 7f6b2ac3f700 1 --2- 172.21.15.6:0/2325763665 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b240fbcd0 0x7f6b240fc0e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6b18004940 tx=0x7f6b18009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.157+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/2325763665 shutdown_connections 2024-03-22T06:12:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.157+0000 7f6b2ac3f700 1 --2- 172.21.15.6:0/2325763665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b240fcf80 0x7f6b240fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.157+0000 7f6b2ac3f700 1 --2- 172.21.15.6:0/2325763665 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b240fbcd0 0x7f6b240fc0e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.157+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/2325763665 >> 172.21.15.6:0/2325763665 conn(0x7f6b240f7450 msgr2=0x7f6b240f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.157+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/2325763665 shutdown_connections 2024-03-22T06:12:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.157+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/2325763665 wait complete. 2024-03-22T06:12:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.157+0000 7f6b2ac3f700 1 Processor -- start 2024-03-22T06:12:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.157+0000 7f6b2ac3f700 1 -- start start 2024-03-22T06:12:24.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b2ac3f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b240fbcd0 0x7f6b24066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:24.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b2ac3f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b240fcf80 0x7f6b24112710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:24.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b2ac3f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b24112c50 con 0x7f6b240fbcd0 2024-03-22T06:12:24.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b2ac3f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b24112d90 con 0x7f6b240fcf80 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b289db700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b240fbcd0 0x7f6b24066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b289db700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b240fbcd0 0x7f6b24066cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33114/0 (socket says 172.21.15.6:33114) 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b289db700 1 -- 172.21.15.6:0/3898138509 learned_addr learned my addr 172.21.15.6:0/3898138509 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b23fff700 1 --2- 172.21.15.6:0/3898138509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b240fcf80 0x7f6b24112710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b289db700 1 -- 172.21.15.6:0/3898138509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b240fcf80 msgr2=0x7f6b24112710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b289db700 1 --2- 172.21.15.6:0/3898138509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b240fcf80 0x7f6b24112710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b289db700 1 -- 172.21.15.6:0/3898138509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b18005040 con 0x7f6b240fbcd0 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b289db700 1 --2- 172.21.15.6:0/3898138509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b240fbcd0 0x7f6b24066cf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009150 tx=0x7f6b18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b217fa700 1 -- 172.21.15.6:0/3898138509 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b1800e040 con 0x7f6b240fbcd0 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b217fa700 1 -- 172.21.15.6:0/3898138509 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b18004290 con 0x7f6b240fbcd0 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/3898138509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b24112fc0 con 0x7f6b240fbcd0 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.158+0000 7f6b217fa700 1 -- 172.21.15.6:0/3898138509 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b1801c630 con 0x7f6b240fbcd0 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.159+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/3898138509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b241134a0 con 0x7f6b240fbcd0 2024-03-22T06:12:24.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.159+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/3898138509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2404c710 con 0x7f6b240fbcd0 2024-03-22T06:12:24.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.160+0000 7f6b217fa700 1 -- 172.21.15.6:0/3898138509 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f6b1800bbf0 con 0x7f6b240fbcd0 2024-03-22T06:12:24.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.160+0000 7f6b217fa700 1 --2- 172.21.15.6:0/3898138509 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6b140692a0 0x7f6b1406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:24.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.160+0000 7f6b217fa700 1 -- 172.21.15.6:0/3898138509 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6b18087e00 con 0x7f6b240fbcd0 2024-03-22T06:12:24.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.160+0000 7f6b23fff700 1 --2- 172.21.15.6:0/3898138509 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6b140692a0 0x7f6b1406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:24.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.161+0000 7f6b23fff700 1 --2- 172.21.15.6:0/3898138509 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6b140692a0 0x7f6b1406b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6b10007fd0 tx=0x7f6b10009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:24.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.165+0000 7f6b217fa700 1 -- 172.21.15.6:0/3898138509 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6b18057c60 con 0x7f6b240fbcd0 2024-03-22T06:12:24.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.432+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/3898138509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6b241047c0 con 0x7f6b240fbcd0 2024-03-22T06:12:24.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.433+0000 7f6b217fa700 1 -- 172.21.15.6:0/3898138509 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f6b180577f0 con 0x7f6b240fbcd0 2024-03-22T06:12:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:e2 2024-03-22T06:12:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:12:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:12:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:12:24.434 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:12:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:12:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:12:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 2 2024-03-22T06:12:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:12:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:12:24.434 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:20.386089+0000 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:in 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:up {} 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:12:24.435 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:12:24.436 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:12:24.436 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:12:24.436 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:12:24.436 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:12:24.436 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 0 2024-03-22T06:12:24.436 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:12:24.436 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:12:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.435+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/3898138509 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6b140692a0 msgr2=0x7f6b1406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.435+0000 7f6b2ac3f700 1 --2- 172.21.15.6:0/3898138509 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6b140692a0 0x7f6b1406b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6b10007fd0 tx=0x7f6b10009040 comp rx=0 tx=0).stop 2024-03-22T06:12:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.435+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/3898138509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b240fbcd0 msgr2=0x7f6b24066cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.435+0000 7f6b2ac3f700 1 --2- 172.21.15.6:0/3898138509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b240fbcd0 0x7f6b24066cf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009150 tx=0x7f6b18009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.436+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/3898138509 shutdown_connections 2024-03-22T06:12:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.436+0000 7f6b2ac3f700 1 --2- 172.21.15.6:0/3898138509 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6b140692a0 0x7f6b1406b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.436+0000 7f6b2ac3f700 1 --2- 172.21.15.6:0/3898138509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b240fbcd0 0x7f6b24066cf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.436+0000 7f6b2ac3f700 1 --2- 172.21.15.6:0/3898138509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b240fcf80 0x7f6b24112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.436+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/3898138509 >> 172.21.15.6:0/3898138509 conn(0x7f6b240f7450 msgr2=0x7f6b240f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.436+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/3898138509 shutdown_connections 2024-03-22T06:12:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:24.436+0000 7f6b2ac3f700 1 -- 172.21.15.6:0/3898138509 wait complete. 2024-03-22T06:12:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 2 2024-03-22T06:12:25.498 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:25 smithi006 ceph-mon[29301]: pgmap v180: 65 pgs: 15 creating+peering, 38 active+clean, 12 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:25.498 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:25 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3898138509' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:12:25.498 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:25 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:25.498 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:25 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:25.498 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:25 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:25.499 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:25 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.fwkech", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:12:25.499 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:25 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.fwkech", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T06:12:25.499 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:25 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:25.523 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T06:12:25.533 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:12:25.533 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-03-22T06:12:25.636 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:25 smithi117 ceph-mon[36514]: pgmap v180: 65 pgs: 15 creating+peering, 38 active+clean, 12 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:25 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3898138509' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:12:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:25 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:25 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:25 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:25 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.fwkech", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:12:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:25 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.fwkech", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T06:12:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:25 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:25.798 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:26.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:26 smithi006 ceph-mon[29301]: Deploying daemon mds.cephfs.smithi117.fwkech on smithi117 2024-03-22T06:12:26.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:26 smithi006 ceph-mon[29301]: mds.? [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] up:boot 2024-03-22T06:12:26.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:26 smithi006 ceph-mon[29301]: daemon mds.cephfs.smithi006.bztopc assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-22T06:12:26.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:26 smithi006 ceph-mon[29301]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T06:12:26.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:26 smithi006 ceph-mon[29301]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T06:12:26.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:26 smithi006 ceph-mon[29301]: Cluster is now healthy 2024-03-22T06:12:26.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:26 smithi006 ceph-mon[29301]: fsmap cephfs:0 1 up:standby 2024-03-22T06:12:26.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:26 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:12:26.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:26 smithi006 ceph-mon[29301]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:creating} 2024-03-22T06:12:26.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:26 smithi006 ceph-mon[29301]: daemon mds.cephfs.smithi006.bztopc is now active in filesystem cephfs as rank 0 2024-03-22T06:12:26.721 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:26 smithi117 ceph-mon[36514]: Deploying daemon mds.cephfs.smithi117.fwkech on smithi117 2024-03-22T06:12:26.721 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:26 smithi117 ceph-mon[36514]: mds.? [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] up:boot 2024-03-22T06:12:26.721 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:26 smithi117 ceph-mon[36514]: daemon mds.cephfs.smithi006.bztopc assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-22T06:12:26.721 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:26 smithi117 ceph-mon[36514]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T06:12:26.721 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:26 smithi117 ceph-mon[36514]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T06:12:26.721 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:26 smithi117 ceph-mon[36514]: Cluster is now healthy 2024-03-22T06:12:26.721 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:26 smithi117 ceph-mon[36514]: fsmap cephfs:0 1 up:standby 2024-03-22T06:12:26.721 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:26 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:12:26.721 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:26 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:creating} 2024-03-22T06:12:26.721 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:26 smithi117 ceph-mon[36514]: daemon mds.cephfs.smithi006.bztopc is now active in filesystem cephfs as rank 0 2024-03-22T06:12:27.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.491+0000 7f0486316700 1 -- 172.21.15.6:0/2656768903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04800fff80 msgr2=0x7f0480102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:27.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.491+0000 7f0486316700 1 --2- 172.21.15.6:0/2656768903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04800fff80 0x7f0480102360 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0470004940 tx=0x7f0470009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:27.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.491+0000 7f0486316700 1 -- 172.21.15.6:0/2656768903 shutdown_connections 2024-03-22T06:12:27.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.491+0000 7f0486316700 1 --2- 172.21.15.6:0/2656768903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04800fff80 0x7f0480102360 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:27.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.491+0000 7f0486316700 1 --2- 172.21.15.6:0/2656768903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0480066260 0x7f04800ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:27.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.491+0000 7f0486316700 1 -- 172.21.15.6:0/2656768903 >> 172.21.15.6:0/2656768903 conn(0x7f04800f7430 msgr2=0x7f04800f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:27.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.491+0000 7f0486316700 1 -- 172.21.15.6:0/2656768903 shutdown_connections 2024-03-22T06:12:27.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.491+0000 7f0486316700 1 -- 172.21.15.6:0/2656768903 wait complete. 2024-03-22T06:12:27.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.492+0000 7f0486316700 1 Processor -- start 2024-03-22T06:12:27.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.492+0000 7f0486316700 1 -- start start 2024-03-22T06:12:27.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.492+0000 7f0486316700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0480066260 0x7f048010da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:27.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.492+0000 7f0486316700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04800fff80 0x7f048010df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:27.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.492+0000 7f0486316700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048010e590 con 0x7f0480066260 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.492+0000 7f0486316700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048010ede0 con 0x7f04800fff80 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.492+0000 7f047ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0480066260 0x7f048010da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f047ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0480066260 0x7f048010da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33160/0 (socket says 172.21.15.6:33160) 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f047ffff700 1 -- 172.21.15.6:0/3516292821 learned_addr learned my addr 172.21.15.6:0/3516292821 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f047f7fe700 1 --2- 172.21.15.6:0/3516292821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04800fff80 0x7f048010df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f047ffff700 1 -- 172.21.15.6:0/3516292821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04800fff80 msgr2=0x7f048010df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f047ffff700 1 --2- 172.21.15.6:0/3516292821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04800fff80 0x7f048010df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f047ffff700 1 -- 172.21.15.6:0/3516292821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0470005040 con 0x7f0480066260 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f047ffff700 1 --2- 172.21.15.6:0/3516292821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0480066260 0x7f048010da50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f046800c8b0 tx=0x7f0468004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f047cff9700 1 -- 172.21.15.6:0/3516292821 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f046800a760 con 0x7f0480066260 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f047cff9700 1 -- 172.21.15.6:0/3516292821 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f046800ad90 con 0x7f0480066260 2024-03-22T06:12:27.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f0486316700 1 -- 172.21.15.6:0/3516292821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048010f0c0 con 0x7f0480066260 2024-03-22T06:12:27.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f047cff9700 1 -- 172.21.15.6:0/3516292821 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04680113a0 con 0x7f0480066260 2024-03-22T06:12:27.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.493+0000 7f0486316700 1 -- 172.21.15.6:0/3516292821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048010f600 con 0x7f0480066260 2024-03-22T06:12:27.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.494+0000 7f0486316700 1 -- 172.21.15.6:0/3516292821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048005f5f0 con 0x7f0480066260 2024-03-22T06:12:27.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.495+0000 7f047cff9700 1 -- 172.21.15.6:0/3516292821 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f0468011610 con 0x7f0480066260 2024-03-22T06:12:27.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.495+0000 7f047cff9700 1 --2- 172.21.15.6:0/3516292821 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f046c0692a0 0x7f046c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:27.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.495+0000 7f047cff9700 1 -- 172.21.15.6:0/3516292821 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0468087200 con 0x7f0480066260 2024-03-22T06:12:27.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.495+0000 7f047f7fe700 1 --2- 172.21.15.6:0/3516292821 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f046c0692a0 0x7f046c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:27.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.495+0000 7f047f7fe700 1 --2- 172.21.15.6:0/3516292821 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f046c0692a0 0x7f046c06b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0470000c00 tx=0x7f0470009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:27.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.500+0000 7f047cff9700 1 -- 172.21.15.6:0/3516292821 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0468056fc0 con 0x7f0480066260 2024-03-22T06:12:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:27 smithi006 ceph-mon[29301]: pgmap v181: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 353 B/s wr, 0 op/s 2024-03-22T06:12:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:27 smithi006 ceph-mon[29301]: mds.? [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] up:active 2024-03-22T06:12:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:27 smithi006 ceph-mon[29301]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 2024-03-22T06:12:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:27.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.dcjpla", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:12:27.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.dcjpla", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T06:12:27.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:27 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:27.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:27.747+0000 7f0486316700 1 -- 172.21.15.6:0/3516292821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f04801058b0 con 0x7f0480066260 2024-03-22T06:12:27.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:27 smithi117 ceph-mon[36514]: pgmap v181: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 353 B/s wr, 0 op/s 2024-03-22T06:12:27.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:27 smithi117 ceph-mon[36514]: mds.? [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] up:active 2024-03-22T06:12:27.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:27 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 2024-03-22T06:12:27.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:27.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:27.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:27.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.dcjpla", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:12:27.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.dcjpla", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T06:12:27.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:27 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:28.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.424+0000 7f047cff9700 1 -- 172.21.15.6:0/3516292821 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f0468056b50 con 0x7f0480066260 2024-03-22T06:12:28.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.426+0000 7f0486316700 1 -- 172.21.15.6:0/3516292821 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f046c0692a0 msgr2=0x7f046c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:28.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.426+0000 7f0486316700 1 --2- 172.21.15.6:0/3516292821 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f046c0692a0 0x7f046c06b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0470000c00 tx=0x7f0470009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:28.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.426+0000 7f0486316700 1 -- 172.21.15.6:0/3516292821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0480066260 msgr2=0x7f048010da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:28.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.426+0000 7f0486316700 1 --2- 172.21.15.6:0/3516292821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0480066260 0x7f048010da50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f046800c8b0 tx=0x7f0468004470 comp rx=0 tx=0).stop 2024-03-22T06:12:28.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.426+0000 7f0486316700 1 -- 172.21.15.6:0/3516292821 shutdown_connections 2024-03-22T06:12:28.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.426+0000 7f0486316700 1 --2- 172.21.15.6:0/3516292821 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f046c0692a0 0x7f046c06b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:28.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.426+0000 7f0486316700 1 --2- 172.21.15.6:0/3516292821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0480066260 0x7f048010da50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:28.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.426+0000 7f0486316700 1 --2- 172.21.15.6:0/3516292821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04800fff80 0x7f048010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:28.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.426+0000 7f0486316700 1 -- 172.21.15.6:0/3516292821 >> 172.21.15.6:0/3516292821 conn(0x7f04800f7430 msgr2=0x7f04800f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:28.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.426+0000 7f0486316700 1 -- 172.21.15.6:0/3516292821 shutdown_connections 2024-03-22T06:12:28.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:28.426+0000 7f0486316700 1 -- 172.21.15.6:0/3516292821 wait complete. 2024-03-22T06:12:28.666 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:28 smithi006 ceph-mon[29301]: Deploying daemon mds.cephfs.smithi006.dcjpla on smithi006 2024-03-22T06:12:28.666 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:28 smithi006 ceph-mon[29301]: mds.? [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] up:boot 2024-03-22T06:12:28.666 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:28 smithi006 ceph-mon[29301]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 1 up:standby 2024-03-22T06:12:28.666 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:28 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:12:28.666 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:28 smithi006 ceph-mon[29301]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 1 up:standby 2024-03-22T06:12:28.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:28 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3516292821' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-22T06:12:28.678 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:28 smithi117 ceph-mon[36514]: Deploying daemon mds.cephfs.smithi006.dcjpla on smithi006 2024-03-22T06:12:28.678 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:28 smithi117 ceph-mon[36514]: mds.? [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] up:boot 2024-03-22T06:12:28.678 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:28 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 1 up:standby 2024-03-22T06:12:28.678 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:28 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:12:28.678 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:28 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 1 up:standby 2024-03-22T06:12:28.678 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:28 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3516292821' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-22T06:12:29.377 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T06:12:29.386 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:12:29.386 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-03-22T06:12:29.711 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:29 smithi006 ceph-mon[29301]: pgmap v182: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 658 B/s wr, 2 op/s 2024-03-22T06:12:29.711 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:29 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3516292821' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-22T06:12:29.712 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:29 smithi006 ceph-mon[29301]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 1 up:standby 2024-03-22T06:12:29.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:29 smithi117 ceph-mon[36514]: pgmap v182: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 658 B/s wr, 2 op/s 2024-03-22T06:12:29.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:29 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3516292821' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-22T06:12:29.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:29 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 1 up:standby 2024-03-22T06:12:30.008 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:31.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:31 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:31.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:31 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:31.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:31 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:31.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:31 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.zqxyql", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:12:31.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:31 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.zqxyql", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T06:12:31.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:31 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:31.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:31 smithi117 ceph-mon[36514]: Deploying daemon mds.cephfs.smithi117.zqxyql on smithi117 2024-03-22T06:12:31.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:31 smithi117 ceph-mon[36514]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 3 op/s 2024-03-22T06:12:31.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:31 smithi117 ceph-mon[36514]: mds.? [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] up:boot 2024-03-22T06:12:31.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:31 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 2 up:standby 2024-03-22T06:12:31.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:31 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:12:31.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:31 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:31.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:31 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:31.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:31 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:31.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:31 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.zqxyql", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:12:31.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:31 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.zqxyql", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T06:12:31.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:31 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:31.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:31 smithi006 ceph-mon[29301]: Deploying daemon mds.cephfs.smithi117.zqxyql on smithi117 2024-03-22T06:12:31.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:31 smithi006 ceph-mon[29301]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 3 op/s 2024-03-22T06:12:31.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:31 smithi006 ceph-mon[29301]: mds.? [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] up:boot 2024-03-22T06:12:31.265 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:31 smithi006 ceph-mon[29301]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 2 up:standby 2024-03-22T06:12:31.265 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:31 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:12:31.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.554+0000 7fbd17d0f700 1 -- 172.21.15.6:0/451562330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd100fbd00 msgr2=0x7fbd100fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:31.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.554+0000 7fbd17d0f700 1 --2- 172.21.15.6:0/451562330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd100fbd00 0x7fbd100fc110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbd08004940 tx=0x7fbd08009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:31.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.554+0000 7fbd17d0f700 1 -- 172.21.15.6:0/451562330 shutdown_connections 2024-03-22T06:12:31.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.554+0000 7fbd17d0f700 1 --2- 172.21.15.6:0/451562330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd100fcfb0 0x7fbd100fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:31.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.554+0000 7fbd17d0f700 1 --2- 172.21.15.6:0/451562330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd100fbd00 0x7fbd100fc110 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:31.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.554+0000 7fbd17d0f700 1 -- 172.21.15.6:0/451562330 >> 172.21.15.6:0/451562330 conn(0x7fbd100f7480 msgr2=0x7fbd100f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:31.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.554+0000 7fbd17d0f700 1 -- 172.21.15.6:0/451562330 shutdown_connections 2024-03-22T06:12:31.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.554+0000 7fbd17d0f700 1 -- 172.21.15.6:0/451562330 wait complete. 2024-03-22T06:12:31.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.554+0000 7fbd17d0f700 1 Processor -- start 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.554+0000 7fbd17d0f700 1 -- start start 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd17d0f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd100fbd00 0x7fbd100ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd17d0f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd100fcfb0 0x7fbd100fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd17d0f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd10100240 con 0x7fbd100fbd00 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd17d0f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100fe830 con 0x7fbd100fcfb0 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd15aab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd100fbd00 0x7fbd100ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd15aab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd100fbd00 0x7fbd100ffc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33186/0 (socket says 172.21.15.6:33186) 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd15aab700 1 -- 172.21.15.6:0/3257110113 learned_addr learned my addr 172.21.15.6:0/3257110113 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd152aa700 1 --2- 172.21.15.6:0/3257110113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd100fcfb0 0x7fbd100fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd15aab700 1 -- 172.21.15.6:0/3257110113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd100fcfb0 msgr2=0x7fbd100fe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd15aab700 1 --2- 172.21.15.6:0/3257110113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd100fcfb0 0x7fbd100fe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd15aab700 1 -- 172.21.15.6:0/3257110113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd08005040 con 0x7fbd100fbd00 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd15aab700 1 --2- 172.21.15.6:0/3257110113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd100fbd00 0x7fbd100ffc40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009150 tx=0x7fbd08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:31.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd0e7fc700 1 -- 172.21.15.6:0/3257110113 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd0800e040 con 0x7fbd100fbd00 2024-03-22T06:12:31.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd0e7fc700 1 -- 172.21.15.6:0/3257110113 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd08004290 con 0x7fbd100fbd00 2024-03-22T06:12:31.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd0e7fc700 1 -- 172.21.15.6:0/3257110113 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd0801c630 con 0x7fbd100fbd00 2024-03-22T06:12:31.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd17d0f700 1 -- 172.21.15.6:0/3257110113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd100fea80 con 0x7fbd100fbd00 2024-03-22T06:12:31.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.555+0000 7fbd17d0f700 1 -- 172.21.15.6:0/3257110113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd100ff040 con 0x7fbd100fbd00 2024-03-22T06:12:31.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.556+0000 7fbd17d0f700 1 -- 172.21.15.6:0/3257110113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd10101b40 con 0x7fbd100fbd00 2024-03-22T06:12:31.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.557+0000 7fbd0e7fc700 1 -- 172.21.15.6:0/3257110113 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fbd0800bbf0 con 0x7fbd100fbd00 2024-03-22T06:12:31.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.557+0000 7fbd0e7fc700 1 --2- 172.21.15.6:0/3257110113 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbcf80692a0 0x7fbcf806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:31.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.557+0000 7fbd0e7fc700 1 -- 172.21.15.6:0/3257110113 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fbd08087e50 con 0x7fbd100fbd00 2024-03-22T06:12:31.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.557+0000 7fbd152aa700 1 --2- 172.21.15.6:0/3257110113 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbcf80692a0 0x7fbcf806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:31.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.557+0000 7fbd152aa700 1 --2- 172.21.15.6:0/3257110113 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbcf80692a0 0x7fbcf806b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbd04004f40 tx=0x7fbd0400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:31.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.562+0000 7fbd0e7fc700 1 -- 172.21.15.6:0/3257110113 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbd08057c10 con 0x7fbd100fbd00 2024-03-22T06:12:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:31.804+0000 7fbd17d0f700 1 -- 172.21.15.6:0/3257110113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbd1004c710 con 0x7fbd100fbd00 2024-03-22T06:12:32.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.066+0000 7fbd0e7fc700 1 -- 172.21.15.6:0/3257110113 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbd080577a0 con 0x7fbd100fbd00 2024-03-22T06:12:32.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.068+0000 7fbd17d0f700 1 -- 172.21.15.6:0/3257110113 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbcf80692a0 msgr2=0x7fbcf806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:32.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.068+0000 7fbd17d0f700 1 --2- 172.21.15.6:0/3257110113 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbcf80692a0 0x7fbcf806b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbd04004f40 tx=0x7fbd0400b040 comp rx=0 tx=0).stop 2024-03-22T06:12:32.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.068+0000 7fbd17d0f700 1 -- 172.21.15.6:0/3257110113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd100fbd00 msgr2=0x7fbd100ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:32.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.068+0000 7fbd17d0f700 1 --2- 172.21.15.6:0/3257110113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd100fbd00 0x7fbd100ffc40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009150 tx=0x7fbd08009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:32.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.069+0000 7fbd17d0f700 1 -- 172.21.15.6:0/3257110113 shutdown_connections 2024-03-22T06:12:32.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.069+0000 7fbd17d0f700 1 --2- 172.21.15.6:0/3257110113 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fbcf80692a0 0x7fbcf806b750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:32.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.069+0000 7fbd17d0f700 1 --2- 172.21.15.6:0/3257110113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd100fbd00 0x7fbd100ffc40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:32.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.069+0000 7fbd17d0f700 1 --2- 172.21.15.6:0/3257110113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd100fcfb0 0x7fbd100fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:32.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.069+0000 7fbd17d0f700 1 -- 172.21.15.6:0/3257110113 >> 172.21.15.6:0/3257110113 conn(0x7fbd100f7480 msgr2=0x7fbd10066860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:32.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.069+0000 7fbd17d0f700 1 -- 172.21.15.6:0/3257110113 shutdown_connections 2024-03-22T06:12:32.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:32.069+0000 7fbd17d0f700 1 -- 172.21.15.6:0/3257110113 wait complete. 2024-03-22T06:12:32.187 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:32 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3257110113' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-22T06:12:32.438 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:32 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3257110113' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-22T06:12:32.771 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T06:12:32.781 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:12:32.782 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-03-22T06:12:33.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:33 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3257110113' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-22T06:12:33.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:33 smithi006 ceph-mon[29301]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 2 up:standby 2024-03-22T06:12:33.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:33 smithi006 ceph-mon[29301]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-22T06:12:33.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:33 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:12:33.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:33 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:33 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:33 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:33 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:33 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:33 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:33 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:12:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:33 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3257110113' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-22T06:12:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:33 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 2 up:standby 2024-03-22T06:12:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:33 smithi117 ceph-mon[36514]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-22T06:12:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:33 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:12:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:33 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:33 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:33 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:33 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:33 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:33 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:33.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:33 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:12:33.592 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:34.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:34 smithi117 ceph-mon[36514]: mds.? [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] up:boot 2024-03-22T06:12:34.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:34 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:12:34.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:34 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:12:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:34 smithi006 ceph-mon[29301]: mds.? [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] up:boot 2024-03-22T06:12:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:34 smithi006 ceph-mon[29301]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:12:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:34 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:12:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:35 smithi117 ceph-mon[36514]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-22T06:12:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:35 smithi117 ceph-mon[36514]: mds.? [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] up:active 2024-03-22T06:12:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:35 smithi117 ceph-mon[36514]: mds.? [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] up:standby 2024-03-22T06:12:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:35 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:12:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:35 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:35.510 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:35 smithi006 ceph-mon[29301]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-22T06:12:35.510 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:35 smithi006 ceph-mon[29301]: mds.? [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] up:active 2024-03-22T06:12:35.510 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:35 smithi006 ceph-mon[29301]: mds.? [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] up:standby 2024-03-22T06:12:35.510 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:35 smithi006 ceph-mon[29301]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:12:35.510 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:35.510 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:35 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.841+0000 7f98418c1700 1 -- 172.21.15.6:0/1127623732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f983c0fed80 msgr2=0x7f983c0faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.841+0000 7f98418c1700 1 --2- 172.21.15.6:0/1127623732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f983c0fed80 0x7f983c0faec0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9830004940 tx=0x7f9830009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.841+0000 7f98418c1700 1 -- 172.21.15.6:0/1127623732 shutdown_connections 2024-03-22T06:12:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.841+0000 7f98418c1700 1 --2- 172.21.15.6:0/1127623732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f983c0fed80 0x7f983c0faec0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.841+0000 7f98418c1700 1 --2- 172.21.15.6:0/1127623732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0fe430 0x7f983c0fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.841+0000 7f98418c1700 1 -- 172.21.15.6:0/1127623732 >> 172.21.15.6:0/1127623732 conn(0x7f983c0f6bd0 msgr2=0x7f983c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.841+0000 7f98418c1700 1 -- 172.21.15.6:0/1127623732 shutdown_connections 2024-03-22T06:12:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.841+0000 7f98418c1700 1 -- 172.21.15.6:0/1127623732 wait complete. 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.842+0000 7f98418c1700 1 Processor -- start 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.842+0000 7f98418c1700 1 -- start start 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.842+0000 7f98418c1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0fe430 0x7f983c10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.842+0000 7f98418c1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f983c0fed80 0x7f983c10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.842+0000 7f98418c1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c10e500 con 0x7f983c0fed80 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.842+0000 7f98418c1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c112ea0 con 0x7f983c0fe430 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.842+0000 7f983a7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f983c0fed80 0x7f983c10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.842+0000 7f983a7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f983c0fed80 0x7f983c10de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52370/0 (socket says 172.21.15.6:52370) 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.842+0000 7f983a7fc700 1 -- 172.21.15.6:0/1763975616 learned_addr learned my addr 172.21.15.6:0/1763975616 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.843+0000 7f983affd700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0fe430 0x7f983c10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.843+0000 7f983a7fc700 1 -- 172.21.15.6:0/1763975616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0fe430 msgr2=0x7f983c10d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.843+0000 7f983a7fc700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0fe430 0x7f983c10d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.843+0000 7f983a7fc700 1 -- 172.21.15.6:0/1763975616 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830005040 con 0x7f983c0fed80 2024-03-22T06:12:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.843+0000 7f983affd700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0fe430 0x7f983c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:12:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.843+0000 7f983a7fc700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f983c0fed80 0x7f983c10de70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f983000b3b0 tx=0x7f9830009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.843+0000 7f98237fe700 1 -- 172.21.15.6:0/1763975616 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f983000e040 con 0x7f983c0fed80 2024-03-22T06:12:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.843+0000 7f98237fe700 1 -- 172.21.15.6:0/1763975616 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f983000bcc0 con 0x7f983c0fed80 2024-03-22T06:12:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.843+0000 7f98418c1700 1 -- 172.21.15.6:0/1763975616 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983c1130a0 con 0x7f983c0fed80 2024-03-22T06:12:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.843+0000 7f98237fe700 1 -- 172.21.15.6:0/1763975616 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f983001c630 con 0x7f983c0fed80 2024-03-22T06:12:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.843+0000 7f98418c1700 1 -- 172.21.15.6:0/1763975616 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983c1135c0 con 0x7f983c0fed80 2024-03-22T06:12:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.844+0000 7f98418c1700 1 -- 172.21.15.6:0/1763975616 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983c04c710 con 0x7f983c0fed80 2024-03-22T06:12:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.845+0000 7f98237fe700 1 -- 172.21.15.6:0/1763975616 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f9830013980 con 0x7f983c0fed80 2024-03-22T06:12:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.845+0000 7f98237fe700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f98240692a0 0x7f982406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.845+0000 7f98237fe700 1 -- 172.21.15.6:0/1763975616 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9830087fb0 con 0x7f983c0fed80 2024-03-22T06:12:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.845+0000 7f983affd700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f98240692a0 0x7f982406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.846+0000 7f983affd700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f98240692a0 0x7f982406b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f982c009b10 tx=0x7f982c008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:35.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:35.850+0000 7f98237fe700 1 -- 172.21.15.6:0/1763975616 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9830057cf0 con 0x7f983c0fed80 2024-03-22T06:12:36.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.106+0000 7f98418c1700 1 -- 172.21.15.6:0/1763975616 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f983c105960 con 0x7f983c0fed80 2024-03-22T06:12:36.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.523+0000 7f98237fe700 1 -- 172.21.15.6:0/1763975616 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v14) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9830057880 con 0x7f983c0fed80 2024-03-22T06:12:36.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.525+0000 7f98418c1700 1 -- 172.21.15.6:0/1763975616 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f98240692a0 msgr2=0x7f982406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:36.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.525+0000 7f98418c1700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f98240692a0 0x7f982406b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f982c009b10 tx=0x7f982c008040 comp rx=0 tx=0).stop 2024-03-22T06:12:36.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.525+0000 7f98418c1700 1 -- 172.21.15.6:0/1763975616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f983c0fed80 msgr2=0x7f983c10de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:36.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.525+0000 7f98418c1700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f983c0fed80 0x7f983c10de70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f983000b3b0 tx=0x7f9830009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.525+0000 7f98418c1700 1 -- 172.21.15.6:0/1763975616 shutdown_connections 2024-03-22T06:12:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.525+0000 7f98418c1700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0fe430 0x7f983c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.525+0000 7f98418c1700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f98240692a0 0x7f982406b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.525+0000 7f98418c1700 1 --2- 172.21.15.6:0/1763975616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f983c0fed80 0x7f983c10de70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.525+0000 7f98418c1700 1 -- 172.21.15.6:0/1763975616 >> 172.21.15.6:0/1763975616 conn(0x7f983c0f6bd0 msgr2=0x7f983c0f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.525+0000 7f98418c1700 1 -- 172.21.15.6:0/1763975616 shutdown_connections 2024-03-22T06:12:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:36.525+0000 7f98418c1700 1 -- 172.21.15.6:0/1763975616 wait complete. 2024-03-22T06:12:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:inline data disabled 2024-03-22T06:12:36.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:36 smithi006 ceph-mon[29301]: mds.? [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] up:standby 2024-03-22T06:12:36.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:36 smithi006 ceph-mon[29301]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:12:36.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:36 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1763975616' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-22T06:12:36.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:36 smithi006 ceph-mon[29301]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-22T06:12:37.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:36 smithi117 ceph-mon[36514]: mds.? [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] up:standby 2024-03-22T06:12:37.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:36 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:12:37.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:36 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1763975616' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-22T06:12:37.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:36 smithi117 ceph-mon[36514]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-22T06:12:37.196 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T06:12:37.207 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:12:37.207 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T06:12:37.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:37 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1763975616' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-22T06:12:37.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:37 smithi006 ceph-mon[29301]: mds.? [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] up:standby 2024-03-22T06:12:37.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:37 smithi006 ceph-mon[29301]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:12:37.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:37 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:37.910 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:38.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:37 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1763975616' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-22T06:12:38.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:37 smithi117 ceph-mon[36514]: mds.? [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] up:standby 2024-03-22T06:12:38.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:37 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:12:38.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:37 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:38.764 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:38 smithi006 ceph-mon[29301]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 3 op/s 2024-03-22T06:12:39.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:38 smithi117 ceph-mon[36514]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 3 op/s 2024-03-22T06:12:40.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.305+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/417346302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bb40fcf80 msgr2=0x7f6bb40fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.305+0000 7f6bbbbbe700 1 --2- 172.21.15.6:0/417346302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bb40fcf80 0x7f6bb40fd3f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0004940 tx=0x7f6bb0009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.305+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/417346302 shutdown_connections 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.305+0000 7f6bbbbbe700 1 --2- 172.21.15.6:0/417346302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bb40fcf80 0x7f6bb40fd3f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.305+0000 7f6bbbbbe700 1 --2- 172.21.15.6:0/417346302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bb40fbcd0 0x7f6bb40fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.305+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/417346302 >> 172.21.15.6:0/417346302 conn(0x7f6bb40f7430 msgr2=0x7f6bb40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.305+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/417346302 shutdown_connections 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.305+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/417346302 wait complete. 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bbbbbe700 1 Processor -- start 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bbbbbe700 1 -- start start 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bbbbbe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bb40fbcd0 0x7f6bb4103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bbbbbe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bb40fcf80 0x7f6bb41022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bbbbbe700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb4104230 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bbbbbe700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb4102820 con 0x7f6bb40fcf80 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bb995a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bb40fbcd0 0x7f6bb4103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bb995a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bb40fbcd0 0x7f6bb4103c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52388/0 (socket says 172.21.15.6:52388) 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bb995a700 1 -- 172.21.15.6:0/882918855 learned_addr learned my addr 172.21.15.6:0/882918855 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bb9159700 1 --2- 172.21.15.6:0/882918855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bb40fcf80 0x7f6bb41022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bb995a700 1 -- 172.21.15.6:0/882918855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bb40fcf80 msgr2=0x7f6bb41022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bb995a700 1 --2- 172.21.15.6:0/882918855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bb40fcf80 0x7f6bb41022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.306+0000 7f6bb995a700 1 -- 172.21.15.6:0/882918855 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb0005040 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.307+0000 7f6bb995a700 1 --2- 172.21.15.6:0/882918855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bb40fbcd0 0x7f6bb4103c30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400eb40 tx=0x7f6ba400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.307+0000 7f6baa7fc700 1 -- 172.21.15.6:0/882918855 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ba4007cc0 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.307+0000 7f6baa7fc700 1 -- 172.21.15.6:0/882918855 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ba400f040 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.307+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/882918855 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb4102a70 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.307+0000 7f6baa7fc700 1 -- 172.21.15.6:0/882918855 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ba400ca10 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.307+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/882918855 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb4102fb0 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.308+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/882918855 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb404c710 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.308+0000 7f6baa7fc700 1 -- 172.21.15.6:0/882918855 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f6ba400cc80 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.309+0000 7f6baa7fc700 1 --2- 172.21.15.6:0/882918855 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ba00692a0 0x7f6ba006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:40.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.309+0000 7f6baa7fc700 1 -- 172.21.15.6:0/882918855 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6ba4086d80 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.309+0000 7f6bb9159700 1 --2- 172.21.15.6:0/882918855 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ba00692a0 0x7f6ba006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:40.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.309+0000 7f6bb9159700 1 --2- 172.21.15.6:0/882918855 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ba00692a0 0x7f6ba006b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009b30 tx=0x7f6bb0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:40.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.313+0000 7f6baa7fc700 1 -- 172.21.15.6:0/882918855 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6ba404e190 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.569+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/882918855 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6bb4115ad0 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.570+0000 7f6baa7fc700 1 -- 172.21.15.6:0/882918855 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1805 (secure 0 0 0) 0x7f6ba4056b70 con 0x7f6bb40fbcd0 2024-03-22T06:12:40.570 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:12:40.570 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:12:40.570 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:12:40.571 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:12:40.572 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:12:40.573 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:12:40.573 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:12:40.573 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:12:40.573 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:12:40.573 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:12:40.573 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.572+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/882918855 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ba00692a0 msgr2=0x7f6ba006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.572+0000 7f6bbbbbe700 1 --2- 172.21.15.6:0/882918855 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ba00692a0 0x7f6ba006b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009b30 tx=0x7f6bb0009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.572+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/882918855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bb40fbcd0 msgr2=0x7f6bb4103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.572+0000 7f6bbbbbe700 1 --2- 172.21.15.6:0/882918855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bb40fbcd0 0x7f6bb4103c30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400eb40 tx=0x7f6ba400b040 comp rx=0 tx=0).stop 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.572+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/882918855 shutdown_connections 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.572+0000 7f6bbbbbe700 1 --2- 172.21.15.6:0/882918855 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f6ba00692a0 0x7f6ba006b750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.572+0000 7f6bbbbbe700 1 --2- 172.21.15.6:0/882918855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bb40fbcd0 0x7f6bb4103c30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.572+0000 7f6bbbbbe700 1 --2- 172.21.15.6:0/882918855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bb40fcf80 0x7f6bb41022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.572+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/882918855 >> 172.21.15.6:0/882918855 conn(0x7f6bb40f7430 msgr2=0x7f6bb40f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.572+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/882918855 shutdown_connections 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:40.573+0000 7f6bbbbbe700 1 -- 172.21.15.6:0/882918855 wait complete. 2024-03-22T06:12:40.574 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:12:40.680 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:40 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:40.680 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:40 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:40.680 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:40 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:40.680 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:40 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:12:40.681 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:40 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:40.681 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:40 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:12:40.689 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:40 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:40.689 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:40 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:40.689 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:40 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:40.689 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:40 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:12:40.689 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:40 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:40.689 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:40 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:12:41.732 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:41 smithi006 ceph-mon[29301]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 2 op/s 2024-03-22T06:12:41.732 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:41 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/882918855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:12:41.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:41 smithi117 ceph-mon[36514]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 2 op/s 2024-03-22T06:12:41.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:41 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/882918855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:12:41.761 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-22T06:12:42.668 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:42.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:42 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:42.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:42 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:42.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:42 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:42.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:42 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:43.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:43 smithi117 ceph-mon[36514]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 1 op/s 2024-03-22T06:12:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:43 smithi006 ceph-mon[29301]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 1 op/s 2024-03-22T06:12:45.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:45 smithi006 ceph-mon[29301]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:45.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:45 smithi117 ceph-mon[36514]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:45.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.832+0000 7f063a817700 1 -- 172.21.15.6:0/4273708054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06340fbd10 msgr2=0x7f06340fc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:45.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.832+0000 7f063a817700 1 --2- 172.21.15.6:0/4273708054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06340fbd10 0x7f06340fc120 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0624004940 tx=0x7f0624009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:45.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.832+0000 7f063a817700 1 -- 172.21.15.6:0/4273708054 shutdown_connections 2024-03-22T06:12:45.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.832+0000 7f063a817700 1 --2- 172.21.15.6:0/4273708054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06340fcfc0 0x7f06340fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:45.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.832+0000 7f063a817700 1 --2- 172.21.15.6:0/4273708054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06340fbd10 0x7f06340fc120 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:45.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.832+0000 7f063a817700 1 -- 172.21.15.6:0/4273708054 >> 172.21.15.6:0/4273708054 conn(0x7f06340f7490 msgr2=0x7f06340f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:45.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.832+0000 7f063a817700 1 -- 172.21.15.6:0/4273708054 shutdown_connections 2024-03-22T06:12:45.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.832+0000 7f063a817700 1 -- 172.21.15.6:0/4273708054 wait complete. 2024-03-22T06:12:45.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f063a817700 1 Processor -- start 2024-03-22T06:12:45.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f063a817700 1 -- start start 2024-03-22T06:12:45.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f063a817700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06340fbd10 0x7f063410c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:45.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f063a817700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06340fcfc0 0x7f063410cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:45.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f063a817700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063410d370 con 0x7f06340fcfc0 2024-03-22T06:12:45.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f063a817700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063410d4e0 con 0x7f06340fbd10 2024-03-22T06:12:45.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f06337fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06340fcfc0 0x7f063410cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:45.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f06337fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06340fcfc0 0x7f063410cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41224/0 (socket says 172.21.15.6:41224) 2024-03-22T06:12:45.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f06337fe700 1 -- 172.21.15.6:0/2191492699 learned_addr learned my addr 172.21.15.6:0/2191492699 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:45.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f0633fff700 1 --2- 172.21.15.6:0/2191492699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06340fbd10 0x7f063410c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:45.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f06337fe700 1 -- 172.21.15.6:0/2191492699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06340fbd10 msgr2=0x7f063410c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:45.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f06337fe700 1 --2- 172.21.15.6:0/2191492699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06340fbd10 0x7f063410c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:45.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.833+0000 7f06337fe700 1 -- 172.21.15.6:0/2191492699 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0624005040 con 0x7f06340fcfc0 2024-03-22T06:12:45.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.834+0000 7f06337fe700 1 --2- 172.21.15.6:0/2191492699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06340fcfc0 0x7f063410cd70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f061c00c830 tx=0x7f061c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:45.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.834+0000 7f0630ff9700 1 -- 172.21.15.6:0/2191492699 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f061c0106f0 con 0x7f06340fcfc0 2024-03-22T06:12:45.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.834+0000 7f063a817700 1 -- 172.21.15.6:0/2191492699 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0634111f40 con 0x7f06340fcfc0 2024-03-22T06:12:45.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.834+0000 7f0630ff9700 1 -- 172.21.15.6:0/2191492699 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f061c010d20 con 0x7f06340fcfc0 2024-03-22T06:12:45.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.834+0000 7f0630ff9700 1 -- 172.21.15.6:0/2191492699 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f061c0113a0 con 0x7f06340fcfc0 2024-03-22T06:12:45.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.834+0000 7f063a817700 1 -- 172.21.15.6:0/2191492699 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06341124d0 con 0x7f06340fcfc0 2024-03-22T06:12:45.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.835+0000 7f063a817700 1 -- 172.21.15.6:0/2191492699 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063404c710 con 0x7f06340fcfc0 2024-03-22T06:12:45.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.835+0000 7f0630ff9700 1 -- 172.21.15.6:0/2191492699 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f061c012400 con 0x7f06340fcfc0 2024-03-22T06:12:45.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.836+0000 7f0630ff9700 1 --2- 172.21.15.6:0/2191492699 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06200692a0 0x7f062006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:45.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.836+0000 7f0633fff700 1 --2- 172.21.15.6:0/2191492699 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06200692a0 0x7f062006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:45.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.836+0000 7f0630ff9700 1 -- 172.21.15.6:0/2191492699 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f061c087150 con 0x7f06340fcfc0 2024-03-22T06:12:45.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.836+0000 7f0633fff700 1 --2- 172.21.15.6:0/2191492699 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06200692a0 0x7f062006b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f062400b970 tx=0x7f0624009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:45.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:45.840+0000 7f0630ff9700 1 -- 172.21.15.6:0/2191492699 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f061c056f10 con 0x7f06340fcfc0 2024-03-22T06:12:46.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.082+0000 7f063a817700 1 -- 172.21.15.6:0/2191492699 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0634066270 con 0x7f06340fcfc0 2024-03-22T06:12:46.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.083+0000 7f0630ff9700 1 -- 172.21.15.6:0/2191492699 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4683 (secure 0 0 0) 0x7f061c056aa0 con 0x7f06340fcfc0 2024-03-22T06:12:46.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.085+0000 7f063a817700 1 -- 172.21.15.6:0/2191492699 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06200692a0 msgr2=0x7f062006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:46.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.085+0000 7f063a817700 1 --2- 172.21.15.6:0/2191492699 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06200692a0 0x7f062006b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f062400b970 tx=0x7f0624009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:46.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.085+0000 7f063a817700 1 -- 172.21.15.6:0/2191492699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06340fcfc0 msgr2=0x7f063410cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:46.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.085+0000 7f063a817700 1 --2- 172.21.15.6:0/2191492699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06340fcfc0 0x7f063410cd70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f061c00c830 tx=0x7f061c004470 comp rx=0 tx=0).stop 2024-03-22T06:12:46.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.085+0000 7f063a817700 1 -- 172.21.15.6:0/2191492699 shutdown_connections 2024-03-22T06:12:46.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.085+0000 7f063a817700 1 --2- 172.21.15.6:0/2191492699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06340fbd10 0x7f063410c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:46.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.085+0000 7f063a817700 1 --2- 172.21.15.6:0/2191492699 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f06200692a0 0x7f062006b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:46.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.085+0000 7f063a817700 1 --2- 172.21.15.6:0/2191492699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06340fcfc0 0x7f063410cd70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:46.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.085+0000 7f063a817700 1 -- 172.21.15.6:0/2191492699 >> 172.21.15.6:0/2191492699 conn(0x7f06340f7490 msgr2=0x7f06340f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:46.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.085+0000 7f063a817700 1 -- 172.21.15.6:0/2191492699 shutdown_connections 2024-03-22T06:12:46.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:46.085+0000 7f063a817700 1 -- 172.21.15.6:0/2191492699 wait complete. 2024-03-22T06:12:46.085 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:12:46.094 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:12:46.745 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:46 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2191492699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T06:12:46.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:46 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2191492699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T06:12:46.769 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-22T06:12:47.056 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:47 smithi117 ceph-mon[36514]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:12:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:12:47.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:47 smithi006 ceph-mon[29301]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:47.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:47.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:47.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:12:47.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:12:47.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:12:47.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:12:48.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.750+0000 7f53a08fb700 1 -- 172.21.15.6:0/954137935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f539c0fcef0 msgr2=0x7f539c0fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:48.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.750+0000 7f53a08fb700 1 --2- 172.21.15.6:0/954137935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f539c0fcef0 0x7f539c0fd300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5394009ce0 tx=0x7f5394004d10 comp rx=0 tx=0).stop 2024-03-22T06:12:48.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.750+0000 7f53a08fb700 1 -- 172.21.15.6:0/954137935 shutdown_connections 2024-03-22T06:12:48.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.750+0000 7f53a08fb700 1 --2- 172.21.15.6:0/954137935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f539c0fe060 0x7f539c0fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:48.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.750+0000 7f53a08fb700 1 --2- 172.21.15.6:0/954137935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f539c0fcef0 0x7f539c0fd300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:48.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.750+0000 7f53a08fb700 1 -- 172.21.15.6:0/954137935 >> 172.21.15.6:0/954137935 conn(0x7f539c0f85b0 msgr2=0x7f539c0fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:48.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.750+0000 7f53a08fb700 1 -- 172.21.15.6:0/954137935 shutdown_connections 2024-03-22T06:12:48.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.750+0000 7f53a08fb700 1 -- 172.21.15.6:0/954137935 wait complete. 2024-03-22T06:12:48.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f53a08fb700 1 Processor -- start 2024-03-22T06:12:48.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f53a08fb700 1 -- start start 2024-03-22T06:12:48.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f53a08fb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f539c0fcef0 0x7f539c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:48.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f53a08fb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f539c0fe060 0x7f539c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:48.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f53a08fb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c0708a0 con 0x7f539c0fcef0 2024-03-22T06:12:48.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f53a08fb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c0709e0 con 0x7f539c0fe060 2024-03-22T06:12:48.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f539a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f539c0fcef0 0x7f539c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:48.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f539a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f539c0fcef0 0x7f539c071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41242/0 (socket says 172.21.15.6:41242) 2024-03-22T06:12:48.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f539a59c700 1 -- 172.21.15.6:0/2755152867 learned_addr learned my addr 172.21.15.6:0/2755152867 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:48.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f5399d9b700 1 --2- 172.21.15.6:0/2755152867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f539c0fe060 0x7f539c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:48.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f539a59c700 1 -- 172.21.15.6:0/2755152867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f539c0fe060 msgr2=0x7f539c070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:48.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f539a59c700 1 --2- 172.21.15.6:0/2755152867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f539c0fe060 0x7f539c070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:48.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f539a59c700 1 -- 172.21.15.6:0/2755152867 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5394009940 con 0x7f539c0fcef0 2024-03-22T06:12:48.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.751+0000 7f539a59c700 1 --2- 172.21.15.6:0/2755152867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f539c0fcef0 0x7f539c071df0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5394009380 tx=0x7f5394004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:48.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.752+0000 7f5392ffd700 1 -- 172.21.15.6:0/2755152867 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f539400e040 con 0x7f539c0fcef0 2024-03-22T06:12:48.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.752+0000 7f5392ffd700 1 -- 172.21.15.6:0/2755152867 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53940128e0 con 0x7f539c0fcef0 2024-03-22T06:12:48.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.752+0000 7f53a08fb700 1 -- 172.21.15.6:0/2755152867 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539c070c60 con 0x7f539c0fcef0 2024-03-22T06:12:48.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.752+0000 7f5392ffd700 1 -- 172.21.15.6:0/2755152867 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f539401c930 con 0x7f539c0fcef0 2024-03-22T06:12:48.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.752+0000 7f53a08fb700 1 -- 172.21.15.6:0/2755152867 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c115c20 con 0x7f539c0fcef0 2024-03-22T06:12:48.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.753+0000 7f53a08fb700 1 -- 172.21.15.6:0/2755152867 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539c05f5f0 con 0x7f539c0fcef0 2024-03-22T06:12:48.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.753+0000 7f5392ffd700 1 -- 172.21.15.6:0/2755152867 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f5394012a50 con 0x7f539c0fcef0 2024-03-22T06:12:48.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.753+0000 7f5392ffd700 1 --2- 172.21.15.6:0/2755152867 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f53880692a0 0x7f538806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:48.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.754+0000 7f5392ffd700 1 -- 172.21.15.6:0/2755152867 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5394088c30 con 0x7f539c0fcef0 2024-03-22T06:12:48.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.754+0000 7f5399d9b700 1 --2- 172.21.15.6:0/2755152867 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f53880692a0 0x7f538806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:48.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.754+0000 7f5399d9b700 1 --2- 172.21.15.6:0/2755152867 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f53880692a0 0x7f538806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f539c0723d0 tx=0x7f538400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:48.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:48.758+0000 7f5392ffd700 1 -- 172.21.15.6:0/2755152867 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5394054f60 con 0x7f539c0fcef0 2024-03-22T06:12:49.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.036+0000 7f53a08fb700 1 -- 172.21.15.6:0/2755152867 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f539c04bed0 con 0x7f539c0fcef0 2024-03-22T06:12:49.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.036+0000 7f5392ffd700 1 -- 172.21.15.6:0/2755152867 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v14) v1 ==== 78+0+99 (secure 0 0 0) 0x7f5394012d00 con 0x7f539c0fcef0 2024-03-22T06:12:49.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.038+0000 7f53a08fb700 1 -- 172.21.15.6:0/2755152867 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f53880692a0 msgr2=0x7f538806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:49.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.038+0000 7f53a08fb700 1 --2- 172.21.15.6:0/2755152867 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f53880692a0 0x7f538806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f539c0723d0 tx=0x7f538400b040 comp rx=0 tx=0).stop 2024-03-22T06:12:49.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.038+0000 7f53a08fb700 1 -- 172.21.15.6:0/2755152867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f539c0fcef0 msgr2=0x7f539c071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:49.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.038+0000 7f53a08fb700 1 --2- 172.21.15.6:0/2755152867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f539c0fcef0 0x7f539c071df0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5394009380 tx=0x7f5394004d10 comp rx=0 tx=0).stop 2024-03-22T06:12:49.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.038+0000 7f53a08fb700 1 -- 172.21.15.6:0/2755152867 shutdown_connections 2024-03-22T06:12:49.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.038+0000 7f53a08fb700 1 --2- 172.21.15.6:0/2755152867 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f53880692a0 0x7f538806b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:49.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.038+0000 7f53a08fb700 1 --2- 172.21.15.6:0/2755152867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f539c0fcef0 0x7f539c071df0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:49.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.038+0000 7f53a08fb700 1 --2- 172.21.15.6:0/2755152867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f539c0fe060 0x7f539c070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:49.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.038+0000 7f53a08fb700 1 -- 172.21.15.6:0/2755152867 >> 172.21.15.6:0/2755152867 conn(0x7f539c0f85b0 msgr2=0x7f539c0f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:49.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.038+0000 7f53a08fb700 1 -- 172.21.15.6:0/2755152867 shutdown_connections 2024-03-22T06:12:49.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.038+0000 7f53a08fb700 1 -- 172.21.15.6:0/2755152867 wait complete. 2024-03-22T06:12:49.049 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:12:49.627 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:49 smithi006 ceph-mon[29301]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:49.627 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:49 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2755152867' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-22T06:12:49.655 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-22T06:12:49.675 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T06:12:49.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:49 smithi117 ceph-mon[36514]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:49.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:49 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2755152867' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-22T06:12:49.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.802+0000 7f032635d640 1 -- 172.21.15.6:0/3952443946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03200fc380 msgr2=0x7f03200fc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:49.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.802+0000 7f032635d640 1 --2- 172.21.15.6:0/3952443946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03200fc380 0x7f03200fc7f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f03140048d0 tx=0x7f0314009da0 comp rx=0 tx=0).stop 2024-03-22T06:12:49.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.802+0000 7f032635d640 1 -- 172.21.15.6:0/3952443946 shutdown_connections 2024-03-22T06:12:49.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.802+0000 7f032635d640 1 --2- 172.21.15.6:0/3952443946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03200fc380 0x7f03200fc7f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:49.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.802+0000 7f032635d640 1 --2- 172.21.15.6:0/3952443946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03200fb0b0 0x7f03200fb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:49.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.802+0000 7f032635d640 1 -- 172.21.15.6:0/3952443946 >> 172.21.15.6:0/3952443946 conn(0x7f03200f6850 msgr2=0x7f03200f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:49.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.802+0000 7f032635d640 1 -- 172.21.15.6:0/3952443946 shutdown_connections 2024-03-22T06:12:49.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.802+0000 7f032635d640 1 -- 172.21.15.6:0/3952443946 wait complete. 2024-03-22T06:12:49.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.803+0000 7f032635d640 1 Processor -- start 2024-03-22T06:12:49.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.803+0000 7f032635d640 1 -- start start 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.803+0000 7f032635d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03200fb0b0 0x7f032010cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.803+0000 7f032635d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03200fc380 0x7f032010d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.803+0000 7f032635d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032010d8e0 con 0x7f03200fc380 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.803+0000 7f032635d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03201122a0 con 0x7f03200fb0b0 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.803+0000 7f031f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03200fc380 0x7f032010d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.803+0000 7f031f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03200fc380 0x7f032010d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41260/0 (socket says 172.21.15.6:41260) 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.803+0000 7f031f7fe640 1 -- 172.21.15.6:0/2690189749 learned_addr learned my addr 172.21.15.6:0/2690189749 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.804+0000 7f031ffff640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03200fb0b0 0x7f032010cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.804+0000 7f031f7fe640 1 -- 172.21.15.6:0/2690189749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03200fb0b0 msgr2=0x7f032010cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.804+0000 7f031f7fe640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03200fb0b0 0x7f032010cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.804+0000 7f031f7fe640 1 -- 172.21.15.6:0/2690189749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0314005040 con 0x7f03200fc380 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.804+0000 7f031ffff640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03200fb0b0 0x7f032010cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.804+0000 7f031f7fe640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03200fc380 0x7f032010d290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0314007da0 tx=0x7f0314009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:49.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.804+0000 7f031cff9640 1 -- 172.21.15.6:0/2690189749 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03140094c0 con 0x7f03200fc380 2024-03-22T06:12:49.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.804+0000 7f032635d640 1 -- 172.21.15.6:0/2690189749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03201123e0 con 0x7f03200fc380 2024-03-22T06:12:49.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.804+0000 7f031cff9640 1 -- 172.21.15.6:0/2690189749 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0314009620 con 0x7f03200fc380 2024-03-22T06:12:49.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.804+0000 7f031cff9640 1 -- 172.21.15.6:0/2690189749 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f031403b620 con 0x7f03200fc380 2024-03-22T06:12:49.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.804+0000 7f032635d640 1 -- 172.21.15.6:0/2690189749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03201128b0 con 0x7f03200fc380 2024-03-22T06:12:49.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.805+0000 7f032635d640 1 -- 172.21.15.6:0/2690189749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032004bcc0 con 0x7f03200fc380 2024-03-22T06:12:49.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.806+0000 7f031cff9640 1 -- 172.21.15.6:0/2690189749 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f031402f070 con 0x7f03200fc380 2024-03-22T06:12:49.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.806+0000 7f031cff9640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0308069210 0x7f030806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:49.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.806+0000 7f031cff9640 1 -- 172.21.15.6:0/2690189749 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f03140a9730 con 0x7f03200fc380 2024-03-22T06:12:49.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.806+0000 7f031ffff640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0308069210 0x7f030806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:49.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.806+0000 7f031ffff640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0308069210 0x7f030806b6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0310009af0 tx=0x7f031000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:49.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.810+0000 7f031cff9640 1 -- 172.21.15.6:0/2690189749 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f03140759e0 con 0x7f03200fc380 2024-03-22T06:12:49.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.987+0000 7f032635d640 1 -- 172.21.15.6:0/2690189749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0320103c00 con 0x7f03200fc380 2024-03-22T06:12:49.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.988+0000 7f031cff9640 1 -- 172.21.15.6:0/2690189749 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14203 (secure 0 0 0) 0x7f0314079000 con 0x7f03200fc380 2024-03-22T06:12:49.988 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:12:49.988 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":46,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","created":"2024-03-22T06:05:27.379207+0000","modified":"2024-03-22T06:12:22.391030+0000","last_up_change":"2024-03-22T06:10:59.072791+0000","last_in_change":"2024-03-22T06:10:39.982933+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T06:09:46.449392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T06:12:19.302857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T06:12:19.628730+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ecd88bb1-d643-4505-a098-3728178c89e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6803","nonce":3577742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6805","nonce":3577742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6809","nonce":3577742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3577742022},{"type":"v1","addr":"172.21.15.6:6807","nonce":3577742022}]},"public_addr":"172.21.15.6:6803/3577742022","cluster_addr":"172.21.15.6:6805/3577742022","heartbeat_back_addr":"172.21.15.6:6809/3577742022","heartbeat_front_addr":"172.21.15.6:6807/3577742022","state":["exists","up"]},{"osd":1,"uuid":"f1b55307-2c3b-4ab8-a893-af8072f333ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6811","nonce":3758615829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6813","nonce":3758615829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6817","nonce":3758615829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3758615829},{"type":"v1","addr":"172.21.15.6:6815","nonce":3758615829}]},"public_addr":"172.21.15.6:6811/3758615829","cluster_addr":"172.21.15.6:6813/3758615829","heartbeat_back_addr":"172.21.15.6:6817/3758615829","heartbeat_front_addr":"172.21.15.6:6815/3758615829","state":["exists","up"]},{"osd":2,"uuid":"9bacfe42-3c78-4d86-bac0-0142cde76ff5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6819","nonce":907023022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6821","nonce":907023022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6825","nonce":907023022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":907023022},{"type":"v1","addr":"172.21.15.6:6823","nonce":907023022}]},"public_addr":"172.21.15.6:6819/907023022","cluster_addr":"172.21.15.6:6821/907023022","heartbeat_back_addr":"172.21.15.6:6825/907023022","heartbeat_front_addr":"172.21.15.6:6823/907023022","state":["exists","up"]},{"osd":3,"uuid":"2c5a6acd-de41-455d-b6c5-d53bb76197ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6801","nonce":769002526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6803","nonce":769002526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6807","nonce":769002526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":769002526},{"type":"v1","addr":"172.21.15.117:6805","nonce":769002526}]},"public_addr":"172.21.15.117:6801/769002526","cluster_addr":"172.21.15.117:6803/769002526","heartbeat_back_addr":"172.21.15.117:6807/769002526","heartbeat_front_addr":"172.21.15.117:6805/769002526","state":["exists","up"]},{"osd":4,"uuid":"be424041-1e07-41f4-abe0-68fdd21b1eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6809","nonce":1798261586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6811","nonce":1798261586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6815","nonce":1798261586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1798261586},{"type":"v1","addr":"172.21.15.117:6813","nonce":1798261586}]},"public_addr":"172.21.15.117:6809/1798261586","cluster_addr":"172.21.15.117:6811/1798261586","heartbeat_back_addr":"172.21.15.117:6815/1798261586","heartbeat_front_addr":"172.21.15.117:6813/1798261586","state":["exists","up"]},{"osd":5,"uuid":"bf61ff67-1f07-47ed-ad58-e6c946369d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6817","nonce":2701911486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6819","nonce":2701911486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6823","nonce":2701911486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2701911486},{"type":"v1","addr":"172.21.15.117:6821","nonce":2701911486}]},"public_addr":"172.21.15.117:6817/2701911486","cluster_addr":"172.21.15.117:6819/2701911486","heartbeat_back_addr":"172.21.15.117:6823/2701911486","heartbeat_front_addr":"172.21.15.117:6821/2701911486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:08:45.922485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:13.118065+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:42.409903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:05.760002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:31.339234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:57.286867+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3489834288":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/1533067608":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/791727884":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/3665872066":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6801/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6801/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2826360973":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6800/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2671132128":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/619515244":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2939515329":"2024-03-23T06:05:54.100943+0000","172.21.15.6:6801/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.6:0/2042811963":"2024-03-23T06:06:24.832062+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T06:12:49.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.989+0000 7f032635d640 1 -- 172.21.15.6:0/2690189749 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0308069210 msgr2=0x7f030806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:49.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.989+0000 7f032635d640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0308069210 0x7f030806b6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0310009af0 tx=0x7f031000b040 comp rx=0 tx=0).stop 2024-03-22T06:12:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.989+0000 7f032635d640 1 -- 172.21.15.6:0/2690189749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03200fc380 msgr2=0x7f032010d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.989+0000 7f032635d640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03200fc380 0x7f032010d290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0314007da0 tx=0x7f0314009da0 comp rx=0 tx=0).stop 2024-03-22T06:12:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.989+0000 7f032635d640 1 -- 172.21.15.6:0/2690189749 shutdown_connections 2024-03-22T06:12:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.989+0000 7f032635d640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0308069210 0x7f030806b6c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.989+0000 7f032635d640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03200fc380 0x7f032010d290 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.989+0000 7f032635d640 1 --2- 172.21.15.6:0/2690189749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03200fb0b0 0x7f032010cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.989+0000 7f032635d640 1 -- 172.21.15.6:0/2690189749 >> 172.21.15.6:0/2690189749 conn(0x7f03200f6850 msgr2=0x7f03200f83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.990+0000 7f032635d640 1 -- 172.21.15.6:0/2690189749 shutdown_connections 2024-03-22T06:12:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:49.990+0000 7f032635d640 1 -- 172.21.15.6:0/2690189749 wait complete. 2024-03-22T06:12:50.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T06:09:46.449392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T06:12:19.302857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T06:12:19.628730+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T06:12:50.001 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T06:12:50.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.102+0000 7f366b66e640 1 -- 172.21.15.6:0/891929046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36640ff2f0 msgr2=0x7f36641016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.102+0000 7f366b66e640 1 --2- 172.21.15.6:0/891929046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36640ff2f0 0x7f36641016d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f36540048d0 tx=0x7f3654009da0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.102+0000 7f366b66e640 1 -- 172.21.15.6:0/891929046 shutdown_connections 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.102+0000 7f366b66e640 1 --2- 172.21.15.6:0/891929046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36640ff2f0 0x7f36641016d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.102+0000 7f366b66e640 1 --2- 172.21.15.6:0/891929046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3664066210 0x7f36640fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.102+0000 7f366b66e640 1 -- 172.21.15.6:0/891929046 >> 172.21.15.6:0/891929046 conn(0x7f36640f6880 msgr2=0x7f36640f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.102+0000 7f366b66e640 1 -- 172.21.15.6:0/891929046 shutdown_connections 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.102+0000 7f366b66e640 1 -- 172.21.15.6:0/891929046 wait complete. 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f366b66e640 1 Processor -- start 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f366b66e640 1 -- start start 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f366b66e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3664066210 0x7f366410cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f366b66e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36640ff2f0 0x7f366410d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f366b66e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366410d8d0 con 0x7f36640ff2f0 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f366b66e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3664112240 con 0x7f3664066210 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f3668c09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36640ff2f0 0x7f366410d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f3668c09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36640ff2f0 0x7f366410d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41292/0 (socket says 172.21.15.6:41292) 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f3668c09640 1 -- 172.21.15.6:0/2909989438 learned_addr learned my addr 172.21.15.6:0/2909989438 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:50.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f366940a640 1 --2- 172.21.15.6:0/2909989438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3664066210 0x7f366410cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f3668c09640 1 -- 172.21.15.6:0/2909989438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3664066210 msgr2=0x7f366410cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f3668c09640 1 --2- 172.21.15.6:0/2909989438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3664066210 0x7f366410cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f3668c09640 1 -- 172.21.15.6:0/2909989438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3654005040 con 0x7f36640ff2f0 2024-03-22T06:12:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f3668c09640 1 --2- 172.21.15.6:0/2909989438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36640ff2f0 0x7f366410d250 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3654007dd0 tx=0x7f3654009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f3659ffb640 1 -- 172.21.15.6:0/2909989438 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36540094c0 con 0x7f36640ff2f0 2024-03-22T06:12:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f366b66e640 1 -- 172.21.15.6:0/2909989438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3664112380 con 0x7f36640ff2f0 2024-03-22T06:12:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.103+0000 7f3659ffb640 1 -- 172.21.15.6:0/2909989438 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3654009620 con 0x7f36640ff2f0 2024-03-22T06:12:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.104+0000 7f366b66e640 1 -- 172.21.15.6:0/2909989438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3664112870 con 0x7f36640ff2f0 2024-03-22T06:12:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.104+0000 7f3659ffb640 1 -- 172.21.15.6:0/2909989438 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36540465e0 con 0x7f36640ff2f0 2024-03-22T06:12:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.105+0000 7f364f7fe640 1 -- 172.21.15.6:0/2909989438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3644005210 con 0x7f36640ff2f0 2024-03-22T06:12:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.105+0000 7f3659ffb640 1 -- 172.21.15.6:0/2909989438 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f365402f070 con 0x7f36640ff2f0 2024-03-22T06:12:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.105+0000 7f3659ffb640 1 --2- 172.21.15.6:0/2909989438 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3650069210 0x7f365006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.105+0000 7f3659ffb640 1 -- 172.21.15.6:0/2909989438 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f36540a96e0 con 0x7f36640ff2f0 2024-03-22T06:12:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.105+0000 7f366940a640 1 --2- 172.21.15.6:0/2909989438 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3650069210 0x7f365006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.106+0000 7f366940a640 1 --2- 172.21.15.6:0/2909989438 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3650069210 0x7f365006b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f36640fee90 tx=0x7f3660009ef0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:50.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.110+0000 7f3659ffb640 1 -- 172.21.15.6:0/2909989438 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3654075990 con 0x7f36640ff2f0 2024-03-22T06:12:50.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.282+0000 7f364f7fe640 1 -- 172.21.15.6:0/2909989438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3644005a40 con 0x7f36640ff2f0 2024-03-22T06:12:50.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.283+0000 7f3659ffb640 1 -- 172.21.15.6:0/2909989438 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3654078fb0 con 0x7f36640ff2f0 2024-03-22T06:12:50.283 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-03-22T06:12:50.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.284+0000 7f364f7fe640 1 -- 172.21.15.6:0/2909989438 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3650069210 msgr2=0x7f365006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.284+0000 7f364f7fe640 1 --2- 172.21.15.6:0/2909989438 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3650069210 0x7f365006b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f36640fee90 tx=0x7f3660009ef0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.284+0000 7f364f7fe640 1 -- 172.21.15.6:0/2909989438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36640ff2f0 msgr2=0x7f366410d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.284+0000 7f364f7fe640 1 --2- 172.21.15.6:0/2909989438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36640ff2f0 0x7f366410d250 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3654007dd0 tx=0x7f3654009da0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.284+0000 7f364f7fe640 1 -- 172.21.15.6:0/2909989438 shutdown_connections 2024-03-22T06:12:50.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.284+0000 7f364f7fe640 1 --2- 172.21.15.6:0/2909989438 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f3650069210 0x7f365006b6c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.284+0000 7f364f7fe640 1 --2- 172.21.15.6:0/2909989438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36640ff2f0 0x7f366410d250 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.284+0000 7f364f7fe640 1 --2- 172.21.15.6:0/2909989438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3664066210 0x7f366410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.284+0000 7f364f7fe640 1 -- 172.21.15.6:0/2909989438 >> 172.21.15.6:0/2909989438 conn(0x7f36640f6880 msgr2=0x7f36640f6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.285+0000 7f364f7fe640 1 -- 172.21.15.6:0/2909989438 shutdown_connections 2024-03-22T06:12:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.285+0000 7f364f7fe640 1 -- 172.21.15.6:0/2909989438 wait complete. 2024-03-22T06:12:50.296 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T06:12:50.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.395+0000 7fdaa506e640 1 -- 172.21.15.6:0/2073125591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaa0064fb0 msgr2=0x7fdaa00fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.395+0000 7fdaa506e640 1 --2- 172.21.15.6:0/2073125591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaa0064fb0 0x7fdaa00fee20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fda900048d0 tx=0x7fda90009da0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.395+0000 7fdaa506e640 1 -- 172.21.15.6:0/2073125591 shutdown_connections 2024-03-22T06:12:50.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.395+0000 7fdaa506e640 1 --2- 172.21.15.6:0/2073125591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa00ff360 0x7fdaa0101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.395+0000 7fdaa506e640 1 --2- 172.21.15.6:0/2073125591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaa0064fb0 0x7fdaa00fee20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.395+0000 7fdaa506e640 1 -- 172.21.15.6:0/2073125591 >> 172.21.15.6:0/2073125591 conn(0x7fdaa00f6870 msgr2=0x7fdaa00f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:50.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fdaa506e640 1 -- 172.21.15.6:0/2073125591 shutdown_connections 2024-03-22T06:12:50.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fdaa506e640 1 -- 172.21.15.6:0/2073125591 wait complete. 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fdaa506e640 1 Processor -- start 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fdaa506e640 1 -- start start 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fdaa506e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaa0064fb0 0x7fdaa00fc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fdaa506e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa00ff360 0x7fdaa00fad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fdaa506e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa00fcca0 con 0x7fdaa0064fb0 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fdaa506e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa00fb260 con 0x7fdaa00ff360 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fda9ed9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaa0064fb0 0x7fdaa00fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fda9ed9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaa0064fb0 0x7fdaa00fc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41300/0 (socket says 172.21.15.6:41300) 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fda9ed9d640 1 -- 172.21.15.6:0/1298565644 learned_addr learned my addr 172.21.15.6:0/1298565644 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fda97fff640 1 --2- 172.21.15.6:0/1298565644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa00ff360 0x7fdaa00fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.396+0000 7fda9ed9d640 1 -- 172.21.15.6:0/1298565644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa00ff360 msgr2=0x7fdaa00fad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.397+0000 7fda9ed9d640 1 --2- 172.21.15.6:0/1298565644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa00ff360 0x7fdaa00fad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.397+0000 7fda9ed9d640 1 -- 172.21.15.6:0/1298565644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90005040 con 0x7fdaa0064fb0 2024-03-22T06:12:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.397+0000 7fda9ed9d640 1 --2- 172.21.15.6:0/1298565644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaa0064fb0 0x7fdaa00fc650 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fda9002bda0 tx=0x7fda90009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:50.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.397+0000 7fda977fe640 1 -- 172.21.15.6:0/1298565644 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda900094c0 con 0x7fdaa0064fb0 2024-03-22T06:12:50.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.397+0000 7fda977fe640 1 -- 172.21.15.6:0/1298565644 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda90009620 con 0x7fdaa0064fb0 2024-03-22T06:12:50.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.397+0000 7fdaa506e640 1 -- 172.21.15.6:0/1298565644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa00fb4e0 con 0x7fdaa0064fb0 2024-03-22T06:12:50.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.397+0000 7fda977fe640 1 -- 172.21.15.6:0/1298565644 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9003b640 con 0x7fdaa0064fb0 2024-03-22T06:12:50.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.397+0000 7fdaa506e640 1 -- 172.21.15.6:0/1298565644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa00fba00 con 0x7fdaa0064fb0 2024-03-22T06:12:50.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.398+0000 7fdaa506e640 1 -- 172.21.15.6:0/1298565644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa0105540 con 0x7fdaa0064fb0 2024-03-22T06:12:50.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.398+0000 7fda977fe640 1 -- 172.21.15.6:0/1298565644 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fda9002f070 con 0x7fdaa0064fb0 2024-03-22T06:12:50.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.399+0000 7fda977fe640 1 --2- 172.21.15.6:0/1298565644 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fda8c069210 0x7fda8c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.399+0000 7fda977fe640 1 -- 172.21.15.6:0/1298565644 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fda900a9650 con 0x7fdaa0064fb0 2024-03-22T06:12:50.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.399+0000 7fda97fff640 1 --2- 172.21.15.6:0/1298565644 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fda8c069210 0x7fda8c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.399+0000 7fda97fff640 1 --2- 172.21.15.6:0/1298565644 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fda8c069210 0x7fda8c06b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00fc400 tx=0x7fda8800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:50.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.403+0000 7fda977fe640 1 -- 172.21.15.6:0/1298565644 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fda90075980 con 0x7fdaa0064fb0 2024-03-22T06:12:50.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.578+0000 7fdaa506e640 1 -- 172.21.15.6:0/1298565644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fdaa004bcc0 con 0x7fdaa0064fb0 2024-03-22T06:12:50.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.578+0000 7fda977fe640 1 -- 172.21.15.6:0/1298565644 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7fda90078fa0 con 0x7fdaa0064fb0 2024-03-22T06:12:50.579 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-03-22T06:12:50.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.580+0000 7fdaa506e640 1 -- 172.21.15.6:0/1298565644 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fda8c069210 msgr2=0x7fda8c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.580+0000 7fdaa506e640 1 --2- 172.21.15.6:0/1298565644 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fda8c069210 0x7fda8c06b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00fc400 tx=0x7fda8800b040 comp rx=0 tx=0).stop 2024-03-22T06:12:50.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.580+0000 7fdaa506e640 1 -- 172.21.15.6:0/1298565644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaa0064fb0 msgr2=0x7fdaa00fc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.580+0000 7fdaa506e640 1 --2- 172.21.15.6:0/1298565644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaa0064fb0 0x7fdaa00fc650 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fda9002bda0 tx=0x7fda90009da0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.580+0000 7fdaa506e640 1 -- 172.21.15.6:0/1298565644 shutdown_connections 2024-03-22T06:12:50.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.580+0000 7fdaa506e640 1 --2- 172.21.15.6:0/1298565644 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fda8c069210 0x7fda8c06b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.580+0000 7fdaa506e640 1 --2- 172.21.15.6:0/1298565644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa00ff360 0x7fdaa00fad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.580+0000 7fdaa506e640 1 --2- 172.21.15.6:0/1298565644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaa0064fb0 0x7fdaa00fc650 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.580+0000 7fdaa506e640 1 -- 172.21.15.6:0/1298565644 >> 172.21.15.6:0/1298565644 conn(0x7fdaa00f6870 msgr2=0x7fdaa00f6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:50.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.580+0000 7fdaa506e640 1 -- 172.21.15.6:0/1298565644 shutdown_connections 2024-03-22T06:12:50.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.580+0000 7fdaa506e640 1 -- 172.21.15.6:0/1298565644 wait complete. 2024-03-22T06:12:50.590 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:50 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2690189749' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:12:50.590 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:50 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2909989438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T06:12:50.591 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T06:12:50.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.691+0000 7f9fddb9d640 1 -- 172.21.15.6:0/1070277567 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fd80fc370 msgr2=0x7f9fd80fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.691+0000 7f9fddb9d640 1 --2- 172.21.15.6:0/1070277567 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fd80fc370 0x7f9fd80fc7e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00048d0 tx=0x7f9fc0009da0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.691+0000 7f9fddb9d640 1 -- 172.21.15.6:0/1070277567 shutdown_connections 2024-03-22T06:12:50.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.691+0000 7f9fddb9d640 1 --2- 172.21.15.6:0/1070277567 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fd80fc370 0x7f9fd80fc7e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.691+0000 7f9fddb9d640 1 --2- 172.21.15.6:0/1070277567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fd80fb0a0 0x7f9fd80fb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.691+0000 7f9fddb9d640 1 -- 172.21.15.6:0/1070277567 >> 172.21.15.6:0/1070277567 conn(0x7f9fd80f6880 msgr2=0x7f9fd80f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.691+0000 7f9fddb9d640 1 -- 172.21.15.6:0/1070277567 shutdown_connections 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.691+0000 7f9fddb9d640 1 -- 172.21.15.6:0/1070277567 wait complete. 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.691+0000 7f9fddb9d640 1 Processor -- start 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fddb9d640 1 -- start start 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fddb9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fd80fb0a0 0x7f9fd810ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fddb9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fd80fc370 0x7f9fd810d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fddb9d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd810d870 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fddb9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd8112230 con 0x7f9fd80fc370 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fd77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fd80fb0a0 0x7f9fd810ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fd77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fd80fb0a0 0x7f9fd810ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41332/0 (socket says 172.21.15.6:41332) 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fd77fe640 1 -- 172.21.15.6:0/2304517649 learned_addr learned my addr 172.21.15.6:0/2304517649 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fd6ffd640 1 --2- 172.21.15.6:0/2304517649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fd80fc370 0x7f9fd810d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fd77fe640 1 -- 172.21.15.6:0/2304517649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fd80fc370 msgr2=0x7f9fd810d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fd77fe640 1 --2- 172.21.15.6:0/2304517649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fd80fc370 0x7f9fd810d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fd77fe640 1 -- 172.21.15.6:0/2304517649 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc0005040 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fd77fe640 1 --2- 172.21.15.6:0/2304517649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fd80fb0a0 0x7f9fd810ccb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8002a80 tx=0x7f9fc8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:50.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fcffff640 1 -- 172.21.15.6:0/2304517649 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fc8007c30 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fcffff640 1 -- 172.21.15.6:0/2304517649 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9fc800f040 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fddb9d640 1 -- 172.21.15.6:0/2304517649 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd81123d0 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fcffff640 1 -- 172.21.15.6:0/2304517649 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fc800db80 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.692+0000 7f9fddb9d640 1 -- 172.21.15.6:0/2304517649 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd81128c0 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.693+0000 7f9fddb9d640 1 -- 172.21.15.6:0/2304517649 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd804bcc0 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.694+0000 7f9fcffff640 1 -- 172.21.15.6:0/2304517649 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f9fc801c400 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.694+0000 7f9fcffff640 1 --2- 172.21.15.6:0/2304517649 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fc4069210 0x7f9fc406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.694+0000 7f9fcffff640 1 -- 172.21.15.6:0/2304517649 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9fc8086ac0 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.694+0000 7f9fd6ffd640 1 --2- 172.21.15.6:0/2304517649 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fc4069210 0x7f9fc406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.695+0000 7f9fd6ffd640 1 --2- 172.21.15.6:0/2304517649 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fc4069210 0x7f9fc406b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9fd810e1a0 tx=0x7f9fc0009da0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:50.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.699+0000 7f9fcffff640 1 -- 172.21.15.6:0/2304517649 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9fc8052d70 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:50 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2690189749' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:12:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:50 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2909989438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T06:12:50.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.871+0000 7f9fddb9d640 1 -- 172.21.15.6:0/2304517649 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f9fd8112bb0 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.871+0000 7f9fcffff640 1 -- 172.21.15.6:0/2304517649 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f9fc8056390 con 0x7f9fd80fb0a0 2024-03-22T06:12:50.872 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-03-22T06:12:50.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.873+0000 7f9fddb9d640 1 -- 172.21.15.6:0/2304517649 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fc4069210 msgr2=0x7f9fc406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.873+0000 7f9fddb9d640 1 --2- 172.21.15.6:0/2304517649 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fc4069210 0x7f9fc406b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9fd810e1a0 tx=0x7f9fc0009da0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.873+0000 7f9fddb9d640 1 -- 172.21.15.6:0/2304517649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fd80fb0a0 msgr2=0x7f9fd810ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.873+0000 7f9fddb9d640 1 --2- 172.21.15.6:0/2304517649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fd80fb0a0 0x7f9fd810ccb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8002a80 tx=0x7f9fc8004600 comp rx=0 tx=0).stop 2024-03-22T06:12:50.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.873+0000 7f9fddb9d640 1 -- 172.21.15.6:0/2304517649 shutdown_connections 2024-03-22T06:12:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.873+0000 7f9fddb9d640 1 --2- 172.21.15.6:0/2304517649 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fc4069210 0x7f9fc406b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.873+0000 7f9fddb9d640 1 --2- 172.21.15.6:0/2304517649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fd80fc370 0x7f9fd810d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.873+0000 7f9fddb9d640 1 --2- 172.21.15.6:0/2304517649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fd80fb0a0 0x7f9fd810ccb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.873+0000 7f9fddb9d640 1 -- 172.21.15.6:0/2304517649 >> 172.21.15.6:0/2304517649 conn(0x7f9fd80f6880 msgr2=0x7f9fd8065550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.873+0000 7f9fddb9d640 1 -- 172.21.15.6:0/2304517649 shutdown_connections 2024-03-22T06:12:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.873+0000 7f9fddb9d640 1 -- 172.21.15.6:0/2304517649 wait complete. 2024-03-22T06:12:50.885 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T06:12:50.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.982+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1092754917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d8066210 msgr2=0x7fc7d80fedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.982+0000 7fc7ddd9d640 1 --2- 172.21.15.6:0/1092754917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d8066210 0x7fc7d80fedd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0048d0 tx=0x7fc7cc009da0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.982+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1092754917 shutdown_connections 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.982+0000 7fc7ddd9d640 1 --2- 172.21.15.6:0/1092754917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d80ff310 0x7fc7d81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.982+0000 7fc7ddd9d640 1 --2- 172.21.15.6:0/1092754917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d8066210 0x7fc7d80fedd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.982+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1092754917 >> 172.21.15.6:0/1092754917 conn(0x7fc7d80f68a0 msgr2=0x7fc7d80f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.982+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1092754917 shutdown_connections 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.982+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1092754917 wait complete. 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.982+0000 7fc7ddd9d640 1 Processor -- start 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7ddd9d640 1 -- start start 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7ddd9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d8066210 0x7fc7d810cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7ddd9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d80ff310 0x7fc7d810d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7ddd9d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d810d900 con 0x7fc7d80ff310 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7ddd9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8112270 con 0x7fc7d8066210 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7d6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d80ff310 0x7fc7d810d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7d6ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d80ff310 0x7fc7d810d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41342/0 (socket says 172.21.15.6:41342) 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7d6ffd640 1 -- 172.21.15.6:0/1302147895 learned_addr learned my addr 172.21.15.6:0/1302147895 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:50.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7d77fe640 1 --2- 172.21.15.6:0/1302147895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d8066210 0x7fc7d810cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7d6ffd640 1 -- 172.21.15.6:0/1302147895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d8066210 msgr2=0x7fc7d810cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:50.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7d6ffd640 1 --2- 172.21.15.6:0/1302147895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d8066210 0x7fc7d810cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:50.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7d6ffd640 1 -- 172.21.15.6:0/1302147895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc005040 con 0x7fc7d80ff310 2024-03-22T06:12:50.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7d6ffd640 1 --2- 172.21.15.6:0/1302147895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d80ff310 0x7fc7d810d2b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8002a50 tx=0x7fc7c8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:50.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7bffff640 1 -- 172.21.15.6:0/1302147895 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7c8007c30 con 0x7fc7d80ff310 2024-03-22T06:12:50.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7bffff640 1 -- 172.21.15.6:0/1302147895 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7c800f040 con 0x7fc7d80ff310 2024-03-22T06:12:50.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1302147895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d8112410 con 0x7fc7d80ff310 2024-03-22T06:12:50.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.983+0000 7fc7bffff640 1 -- 172.21.15.6:0/1302147895 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7c800db50 con 0x7fc7d80ff310 2024-03-22T06:12:50.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.984+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1302147895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d81128e0 con 0x7fc7d80ff310 2024-03-22T06:12:50.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.984+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1302147895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d804bcc0 con 0x7fc7d80ff310 2024-03-22T06:12:50.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.985+0000 7fc7bffff640 1 -- 172.21.15.6:0/1302147895 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fc7c801c400 con 0x7fc7d80ff310 2024-03-22T06:12:50.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.985+0000 7fc7bffff640 1 --2- 172.21.15.6:0/1302147895 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc7c0069210 0x7fc7c006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:50.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.985+0000 7fc7d77fe640 1 --2- 172.21.15.6:0/1302147895 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc7c0069210 0x7fc7c006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:50.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.986+0000 7fc7bffff640 1 -- 172.21.15.6:0/1302147895 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc7c8086ab0 con 0x7fc7d80ff310 2024-03-22T06:12:50.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.986+0000 7fc7d77fe640 1 --2- 172.21.15.6:0/1302147895 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc7c0069210 0x7fc7c006b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc7d810e1e0 tx=0x7fc7cc009da0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:50.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:50.990+0000 7fc7bffff640 1 -- 172.21.15.6:0/1302147895 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc7c804d190 con 0x7fc7d80ff310 2024-03-22T06:12:51.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.203+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1302147895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7d8112b00 con 0x7fc7d80ff310 2024-03-22T06:12:51.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.203+0000 7fc7bffff640 1 -- 172.21.15.6:0/1302147895 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4683 (secure 0 0 0) 0x7fc7c8088e60 con 0x7fc7d80ff310 2024-03-22T06:12:51.204 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:12:51.204 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14476,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.6:6829/947853733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":947853733},{"type":"v1","addr":"172.21.15.6:6829","nonce":947853733}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24269,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6825/1251227895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1251227895},{"type":"v1","addr":"172.21.15.117:6825","nonce":1251227895}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:12:36.515420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"cephfs.smithi006.bztopc","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.6:6827/1260089028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1260089028},{"type":"v1","addr":"172.21.15.6:6827","nonce":1260089028}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-03-22T06:12:51.204 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:12:51.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.205+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1302147895 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc7c0069210 msgr2=0x7fc7c006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:51.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.205+0000 7fc7ddd9d640 1 --2- 172.21.15.6:0/1302147895 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc7c0069210 0x7fc7c006b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc7d810e1e0 tx=0x7fc7cc009da0 comp rx=0 tx=0).stop 2024-03-22T06:12:51.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.205+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1302147895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d80ff310 msgr2=0x7fc7d810d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:51.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.205+0000 7fc7ddd9d640 1 --2- 172.21.15.6:0/1302147895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d80ff310 0x7fc7d810d2b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8002a50 tx=0x7fc7c8004600 comp rx=0 tx=0).stop 2024-03-22T06:12:51.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.205+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1302147895 shutdown_connections 2024-03-22T06:12:51.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.205+0000 7fc7ddd9d640 1 --2- 172.21.15.6:0/1302147895 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc7c0069210 0x7fc7c006b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:51.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.205+0000 7fc7ddd9d640 1 --2- 172.21.15.6:0/1302147895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d80ff310 0x7fc7d810d2b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:51.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.205+0000 7fc7ddd9d640 1 --2- 172.21.15.6:0/1302147895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d8066210 0x7fc7d810cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:51.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.205+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1302147895 >> 172.21.15.6:0/1302147895 conn(0x7fc7d80f68a0 msgr2=0x7fc7d80f6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:51.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.205+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1302147895 shutdown_connections 2024-03-22T06:12:51.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:51.205+0000 7fc7ddd9d640 1 -- 172.21.15.6:0/1302147895 wait complete. 2024-03-22T06:12:51.217 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 14, 'max_mds': 1, 'flags': 18} 2024-03-22T06:12:51.217 INFO:teuthology.run_tasks:Running task kclient... 2024-03-22T06:12:51.261 INFO:tasks.kclient:Mounting kernel clients... 2024-03-22T06:12:51.262 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-22T06:12:51.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:51.262 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-03-22T06:12:51.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:51.277 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2024-03-22T06:12:51.346 INFO:teuthology.orchestra.run.smithi006.stderr:Cannot find device "ceph-brx" 2024-03-22T06:12:51.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:12:51.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:51.348 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-22T06:12:51.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:51.368 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-03-22T06:12:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-03-22T06:12:51.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:12:51.446 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-22T06:12:51.446 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T06:12:51.446 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T06:12:51.446 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-22T06:12:51.714 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:51.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:51 smithi117 ceph-mon[36514]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:51.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:51 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1298565644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T06:12:51.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:51 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2304517649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T06:12:51.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:51 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1302147895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T06:12:51.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:51 smithi006 ceph-mon[29301]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:51.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:51 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1298565644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T06:12:51.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:51 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2304517649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T06:12:51.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:51 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1302147895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T06:12:53.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.336+0000 7ffb442ea700 1 -- 172.21.15.6:0/1384672111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb3c0fbc80 msgr2=0x7ffb3c0fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:53.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.336+0000 7ffb442ea700 1 --2- 172.21.15.6:0/1384672111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb3c0fbc80 0x7ffb3c0fc050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb38004940 tx=0x7ffb38009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:53.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.336+0000 7ffb442ea700 1 -- 172.21.15.6:0/1384672111 shutdown_connections 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.336+0000 7ffb442ea700 1 --2- 172.21.15.6:0/1384672111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3c0fc590 0x7ffb3c1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.336+0000 7ffb442ea700 1 --2- 172.21.15.6:0/1384672111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb3c0fbc80 0x7ffb3c0fc050 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.336+0000 7ffb442ea700 1 -- 172.21.15.6:0/1384672111 >> 172.21.15.6:0/1384672111 conn(0x7ffb3c072bc0 msgr2=0x7ffb3c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.336+0000 7ffb442ea700 1 -- 172.21.15.6:0/1384672111 shutdown_connections 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.336+0000 7ffb442ea700 1 -- 172.21.15.6:0/1384672111 wait complete. 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb442ea700 1 Processor -- start 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb442ea700 1 -- start start 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb442ea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb3c0fbc80 0x7ffb3c115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb442ea700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3c0fc590 0x7ffb3c116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb442ea700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c116950 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb442ea700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c10fc90 con 0x7ffb3c0fc590 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb42086700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb3c0fbc80 0x7ffb3c115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb42086700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb3c0fbc80 0x7ffb3c115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41358/0 (socket says 172.21.15.6:41358) 2024-03-22T06:12:53.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb42086700 1 -- 172.21.15.6:0/3744701735 learned_addr learned my addr 172.21.15.6:0/3744701735 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:53.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb41885700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3c0fc590 0x7ffb3c116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:53.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb42086700 1 -- 172.21.15.6:0/3744701735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3c0fc590 msgr2=0x7ffb3c116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:53.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb42086700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3c0fc590 0x7ffb3c116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:53.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb42086700 1 -- 172.21.15.6:0/3744701735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb38005040 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb41885700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3c0fc590 0x7ffb3c116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:12:53.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb42086700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb3c0fbc80 0x7ffb3c115d30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800b3b0 tx=0x7ffb38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:53.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.337+0000 7ffb32ffd700 1 -- 172.21.15.6:0/3744701735 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3800e040 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.338+0000 7ffb32ffd700 1 -- 172.21.15.6:0/3744701735 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb3800bbc0 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.338+0000 7ffb442ea700 1 -- 172.21.15.6:0/3744701735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c10feb0 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.338+0000 7ffb32ffd700 1 -- 172.21.15.6:0/3744701735 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3801c3f0 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.338+0000 7ffb442ea700 1 -- 172.21.15.6:0/3744701735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c110470 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.339+0000 7ffb442ea700 1 -- 172.21.15.6:0/3744701735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c103c30 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.339+0000 7ffb32ffd700 1 -- 172.21.15.6:0/3744701735 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7ffb3800bd30 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.339+0000 7ffb32ffd700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb280692a0 0x7ffb2806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:53.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.340+0000 7ffb41885700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb280692a0 0x7ffb2806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:53.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.340+0000 7ffb32ffd700 1 -- 172.21.15.6:0/3744701735 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ffb38088360 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.340+0000 7ffb41885700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb280692a0 0x7ffb2806b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c006fd0 tx=0x7ffb2c008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:53.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.344+0000 7ffb32ffd700 1 -- 172.21.15.6:0/3744701735 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ffb380581d0 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.588+0000 7ffb442ea700 1 -- 172.21.15.6:0/3744701735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffb3c04bed0 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.589+0000 7ffb32ffd700 1 -- 172.21.15.6:0/3744701735 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+83 (secure 0 0 0) 0x7ffb38057d60 con 0x7ffb3c0fbc80 2024-03-22T06:12:53.590 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-22T06:12:53.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.591+0000 7ffb442ea700 1 -- 172.21.15.6:0/3744701735 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb280692a0 msgr2=0x7ffb2806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:53.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.591+0000 7ffb442ea700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb280692a0 0x7ffb2806b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c006fd0 tx=0x7ffb2c008040 comp rx=0 tx=0).stop 2024-03-22T06:12:53.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.591+0000 7ffb442ea700 1 -- 172.21.15.6:0/3744701735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb3c0fbc80 msgr2=0x7ffb3c115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:53.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.591+0000 7ffb442ea700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb3c0fbc80 0x7ffb3c115d30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800b3b0 tx=0x7ffb38009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:53.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.591+0000 7ffb442ea700 1 -- 172.21.15.6:0/3744701735 shutdown_connections 2024-03-22T06:12:53.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.591+0000 7ffb442ea700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7ffb280692a0 0x7ffb2806b750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:53.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.591+0000 7ffb442ea700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb3c0fbc80 0x7ffb3c115d30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:53.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.591+0000 7ffb442ea700 1 --2- 172.21.15.6:0/3744701735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb3c0fc590 0x7ffb3c116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:53.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.591+0000 7ffb442ea700 1 -- 172.21.15.6:0/3744701735 >> 172.21.15.6:0/3744701735 conn(0x7ffb3c072bc0 msgr2=0x7ffb3c0fa450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:53.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.591+0000 7ffb442ea700 1 -- 172.21.15.6:0/3744701735 shutdown_connections 2024-03-22T06:12:53.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:53.591+0000 7ffb442ea700 1 -- 172.21.15.6:0/3744701735 wait complete. 2024-03-22T06:12:53.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:53 smithi117 ceph-mon[36514]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:53 smithi006 ceph-mon[29301]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:54.191 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T06:12:54.191 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T06:12:54.191 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2024-03-22T06:12:54.191 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T06:12:54.191 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T06:12:54.191 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T06:12:54.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T06:12:54.191 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T06:12:54.191 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T06:12:54.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:54.192 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2024-03-22T06:12:54.207 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2024-03-22T06:12:54.208 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2024-03-22T06:12:54.209 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-22T06:12:54.209 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42224sec preferred_lft 42224sec 2024-03-22T06:12:54.209 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2024-03-22T06:12:54.209 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-03-22T06:12:54.209 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T06:12:54.209 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2024-03-22T06:12:54.209 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2024-03-22T06:12:54.209 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T06:12:54.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:54.210 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2024-03-22T06:12:54.210 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-03-22T06:12:54.210 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add name ceph-brx type bridge 2024-03-22T06:12:54.210 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr flush dev ceph-brx 2024-03-22T06:12:54.210 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set ceph-brx up 2024-03-22T06:12:54.210 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T06:12:54.210 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-03-22T06:12:54.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:54.394 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T06:12:54.421 INFO:teuthology.orchestra.run.smithi006.stdout:1 2024-03-22T06:12:54.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:54.423 DEBUG:teuthology.orchestra.run.smithi006:> route 2024-03-22T06:12:54.487 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2024-03-22T06:12:54.487 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T06:12:54.487 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-22T06:12:54.487 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-22T06:12:54.487 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T06:12:54.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:54.488 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2024-03-22T06:12:54.488 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-03-22T06:12:54.488 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-22T06:12:54.488 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-22T06:12:54.488 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-22T06:12:54.488 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-03-22T06:12:54.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:54 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3744701735' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T06:12:54.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:54 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3744701735' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T06:12:54.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:54.787 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-03-22T06:12:54.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:54.801 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2024-03-22T06:12:54.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:54.856 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2024-03-22T06:12:54.856 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-03-22T06:12:54.856 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T06:12:54.856 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-22T06:12:54.856 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-03-22T06:12:54.938 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-22T06:12:54.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:54.938 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2024-03-22T06:12:54.938 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-03-22T06:12:54.938 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-22T06:12:54.938 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T06:12:54.939 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-22T06:12:54.939 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-22T06:12:54.939 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-22T06:12:54.939 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-03-22T06:12:55.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:55.071 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2024-03-22T06:12:55.071 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-03-22T06:12:55.071 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2024-03-22T06:12:55.071 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T06:12:55.071 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-03-22T06:12:55.112 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T06:12:55.112 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T06:12:55.167 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T06:12:55.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T06:12:55.167 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T06:12:55.221 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T06:12:55.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:55.222 DEBUG:teuthology.orchestra.run.smithi006:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T06:12:55.319 INFO:teuthology.orchestra.run.smithi006.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T06:12:55.319 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T06:12:55.319 INFO:teuthology.orchestra.run.smithi006.stdout:invalid new device string format 2024-03-22T06:12:55.445 INFO:teuthology.orchestra.run.smithi006.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T06:12:55.445 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T06:12:55.446 INFO:teuthology.orchestra.run.smithi006.stdout:invalid new device string format 2024-03-22T06:12:55.446 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: resolved to: "172.21.15.6:3300,172.21.15.117:3300" 2024-03-22T06:12:55.446 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T06:12:55.451 INFO:teuthology.orchestra.run.smithi006.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T06:12:55.451 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T06:12:55.451 INFO:teuthology.orchestra.run.smithi006.stdout:invalid new device string format 2024-03-22T06:12:55.451 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: resolved to: "172.21.15.6:3300,172.21.15.117:3300" 2024-03-22T06:12:55.451 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T06:12:55.451 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: resolved to: "172.21.15.6:6789,172.21.15.117:6789" 2024-03-22T06:12:55.451 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.6:6789,172.21.15.117:6789:/ 2024-03-22T06:12:55.451 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=fa0e13c2-e811-11ee-95cd-87774f69a715" will pass to kernel 2024-03-22T06:12:55.452 INFO:teuthology.orchestra.run.smithi006.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-22T06:12:55.452 INFO:teuthology.orchestra.run.smithi006.stdout: You just mounted a file system that supports labels which does not 2024-03-22T06:12:55.452 INFO:teuthology.orchestra.run.smithi006.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-22T06:12:55.452 INFO:teuthology.orchestra.run.smithi006.stdout: applications will generate AVC messages and not be allowed access to 2024-03-22T06:12:55.452 INFO:teuthology.orchestra.run.smithi006.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-22T06:12:55.454 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T06:12:55.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:55.454 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T06:12:55.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:55.481 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T06:12:55.541 INFO:teuthology.orchestra.run.smithi006.stdout:client24313 2024-03-22T06:12:55.541 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:12:55.541 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T06:12:55.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:55.598 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T06:12:55.657 INFO:teuthology.orchestra.run.smithi006.stdout:client24313 2024-03-22T06:12:55.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:55.658 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T06:12:55.718 INFO:teuthology.orchestra.run.smithi006.stdout:client24313 2024-03-22T06:12:55.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:55.718 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715.client24313/status') 2024-03-22T06:12:55.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:55 smithi117 ceph-mon[36514]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:55.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:55 smithi006 ceph-mon[29301]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:55.788 INFO:teuthology.orchestra.run.smithi006.stdout:instance: client.24313 (3)172.21.15.6:0/1969668997 2024-03-22T06:12:55.788 INFO:teuthology.orchestra.run.smithi006.stdout:blocklisted: false 2024-03-22T06:12:55.789 INFO:teuthology.orchestra.run.smithi006.stderr:0+1 records in 2024-03-22T06:12:55.789 INFO:teuthology.orchestra.run.smithi006.stderr:0+1 records out 2024-03-22T06:12:55.789 INFO:teuthology.orchestra.run.smithi006.stderr:70 bytes copied, 2.3376e-05 s, 3.0 MB/s 2024-03-22T06:12:55.790 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-22T06:12:55.790 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T06:12:55.790 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-22T06:12:55.790 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-22T06:12:56.089 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:12:57.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:57 smithi117 ceph-mon[36514]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:57.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.755+0000 7f6854518700 1 -- 172.21.15.6:0/1918084933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f684c101d20 msgr2=0x7f684c1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:57.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.755+0000 7f6854518700 1 --2- 172.21.15.6:0/1918084933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f684c101d20 0x7f684c1020f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6848004940 tx=0x7f6848009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:57.757 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:57 smithi006 ceph-mon[29301]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:12:57.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 -- 172.21.15.6:0/1918084933 shutdown_connections 2024-03-22T06:12:57.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 --2- 172.21.15.6:0/1918084933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f684c1026c0 0x7f684c0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:57.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 --2- 172.21.15.6:0/1918084933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f684c101d20 0x7f684c1020f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:57.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 -- 172.21.15.6:0/1918084933 >> 172.21.15.6:0/1918084933 conn(0x7f684c0f78b0 msgr2=0x7f684c0f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:57.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 -- 172.21.15.6:0/1918084933 shutdown_connections 2024-03-22T06:12:57.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 -- 172.21.15.6:0/1918084933 wait complete. 2024-03-22T06:12:57.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 Processor -- start 2024-03-22T06:12:57.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 -- start start 2024-03-22T06:12:57.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f684c101d20 0x7f684c115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f684c1026c0 0x7f684c1160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c116790 con 0x7f684c101d20 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.756+0000 7f6854518700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c10fce0 con 0x7f684c1026c0 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f68522b4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f684c101d20 0x7f684c115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f68522b4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f684c101d20 0x7f684c115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41402/0 (socket says 172.21.15.6:41402) 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f68522b4700 1 -- 172.21.15.6:0/469073161 learned_addr learned my addr 172.21.15.6:0/469073161 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f6851ab3700 1 --2- 172.21.15.6:0/469073161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f684c1026c0 0x7f684c1160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f68522b4700 1 -- 172.21.15.6:0/469073161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f684c1026c0 msgr2=0x7f684c1160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f68522b4700 1 --2- 172.21.15.6:0/469073161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f684c1026c0 0x7f684c1160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f68522b4700 1 -- 172.21.15.6:0/469073161 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848005040 con 0x7f684c101d20 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f68522b4700 1 --2- 172.21.15.6:0/469073161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f684c101d20 0x7f684c115b70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f684800b3b0 tx=0x7f6848009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f6842ffd700 1 -- 172.21.15.6:0/469073161 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684800e040 con 0x7f684c101d20 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f6854518700 1 -- 172.21.15.6:0/469073161 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c10ff00 con 0x7f684c101d20 2024-03-22T06:12:57.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f6842ffd700 1 -- 172.21.15.6:0/469073161 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f684800bbc0 con 0x7f684c101d20 2024-03-22T06:12:57.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f6842ffd700 1 -- 172.21.15.6:0/469073161 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684801c3f0 con 0x7f684c101d20 2024-03-22T06:12:57.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.757+0000 7f6854518700 1 -- 172.21.15.6:0/469073161 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c1104c0 con 0x7f684c101d20 2024-03-22T06:12:57.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.758+0000 7f6854518700 1 -- 172.21.15.6:0/469073161 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68300052b0 con 0x7f684c101d20 2024-03-22T06:12:57.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.759+0000 7f6842ffd700 1 -- 172.21.15.6:0/469073161 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f684800bd30 con 0x7f684c101d20 2024-03-22T06:12:57.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.759+0000 7f6842ffd700 1 --2- 172.21.15.6:0/469073161 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68380692a0 0x7f683806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:12:57.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.759+0000 7f6842ffd700 1 -- 172.21.15.6:0/469073161 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6848088390 con 0x7f684c101d20 2024-03-22T06:12:57.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.759+0000 7f6851ab3700 1 --2- 172.21.15.6:0/469073161 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68380692a0 0x7f683806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:12:57.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.760+0000 7f6851ab3700 1 --2- 172.21.15.6:0/469073161 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68380692a0 0x7f683806b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f683c006fd0 tx=0x7f683c008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:12:57.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:57.764+0000 7f6842ffd700 1 -- 172.21.15.6:0/469073161 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6848058180 con 0x7f684c101d20 2024-03-22T06:12:58.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.004+0000 7f6854518700 1 -- 172.21.15.6:0/469073161 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6830005d70 con 0x7f684c101d20 2024-03-22T06:12:58.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.005+0000 7f6842ffd700 1 -- 172.21.15.6:0/469073161 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+83 (secure 0 0 0) 0x7f6848057d10 con 0x7f684c101d20 2024-03-22T06:12:58.006 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-22T06:12:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.006+0000 7f6854518700 1 -- 172.21.15.6:0/469073161 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68380692a0 msgr2=0x7f683806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.006+0000 7f6854518700 1 --2- 172.21.15.6:0/469073161 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68380692a0 0x7f683806b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f683c006fd0 tx=0x7f683c008040 comp rx=0 tx=0).stop 2024-03-22T06:12:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.006+0000 7f6854518700 1 -- 172.21.15.6:0/469073161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f684c101d20 msgr2=0x7f684c115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:12:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.006+0000 7f6854518700 1 --2- 172.21.15.6:0/469073161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f684c101d20 0x7f684c115b70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f684800b3b0 tx=0x7f6848009e30 comp rx=0 tx=0).stop 2024-03-22T06:12:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.007+0000 7f6854518700 1 -- 172.21.15.6:0/469073161 shutdown_connections 2024-03-22T06:12:58.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.007+0000 7f6854518700 1 --2- 172.21.15.6:0/469073161 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68380692a0 0x7f683806b750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:58.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.007+0000 7f6854518700 1 --2- 172.21.15.6:0/469073161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f684c101d20 0x7f684c115b70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:58.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.007+0000 7f6854518700 1 --2- 172.21.15.6:0/469073161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f684c1026c0 0x7f684c1160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:12:58.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.007+0000 7f6854518700 1 -- 172.21.15.6:0/469073161 >> 172.21.15.6:0/469073161 conn(0x7f684c0f78b0 msgr2=0x7f684c066c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:12:58.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.007+0000 7f6854518700 1 -- 172.21.15.6:0/469073161 shutdown_connections 2024-03-22T06:12:58.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:12:58.007+0000 7f6854518700 1 -- 172.21.15.6:0/469073161 wait complete. 2024-03-22T06:12:58.654 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:58 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/469073161' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T06:12:58.676 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T06:12:58.676 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T06:12:58.676 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-03-22T06:12:58.676 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-22T06:12:58.676 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-22T06:12:58.676 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T06:12:58.676 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T06:12:58.676 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T06:12:58.677 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-22T06:12:58.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:58.677 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-03-22T06:12:58.696 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T06:12:58.697 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-03-22T06:12:58.697 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T06:12:58.697 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-03-22T06:12:58.697 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-22T06:12:58.697 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42161sec preferred_lft 42161sec 2024-03-22T06:12:58.697 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-03-22T06:12:58.697 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-22T06:12:58.697 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T06:12:58.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:58.698 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-03-22T06:12:58.698 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-22T06:12:58.698 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add name ceph-brx type bridge 2024-03-22T06:12:58.698 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr flush dev ceph-brx 2024-03-22T06:12:58.698 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set ceph-brx up 2024-03-22T06:12:58.698 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T06:12:58.698 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-22T06:12:58.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:58 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/469073161' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T06:12:58.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:58.869 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T06:12:58.936 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-03-22T06:12:58.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:58.938 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-03-22T06:12:59.008 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-03-22T06:12:59.008 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T06:12:59.008 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T06:12:59.008 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T06:12:59.008 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T06:12:59.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:59.008 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-03-22T06:12:59.009 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-22T06:12:59.009 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T06:12:59.009 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T06:12:59.009 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T06:12:59.009 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-22T06:12:59.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:59.267 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-22T06:12:59.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:59.281 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-03-22T06:12:59.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:59.336 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-03-22T06:12:59.336 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-22T06:12:59.336 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T06:12:59.336 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-22T06:12:59.336 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-22T06:12:59.420 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-22T06:12:59.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:59.420 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-03-22T06:12:59.420 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-22T06:12:59.420 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-22T06:12:59.421 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T06:12:59.421 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-22T06:12:59.421 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-22T06:12:59.421 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-22T06:12:59.421 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-22T06:12:59.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:59.566 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-03-22T06:12:59.566 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-22T06:12:59.566 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.0 up 2024-03-22T06:12:59.566 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T06:12:59.566 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-22T06:12:59.603 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:12:59 smithi117 ceph-mon[36514]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T06:12:59.608 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T06:12:59.609 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-22T06:12:59.666 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-22T06:12:59.667 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T06:12:59.667 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-22T06:12:59.721 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T06:12:59.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:59.722 DEBUG:teuthology.orchestra.run.smithi117:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T06:12:59.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:12:59 smithi006 ceph-mon[29301]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T06:12:59.839 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T06:12:59.839 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T06:12:59.839 INFO:teuthology.orchestra.run.smithi117.stdout:invalid new device string format 2024-03-22T06:12:59.937 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T06:12:59.937 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T06:12:59.937 INFO:teuthology.orchestra.run.smithi117.stdout:invalid new device string format 2024-03-22T06:12:59.937 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: resolved to: "172.21.15.6:3300,172.21.15.117:3300" 2024-03-22T06:12:59.937 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T06:12:59.942 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T06:12:59.942 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T06:12:59.942 INFO:teuthology.orchestra.run.smithi117.stdout:invalid new device string format 2024-03-22T06:12:59.942 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: resolved to: "172.21.15.6:3300,172.21.15.117:3300" 2024-03-22T06:12:59.942 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T06:12:59.942 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: resolved to: "172.21.15.6:6789,172.21.15.117:6789" 2024-03-22T06:12:59.942 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.6:6789,172.21.15.117:6789:/ 2024-03-22T06:12:59.942 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=fa0e13c2-e811-11ee-95cd-87774f69a715" will pass to kernel 2024-03-22T06:12:59.943 INFO:teuthology.orchestra.run.smithi117.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-22T06:12:59.943 INFO:teuthology.orchestra.run.smithi117.stdout: You just mounted a file system that supports labels which does not 2024-03-22T06:12:59.943 INFO:teuthology.orchestra.run.smithi117.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-22T06:12:59.943 INFO:teuthology.orchestra.run.smithi117.stdout: applications will generate AVC messages and not be allowed access to 2024-03-22T06:12:59.943 INFO:teuthology.orchestra.run.smithi117.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-22T06:12:59.944 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T06:12:59.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:59.945 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-22T06:12:59.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:12:59.971 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T06:13:00.028 INFO:teuthology.orchestra.run.smithi117.stdout:client14528 2024-03-22T06:13:00.028 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:13:00.028 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T06:13:00.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:13:00.084 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T06:13:00.139 INFO:teuthology.orchestra.run.smithi117.stdout:client14528 2024-03-22T06:13:00.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:13:00.140 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T06:13:00.196 INFO:teuthology.orchestra.run.smithi117.stdout:client14528 2024-03-22T06:13:00.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:13:00.196 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715.client14528/status') 2024-03-22T06:13:00.263 INFO:teuthology.orchestra.run.smithi117.stdout:instance: client.14528 (3)172.21.15.117:0/3855819009 2024-03-22T06:13:00.263 INFO:teuthology.orchestra.run.smithi117.stdout:blocklisted: false 2024-03-22T06:13:00.263 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2024-03-22T06:13:00.263 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2024-03-22T06:13:00.264 INFO:teuthology.orchestra.run.smithi117.stderr:72 bytes copied, 2.1688e-05 s, 3.3 MB/s 2024-03-22T06:13:00.265 INFO:teuthology.run_tasks:Running task print... 2024-03-22T06:13:00.275 INFO:teuthology.task.print:**** done client 2024-03-22T06:13:00.275 INFO:teuthology.run_tasks:Running task parallel... 2024-03-22T06:13:00.319 INFO:teuthology.task.parallel:starting parallel... 2024-03-22T06:13:00.319 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-22T06:13:00.320 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T06:13:00.320 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:13:00.320 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-03-22T06:13:00.321 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-22T06:13:00.321 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-22T06:13:00.324 INFO:tasks.workunit:Pulling workunits from ref ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:13:00.325 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-22T06:13:00.325 INFO:tasks.workunit:timeout=3h 2024-03-22T06:13:00.325 INFO:tasks.workunit:cleanup=True 2024-03-22T06:13:00.325 DEBUG:teuthology.orchestra.run.smithi006:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-22T06:13:00.346 INFO:teuthology.orchestra.run.smithi006.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-22T06:13:00.346 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-22T06:13:00.346 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 94h/148d Inode: 1 Links: 2 2024-03-22T06:13:00.346 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T06:13:00.346 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-22T06:13:00.346 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-22T06:13:00.346 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-03-22 06:12:25.406245866 +0000 2024-03-22T06:13:00.346 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-03-22 06:12:55.478915547 +0000 2024-03-22T06:13:00.346 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-03-22 06:12:25.406245866 +0000 2024-03-22T06:13:00.346 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-22T06:13:00.347 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-22T06:13:00.422 DEBUG:teuthology.orchestra.run.smithi117:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-22T06:13:00.438 INFO:teuthology.orchestra.run.smithi117.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-22T06:13:00.438 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-03-22T06:13:00.438 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 73h/115d Inode: 1 Links: 3 2024-03-22T06:13:00.438 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T06:13:00.438 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-22T06:13:00.438 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-22T06:13:00.438 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-22 06:13:00.417866453 +0000 2024-03-22T06:13:00.438 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-22 06:13:00.417866453 +0000 2024-03-22T06:13:00.438 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-22 06:12:25.406245866 +0000 2024-03-22T06:13:00.439 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-22T06:13:00.439 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-22T06:13:00.510 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:13:00.511 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:13:00.526 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-22T06:13:00.566 INFO:tasks.workunit.client.1.smithi117.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-22T06:13:00.614 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:13:00.737 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:00 smithi006 ceph-mon[29301]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T06:13:00.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:00 smithi117 ceph-mon[36514]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T06:13:02.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.141+0000 7fe1d35ba700 1 -- 172.21.15.6:0/1631193702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0fce70 msgr2=0x7fe1cc0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:02.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.141+0000 7fe1d35ba700 1 --2- 172.21.15.6:0/1631193702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0fce70 0x7fe1cc0fd280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8004940 tx=0x7fe1c8009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:02.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 -- 172.21.15.6:0/1631193702 shutdown_connections 2024-03-22T06:13:02.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 --2- 172.21.15.6:0/1631193702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1cc0fdfe0 0x7fe1cc0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:02.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 --2- 172.21.15.6:0/1631193702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0fce70 0x7fe1cc0fd280 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:02.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 -- 172.21.15.6:0/1631193702 >> 172.21.15.6:0/1631193702 conn(0x7fe1cc0f8550 msgr2=0x7fe1cc0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:02.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 -- 172.21.15.6:0/1631193702 shutdown_connections 2024-03-22T06:13:02.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 -- 172.21.15.6:0/1631193702 wait complete. 2024-03-22T06:13:02.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 Processor -- start 2024-03-22T06:13:02.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 -- start start 2024-03-22T06:13:02.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1cc0fce70 0x7fe1cc071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:02.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0fdfe0 0x7fe1cc070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:02.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc072260 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d35ba700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc070850 con 0x7fe1cc0fce70 2024-03-22T06:13:02.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d0b55700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0fdfe0 0x7fe1cc070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.142+0000 7fe1d0b55700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0fdfe0 0x7fe1cc070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41416/0 (socket says 172.21.15.6:41416) 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.143+0000 7fe1d0b55700 1 -- 172.21.15.6:0/2113537101 learned_addr learned my addr 172.21.15.6:0/2113537101 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.143+0000 7fe1d1356700 1 --2- 172.21.15.6:0/2113537101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1cc0fce70 0x7fe1cc071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.143+0000 7fe1d0b55700 1 -- 172.21.15.6:0/2113537101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1cc0fce70 msgr2=0x7fe1cc071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.143+0000 7fe1d0b55700 1 --2- 172.21.15.6:0/2113537101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1cc0fce70 0x7fe1cc071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.143+0000 7fe1d0b55700 1 -- 172.21.15.6:0/2113537101 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c8005040 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.143+0000 7fe1d0b55700 1 --2- 172.21.15.6:0/2113537101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0fdfe0 0x7fe1cc070310 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc002a50 tx=0x7fe1bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.143+0000 7fe1c1ffb700 1 -- 172.21.15.6:0/2113537101 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1bc007cc0 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.143+0000 7fe1c1ffb700 1 -- 172.21.15.6:0/2113537101 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1bc00f040 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.143+0000 7fe1d35ba700 1 -- 172.21.15.6:0/2113537101 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc070b00 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.143+0000 7fe1c1ffb700 1 -- 172.21.15.6:0/2113537101 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1bc00da30 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.143+0000 7fe1d35ba700 1 -- 172.21.15.6:0/2113537101 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc0710c0 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.144+0000 7fe1d35ba700 1 -- 172.21.15.6:0/2113537101 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc04c710 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.145+0000 7fe1c1ffb700 1 -- 172.21.15.6:0/2113537101 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fe1bc00db90 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.145+0000 7fe1c1ffb700 1 --2- 172.21.15.6:0/2113537101 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fe1b80692a0 0x7fe1b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:02.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.145+0000 7fe1d1356700 1 --2- 172.21.15.6:0/2113537101 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fe1b80692a0 0x7fe1b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:02.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.145+0000 7fe1c1ffb700 1 -- 172.21.15.6:0/2113537101 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe1bc086f40 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.146+0000 7fe1d1356700 1 --2- 172.21.15.6:0/2113537101 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fe1b80692a0 0x7fe1b806b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800b970 tx=0x7fe1c8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:02.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.149+0000 7fe1c1ffb700 1 -- 172.21.15.6:0/2113537101 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe1bc056c80 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.344+0000 7fe1d35ba700 1 -- 172.21.15.6:0/2113537101 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fe1cc1058d0 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:02.344+0000 7fe1c1ffb700 1 -- 172.21.15.6:0/2113537101 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7fe1bc056810 con 0x7fe1cc0fdfe0 2024-03-22T06:13:02.346 INFO:teuthology.orchestra.run.smithi006.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-03-22T06:13:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:13:02.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:13:02.964 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T06:13:02.965 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:13:02.965 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-22T06:13:03.268 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:13:03.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:03 smithi006 ceph-mon[29301]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T06:13:03.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:03 smithi117 ceph-mon[36514]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T06:13:04.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.989+0000 7f686e12d700 1 -- 172.21.15.6:0/4200413103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68680fbcd0 msgr2=0x7f68680fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.989+0000 7f686e12d700 1 --2- 172.21.15.6:0/4200413103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68680fbcd0 0x7f68680fc0e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6858004940 tx=0x7f6858009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.989+0000 7f686e12d700 1 -- 172.21.15.6:0/4200413103 shutdown_connections 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.989+0000 7f686e12d700 1 --2- 172.21.15.6:0/4200413103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68680fcf80 0x7f68680fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.989+0000 7f686e12d700 1 --2- 172.21.15.6:0/4200413103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68680fbcd0 0x7f68680fc0e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.989+0000 7f686e12d700 1 -- 172.21.15.6:0/4200413103 >> 172.21.15.6:0/4200413103 conn(0x7f68680f7430 msgr2=0x7f68680f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.989+0000 7f686e12d700 1 -- 172.21.15.6:0/4200413103 shutdown_connections 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.989+0000 7f686e12d700 1 -- 172.21.15.6:0/4200413103 wait complete. 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f686e12d700 1 Processor -- start 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f686e12d700 1 -- start start 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f686e12d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68680fbcd0 0x7f6868110740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f686e12d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68680fcf80 0x7f6868112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f686e12d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6868066350 con 0x7f68680fbcd0 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f686e12d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6868066490 con 0x7f68680fcf80 2024-03-22T06:13:04.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f68677fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68680fbcd0 0x7f6868110740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f68677fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68680fbcd0 0x7f6868110740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38858/0 (socket says 172.21.15.6:38858) 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f68677fe700 1 -- 172.21.15.6:0/3992219872 learned_addr learned my addr 172.21.15.6:0/3992219872 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f6866ffd700 1 --2- 172.21.15.6:0/3992219872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68680fcf80 0x7f6868112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f68677fe700 1 -- 172.21.15.6:0/3992219872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68680fcf80 msgr2=0x7f6868112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f68677fe700 1 --2- 172.21.15.6:0/3992219872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68680fcf80 0x7f6868112c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.990+0000 7f68677fe700 1 -- 172.21.15.6:0/3992219872 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6858005040 con 0x7f68680fbcd0 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.991+0000 7f68677fe700 1 --2- 172.21.15.6:0/3992219872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68680fbcd0 0x7f6868110740 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f685800b3b0 tx=0x7f6858009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.991+0000 7f684ffff700 1 -- 172.21.15.6:0/3992219872 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f685800e040 con 0x7f68680fbcd0 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.991+0000 7f686e12d700 1 -- 172.21.15.6:0/3992219872 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68681131d0 con 0x7f68680fbcd0 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.991+0000 7f684ffff700 1 -- 172.21.15.6:0/3992219872 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f685800bbf0 con 0x7f68680fbcd0 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.991+0000 7f684ffff700 1 -- 172.21.15.6:0/3992219872 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f685801c630 con 0x7f68680fbcd0 2024-03-22T06:13:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.991+0000 7f686e12d700 1 -- 172.21.15.6:0/3992219872 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6868113790 con 0x7f68680fbcd0 2024-03-22T06:13:04.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.992+0000 7f686e12d700 1 -- 172.21.15.6:0/3992219872 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686805f5f0 con 0x7f68680fbcd0 2024-03-22T06:13:04.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.992+0000 7f684ffff700 1 -- 172.21.15.6:0/3992219872 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f685800bd60 con 0x7f68680fbcd0 2024-03-22T06:13:04.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.993+0000 7f684ffff700 1 --2- 172.21.15.6:0/3992219872 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68500692a0 0x7f685006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:04.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.993+0000 7f684ffff700 1 -- 172.21.15.6:0/3992219872 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6858087e40 con 0x7f68680fbcd0 2024-03-22T06:13:04.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.993+0000 7f6866ffd700 1 --2- 172.21.15.6:0/3992219872 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68500692a0 0x7f685006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:04.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.993+0000 7f6866ffd700 1 --2- 172.21.15.6:0/3992219872 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68500692a0 0x7f685006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f685c004f40 tx=0x7f685c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:04.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:04.997+0000 7f684ffff700 1 -- 172.21.15.6:0/3992219872 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6858054170 con 0x7f68680fbcd0 2024-03-22T06:13:05.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.193+0000 7f686e12d700 1 -- 172.21.15.6:0/3992219872 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f686804bed0 con 0x7f68680fbcd0 2024-03-22T06:13:05.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.193+0000 7f684ffff700 1 -- 172.21.15.6:0/3992219872 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f6858057790 con 0x7f68680fbcd0 2024-03-22T06:13:05.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.195+0000 7f686e12d700 1 -- 172.21.15.6:0/3992219872 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68500692a0 msgr2=0x7f685006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.195+0000 7f686e12d700 1 --2- 172.21.15.6:0/3992219872 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68500692a0 0x7f685006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f685c004f40 tx=0x7f685c00b040 comp rx=0 tx=0).stop 2024-03-22T06:13:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.195+0000 7f686e12d700 1 -- 172.21.15.6:0/3992219872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68680fbcd0 msgr2=0x7f6868110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.195+0000 7f686e12d700 1 --2- 172.21.15.6:0/3992219872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68680fbcd0 0x7f6868110740 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f685800b3b0 tx=0x7f6858009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.195+0000 7f686e12d700 1 -- 172.21.15.6:0/3992219872 shutdown_connections 2024-03-22T06:13:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.195+0000 7f686e12d700 1 --2- 172.21.15.6:0/3992219872 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f68500692a0 0x7f685006b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.195+0000 7f686e12d700 1 --2- 172.21.15.6:0/3992219872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68680fbcd0 0x7f6868110740 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.195+0000 7f686e12d700 1 --2- 172.21.15.6:0/3992219872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68680fcf80 0x7f6868112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.195+0000 7f686e12d700 1 -- 172.21.15.6:0/3992219872 >> 172.21.15.6:0/3992219872 conn(0x7f68680f7430 msgr2=0x7f68680f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.195+0000 7f686e12d700 1 -- 172.21.15.6:0/3992219872 shutdown_connections 2024-03-22T06:13:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:05.195+0000 7f686e12d700 1 -- 172.21.15.6:0/3992219872 wait complete. 2024-03-22T06:13:05.301 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:05 smithi006 ceph-mon[29301]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-22T06:13:05.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:05 smithi117 ceph-mon[36514]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-22T06:13:05.913 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-22T06:13:06.177 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:13:07.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:07 smithi117 ceph-mon[36514]: pgmap v201: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T06:13:07.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:07 smithi006 ceph-mon[29301]: pgmap v201: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T06:13:08.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.019+0000 7fece972b700 1 -- 172.21.15.6:0/2628670034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece40fcf80 msgr2=0x7fece40fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:08.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.019+0000 7fece972b700 1 --2- 172.21.15.6:0/2628670034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece40fcf80 0x7fece40fd3f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fecd8004940 tx=0x7fecd8009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:08.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.019+0000 7fece972b700 1 -- 172.21.15.6:0/2628670034 shutdown_connections 2024-03-22T06:13:08.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.019+0000 7fece972b700 1 --2- 172.21.15.6:0/2628670034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece40fcf80 0x7fece40fd3f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:08.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.019+0000 7fece972b700 1 --2- 172.21.15.6:0/2628670034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fece40fbcd0 0x7fece40fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:08.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.019+0000 7fece972b700 1 -- 172.21.15.6:0/2628670034 >> 172.21.15.6:0/2628670034 conn(0x7fece40f7450 msgr2=0x7fece40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:08.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.019+0000 7fece972b700 1 -- 172.21.15.6:0/2628670034 shutdown_connections 2024-03-22T06:13:08.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.019+0000 7fece972b700 1 -- 172.21.15.6:0/2628670034 wait complete. 2024-03-22T06:13:08.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.020+0000 7fece972b700 1 Processor -- start 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.020+0000 7fece972b700 1 -- start start 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.020+0000 7fece972b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece40fbcd0 0x7fece410d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.020+0000 7fece972b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fece40fcf80 0x7fece410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.020+0000 7fece972b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece410e4d0 con 0x7fece40fbcd0 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.020+0000 7fece972b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece410e640 con 0x7fece40fcf80 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.020+0000 7fece2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece40fbcd0 0x7fece410d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.020+0000 7fece2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece40fbcd0 0x7fece410d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38870/0 (socket says 172.21.15.6:38870) 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.020+0000 7fece2ffd700 1 -- 172.21.15.6:0/921153937 learned_addr learned my addr 172.21.15.6:0/921153937 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.020+0000 7fece27fc700 1 --2- 172.21.15.6:0/921153937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fece40fcf80 0x7fece410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.021+0000 7fece2ffd700 1 -- 172.21.15.6:0/921153937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fece40fcf80 msgr2=0x7fece410ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.021+0000 7fece2ffd700 1 --2- 172.21.15.6:0/921153937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fece40fcf80 0x7fece410ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.021+0000 7fece2ffd700 1 -- 172.21.15.6:0/921153937 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd8005040 con 0x7fece40fbcd0 2024-03-22T06:13:08.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.021+0000 7fece2ffd700 1 --2- 172.21.15.6:0/921153937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece40fbcd0 0x7fece410d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fecd4002a50 tx=0x7fecd40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:08.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.021+0000 7feccb7fe700 1 -- 172.21.15.6:0/921153937 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd4007cc0 con 0x7fece40fbcd0 2024-03-22T06:13:08.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.021+0000 7feccb7fe700 1 -- 172.21.15.6:0/921153937 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecd400f040 con 0x7fece40fbcd0 2024-03-22T06:13:08.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.021+0000 7fece972b700 1 -- 172.21.15.6:0/921153937 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece4112ff0 con 0x7fece40fbcd0 2024-03-22T06:13:08.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.021+0000 7feccb7fe700 1 -- 172.21.15.6:0/921153937 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd400da30 con 0x7fece40fbcd0 2024-03-22T06:13:08.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.021+0000 7fece972b700 1 -- 172.21.15.6:0/921153937 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece4113580 con 0x7fece40fbcd0 2024-03-22T06:13:08.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.022+0000 7fece972b700 1 -- 172.21.15.6:0/921153937 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece4101b10 con 0x7fece40fbcd0 2024-03-22T06:13:08.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.022+0000 7feccb7fe700 1 -- 172.21.15.6:0/921153937 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fecd400dbb0 con 0x7fece40fbcd0 2024-03-22T06:13:08.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.023+0000 7feccb7fe700 1 --2- 172.21.15.6:0/921153937 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7feccc0692a0 0x7feccc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:08.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.023+0000 7feccb7fe700 1 -- 172.21.15.6:0/921153937 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fecd4086f40 con 0x7fece40fbcd0 2024-03-22T06:13:08.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.023+0000 7fece27fc700 1 --2- 172.21.15.6:0/921153937 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7feccc0692a0 0x7feccc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:08.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.023+0000 7fece27fc700 1 --2- 172.21.15.6:0/921153937 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7feccc0692a0 0x7feccc06b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fecd800b970 tx=0x7fecd8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:08.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.027+0000 7feccb7fe700 1 -- 172.21.15.6:0/921153937 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fecd4056c80 con 0x7fece40fbcd0 2024-03-22T06:13:08.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.221+0000 7fece972b700 1 -- 172.21.15.6:0/921153937 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fece404c710 con 0x7fece40fbcd0 2024-03-22T06:13:08.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.221+0000 7feccb7fe700 1 -- 172.21.15.6:0/921153937 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7fecd4056810 con 0x7fece40fbcd0 2024-03-22T06:13:08.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.224+0000 7fece972b700 1 -- 172.21.15.6:0/921153937 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7feccc0692a0 msgr2=0x7feccc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:08.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.224+0000 7fece972b700 1 --2- 172.21.15.6:0/921153937 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7feccc0692a0 0x7feccc06b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fecd800b970 tx=0x7fecd8009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:08.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.224+0000 7fece972b700 1 -- 172.21.15.6:0/921153937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece40fbcd0 msgr2=0x7fece410d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:08.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.224+0000 7fece972b700 1 --2- 172.21.15.6:0/921153937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece40fbcd0 0x7fece410d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fecd4002a50 tx=0x7fecd40046b0 comp rx=0 tx=0).stop 2024-03-22T06:13:08.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.224+0000 7fece972b700 1 -- 172.21.15.6:0/921153937 shutdown_connections 2024-03-22T06:13:08.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.224+0000 7fece972b700 1 --2- 172.21.15.6:0/921153937 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7feccc0692a0 0x7feccc06b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:08.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.224+0000 7fece972b700 1 --2- 172.21.15.6:0/921153937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece40fbcd0 0x7fece410d990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:08.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.224+0000 7fece972b700 1 --2- 172.21.15.6:0/921153937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fece40fcf80 0x7fece410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:08.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.224+0000 7fece972b700 1 -- 172.21.15.6:0/921153937 >> 172.21.15.6:0/921153937 conn(0x7fece40f7450 msgr2=0x7fece40f8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:08.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.224+0000 7fece972b700 1 -- 172.21.15.6:0/921153937 shutdown_connections 2024-03-22T06:13:08.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:08.224+0000 7fece972b700 1 -- 172.21.15.6:0/921153937 wait complete. 2024-03-22T06:13:08.838 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-22T06:13:09.101 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:13:09.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:09 smithi117 ceph-mon[36514]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T06:13:09.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:09 smithi006 ceph-mon[29301]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T06:13:10.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.835+0000 7fe2f42d0700 1 -- 172.21.15.6:0/3819261322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2ec0fbcd0 msgr2=0x7fe2ec0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:10.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.835+0000 7fe2f42d0700 1 --2- 172.21.15.6:0/3819261322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2ec0fbcd0 0x7fe2ec0fc0e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4004940 tx=0x7fe2e4009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:10.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.835+0000 7fe2f42d0700 1 -- 172.21.15.6:0/3819261322 shutdown_connections 2024-03-22T06:13:10.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.835+0000 7fe2f42d0700 1 --2- 172.21.15.6:0/3819261322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec0fcf80 0x7fe2ec0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.835+0000 7fe2f42d0700 1 --2- 172.21.15.6:0/3819261322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2ec0fbcd0 0x7fe2ec0fc0e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.835+0000 7fe2f42d0700 1 -- 172.21.15.6:0/3819261322 >> 172.21.15.6:0/3819261322 conn(0x7fe2ec0f7430 msgr2=0x7fe2ec0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.835+0000 7fe2f42d0700 1 -- 172.21.15.6:0/3819261322 shutdown_connections 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.835+0000 7fe2f42d0700 1 -- 172.21.15.6:0/3819261322 wait complete. 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.835+0000 7fe2f42d0700 1 Processor -- start 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.835+0000 7fe2f42d0700 1 -- start start 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f42d0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2ec0fbcd0 0x7fe2ec10c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f42d0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec0fcf80 0x7fe2ec10ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f42d0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec10d4e0 con 0x7fe2ec0fbcd0 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f42d0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec10dcd0 con 0x7fe2ec0fcf80 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f206c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2ec0fbcd0 0x7fe2ec10c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f206c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2ec0fbcd0 0x7fe2ec10c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38888/0 (socket says 172.21.15.6:38888) 2024-03-22T06:13:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f206c700 1 -- 172.21.15.6:0/1480850353 learned_addr learned my addr 172.21.15.6:0/1480850353 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:10.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f186b700 1 --2- 172.21.15.6:0/1480850353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec0fcf80 0x7fe2ec10ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:10.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f206c700 1 -- 172.21.15.6:0/1480850353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec0fcf80 msgr2=0x7fe2ec10ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:10.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f206c700 1 --2- 172.21.15.6:0/1480850353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec0fcf80 0x7fe2ec10ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:10.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f206c700 1 -- 172.21.15.6:0/1480850353 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4005040 con 0x7fe2ec0fbcd0 2024-03-22T06:13:10.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f206c700 1 --2- 172.21.15.6:0/1480850353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2ec0fbcd0 0x7fe2ec10c910 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009150 tx=0x7fe2e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:10.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2eaffd700 1 -- 172.21.15.6:0/1480850353 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e400e040 con 0x7fe2ec0fbcd0 2024-03-22T06:13:10.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2eaffd700 1 -- 172.21.15.6:0/1480850353 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2e400bcc0 con 0x7fe2ec0fbcd0 2024-03-22T06:13:10.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f42d0700 1 -- 172.21.15.6:0/1480850353 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec10df50 con 0x7fe2ec0fbcd0 2024-03-22T06:13:10.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2eaffd700 1 -- 172.21.15.6:0/1480850353 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e401c630 con 0x7fe2ec0fbcd0 2024-03-22T06:13:10.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.836+0000 7fe2f42d0700 1 -- 172.21.15.6:0/1480850353 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec10e510 con 0x7fe2ec0fbcd0 2024-03-22T06:13:10.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.837+0000 7fe2f42d0700 1 -- 172.21.15.6:0/1480850353 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ec05f5f0 con 0x7fe2ec0fbcd0 2024-03-22T06:13:10.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.838+0000 7fe2eaffd700 1 -- 172.21.15.6:0/1480850353 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fe2e4013420 con 0x7fe2ec0fbcd0 2024-03-22T06:13:10.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.838+0000 7fe2eaffd700 1 --2- 172.21.15.6:0/1480850353 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fe2d40692a0 0x7fe2d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:10.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.838+0000 7fe2f186b700 1 --2- 172.21.15.6:0/1480850353 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fe2d40692a0 0x7fe2d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:10.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.838+0000 7fe2eaffd700 1 -- 172.21.15.6:0/1480850353 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe2e4087e60 con 0x7fe2ec0fbcd0 2024-03-22T06:13:10.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.839+0000 7fe2f186b700 1 --2- 172.21.15.6:0/1480850353 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fe2d40692a0 0x7fe2d406b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0006fd0 tx=0x7fe2e0008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:10.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:10.843+0000 7fe2eaffd700 1 -- 172.21.15.6:0/1480850353 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe2e4057cc0 con 0x7fe2ec0fbcd0 2024-03-22T06:13:11.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.040+0000 7fe2f42d0700 1 -- 172.21.15.6:0/1480850353 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fe2ec04bed0 con 0x7fe2ec0fbcd0 2024-03-22T06:13:11.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.040+0000 7fe2eaffd700 1 -- 172.21.15.6:0/1480850353 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fe2e4057850 con 0x7fe2ec0fbcd0 2024-03-22T06:13:11.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.042+0000 7fe2f42d0700 1 -- 172.21.15.6:0/1480850353 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fe2d40692a0 msgr2=0x7fe2d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:11.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.042+0000 7fe2f42d0700 1 --2- 172.21.15.6:0/1480850353 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fe2d40692a0 0x7fe2d406b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0006fd0 tx=0x7fe2e0008040 comp rx=0 tx=0).stop 2024-03-22T06:13:11.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.042+0000 7fe2f42d0700 1 -- 172.21.15.6:0/1480850353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2ec0fbcd0 msgr2=0x7fe2ec10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:11.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.042+0000 7fe2f42d0700 1 --2- 172.21.15.6:0/1480850353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2ec0fbcd0 0x7fe2ec10c910 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009150 tx=0x7fe2e4009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:11.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.042+0000 7fe2f42d0700 1 -- 172.21.15.6:0/1480850353 shutdown_connections 2024-03-22T06:13:11.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.042+0000 7fe2f42d0700 1 --2- 172.21.15.6:0/1480850353 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fe2d40692a0 0x7fe2d406b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:11.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.042+0000 7fe2f42d0700 1 --2- 172.21.15.6:0/1480850353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2ec0fbcd0 0x7fe2ec10c910 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:11.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.042+0000 7fe2f42d0700 1 --2- 172.21.15.6:0/1480850353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec0fcf80 0x7fe2ec10ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:11.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.042+0000 7fe2f42d0700 1 -- 172.21.15.6:0/1480850353 >> 172.21.15.6:0/1480850353 conn(0x7fe2ec0f7430 msgr2=0x7fe2ec0f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:11.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.042+0000 7fe2f42d0700 1 -- 172.21.15.6:0/1480850353 shutdown_connections 2024-03-22T06:13:11.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:11.042+0000 7fe2f42d0700 1 -- 172.21.15.6:0/1480850353 wait complete. 2024-03-22T06:13:11.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:11 smithi117 ceph-mon[36514]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T06:13:11.515 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:11 smithi006 ceph-mon[29301]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T06:13:11.847 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-22T06:13:12.114 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:13:13.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:13 smithi006 ceph-mon[29301]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T06:13:13.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:13 smithi117 ceph-mon[36514]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T06:13:13.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.842+0000 7fb5596c4700 1 -- 172.21.15.6:0/493624626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5540fbce0 msgr2=0x7fb5540fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:13.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.842+0000 7fb5596c4700 1 --2- 172.21.15.6:0/493624626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5540fbce0 0x7fb5540fc0f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb544004940 tx=0x7fb544009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:13.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.842+0000 7fb5596c4700 1 -- 172.21.15.6:0/493624626 shutdown_connections 2024-03-22T06:13:13.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.842+0000 7fb5596c4700 1 --2- 172.21.15.6:0/493624626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5540fcf90 0x7fb5540fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:13.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.842+0000 7fb5596c4700 1 --2- 172.21.15.6:0/493624626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5540fbce0 0x7fb5540fc0f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:13.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.842+0000 7fb5596c4700 1 -- 172.21.15.6:0/493624626 >> 172.21.15.6:0/493624626 conn(0x7fb5540f7440 msgr2=0x7fb5540f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:13.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.842+0000 7fb5596c4700 1 -- 172.21.15.6:0/493624626 shutdown_connections 2024-03-22T06:13:13.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.842+0000 7fb5596c4700 1 -- 172.21.15.6:0/493624626 wait complete. 2024-03-22T06:13:13.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb5596c4700 1 Processor -- start 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb5596c4700 1 -- start start 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb5596c4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5540fbce0 0x7fb554066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb5596c4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5540fcf90 0x7fb554112780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb5596c4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554112cc0 con 0x7fb5540fbce0 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb5596c4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554112e30 con 0x7fb5540fcf90 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb552ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5540fbce0 0x7fb554066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb552ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5540fbce0 0x7fb554066d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34344/0 (socket says 172.21.15.6:34344) 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb552ffd700 1 -- 172.21.15.6:0/2182014241 learned_addr learned my addr 172.21.15.6:0/2182014241 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb552ffd700 1 -- 172.21.15.6:0/2182014241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5540fcf90 msgr2=0x7fb554112780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb552ffd700 1 --2- 172.21.15.6:0/2182014241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5540fcf90 0x7fb554112780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.843+0000 7fb552ffd700 1 -- 172.21.15.6:0/2182014241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544005040 con 0x7fb5540fbce0 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.844+0000 7fb552ffd700 1 --2- 172.21.15.6:0/2182014241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5540fbce0 0x7fb554066d50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb54400b3b0 tx=0x7fb544009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.844+0000 7fb53b7fe700 1 -- 172.21.15.6:0/2182014241 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54400e040 con 0x7fb5540fbce0 2024-03-22T06:13:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.844+0000 7fb53b7fe700 1 -- 172.21.15.6:0/2182014241 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb544004290 con 0x7fb5540fbce0 2024-03-22T06:13:13.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.844+0000 7fb5596c4700 1 -- 172.21.15.6:0/2182014241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5541130b0 con 0x7fb5540fbce0 2024-03-22T06:13:13.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.844+0000 7fb53b7fe700 1 -- 172.21.15.6:0/2182014241 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54401c630 con 0x7fb5540fbce0 2024-03-22T06:13:13.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.844+0000 7fb5596c4700 1 -- 172.21.15.6:0/2182014241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb554113670 con 0x7fb5540fbce0 2024-03-22T06:13:13.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.845+0000 7fb5596c4700 1 -- 172.21.15.6:0/2182014241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55404c710 con 0x7fb5540fbce0 2024-03-22T06:13:13.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.845+0000 7fb53b7fe700 1 -- 172.21.15.6:0/2182014241 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fb54400bbf0 con 0x7fb5540fbce0 2024-03-22T06:13:13.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.845+0000 7fb53b7fe700 1 --2- 172.21.15.6:0/2182014241 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb53c0692a0 0x7fb53c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:13.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.846+0000 7fb53b7fe700 1 -- 172.21.15.6:0/2182014241 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb5440885e0 con 0x7fb5540fbce0 2024-03-22T06:13:13.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.846+0000 7fb5527fc700 1 --2- 172.21.15.6:0/2182014241 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb53c0692a0 0x7fb53c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:13.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.846+0000 7fb5527fc700 1 --2- 172.21.15.6:0/2182014241 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb53c0692a0 0x7fb53c06b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb548006fd0 tx=0x7fb548006db0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:13.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:13.850+0000 7fb53b7fe700 1 -- 172.21.15.6:0/2182014241 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb5440583a0 con 0x7fb5540fbce0 2024-03-22T06:13:14.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.066+0000 7fb5596c4700 1 -- 172.21.15.6:0/2182014241 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:ea01217f8a79678e64ef6a5d68a043a4ef21ca37", "target": ["mon-mgr", ""]}) v1 -- 0x7fb554101230 con 0x7fb53c0692a0 2024-03-22T06:13:14.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.071+0000 7fb53b7fe700 1 -- 172.21.15.6:0/2182014241 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fb554101230 con 0x7fb53c0692a0 2024-03-22T06:13:14.072 INFO:teuthology.orchestra.run.smithi006.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:13:14.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.073+0000 7fb5596c4700 1 -- 172.21.15.6:0/2182014241 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb53c0692a0 msgr2=0x7fb53c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:14.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.073+0000 7fb5596c4700 1 --2- 172.21.15.6:0/2182014241 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb53c0692a0 0x7fb53c06b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb548006fd0 tx=0x7fb548006db0 comp rx=0 tx=0).stop 2024-03-22T06:13:14.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.073+0000 7fb5596c4700 1 -- 172.21.15.6:0/2182014241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5540fbce0 msgr2=0x7fb554066d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:14.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.073+0000 7fb5596c4700 1 --2- 172.21.15.6:0/2182014241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5540fbce0 0x7fb554066d50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb54400b3b0 tx=0x7fb544009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:14.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.073+0000 7fb5596c4700 1 -- 172.21.15.6:0/2182014241 shutdown_connections 2024-03-22T06:13:14.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.073+0000 7fb5596c4700 1 --2- 172.21.15.6:0/2182014241 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb53c0692a0 0x7fb53c06b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:14.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.074+0000 7fb5596c4700 1 --2- 172.21.15.6:0/2182014241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5540fbce0 0x7fb554066d50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:14.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.074+0000 7fb5596c4700 1 --2- 172.21.15.6:0/2182014241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5540fcf90 0x7fb554112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:14.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.074+0000 7fb5596c4700 1 -- 172.21.15.6:0/2182014241 >> 172.21.15.6:0/2182014241 conn(0x7fb5540f7440 msgr2=0x7fb5540f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:14.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.074+0000 7fb5596c4700 1 -- 172.21.15.6:0/2182014241 shutdown_connections 2024-03-22T06:13:14.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:14.074+0000 7fb5596c4700 1 -- 172.21.15.6:0/2182014241 wait complete. 2024-03-22T06:13:14.724 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T06:13:14.725 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:13:14.725 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-22T06:13:15.009 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:13:15.353 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:15 smithi117 ceph-mon[36514]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:ea01217f8a79678e64ef6a5d68a043a4ef21ca37", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:15.354 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:15 smithi117 ceph-mon[36514]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:13:15.354 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:15 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:13:15.354 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:15 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:13:15.354 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:15 smithi117 ceph-mon[36514]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T06:13:15.354 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:15 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:13:15.354 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:15 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:13:15.354 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:15 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:13:15.405 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:15 smithi006 ceph-mon[29301]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:ea01217f8a79678e64ef6a5d68a043a4ef21ca37", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:15.405 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:15 smithi006 ceph-mon[29301]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:13:15.405 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:15 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:13:15.405 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:15 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:13:15.405 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:15 smithi006 ceph-mon[29301]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T06:13:15.405 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:15 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:13:15.405 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:15 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:13:15.405 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:15 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:13:16.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:16 smithi117 ceph-mon[36514]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:13:16.340 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:16 smithi006 ceph-mon[29301]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:ea01217f8a79678e64ef6a5d68a043a4ef21ca37 2024-03-22T06:13:16.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.835+0000 7f46b2157700 1 -- 172.21.15.6:0/3174664444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46ac0fe400 msgr2=0x7f46ac0fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:16.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.835+0000 7f46b2157700 1 --2- 172.21.15.6:0/3174664444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46ac0fe400 0x7f46ac0fe810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f469c004940 tx=0x7f469c009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.835+0000 7f46b2157700 1 -- 172.21.15.6:0/3174664444 shutdown_connections 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.835+0000 7f46b2157700 1 --2- 172.21.15.6:0/3174664444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46ac0fed50 0x7f46ac0fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.835+0000 7f46b2157700 1 --2- 172.21.15.6:0/3174664444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46ac0fe400 0x7f46ac0fe810 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.835+0000 7f46b2157700 1 -- 172.21.15.6:0/3174664444 >> 172.21.15.6:0/3174664444 conn(0x7f46ac0f6ba0 msgr2=0x7f46ac0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.835+0000 7f46b2157700 1 -- 172.21.15.6:0/3174664444 shutdown_connections 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.835+0000 7f46b2157700 1 -- 172.21.15.6:0/3174664444 wait complete. 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.836+0000 7f46b2157700 1 Processor -- start 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.836+0000 7f46b2157700 1 -- start start 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.836+0000 7f46b2157700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46ac0fe400 0x7f46ac10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.836+0000 7f46b2157700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46ac0fed50 0x7f46ac10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.836+0000 7f46b2157700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac10e5d0 con 0x7f46ac0fed50 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.836+0000 7f46b2157700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac10edc0 con 0x7f46ac0fe400 2024-03-22T06:13:16.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.836+0000 7f46aaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46ac0fed50 0x7f46ac10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.836+0000 7f46aaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46ac0fed50 0x7f46ac10df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34356/0 (socket says 172.21.15.6:34356) 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.836+0000 7f46aaffd700 1 -- 172.21.15.6:0/2023398800 learned_addr learned my addr 172.21.15.6:0/2023398800 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.836+0000 7f46ab7fe700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46ac0fe400 0x7f46ac10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.837+0000 7f46aaffd700 1 -- 172.21.15.6:0/2023398800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46ac0fe400 msgr2=0x7f46ac10da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.837+0000 7f46aaffd700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46ac0fe400 0x7f46ac10da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.837+0000 7f46aaffd700 1 -- 172.21.15.6:0/2023398800 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c005040 con 0x7f46ac0fed50 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.837+0000 7f46ab7fe700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46ac0fe400 0x7f46ac10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.837+0000 7f46aaffd700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46ac0fed50 0x7f46ac10df40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f46a000b790 tx=0x7f46a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.837+0000 7f4693fff700 1 -- 172.21.15.6:0/2023398800 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46a000f770 con 0x7f46ac0fed50 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.837+0000 7f46b2157700 1 -- 172.21.15.6:0/2023398800 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ac10f0a0 con 0x7f46ac0fed50 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.837+0000 7f4693fff700 1 -- 172.21.15.6:0/2023398800 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46a000fda0 con 0x7f46ac0fed50 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.837+0000 7f4693fff700 1 -- 172.21.15.6:0/2023398800 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46a000d310 con 0x7f46ac0fed50 2024-03-22T06:13:16.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.837+0000 7f46b2157700 1 -- 172.21.15.6:0/2023398800 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac10f660 con 0x7f46ac0fed50 2024-03-22T06:13:16.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.838+0000 7f46b2157700 1 -- 172.21.15.6:0/2023398800 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ac05f5f0 con 0x7f46ac0fed50 2024-03-22T06:13:16.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.838+0000 7f4693fff700 1 -- 172.21.15.6:0/2023398800 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f46a000d4d0 con 0x7f46ac0fed50 2024-03-22T06:13:16.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.839+0000 7f4693fff700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f46940692b0 0x7f469406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:16.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.839+0000 7f4693fff700 1 -- 172.21.15.6:0/2023398800 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f46a0087200 con 0x7f46ac0fed50 2024-03-22T06:13:16.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.839+0000 7f46ab7fe700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f46940692b0 0x7f469406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:16.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.839+0000 7f46ab7fe700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f46940692b0 0x7f469406b760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f469c00b3b0 tx=0x7f469c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:16.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:16.843+0000 7f4693fff700 1 -- 172.21.15.6:0/2023398800 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f46a0056c10 con 0x7f46ac0fed50 2024-03-22T06:13:17.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.075+0000 7f46b2157700 1 -- 172.21.15.6:0/2023398800 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46ac10f950 con 0x7f46940692b0 2024-03-22T06:13:17.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.076+0000 7f4693fff700 1 -- 172.21.15.6:0/2023398800 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f46ac10f950 con 0x7f46940692b0 2024-03-22T06:13:17.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.078+0000 7f46b2157700 1 -- 172.21.15.6:0/2023398800 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f46940692b0 msgr2=0x7f469406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.078+0000 7f46b2157700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f46940692b0 0x7f469406b760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f469c00b3b0 tx=0x7f469c009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:17.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.078+0000 7f46b2157700 1 -- 172.21.15.6:0/2023398800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46ac0fed50 msgr2=0x7f46ac10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.078+0000 7f46b2157700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46ac0fed50 0x7f46ac10df40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f46a000b790 tx=0x7f46a0009250 comp rx=0 tx=0).stop 2024-03-22T06:13:17.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.078+0000 7f46b2157700 1 -- 172.21.15.6:0/2023398800 shutdown_connections 2024-03-22T06:13:17.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.078+0000 7f46b2157700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46ac0fe400 0x7f46ac10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.078+0000 7f46b2157700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f46940692b0 0x7f469406b760 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.078+0000 7f46b2157700 1 --2- 172.21.15.6:0/2023398800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46ac0fed50 0x7f46ac10df40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.078+0000 7f46b2157700 1 -- 172.21.15.6:0/2023398800 >> 172.21.15.6:0/2023398800 conn(0x7f46ac0f6ba0 msgr2=0x7f46ac0f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:17.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.079+0000 7f46b2157700 1 -- 172.21.15.6:0/2023398800 shutdown_connections 2024-03-22T06:13:17.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.079+0000 7f46b2157700 1 -- 172.21.15.6:0/2023398800 wait complete. 2024-03-22T06:13:17.092 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:13:17.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.202+0000 7fcc6d944700 1 -- 172.21.15.6:0/1806896438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc680fff60 msgr2=0x7fcc68102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.202+0000 7fcc6d944700 1 --2- 172.21.15.6:0/1806896438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc680fff60 0x7fcc68102340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc50004940 tx=0x7fcc50009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.202+0000 7fcc6d944700 1 -- 172.21.15.6:0/1806896438 shutdown_connections 2024-03-22T06:13:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.202+0000 7fcc6d944700 1 --2- 172.21.15.6:0/1806896438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc680fff60 0x7fcc68102340 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.202+0000 7fcc6d944700 1 --2- 172.21.15.6:0/1806896438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc68066280 0x7fcc680ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.202+0000 7fcc6d944700 1 -- 172.21.15.6:0/1806896438 >> 172.21.15.6:0/1806896438 conn(0x7fcc680f7430 msgr2=0x7fcc680f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.202+0000 7fcc6d944700 1 -- 172.21.15.6:0/1806896438 shutdown_connections 2024-03-22T06:13:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.202+0000 7fcc6d944700 1 -- 172.21.15.6:0/1806896438 wait complete. 2024-03-22T06:13:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.203+0000 7fcc6d944700 1 Processor -- start 2024-03-22T06:13:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.203+0000 7fcc6d944700 1 -- start start 2024-03-22T06:13:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.203+0000 7fcc6d944700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc68066280 0x7fcc6810da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.203+0000 7fcc6d944700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc680fff60 0x7fcc6810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.203+0000 7fcc6d944700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6810e590 con 0x7fcc68066280 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.203+0000 7fcc6d944700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6810ede0 con 0x7fcc680fff60 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.203+0000 7fcc66ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc68066280 0x7fcc6810da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.203+0000 7fcc66ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc68066280 0x7fcc6810da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34380/0 (socket says 172.21.15.6:34380) 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.203+0000 7fcc66ffd700 1 -- 172.21.15.6:0/1068618847 learned_addr learned my addr 172.21.15.6:0/1068618847 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.203+0000 7fcc5ffff700 1 --2- 172.21.15.6:0/1068618847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc680fff60 0x7fcc6810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.204+0000 7fcc66ffd700 1 -- 172.21.15.6:0/1068618847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc680fff60 msgr2=0x7fcc6810df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.204+0000 7fcc66ffd700 1 --2- 172.21.15.6:0/1068618847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc680fff60 0x7fcc6810df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.204+0000 7fcc66ffd700 1 -- 172.21.15.6:0/1068618847 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50005040 con 0x7fcc68066280 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.204+0000 7fcc66ffd700 1 --2- 172.21.15.6:0/1068618847 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc68066280 0x7fcc6810da50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcc58002a50 tx=0x7fcc58004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.204+0000 7fcc5f7fe700 1 -- 172.21.15.6:0/1068618847 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc58007cc0 con 0x7fcc68066280 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.204+0000 7fcc5f7fe700 1 -- 172.21.15.6:0/1068618847 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc5800f040 con 0x7fcc68066280 2024-03-22T06:13:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.204+0000 7fcc5f7fe700 1 -- 172.21.15.6:0/1068618847 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc5800da40 con 0x7fcc68066280 2024-03-22T06:13:17.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.204+0000 7fcc6d944700 1 -- 172.21.15.6:0/1068618847 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6810f0c0 con 0x7fcc68066280 2024-03-22T06:13:17.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.204+0000 7fcc6d944700 1 -- 172.21.15.6:0/1068618847 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6810f600 con 0x7fcc68066280 2024-03-22T06:13:17.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.205+0000 7fcc6d944700 1 -- 172.21.15.6:0/1068618847 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6805f5f0 con 0x7fcc68066280 2024-03-22T06:13:17.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.206+0000 7fcc5f7fe700 1 -- 172.21.15.6:0/1068618847 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fcc580097c0 con 0x7fcc68066280 2024-03-22T06:13:17.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.206+0000 7fcc5f7fe700 1 --2- 172.21.15.6:0/1068618847 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcc540692a0 0x7fcc5406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:17.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.206+0000 7fcc5f7fe700 1 -- 172.21.15.6:0/1068618847 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fcc58086e00 con 0x7fcc68066280 2024-03-22T06:13:17.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.206+0000 7fcc5ffff700 1 --2- 172.21.15.6:0/1068618847 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcc540692a0 0x7fcc5406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:17.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.207+0000 7fcc5ffff700 1 --2- 172.21.15.6:0/1068618847 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcc540692a0 0x7fcc5406b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000b3b0 tx=0x7fcc50009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:17.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.211+0000 7fcc5f7fe700 1 -- 172.21.15.6:0/1068618847 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcc58056930 con 0x7fcc68066280 2024-03-22T06:13:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.440+0000 7fcc6d944700 1 -- 172.21.15.6:0/1068618847 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc6810f9d0 con 0x7fcc540692a0 2024-03-22T06:13:17.441 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:17 smithi006 ceph-mon[29301]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:17.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.441+0000 7fcc5f7fe700 1 -- 172.21.15.6:0/1068618847 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fcc6810f9d0 con 0x7fcc540692a0 2024-03-22T06:13:17.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.443+0000 7fcc6d944700 1 -- 172.21.15.6:0/1068618847 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcc540692a0 msgr2=0x7fcc5406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.443+0000 7fcc6d944700 1 --2- 172.21.15.6:0/1068618847 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcc540692a0 0x7fcc5406b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000b3b0 tx=0x7fcc50009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:17.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.443+0000 7fcc6d944700 1 -- 172.21.15.6:0/1068618847 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc68066280 msgr2=0x7fcc6810da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.443+0000 7fcc6d944700 1 --2- 172.21.15.6:0/1068618847 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc68066280 0x7fcc6810da50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcc58002a50 tx=0x7fcc58004600 comp rx=0 tx=0).stop 2024-03-22T06:13:17.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.444+0000 7fcc6d944700 1 -- 172.21.15.6:0/1068618847 shutdown_connections 2024-03-22T06:13:17.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.444+0000 7fcc6d944700 1 --2- 172.21.15.6:0/1068618847 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcc540692a0 0x7fcc5406b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.444+0000 7fcc6d944700 1 --2- 172.21.15.6:0/1068618847 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc68066280 0x7fcc6810da50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.444+0000 7fcc6d944700 1 --2- 172.21.15.6:0/1068618847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc680fff60 0x7fcc6810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.444+0000 7fcc6d944700 1 -- 172.21.15.6:0/1068618847 >> 172.21.15.6:0/1068618847 conn(0x7fcc680f7430 msgr2=0x7fcc680f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:17.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.444+0000 7fcc6d944700 1 -- 172.21.15.6:0/1068618847 shutdown_connections 2024-03-22T06:13:17.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.444+0000 7fcc6d944700 1 -- 172.21.15.6:0/1068618847 wait complete. 2024-03-22T06:13:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:17 smithi117 ceph-mon[36514]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:17.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.553+0000 7f9b5011b700 1 -- 172.21.15.6:0/556877426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b48066280 msgr2=0x7f9b480ff990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.553+0000 7f9b5011b700 1 --2- 172.21.15.6:0/556877426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b48066280 0x7f9b480ff990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9b44004940 tx=0x7f9b44009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:17.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.553+0000 7f9b5011b700 1 -- 172.21.15.6:0/556877426 shutdown_connections 2024-03-22T06:13:17.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.553+0000 7f9b5011b700 1 --2- 172.21.15.6:0/556877426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b480ffed0 0x7f9b481022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.553+0000 7f9b5011b700 1 --2- 172.21.15.6:0/556877426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b48066280 0x7f9b480ff990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.553+0000 7f9b5011b700 1 -- 172.21.15.6:0/556877426 >> 172.21.15.6:0/556877426 conn(0x7f9b480f73c0 msgr2=0x7f9b480f97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:17.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.553+0000 7f9b5011b700 1 -- 172.21.15.6:0/556877426 shutdown_connections 2024-03-22T06:13:17.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.553+0000 7f9b5011b700 1 -- 172.21.15.6:0/556877426 wait complete. 2024-03-22T06:13:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b5011b700 1 Processor -- start 2024-03-22T06:13:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b5011b700 1 -- start start 2024-03-22T06:13:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b5011b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b48066280 0x7f9b4810da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b5011b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480ffed0 0x7f9b4810df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b5011b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4810e560 con 0x7f9b480ffed0 2024-03-22T06:13:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b5011b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4810edb0 con 0x7f9b48066280 2024-03-22T06:13:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b4d6b6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480ffed0 0x7f9b4810df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b4d6b6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480ffed0 0x7f9b4810df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34388/0 (socket says 172.21.15.6:34388) 2024-03-22T06:13:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b4d6b6700 1 -- 172.21.15.6:0/826051342 learned_addr learned my addr 172.21.15.6:0/826051342 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b4deb7700 1 --2- 172.21.15.6:0/826051342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b48066280 0x7f9b4810da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b4d6b6700 1 -- 172.21.15.6:0/826051342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b48066280 msgr2=0x7f9b4810da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b4d6b6700 1 --2- 172.21.15.6:0/826051342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b48066280 0x7f9b4810da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.554+0000 7f9b4d6b6700 1 -- 172.21.15.6:0/826051342 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b44005040 con 0x7f9b480ffed0 2024-03-22T06:13:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.555+0000 7f9b4d6b6700 1 --2- 172.21.15.6:0/826051342 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480ffed0 0x7f9b4810df60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c8c0 tx=0x7f9b380044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.555+0000 7f9b3e7fc700 1 -- 172.21.15.6:0/826051342 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3800a760 con 0x7f9b480ffed0 2024-03-22T06:13:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.555+0000 7f9b5011b700 1 -- 172.21.15.6:0/826051342 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4810f090 con 0x7f9b480ffed0 2024-03-22T06:13:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.555+0000 7f9b3e7fc700 1 -- 172.21.15.6:0/826051342 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b3800ad90 con 0x7f9b480ffed0 2024-03-22T06:13:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.555+0000 7f9b3e7fc700 1 -- 172.21.15.6:0/826051342 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b380113a0 con 0x7f9b480ffed0 2024-03-22T06:13:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.555+0000 7f9b5011b700 1 -- 172.21.15.6:0/826051342 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4810f5d0 con 0x7f9b480ffed0 2024-03-22T06:13:17.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.556+0000 7f9b5011b700 1 -- 172.21.15.6:0/826051342 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4805f5f0 con 0x7f9b480ffed0 2024-03-22T06:13:17.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.556+0000 7f9b3e7fc700 1 -- 172.21.15.6:0/826051342 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f9b38012400 con 0x7f9b480ffed0 2024-03-22T06:13:17.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.557+0000 7f9b3e7fc700 1 --2- 172.21.15.6:0/826051342 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9b340692a0 0x7f9b3406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:17.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.557+0000 7f9b3e7fc700 1 -- 172.21.15.6:0/826051342 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9b38087110 con 0x7f9b480ffed0 2024-03-22T06:13:17.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.557+0000 7f9b4deb7700 1 --2- 172.21.15.6:0/826051342 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9b340692a0 0x7f9b3406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:17.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.558+0000 7f9b4deb7700 1 --2- 172.21.15.6:0/826051342 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9b340692a0 0x7f9b3406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400b970 tx=0x7f9b44009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.562+0000 7f9b3e7fc700 1 -- 172.21.15.6:0/826051342 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9b38056f70 con 0x7f9b480ffed0 2024-03-22T06:13:17.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.778+0000 7f9b5011b700 1 -- 172.21.15.6:0/826051342 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b4810f940 con 0x7f9b340692a0 2024-03-22T06:13:17.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.789+0000 7f9b3e7fc700 1 -- 172.21.15.6:0/826051342 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f9b4810f940 con 0x7f9b340692a0 2024-03-22T06:13:17.789 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (5m) 30s ago 6m 23.6M - 0.25.0 c8568f914cd2 bf3cfbba5c2f 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (6m) 30s ago 6m 6656k - 17.2.7-726-g9c4e0c34 837124da804f a49ab95d42e6 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (5m) 36s ago 5m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0a36af1f8ed0 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 running (4m) 30s ago 6m 84.9M - 9.4.7 954c08fa6188 8b365bc3fd8b 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (53s) 30s ago 52s 14.6M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (48s) 30s ago 47s 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (50s) 36s ago 50s 11.9M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (45s) 36s ago 45s 13.4M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:9283 running (7m) 30s ago 7m 479M - 17.2.7-726-g9c4e0c34 837124da804f 2cea0b1a5c60 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283 running (5m) 36s ago 5m 435M - 17.2.7-726-g9c4e0c34 837124da804f b14c221438cc 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (7m) 30s ago 7m 47.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f b2fe36c6b005 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (5m) 36s ago 5m 37.5M 2048M 17.2.7-726-g9c4e0c34 837124da804f 0559848da9ab 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (6m) 30s ago 6m 16.2M - 1.5.0 0da6a335fe13 0ef65efb379b 2024-03-22T06:13:17.790 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (5m) 36s ago 5m 15.3M - 1.5.0 0da6a335fe13 58a93ac942d3 2024-03-22T06:13:17.791 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (4m) 30s ago 4m 48.5M 4289M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:13:17.791 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (4m) 30s ago 4m 45.1M 4289M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:13:17.791 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (3m) 30s ago 3m 47.2M 4289M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:13:17.791 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (3m) 36s ago 3m 45.5M 5312M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:13:17.791 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (2m) 36s ago 2m 43.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:13:17.791 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (2m) 36s ago 2m 40.8M 5312M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:13:17.791 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (4m) 30s ago 5m 56.6M - 2.43.0 a07b618ecd1d 59d1a24d5af8 2024-03-22T06:13:17.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.791+0000 7f9b5011b700 1 -- 172.21.15.6:0/826051342 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9b340692a0 msgr2=0x7f9b3406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.791+0000 7f9b5011b700 1 --2- 172.21.15.6:0/826051342 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9b340692a0 0x7f9b3406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400b970 tx=0x7f9b44009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:17.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.791+0000 7f9b5011b700 1 -- 172.21.15.6:0/826051342 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480ffed0 msgr2=0x7f9b4810df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.791+0000 7f9b5011b700 1 --2- 172.21.15.6:0/826051342 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480ffed0 0x7f9b4810df60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c8c0 tx=0x7f9b380044c0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.791+0000 7f9b5011b700 1 -- 172.21.15.6:0/826051342 shutdown_connections 2024-03-22T06:13:17.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.791+0000 7f9b5011b700 1 --2- 172.21.15.6:0/826051342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b48066280 0x7f9b4810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.791+0000 7f9b5011b700 1 --2- 172.21.15.6:0/826051342 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9b340692a0 0x7f9b3406b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.791+0000 7f9b5011b700 1 --2- 172.21.15.6:0/826051342 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480ffed0 0x7f9b4810df60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.791+0000 7f9b5011b700 1 -- 172.21.15.6:0/826051342 >> 172.21.15.6:0/826051342 conn(0x7f9b480f73c0 msgr2=0x7f9b480f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:17.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.791+0000 7f9b5011b700 1 -- 172.21.15.6:0/826051342 shutdown_connections 2024-03-22T06:13:17.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.792+0000 7f9b5011b700 1 -- 172.21.15.6:0/826051342 wait complete. 2024-03-22T06:13:17.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.903+0000 7f25ff9b3700 1 -- 172.21.15.6:0/1473490017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f8066280 msgr2=0x7f25f80ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.903+0000 7f25ff9b3700 1 --2- 172.21.15.6:0/1473490017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f8066280 0x7f25f80ffa20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f25f4004940 tx=0x7f25f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:17.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.903+0000 7f25ff9b3700 1 -- 172.21.15.6:0/1473490017 shutdown_connections 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.903+0000 7f25ff9b3700 1 --2- 172.21.15.6:0/1473490017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f80fff60 0x7f25f8102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.903+0000 7f25ff9b3700 1 --2- 172.21.15.6:0/1473490017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f8066280 0x7f25f80ffa20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.903+0000 7f25ff9b3700 1 -- 172.21.15.6:0/1473490017 >> 172.21.15.6:0/1473490017 conn(0x7f25f80f7430 msgr2=0x7f25f80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.903+0000 7f25ff9b3700 1 -- 172.21.15.6:0/1473490017 shutdown_connections 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.903+0000 7f25ff9b3700 1 -- 172.21.15.6:0/1473490017 wait complete. 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25ff9b3700 1 Processor -- start 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25ff9b3700 1 -- start start 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25ff9b3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8066280 0x7f25f80fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25ff9b3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f80fff60 0x7f25f80fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25ff9b3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f80fbed0 con 0x7f25f80fff60 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25ff9b3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f80fc040 con 0x7f25f8066280 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25fcf4e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f80fff60 0x7f25f80fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25fcf4e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f80fff60 0x7f25f80fb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34394/0 (socket says 172.21.15.6:34394) 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25fcf4e700 1 -- 172.21.15.6:0/3725436263 learned_addr learned my addr 172.21.15.6:0/3725436263 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:17.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25fd74f700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8066280 0x7f25f80fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25fcf4e700 1 -- 172.21.15.6:0/3725436263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8066280 msgr2=0x7f25f80fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25fcf4e700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8066280 0x7f25f80fd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25fcf4e700 1 -- 172.21.15.6:0/3725436263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f4005040 con 0x7f25f80fff60 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25fd74f700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8066280 0x7f25f80fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.904+0000 7f25fcf4e700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f80fff60 0x7f25f80fb920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f25e800c8c0 tx=0x7f25e80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.905+0000 7f25edffb700 1 -- 172.21.15.6:0/3725436263 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25e800a980 con 0x7f25f80fff60 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.905+0000 7f25ff9b3700 1 -- 172.21.15.6:0/3725436263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f80fc320 con 0x7f25f80fff60 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.905+0000 7f25edffb700 1 -- 172.21.15.6:0/3725436263 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25e800ce60 con 0x7f25f80fff60 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.905+0000 7f25edffb700 1 -- 172.21.15.6:0/3725436263 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25e80115e0 con 0x7f25f80fff60 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.905+0000 7f25ff9b3700 1 -- 172.21.15.6:0/3725436263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f8115ad0 con 0x7f25f80fff60 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.906+0000 7f25ff9b3700 1 -- 172.21.15.6:0/3725436263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f805f5f0 con 0x7f25f80fff60 2024-03-22T06:13:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.906+0000 7f25edffb700 1 -- 172.21.15.6:0/3725436263 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f25e8012400 con 0x7f25f80fff60 2024-03-22T06:13:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.907+0000 7f25edffb700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f25e40692b0 0x7f25e406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.907+0000 7f25edffb700 1 -- 172.21.15.6:0/3725436263 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f25e8087190 con 0x7f25f80fff60 2024-03-22T06:13:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.907+0000 7f25fd74f700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f25e40692b0 0x7f25e406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.907+0000 7f25fd74f700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f25e40692b0 0x7f25e406b760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f25f4000c00 tx=0x7f25f4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:17.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:17.911+0000 7f25edffb700 1 -- 172.21.15.6:0/3725436263 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f25e8053560 con 0x7f25f80fff60 2024-03-22T06:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.216+0000 7f25ff9b3700 1 -- 172.21.15.6:0/3725436263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f25f8115e10 con 0x7f25f80fff60 2024-03-22T06:13:18.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.216+0000 7f25edffb700 1 -- 172.21.15.6:0/3725436263 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f25e80126b0 con 0x7f25f80fff60 2024-03-22T06:13:18.217 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:13:18.217 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:13:18.217 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T06:13:18.218 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:13:18.218 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:13:18.218 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T06:13:18.218 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:13:18.218 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:13:18.218 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T06:13:18.219 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:13:18.219 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:13:18.219 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:13:18.219 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:13:18.219 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:13:18.219 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-22T06:13:18.220 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:13:18.220 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:13:18.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.218+0000 7f25ff9b3700 1 -- 172.21.15.6:0/3725436263 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f25e40692b0 msgr2=0x7f25e406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:18.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.218+0000 7f25ff9b3700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f25e40692b0 0x7f25e406b760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f25f4000c00 tx=0x7f25f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:18.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.218+0000 7f25ff9b3700 1 -- 172.21.15.6:0/3725436263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f80fff60 msgr2=0x7f25f80fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:18.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.218+0000 7f25ff9b3700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f80fff60 0x7f25f80fb920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f25e800c8c0 tx=0x7f25e80044c0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.218+0000 7f25ff9b3700 1 -- 172.21.15.6:0/3725436263 shutdown_connections 2024-03-22T06:13:18.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.218+0000 7f25ff9b3700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8066280 0x7f25f80fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.218+0000 7f25ff9b3700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f25e40692b0 0x7f25e406b760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.218+0000 7f25ff9b3700 1 --2- 172.21.15.6:0/3725436263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f80fff60 0x7f25f80fb920 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.218+0000 7f25ff9b3700 1 -- 172.21.15.6:0/3725436263 >> 172.21.15.6:0/3725436263 conn(0x7f25f80f7430 msgr2=0x7f25f80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:18.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.218+0000 7f25ff9b3700 1 -- 172.21.15.6:0/3725436263 shutdown_connections 2024-03-22T06:13:18.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.218+0000 7f25ff9b3700 1 -- 172.21.15.6:0/3725436263 wait complete. 2024-03-22T06:13:18.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.325+0000 7fd91c579700 1 -- 172.21.15.6:0/2096270620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd914066af0 msgr2=0x7fd9140ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.325+0000 7fd91c579700 1 --2- 172.21.15.6:0/2096270620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd914066af0 0x7fd9140ffad0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd910004940 tx=0x7fd910009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.325+0000 7fd91c579700 1 -- 172.21.15.6:0/2096270620 shutdown_connections 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.325+0000 7fd91c579700 1 --2- 172.21.15.6:0/2096270620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd914100010 0x7fd9141023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.325+0000 7fd91c579700 1 --2- 172.21.15.6:0/2096270620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd914066af0 0x7fd9140ffad0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.325+0000 7fd91c579700 1 -- 172.21.15.6:0/2096270620 >> 172.21.15.6:0/2096270620 conn(0x7fd9140f74e0 msgr2=0x7fd9140f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.325+0000 7fd91c579700 1 -- 172.21.15.6:0/2096270620 shutdown_connections 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.325+0000 7fd91c579700 1 -- 172.21.15.6:0/2096270620 wait complete. 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.325+0000 7fd91c579700 1 Processor -- start 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.325+0000 7fd91c579700 1 -- start start 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd91c579700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd914066af0 0x7fd91410dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd91c579700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd914100010 0x7fd91410dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd91c579700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91410e5f0 con 0x7fd914100010 2024-03-22T06:13:18.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd91c579700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91410ee40 con 0x7fd914066af0 2024-03-22T06:13:18.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd919b14700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd914100010 0x7fd91410dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:18.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd919b14700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd914100010 0x7fd91410dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34400/0 (socket says 172.21.15.6:34400) 2024-03-22T06:13:18.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd919b14700 1 -- 172.21.15.6:0/2026891306 learned_addr learned my addr 172.21.15.6:0/2026891306 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:18.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd91a315700 1 --2- 172.21.15.6:0/2026891306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd914066af0 0x7fd91410dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:18.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd919b14700 1 -- 172.21.15.6:0/2026891306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd914066af0 msgr2=0x7fd91410dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:18.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd919b14700 1 --2- 172.21.15.6:0/2026891306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd914066af0 0x7fd91410dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd919b14700 1 -- 172.21.15.6:0/2026891306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910005040 con 0x7fd914100010 2024-03-22T06:13:18.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd919b14700 1 --2- 172.21.15.6:0/2026891306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd914100010 0x7fd91410dff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd90400c8c0 tx=0x7fd9040044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:18.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.326+0000 7fd90affd700 1 -- 172.21.15.6:0/2026891306 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd90400a760 con 0x7fd914100010 2024-03-22T06:13:18.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.327+0000 7fd91c579700 1 -- 172.21.15.6:0/2026891306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91410f120 con 0x7fd914100010 2024-03-22T06:13:18.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.327+0000 7fd90affd700 1 -- 172.21.15.6:0/2026891306 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd90400ad90 con 0x7fd914100010 2024-03-22T06:13:18.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.327+0000 7fd90affd700 1 -- 172.21.15.6:0/2026891306 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9040113a0 con 0x7fd914100010 2024-03-22T06:13:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.327+0000 7fd91c579700 1 -- 172.21.15.6:0/2026891306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91410f660 con 0x7fd914100010 2024-03-22T06:13:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.328+0000 7fd91c579700 1 -- 172.21.15.6:0/2026891306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91405f5f0 con 0x7fd914100010 2024-03-22T06:13:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.328+0000 7fd90affd700 1 -- 172.21.15.6:0/2026891306 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fd904012400 con 0x7fd914100010 2024-03-22T06:13:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.328+0000 7fd90affd700 1 --2- 172.21.15.6:0/2026891306 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd9000692a0 0x7fd90006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.329+0000 7fd90affd700 1 -- 172.21.15.6:0/2026891306 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fd9040871a0 con 0x7fd914100010 2024-03-22T06:13:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.329+0000 7fd91a315700 1 --2- 172.21.15.6:0/2026891306 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd9000692a0 0x7fd90006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.329+0000 7fd91a315700 1 --2- 172.21.15.6:0/2026891306 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd9000692a0 0x7fd90006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b970 tx=0x7fd910009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:18.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.333+0000 7fd90affd700 1 -- 172.21.15.6:0/2026891306 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd904056f60 con 0x7fd914100010 2024-03-22T06:13:18.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:18 smithi117 ceph-mon[36514]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:18.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:18 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:13:18.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:18 smithi117 ceph-mon[36514]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:18.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:18 smithi006 ceph-mon[29301]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:18.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:18 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:13:18.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:18 smithi006 ceph-mon[29301]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:18.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.596+0000 7fd91c579700 1 -- 172.21.15.6:0/2026891306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd91404bed0 con 0x7fd914100010 2024-03-22T06:13:18.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.597+0000 7fd90affd700 1 -- 172.21.15.6:0/2026891306 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1805 (secure 0 0 0) 0x7fd904056af0 con 0x7fd914100010 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:13:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:13:18.599 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:13:18.600 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:13:18.600 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:13:18.600 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:13:18.600 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:13:18.600 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:13:18.600 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:13:18.600 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:13:18.600 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:13:18.600 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:13:18.600 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:13:18.600 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:13:18.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.599+0000 7fd91c579700 1 -- 172.21.15.6:0/2026891306 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd9000692a0 msgr2=0x7fd90006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:18.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.599+0000 7fd91c579700 1 --2- 172.21.15.6:0/2026891306 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd9000692a0 0x7fd90006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b970 tx=0x7fd910009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:18.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.599+0000 7fd91c579700 1 -- 172.21.15.6:0/2026891306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd914100010 msgr2=0x7fd91410dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:18.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.599+0000 7fd91c579700 1 --2- 172.21.15.6:0/2026891306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd914100010 0x7fd91410dff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd90400c8c0 tx=0x7fd9040044c0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.599+0000 7fd91c579700 1 -- 172.21.15.6:0/2026891306 shutdown_connections 2024-03-22T06:13:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.599+0000 7fd91c579700 1 --2- 172.21.15.6:0/2026891306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd914066af0 0x7fd91410dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.599+0000 7fd91c579700 1 --2- 172.21.15.6:0/2026891306 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fd9000692a0 0x7fd90006b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.599+0000 7fd91c579700 1 --2- 172.21.15.6:0/2026891306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd914100010 0x7fd91410dff0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.599+0000 7fd91c579700 1 -- 172.21.15.6:0/2026891306 >> 172.21.15.6:0/2026891306 conn(0x7fd9140f74e0 msgr2=0x7fd9140f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.599+0000 7fd91c579700 1 -- 172.21.15.6:0/2026891306 shutdown_connections 2024-03-22T06:13:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.599+0000 7fd91c579700 1 -- 172.21.15.6:0/2026891306 wait complete. 2024-03-22T06:13:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:13:18.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.707+0000 7f559b032700 1 -- 172.21.15.6:0/2119110831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5594066280 msgr2=0x7f55940ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:18.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.707+0000 7f559b032700 1 --2- 172.21.15.6:0/2119110831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5594066280 0x7f55940ffa20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5588004940 tx=0x7f5588009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:18.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.707+0000 7f559b032700 1 -- 172.21.15.6:0/2119110831 shutdown_connections 2024-03-22T06:13:18.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.707+0000 7f559b032700 1 --2- 172.21.15.6:0/2119110831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55940fff60 0x7f5594102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.707+0000 7f559b032700 1 --2- 172.21.15.6:0/2119110831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5594066280 0x7f55940ffa20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.707+0000 7f559b032700 1 -- 172.21.15.6:0/2119110831 >> 172.21.15.6:0/2119110831 conn(0x7f55940f7430 msgr2=0x7f55940f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:18.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.708+0000 7f559b032700 1 -- 172.21.15.6:0/2119110831 shutdown_connections 2024-03-22T06:13:18.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.708+0000 7f559b032700 1 -- 172.21.15.6:0/2119110831 wait complete. 2024-03-22T06:13:18.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.708+0000 7f559b032700 1 Processor -- start 2024-03-22T06:13:18.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.708+0000 7f559b032700 1 -- start start 2024-03-22T06:13:18.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.709+0000 7f559b032700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5594066280 0x7f559410fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:18.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.709+0000 7f559b032700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55940fff60 0x7f5594110280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.709+0000 7f559b032700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5594110880 con 0x7f5594066280 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.709+0000 7f559b032700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5594110fc0 con 0x7f55940fff60 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.709+0000 7f5598dce700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5594066280 0x7f559410fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.709+0000 7f5598dce700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5594066280 0x7f559410fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34422/0 (socket says 172.21.15.6:34422) 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.709+0000 7f5598dce700 1 -- 172.21.15.6:0/3631626075 learned_addr learned my addr 172.21.15.6:0/3631626075 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.709+0000 7f5593fff700 1 --2- 172.21.15.6:0/3631626075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55940fff60 0x7f5594110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.709+0000 7f5598dce700 1 -- 172.21.15.6:0/3631626075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55940fff60 msgr2=0x7f5594110280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.709+0000 7f5598dce700 1 --2- 172.21.15.6:0/3631626075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55940fff60 0x7f5594110280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.709+0000 7f5598dce700 1 -- 172.21.15.6:0/3631626075 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5588005040 con 0x7f5594066280 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.710+0000 7f5598dce700 1 --2- 172.21.15.6:0/3631626075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5594066280 0x7f559410fd40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f558800b3b0 tx=0x7f5588009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.710+0000 7f55917fa700 1 -- 172.21.15.6:0/3631626075 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558800e040 con 0x7f5594066280 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.710+0000 7f55917fa700 1 -- 172.21.15.6:0/3631626075 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f558800bbf0 con 0x7f5594066280 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.710+0000 7f559b032700 1 -- 172.21.15.6:0/3631626075 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5594111240 con 0x7f5594066280 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.710+0000 7f55917fa700 1 -- 172.21.15.6:0/3631626075 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558801c630 con 0x7f5594066280 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.710+0000 7f559b032700 1 -- 172.21.15.6:0/3631626075 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5594111800 con 0x7f5594066280 2024-03-22T06:13:18.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.711+0000 7f559b032700 1 -- 172.21.15.6:0/3631626075 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559405f5f0 con 0x7f5594066280 2024-03-22T06:13:18.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.712+0000 7f55917fa700 1 -- 172.21.15.6:0/3631626075 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f558800bd60 con 0x7f5594066280 2024-03-22T06:13:18.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.712+0000 7f55917fa700 1 --2- 172.21.15.6:0/3631626075 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55840692a0 0x7f558406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:18.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.712+0000 7f55917fa700 1 -- 172.21.15.6:0/3631626075 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5588087e30 con 0x7f5594066280 2024-03-22T06:13:18.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.712+0000 7f5593fff700 1 --2- 172.21.15.6:0/3631626075 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55840692a0 0x7f558406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:18.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.713+0000 7f5593fff700 1 --2- 172.21.15.6:0/3631626075 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55840692a0 0x7f558406b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5580004f40 tx=0x7f558000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:18.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.717+0000 7f55917fa700 1 -- 172.21.15.6:0/3631626075 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5588057bf0 con 0x7f5594066280 2024-03-22T06:13:18.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.941+0000 7f559b032700 1 -- 172.21.15.6:0/3631626075 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5594111c00 con 0x7f55840692a0 2024-03-22T06:13:18.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.942+0000 7f55917fa700 1 -- 172.21.15.6:0/3631626075 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f5594111c00 con 0x7f55840692a0 2024-03-22T06:13:18.943 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:13:18.943 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:ea01217f8a79678e64ef6a5d68a043a4ef21ca37", 2024-03-22T06:13:18.943 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:13:18.943 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:13:18.943 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [], 2024-03-22T06:13:18.943 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "", 2024-03-22T06:13:18.943 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:ea01217f8a79678e64ef6a5d68a043a4ef21ca37 image", 2024-03-22T06:13:18.943 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:13:18.943 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:13:18.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.945+0000 7f559b032700 1 -- 172.21.15.6:0/3631626075 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55840692a0 msgr2=0x7f558406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:18.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.945+0000 7f559b032700 1 --2- 172.21.15.6:0/3631626075 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55840692a0 0x7f558406b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5580004f40 tx=0x7f558000b040 comp rx=0 tx=0).stop 2024-03-22T06:13:18.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.945+0000 7f559b032700 1 -- 172.21.15.6:0/3631626075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5594066280 msgr2=0x7f559410fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:18.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.945+0000 7f559b032700 1 --2- 172.21.15.6:0/3631626075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5594066280 0x7f559410fd40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f558800b3b0 tx=0x7f5588009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:18.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.945+0000 7f559b032700 1 -- 172.21.15.6:0/3631626075 shutdown_connections 2024-03-22T06:13:18.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.945+0000 7f559b032700 1 --2- 172.21.15.6:0/3631626075 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55840692a0 0x7f558406b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.945+0000 7f559b032700 1 --2- 172.21.15.6:0/3631626075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5594066280 0x7f559410fd40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.945+0000 7f559b032700 1 --2- 172.21.15.6:0/3631626075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55940fff60 0x7f5594110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:18.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.945+0000 7f559b032700 1 -- 172.21.15.6:0/3631626075 >> 172.21.15.6:0/3631626075 conn(0x7f55940f7430 msgr2=0x7f55940fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:18.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.945+0000 7f559b032700 1 -- 172.21.15.6:0/3631626075 shutdown_connections 2024-03-22T06:13:18.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:18.945+0000 7f559b032700 1 -- 172.21.15.6:0/3631626075 wait complete. 2024-03-22T06:13:19.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.055+0000 7f0c3e26f700 1 -- 172.21.15.6:0/1873964137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c380fff00 msgr2=0x7f0c38102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:19.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.055+0000 7f0c3e26f700 1 --2- 172.21.15.6:0/1873964137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c380fff00 0x7f0c38102320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c004940 tx=0x7f0c2c009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:19.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.055+0000 7f0c3e26f700 1 -- 172.21.15.6:0/1873964137 shutdown_connections 2024-03-22T06:13:19.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.055+0000 7f0c3e26f700 1 --2- 172.21.15.6:0/1873964137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c380fff00 0x7f0c38102320 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:19.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.055+0000 7f0c3e26f700 1 --2- 172.21.15.6:0/1873964137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c380662e0 0x7f0c380ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:19.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.055+0000 7f0c3e26f700 1 -- 172.21.15.6:0/1873964137 >> 172.21.15.6:0/1873964137 conn(0x7f0c380f7430 msgr2=0x7f0c380f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:19.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.055+0000 7f0c3e26f700 1 -- 172.21.15.6:0/1873964137 shutdown_connections 2024-03-22T06:13:19.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.055+0000 7f0c3e26f700 1 -- 172.21.15.6:0/1873964137 wait complete. 2024-03-22T06:13:19.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.055+0000 7f0c3e26f700 1 Processor -- start 2024-03-22T06:13:19.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c3e26f700 1 -- start start 2024-03-22T06:13:19.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c3e26f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c380662e0 0x7f0c3810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:19.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c3e26f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c380fff00 0x7f0c3810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:19.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c3e26f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3810e5a0 con 0x7f0c380fff00 2024-03-22T06:13:19.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c3e26f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3810edf0 con 0x7f0c380662e0 2024-03-22T06:13:19.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c377fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c380fff00 0x7f0c3810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:19.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c377fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c380fff00 0x7f0c3810dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34442/0 (socket says 172.21.15.6:34442) 2024-03-22T06:13:19.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c377fe700 1 -- 172.21.15.6:0/2069678529 learned_addr learned my addr 172.21.15.6:0/2069678529 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:19.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c37fff700 1 --2- 172.21.15.6:0/2069678529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c380662e0 0x7f0c3810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:19.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c377fe700 1 -- 172.21.15.6:0/2069678529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c380662e0 msgr2=0x7f0c3810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:19.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c377fe700 1 --2- 172.21.15.6:0/2069678529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c380662e0 0x7f0c3810da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:19.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c377fe700 1 -- 172.21.15.6:0/2069678529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c005040 con 0x7f0c380fff00 2024-03-22T06:13:19.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.056+0000 7f0c377fe700 1 --2- 172.21.15.6:0/2069678529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c380fff00 0x7f0c3810dfa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009150 tx=0x7f0c2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:19.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.057+0000 7f0c34ff9700 1 -- 172.21.15.6:0/2069678529 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c2c00e040 con 0x7f0c380fff00 2024-03-22T06:13:19.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.057+0000 7f0c3e26f700 1 -- 172.21.15.6:0/2069678529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3810f070 con 0x7f0c380fff00 2024-03-22T06:13:19.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.057+0000 7f0c34ff9700 1 -- 172.21.15.6:0/2069678529 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c2c004290 con 0x7f0c380fff00 2024-03-22T06:13:19.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.057+0000 7f0c34ff9700 1 -- 172.21.15.6:0/2069678529 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c2c01c630 con 0x7f0c380fff00 2024-03-22T06:13:19.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.057+0000 7f0c3e26f700 1 -- 172.21.15.6:0/2069678529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3810f5b0 con 0x7f0c380fff00 2024-03-22T06:13:19.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.058+0000 7f0c3e26f700 1 -- 172.21.15.6:0/2069678529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3805f5f0 con 0x7f0c380fff00 2024-03-22T06:13:19.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.058+0000 7f0c34ff9700 1 -- 172.21.15.6:0/2069678529 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f0c2c00bbf0 con 0x7f0c380fff00 2024-03-22T06:13:19.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.059+0000 7f0c34ff9700 1 --2- 172.21.15.6:0/2069678529 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c2006d6b0 0x7f0c2006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:19.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.059+0000 7f0c34ff9700 1 -- 172.21.15.6:0/2069678529 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0c2c087d60 con 0x7f0c380fff00 2024-03-22T06:13:19.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.059+0000 7f0c37fff700 1 --2- 172.21.15.6:0/2069678529 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c2006d6b0 0x7f0c2006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:19.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.059+0000 7f0c37fff700 1 --2- 172.21.15.6:0/2069678529 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c2006d6b0 0x7f0c2006fb60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0c38066020 tx=0x7f0c2800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:19.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.063+0000 7f0c34ff9700 1 -- 172.21.15.6:0/2069678529 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0c2c057bc0 con 0x7f0c380fff00 2024-03-22T06:13:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:19 smithi006 ceph-mon[29301]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:19.320 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:19 smithi006 ceph-mon[29301]: pgmap v207: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:19.320 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:19 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3725436263' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:13:19.321 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:19 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2026891306' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:13:19.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.361+0000 7f0c3e26f700 1 -- 172.21.15.6:0/2069678529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0c381058d0 con 0x7f0c380fff00 2024-03-22T06:13:19.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.362+0000 7f0c34ff9700 1 -- 172.21.15.6:0/2069678529 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f0c2c057750 con 0x7f0c380fff00 2024-03-22T06:13:19.363 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_OK 2024-03-22T06:13:19.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.364+0000 7f0c3e26f700 1 -- 172.21.15.6:0/2069678529 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c2006d6b0 msgr2=0x7f0c2006fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:19.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.364+0000 7f0c3e26f700 1 --2- 172.21.15.6:0/2069678529 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c2006d6b0 0x7f0c2006fb60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0c38066020 tx=0x7f0c2800b040 comp rx=0 tx=0).stop 2024-03-22T06:13:19.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.364+0000 7f0c3e26f700 1 -- 172.21.15.6:0/2069678529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c380fff00 msgr2=0x7f0c3810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:19.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.364+0000 7f0c3e26f700 1 --2- 172.21.15.6:0/2069678529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c380fff00 0x7f0c3810dfa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009150 tx=0x7f0c2c009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:19.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.364+0000 7f0c3e26f700 1 -- 172.21.15.6:0/2069678529 shutdown_connections 2024-03-22T06:13:19.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.364+0000 7f0c3e26f700 1 --2- 172.21.15.6:0/2069678529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c380662e0 0x7f0c3810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:19.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.364+0000 7f0c3e26f700 1 --2- 172.21.15.6:0/2069678529 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f0c2006d6b0 0x7f0c2006fb60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:19.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.364+0000 7f0c3e26f700 1 --2- 172.21.15.6:0/2069678529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c380fff00 0x7f0c3810dfa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:19.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.364+0000 7f0c3e26f700 1 -- 172.21.15.6:0/2069678529 >> 172.21.15.6:0/2069678529 conn(0x7f0c380f7430 msgr2=0x7f0c380f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:19.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.364+0000 7f0c3e26f700 1 -- 172.21.15.6:0/2069678529 shutdown_connections 2024-03-22T06:13:19.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:19.364+0000 7f0c3e26f700 1 -- 172.21.15.6:0/2069678529 wait complete. 2024-03-22T06:13:19.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:19 smithi117 ceph-mon[36514]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:19.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:19 smithi117 ceph-mon[36514]: pgmap v207: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:19.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:19 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3725436263' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:13:19.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:19 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2026891306' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:13:20.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:20 smithi117 ceph-mon[36514]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:20.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:20 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2069678529' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:13:20.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:20 smithi006 ceph-mon[29301]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:20.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:20 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2069678529' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:13:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:21 smithi117 ceph-mon[36514]: pgmap v208: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:21.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:21 smithi006 ceph-mon[29301]: pgmap v208: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:23.496 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:23 smithi006 ceph-mon[29301]: pgmap v209: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:23.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:23 smithi117 ceph-mon[36514]: pgmap v209: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:25 smithi117 ceph-mon[36514]: pgmap v210: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:25 smithi006 ceph-mon[29301]: pgmap v210: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:27 smithi117 ceph-mon[36514]: pgmap v211: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:27 smithi006 ceph-mon[29301]: pgmap v211: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:29.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:29 smithi117 ceph-mon[36514]: pgmap v212: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:29.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:29 smithi006 ceph-mon[29301]: pgmap v212: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:31.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:31 smithi117 ceph-mon[36514]: pgmap v213: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:31.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:31 smithi006 ceph-mon[29301]: pgmap v213: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:32.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:32 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:13:32.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:13:33.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:33 smithi117 ceph-mon[36514]: pgmap v214: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:33.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:33 smithi006 ceph-mon[29301]: pgmap v214: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:35.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:35 smithi117 ceph-mon[36514]: pgmap v215: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:35.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:35 smithi006 ceph-mon[29301]: pgmap v215: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:37.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:37 smithi117 ceph-mon[36514]: pgmap v216: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:37.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:37 smithi006 ceph-mon[29301]: pgmap v216: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:39.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:39 smithi117 ceph-mon[36514]: pgmap v217: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T06:13:39.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:39 smithi006 ceph-mon[29301]: pgmap v217: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T06:13:41.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:41 smithi117 ceph-mon[36514]: pgmap v218: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T06:13:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:41 smithi006 ceph-mon[29301]: pgmap v218: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T06:13:43.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:43 smithi117 ceph-mon[36514]: pgmap v219: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T06:13:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:43 smithi006 ceph-mon[29301]: pgmap v219: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T06:13:45.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:45 smithi117 ceph-mon[36514]: pgmap v220: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T06:13:45.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:45 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:13:45.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:45 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:13:45.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:45 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:13:45.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:45 smithi006 ceph-mon[29301]: pgmap v220: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T06:13:45.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:45 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:13:45.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:45 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:13:45.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:45 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:13:46.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:46 smithi117 ceph-mon[36514]: Upgrade: Target is version 19.0.0-2299-gea01217f (unknown) 2024-03-22T06:13:46.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:46 smithi117 ceph-mon[36514]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, digests ['quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9'] 2024-03-22T06:13:46.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:46 smithi117 ceph-mon[36514]: Upgrade: Need to upgrade myself (mgr.smithi006.sfpxdy) 2024-03-22T06:13:46.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:46 smithi117 ceph-mon[36514]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 on smithi117 2024-03-22T06:13:46.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:46 smithi006 ceph-mon[29301]: Upgrade: Target is version 19.0.0-2299-gea01217f (unknown) 2024-03-22T06:13:46.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:46 smithi006 ceph-mon[29301]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, digests ['quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9'] 2024-03-22T06:13:46.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:46 smithi006 ceph-mon[29301]: Upgrade: Need to upgrade myself (mgr.smithi006.sfpxdy) 2024-03-22T06:13:46.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:46 smithi006 ceph-mon[29301]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 on smithi117 2024-03-22T06:13:47.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:47 smithi117 ceph-mon[36514]: pgmap v221: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T06:13:47.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:47 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:13:47.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:47 smithi006 ceph-mon[29301]: pgmap v221: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T06:13:47.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:47 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:13:49.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.472+0000 7f5a0c929700 1 -- 172.21.15.6:0/2886291101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a080fcf90 msgr2=0x7f5a080fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.472+0000 7f5a0c929700 1 --2- 172.21.15.6:0/2886291101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a080fcf90 0x7f5a080fd400 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f59fc004940 tx=0x7f59fc009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.472+0000 7f5a0c929700 1 -- 172.21.15.6:0/2886291101 shutdown_connections 2024-03-22T06:13:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.472+0000 7f5a0c929700 1 --2- 172.21.15.6:0/2886291101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a080fcf90 0x7f5a080fd400 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.472+0000 7f5a0c929700 1 --2- 172.21.15.6:0/2886291101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a080fbce0 0x7f5a080fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.472+0000 7f5a0c929700 1 -- 172.21.15.6:0/2886291101 >> 172.21.15.6:0/2886291101 conn(0x7f5a080f7480 msgr2=0x7f5a080f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.472+0000 7f5a0c929700 1 -- 172.21.15.6:0/2886291101 shutdown_connections 2024-03-22T06:13:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.472+0000 7f5a0c929700 1 -- 172.21.15.6:0/2886291101 wait complete. 2024-03-22T06:13:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a0c929700 1 Processor -- start 2024-03-22T06:13:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a0c929700 1 -- start start 2024-03-22T06:13:49.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a0c929700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a080fbce0 0x7f5a0810fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:49.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a0c929700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a080fcf90 0x7f5a081100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:49.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a0c929700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a081106d0 con 0x7f5a080fcf90 2024-03-22T06:13:49.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a0c929700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08110f20 con 0x7f5a080fbce0 2024-03-22T06:13:49.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a05d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a080fcf90 0x7f5a081100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:49.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a0659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a080fbce0 0x7f5a0810fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:49.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a05d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a080fcf90 0x7f5a081100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55486/0 (socket says 172.21.15.6:55486) 2024-03-22T06:13:49.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a05d9b700 1 -- 172.21.15.6:0/2164886096 learned_addr learned my addr 172.21.15.6:0/2164886096 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a05d9b700 1 -- 172.21.15.6:0/2164886096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a080fbce0 msgr2=0x7f5a0810fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a05d9b700 1 --2- 172.21.15.6:0/2164886096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a080fbce0 0x7f5a0810fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a05d9b700 1 -- 172.21.15.6:0/2164886096 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc005040 con 0x7f5a080fcf90 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.473+0000 7f5a0659c700 1 --2- 172.21.15.6:0/2164886096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a080fbce0 0x7f5a0810fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.474+0000 7f5a05d9b700 1 --2- 172.21.15.6:0/2164886096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a080fcf90 0x7f5a081100d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b3b0 tx=0x7f59fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.474+0000 7f59f6ffd700 1 -- 172.21.15.6:0/2164886096 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59fc00e040 con 0x7f5a080fcf90 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.474+0000 7f59f6ffd700 1 -- 172.21.15.6:0/2164886096 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59fc004290 con 0x7f5a080fcf90 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.474+0000 7f59f6ffd700 1 -- 172.21.15.6:0/2164886096 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59fc01c3f0 con 0x7f5a080fcf90 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.474+0000 7f5a0c929700 1 -- 172.21.15.6:0/2164886096 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a081111a0 con 0x7f5a080fcf90 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.474+0000 7f5a0c929700 1 -- 172.21.15.6:0/2164886096 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a081116e0 con 0x7f5a080fcf90 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.475+0000 7f5a0c929700 1 -- 172.21.15.6:0/2164886096 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a08101920 con 0x7f5a080fcf90 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.476+0000 7f59f6ffd700 1 -- 172.21.15.6:0/2164886096 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f59fc00bbf0 con 0x7f5a080fcf90 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.476+0000 7f59f6ffd700 1 --2- 172.21.15.6:0/2164886096 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f59f00692b0 0x7f59f006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.476+0000 7f59f6ffd700 1 -- 172.21.15.6:0/2164886096 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f59fc0883f0 con 0x7f5a080fcf90 2024-03-22T06:13:49.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.476+0000 7f5a0659c700 1 --2- 172.21.15.6:0/2164886096 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f59f00692b0 0x7f59f006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:49.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.477+0000 7f5a0659c700 1 --2- 172.21.15.6:0/2164886096 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f59f00692b0 0x7f59f006b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009fd0 tx=0x7f59f800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:49.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.481+0000 7f59f6ffd700 1 -- 172.21.15.6:0/2164886096 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f59fc058250 con 0x7f5a080fcf90 2024-03-22T06:13:49.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.702+0000 7f5a0c929700 1 -- 172.21.15.6:0/2164886096 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a08111950 con 0x7f59f00692b0 2024-03-22T06:13:49.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.704+0000 7f59f6ffd700 1 -- 172.21.15.6:0/2164886096 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f5a08111950 con 0x7f59f00692b0 2024-03-22T06:13:49.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.706+0000 7f5a0c929700 1 -- 172.21.15.6:0/2164886096 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f59f00692b0 msgr2=0x7f59f006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:49.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.706+0000 7f5a0c929700 1 --2- 172.21.15.6:0/2164886096 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f59f00692b0 0x7f59f006b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009fd0 tx=0x7f59f800b040 comp rx=0 tx=0).stop 2024-03-22T06:13:49.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.706+0000 7f5a0c929700 1 -- 172.21.15.6:0/2164886096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a080fcf90 msgr2=0x7f5a081100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:49.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.706+0000 7f5a0c929700 1 --2- 172.21.15.6:0/2164886096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a080fcf90 0x7f5a081100d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b3b0 tx=0x7f59fc009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:49.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.706+0000 7f5a0c929700 1 -- 172.21.15.6:0/2164886096 shutdown_connections 2024-03-22T06:13:49.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.706+0000 7f5a0c929700 1 --2- 172.21.15.6:0/2164886096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a080fbce0 0x7f5a0810fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:49.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.706+0000 7f5a0c929700 1 --2- 172.21.15.6:0/2164886096 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f59f00692b0 0x7f59f006b760 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:49.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.706+0000 7f5a0c929700 1 --2- 172.21.15.6:0/2164886096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a080fcf90 0x7f5a081100d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:49.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.706+0000 7f5a0c929700 1 -- 172.21.15.6:0/2164886096 >> 172.21.15.6:0/2164886096 conn(0x7f5a080f7480 msgr2=0x7f5a080f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:49.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.706+0000 7f5a0c929700 1 -- 172.21.15.6:0/2164886096 shutdown_connections 2024-03-22T06:13:49.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.706+0000 7f5a0c929700 1 -- 172.21.15.6:0/2164886096 wait complete. 2024-03-22T06:13:49.717 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:13:49.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:49 smithi006 ceph-mon[29301]: pgmap v222: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.812+0000 7f5594ee8700 1 -- 172.21.15.6:0/450578075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55900fe400 msgr2=0x7f55900fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.812+0000 7f5594ee8700 1 --2- 172.21.15.6:0/450578075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55900fe400 0x7f55900fe810 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5580004940 tx=0x7f5580009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 -- 172.21.15.6:0/450578075 shutdown_connections 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 --2- 172.21.15.6:0/450578075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55900fed50 0x7f55900fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 --2- 172.21.15.6:0/450578075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55900fe400 0x7f55900fe810 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 -- 172.21.15.6:0/450578075 >> 172.21.15.6:0/450578075 conn(0x7f55900f6ba0 msgr2=0x7f55900f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 -- 172.21.15.6:0/450578075 shutdown_connections 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 -- 172.21.15.6:0/450578075 wait complete. 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 Processor -- start 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 -- start start 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55900fe400 0x7f559010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:49.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55900fed50 0x7f559010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559010e4c0 con 0x7f55900fe400 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.813+0000 7f5594ee8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559010ed10 con 0x7f55900fed50 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f558e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55900fe400 0x7f559010d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f558e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55900fe400 0x7f559010d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55502/0 (socket says 172.21.15.6:55502) 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f558e59c700 1 -- 172.21.15.6:0/4200302647 learned_addr learned my addr 172.21.15.6:0/4200302647 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f558dd9b700 1 --2- 172.21.15.6:0/4200302647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55900fed50 0x7f559010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f558e59c700 1 -- 172.21.15.6:0/4200302647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55900fed50 msgr2=0x7f559010dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f558e59c700 1 --2- 172.21.15.6:0/4200302647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55900fed50 0x7f559010dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f558e59c700 1 -- 172.21.15.6:0/4200302647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5580005040 con 0x7f55900fe400 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f558e59c700 1 --2- 172.21.15.6:0/4200302647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55900fe400 0x7f559010d980 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f558000b0d0 tx=0x7f5580009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f557effd700 1 -- 172.21.15.6:0/4200302647 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558000e030 con 0x7f55900fe400 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f5594ee8700 1 -- 172.21.15.6:0/4200302647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559010ef90 con 0x7f55900fe400 2024-03-22T06:13:49.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f557effd700 1 -- 172.21.15.6:0/4200302647 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f558001c400 con 0x7f55900fe400 2024-03-22T06:13:49.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f557effd700 1 -- 172.21.15.6:0/4200302647 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558000b630 con 0x7f55900fe400 2024-03-22T06:13:49.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.814+0000 7f5594ee8700 1 -- 172.21.15.6:0/4200302647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559010f4d0 con 0x7f55900fe400 2024-03-22T06:13:49.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.815+0000 7f5594ee8700 1 -- 172.21.15.6:0/4200302647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559005f5f0 con 0x7f55900fe400 2024-03-22T06:13:49.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.816+0000 7f557effd700 1 -- 172.21.15.6:0/4200302647 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f558001c570 con 0x7f55900fe400 2024-03-22T06:13:49.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.816+0000 7f557effd700 1 --2- 172.21.15.6:0/4200302647 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55780692a0 0x7f557806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:49.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.816+0000 7f557effd700 1 -- 172.21.15.6:0/4200302647 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5580088b50 con 0x7f55900fe400 2024-03-22T06:13:49.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.816+0000 7f558dd9b700 1 --2- 172.21.15.6:0/4200302647 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55780692a0 0x7f557806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:49.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.817+0000 7f558dd9b700 1 --2- 172.21.15.6:0/4200302647 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55780692a0 0x7f557806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5584009b80 tx=0x7f558400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:49.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:49.821+0000 7f557effd700 1 -- 172.21.15.6:0/4200302647 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5580058910 con 0x7f55900fe400 2024-03-22T06:13:50.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:49 smithi117 ceph-mon[36514]: pgmap v222: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T06:13:50.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.035+0000 7f5594ee8700 1 -- 172.21.15.6:0/4200302647 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f559010f850 con 0x7f55780692a0 2024-03-22T06:13:50.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.036+0000 7f557effd700 1 -- 172.21.15.6:0/4200302647 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f559010f850 con 0x7f55780692a0 2024-03-22T06:13:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.038+0000 7f5594ee8700 1 -- 172.21.15.6:0/4200302647 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55780692a0 msgr2=0x7f557806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.038+0000 7f5594ee8700 1 --2- 172.21.15.6:0/4200302647 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55780692a0 0x7f557806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5584009b80 tx=0x7f558400b040 comp rx=0 tx=0).stop 2024-03-22T06:13:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.038+0000 7f5594ee8700 1 -- 172.21.15.6:0/4200302647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55900fe400 msgr2=0x7f559010d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.038+0000 7f5594ee8700 1 --2- 172.21.15.6:0/4200302647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55900fe400 0x7f559010d980 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f558000b0d0 tx=0x7f5580009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.038+0000 7f5594ee8700 1 -- 172.21.15.6:0/4200302647 shutdown_connections 2024-03-22T06:13:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.038+0000 7f5594ee8700 1 --2- 172.21.15.6:0/4200302647 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f55780692a0 0x7f557806b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.038+0000 7f5594ee8700 1 --2- 172.21.15.6:0/4200302647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55900fe400 0x7f559010d980 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.038+0000 7f5594ee8700 1 --2- 172.21.15.6:0/4200302647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55900fed50 0x7f559010dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.038+0000 7f5594ee8700 1 -- 172.21.15.6:0/4200302647 >> 172.21.15.6:0/4200302647 conn(0x7f55900f6ba0 msgr2=0x7f55900f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.039+0000 7f5594ee8700 1 -- 172.21.15.6:0/4200302647 shutdown_connections 2024-03-22T06:13:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.039+0000 7f5594ee8700 1 -- 172.21.15.6:0/4200302647 wait complete. 2024-03-22T06:13:50.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.157+0000 7f84e1c2c700 1 -- 172.21.15.6:0/1093187112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84dc0fff00 msgr2=0x7f84dc102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:50.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.157+0000 7f84e1c2c700 1 --2- 172.21.15.6:0/1093187112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84dc0fff00 0x7f84dc102320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f84d0004940 tx=0x7f84d0009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:50.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.157+0000 7f84e1c2c700 1 -- 172.21.15.6:0/1093187112 shutdown_connections 2024-03-22T06:13:50.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.157+0000 7f84e1c2c700 1 --2- 172.21.15.6:0/1093187112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84dc0fff00 0x7f84dc102320 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.157+0000 7f84e1c2c700 1 --2- 172.21.15.6:0/1093187112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc0662e0 0x7f84dc0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.157+0000 7f84e1c2c700 1 -- 172.21.15.6:0/1093187112 >> 172.21.15.6:0/1093187112 conn(0x7f84dc0f7430 msgr2=0x7f84dc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:50.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.157+0000 7f84e1c2c700 1 -- 172.21.15.6:0/1093187112 shutdown_connections 2024-03-22T06:13:50.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.158+0000 7f84e1c2c700 1 -- 172.21.15.6:0/1093187112 wait complete. 2024-03-22T06:13:50.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.158+0000 7f84e1c2c700 1 Processor -- start 2024-03-22T06:13:50.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.158+0000 7f84e1c2c700 1 -- start start 2024-03-22T06:13:50.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84e1c2c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc0662e0 0x7f84dc10da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:50.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84e1c2c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84dc0fff00 0x7f84dc10df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:50.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84e1c2c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc10e580 con 0x7f84dc0fff00 2024-03-22T06:13:50.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84e1c2c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc10ed80 con 0x7f84dc0662e0 2024-03-22T06:13:50.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84daffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84dc0fff00 0x7f84dc10df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84daffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84dc0fff00 0x7f84dc10df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55522/0 (socket says 172.21.15.6:55522) 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84daffd700 1 -- 172.21.15.6:0/3635648197 learned_addr learned my addr 172.21.15.6:0/3635648197 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84db7fe700 1 --2- 172.21.15.6:0/3635648197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc0662e0 0x7f84dc10da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84daffd700 1 -- 172.21.15.6:0/3635648197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc0662e0 msgr2=0x7f84dc10da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84daffd700 1 --2- 172.21.15.6:0/3635648197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc0662e0 0x7f84dc10da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84daffd700 1 -- 172.21.15.6:0/3635648197 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d0005040 con 0x7f84dc0fff00 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.159+0000 7f84daffd700 1 --2- 172.21.15.6:0/3635648197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84dc0fff00 0x7f84dc10df80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f84d0013870 tx=0x7f84d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.160+0000 7f84c3fff700 1 -- 172.21.15.6:0/3635648197 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84d0021070 con 0x7f84dc0fff00 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.160+0000 7f84c3fff700 1 -- 172.21.15.6:0/3635648197 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84d0009430 con 0x7f84dc0fff00 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.160+0000 7f84e1c2c700 1 -- 172.21.15.6:0/3635648197 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84dc10f000 con 0x7f84dc0fff00 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.160+0000 7f84c3fff700 1 -- 172.21.15.6:0/3635648197 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84d000e070 con 0x7f84dc0fff00 2024-03-22T06:13:50.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.160+0000 7f84e1c2c700 1 -- 172.21.15.6:0/3635648197 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84dc10f540 con 0x7f84dc0fff00 2024-03-22T06:13:50.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.161+0000 7f84e1c2c700 1 -- 172.21.15.6:0/3635648197 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84dc05f5f0 con 0x7f84dc0fff00 2024-03-22T06:13:50.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.161+0000 7f84c3fff700 1 -- 172.21.15.6:0/3635648197 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f84d0026410 con 0x7f84dc0fff00 2024-03-22T06:13:50.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.162+0000 7f84c3fff700 1 --2- 172.21.15.6:0/3635648197 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f84c40692a0 0x7f84c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:50.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.162+0000 7f84c3fff700 1 -- 172.21.15.6:0/3635648197 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f84d0088330 con 0x7f84dc0fff00 2024-03-22T06:13:50.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.162+0000 7f84db7fe700 1 --2- 172.21.15.6:0/3635648197 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f84c40692a0 0x7f84c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:50.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.162+0000 7f84db7fe700 1 --2- 172.21.15.6:0/3635648197 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f84c40692a0 0x7f84c406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009b80 tx=0x7f84cc00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:50.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.170+0000 7f84c3fff700 1 -- 172.21.15.6:0/3635648197 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f84d0058200 con 0x7f84dc0fff00 2024-03-22T06:13:50.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.444+0000 7f84e1c2c700 1 -- 172.21.15.6:0/3635648197 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f84dc10f820 con 0x7f84c40692a0 2024-03-22T06:13:50.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.461+0000 7f84c3fff700 1 -- 172.21.15.6:0/3635648197 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f84dc10f820 con 0x7f84c40692a0 2024-03-22T06:13:50.462 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:13:50.462 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (5m) 63s ago 6m 23.6M - 0.25.0 c8568f914cd2 bf3cfbba5c2f 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (7m) 63s ago 7m 6656k - 17.2.7-726-g9c4e0c34 837124da804f a49ab95d42e6 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (6m) 68s ago 6m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0a36af1f8ed0 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 running (5m) 63s ago 6m 84.9M - 9.4.7 954c08fa6188 8b365bc3fd8b 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (86s) 63s ago 85s 14.6M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (80s) 63s ago 80s 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (83s) 68s ago 83s 11.9M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (78s) 68s ago 78s 13.4M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:9283 running (8m) 63s ago 8m 479M - 17.2.7-726-g9c4e0c34 837124da804f 2cea0b1a5c60 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283 running (6m) 68s ago 5m 435M - 17.2.7-726-g9c4e0c34 837124da804f b14c221438cc 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (8m) 63s ago 8m 47.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f b2fe36c6b005 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (5m) 68s ago 5m 37.5M 2048M 17.2.7-726-g9c4e0c34 837124da804f 0559848da9ab 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (7m) 63s ago 7m 16.2M - 1.5.0 0da6a335fe13 0ef65efb379b 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (6m) 68s ago 6m 15.3M - 1.5.0 0da6a335fe13 58a93ac942d3 2024-03-22T06:13:50.463 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (5m) 63s ago 5m 48.5M 4289M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:13:50.464 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (4m) 63s ago 4m 45.1M 4289M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:13:50.464 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (4m) 63s ago 4m 47.2M 4289M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:13:50.464 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (3m) 68s ago 3m 45.5M 5312M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:13:50.464 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (3m) 68s ago 3m 43.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:13:50.464 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (2m) 68s ago 2m 40.8M 5312M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:13:50.464 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (5m) 63s ago 6m 56.6M - 2.43.0 a07b618ecd1d 59d1a24d5af8 2024-03-22T06:13:50.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.464+0000 7f84e1c2c700 1 -- 172.21.15.6:0/3635648197 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f84c40692a0 msgr2=0x7f84c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.464+0000 7f84e1c2c700 1 --2- 172.21.15.6:0/3635648197 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f84c40692a0 0x7f84c406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009b80 tx=0x7f84cc00b040 comp rx=0 tx=0).stop 2024-03-22T06:13:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.464+0000 7f84e1c2c700 1 -- 172.21.15.6:0/3635648197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84dc0fff00 msgr2=0x7f84dc10df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.464+0000 7f84e1c2c700 1 --2- 172.21.15.6:0/3635648197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84dc0fff00 0x7f84dc10df80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f84d0013870 tx=0x7f84d0009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.464+0000 7f84e1c2c700 1 -- 172.21.15.6:0/3635648197 shutdown_connections 2024-03-22T06:13:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.464+0000 7f84e1c2c700 1 --2- 172.21.15.6:0/3635648197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc0662e0 0x7f84dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.464+0000 7f84e1c2c700 1 --2- 172.21.15.6:0/3635648197 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f84c40692a0 0x7f84c406b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.464+0000 7f84e1c2c700 1 --2- 172.21.15.6:0/3635648197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84dc0fff00 0x7f84dc10df80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.464+0000 7f84e1c2c700 1 -- 172.21.15.6:0/3635648197 >> 172.21.15.6:0/3635648197 conn(0x7f84dc0f7430 msgr2=0x7f84dc0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.464+0000 7f84e1c2c700 1 -- 172.21.15.6:0/3635648197 shutdown_connections 2024-03-22T06:13:50.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.464+0000 7f84e1c2c700 1 -- 172.21.15.6:0/3635648197 wait complete. 2024-03-22T06:13:50.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.604+0000 7fec63808700 1 -- 172.21.15.6:0/1853415054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec5c066b70 msgr2=0x7fec5c0ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:50.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.604+0000 7fec63808700 1 --2- 172.21.15.6:0/1853415054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec5c066b70 0x7fec5c0ff9d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fec58004940 tx=0x7fec58009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:50.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.604+0000 7fec63808700 1 -- 172.21.15.6:0/1853415054 shutdown_connections 2024-03-22T06:13:50.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.604+0000 7fec63808700 1 --2- 172.21.15.6:0/1853415054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c0fff10 0x7fec5c102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.604+0000 7fec63808700 1 --2- 172.21.15.6:0/1853415054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec5c066b70 0x7fec5c0ff9d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.604+0000 7fec63808700 1 -- 172.21.15.6:0/1853415054 >> 172.21.15.6:0/1853415054 conn(0x7fec5c0f7440 msgr2=0x7fec5c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:50.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.605+0000 7fec63808700 1 -- 172.21.15.6:0/1853415054 shutdown_connections 2024-03-22T06:13:50.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.605+0000 7fec63808700 1 -- 172.21.15.6:0/1853415054 wait complete. 2024-03-22T06:13:50.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.605+0000 7fec63808700 1 Processor -- start 2024-03-22T06:13:50.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.605+0000 7fec63808700 1 -- start start 2024-03-22T06:13:50.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.606+0000 7fec63808700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec5c066b70 0x7fec5c0fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:50.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.606+0000 7fec63808700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c0fff10 0x7fec5c0fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:50.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.606+0000 7fec63808700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c0fbe80 con 0x7fec5c066b70 2024-03-22T06:13:50.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.606+0000 7fec63808700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c0fbff0 con 0x7fec5c0fff10 2024-03-22T06:13:50.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.606+0000 7fec615a4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec5c066b70 0x7fec5c0fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:50.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.606+0000 7fec615a4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec5c066b70 0x7fec5c0fd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55542/0 (socket says 172.21.15.6:55542) 2024-03-22T06:13:50.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.606+0000 7fec615a4700 1 -- 172.21.15.6:0/3660847747 learned_addr learned my addr 172.21.15.6:0/3660847747 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:50.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.606+0000 7fec60da3700 1 --2- 172.21.15.6:0/3660847747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c0fff10 0x7fec5c0fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:50.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.607+0000 7fec615a4700 1 -- 172.21.15.6:0/3660847747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c0fff10 msgr2=0x7fec5c0fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:50.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.607+0000 7fec615a4700 1 --2- 172.21.15.6:0/3660847747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c0fff10 0x7fec5c0fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:50.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.607+0000 7fec615a4700 1 -- 172.21.15.6:0/3660847747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec58005040 con 0x7fec5c066b70 2024-03-22T06:13:50.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.607+0000 7fec615a4700 1 --2- 172.21.15.6:0/3660847747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec5c066b70 0x7fec5c0fd400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fec58000c00 tx=0x7fec58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:50.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.607+0000 7fec51ffb700 1 -- 172.21.15.6:0/3660847747 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec5800e030 con 0x7fec5c066b70 2024-03-22T06:13:50.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.607+0000 7fec51ffb700 1 -- 172.21.15.6:0/3660847747 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec58013de0 con 0x7fec5c066b70 2024-03-22T06:13:50.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.607+0000 7fec51ffb700 1 -- 172.21.15.6:0/3660847747 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec5801ce50 con 0x7fec5c066b70 2024-03-22T06:13:50.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.607+0000 7fec63808700 1 -- 172.21.15.6:0/3660847747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5c0fc270 con 0x7fec5c066b70 2024-03-22T06:13:50.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.608+0000 7fec63808700 1 -- 172.21.15.6:0/3660847747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5c0fc6f0 con 0x7fec5c066b70 2024-03-22T06:13:50.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.609+0000 7fec51ffb700 1 -- 172.21.15.6:0/3660847747 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fec5801c400 con 0x7fec5c066b70 2024-03-22T06:13:50.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.609+0000 7fec63808700 1 -- 172.21.15.6:0/3660847747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5c05f5f0 con 0x7fec5c066b70 2024-03-22T06:13:50.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.609+0000 7fec51ffb700 1 --2- 172.21.15.6:0/3660847747 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fec480692a0 0x7fec4806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:50.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.610+0000 7fec51ffb700 1 -- 172.21.15.6:0/3660847747 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fec58088b30 con 0x7fec5c066b70 2024-03-22T06:13:50.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.610+0000 7fec60da3700 1 --2- 172.21.15.6:0/3660847747 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fec480692a0 0x7fec4806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:50.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.610+0000 7fec60da3700 1 --2- 172.21.15.6:0/3660847747 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fec480692a0 0x7fec4806b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fec4c004fd0 tx=0x7fec4c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:50.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:50.614+0000 7fec51ffb700 1 -- 172.21.15.6:0/3660847747 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fec58054e60 con 0x7fec5c066b70 2024-03-22T06:13:51.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.039+0000 7fec63808700 1 -- 172.21.15.6:0/3660847747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec5c115ae0 con 0x7fec5c066b70 2024-03-22T06:13:51.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.040+0000 7fec51ffb700 1 -- 172.21.15.6:0/3660847747 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fec58058480 con 0x7fec5c066b70 2024-03-22T06:13:51.041 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:13:51.041 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:13:51.041 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T06:13:51.041 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:13:51.041 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:13:51.041 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T06:13:51.041 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:13:51.041 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:13:51.041 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T06:13:51.042 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:13:51.042 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:13:51.042 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:13:51.042 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:13:51.042 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:13:51.042 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-22T06:13:51.042 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:13:51.042 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:13:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.042+0000 7fec63808700 1 -- 172.21.15.6:0/3660847747 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fec480692a0 msgr2=0x7fec4806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.042+0000 7fec63808700 1 --2- 172.21.15.6:0/3660847747 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fec480692a0 0x7fec4806b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fec4c004fd0 tx=0x7fec4c00b040 comp rx=0 tx=0).stop 2024-03-22T06:13:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.042+0000 7fec63808700 1 -- 172.21.15.6:0/3660847747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec5c066b70 msgr2=0x7fec5c0fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.042+0000 7fec63808700 1 --2- 172.21.15.6:0/3660847747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec5c066b70 0x7fec5c0fd400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fec58000c00 tx=0x7fec58009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.042+0000 7fec63808700 1 -- 172.21.15.6:0/3660847747 shutdown_connections 2024-03-22T06:13:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.042+0000 7fec63808700 1 --2- 172.21.15.6:0/3660847747 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fec480692a0 0x7fec4806b750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.042+0000 7fec63808700 1 --2- 172.21.15.6:0/3660847747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec5c066b70 0x7fec5c0fd400 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.042+0000 7fec63808700 1 --2- 172.21.15.6:0/3660847747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c0fff10 0x7fec5c0fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.042+0000 7fec63808700 1 -- 172.21.15.6:0/3660847747 >> 172.21.15.6:0/3660847747 conn(0x7fec5c0f7440 msgr2=0x7fec5c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.042+0000 7fec63808700 1 -- 172.21.15.6:0/3660847747 shutdown_connections 2024-03-22T06:13:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.042+0000 7fec63808700 1 -- 172.21.15.6:0/3660847747 wait complete. 2024-03-22T06:13:51.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.168+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/2827930118 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80fcee0 msgr2=0x7fcdf80fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:51.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.168+0000 7fcdfdc7b700 1 --2- 172.21.15.6:0/2827930118 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80fcee0 0x7fcdf80fd310 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcde8004940 tx=0x7fcde8009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:51.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.168+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/2827930118 shutdown_connections 2024-03-22T06:13:51.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.168+0000 7fcdfdc7b700 1 --2- 172.21.15.6:0/2827930118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf80fe0d0 0x7fcdf80fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.168+0000 7fcdfdc7b700 1 --2- 172.21.15.6:0/2827930118 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80fcee0 0x7fcdf80fd310 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.168+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/2827930118 >> 172.21.15.6:0/2827930118 conn(0x7fcdf80f8560 msgr2=0x7fcdf80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:51.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.169+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/2827930118 shutdown_connections 2024-03-22T06:13:51.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.169+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/2827930118 wait complete. 2024-03-22T06:13:51.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.169+0000 7fcdfdc7b700 1 Processor -- start 2024-03-22T06:13:51.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.169+0000 7fcdfdc7b700 1 -- start start 2024-03-22T06:13:51.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.169+0000 7fcdfdc7b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf80fcee0 0x7fcdf8071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:51.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdfdc7b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80fe0d0 0x7fcdf8070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:51.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdfdc7b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80708c0 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdfdc7b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8070a30 con 0x7fcdf80fcee0 2024-03-22T06:13:51.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdf6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80fe0d0 0x7fcdf8070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:51.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdf6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80fe0d0 0x7fcdf8070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55556/0 (socket says 172.21.15.6:55556) 2024-03-22T06:13:51.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdf77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf80fcee0 0x7fcdf8071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:51.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdf6ffd700 1 -- 172.21.15.6:0/4105099162 learned_addr learned my addr 172.21.15.6:0/4105099162 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:51.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdf6ffd700 1 -- 172.21.15.6:0/4105099162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf80fcee0 msgr2=0x7fcdf8071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:51.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdf6ffd700 1 --2- 172.21.15.6:0/4105099162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf80fcee0 0x7fcdf8071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdf6ffd700 1 -- 172.21.15.6:0/4105099162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde8005040 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdf77fe700 1 --2- 172.21.15.6:0/4105099162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf80fcee0 0x7fcdf8071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:13:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcdf6ffd700 1 --2- 172.21.15.6:0/4105099162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80fe0d0 0x7fcdf8070310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcdec002a80 tx=0x7fcdec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.170+0000 7fcddffff700 1 -- 172.21.15.6:0/4105099162 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdec007cc0 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.171+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/4105099162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf8070d10 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.171+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/4105099162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf8115a00 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.171+0000 7fcddffff700 1 -- 172.21.15.6:0/4105099162 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdec00f040 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.171+0000 7fcddffff700 1 -- 172.21.15.6:0/4105099162 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdec00dac0 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.172+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/4105099162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf805f5f0 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.172+0000 7fcddffff700 1 -- 172.21.15.6:0/4105099162 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fcdec00dcc0 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.173+0000 7fcddffff700 1 --2- 172.21.15.6:0/4105099162 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcde00692a0 0x7fcde006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.173+0000 7fcddffff700 1 -- 172.21.15.6:0/4105099162 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fcdec086f40 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.173+0000 7fcdf77fe700 1 --2- 172.21.15.6:0/4105099162 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcde00692a0 0x7fcde006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:51.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.174+0000 7fcdf77fe700 1 --2- 172.21.15.6:0/4105099162 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcde00692a0 0x7fcde006b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcde8004890 tx=0x7fcde8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:51.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.178+0000 7fcddffff700 1 -- 172.21.15.6:0/4105099162 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcdec0531f0 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.420+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/4105099162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcdf8115d70 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.420+0000 7fcddffff700 1 -- 172.21.15.6:0/4105099162 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1805 (secure 0 0 0) 0x7fcdec056810 con 0x7fcdf80fe0d0 2024-03-22T06:13:51.421 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:13:51.421 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:13:51.422 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:13:51.423 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:13:51.424 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:13:51.424 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:13:51.424 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:13:51.424 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:13:51.424 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:13:51.424 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:13:51.424 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:13:51.424 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:13:51.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.424+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/4105099162 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcde00692a0 msgr2=0x7fcde006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:51.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.424+0000 7fcdfdc7b700 1 --2- 172.21.15.6:0/4105099162 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcde00692a0 0x7fcde006b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcde8004890 tx=0x7fcde8009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:51.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.424+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/4105099162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80fe0d0 msgr2=0x7fcdf8070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:51.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.424+0000 7fcdfdc7b700 1 --2- 172.21.15.6:0/4105099162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80fe0d0 0x7fcdf8070310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcdec002a80 tx=0x7fcdec0046b0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.424+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/4105099162 shutdown_connections 2024-03-22T06:13:51.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.424+0000 7fcdfdc7b700 1 --2- 172.21.15.6:0/4105099162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf80fcee0 0x7fcdf8071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.424+0000 7fcdfdc7b700 1 --2- 172.21.15.6:0/4105099162 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fcde00692a0 0x7fcde006b750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.424+0000 7fcdfdc7b700 1 --2- 172.21.15.6:0/4105099162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80fe0d0 0x7fcdf8070310 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.424+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/4105099162 >> 172.21.15.6:0/4105099162 conn(0x7fcdf80f8560 msgr2=0x7fcdf80f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:51.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.424+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/4105099162 shutdown_connections 2024-03-22T06:13:51.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.424+0000 7fcdfdc7b700 1 -- 172.21.15.6:0/4105099162 wait complete. 2024-03-22T06:13:51.426 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:13:51.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.539+0000 7f9139921700 1 -- 172.21.15.6:0/3755208473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340fe040 msgr2=0x7f91340fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:51.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.539+0000 7f9139921700 1 --2- 172.21.15.6:0/3755208473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340fe040 0x7f91340fe4b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9128004940 tx=0x7f9128009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:51.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.539+0000 7f9139921700 1 -- 172.21.15.6:0/3755208473 shutdown_connections 2024-03-22T06:13:51.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.539+0000 7f9139921700 1 --2- 172.21.15.6:0/3755208473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340fe040 0x7f91340fe4b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.539+0000 7f9139921700 1 --2- 172.21.15.6:0/3755208473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91340fce40 0x7f91340fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.539+0000 7f9139921700 1 -- 172.21.15.6:0/3755208473 >> 172.21.15.6:0/3755208473 conn(0x7f91340f8550 msgr2=0x7f91340fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:51.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.539+0000 7f9139921700 1 -- 172.21.15.6:0/3755208473 shutdown_connections 2024-03-22T06:13:51.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.539+0000 7f9139921700 1 -- 172.21.15.6:0/3755208473 wait complete. 2024-03-22T06:13:51.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.540+0000 7f9139921700 1 Processor -- start 2024-03-22T06:13:51.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.540+0000 7f9139921700 1 -- start start 2024-03-22T06:13:51.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.540+0000 7f9139921700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340fce40 0x7f913410d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:51.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.540+0000 7f9139921700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91340fe040 0x7f913410df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:51.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.540+0000 7f9139921700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913410e510 con 0x7f91340fce40 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.540+0000 7f9139921700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913410ed60 con 0x7f91340fe040 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.540+0000 7f9132ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340fce40 0x7f913410d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.540+0000 7f9132ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340fce40 0x7f913410d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55566/0 (socket says 172.21.15.6:55566) 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.540+0000 7f9132ffd700 1 -- 172.21.15.6:0/1325108651 learned_addr learned my addr 172.21.15.6:0/1325108651 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.541+0000 7f91327fc700 1 --2- 172.21.15.6:0/1325108651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91340fe040 0x7f913410df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.541+0000 7f9132ffd700 1 -- 172.21.15.6:0/1325108651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91340fe040 msgr2=0x7f913410df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.541+0000 7f9132ffd700 1 --2- 172.21.15.6:0/1325108651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91340fe040 0x7f913410df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.541+0000 7f9132ffd700 1 -- 172.21.15.6:0/1325108651 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128005040 con 0x7f91340fce40 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.541+0000 7f9132ffd700 1 --2- 172.21.15.6:0/1325108651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340fce40 0x7f913410d9d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f912400eb70 tx=0x7f912400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.541+0000 7f911b7fe700 1 -- 172.21.15.6:0/1325108651 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9124007cc0 con 0x7f91340fce40 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.541+0000 7f911b7fe700 1 -- 172.21.15.6:0/1325108651 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f912400f040 con 0x7f91340fce40 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.541+0000 7f911b7fe700 1 -- 172.21.15.6:0/1325108651 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912400cb20 con 0x7f91340fce40 2024-03-22T06:13:51.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.542+0000 7f9139921700 1 -- 172.21.15.6:0/1325108651 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913410efe0 con 0x7f91340fce40 2024-03-22T06:13:51.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.542+0000 7f9139921700 1 -- 172.21.15.6:0/1325108651 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913410f460 con 0x7f91340fce40 2024-03-22T06:13:51.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.543+0000 7f9139921700 1 -- 172.21.15.6:0/1325108651 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913405f5f0 con 0x7f91340fce40 2024-03-22T06:13:51.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.543+0000 7f911b7fe700 1 -- 172.21.15.6:0/1325108651 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f912400cc80 con 0x7f91340fce40 2024-03-22T06:13:51.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.544+0000 7f911b7fe700 1 --2- 172.21.15.6:0/1325108651 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f911c0692a0 0x7f911c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:51.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.544+0000 7f911b7fe700 1 -- 172.21.15.6:0/1325108651 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9124086b30 con 0x7f91340fce40 2024-03-22T06:13:51.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.544+0000 7f91327fc700 1 --2- 172.21.15.6:0/1325108651 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f911c0692a0 0x7f911c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:51.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.545+0000 7f91327fc700 1 --2- 172.21.15.6:0/1325108651 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f911c0692a0 0x7f911c06b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9128009b30 tx=0x7f9128009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:51.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.552+0000 7f911b7fe700 1 -- 172.21.15.6:0/1325108651 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f912404e190 con 0x7f91340fce40 2024-03-22T06:13:51.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:51 smithi006 ceph-mon[29301]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:51.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:51 smithi006 ceph-mon[29301]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:51.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:51 smithi006 ceph-mon[29301]: pgmap v223: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:51.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:51 smithi006 ceph-mon[29301]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:51.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:51 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3660847747' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:13:51.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:51 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/4105099162' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:13:51.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.857+0000 7f9139921700 1 -- 172.21.15.6:0/1325108651 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f913410f8a0 con 0x7f911c0692a0 2024-03-22T06:13:51.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.858+0000 7f911b7fe700 1 -- 172.21.15.6:0/1325108651 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f913410f8a0 con 0x7f911c0692a0 2024-03-22T06:13:51.859 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:13:51.859 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:13:51.859 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:13:51.859 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:13:51.859 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [], 2024-03-22T06:13:51.859 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "0/21 daemons upgraded", 2024-03-22T06:13:51.859 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 image on host smithi117", 2024-03-22T06:13:51.860 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:13:51.860 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:13:51.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.861+0000 7f9139921700 1 -- 172.21.15.6:0/1325108651 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f911c0692a0 msgr2=0x7f911c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:51.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.861+0000 7f9139921700 1 --2- 172.21.15.6:0/1325108651 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f911c0692a0 0x7f911c06b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9128009b30 tx=0x7f9128009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:51.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.861+0000 7f9139921700 1 -- 172.21.15.6:0/1325108651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340fce40 msgr2=0x7f913410d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:51.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.861+0000 7f9139921700 1 --2- 172.21.15.6:0/1325108651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340fce40 0x7f913410d9d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f912400eb70 tx=0x7f912400b040 comp rx=0 tx=0).stop 2024-03-22T06:13:51.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.861+0000 7f9139921700 1 -- 172.21.15.6:0/1325108651 shutdown_connections 2024-03-22T06:13:51.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.861+0000 7f9139921700 1 --2- 172.21.15.6:0/1325108651 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f911c0692a0 0x7f911c06b750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.861+0000 7f9139921700 1 --2- 172.21.15.6:0/1325108651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340fce40 0x7f913410d9d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.861+0000 7f9139921700 1 --2- 172.21.15.6:0/1325108651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91340fe040 0x7f913410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:51.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.861+0000 7f9139921700 1 -- 172.21.15.6:0/1325108651 >> 172.21.15.6:0/1325108651 conn(0x7f91340f8550 msgr2=0x7f91340f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:51.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.861+0000 7f9139921700 1 -- 172.21.15.6:0/1325108651 shutdown_connections 2024-03-22T06:13:51.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:51.862+0000 7f9139921700 1 -- 172.21.15.6:0/1325108651 wait complete. 2024-03-22T06:13:52.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:51 smithi117 ceph-mon[36514]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:51 smithi117 ceph-mon[36514]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:51 smithi117 ceph-mon[36514]: pgmap v223: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:51 smithi117 ceph-mon[36514]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:51 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3660847747' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:13:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:51 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/4105099162' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:13:52.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.016+0000 7fa9a2b76700 1 -- 172.21.15.6:0/3699442356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa99c0fe040 msgr2=0x7fa99c0fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:52.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.016+0000 7fa9a2b76700 1 --2- 172.21.15.6:0/3699442356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa99c0fe040 0x7fa99c0fe4b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa988004940 tx=0x7fa988009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:52.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.016+0000 7fa9a2b76700 1 -- 172.21.15.6:0/3699442356 shutdown_connections 2024-03-22T06:13:52.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.016+0000 7fa9a2b76700 1 --2- 172.21.15.6:0/3699442356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa99c0fe040 0x7fa99c0fe4b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:52.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.016+0000 7fa9a2b76700 1 --2- 172.21.15.6:0/3699442356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa99c0fce40 0x7fa99c0fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:52.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.016+0000 7fa9a2b76700 1 -- 172.21.15.6:0/3699442356 >> 172.21.15.6:0/3699442356 conn(0x7fa99c0f8550 msgr2=0x7fa99c0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:52.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.016+0000 7fa9a2b76700 1 -- 172.21.15.6:0/3699442356 shutdown_connections 2024-03-22T06:13:52.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.016+0000 7fa9a2b76700 1 -- 172.21.15.6:0/3699442356 wait complete. 2024-03-22T06:13:52.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.016+0000 7fa9a2b76700 1 Processor -- start 2024-03-22T06:13:52.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.016+0000 7fa9a2b76700 1 -- start start 2024-03-22T06:13:52.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa9a2b76700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa99c0fce40 0x7fa99c10d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:52.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa9a2b76700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa99c0fe040 0x7fa99c10df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:52.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa9a2b76700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c10e510 con 0x7fa99c0fe040 2024-03-22T06:13:52.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa9a2b76700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c10ed60 con 0x7fa99c0fce40 2024-03-22T06:13:52.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa99bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa99c0fe040 0x7fa99c10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:52.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa99bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa99c0fe040 0x7fa99c10df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55578/0 (socket says 172.21.15.6:55578) 2024-03-22T06:13:52.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa99bfff700 1 -- 172.21.15.6:0/2748927138 learned_addr learned my addr 172.21.15.6:0/2748927138 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:13:52.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa9a0912700 1 --2- 172.21.15.6:0/2748927138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa99c0fce40 0x7fa99c10d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:52.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa99bfff700 1 -- 172.21.15.6:0/2748927138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa99c0fce40 msgr2=0x7fa99c10d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:52.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa99bfff700 1 --2- 172.21.15.6:0/2748927138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa99c0fce40 0x7fa99c10d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:52.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa99bfff700 1 -- 172.21.15.6:0/2748927138 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa988005040 con 0x7fa99c0fe040 2024-03-22T06:13:52.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.017+0000 7fa99bfff700 1 --2- 172.21.15.6:0/2748927138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa99c0fe040 0x7fa99c10df10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa98800b0d0 tx=0x7fa988009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:52.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.018+0000 7fa9997fa700 1 -- 172.21.15.6:0/2748927138 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa98800e070 con 0x7fa99c0fe040 2024-03-22T06:13:52.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.018+0000 7fa9997fa700 1 -- 172.21.15.6:0/2748927138 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa98800b410 con 0x7fa99c0fe040 2024-03-22T06:13:52.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.018+0000 7fa9a2b76700 1 -- 172.21.15.6:0/2748927138 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c10efe0 con 0x7fa99c0fe040 2024-03-22T06:13:52.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.018+0000 7fa9997fa700 1 -- 172.21.15.6:0/2748927138 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa988012600 con 0x7fa99c0fe040 2024-03-22T06:13:52.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.018+0000 7fa9a2b76700 1 -- 172.21.15.6:0/2748927138 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c10f520 con 0x7fa99c0fe040 2024-03-22T06:13:52.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.019+0000 7fa9997fa700 1 -- 172.21.15.6:0/2748927138 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fa988013af0 con 0x7fa99c0fe040 2024-03-22T06:13:52.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.020+0000 7fa9a2b76700 1 -- 172.21.15.6:0/2748927138 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99c05f5f0 con 0x7fa99c0fe040 2024-03-22T06:13:52.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.020+0000 7fa9997fa700 1 --2- 172.21.15.6:0/2748927138 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa98c069380 0x7fa98c06b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:13:52.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.020+0000 7fa9a0912700 1 --2- 172.21.15.6:0/2748927138 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa98c069380 0x7fa98c06b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:13:52.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.021+0000 7fa9a0912700 1 --2- 172.21.15.6:0/2748927138 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa98c069380 0x7fa98c06b830 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa990004600 tx=0x7fa99000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:13:52.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.021+0000 7fa9997fa700 1 -- 172.21.15.6:0/2748927138 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa988087e50 con 0x7fa99c0fe040 2024-03-22T06:13:52.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.025+0000 7fa9997fa700 1 -- 172.21.15.6:0/2748927138 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa988057ca0 con 0x7fa99c0fe040 2024-03-22T06:13:52.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.379+0000 7fa9a2b76700 1 -- 172.21.15.6:0/2748927138 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa99c10f830 con 0x7fa99c0fe040 2024-03-22T06:13:52.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.379+0000 7fa9997fa700 1 -- 172.21.15.6:0/2748927138 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa988022020 con 0x7fa99c0fe040 2024-03-22T06:13:52.380 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_OK 2024-03-22T06:13:52.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.381+0000 7fa9a2b76700 1 -- 172.21.15.6:0/2748927138 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa98c069380 msgr2=0x7fa98c06b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:52.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.381+0000 7fa9a2b76700 1 --2- 172.21.15.6:0/2748927138 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa98c069380 0x7fa98c06b830 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa990004600 tx=0x7fa99000b040 comp rx=0 tx=0).stop 2024-03-22T06:13:52.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.381+0000 7fa9a2b76700 1 -- 172.21.15.6:0/2748927138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa99c0fe040 msgr2=0x7fa99c10df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:13:52.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.381+0000 7fa9a2b76700 1 --2- 172.21.15.6:0/2748927138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa99c0fe040 0x7fa99c10df10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa98800b0d0 tx=0x7fa988009e30 comp rx=0 tx=0).stop 2024-03-22T06:13:52.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.381+0000 7fa9a2b76700 1 -- 172.21.15.6:0/2748927138 shutdown_connections 2024-03-22T06:13:52.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.381+0000 7fa9a2b76700 1 --2- 172.21.15.6:0/2748927138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa99c0fce40 0x7fa99c10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:52.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.381+0000 7fa9a2b76700 1 --2- 172.21.15.6:0/2748927138 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fa98c069380 0x7fa98c06b830 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:52.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.381+0000 7fa9a2b76700 1 --2- 172.21.15.6:0/2748927138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa99c0fe040 0x7fa99c10df10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:13:52.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.381+0000 7fa9a2b76700 1 -- 172.21.15.6:0/2748927138 >> 172.21.15.6:0/2748927138 conn(0x7fa99c0f8550 msgr2=0x7fa99c0f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:13:52.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.382+0000 7fa9a2b76700 1 -- 172.21.15.6:0/2748927138 shutdown_connections 2024-03-22T06:13:52.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:13:52.382+0000 7fa9a2b76700 1 -- 172.21.15.6:0/2748927138 wait complete. 2024-03-22T06:13:52.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:52 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2748927138' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:13:53.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:52 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2748927138' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:13:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:53 smithi006 ceph-mon[29301]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:53 smithi006 ceph-mon[29301]: pgmap v224: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:54.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:53 smithi117 ceph-mon[36514]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:13:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:53 smithi117 ceph-mon[36514]: pgmap v224: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:54.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:54 smithi006 ceph-mon[29301]: pgmap v225: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:55.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:54 smithi117 ceph-mon[36514]: pgmap v225: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:57.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:57 smithi117 ceph-mon[36514]: pgmap v226: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:57 smithi006 ceph-mon[29301]: pgmap v226: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T06:13:59.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:13:59 smithi117 ceph-mon[36514]: pgmap v227: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:13:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:13:59 smithi006 ceph-mon[29301]: pgmap v227: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:01.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:01 smithi117 ceph-mon[36514]: pgmap v228: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:01.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:01 smithi006 ceph-mon[29301]: pgmap v228: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:02.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:02 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:14:02.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:02 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:14:03.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:03 smithi117 ceph-mon[36514]: pgmap v229: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:03.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:03 smithi006 ceph-mon[29301]: pgmap v229: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:05.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:05 smithi117 ceph-mon[36514]: pgmap v230: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:05.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:05 smithi006 ceph-mon[29301]: pgmap v230: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:07.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:07 smithi117 ceph-mon[36514]: pgmap v231: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:07.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:07 smithi006 ceph-mon[29301]: pgmap v231: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:09.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:09 smithi117 ceph-mon[36514]: pgmap v232: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:09.534 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:09 smithi006 ceph-mon[29301]: pgmap v232: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:11.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:11 smithi117 ceph-mon[36514]: pgmap v233: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:11.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:11 smithi006 ceph-mon[29301]: pgmap v233: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:13.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:13 smithi006 ceph-mon[29301]: pgmap v234: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:13.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:13 smithi117 ceph-mon[36514]: pgmap v234: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:15.379 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:15 smithi117 ceph-mon[36514]: pgmap v235: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:15.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:15 smithi006 ceph-mon[29301]: pgmap v235: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:17.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:17 smithi117 ceph-mon[36514]: Upgrade: Updating mgr.smithi117.bqeskk 2024-03-22T06:14:17.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:17.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:14:17.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:14:17.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:17 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:14:17.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:17 smithi117 ceph-mon[36514]: Deploying daemon mgr.smithi117.bqeskk on smithi117 2024-03-22T06:14:17.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:17 smithi117 ceph-mon[36514]: pgmap v236: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:17 smithi006 ceph-mon[29301]: Upgrade: Updating mgr.smithi117.bqeskk 2024-03-22T06:14:17.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:17.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:14:17.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:14:17.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:17 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:14:17.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:17 smithi006 ceph-mon[29301]: Deploying daemon mgr.smithi117.bqeskk on smithi117 2024-03-22T06:14:17.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:17 smithi006 ceph-mon[29301]: pgmap v236: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:18.415 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:18 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:14:18.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:18 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:14:19.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:19 smithi117 ceph-mon[36514]: pgmap v237: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:19.433 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:19 smithi006 ceph-mon[29301]: pgmap v237: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:19.892 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 94% (11657/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-22T06:14:20.655 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to 'ea01217f8a79678e64ef6a5d68a043a4ef21ca37'. 2024-03-22T06:14:20.655 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-22T06:14:20.656 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-22T06:14:20.657 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-22T06:14:20.657 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at ea01217f8a7 Merge branch 'jjm-cephadm-smb-svc-spec' of https://github.com/phlogistonjohn/ceph into wip-phlogistonjohn-testing-2024-03-21-1835 2024-03-22T06:14:20.660 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-22T06:14:20.688 INFO:tasks.workunit.client.0.smithi006.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-22T06:14:20.690 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-22T06:14:20.690 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-22T06:14:21.484 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-22T06:14:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:21 smithi117 ceph-mon[36514]: pgmap v238: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:21 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:14:21.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:21 smithi006 ceph-mon[29301]: pgmap v238: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:21.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:21.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:21.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:21 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:14:21.553 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-22T06:14:21.592 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-22T06:14:21.594 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-22T06:14:21.594 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-22T06:14:21.632 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-22T06:14:21.635 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:14:21.635 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-22T06:14:21.690 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-22T06:14:21.693 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-22T06:14:21.693 DEBUG:teuthology.orchestra.run.smithi006:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-22T06:14:21.751 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir -p fsstress 2024-03-22T06:14:21.752 INFO:tasks.workunit.client.0.smithi006.stderr:+ pushd fsstress 2024-03-22T06:14:21.752 INFO:tasks.workunit.client.0.smithi006.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T06:14:21.753 INFO:tasks.workunit.client.0.smithi006.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-22T06:14:22.431 INFO:tasks.workunit.client.0.smithi006.stderr:+ tar xzf ltp-full.tgz 2024-03-22T06:14:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.492+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1853422377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf00662e0 msgr2=0x7fedf00ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.492+0000 7fedf5e3a700 1 --2- 172.21.15.6:0/1853422377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf00662e0 0x7fedf00ff9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fede0004940 tx=0x7fede0009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.492+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1853422377 shutdown_connections 2024-03-22T06:14:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.492+0000 7fedf5e3a700 1 --2- 172.21.15.6:0/1853422377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf00fff00 0x7fedf0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.492+0000 7fedf5e3a700 1 --2- 172.21.15.6:0/1853422377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf00662e0 0x7fedf00ff9c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.492+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1853422377 >> 172.21.15.6:0/1853422377 conn(0x7fedf00f7430 msgr2=0x7fedf00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.492+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1853422377 shutdown_connections 2024-03-22T06:14:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.492+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1853422377 wait complete. 2024-03-22T06:14:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.493+0000 7fedf5e3a700 1 Processor -- start 2024-03-22T06:14:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.493+0000 7fedf5e3a700 1 -- start start 2024-03-22T06:14:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.493+0000 7fedf5e3a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf00662e0 0x7fedf010fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.493+0000 7fedf5e3a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf00fff00 0x7fedf01101d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.493+0000 7fedf5e3a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf01107d0 con 0x7fedf00fff00 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.493+0000 7fedf5e3a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0110fd0 con 0x7fedf00662e0 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.493+0000 7fedeeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf00fff00 0x7fedf01101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.493+0000 7fedeeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf00fff00 0x7fedf01101d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40500/0 (socket says 172.21.15.6:40500) 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.493+0000 7fedeeffd700 1 -- 172.21.15.6:0/1915157793 learned_addr learned my addr 172.21.15.6:0/1915157793 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.494+0000 7fedeeffd700 1 -- 172.21.15.6:0/1915157793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf00662e0 msgr2=0x7fedf010fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.494+0000 7fedeeffd700 1 --2- 172.21.15.6:0/1915157793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf00662e0 0x7fedf010fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.494+0000 7fedeeffd700 1 -- 172.21.15.6:0/1915157793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede0005040 con 0x7fedf00fff00 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.494+0000 7fedeeffd700 1 --2- 172.21.15.6:0/1915157793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf00fff00 0x7fedf01101d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fede4002a50 tx=0x7fede40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.494+0000 7fedd7fff700 1 -- 172.21.15.6:0/1915157793 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede4007cc0 con 0x7fedf00fff00 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.494+0000 7fedd7fff700 1 -- 172.21.15.6:0/1915157793 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fede400f040 con 0x7fedf00fff00 2024-03-22T06:14:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.494+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1915157793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf01112b0 con 0x7fedf00fff00 2024-03-22T06:14:22.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.494+0000 7fedd7fff700 1 -- 172.21.15.6:0/1915157793 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede4009640 con 0x7fedf00fff00 2024-03-22T06:14:22.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.494+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1915157793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf01117f0 con 0x7fedf00fff00 2024-03-22T06:14:22.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.495+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1915157793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf005f5f0 con 0x7fedf00fff00 2024-03-22T06:14:22.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.496+0000 7fedd7fff700 1 -- 172.21.15.6:0/1915157793 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fede40097a0 con 0x7fedf00fff00 2024-03-22T06:14:22.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.496+0000 7fedd7fff700 1 --2- 172.21.15.6:0/1915157793 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fedd80692a0 0x7fedd806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:22.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.497+0000 7fedd7fff700 1 -- 172.21.15.6:0/1915157793 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fede4086ed0 con 0x7fedf00fff00 2024-03-22T06:14:22.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.497+0000 7fedef7fe700 1 --2- 172.21.15.6:0/1915157793 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fedd80692a0 0x7fedd806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:22.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.498+0000 7fedef7fe700 1 --2- 172.21.15.6:0/1915157793 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fedd80692a0 0x7fedd806b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fede0004f90 tx=0x7fede0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:22.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.504+0000 7fedd7fff700 1 -- 172.21.15.6:0/1915157793 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fede4056d00 con 0x7fedf00fff00 2024-03-22T06:14:22.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.799+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1915157793 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fedf0111bd0 con 0x7fedd80692a0 2024-03-22T06:14:22.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.800+0000 7fedd7fff700 1 -- 172.21.15.6:0/1915157793 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7fedf0111bd0 con 0x7fedd80692a0 2024-03-22T06:14:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.802+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1915157793 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fedd80692a0 msgr2=0x7fedd806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.802+0000 7fedf5e3a700 1 --2- 172.21.15.6:0/1915157793 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fedd80692a0 0x7fedd806b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fede0004f90 tx=0x7fede0009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.802+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1915157793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf00fff00 msgr2=0x7fedf01101d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.802+0000 7fedf5e3a700 1 --2- 172.21.15.6:0/1915157793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf00fff00 0x7fedf01101d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fede4002a50 tx=0x7fede40046b0 comp rx=0 tx=0).stop 2024-03-22T06:14:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.802+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1915157793 shutdown_connections 2024-03-22T06:14:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.802+0000 7fedf5e3a700 1 --2- 172.21.15.6:0/1915157793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedf00662e0 0x7fedf010fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.802+0000 7fedf5e3a700 1 --2- 172.21.15.6:0/1915157793 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fedd80692a0 0x7fedd806b750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.802+0000 7fedf5e3a700 1 --2- 172.21.15.6:0/1915157793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf00fff00 0x7fedf01101d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.802+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1915157793 >> 172.21.15.6:0/1915157793 conn(0x7fedf00f7430 msgr2=0x7fedf00fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.802+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1915157793 shutdown_connections 2024-03-22T06:14:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.802+0000 7fedf5e3a700 1 -- 172.21.15.6:0/1915157793 wait complete. 2024-03-22T06:14:22.813 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:14:22.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.910+0000 7fb876eba700 1 -- 172.21.15.6:0/1670080533 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb870071e20 msgr2=0x7fb870070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:22.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.910+0000 7fb876eba700 1 --2- 172.21.15.6:0/1670080533 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb870071e20 0x7fb870070330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb864004940 tx=0x7fb864009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:22.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.910+0000 7fb876eba700 1 -- 172.21.15.6:0/1670080533 shutdown_connections 2024-03-22T06:14:22.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.910+0000 7fb876eba700 1 --2- 172.21.15.6:0/1670080533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb870070870 0x7fb870070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:22.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.910+0000 7fb876eba700 1 --2- 172.21.15.6:0/1670080533 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb870071e20 0x7fb870070330 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:22.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.910+0000 7fb876eba700 1 -- 172.21.15.6:0/1670080533 >> 172.21.15.6:0/1670080533 conn(0x7fb8700f85b0 msgr2=0x7fb8700fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:22.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.910+0000 7fb876eba700 1 -- 172.21.15.6:0/1670080533 shutdown_connections 2024-03-22T06:14:22.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.910+0000 7fb876eba700 1 -- 172.21.15.6:0/1670080533 wait complete. 2024-03-22T06:14:22.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.911+0000 7fb876eba700 1 Processor -- start 2024-03-22T06:14:22.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.911+0000 7fb876eba700 1 -- start start 2024-03-22T06:14:22.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.911+0000 7fb876eba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb870070870 0x7fb87010fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:22.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.911+0000 7fb876eba700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb870071e20 0x7fb870110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:22.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.911+0000 7fb876eba700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb870110790 con 0x7fb870070870 2024-03-22T06:14:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.911+0000 7fb876eba700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb870110fe0 con 0x7fb870071e20 2024-03-22T06:14:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb874c56700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb870070870 0x7fb87010fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb874c56700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb870070870 0x7fb87010fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40512/0 (socket says 172.21.15.6:40512) 2024-03-22T06:14:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb874c56700 1 -- 172.21.15.6:0/4069563095 learned_addr learned my addr 172.21.15.6:0/4069563095 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb86ffff700 1 --2- 172.21.15.6:0/4069563095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb870071e20 0x7fb870110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb874c56700 1 -- 172.21.15.6:0/4069563095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb870071e20 msgr2=0x7fb870110190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb874c56700 1 --2- 172.21.15.6:0/4069563095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb870071e20 0x7fb870110190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb874c56700 1 -- 172.21.15.6:0/4069563095 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb864005040 con 0x7fb870070870 2024-03-22T06:14:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb874c56700 1 --2- 172.21.15.6:0/4069563095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb870070870 0x7fb87010fc50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb86400b3b0 tx=0x7fb864009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb86d7fa700 1 -- 172.21.15.6:0/4069563095 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb86400e040 con 0x7fb870070870 2024-03-22T06:14:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb876eba700 1 -- 172.21.15.6:0/4069563095 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb870111260 con 0x7fb870070870 2024-03-22T06:14:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb876eba700 1 -- 172.21.15.6:0/4069563095 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8701117a0 con 0x7fb870070870 2024-03-22T06:14:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb86d7fa700 1 -- 172.21.15.6:0/4069563095 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb86400bbf0 con 0x7fb870070870 2024-03-22T06:14:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.912+0000 7fb86d7fa700 1 -- 172.21.15.6:0/4069563095 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb86401c630 con 0x7fb870070870 2024-03-22T06:14:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.913+0000 7fb876eba700 1 -- 172.21.15.6:0/4069563095 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87005f5f0 con 0x7fb870070870 2024-03-22T06:14:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.914+0000 7fb86d7fa700 1 -- 172.21.15.6:0/4069563095 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fb86400bd60 con 0x7fb870070870 2024-03-22T06:14:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.915+0000 7fb86d7fa700 1 --2- 172.21.15.6:0/4069563095 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb8600692a0 0x7fb86006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.915+0000 7fb86d7fa700 1 -- 172.21.15.6:0/4069563095 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb864087e90 con 0x7fb870070870 2024-03-22T06:14:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.915+0000 7fb86ffff700 1 --2- 172.21.15.6:0/4069563095 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb8600692a0 0x7fb86006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.915+0000 7fb86ffff700 1 --2- 172.21.15.6:0/4069563095 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb8600692a0 0x7fb86006b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004f40 tx=0x7fb85c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:22.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:22.919+0000 7fb86d7fa700 1 -- 172.21.15.6:0/4069563095 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb864057c50 con 0x7fb870070870 2024-03-22T06:14:23.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.134+0000 7fb876eba700 1 -- 172.21.15.6:0/4069563095 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb870111b90 con 0x7fb8600692a0 2024-03-22T06:14:23.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.135+0000 7fb86d7fa700 1 -- 172.21.15.6:0/4069563095 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7fb870111b90 con 0x7fb8600692a0 2024-03-22T06:14:23.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.138+0000 7fb876eba700 1 -- 172.21.15.6:0/4069563095 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb8600692a0 msgr2=0x7fb86006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:23.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.138+0000 7fb876eba700 1 --2- 172.21.15.6:0/4069563095 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb8600692a0 0x7fb86006b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004f40 tx=0x7fb85c00b040 comp rx=0 tx=0).stop 2024-03-22T06:14:23.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.138+0000 7fb876eba700 1 -- 172.21.15.6:0/4069563095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb870070870 msgr2=0x7fb87010fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:23.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.138+0000 7fb876eba700 1 --2- 172.21.15.6:0/4069563095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb870070870 0x7fb87010fc50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb86400b3b0 tx=0x7fb864009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:23.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.138+0000 7fb876eba700 1 -- 172.21.15.6:0/4069563095 shutdown_connections 2024-03-22T06:14:23.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.138+0000 7fb876eba700 1 --2- 172.21.15.6:0/4069563095 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fb8600692a0 0x7fb86006b750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.138+0000 7fb876eba700 1 --2- 172.21.15.6:0/4069563095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb870070870 0x7fb87010fc50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.138+0000 7fb876eba700 1 --2- 172.21.15.6:0/4069563095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb870071e20 0x7fb870110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.138+0000 7fb876eba700 1 -- 172.21.15.6:0/4069563095 >> 172.21.15.6:0/4069563095 conn(0x7fb8700f85b0 msgr2=0x7fb8700f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:23.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.138+0000 7fb876eba700 1 -- 172.21.15.6:0/4069563095 shutdown_connections 2024-03-22T06:14:23.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.138+0000 7fb876eba700 1 -- 172.21.15.6:0/4069563095 wait complete. 2024-03-22T06:14:23.198 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:23 smithi117 ceph-mon[36514]: pgmap v239: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:23.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.258+0000 7f47340ca700 1 -- 172.21.15.6:0/1315682345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f472c0fcf00 msgr2=0x7f472c0fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:23.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.258+0000 7f47340ca700 1 --2- 172.21.15.6:0/1315682345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f472c0fcf00 0x7f472c0fd310 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4728004940 tx=0x7f4728009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:23.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.258+0000 7f47340ca700 1 -- 172.21.15.6:0/1315682345 shutdown_connections 2024-03-22T06:14:23.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.258+0000 7f47340ca700 1 --2- 172.21.15.6:0/1315682345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f472c0fe070 0x7f472c0fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.258+0000 7f47340ca700 1 --2- 172.21.15.6:0/1315682345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f472c0fcf00 0x7f472c0fd310 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.258+0000 7f47340ca700 1 -- 172.21.15.6:0/1315682345 >> 172.21.15.6:0/1315682345 conn(0x7f472c0f85a0 msgr2=0x7f472c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:23.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.258+0000 7f47340ca700 1 -- 172.21.15.6:0/1315682345 shutdown_connections 2024-03-22T06:14:23.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.258+0000 7f47340ca700 1 -- 172.21.15.6:0/1315682345 wait complete. 2024-03-22T06:14:23.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.259+0000 7f47340ca700 1 Processor -- start 2024-03-22T06:14:23.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.259+0000 7f47340ca700 1 -- start start 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.259+0000 7f47340ca700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f472c0fcf00 0x7f472c10da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.259+0000 7f47340ca700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f472c0fe070 0x7f472c10dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.259+0000 7f47340ca700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c10e5d0 con 0x7f472c0fcf00 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.259+0000 7f47340ca700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c10edd0 con 0x7f472c0fe070 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.259+0000 7f4731e66700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f472c0fcf00 0x7f472c10da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.259+0000 7f4731e66700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f472c0fcf00 0x7f472c10da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40534/0 (socket says 172.21.15.6:40534) 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.259+0000 7f4731e66700 1 -- 172.21.15.6:0/4151571003 learned_addr learned my addr 172.21.15.6:0/4151571003 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.259+0000 7f4731665700 1 --2- 172.21.15.6:0/4151571003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f472c0fe070 0x7f472c10dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.260+0000 7f4731e66700 1 -- 172.21.15.6:0/4151571003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f472c0fe070 msgr2=0x7f472c10dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.260+0000 7f4731e66700 1 --2- 172.21.15.6:0/4151571003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f472c0fe070 0x7f472c10dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.260+0000 7f4731e66700 1 -- 172.21.15.6:0/4151571003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728005040 con 0x7f472c0fcf00 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.260+0000 7f4731e66700 1 --2- 172.21.15.6:0/4151571003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f472c0fcf00 0x7f472c10da90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f472800b3b0 tx=0x7f4728009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:23.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.260+0000 7f47227fc700 1 -- 172.21.15.6:0/4151571003 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f472800e040 con 0x7f472c0fcf00 2024-03-22T06:14:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.260+0000 7f47340ca700 1 -- 172.21.15.6:0/4151571003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472c10f050 con 0x7f472c0fcf00 2024-03-22T06:14:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.260+0000 7f47227fc700 1 -- 172.21.15.6:0/4151571003 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f472800bbf0 con 0x7f472c0fcf00 2024-03-22T06:14:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.260+0000 7f47227fc700 1 -- 172.21.15.6:0/4151571003 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f472801c630 con 0x7f472c0fcf00 2024-03-22T06:14:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.260+0000 7f47340ca700 1 -- 172.21.15.6:0/4151571003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472c10f590 con 0x7f472c0fcf00 2024-03-22T06:14:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.261+0000 7f47340ca700 1 -- 172.21.15.6:0/4151571003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472c05f5f0 con 0x7f472c0fcf00 2024-03-22T06:14:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.262+0000 7f47227fc700 1 -- 172.21.15.6:0/4151571003 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f472800bd60 con 0x7f472c0fcf00 2024-03-22T06:14:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.263+0000 7f47227fc700 1 --2- 172.21.15.6:0/4151571003 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f47180692a0 0x7f471806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.263+0000 7f47227fc700 1 -- 172.21.15.6:0/4151571003 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f4728088540 con 0x7f472c0fcf00 2024-03-22T06:14:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.263+0000 7f4731665700 1 --2- 172.21.15.6:0/4151571003 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f47180692a0 0x7f471806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.264+0000 7f4731665700 1 --2- 172.21.15.6:0/4151571003 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f47180692a0 0x7f471806b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f471c004f40 tx=0x7f471c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:23.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.267+0000 7f47227fc700 1 -- 172.21.15.6:0/4151571003 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f47280583a0 con 0x7f472c0fcf00 2024-03-22T06:14:23.495 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:23 smithi006 ceph-mon[29301]: pgmap v239: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:14:23.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.502+0000 7f47340ca700 1 -- 172.21.15.6:0/4151571003 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f472c10f980 con 0x7f47180692a0 2024-03-22T06:14:23.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.517+0000 7f47227fc700 1 -- 172.21.15.6:0/4151571003 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3432 (secure 0 0 0) 0x7f472c10f980 con 0x7f47180692a0 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (6m) 96s ago 7m 23.6M - 0.25.0 c8568f914cd2 bf3cfbba5c2f 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (7m) 96s ago 7m 6656k - 17.2.7-726-g9c4e0c34 837124da804f a49ab95d42e6 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (6m) 0s ago 6m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0a36af1f8ed0 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 running (6m) 96s ago 7m 84.9M - 9.4.7 954c08fa6188 8b365bc3fd8b 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (119s) 96s ago 118s 14.6M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (114s) 96s ago 113s 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (116s) 0s ago 116s 12.4M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (111s) 0s ago 111s 13.9M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:9283 running (8m) 96s ago 8m 479M - 17.2.7-726-g9c4e0c34 837124da804f 2cea0b1a5c60 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283 running (3s) 0s ago 6m 43.4M - 19.0.0-2299-gea01217f aa461e7c5e7f ff1047688325 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (8m) 96s ago 8m 47.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f b2fe36c6b005 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (6m) 0s ago 6m 37.9M 2048M 17.2.7-726-g9c4e0c34 837124da804f 0559848da9ab 2024-03-22T06:14:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (7m) 96s ago 7m 16.2M - 1.5.0 0da6a335fe13 0ef65efb379b 2024-03-22T06:14:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (6m) 0s ago 6m 15.3M - 1.5.0 0da6a335fe13 58a93ac942d3 2024-03-22T06:14:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (5m) 96s ago 5m 48.5M 4289M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:14:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (5m) 96s ago 5m 45.1M 4289M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:14:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (4m) 96s ago 4m 47.2M 4289M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:14:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (4m) 0s ago 4m 46.3M 5312M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:14:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (3m) 0s ago 3m 45.0M 5312M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:14:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (3m) 0s ago 3m 41.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:14:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (5m) 96s ago 6m 56.6M - 2.43.0 a07b618ecd1d 59d1a24d5af8 2024-03-22T06:14:23.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.519+0000 7f47340ca700 1 -- 172.21.15.6:0/4151571003 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f47180692a0 msgr2=0x7f471806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:23.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.519+0000 7f47340ca700 1 --2- 172.21.15.6:0/4151571003 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f47180692a0 0x7f471806b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f471c004f40 tx=0x7f471c00b040 comp rx=0 tx=0).stop 2024-03-22T06:14:23.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.519+0000 7f47340ca700 1 -- 172.21.15.6:0/4151571003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f472c0fcf00 msgr2=0x7f472c10da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:23.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.519+0000 7f47340ca700 1 --2- 172.21.15.6:0/4151571003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f472c0fcf00 0x7f472c10da90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f472800b3b0 tx=0x7f4728009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:23.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.519+0000 7f47340ca700 1 -- 172.21.15.6:0/4151571003 shutdown_connections 2024-03-22T06:14:23.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.519+0000 7f47340ca700 1 --2- 172.21.15.6:0/4151571003 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f47180692a0 0x7f471806b750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.519+0000 7f47340ca700 1 --2- 172.21.15.6:0/4151571003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f472c0fcf00 0x7f472c10da90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.519+0000 7f47340ca700 1 --2- 172.21.15.6:0/4151571003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f472c0fe070 0x7f472c10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.519+0000 7f47340ca700 1 -- 172.21.15.6:0/4151571003 >> 172.21.15.6:0/4151571003 conn(0x7f472c0f85a0 msgr2=0x7f472c0f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:23.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.520+0000 7f47340ca700 1 -- 172.21.15.6:0/4151571003 shutdown_connections 2024-03-22T06:14:23.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.520+0000 7f47340ca700 1 -- 172.21.15.6:0/4151571003 wait complete. 2024-03-22T06:14:23.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.631+0000 7fc13db9e700 1 -- 172.21.15.6:0/459743618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1380fdec0 msgr2=0x7fc1380fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:23.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.631+0000 7fc13db9e700 1 --2- 172.21.15.6:0/459743618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1380fdec0 0x7fc1380fe330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc124004940 tx=0x7fc124009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:23.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.631+0000 7fc13db9e700 1 -- 172.21.15.6:0/459743618 shutdown_connections 2024-03-22T06:14:23.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.631+0000 7fc13db9e700 1 --2- 172.21.15.6:0/459743618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1380fdec0 0x7fc1380fe330 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.631+0000 7fc13db9e700 1 --2- 172.21.15.6:0/459743618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1380fcc10 0x7fc1380fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.631+0000 7fc13db9e700 1 -- 172.21.15.6:0/459743618 >> 172.21.15.6:0/459743618 conn(0x7fc1380f83d0 msgr2=0x7fc1380fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:23.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.632+0000 7fc13db9e700 1 -- 172.21.15.6:0/459743618 shutdown_connections 2024-03-22T06:14:23.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.632+0000 7fc13db9e700 1 -- 172.21.15.6:0/459743618 wait complete. 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.632+0000 7fc13db9e700 1 Processor -- start 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.632+0000 7fc13db9e700 1 -- start start 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.633+0000 7fc13db9e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1380fcc10 0x7fc13810d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.633+0000 7fc13db9e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1380fdec0 0x7fc13810dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.633+0000 7fc13db9e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13810e370 con 0x7fc1380fdec0 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.633+0000 7fc13db9e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13810ebc0 con 0x7fc1380fcc10 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.633+0000 7fc13cb9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1380fcc10 0x7fc13810d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.633+0000 7fc13cb9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1380fcc10 0x7fc13810d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:43850/0 (socket says 172.21.15.6:43850) 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.633+0000 7fc13cb9c700 1 -- 172.21.15.6:0/107260499 learned_addr learned my addr 172.21.15.6:0/107260499 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.633+0000 7fc137fff700 1 --2- 172.21.15.6:0/107260499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1380fdec0 0x7fc13810dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.633+0000 7fc137fff700 1 -- 172.21.15.6:0/107260499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1380fcc10 msgr2=0x7fc13810d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.633+0000 7fc137fff700 1 --2- 172.21.15.6:0/107260499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1380fcc10 0x7fc13810d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.633+0000 7fc137fff700 1 -- 172.21.15.6:0/107260499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc124005040 con 0x7fc1380fdec0 2024-03-22T06:14:23.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.634+0000 7fc137fff700 1 --2- 172.21.15.6:0/107260499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1380fdec0 0x7fc13810dd70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc12400b3b0 tx=0x7fc124009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:23.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.634+0000 7fc1357fa700 1 -- 172.21.15.6:0/107260499 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc12400e040 con 0x7fc1380fdec0 2024-03-22T06:14:23.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.634+0000 7fc13db9e700 1 -- 172.21.15.6:0/107260499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13810ee40 con 0x7fc1380fdec0 2024-03-22T06:14:23.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.634+0000 7fc1357fa700 1 -- 172.21.15.6:0/107260499 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc12400bbf0 con 0x7fc1380fdec0 2024-03-22T06:14:23.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.634+0000 7fc1357fa700 1 -- 172.21.15.6:0/107260499 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc124013660 con 0x7fc1380fdec0 2024-03-22T06:14:23.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.634+0000 7fc13db9e700 1 -- 172.21.15.6:0/107260499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13810f380 con 0x7fc1380fdec0 2024-03-22T06:14:23.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.635+0000 7fc13db9e700 1 -- 172.21.15.6:0/107260499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13805f5f0 con 0x7fc1380fdec0 2024-03-22T06:14:23.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.636+0000 7fc1357fa700 1 -- 172.21.15.6:0/107260499 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fc12400bd60 con 0x7fc1380fdec0 2024-03-22T06:14:23.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.636+0000 7fc1357fa700 1 --2- 172.21.15.6:0/107260499 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc1280692a0 0x7fc12806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:23.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.636+0000 7fc1357fa700 1 -- 172.21.15.6:0/107260499 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc1240885a0 con 0x7fc1380fdec0 2024-03-22T06:14:23.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.637+0000 7fc13cb9c700 1 --2- 172.21.15.6:0/107260499 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc1280692a0 0x7fc12806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:23.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.637+0000 7fc13cb9c700 1 --2- 172.21.15.6:0/107260499 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc1280692a0 0x7fc12806b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0041d0 tx=0x7fc12c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.645+0000 7fc1357fa700 1 -- 172.21.15.6:0/107260499 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc124058360 con 0x7fc1380fdec0 2024-03-22T06:14:23.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.991+0000 7fc13db9e700 1 -- 172.21.15.6:0/107260499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc13804bed0 con 0x7fc1380fdec0 2024-03-22T06:14:23.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.992+0000 7fc1357fa700 1 -- 172.21.15.6:0/107260499 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fc124057ef0 con 0x7fc1380fdec0 2024-03-22T06:14:23.992 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:14:23.992 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:14:23.992 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T06:14:23.992 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:14:23.992 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:14:23.993 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:14:23.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.993+0000 7fc13db9e700 1 -- 172.21.15.6:0/107260499 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc1280692a0 msgr2=0x7fc12806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:23.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.993+0000 7fc13db9e700 1 --2- 172.21.15.6:0/107260499 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc1280692a0 0x7fc12806b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0041d0 tx=0x7fc12c00b040 comp rx=0 tx=0).stop 2024-03-22T06:14:23.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.993+0000 7fc13db9e700 1 -- 172.21.15.6:0/107260499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1380fdec0 msgr2=0x7fc13810dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:23.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.993+0000 7fc13db9e700 1 --2- 172.21.15.6:0/107260499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1380fdec0 0x7fc13810dd70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc12400b3b0 tx=0x7fc124009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:23.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.994+0000 7fc13db9e700 1 -- 172.21.15.6:0/107260499 shutdown_connections 2024-03-22T06:14:23.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.994+0000 7fc13db9e700 1 --2- 172.21.15.6:0/107260499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1380fcc10 0x7fc13810d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.994+0000 7fc13db9e700 1 --2- 172.21.15.6:0/107260499 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7fc1280692a0 0x7fc12806b750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.994+0000 7fc13db9e700 1 --2- 172.21.15.6:0/107260499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1380fdec0 0x7fc13810dd70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.994+0000 7fc13db9e700 1 -- 172.21.15.6:0/107260499 >> 172.21.15.6:0/107260499 conn(0x7fc1380f83d0 msgr2=0x7fc1380f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.994+0000 7fc13db9e700 1 -- 172.21.15.6:0/107260499 shutdown_connections 2024-03-22T06:14:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:23.994+0000 7fc13db9e700 1 -- 172.21.15.6:0/107260499 wait complete. 2024-03-22T06:14:24.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.098+0000 7fa007726700 1 -- 172.21.15.6:0/4060525691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0000662e0 msgr2=0x7fa0000ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:24.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.098+0000 7fa007726700 1 --2- 172.21.15.6:0/4060525691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0000662e0 0x7fa0000ff9c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc004940 tx=0x7f9ffc009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:24.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.098+0000 7fa007726700 1 -- 172.21.15.6:0/4060525691 shutdown_connections 2024-03-22T06:14:24.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.098+0000 7fa007726700 1 --2- 172.21.15.6:0/4060525691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0000fff00 0x7fa000102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.098+0000 7fa007726700 1 --2- 172.21.15.6:0/4060525691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0000662e0 0x7fa0000ff9c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.098+0000 7fa007726700 1 -- 172.21.15.6:0/4060525691 >> 172.21.15.6:0/4060525691 conn(0x7fa0000f7430 msgr2=0x7fa0000f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:24.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.098+0000 7fa007726700 1 -- 172.21.15.6:0/4060525691 shutdown_connections 2024-03-22T06:14:24.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.098+0000 7fa007726700 1 -- 172.21.15.6:0/4060525691 wait complete. 2024-03-22T06:14:24.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.098+0000 7fa007726700 1 Processor -- start 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.098+0000 7fa007726700 1 -- start start 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa007726700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0000662e0 0x7fa00010fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa007726700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0000fff00 0x7fa0001101c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa007726700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa000110850 con 0x7fa0000fff00 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa007726700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa000110ff0 con 0x7fa0000662e0 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa004cc1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0000fff00 0x7fa0001101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa004cc1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0000fff00 0x7fa0001101c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57232/0 (socket says 172.21.15.6:57232) 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa004cc1700 1 -- 172.21.15.6:0/3784830346 learned_addr learned my addr 172.21.15.6:0/3784830346 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa0054c2700 1 --2- 172.21.15.6:0/3784830346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0000662e0 0x7fa00010fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa004cc1700 1 -- 172.21.15.6:0/3784830346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0000662e0 msgr2=0x7fa00010fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa004cc1700 1 --2- 172.21.15.6:0/3784830346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0000662e0 0x7fa00010fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa004cc1700 1 -- 172.21.15.6:0/3784830346 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc005040 con 0x7fa0000fff00 2024-03-22T06:14:24.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.099+0000 7fa004cc1700 1 --2- 172.21.15.6:0/3784830346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0000fff00 0x7fa0001101c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000b790 tx=0x7f9ff0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:24.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.100+0000 7f9ff5ffb700 1 -- 172.21.15.6:0/3784830346 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ff000f6f0 con 0x7fa0000fff00 2024-03-22T06:14:24.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.100+0000 7f9ff5ffb700 1 -- 172.21.15.6:0/3784830346 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ff000fd20 con 0x7fa0000fff00 2024-03-22T06:14:24.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.100+0000 7fa007726700 1 -- 172.21.15.6:0/3784830346 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0001112d0 con 0x7fa0000fff00 2024-03-22T06:14:24.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.100+0000 7f9ff5ffb700 1 -- 172.21.15.6:0/3784830346 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ff00103a0 con 0x7fa0000fff00 2024-03-22T06:14:24.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.100+0000 7fa007726700 1 -- 172.21.15.6:0/3784830346 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa000111890 con 0x7fa0000fff00 2024-03-22T06:14:24.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.101+0000 7f9ff5ffb700 1 -- 172.21.15.6:0/3784830346 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f9ff0011400 con 0x7fa0000fff00 2024-03-22T06:14:24.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.101+0000 7fa007726700 1 -- 172.21.15.6:0/3784830346 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00005f5f0 con 0x7fa0000fff00 2024-03-22T06:14:24.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.102+0000 7f9ff5ffb700 1 --2- 172.21.15.6:0/3784830346 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fec0692a0 0x7f9fec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:24.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.102+0000 7f9ff5ffb700 1 -- 172.21.15.6:0/3784830346 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9ff0086ff0 con 0x7fa0000fff00 2024-03-22T06:14:24.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.102+0000 7fa0054c2700 1 --2- 172.21.15.6:0/3784830346 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fec0692a0 0x7f9fec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:24.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.102+0000 7fa0054c2700 1 --2- 172.21.15.6:0/3784830346 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fec0692a0 0x7f9fec06b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00b970 tx=0x7f9ffc009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:24.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.107+0000 7f9ff5ffb700 1 -- 172.21.15.6:0/3784830346 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9ff0056d30 con 0x7fa0000fff00 2024-03-22T06:14:24.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.461+0000 7fa007726700 1 -- 172.21.15.6:0/3784830346 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa000111b60 con 0x7fa0000fff00 2024-03-22T06:14:24.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.462+0000 7f9ff5ffb700 1 -- 172.21.15.6:0/3784830346 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1805 (secure 0 0 0) 0x7f9ff00568c0 con 0x7fa0000fff00 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:14:24.463 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:14:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:14:24.465 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:14:24.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.464+0000 7fa007726700 1 -- 172.21.15.6:0/3784830346 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fec0692a0 msgr2=0x7f9fec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:24.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.464+0000 7fa007726700 1 --2- 172.21.15.6:0/3784830346 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fec0692a0 0x7f9fec06b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00b970 tx=0x7f9ffc009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:24.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.464+0000 7fa007726700 1 -- 172.21.15.6:0/3784830346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0000fff00 msgr2=0x7fa0001101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:24.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.464+0000 7fa007726700 1 --2- 172.21.15.6:0/3784830346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0000fff00 0x7fa0001101c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000b790 tx=0x7f9ff0009250 comp rx=0 tx=0).stop 2024-03-22T06:14:24.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.464+0000 7fa007726700 1 -- 172.21.15.6:0/3784830346 shutdown_connections 2024-03-22T06:14:24.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.464+0000 7fa007726700 1 --2- 172.21.15.6:0/3784830346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0000662e0 0x7fa00010fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.464+0000 7fa007726700 1 --2- 172.21.15.6:0/3784830346 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f9fec0692a0 0x7f9fec06b750 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.464+0000 7fa007726700 1 --2- 172.21.15.6:0/3784830346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0000fff00 0x7fa0001101c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.464+0000 7fa007726700 1 -- 172.21.15.6:0/3784830346 >> 172.21.15.6:0/3784830346 conn(0x7fa0000f7430 msgr2=0x7fa0000fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:24.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.464+0000 7fa007726700 1 -- 172.21.15.6:0/3784830346 shutdown_connections 2024-03-22T06:14:24.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.464+0000 7fa007726700 1 -- 172.21.15.6:0/3784830346 wait complete. 2024-03-22T06:14:24.468 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:14:24.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.595+0000 7f4b38d44700 1 -- 172.21.15.6:0/258930760 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340fbcd0 msgr2=0x7f4b340fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.595+0000 7f4b38d44700 1 --2- 172.21.15.6:0/258930760 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340fbcd0 0x7f4b340fc0e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4b24004940 tx=0x7f4b24009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.595+0000 7f4b38d44700 1 -- 172.21.15.6:0/258930760 shutdown_connections 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.595+0000 7f4b38d44700 1 --2- 172.21.15.6:0/258930760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b340fcf80 0x7f4b340fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.595+0000 7f4b38d44700 1 --2- 172.21.15.6:0/258930760 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340fbcd0 0x7f4b340fc0e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.595+0000 7f4b38d44700 1 -- 172.21.15.6:0/258930760 >> 172.21.15.6:0/258930760 conn(0x7f4b340f7430 msgr2=0x7f4b340f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.595+0000 7f4b38d44700 1 -- 172.21.15.6:0/258930760 shutdown_connections 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.595+0000 7f4b38d44700 1 -- 172.21.15.6:0/258930760 wait complete. 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.595+0000 7f4b38d44700 1 Processor -- start 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.595+0000 7f4b38d44700 1 -- start start 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b38d44700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340fbcd0 0x7f4b34110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b38d44700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b340fcf80 0x7f4b34112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b38d44700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b34066350 con 0x7f4b340fbcd0 2024-03-22T06:14:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b38d44700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b34066490 con 0x7f4b340fcf80 2024-03-22T06:14:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b3259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340fbcd0 0x7f4b34110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b3259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340fbcd0 0x7f4b34110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57258/0 (socket says 172.21.15.6:57258) 2024-03-22T06:14:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b3259c700 1 -- 172.21.15.6:0/1626783965 learned_addr learned my addr 172.21.15.6:0/1626783965 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b31d9b700 1 --2- 172.21.15.6:0/1626783965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b340fcf80 0x7f4b34112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b3259c700 1 -- 172.21.15.6:0/1626783965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b340fcf80 msgr2=0x7f4b34112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b3259c700 1 --2- 172.21.15.6:0/1626783965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b340fcf80 0x7f4b34112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b3259c700 1 -- 172.21.15.6:0/1626783965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b24005040 con 0x7f4b340fbcd0 2024-03-22T06:14:24.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b3259c700 1 --2- 172.21.15.6:0/1626783965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340fbcd0 0x7f4b34110720 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009150 tx=0x7f4b24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:24.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b22ffd700 1 -- 172.21.15.6:0/1626783965 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2400e040 con 0x7f4b340fbcd0 2024-03-22T06:14:24.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.596+0000 7f4b22ffd700 1 -- 172.21.15.6:0/1626783965 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b24004290 con 0x7f4b340fbcd0 2024-03-22T06:14:24.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.597+0000 7f4b38d44700 1 -- 172.21.15.6:0/1626783965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b341131b0 con 0x7f4b340fbcd0 2024-03-22T06:14:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.597+0000 7f4b22ffd700 1 -- 172.21.15.6:0/1626783965 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2401c630 con 0x7f4b340fbcd0 2024-03-22T06:14:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.597+0000 7f4b38d44700 1 -- 172.21.15.6:0/1626783965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b34113770 con 0x7f4b340fbcd0 2024-03-22T06:14:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.598+0000 7f4b38d44700 1 -- 172.21.15.6:0/1626783965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3405f5f0 con 0x7f4b340fbcd0 2024-03-22T06:14:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.598+0000 7f4b22ffd700 1 -- 172.21.15.6:0/1626783965 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f4b2400bbf0 con 0x7f4b340fbcd0 2024-03-22T06:14:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.599+0000 7f4b22ffd700 1 --2- 172.21.15.6:0/1626783965 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4b1c0692a0 0x7f4b1c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.599+0000 7f4b22ffd700 1 -- 172.21.15.6:0/1626783965 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f4b24087e50 con 0x7f4b340fbcd0 2024-03-22T06:14:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.599+0000 7f4b31d9b700 1 --2- 172.21.15.6:0/1626783965 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4b1c0692a0 0x7f4b1c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.599+0000 7f4b31d9b700 1 --2- 172.21.15.6:0/1626783965 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4b1c0692a0 0x7f4b1c06b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b28004f40 tx=0x7f4b2800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:24.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.603+0000 7f4b22ffd700 1 -- 172.21.15.6:0/1626783965 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4b24054180 con 0x7f4b340fbcd0 2024-03-22T06:14:24.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:24 smithi117 ceph-mon[36514]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:24.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:24 smithi117 ceph-mon[36514]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:24.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:24 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:24.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:24 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:24.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:24 smithi117 ceph-mon[36514]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:24.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:24 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/107260499' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:14:24.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:24 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:24.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:24 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:24.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:24 smithi006 ceph-mon[29301]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:24.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:24 smithi006 ceph-mon[29301]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:24.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:24 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:24.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:24 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:24.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:24 smithi006 ceph-mon[29301]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:24.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:24 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/107260499' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:14:24.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:24 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:24.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:24 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:24.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.853+0000 7f4b38d44700 1 -- 172.21.15.6:0/1626783965 --> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b34101290 con 0x7f4b1c0692a0 2024-03-22T06:14:24.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.854+0000 7f4b22ffd700 1 -- 172.21.15.6:0/1626783965 <== mgr.14212 v2:172.21.15.6:6800/602685778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f4b34101290 con 0x7f4b1c0692a0 2024-03-22T06:14:24.855 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:14:24.855 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:14:24.855 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:14:24.855 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:14:24.855 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [], 2024-03-22T06:14:24.855 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "1/21 daemons upgraded", 2024-03-22T06:14:24.855 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading mgr daemons", 2024-03-22T06:14:24.856 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:14:24.856 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:14:24.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.856+0000 7f4b38d44700 1 -- 172.21.15.6:0/1626783965 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4b1c0692a0 msgr2=0x7f4b1c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:24.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.856+0000 7f4b38d44700 1 --2- 172.21.15.6:0/1626783965 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4b1c0692a0 0x7f4b1c06b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b28004f40 tx=0x7f4b2800b040 comp rx=0 tx=0).stop 2024-03-22T06:14:24.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.856+0000 7f4b38d44700 1 -- 172.21.15.6:0/1626783965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340fbcd0 msgr2=0x7f4b34110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:24.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.856+0000 7f4b38d44700 1 --2- 172.21.15.6:0/1626783965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340fbcd0 0x7f4b34110720 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009150 tx=0x7f4b24009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:24.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.856+0000 7f4b38d44700 1 -- 172.21.15.6:0/1626783965 shutdown_connections 2024-03-22T06:14:24.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.856+0000 7f4b38d44700 1 --2- 172.21.15.6:0/1626783965 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f4b1c0692a0 0x7f4b1c06b750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.856+0000 7f4b38d44700 1 --2- 172.21.15.6:0/1626783965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340fbcd0 0x7f4b34110720 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.856+0000 7f4b38d44700 1 --2- 172.21.15.6:0/1626783965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b340fcf80 0x7f4b34112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.857+0000 7f4b38d44700 1 -- 172.21.15.6:0/1626783965 >> 172.21.15.6:0/1626783965 conn(0x7f4b340f7430 msgr2=0x7f4b340f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:24.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.857+0000 7f4b38d44700 1 -- 172.21.15.6:0/1626783965 shutdown_connections 2024-03-22T06:14:24.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.857+0000 7f4b38d44700 1 -- 172.21.15.6:0/1626783965 wait complete. 2024-03-22T06:14:24.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 -- 172.21.15.6:0/1211125326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39040fffa0 msgr2=0x7f3904102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:24.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 --2- 172.21.15.6:0/1211125326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39040fffa0 0x7f3904102380 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f38f0004940 tx=0x7f38f0009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:24.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 -- 172.21.15.6:0/1211125326 shutdown_connections 2024-03-22T06:14:24.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 --2- 172.21.15.6:0/1211125326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39040fffa0 0x7f3904102380 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 --2- 172.21.15.6:0/1211125326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3904066240 0x7f39040ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 -- 172.21.15.6:0/1211125326 >> 172.21.15.6:0/1211125326 conn(0x7f39040f7430 msgr2=0x7f39040f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:24.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 -- 172.21.15.6:0/1211125326 shutdown_connections 2024-03-22T06:14:24.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 -- 172.21.15.6:0/1211125326 wait complete. 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 Processor -- start 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 -- start start 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3904066240 0x7f390410da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39040fffa0 0x7f390410dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390410e5c0 con 0x7f3904066240 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f390b1a5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390410ee10 con 0x7f39040fffa0 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f3908f41700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3904066240 0x7f390410da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f3908f41700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3904066240 0x7f390410da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57272/0 (socket says 172.21.15.6:57272) 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f3908f41700 1 -- 172.21.15.6:0/499275428 learned_addr learned my addr 172.21.15.6:0/499275428 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.968+0000 7f3903fff700 1 --2- 172.21.15.6:0/499275428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39040fffa0 0x7f390410dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.969+0000 7f3908f41700 1 -- 172.21.15.6:0/499275428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39040fffa0 msgr2=0x7f390410dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:24.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.969+0000 7f3908f41700 1 --2- 172.21.15.6:0/499275428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39040fffa0 0x7f390410dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:24.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.969+0000 7f3908f41700 1 -- 172.21.15.6:0/499275428 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f0005040 con 0x7f3904066240 2024-03-22T06:14:24.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.969+0000 7f3908f41700 1 --2- 172.21.15.6:0/499275428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3904066240 0x7f390410da80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f38f8002a50 tx=0x7f38f8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:24.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.969+0000 7f39017fa700 1 -- 172.21.15.6:0/499275428 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38f8007cc0 con 0x7f3904066240 2024-03-22T06:14:24.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.969+0000 7f390b1a5700 1 -- 172.21.15.6:0/499275428 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390410f0f0 con 0x7f3904066240 2024-03-22T06:14:24.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.969+0000 7f39017fa700 1 -- 172.21.15.6:0/499275428 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38f800f040 con 0x7f3904066240 2024-03-22T06:14:24.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.969+0000 7f39017fa700 1 -- 172.21.15.6:0/499275428 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38f800d940 con 0x7f3904066240 2024-03-22T06:14:24.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.969+0000 7f390b1a5700 1 -- 172.21.15.6:0/499275428 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390410f630 con 0x7f3904066240 2024-03-22T06:14:24.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.970+0000 7f390b1a5700 1 -- 172.21.15.6:0/499275428 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390405f5f0 con 0x7f3904066240 2024-03-22T06:14:24.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.972+0000 7f39017fa700 1 -- 172.21.15.6:0/499275428 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f38f800dbc0 con 0x7f3904066240 2024-03-22T06:14:24.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.972+0000 7f39017fa700 1 --2- 172.21.15.6:0/499275428 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f38f40692a0 0x7f38f406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:24.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.972+0000 7f39017fa700 1 -- 172.21.15.6:0/499275428 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f38f80874a0 con 0x7f3904066240 2024-03-22T06:14:24.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.972+0000 7f3903fff700 1 --2- 172.21.15.6:0/499275428 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f38f40692a0 0x7f38f406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:24.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.972+0000 7f3903fff700 1 --2- 172.21.15.6:0/499275428 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f38f40692a0 0x7f38f406b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38f000b970 tx=0x7f38f0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:24.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:24.979+0000 7f39017fa700 1 -- 172.21.15.6:0/499275428 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f38f8057300 con 0x7f3904066240 2024-03-22T06:14:25.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.272+0000 7f390b1a5700 1 -- 172.21.15.6:0/499275428 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f39041058e0 con 0x7f3904066240 2024-03-22T06:14:25.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.272+0000 7f39017fa700 1 -- 172.21.15.6:0/499275428 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f38f8056e90 con 0x7f3904066240 2024-03-22T06:14:25.272 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_OK 2024-03-22T06:14:25.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.275+0000 7f390b1a5700 1 -- 172.21.15.6:0/499275428 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f38f40692a0 msgr2=0x7f38f406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:25.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.275+0000 7f390b1a5700 1 --2- 172.21.15.6:0/499275428 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f38f40692a0 0x7f38f406b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38f000b970 tx=0x7f38f0009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.275+0000 7f390b1a5700 1 -- 172.21.15.6:0/499275428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3904066240 msgr2=0x7f390410da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.275+0000 7f390b1a5700 1 --2- 172.21.15.6:0/499275428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3904066240 0x7f390410da80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f38f8002a50 tx=0x7f38f8004600 comp rx=0 tx=0).stop 2024-03-22T06:14:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.275+0000 7f390b1a5700 1 -- 172.21.15.6:0/499275428 shutdown_connections 2024-03-22T06:14:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.275+0000 7f390b1a5700 1 --2- 172.21.15.6:0/499275428 >> [v2:172.21.15.6:6800/602685778,v1:172.21.15.6:6801/602685778] conn(0x7f38f40692a0 0x7f38f406b750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.275+0000 7f390b1a5700 1 --2- 172.21.15.6:0/499275428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3904066240 0x7f390410da80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.275+0000 7f390b1a5700 1 --2- 172.21.15.6:0/499275428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39040fffa0 0x7f390410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.275+0000 7f390b1a5700 1 -- 172.21.15.6:0/499275428 >> 172.21.15.6:0/499275428 conn(0x7f39040f7430 msgr2=0x7f39040f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.275+0000 7f390b1a5700 1 -- 172.21.15.6:0/499275428 shutdown_connections 2024-03-22T06:14:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:25.275+0000 7f390b1a5700 1 -- 172.21.15.6:0/499275428 wait complete. 2024-03-22T06:14:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:25 smithi006 ceph-mon[29301]: pgmap v240: 65 pgs: 65 active+clean; 601 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 0 op/s 2024-03-22T06:14:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:25 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3784830346' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:14:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:25 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/499275428' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:14:25.635 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:25 smithi117 ceph-mon[36514]: pgmap v240: 65 pgs: 65 active+clean; 601 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 0 op/s 2024-03-22T06:14:25.635 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:25 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3784830346' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:14:25.635 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:25 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/499275428' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:14:26.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:26 smithi117 ceph-mon[36514]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:26.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:26 smithi006 ceph-mon[29301]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:27.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:27 smithi117 ceph-mon[36514]: pgmap v241: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 358 KiB/s wr, 0 op/s 2024-03-22T06:14:27.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:27 smithi006 ceph-mon[29301]: pgmap v241: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 358 KiB/s wr, 0 op/s 2024-03-22T06:14:28.867 INFO:tasks.workunit.client.1.smithi117.stderr:Updating files: 79% (9707/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-22T06:14:29.630 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:29 smithi006 ceph-mon[29301]: pgmap v242: 65 pgs: 65 active+clean; 17 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s wr, 57 op/s 2024-03-22T06:14:29.631 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:29 smithi006 ceph-mon[29301]: Standby manager daemon smithi117.bqeskk restarted 2024-03-22T06:14:29.631 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:29 smithi006 ceph-mon[29301]: Standby manager daemon smithi117.bqeskk started 2024-03-22T06:14:29.631 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:29 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/crt"}]: dispatch 2024-03-22T06:14:29.631 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:29 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:14:29.631 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:29 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/key"}]: dispatch 2024-03-22T06:14:29.631 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:29 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:14:29.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:29 smithi117 ceph-mon[36514]: pgmap v242: 65 pgs: 65 active+clean; 17 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s wr, 57 op/s 2024-03-22T06:14:29.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:29 smithi117 ceph-mon[36514]: Standby manager daemon smithi117.bqeskk restarted 2024-03-22T06:14:29.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:29 smithi117 ceph-mon[36514]: Standby manager daemon smithi117.bqeskk started 2024-03-22T06:14:29.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:29 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/crt"}]: dispatch 2024-03-22T06:14:29.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:29 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:14:29.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:29 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/key"}]: dispatch 2024-03-22T06:14:29.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:29 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:14:29.865 INFO:tasks.workunit.client.1.smithi117.stderr:Note: switching to 'ea01217f8a79678e64ef6a5d68a043a4ef21ca37'. 2024-03-22T06:14:29.865 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-03-22T06:14:29.865 INFO:tasks.workunit.client.1.smithi117.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-22T06:14:29.865 INFO:tasks.workunit.client.1.smithi117.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-22T06:14:29.865 INFO:tasks.workunit.client.1.smithi117.stderr:state without impacting any branches by switching back to a branch. 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr: git switch -c 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr:Or undo this operation with: 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr: git switch - 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-03-22T06:14:29.866 INFO:tasks.workunit.client.1.smithi117.stderr:HEAD is now at ea01217f8a7 Merge branch 'jjm-cephadm-smb-svc-spec' of https://github.com/phlogistonjohn/ceph into wip-phlogistonjohn-testing-2024-03-21-1835 2024-03-22T06:14:29.870 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-22T06:14:30.068 INFO:tasks.workunit.client.1.smithi117.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-22T06:14:30.070 INFO:tasks.workunit.client.1.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-22T06:14:30.070 INFO:tasks.workunit.client.1.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-22T06:14:30.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:30 smithi117 ceph-mon[36514]: mgrmap e20: smithi006.sfpxdy(active, since 6m), standbys: smithi117.bqeskk 2024-03-22T06:14:30.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:30 smithi006 ceph-mon[29301]: mgrmap e20: smithi006.sfpxdy(active, since 6m), standbys: smithi117.bqeskk 2024-03-22T06:14:31.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:31 smithi117 ceph-mon[36514]: pgmap v243: 65 pgs: 65 active+clean; 39 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 3.4 MiB/s wr, 296 op/s 2024-03-22T06:14:31.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:31 smithi006 ceph-mon[29301]: pgmap v243: 65 pgs: 65 active+clean; 39 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 3.4 MiB/s wr, 296 op/s 2024-03-22T06:14:32.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:32 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:14:32.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:32 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:14:33.453 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:33 smithi117 ceph-mon[36514]: pgmap v244: 65 pgs: 65 active+clean; 47 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 4.1 MiB/s wr, 421 op/s 2024-03-22T06:14:33.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:33 smithi006 ceph-mon[29301]: pgmap v244: 65 pgs: 65 active+clean; 47 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 4.1 MiB/s wr, 421 op/s 2024-03-22T06:14:35.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:35 smithi117 ceph-mon[36514]: pgmap v245: 65 pgs: 65 active+clean; 59 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 5.1 MiB/s wr, 452 op/s 2024-03-22T06:14:35.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:35 smithi006 ceph-mon[29301]: pgmap v245: 65 pgs: 65 active+clean; 59 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 5.1 MiB/s wr, 452 op/s 2024-03-22T06:14:36.174 INFO:tasks.workunit.client.0.smithi006.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-22T06:14:36.174 INFO:tasks.workunit.client.0.smithi006.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T06:14:36.174 INFO:tasks.workunit.client.0.smithi006.stderr:+ make 2024-03-22T06:14:36.226 INFO:tasks.workunit.client.0.smithi006.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-22T06:14:37.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:37 smithi117 ceph-mon[36514]: pgmap v246: 65 pgs: 65 active+clean; 62 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 5.4 MiB/s wr, 457 op/s 2024-03-22T06:14:37.721 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:37 smithi006 ceph-mon[29301]: pgmap v246: 65 pgs: 65 active+clean; 62 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 5.4 MiB/s wr, 457 op/s 2024-03-22T06:14:37.941 INFO:tasks.workunit.client.0.smithi006.stderr:++ readlink -f fsstress 2024-03-22T06:14:37.942 INFO:tasks.workunit.client.0.smithi006.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-22T06:14:37.942 INFO:tasks.workunit.client.0.smithi006.stderr:+ popd 2024-03-22T06:14:37.942 INFO:tasks.workunit.client.0.smithi006.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T06:14:37.943 INFO:tasks.workunit.client.0.smithi006.stderr:+ popd 2024-03-22T06:14:37.943 INFO:tasks.workunit.client.0.smithi006.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-22T06:14:37.943 INFO:tasks.workunit.client.0.smithi006.stderr:++ mktemp -d -p . 2024-03-22T06:14:38.441 INFO:tasks.workunit.client.1.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-22T06:14:38.509 INFO:tasks.workunit.client.1.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-22T06:14:38.543 INFO:tasks.workunit.client.0.smithi006.stderr:+ T=./tmp.Oar7AqeBar 2024-03-22T06:14:38.543 INFO:tasks.workunit.client.0.smithi006.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.Oar7AqeBar -l 1 -n 1000 -p 10 -v 2024-03-22T06:14:38.548 INFO:tasks.workunit.client.1.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-22T06:14:38.549 INFO:tasks.workunit.client.1.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-22T06:14:38.549 INFO:tasks.workunit.client.1.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-22T06:14:38.591 INFO:tasks.workunit.client.1.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-22T06:14:38.594 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:14:38.594 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-22T06:14:38.610 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-22T06:14:38.612 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-22T06:14:38.613 DEBUG:teuthology.orchestra.run.smithi117:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-22T06:14:38.674 INFO:tasks.workunit.client.1.smithi117.stderr:+ mkdir -p fsstress 2024-03-22T06:14:38.677 INFO:tasks.workunit.client.1.smithi117.stderr:+ pushd fsstress 2024-03-22T06:14:38.677 INFO:tasks.workunit.client.1.smithi117.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T06:14:38.677 INFO:tasks.workunit.client.1.smithi117.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-22T06:14:39.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: pgmap v247: 65 pgs: 65 active+clean; 71 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 5.8 MiB/s wr, 462 op/s 2024-03-22T06:14:39.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:39.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:39.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr fail", "who": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: osdmap e47: 6 total, 6 up, 6 in 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "mgr fail", "who": "smithi006.sfpxdy"}]': finished 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: mgrmap e21: smithi117.bqeskk(active, starting, since 0.00920251s) 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:14:39.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mgr metadata", "who": "smithi117.bqeskk", "id": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: Manager daemon smithi117.bqeskk is now available 2024-03-22T06:14:39.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:14:39.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:14:39.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.bqeskk/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:14:39.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.bqeskk/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:14:39.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.bqeskk/trash_purge_schedule"}]: dispatch 2024-03-22T06:14:39.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:39 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.bqeskk/trash_purge_schedule"}]: dispatch 2024-03-22T06:14:39.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: pgmap v247: 65 pgs: 65 active+clean; 71 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 5.8 MiB/s wr, 462 op/s 2024-03-22T06:14:39.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:39.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:39.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:14:39.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:14:39.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:14:39.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:14:39.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr fail", "who": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:14:39.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: osdmap e47: 6 total, 6 up, 6 in 2024-03-22T06:14:39.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14212 172.21.15.6:0/348738352' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "mgr fail", "who": "smithi006.sfpxdy"}]': finished 2024-03-22T06:14:39.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: mgrmap e21: smithi117.bqeskk(active, starting, since 0.00920251s) 2024-03-22T06:14:39.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mgr metadata", "who": "smithi117.bqeskk", "id": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T06:14:39.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T06:14:39.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T06:14:39.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: Manager daemon smithi117.bqeskk is now available 2024-03-22T06:14:39.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:14:39.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:14:39.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.bqeskk/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:14:39.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.bqeskk/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:14:39.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.bqeskk/trash_purge_schedule"}]: dispatch 2024-03-22T06:14:39.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:39 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.bqeskk/trash_purge_schedule"}]: dispatch 2024-03-22T06:14:40.274 INFO:tasks.workunit.client.0.smithi006.stdout:seed = 1710822888 2024-03-22T06:14:40.381 INFO:tasks.workunit.client.0.smithi006.stdout:0/0: stat - no entries 2024-03-22T06:14:40.381 INFO:tasks.workunit.client.0.smithi006.stdout:0/1: stat - no entries 2024-03-22T06:14:40.381 INFO:tasks.workunit.client.0.smithi006.stdout:0/2: dread - no filename 2024-03-22T06:14:40.381 INFO:tasks.workunit.client.0.smithi006.stdout:0/3: write - no filename 2024-03-22T06:14:40.382 INFO:tasks.workunit.client.0.smithi006.stdout:1/0: chown . 63446408 1 2024-03-22T06:14:40.385 INFO:tasks.workunit.client.0.smithi006.stdout:0/4: creat f0 x:0 0 0 2024-03-22T06:14:40.386 INFO:tasks.workunit.client.0.smithi006.stdout:0/5: stat f0 0 2024-03-22T06:14:40.391 INFO:tasks.workunit.client.0.smithi006.stdout:1/1: symlink l0 0 2024-03-22T06:14:40.392 INFO:tasks.workunit.client.0.smithi006.stdout:0/6: symlink l1 0 2024-03-22T06:14:40.395 INFO:tasks.workunit.client.0.smithi006.stdout:2/0: creat f0 x:0 0 0 2024-03-22T06:14:40.395 INFO:tasks.workunit.client.0.smithi006.stdout:2/1: write f0 [322117,101674] 0 2024-03-22T06:14:40.395 INFO:tasks.workunit.client.0.smithi006.stdout:2/2: write f0 [1100362,99929] 0 2024-03-22T06:14:40.395 INFO:tasks.workunit.client.0.smithi006.stdout:2/3: read f0 [141210,18403] 0 2024-03-22T06:14:40.395 INFO:tasks.workunit.client.0.smithi006.stdout:2/4: creat f1 x:0 0 0 2024-03-22T06:14:40.396 INFO:tasks.workunit.client.0.smithi006.stdout:1/2: creat f1 x:0 0 0 2024-03-22T06:14:40.398 INFO:tasks.workunit.client.0.smithi006.stdout:0/7: mknod c2 0 2024-03-22T06:14:40.398 INFO:tasks.workunit.client.0.smithi006.stdout:0/8: write f0 [684297,110022] 0 2024-03-22T06:14:40.398 INFO:tasks.workunit.client.0.smithi006.stdout:0/9: truncate f0 1163568 0 2024-03-22T06:14:40.400 INFO:tasks.workunit.client.0.smithi006.stdout:4/0: truncate - no filename 2024-03-22T06:14:40.400 INFO:tasks.workunit.client.0.smithi006.stdout:4/1: write - no filename 2024-03-22T06:14:40.402 INFO:tasks.workunit.client.0.smithi006.stdout:3/0: mkdir d0 0 2024-03-22T06:14:40.402 INFO:tasks.workunit.client.0.smithi006.stdout:3/1: truncate - no filename 2024-03-22T06:14:40.402 INFO:tasks.workunit.client.0.smithi006.stdout:3/2: readlink - no filename 2024-03-22T06:14:40.402 INFO:tasks.workunit.client.0.smithi006.stdout:3/3: getdents d0 0 2024-03-22T06:14:40.409 INFO:tasks.workunit.client.0.smithi006.stdout:4/2: creat f0 x:0 0 0 2024-03-22T06:14:40.409 INFO:tasks.workunit.client.0.smithi006.stdout:4/3: creat f1 x:0 0 0 2024-03-22T06:14:40.409 INFO:tasks.workunit.client.0.smithi006.stdout:4/4: readlink - no filename 2024-03-22T06:14:40.409 INFO:tasks.workunit.client.0.smithi006.stdout:4/5: write f0 [879277,91047] 0 2024-03-22T06:14:40.411 INFO:tasks.workunit.client.0.smithi006.stdout:5/0: mkdir d0 0 2024-03-22T06:14:40.411 INFO:tasks.workunit.client.0.smithi006.stdout:5/1: write - no filename 2024-03-22T06:14:40.411 INFO:tasks.workunit.client.0.smithi006.stdout:5/2: write - no filename 2024-03-22T06:14:40.412 INFO:tasks.workunit.client.0.smithi006.stdout:5/3: dwrite - no filename 2024-03-22T06:14:40.418 INFO:tasks.workunit.client.0.smithi006.stdout:5/4: creat d0/f1 x:0 0 0 2024-03-22T06:14:40.418 INFO:tasks.workunit.client.0.smithi006.stdout:5/5: truncate d0/f1 741739 0 2024-03-22T06:14:40.422 INFO:tasks.workunit.client.0.smithi006.stdout:5/6: mknod d0/c2 0 2024-03-22T06:14:40.422 INFO:tasks.workunit.client.0.smithi006.stdout:5/7: creat d0/f3 x:0 0 0 2024-03-22T06:14:40.493 INFO:tasks.workunit.client.0.smithi006.stdout:1/3: dwrite f1 [0,4194304] 0 2024-03-22T06:14:40.684 INFO:tasks.workunit.client.0.smithi006.stdout:1/4: write f1 [2022873,67377] 0 2024-03-22T06:14:40.706 INFO:tasks.workunit.client.0.smithi006.stdout:7/0: dwrite - no filename 2024-03-22T06:14:40.706 INFO:tasks.workunit.client.0.smithi006.stdout:7/1: dwrite - no filename 2024-03-22T06:14:40.716 INFO:tasks.workunit.client.1.smithi117.stderr:+ tar xzf ltp-full.tgz 2024-03-22T06:14:40.730 INFO:tasks.workunit.client.0.smithi006.stdout:1/5: dread f1 [0,4194304] 0 2024-03-22T06:14:40.730 INFO:tasks.workunit.client.0.smithi006.stdout:1/6: write f1 [4020299,36825] 0 2024-03-22T06:14:40.784 INFO:tasks.workunit.client.0.smithi006.stdout:4/6: dwrite f0 [0,4194304] 0 2024-03-22T06:14:40.795 INFO:tasks.workunit.client.0.smithi006.stdout:4/7: dread f0 [0,4194304] 0 2024-03-22T06:14:40.800 INFO:tasks.workunit.client.0.smithi006.stdout:4/8: write f0 [2652522,102778] 0 2024-03-22T06:14:41.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:40 smithi117 ceph-mon[36514]: Deploying cephadm binary to smithi117 2024-03-22T06:14:41.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:40 smithi117 ceph-mon[36514]: Deploying cephadm binary to smithi006 2024-03-22T06:14:41.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:40 smithi117 ceph-mon[36514]: mgrmap e22: smithi117.bqeskk(active, since 1.00874s) 2024-03-22T06:14:41.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:40 smithi117 ceph-mon[36514]: pgmap v3: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:14:41.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:40 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:41.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:40 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:41.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:40 smithi006 ceph-mon[29301]: Deploying cephadm binary to smithi117 2024-03-22T06:14:41.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:40 smithi006 ceph-mon[29301]: Deploying cephadm binary to smithi006 2024-03-22T06:14:41.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:40 smithi006 ceph-mon[29301]: mgrmap e22: smithi117.bqeskk(active, since 1.00874s) 2024-03-22T06:14:41.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:40 smithi006 ceph-mon[29301]: pgmap v3: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:14:41.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:40 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:41.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:40 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:41.234 INFO:tasks.workunit.client.0.smithi006.stdout:0/10: fsync f0 0 2024-03-22T06:14:41.235 INFO:tasks.workunit.client.0.smithi006.stdout:0/11: write f0 [880335,99979] 0 2024-03-22T06:14:41.258 INFO:tasks.workunit.client.0.smithi006.stdout:2/5: getdents . 0 2024-03-22T06:14:41.264 INFO:tasks.workunit.client.0.smithi006.stdout:0/12: dwrite f0 [0,4194304] 0 2024-03-22T06:14:41.264 INFO:tasks.workunit.client.0.smithi006.stdout:0/13: chown l1 223837 1 2024-03-22T06:14:41.289 INFO:tasks.workunit.client.0.smithi006.stdout:0/14: dwrite f0 [0,4194304] 0 2024-03-22T06:14:41.298 INFO:tasks.workunit.client.0.smithi006.stdout:0/15: read f0 [3505177,101097] 0 2024-03-22T06:14:41.303 INFO:tasks.workunit.client.0.smithi006.stdout:0/16: dread f0 [0,4194304] 0 2024-03-22T06:14:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:41 smithi006 ceph-mon[29301]: [22/Mar/2024:06:14:40] ENGINE Bus STARTING 2024-03-22T06:14:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:41 smithi006 ceph-mon[29301]: [22/Mar/2024:06:14:40] ENGINE Serving on http://172.21.15.117:8765 2024-03-22T06:14:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:41 smithi006 ceph-mon[29301]: [22/Mar/2024:06:14:40] ENGINE Serving on https://172.21.15.117:7150 2024-03-22T06:14:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:41 smithi006 ceph-mon[29301]: [22/Mar/2024:06:14:40] ENGINE Bus STARTED 2024-03-22T06:14:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:41 smithi006 ceph-mon[29301]: pgmap v4: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:14:42.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:41 smithi117 ceph-mon[36514]: [22/Mar/2024:06:14:40] ENGINE Bus STARTING 2024-03-22T06:14:42.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:41 smithi117 ceph-mon[36514]: [22/Mar/2024:06:14:40] ENGINE Serving on http://172.21.15.117:8765 2024-03-22T06:14:42.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:41 smithi117 ceph-mon[36514]: [22/Mar/2024:06:14:40] ENGINE Serving on https://172.21.15.117:7150 2024-03-22T06:14:42.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:41 smithi117 ceph-mon[36514]: [22/Mar/2024:06:14:40] ENGINE Bus STARTED 2024-03-22T06:14:42.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:41 smithi117 ceph-mon[36514]: pgmap v4: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:14:42.052 INFO:tasks.workunit.client.0.smithi006.stdout:6/0: creat f0 x:0 0 0 2024-03-22T06:14:42.052 INFO:tasks.workunit.client.0.smithi006.stdout:6/1: creat f1 x:0 0 0 2024-03-22T06:14:42.052 INFO:tasks.workunit.client.0.smithi006.stdout:6/2: dread - f1 zero size 2024-03-22T06:14:42.052 INFO:tasks.workunit.client.0.smithi006.stdout:6/3: rmdir - no directory 2024-03-22T06:14:42.052 INFO:tasks.workunit.client.0.smithi006.stdout:6/4: dread - f1 zero size 2024-03-22T06:14:42.052 INFO:tasks.workunit.client.0.smithi006.stdout:6/5: dread - f1 zero size 2024-03-22T06:14:42.053 INFO:tasks.workunit.client.0.smithi006.stdout:8/0: symlink l0 0 2024-03-22T06:14:42.053 INFO:tasks.workunit.client.0.smithi006.stdout:8/1: fsync - no filename 2024-03-22T06:14:42.055 INFO:tasks.workunit.client.0.smithi006.stdout:9/0: creat f0 x:0 0 0 2024-03-22T06:14:42.055 INFO:tasks.workunit.client.0.smithi006.stdout:9/1: write f0 [702941,34726] 0 2024-03-22T06:14:42.058 INFO:tasks.workunit.client.0.smithi006.stdout:7/2: creat f0 x:0 0 0 2024-03-22T06:14:42.058 INFO:tasks.workunit.client.0.smithi006.stdout:7/3: creat f1 x:0 0 0 2024-03-22T06:14:42.077 INFO:tasks.workunit.client.0.smithi006.stdout:7/4: dwrite f1 [0,4194304] 0 2024-03-22T06:14:42.092 INFO:tasks.workunit.client.0.smithi006.stdout:7/5: dwrite f0 [0,4194304] 0 2024-03-22T06:14:42.100 INFO:tasks.workunit.client.0.smithi006.stdout:7/6: write f1 [2679183,37473] 0 2024-03-22T06:14:42.156 INFO:tasks.workunit.client.0.smithi006.stdout:1/7: symlink l2 0 2024-03-22T06:14:42.156 INFO:tasks.workunit.client.0.smithi006.stdout:1/8: write f1 [181424,25940] 0 2024-03-22T06:14:42.156 INFO:tasks.workunit.client.0.smithi006.stdout:1/9: chown f1 2880 1 2024-03-22T06:14:42.157 INFO:tasks.workunit.client.0.smithi006.stdout:1/10: write f1 [3087686,62093] 0 2024-03-22T06:14:42.157 INFO:tasks.workunit.client.0.smithi006.stdout:1/11: creat f3 x:0 0 0 2024-03-22T06:14:42.173 INFO:tasks.workunit.client.0.smithi006.stdout:1/12: dwrite f3 [0,4194304] 0 2024-03-22T06:14:42.183 INFO:tasks.workunit.client.0.smithi006.stdout:1/13: dread f1 [0,4194304] 0 2024-03-22T06:14:42.183 INFO:tasks.workunit.client.0.smithi006.stdout:1/14: write f1 [632081,76408] 0 2024-03-22T06:14:42.183 INFO:tasks.workunit.client.0.smithi006.stdout:1/15: readlink l2 0 2024-03-22T06:14:42.648 INFO:tasks.workunit.client.0.smithi006.stdout:2/6: link f0 f2 0 2024-03-22T06:14:42.648 INFO:tasks.workunit.client.0.smithi006.stdout:2/7: truncate f2 1251439 0 2024-03-22T06:14:42.652 INFO:tasks.workunit.client.0.smithi006.stdout:6/6: unlink f0 0 2024-03-22T06:14:42.654 INFO:tasks.workunit.client.0.smithi006.stdout:8/2: creat f1 x:0 0 0 2024-03-22T06:14:42.654 INFO:tasks.workunit.client.0.smithi006.stdout:8/3: creat f2 x:0 0 0 2024-03-22T06:14:42.654 INFO:tasks.workunit.client.0.smithi006.stdout:8/4: chown f2 398179 1 2024-03-22T06:14:42.661 INFO:tasks.workunit.client.0.smithi006.stdout:7/7: link f0 f2 0 2024-03-22T06:14:42.666 INFO:tasks.workunit.client.0.smithi006.stdout:1/16: symlink l4 0 2024-03-22T06:14:42.666 INFO:tasks.workunit.client.0.smithi006.stdout:2/8: chown f1 49303593 1 2024-03-22T06:14:42.668 INFO:tasks.workunit.client.0.smithi006.stdout:6/7: rename f1 to f2 0 2024-03-22T06:14:42.668 INFO:tasks.workunit.client.0.smithi006.stdout:6/8: fdatasync f2 0 2024-03-22T06:14:42.669 INFO:tasks.workunit.client.0.smithi006.stdout:7/8: symlink l3 0 2024-03-22T06:14:42.671 INFO:tasks.workunit.client.0.smithi006.stdout:8/5: dwrite f2 [0,4194304] 0 2024-03-22T06:14:42.689 INFO:tasks.workunit.client.0.smithi006.stdout:7/9: write f2 [1394522,12636] 0 2024-03-22T06:14:42.690 INFO:tasks.workunit.client.0.smithi006.stdout:8/6: write f2 [3190728,102190] 0 2024-03-22T06:14:42.693 INFO:tasks.workunit.client.0.smithi006.stdout:7/10: link f1 f4 0 2024-03-22T06:14:42.697 INFO:tasks.workunit.client.0.smithi006.stdout:2/9: dwrite f0 [0,4194304] 0 2024-03-22T06:14:42.698 INFO:tasks.workunit.client.0.smithi006.stdout:2/10: creat f3 x:0 0 0 2024-03-22T06:14:42.703 INFO:tasks.workunit.client.0.smithi006.stdout:8/7: dread f2 [0,4194304] 0 2024-03-22T06:14:42.704 INFO:tasks.workunit.client.0.smithi006.stdout:8/8: creat f3 x:0 0 0 2024-03-22T06:14:42.706 INFO:tasks.workunit.client.0.smithi006.stdout:8/9: symlink l4 0 2024-03-22T06:14:42.706 INFO:tasks.workunit.client.0.smithi006.stdout:8/10: read - f3 zero size 2024-03-22T06:14:42.706 INFO:tasks.workunit.client.0.smithi006.stdout:8/11: write f1 [951979,86383] 0 2024-03-22T06:14:42.708 INFO:tasks.workunit.client.0.smithi006.stdout:8/12: rename f3 to f5 0 2024-03-22T06:14:42.709 INFO:tasks.workunit.client.0.smithi006.stdout:8/13: write f1 [51822,122615] 0 2024-03-22T06:14:42.709 INFO:tasks.workunit.client.0.smithi006.stdout:8/14: creat f6 x:0 0 0 2024-03-22T06:14:42.712 INFO:tasks.workunit.client.0.smithi006.stdout:8/15: rename f6 to f7 0 2024-03-22T06:14:42.712 INFO:tasks.workunit.client.0.smithi006.stdout:8/16: chown f1 59404 1 2024-03-22T06:14:42.712 INFO:tasks.workunit.client.0.smithi006.stdout:8/17: write f2 [3439546,65188] 0 2024-03-22T06:14:42.713 INFO:tasks.workunit.client.0.smithi006.stdout:8/18: mkdir d8 0 2024-03-22T06:14:42.714 INFO:tasks.workunit.client.0.smithi006.stdout:8/19: rmdir d8 0 2024-03-22T06:14:42.714 INFO:tasks.workunit.client.0.smithi006.stdout:8/20: write f5 [434083,80573] 0 2024-03-22T06:14:42.719 INFO:tasks.workunit.client.0.smithi006.stdout:8/21: mkdir d9 0 2024-03-22T06:14:42.719 INFO:tasks.workunit.client.0.smithi006.stdout:2/11: dwrite f3 [0,4194304] 0 2024-03-22T06:14:42.719 INFO:tasks.workunit.client.0.smithi006.stdout:8/22: creat d9/fa x:0 0 0 2024-03-22T06:14:42.719 INFO:tasks.workunit.client.0.smithi006.stdout:7/11: dwrite f1 [0,4194304] 0 2024-03-22T06:14:42.719 INFO:tasks.workunit.client.0.smithi006.stdout:7/12: readlink l3 0 2024-03-22T06:14:42.720 INFO:tasks.workunit.client.0.smithi006.stdout:7/13: mkdir d5 0 2024-03-22T06:14:42.722 INFO:tasks.workunit.client.0.smithi006.stdout:2/12: rename f0 to f4 0 2024-03-22T06:14:42.724 INFO:tasks.workunit.client.0.smithi006.stdout:7/14: creat d5/f6 x:0 0 0 2024-03-22T06:14:42.724 INFO:tasks.workunit.client.0.smithi006.stdout:7/15: fdatasync f4 0 2024-03-22T06:14:42.724 INFO:tasks.workunit.client.0.smithi006.stdout:7/16: creat d5/f7 x:0 0 0 2024-03-22T06:14:42.726 INFO:tasks.workunit.client.0.smithi006.stdout:2/13: symlink l5 0 2024-03-22T06:14:42.726 INFO:tasks.workunit.client.0.smithi006.stdout:2/14: dread - f1 zero size 2024-03-22T06:14:42.726 INFO:tasks.workunit.client.0.smithi006.stdout:2/15: write f1 [289523,128680] 0 2024-03-22T06:14:42.741 INFO:tasks.workunit.client.0.smithi006.stdout:8/23: dwrite f2 [0,4194304] 0 2024-03-22T06:14:42.744 INFO:tasks.workunit.client.0.smithi006.stdout:7/17: dwrite f2 [0,4194304] 0 2024-03-22T06:14:42.766 INFO:tasks.workunit.client.0.smithi006.stdout:8/24: dwrite f2 [0,4194304] 0 2024-03-22T06:14:42.766 INFO:tasks.workunit.client.0.smithi006.stdout:7/18: dwrite f4 [0,4194304] 0 2024-03-22T06:14:42.766 INFO:tasks.workunit.client.0.smithi006.stdout:7/19: creat d5/f8 x:0 0 0 2024-03-22T06:14:42.781 INFO:tasks.workunit.client.0.smithi006.stdout:7/20: dwrite d5/f7 [0,4194304] 0 2024-03-22T06:14:42.942 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:42 smithi006 ceph-mon[29301]: mgrmap e23: smithi117.bqeskk(active, since 3s) 2024-03-22T06:14:43.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:42 smithi117 ceph-mon[36514]: mgrmap e23: smithi117.bqeskk(active, since 3s) 2024-03-22T06:14:43.243 INFO:tasks.workunit.client.0.smithi006.stdout:0/17: truncate f0 1689083 0 2024-03-22T06:14:43.247 INFO:tasks.workunit.client.0.smithi006.stdout:0/18: symlink l3 0 2024-03-22T06:14:43.251 INFO:tasks.workunit.client.0.smithi006.stdout:4/9: truncate f0 2649949 0 2024-03-22T06:14:43.251 INFO:tasks.workunit.client.0.smithi006.stdout:4/10: chown f0 233920841 1 2024-03-22T06:14:43.251 INFO:tasks.workunit.client.0.smithi006.stdout:4/11: chown f1 12 1 2024-03-22T06:14:43.251 INFO:tasks.workunit.client.0.smithi006.stdout:4/12: creat f2 x:0 0 0 2024-03-22T06:14:43.251 INFO:tasks.workunit.client.0.smithi006.stdout:4/13: creat f3 x:0 0 0 2024-03-22T06:14:43.251 INFO:tasks.workunit.client.0.smithi006.stdout:4/14: creat f4 x:0 0 0 2024-03-22T06:14:43.257 INFO:tasks.workunit.client.0.smithi006.stdout:6/9: rename f2 to f3 0 2024-03-22T06:14:43.257 INFO:tasks.workunit.client.0.smithi006.stdout:6/10: creat f4 x:0 0 0 2024-03-22T06:14:43.257 INFO:tasks.workunit.client.0.smithi006.stdout:6/11: creat f5 x:0 0 0 2024-03-22T06:14:43.266 INFO:tasks.workunit.client.0.smithi006.stdout:4/15: unlink f4 0 2024-03-22T06:14:43.266 INFO:tasks.workunit.client.0.smithi006.stdout:4/16: rmdir - no directory 2024-03-22T06:14:43.267 INFO:tasks.workunit.client.0.smithi006.stdout:7/21: fsync f4 0 2024-03-22T06:14:43.275 INFO:tasks.workunit.client.0.smithi006.stdout:7/22: dread f2 [0,4194304] 0 2024-03-22T06:14:43.283 INFO:tasks.workunit.client.0.smithi006.stdout:2/16: getdents . 0 2024-03-22T06:14:43.285 INFO:tasks.workunit.client.0.smithi006.stdout:2/17: fdatasync f1 0 2024-03-22T06:14:43.285 INFO:tasks.workunit.client.0.smithi006.stdout:8/25: truncate f1 384123 0 2024-03-22T06:14:43.288 INFO:tasks.workunit.client.0.smithi006.stdout:2/18: link f2 f6 0 2024-03-22T06:14:43.288 INFO:tasks.workunit.client.0.smithi006.stdout:2/19: fdatasync f3 0 2024-03-22T06:14:43.289 INFO:tasks.workunit.client.0.smithi006.stdout:7/23: rmdir d5 39 2024-03-22T06:14:43.289 INFO:tasks.workunit.client.0.smithi006.stdout:7/24: stat l3 0 2024-03-22T06:14:43.292 INFO:tasks.workunit.client.0.smithi006.stdout:2/20: creat f7 x:0 0 0 2024-03-22T06:14:43.293 INFO:tasks.workunit.client.0.smithi006.stdout:7/25: dread f2 [0,4194304] 0 2024-03-22T06:14:43.294 INFO:tasks.workunit.client.0.smithi006.stdout:2/21: mknod c8 0 2024-03-22T06:14:43.297 INFO:tasks.workunit.client.0.smithi006.stdout:7/26: rename d5/f6 to d5/f9 0 2024-03-22T06:14:43.299 INFO:tasks.workunit.client.0.smithi006.stdout:7/27: rmdir d5 39 2024-03-22T06:14:43.299 INFO:tasks.workunit.client.0.smithi006.stdout:7/28: write f1 [4438955,29983] 0 2024-03-22T06:14:43.301 INFO:tasks.workunit.client.0.smithi006.stdout:2/22: write f3 [3946935,124878] 0 2024-03-22T06:14:43.302 INFO:tasks.workunit.client.0.smithi006.stdout:7/29: symlink d5/la 0 2024-03-22T06:14:43.549 INFO:tasks.workunit.client.0.smithi006.stdout:6/12: unlink f3 0 2024-03-22T06:14:43.557 INFO:tasks.workunit.client.0.smithi006.stdout:4/17: dwrite f0 [0,4194304] 0 2024-03-22T06:14:43.560 INFO:tasks.workunit.client.0.smithi006.stdout:4/18: mknod c5 0 2024-03-22T06:14:43.571 INFO:tasks.workunit.client.0.smithi006.stdout:6/13: dwrite f4 [0,4194304] 0 2024-03-22T06:14:43.571 INFO:tasks.workunit.client.0.smithi006.stdout:6/14: fsync f4 0 2024-03-22T06:14:43.579 INFO:tasks.workunit.client.0.smithi006.stdout:6/15: mkdir d6 0 2024-03-22T06:14:43.582 INFO:tasks.workunit.client.0.smithi006.stdout:4/19: dwrite f3 [0,4194304] 0 2024-03-22T06:14:43.588 INFO:tasks.workunit.client.0.smithi006.stdout:6/16: dread f4 [0,4194304] 0 2024-03-22T06:14:43.588 INFO:tasks.workunit.client.0.smithi006.stdout:6/17: truncate f5 75996 0 2024-03-22T06:14:43.590 INFO:tasks.workunit.client.0.smithi006.stdout:4/20: mkdir d6 0 2024-03-22T06:14:43.590 INFO:tasks.workunit.client.0.smithi006.stdout:4/21: write f0 [4958090,124656] 0 2024-03-22T06:14:43.592 INFO:tasks.workunit.client.0.smithi006.stdout:6/18: link f4 d6/f7 0 2024-03-22T06:14:43.592 INFO:tasks.workunit.client.0.smithi006.stdout:2/23: getdents . 0 2024-03-22T06:14:43.594 INFO:tasks.workunit.client.0.smithi006.stdout:2/24: dread f1 [0,4194304] 0 2024-03-22T06:14:43.595 INFO:tasks.workunit.client.0.smithi006.stdout:4/22: creat d6/f7 x:0 0 0 2024-03-22T06:14:43.596 INFO:tasks.workunit.client.0.smithi006.stdout:4/23: chown f1 6082 1 2024-03-22T06:14:43.608 INFO:tasks.workunit.client.0.smithi006.stdout:4/24: link f2 d6/f8 0 2024-03-22T06:14:43.608 INFO:tasks.workunit.client.0.smithi006.stdout:4/25: dread - f1 zero size 2024-03-22T06:14:43.608 INFO:tasks.workunit.client.0.smithi006.stdout:4/26: fsync f3 0 2024-03-22T06:14:43.608 INFO:tasks.workunit.client.0.smithi006.stdout:4/27: truncate d6/f7 865509 0 2024-03-22T06:14:43.608 INFO:tasks.workunit.client.0.smithi006.stdout:4/28: creat d6/f9 x:0 0 0 2024-03-22T06:14:43.608 INFO:tasks.workunit.client.0.smithi006.stdout:4/29: creat d6/fa x:0 0 0 2024-03-22T06:14:43.608 INFO:tasks.workunit.client.0.smithi006.stdout:4/30: dread - f1 zero size 2024-03-22T06:14:43.608 INFO:tasks.workunit.client.0.smithi006.stdout:4/31: write f2 [522816,45775] 0 2024-03-22T06:14:43.608 INFO:tasks.workunit.client.0.smithi006.stdout:4/32: truncate f2 643671 0 2024-03-22T06:14:43.608 INFO:tasks.workunit.client.0.smithi006.stdout:4/33: creat d6/fb x:0 0 0 2024-03-22T06:14:43.619 INFO:tasks.workunit.client.0.smithi006.stdout:6/19: dwrite f4 [0,4194304] 0 2024-03-22T06:14:43.619 INFO:tasks.workunit.client.0.smithi006.stdout:2/25: dwrite f3 [0,4194304] 0 2024-03-22T06:14:43.619 INFO:tasks.workunit.client.0.smithi006.stdout:2/26: getdents . 0 2024-03-22T06:14:43.620 INFO:tasks.workunit.client.0.smithi006.stdout:6/20: mknod d6/c8 0 2024-03-22T06:14:43.621 INFO:tasks.workunit.client.0.smithi006.stdout:4/34: dread f2 [0,4194304] 0 2024-03-22T06:14:43.621 INFO:tasks.workunit.client.0.smithi006.stdout:4/35: fdatasync d6/f8 0 2024-03-22T06:14:43.625 INFO:tasks.workunit.client.0.smithi006.stdout:4/36: symlink d6/lc 0 2024-03-22T06:14:43.625 INFO:tasks.workunit.client.0.smithi006.stdout:4/37: readlink d6/lc 0 2024-03-22T06:14:43.630 INFO:tasks.workunit.client.0.smithi006.stdout:6/21: read d6/f7 [466929,6512] 0 2024-03-22T06:14:43.645 INFO:tasks.workunit.client.0.smithi006.stdout:4/38: dwrite d6/f8 [0,4194304] 0 2024-03-22T06:14:43.645 INFO:tasks.workunit.client.0.smithi006.stdout:4/39: creat d6/fd x:0 0 0 2024-03-22T06:14:43.645 INFO:tasks.workunit.client.0.smithi006.stdout:4/40: chown d6/f7 683104949 1 2024-03-22T06:14:43.650 INFO:tasks.workunit.client.0.smithi006.stdout:4/41: link d6/fa d6/fe 0 2024-03-22T06:14:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:43 smithi006 ceph-mon[29301]: pgmap v5: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:14:43.814 INFO:tasks.workunit.client.0.smithi006.stdout:0/19: truncate f0 348639 0 2024-03-22T06:14:43.814 INFO:tasks.workunit.client.0.smithi006.stdout:0/20: creat f4 x:0 0 0 2024-03-22T06:14:43.823 INFO:tasks.workunit.client.0.smithi006.stdout:6/22: fsync d6/f7 0 2024-03-22T06:14:43.823 INFO:tasks.workunit.client.0.smithi006.stdout:6/23: chown d6/f7 1 1 2024-03-22T06:14:43.839 INFO:tasks.workunit.client.0.smithi006.stdout:6/24: dwrite d6/f7 [0,4194304] 0 2024-03-22T06:14:43.843 INFO:tasks.workunit.client.0.smithi006.stdout:6/25: mknod d6/c9 0 2024-03-22T06:14:43.843 INFO:tasks.workunit.client.0.smithi006.stdout:6/26: fsync f5 0 2024-03-22T06:14:43.846 INFO:tasks.workunit.client.0.smithi006.stdout:2/27: write f4 [4335200,6959] 0 2024-03-22T06:14:43.848 INFO:tasks.workunit.client.0.smithi006.stdout:6/27: symlink d6/la 0 2024-03-22T06:14:43.857 INFO:tasks.workunit.client.0.smithi006.stdout:2/28: dread f6 [0,4194304] 0 2024-03-22T06:14:43.857 INFO:tasks.workunit.client.0.smithi006.stdout:2/29: getdents . 0 2024-03-22T06:14:43.859 INFO:tasks.workunit.client.0.smithi006.stdout:2/30: rename f2 to f9 0 2024-03-22T06:14:43.862 INFO:tasks.workunit.client.0.smithi006.stdout:2/31: link c8 ca 0 2024-03-22T06:14:43.868 INFO:tasks.workunit.client.0.smithi006.stdout:6/28: dwrite f4 [0,4194304] 0 2024-03-22T06:14:43.871 INFO:tasks.workunit.client.0.smithi006.stdout:6/29: unlink d6/f7 0 2024-03-22T06:14:43.874 INFO:tasks.workunit.client.0.smithi006.stdout:6/30: mknod d6/cb 0 2024-03-22T06:14:43.876 INFO:tasks.workunit.client.0.smithi006.stdout:4/42: truncate d6/f7 295169 0 2024-03-22T06:14:43.879 INFO:tasks.workunit.client.0.smithi006.stdout:6/31: link f5 d6/fc 0 2024-03-22T06:14:43.883 INFO:tasks.workunit.client.0.smithi006.stdout:2/32: dwrite f1 [0,4194304] 0 2024-03-22T06:14:43.883 INFO:tasks.workunit.client.0.smithi006.stdout:2/33: chown f7 1812056 1 2024-03-22T06:14:43.883 INFO:tasks.workunit.client.0.smithi006.stdout:2/34: readlink l5 0 2024-03-22T06:14:43.884 INFO:tasks.workunit.client.0.smithi006.stdout:6/32: creat d6/fd x:0 0 0 2024-03-22T06:14:43.884 INFO:tasks.workunit.client.0.smithi006.stdout:6/33: dread d6/fc [0,4194304] 0 2024-03-22T06:14:43.895 INFO:tasks.workunit.client.0.smithi006.stdout:2/35: dread f1 [0,4194304] 0 2024-03-22T06:14:43.899 INFO:tasks.workunit.client.0.smithi006.stdout:2/36: write f6 [1032186,12094] 0 2024-03-22T06:14:43.902 INFO:tasks.workunit.client.0.smithi006.stdout:2/37: creat fb x:0 0 0 2024-03-22T06:14:43.902 INFO:tasks.workunit.client.0.smithi006.stdout:2/38: chown f4 2001124 1 2024-03-22T06:14:43.905 INFO:tasks.workunit.client.0.smithi006.stdout:2/39: rename f1 to fc 0 2024-03-22T06:14:43.908 INFO:tasks.workunit.client.0.smithi006.stdout:2/40: symlink ld 0 2024-03-22T06:14:43.911 INFO:tasks.workunit.client.0.smithi006.stdout:2/41: link f7 fe 0 2024-03-22T06:14:43.914 INFO:tasks.workunit.client.0.smithi006.stdout:2/42: mknod cf 0 2024-03-22T06:14:43.917 INFO:tasks.workunit.client.0.smithi006.stdout:2/43: symlink l10 0 2024-03-22T06:14:43.917 INFO:tasks.workunit.client.0.smithi006.stdout:2/44: write f9 [368249,34496] 0 2024-03-22T06:14:43.919 INFO:tasks.workunit.client.0.smithi006.stdout:2/45: mknod c11 0 2024-03-22T06:14:43.919 INFO:tasks.workunit.client.0.smithi006.stdout:2/46: creat f12 x:0 0 0 2024-03-22T06:14:43.924 INFO:tasks.workunit.client.0.smithi006.stdout:2/47: mkdir d13 0 2024-03-22T06:14:43.926 INFO:tasks.workunit.client.0.smithi006.stdout:2/48: mkdir d13/d14 0 2024-03-22T06:14:43.926 INFO:tasks.workunit.client.0.smithi006.stdout:2/49: chown c8 785763184 1 2024-03-22T06:14:43.926 INFO:tasks.workunit.client.0.smithi006.stdout:2/50: dread - f7 zero size 2024-03-22T06:14:43.928 INFO:tasks.workunit.client.0.smithi006.stdout:2/51: creat d13/d14/f15 x:0 0 0 2024-03-22T06:14:43.928 INFO:tasks.workunit.client.0.smithi006.stdout:2/52: truncate f4 5168801 0 2024-03-22T06:14:43.929 INFO:tasks.workunit.client.0.smithi006.stdout:2/53: rmdir d13 39 2024-03-22T06:14:43.931 INFO:tasks.workunit.client.0.smithi006.stdout:2/54: mknod d13/c16 0 2024-03-22T06:14:43.933 INFO:tasks.workunit.client.0.smithi006.stdout:2/55: symlink d13/l17 0 2024-03-22T06:14:43.935 INFO:tasks.workunit.client.0.smithi006.stdout:2/56: creat d13/f18 x:0 0 0 2024-03-22T06:14:43.937 INFO:tasks.workunit.client.0.smithi006.stdout:2/57: mkdir d13/d14/d19 0 2024-03-22T06:14:43.940 INFO:tasks.workunit.client.0.smithi006.stdout:2/58: rename c11 to d13/c1a 0 2024-03-22T06:14:43.940 INFO:tasks.workunit.client.0.smithi006.stdout:2/59: chown d13/f18 0 1 2024-03-22T06:14:44.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:43 smithi117 ceph-mon[36514]: pgmap v5: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:14:44.032 INFO:tasks.workunit.client.0.smithi006.stdout:0/21: dread f0 [0,4194304] 0 2024-03-22T06:14:44.050 INFO:tasks.workunit.client.0.smithi006.stdout:0/22: dwrite f0 [0,4194304] 0 2024-03-22T06:14:44.068 INFO:tasks.workunit.client.0.smithi006.stdout:4/43: rmdir d6 39 2024-03-22T06:14:44.068 INFO:tasks.workunit.client.0.smithi006.stdout:4/44: write d6/f7 [1190839,17528] 0 2024-03-22T06:14:44.071 INFO:tasks.workunit.client.0.smithi006.stdout:0/23: dwrite f0 [0,4194304] 0 2024-03-22T06:14:44.071 INFO:tasks.workunit.client.0.smithi006.stdout:4/45: getdents d6 0 2024-03-22T06:14:44.071 INFO:tasks.workunit.client.0.smithi006.stdout:4/46: fsync d6/f8 0 2024-03-22T06:14:44.074 INFO:tasks.workunit.client.0.smithi006.stdout:0/24: mknod c5 0 2024-03-22T06:14:44.074 INFO:tasks.workunit.client.0.smithi006.stdout:0/25: creat f6 x:0 0 0 2024-03-22T06:14:44.074 INFO:tasks.workunit.client.0.smithi006.stdout:6/34: getdents d6 0 2024-03-22T06:14:44.076 INFO:tasks.workunit.client.0.smithi006.stdout:6/35: dread d6/fc [0,4194304] 0 2024-03-22T06:14:44.082 INFO:tasks.workunit.client.0.smithi006.stdout:0/26: mknod c7 0 2024-03-22T06:14:44.085 INFO:tasks.workunit.client.0.smithi006.stdout:4/47: dwrite d6/fa [0,4194304] 0 2024-03-22T06:14:44.086 INFO:tasks.workunit.client.0.smithi006.stdout:6/36: rename f4 to d6/fe 0 2024-03-22T06:14:44.092 INFO:tasks.workunit.client.0.smithi006.stdout:4/48: rename d6/fd to d6/ff 0 2024-03-22T06:14:44.094 INFO:tasks.workunit.client.0.smithi006.stdout:6/37: mknod d6/cf 0 2024-03-22T06:14:44.096 INFO:tasks.workunit.client.0.smithi006.stdout:4/49: symlink d6/l10 0 2024-03-22T06:14:44.100 INFO:tasks.workunit.client.0.smithi006.stdout:0/27: dwrite f4 [0,4194304] 0 2024-03-22T06:14:44.100 INFO:tasks.workunit.client.0.smithi006.stdout:0/28: fsync f0 0 2024-03-22T06:14:44.100 INFO:tasks.workunit.client.0.smithi006.stdout:6/38: rename d6/la to d6/l10 0 2024-03-22T06:14:44.108 INFO:tasks.workunit.client.0.smithi006.stdout:0/29: mknod c8 0 2024-03-22T06:14:44.113 INFO:tasks.workunit.client.0.smithi006.stdout:0/30: rename f0 to f9 0 2024-03-22T06:14:44.117 INFO:tasks.workunit.client.0.smithi006.stdout:0/31: symlink la 0 2024-03-22T06:14:44.119 INFO:tasks.workunit.client.0.smithi006.stdout:4/50: dwrite d6/fb [0,4194304] 0 2024-03-22T06:14:44.119 INFO:tasks.workunit.client.0.smithi006.stdout:4/51: fsync d6/fe 0 2024-03-22T06:14:44.119 INFO:tasks.workunit.client.0.smithi006.stdout:4/52: write d6/ff [319877,71932] 0 2024-03-22T06:14:44.120 INFO:tasks.workunit.client.0.smithi006.stdout:6/39: dwrite d6/fd [0,4194304] 0 2024-03-22T06:14:44.127 INFO:tasks.workunit.client.0.smithi006.stdout:4/53: creat d6/f11 x:0 0 0 2024-03-22T06:14:44.147 INFO:tasks.workunit.client.0.smithi006.stdout:6/40: rename f5 to d6/f11 0 2024-03-22T06:14:44.150 INFO:tasks.workunit.client.0.smithi006.stdout:4/54: rename d6/fe to d6/f12 0 2024-03-22T06:14:44.150 INFO:tasks.workunit.client.0.smithi006.stdout:4/55: fdatasync d6/fa 0 2024-03-22T06:14:44.151 INFO:tasks.workunit.client.0.smithi006.stdout:4/56: readlink d6/lc 0 2024-03-22T06:14:44.154 INFO:tasks.workunit.client.0.smithi006.stdout:6/41: link d6/c8 d6/c12 0 2024-03-22T06:14:44.158 INFO:tasks.workunit.client.0.smithi006.stdout:6/42: stat d6/l10 0 2024-03-22T06:14:44.158 INFO:tasks.workunit.client.0.smithi006.stdout:6/43: stat d6/fe 0 2024-03-22T06:14:44.158 INFO:tasks.workunit.client.0.smithi006.stdout:6/44: fdatasync d6/fd 0 2024-03-22T06:14:44.159 INFO:tasks.workunit.client.0.smithi006.stdout:6/45: chown d6/f11 185803 1 2024-03-22T06:14:44.167 INFO:tasks.workunit.client.0.smithi006.stdout:6/46: link d6/fd d6/f13 0 2024-03-22T06:14:44.167 INFO:tasks.workunit.client.0.smithi006.stdout:6/47: chown d6/cb 459610 1 2024-03-22T06:14:44.448 INFO:tasks.workunit.client.0.smithi006.stdout:6/48: dwrite d6/fc [0,4194304] 0 2024-03-22T06:14:44.473 INFO:tasks.workunit.client.0.smithi006.stdout:6/49: write d6/f13 [1868875,33937] 0 2024-03-22T06:14:44.486 INFO:tasks.workunit.client.0.smithi006.stdout:6/50: dread d6/fe [0,4194304] 0 2024-03-22T06:14:44.493 INFO:tasks.workunit.client.0.smithi006.stdout:6/51: creat d6/f14 x:0 0 0 2024-03-22T06:14:44.494 INFO:tasks.workunit.client.0.smithi006.stdout:6/52: link d6/cf d6/c15 0 2024-03-22T06:14:44.684 INFO:tasks.workunit.client.0.smithi006.stdout:6/53: link d6/f14 d6/f16 0 2024-03-22T06:14:44.742 INFO:tasks.workunit.client.0.smithi006.stdout:4/57: rmdir d6 39 2024-03-22T06:14:44.819 INFO:tasks.workunit.client.0.smithi006.stdout:6/54: symlink d6/l17 0 2024-03-22T06:14:44.824 INFO:tasks.workunit.client.0.smithi006.stdout:6/55: rename d6/f14 to d6/f18 0 2024-03-22T06:14:44.824 INFO:tasks.workunit.client.0.smithi006.stdout:6/56: creat d6/f19 x:0 0 0 2024-03-22T06:14:44.824 INFO:tasks.workunit.client.0.smithi006.stdout:6/57: dread - d6/f19 zero size 2024-03-22T06:14:44.828 INFO:tasks.workunit.client.0.smithi006.stdout:6/58: symlink d6/l1a 0 2024-03-22T06:14:44.830 INFO:tasks.workunit.client.0.smithi006.stdout:6/59: stat d6/c8 0 2024-03-22T06:14:44.830 INFO:tasks.workunit.client.0.smithi006.stdout:6/60: readlink d6/l10 0 2024-03-22T06:14:44.832 INFO:tasks.workunit.client.0.smithi006.stdout:6/61: mkdir d6/d1b 0 2024-03-22T06:14:44.835 INFO:tasks.workunit.client.0.smithi006.stdout:6/62: creat d6/d1b/f1c x:0 0 0 2024-03-22T06:14:44.835 INFO:tasks.workunit.client.0.smithi006.stdout:6/63: fdatasync d6/f18 0 2024-03-22T06:14:44.837 INFO:tasks.workunit.client.0.smithi006.stdout:6/64: mkdir d6/d1d 0 2024-03-22T06:14:44.837 INFO:tasks.workunit.client.0.smithi006.stdout:6/65: creat d6/f1e x:0 0 0 2024-03-22T06:14:44.841 INFO:tasks.workunit.client.0.smithi006.stdout:6/66: unlink d6/c15 0 2024-03-22T06:14:44.841 INFO:tasks.workunit.client.0.smithi006.stdout:6/67: truncate d6/d1b/f1c 151464 0 2024-03-22T06:14:44.841 INFO:tasks.workunit.client.0.smithi006.stdout:6/68: truncate d6/f18 628448 0 2024-03-22T06:14:44.841 INFO:tasks.workunit.client.0.smithi006.stdout:6/69: truncate d6/f13 4754923 0 2024-03-22T06:14:44.859 INFO:tasks.workunit.client.0.smithi006.stdout:6/70: dwrite d6/f13 [0,4194304] 0 2024-03-22T06:14:44.875 INFO:tasks.workunit.client.0.smithi006.stdout:6/71: write d6/f13 [3036988,124155] 0 2024-03-22T06:14:44.875 INFO:tasks.workunit.client.0.smithi006.stdout:6/72: creat d6/d1b/f1f x:0 0 0 2024-03-22T06:14:45.027 INFO:tasks.workunit.client.0.smithi006.stdout:6/73: link d6/f18 d6/d1d/f20 0 2024-03-22T06:14:45.028 INFO:tasks.workunit.client.0.smithi006.stdout:6/74: dread d6/d1d/f20 [0,4194304] 0 2024-03-22T06:14:45.028 INFO:tasks.workunit.client.0.smithi006.stdout:6/75: stat d6/d1d/f20 0 2024-03-22T06:14:45.029 INFO:tasks.workunit.client.0.smithi006.stdout:6/76: stat d6/fe 0 2024-03-22T06:14:45.029 INFO:tasks.workunit.client.0.smithi006.stdout:6/77: write d6/fd [2107754,67314] 0 2024-03-22T06:14:45.029 INFO:tasks.workunit.client.0.smithi006.stdout:6/78: creat d6/d1b/f21 x:0 0 0 2024-03-22T06:14:45.032 INFO:tasks.workunit.client.0.smithi006.stdout:6/79: symlink d6/d1b/l22 0 2024-03-22T06:14:45.035 INFO:tasks.workunit.client.0.smithi006.stdout:6/80: link d6/fe d6/d1b/f23 0 2024-03-22T06:14:45.039 INFO:tasks.workunit.client.0.smithi006.stdout:6/81: dread d6/fe [0,4194304] 0 2024-03-22T06:14:45.042 INFO:tasks.workunit.client.0.smithi006.stdout:6/82: unlink d6/l10 0 2024-03-22T06:14:45.046 INFO:tasks.workunit.client.0.smithi006.stdout:6/83: dread d6/fe [0,4194304] 0 2024-03-22T06:14:45.048 INFO:tasks.workunit.client.0.smithi006.stdout:6/84: mkdir d6/d24 0 2024-03-22T06:14:45.050 INFO:tasks.workunit.client.0.smithi006.stdout:6/85: symlink d6/d24/l25 0 2024-03-22T06:14:45.074 INFO:tasks.workunit.client.0.smithi006.stdout:6/86: dwrite d6/f18 [0,4194304] 0 2024-03-22T06:14:45.355 INFO:tasks.workunit.client.0.smithi006.stdout:6/87: getdents d6/d1b 0 2024-03-22T06:14:45.364 INFO:tasks.workunit.client.0.smithi006.stdout:6/88: read d6/f16 [573347,23630] 0 2024-03-22T06:14:45.364 INFO:tasks.workunit.client.0.smithi006.stdout:6/89: dread - d6/d1b/f21 zero size 2024-03-22T06:14:45.384 INFO:tasks.workunit.client.0.smithi006.stdout:6/90: dwrite d6/f18 [0,4194304] 0 2024-03-22T06:14:45.387 INFO:tasks.workunit.client.0.smithi006.stdout:6/91: creat d6/d1d/f26 x:0 0 0 2024-03-22T06:14:45.632 INFO:tasks.workunit.client.0.smithi006.stdout:6/92: getdents d6/d1d 0 2024-03-22T06:14:45.632 INFO:tasks.workunit.client.0.smithi006.stdout:6/93: truncate d6/d1b/f21 428666 0 2024-03-22T06:14:45.916 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:45 smithi117 ceph-mon[36514]: pgmap v6: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:14:45.923 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:45 smithi006 ceph-mon[29301]: pgmap v6: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:14:46.844 INFO:tasks.workunit.client.0.smithi006.stdout:6/94: rename d6/c12 to d6/d1b/c27 0 2024-03-22T06:14:46.845 INFO:tasks.workunit.client.0.smithi006.stdout:6/95: write d6/d1b/f1c [1167949,26483] 0 2024-03-22T06:14:46.847 INFO:tasks.workunit.client.0.smithi006.stdout:6/96: creat d6/d1d/f28 x:0 0 0 2024-03-22T06:14:46.847 INFO:tasks.workunit.client.0.smithi006.stdout:6/97: write d6/d1d/f26 [141868,58883] 0 2024-03-22T06:14:46.847 INFO:tasks.workunit.client.0.smithi006.stdout:6/98: fdatasync d6/f11 0 2024-03-22T06:14:46.847 INFO:tasks.workunit.client.0.smithi006.stdout:6/99: stat d6 0 2024-03-22T06:14:46.866 INFO:tasks.workunit.client.0.smithi006.stdout:6/100: dwrite d6/f18 [0,4194304] 0 2024-03-22T06:14:46.875 INFO:tasks.workunit.client.0.smithi006.stdout:6/101: dread d6/f16 [0,4194304] 0 2024-03-22T06:14:46.882 INFO:tasks.workunit.client.0.smithi006.stdout:6/102: write d6/f11 [2680294,19417] 0 2024-03-22T06:14:46.888 INFO:tasks.workunit.client.0.smithi006.stdout:6/103: symlink d6/d1b/l29 0 2024-03-22T06:14:46.890 INFO:tasks.workunit.client.0.smithi006.stdout:6/104: rename d6/d1b/f1f to d6/d1b/f2a 0 2024-03-22T06:14:46.890 INFO:tasks.workunit.client.0.smithi006.stdout:6/105: chown d6/fe 636 1 2024-03-22T06:14:46.891 INFO:tasks.workunit.client.0.smithi006.stdout:6/106: creat d6/d1d/f2b x:0 0 0 2024-03-22T06:14:46.893 INFO:tasks.workunit.client.0.smithi006.stdout:6/107: rmdir d6/d24 39 2024-03-22T06:14:46.897 INFO:tasks.workunit.client.0.smithi006.stdout:6/108: dread d6/d1d/f26 [0,4194304] 0 2024-03-22T06:14:46.898 INFO:tasks.workunit.client.0.smithi006.stdout:6/109: read d6/d1b/f21 [260999,106937] 0 2024-03-22T06:14:46.900 INFO:tasks.workunit.client.0.smithi006.stdout:6/110: mkdir d6/d1b/d2c 0 2024-03-22T06:14:46.900 INFO:tasks.workunit.client.0.smithi006.stdout:6/111: fdatasync d6/d1b/f21 0 2024-03-22T06:14:46.900 INFO:tasks.workunit.client.0.smithi006.stdout:6/112: dread - d6/f1e zero size 2024-03-22T06:14:46.903 INFO:tasks.workunit.client.0.smithi006.stdout:6/113: creat d6/d1b/f2d x:0 0 0 2024-03-22T06:14:46.906 INFO:tasks.workunit.client.0.smithi006.stdout:6/114: symlink d6/d1d/l2e 0 2024-03-22T06:14:46.927 INFO:tasks.workunit.client.0.smithi006.stdout:6/115: dwrite d6/fe [0,4194304] 0 2024-03-22T06:14:46.929 INFO:tasks.workunit.client.0.smithi006.stdout:6/116: link d6/d1b/f2d d6/d1b/d2c/f2f 0 2024-03-22T06:14:46.933 INFO:tasks.workunit.client.0.smithi006.stdout:6/117: unlink d6/d1b/f1c 0 2024-03-22T06:14:47.808 INFO:tasks.workunit.client.0.smithi006.stdout:7/30: sync 2024-03-22T06:14:47.809 INFO:tasks.workunit.client.0.smithi006.stdout:8/26: sync 2024-03-22T06:14:47.809 INFO:tasks.workunit.client.0.smithi006.stdout:1/17: sync 2024-03-22T06:14:47.809 INFO:tasks.workunit.client.0.smithi006.stdout:2/60: sync 2024-03-22T06:14:47.809 INFO:tasks.workunit.client.0.smithi006.stdout:0/32: sync 2024-03-22T06:14:47.809 INFO:tasks.workunit.client.0.smithi006.stdout:5/8: sync 2024-03-22T06:14:47.809 INFO:tasks.workunit.client.0.smithi006.stdout:5/9: readlink - no filename 2024-03-22T06:14:47.809 INFO:tasks.workunit.client.0.smithi006.stdout:0/33: fdatasync f6 0 2024-03-22T06:14:47.809 INFO:tasks.workunit.client.0.smithi006.stdout:0/34: chown f4 26046 1 2024-03-22T06:14:47.809 INFO:tasks.workunit.client.0.smithi006.stdout:5/10: chown d0/c2 110318 1 2024-03-22T06:14:47.809 INFO:tasks.workunit.client.0.smithi006.stdout:0/35: chown c5 0 1 2024-03-22T06:14:47.809 INFO:tasks.workunit.client.0.smithi006.stdout:5/11: write d0/f3 [744833,81710] 0 2024-03-22T06:14:47.810 INFO:tasks.workunit.client.0.smithi006.stdout:4/58: sync 2024-03-22T06:14:47.810 INFO:tasks.workunit.client.0.smithi006.stdout:9/2: sync 2024-03-22T06:14:47.810 INFO:tasks.workunit.client.0.smithi006.stdout:3/4: sync 2024-03-22T06:14:47.810 INFO:tasks.workunit.client.0.smithi006.stdout:3/5: rename d0 to d0/d1 22 2024-03-22T06:14:47.810 INFO:tasks.workunit.client.0.smithi006.stdout:3/6: chown d0 1863985 1 2024-03-22T06:14:47.810 INFO:tasks.workunit.client.0.smithi006.stdout:3/7: write - no filename 2024-03-22T06:14:47.810 INFO:tasks.workunit.client.0.smithi006.stdout:3/8: stat d0 0 2024-03-22T06:14:47.810 INFO:tasks.workunit.client.0.smithi006.stdout:7/31: mkdir d5/db 0 2024-03-22T06:14:47.811 INFO:tasks.workunit.client.0.smithi006.stdout:8/27: mkdir d9/db 0 2024-03-22T06:14:47.812 INFO:tasks.workunit.client.0.smithi006.stdout:8/28: fdatasync f1 0 2024-03-22T06:14:47.812 INFO:tasks.workunit.client.0.smithi006.stdout:8/29: read f1 [265960,5978] 0 2024-03-22T06:14:47.812 INFO:tasks.workunit.client.0.smithi006.stdout:8/30: truncate d9/fa 110220 0 2024-03-22T06:14:47.813 INFO:tasks.workunit.client.0.smithi006.stdout:2/61: symlink d13/d14/l1b 0 2024-03-22T06:14:47.815 INFO:tasks.workunit.client.0.smithi006.stdout:4/59: getdents d6 0 2024-03-22T06:14:47.815 INFO:tasks.workunit.client.0.smithi006.stdout:4/60: chown d6/f11 1254491 1 2024-03-22T06:14:47.815 INFO:tasks.workunit.client.0.smithi006.stdout:4/61: dread - d6/f9 zero size 2024-03-22T06:14:47.817 INFO:tasks.workunit.client.0.smithi006.stdout:3/9: mknod d0/c2 0 2024-03-22T06:14:47.818 INFO:tasks.workunit.client.0.smithi006.stdout:7/32: mknod d5/cc 0 2024-03-22T06:14:47.818 INFO:tasks.workunit.client.0.smithi006.stdout:7/33: chown f0 0 1 2024-03-22T06:14:47.818 INFO:tasks.workunit.client.0.smithi006.stdout:7/34: fdatasync f0 0 2024-03-22T06:14:47.825 INFO:tasks.workunit.client.0.smithi006.stdout:2/62: rename f4 to d13/d14/d19/f1c 0 2024-03-22T06:14:47.825 INFO:tasks.workunit.client.0.smithi006.stdout:0/36: dread f4 [0,4194304] 0 2024-03-22T06:14:47.826 INFO:tasks.workunit.client.0.smithi006.stdout:3/10: unlink d0/c2 0 2024-03-22T06:14:47.826 INFO:tasks.workunit.client.0.smithi006.stdout:3/11: dwrite - no filename 2024-03-22T06:14:47.829 INFO:tasks.workunit.client.0.smithi006.stdout:0/37: truncate f9 1703570 0 2024-03-22T06:14:47.831 INFO:tasks.workunit.client.0.smithi006.stdout:3/12: creat d0/f3 x:0 0 0 2024-03-22T06:14:47.842 INFO:tasks.workunit.client.0.smithi006.stdout:9/3: dwrite f0 [0,4194304] 0 2024-03-22T06:14:47.850 INFO:tasks.workunit.client.0.smithi006.stdout:5/12: dwrite d0/f3 [0,4194304] 0 2024-03-22T06:14:47.850 INFO:tasks.workunit.client.0.smithi006.stdout:5/13: stat d0/f1 0 2024-03-22T06:14:47.850 INFO:tasks.workunit.client.0.smithi006.stdout:5/14: readlink - no filename 2024-03-22T06:14:47.850 INFO:tasks.workunit.client.0.smithi006.stdout:5/15: fsync d0/f3 0 2024-03-22T06:14:47.851 INFO:tasks.workunit.client.0.smithi006.stdout:5/16: dread d0/f1 [0,4194304] 0 2024-03-22T06:14:47.854 INFO:tasks.workunit.client.0.smithi006.stdout:5/17: rmdir d0 39 2024-03-22T06:14:47.855 INFO:tasks.workunit.client.0.smithi006.stdout:5/18: rmdir d0 39 2024-03-22T06:14:47.859 INFO:tasks.workunit.client.0.smithi006.stdout:9/4: write f0 [1307444,121239] 0 2024-03-22T06:14:47.859 INFO:tasks.workunit.client.0.smithi006.stdout:9/5: creat f1 x:0 0 0 2024-03-22T06:14:47.859 INFO:tasks.workunit.client.0.smithi006.stdout:9/6: fdatasync f1 0 2024-03-22T06:14:47.859 INFO:tasks.workunit.client.0.smithi006.stdout:9/7: dread - f1 zero size 2024-03-22T06:14:47.860 INFO:tasks.workunit.client.0.smithi006.stdout:8/31: dwrite f7 [0,4194304] 0 2024-03-22T06:14:47.861 INFO:tasks.workunit.client.0.smithi006.stdout:9/8: rename f0 to f2 0 2024-03-22T06:14:47.861 INFO:tasks.workunit.client.0.smithi006.stdout:9/9: creat f3 x:0 0 0 2024-03-22T06:14:47.862 INFO:tasks.workunit.client.0.smithi006.stdout:9/10: chown f3 124309 1 2024-03-22T06:14:47.865 INFO:tasks.workunit.client.0.smithi006.stdout:4/62: dwrite f1 [0,4194304] 0 2024-03-22T06:14:47.865 INFO:tasks.workunit.client.0.smithi006.stdout:4/63: chown d6/ff 744 1 2024-03-22T06:14:47.865 INFO:tasks.workunit.client.0.smithi006.stdout:2/63: dwrite f9 [0,4194304] 0 2024-03-22T06:14:47.865 INFO:tasks.workunit.client.0.smithi006.stdout:2/64: readlink d13/d14/l1b 0 2024-03-22T06:14:47.865 INFO:tasks.workunit.client.0.smithi006.stdout:7/35: dwrite d5/f8 [0,4194304] 0 2024-03-22T06:14:47.866 INFO:tasks.workunit.client.0.smithi006.stdout:7/36: getdents d5/db 0 2024-03-22T06:14:47.866 INFO:tasks.workunit.client.0.smithi006.stdout:3/13: dwrite d0/f3 [0,4194304] 0 2024-03-22T06:14:47.872 INFO:tasks.workunit.client.0.smithi006.stdout:2/65: symlink d13/d14/l1d 0 2024-03-22T06:14:47.876 INFO:tasks.workunit.client.0.smithi006.stdout:3/14: mkdir d0/d4 0 2024-03-22T06:14:47.878 INFO:tasks.workunit.client.0.smithi006.stdout:2/66: symlink d13/d14/l1e 0 2024-03-22T06:14:47.879 INFO:tasks.workunit.client.0.smithi006.stdout:2/67: mknod d13/d14/c1f 0 2024-03-22T06:14:47.880 INFO:tasks.workunit.client.0.smithi006.stdout:2/68: dread - f12 zero size 2024-03-22T06:14:47.880 INFO:tasks.workunit.client.0.smithi006.stdout:2/69: fdatasync fe 0 2024-03-22T06:14:47.880 INFO:tasks.workunit.client.0.smithi006.stdout:7/37: write f4 [1675462,99076] 0 2024-03-22T06:14:47.880 INFO:tasks.workunit.client.0.smithi006.stdout:2/70: write d13/f18 [215911,76961] 0 2024-03-22T06:14:47.880 INFO:tasks.workunit.client.0.smithi006.stdout:2/71: dread - fb zero size 2024-03-22T06:14:47.880 INFO:tasks.workunit.client.0.smithi006.stdout:2/72: fdatasync fb 0 2024-03-22T06:14:47.888 INFO:tasks.workunit.client.0.smithi006.stdout:5/19: dwrite d0/f3 [0,4194304] 0 2024-03-22T06:14:47.888 INFO:tasks.workunit.client.0.smithi006.stdout:5/20: readlink - no filename 2024-03-22T06:14:47.888 INFO:tasks.workunit.client.0.smithi006.stdout:5/21: chown d0/f3 3756834 1 2024-03-22T06:14:47.889 INFO:tasks.workunit.client.0.smithi006.stdout:5/22: creat d0/f4 x:0 0 0 2024-03-22T06:14:47.892 INFO:tasks.workunit.client.0.smithi006.stdout:3/15: dread d0/f3 [0,4194304] 0 2024-03-22T06:14:47.896 INFO:tasks.workunit.client.0.smithi006.stdout:4/64: dwrite f3 [0,4194304] 0 2024-03-22T06:14:47.896 INFO:tasks.workunit.client.0.smithi006.stdout:8/32: dwrite f2 [0,4194304] 0 2024-03-22T06:14:47.897 INFO:tasks.workunit.client.0.smithi006.stdout:9/11: dread f2 [0,4194304] 0 2024-03-22T06:14:47.897 INFO:tasks.workunit.client.0.smithi006.stdout:3/16: mknod d0/d4/c5 0 2024-03-22T06:14:47.898 INFO:tasks.workunit.client.0.smithi006.stdout:4/65: mkdir d6/d13 0 2024-03-22T06:14:47.898 INFO:tasks.workunit.client.0.smithi006.stdout:4/66: getdents d6/d13 0 2024-03-22T06:14:47.900 INFO:tasks.workunit.client.0.smithi006.stdout:7/38: dread f1 [4194304,4194304] 0 2024-03-22T06:14:47.900 INFO:tasks.workunit.client.0.smithi006.stdout:8/33: mkdir d9/db/dc 0 2024-03-22T06:14:47.902 INFO:tasks.workunit.client.0.smithi006.stdout:3/17: symlink d0/l6 0 2024-03-22T06:14:47.903 INFO:tasks.workunit.client.0.smithi006.stdout:4/67: symlink d6/l14 0 2024-03-22T06:14:47.904 INFO:tasks.workunit.client.0.smithi006.stdout:7/39: rmdir d5 39 2024-03-22T06:14:47.904 INFO:tasks.workunit.client.0.smithi006.stdout:7/40: chown d5/f9 47593 1 2024-03-22T06:14:47.905 INFO:tasks.workunit.client.0.smithi006.stdout:8/34: creat d9/db/dc/fd x:0 0 0 2024-03-22T06:14:47.908 INFO:tasks.workunit.client.0.smithi006.stdout:2/73: dwrite fb [0,4194304] 0 2024-03-22T06:14:47.908 INFO:tasks.workunit.client.0.smithi006.stdout:5/23: dwrite d0/f4 [0,4194304] 0 2024-03-22T06:14:47.912 INFO:tasks.workunit.client.0.smithi006.stdout:2/74: mkdir d13/d20 0 2024-03-22T06:14:47.912 INFO:tasks.workunit.client.0.smithi006.stdout:2/75: fsync fb 0 2024-03-22T06:14:47.912 INFO:tasks.workunit.client.0.smithi006.stdout:2/76: creat d13/f21 x:0 0 0 2024-03-22T06:14:47.913 INFO:tasks.workunit.client.0.smithi006.stdout:2/77: chown d13/d20 1997725 1 2024-03-22T06:14:47.913 INFO:tasks.workunit.client.0.smithi006.stdout:2/78: chown f6 1497 1 2024-03-22T06:14:47.922 INFO:tasks.workunit.client.0.smithi006.stdout:8/35: dread f2 [0,4194304] 0 2024-03-22T06:14:47.922 INFO:tasks.workunit.client.0.smithi006.stdout:8/36: fdatasync d9/fa 0 2024-03-22T06:14:47.923 INFO:tasks.workunit.client.0.smithi006.stdout:8/37: mkdir d9/db/dc/de 0 2024-03-22T06:14:47.924 INFO:tasks.workunit.client.0.smithi006.stdout:8/38: creat d9/db/dc/de/ff x:0 0 0 2024-03-22T06:14:47.925 INFO:tasks.workunit.client.0.smithi006.stdout:7/41: dwrite d5/f9 [0,4194304] 0 2024-03-22T06:14:47.925 INFO:tasks.workunit.client.0.smithi006.stdout:7/42: fdatasync d5/f7 0 2024-03-22T06:14:47.925 INFO:tasks.workunit.client.0.smithi006.stdout:4/68: dwrite d6/fa [0,4194304] 0 2024-03-22T06:14:47.925 INFO:tasks.workunit.client.0.smithi006.stdout:4/69: chown d6/fb 385074 1 2024-03-22T06:14:47.925 INFO:tasks.workunit.client.0.smithi006.stdout:8/39: creat d9/db/f10 x:0 0 0 2024-03-22T06:14:47.927 INFO:tasks.workunit.client.0.smithi006.stdout:2/79: dread d13/f18 [0,4194304] 0 2024-03-22T06:14:47.928 INFO:tasks.workunit.client.0.smithi006.stdout:7/43: creat d5/fd x:0 0 0 2024-03-22T06:14:47.930 INFO:tasks.workunit.client.0.smithi006.stdout:2/80: mknod d13/c22 0 2024-03-22T06:14:47.930 INFO:tasks.workunit.client.0.smithi006.stdout:2/81: chown d13/d14/l1b 3999843 1 2024-03-22T06:14:47.932 INFO:tasks.workunit.client.0.smithi006.stdout:2/82: dread d13/f18 [0,4194304] 0 2024-03-22T06:14:47.932 INFO:tasks.workunit.client.0.smithi006.stdout:2/83: write d13/f21 [656945,74378] 0 2024-03-22T06:14:47.932 INFO:tasks.workunit.client.0.smithi006.stdout:7/44: rename d5/f8 to d5/fe 0 2024-03-22T06:14:47.933 INFO:tasks.workunit.client.0.smithi006.stdout:4/70: read f3 [620863,1324] 0 2024-03-22T06:14:47.935 INFO:tasks.workunit.client.0.smithi006.stdout:4/71: dread d6/f7 [0,4194304] 0 2024-03-22T06:14:47.938 INFO:tasks.workunit.client.0.smithi006.stdout:4/72: rename c5 to d6/d13/c15 0 2024-03-22T06:14:47.944 INFO:tasks.workunit.client.0.smithi006.stdout:8/40: dwrite d9/db/dc/de/ff [0,4194304] 0 2024-03-22T06:14:47.955 INFO:tasks.workunit.client.0.smithi006.stdout:7/45: dwrite f4 [0,4194304] 0 2024-03-22T06:14:47.956 INFO:tasks.workunit.client.0.smithi006.stdout:7/46: chown d5/f9 130714379 1 2024-03-22T06:14:47.956 INFO:tasks.workunit.client.0.smithi006.stdout:7/47: readlink d5/la 0 2024-03-22T06:14:47.956 INFO:tasks.workunit.client.0.smithi006.stdout:4/73: dwrite d6/f7 [0,4194304] 0 2024-03-22T06:14:47.968 INFO:tasks.workunit.client.0.smithi006.stdout:7/48: write f1 [3618218,60473] 0 2024-03-22T06:14:47.968 INFO:tasks.workunit.client.0.smithi006.stdout:7/49: write f4 [906828,19356] 0 2024-03-22T06:14:47.968 INFO:tasks.workunit.client.0.smithi006.stdout:7/50: chown f4 1578 1 2024-03-22T06:14:47.971 INFO:tasks.workunit.client.0.smithi006.stdout:4/74: dwrite f0 [4194304,4194304] 0 2024-03-22T06:14:47.977 INFO:tasks.workunit.client.0.smithi006.stdout:7/51: dread d5/f9 [0,4194304] 0 2024-03-22T06:14:47.977 INFO:tasks.workunit.client.0.smithi006.stdout:0/38: truncate f9 2728668 0 2024-03-22T06:14:47.983 INFO:tasks.workunit.client.0.smithi006.stdout:0/39: mkdir db 0 2024-03-22T06:14:47.983 INFO:tasks.workunit.client.0.smithi006.stdout:0/40: dread - f6 zero size 2024-03-22T06:14:47.987 INFO:tasks.workunit.client.0.smithi006.stdout:9/12: unlink f2 0 2024-03-22T06:14:47.988 INFO:tasks.workunit.client.0.smithi006.stdout:9/13: chown f3 726593320 1 2024-03-22T06:14:47.988 INFO:tasks.workunit.client.0.smithi006.stdout:9/14: creat f4 x:0 0 0 2024-03-22T06:14:47.988 INFO:tasks.workunit.client.0.smithi006.stdout:9/15: chown f3 11752174 1 2024-03-22T06:14:47.988 INFO:tasks.workunit.client.0.smithi006.stdout:9/16: truncate f4 695200 0 2024-03-22T06:14:47.993 INFO:tasks.workunit.client.0.smithi006.stdout:9/17: symlink l5 0 2024-03-22T06:14:47.994 INFO:tasks.workunit.client.0.smithi006.stdout:9/18: chown f4 357 1 2024-03-22T06:14:47.995 INFO:tasks.workunit.client.0.smithi006.stdout:0/41: dread f4 [0,4194304] 0 2024-03-22T06:14:47.995 INFO:tasks.workunit.client.0.smithi006.stdout:4/75: dwrite f2 [4194304,4194304] 0 2024-03-22T06:14:47.995 INFO:tasks.workunit.client.0.smithi006.stdout:4/76: chown f0 3494248 1 2024-03-22T06:14:47.995 INFO:tasks.workunit.client.0.smithi006.stdout:9/19: dread f4 [0,4194304] 0 2024-03-22T06:14:47.995 INFO:tasks.workunit.client.0.smithi006.stdout:9/20: stat l5 0 2024-03-22T06:14:47.995 INFO:tasks.workunit.client.0.smithi006.stdout:9/21: chown f1 234 1 2024-03-22T06:14:47.998 INFO:tasks.workunit.client.0.smithi006.stdout:0/42: mkdir db/dc 0 2024-03-22T06:14:48.003 INFO:tasks.workunit.client.0.smithi006.stdout:5/24: dwrite d0/f3 [4194304,4194304] 0 2024-03-22T06:14:48.003 INFO:tasks.workunit.client.0.smithi006.stdout:0/43: dread f4 [0,4194304] 0 2024-03-22T06:14:48.003 INFO:tasks.workunit.client.0.smithi006.stdout:5/25: creat d0/f5 x:0 0 0 2024-03-22T06:14:48.003 INFO:tasks.workunit.client.0.smithi006.stdout:5/26: read - d0/f5 zero size 2024-03-22T06:14:48.003 INFO:tasks.workunit.client.0.smithi006.stdout:5/27: stat d0/f5 0 2024-03-22T06:14:48.004 INFO:tasks.workunit.client.0.smithi006.stdout:5/28: chown d0/c2 43 1 2024-03-22T06:14:48.004 INFO:tasks.workunit.client.0.smithi006.stdout:5/29: read d0/f1 [294826,4082] 0 2024-03-22T06:14:48.005 INFO:tasks.workunit.client.0.smithi006.stdout:9/22: link f4 f6 0 2024-03-22T06:14:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:47 smithi117 ceph-mon[36514]: pgmap v7: 65 pgs: 65 active+clean; 214 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 25 MiB/s wr, 556 op/s 2024-03-22T06:14:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:47 smithi117 ceph-mon[36514]: Standby manager daemon smithi006.sfpxdy started 2024-03-22T06:14:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:47 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.6:0/928114473' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi006.sfpxdy/crt"}]: dispatch 2024-03-22T06:14:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:47 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.6:0/928114473' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:14:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:47 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.6:0/928114473' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi006.sfpxdy/key"}]: dispatch 2024-03-22T06:14:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:47 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.6:0/928114473' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:14:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:47 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:48.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:47 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:48.008 INFO:tasks.workunit.client.0.smithi006.stdout:2/84: getdents d13/d14 0 2024-03-22T06:14:48.013 INFO:tasks.workunit.client.0.smithi006.stdout:0/44: dread f4 [0,4194304] 0 2024-03-22T06:14:48.013 INFO:tasks.workunit.client.0.smithi006.stdout:0/45: dread - f6 zero size 2024-03-22T06:14:48.016 INFO:tasks.workunit.client.0.smithi006.stdout:0/46: truncate f4 1961228 0 2024-03-22T06:14:48.016 INFO:tasks.workunit.client.0.smithi006.stdout:0/47: chown c8 0 1 2024-03-22T06:14:48.018 INFO:tasks.workunit.client.0.smithi006.stdout:4/77: dwrite f0 [4194304,4194304] 0 2024-03-22T06:14:48.019 INFO:tasks.workunit.client.0.smithi006.stdout:0/48: symlink db/ld 0 2024-03-22T06:14:48.021 INFO:tasks.workunit.client.0.smithi006.stdout:4/78: symlink d6/d13/l16 0 2024-03-22T06:14:48.023 INFO:tasks.workunit.client.0.smithi006.stdout:0/49: symlink db/dc/le 0 2024-03-22T06:14:48.025 INFO:tasks.workunit.client.0.smithi006.stdout:3/18: dwrite d0/f3 [4194304,4194304] 0 2024-03-22T06:14:48.026 INFO:tasks.workunit.client.0.smithi006.stdout:4/79: link f2 d6/d13/f17 0 2024-03-22T06:14:48.034 INFO:tasks.workunit.client.0.smithi006.stdout:8/41: getdents d9/db 0 2024-03-22T06:14:48.034 INFO:tasks.workunit.client.0.smithi006.stdout:8/42: creat d9/f11 x:0 0 0 2024-03-22T06:14:48.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:47 smithi006 ceph-mon[29301]: pgmap v7: 65 pgs: 65 active+clean; 214 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 25 MiB/s wr, 556 op/s 2024-03-22T06:14:48.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:47 smithi006 ceph-mon[29301]: Standby manager daemon smithi006.sfpxdy started 2024-03-22T06:14:48.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:47 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.6:0/928114473' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi006.sfpxdy/crt"}]: dispatch 2024-03-22T06:14:48.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:47 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.6:0/928114473' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:14:48.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:47 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.6:0/928114473' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi006.sfpxdy/key"}]: dispatch 2024-03-22T06:14:48.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:47 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.6:0/928114473' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:14:48.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:47 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:48.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:47 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:48.035 INFO:tasks.workunit.client.0.smithi006.stdout:8/43: mkdir d9/db/d12 0 2024-03-22T06:14:48.035 INFO:tasks.workunit.client.0.smithi006.stdout:8/44: creat d9/f13 x:0 0 0 2024-03-22T06:14:48.036 INFO:tasks.workunit.client.0.smithi006.stdout:7/52: getdents d5 0 2024-03-22T06:14:48.038 INFO:tasks.workunit.client.0.smithi006.stdout:9/23: dwrite f4 [0,4194304] 0 2024-03-22T06:14:48.040 INFO:tasks.workunit.client.0.smithi006.stdout:2/85: dwrite fb [0,4194304] 0 2024-03-22T06:14:48.040 INFO:tasks.workunit.client.0.smithi006.stdout:4/80: write d6/fb [4120327,122537] 0 2024-03-22T06:14:48.040 INFO:tasks.workunit.client.0.smithi006.stdout:2/86: creat d13/f23 x:0 0 0 2024-03-22T06:14:48.042 INFO:tasks.workunit.client.0.smithi006.stdout:7/53: rmdir d5 39 2024-03-22T06:14:48.042 INFO:tasks.workunit.client.0.smithi006.stdout:7/54: truncate d5/fd 905847 0 2024-03-22T06:14:48.051 INFO:tasks.workunit.client.0.smithi006.stdout:9/24: dread f6 [0,4194304] 0 2024-03-22T06:14:48.054 INFO:tasks.workunit.client.0.smithi006.stdout:4/81: symlink d6/l18 0 2024-03-22T06:14:48.057 INFO:tasks.workunit.client.0.smithi006.stdout:2/87: creat d13/d14/d19/f24 x:0 0 0 2024-03-22T06:14:48.057 INFO:tasks.workunit.client.0.smithi006.stdout:2/88: chown d13/d14/l1d 2 1 2024-03-22T06:14:48.066 INFO:tasks.workunit.client.0.smithi006.stdout:7/55: dwrite d5/f7 [0,4194304] 0 2024-03-22T06:14:48.075 INFO:tasks.workunit.client.0.smithi006.stdout:2/89: link l5 d13/d20/l25 0 2024-03-22T06:14:48.076 INFO:tasks.workunit.client.0.smithi006.stdout:2/90: write d13/d14/d19/f24 [1011226,85234] 0 2024-03-22T06:14:48.081 INFO:tasks.workunit.client.0.smithi006.stdout:9/25: fsync f6 0 2024-03-22T06:14:48.081 INFO:tasks.workunit.client.0.smithi006.stdout:9/26: creat f7 x:0 0 0 2024-03-22T06:14:48.081 INFO:tasks.workunit.client.0.smithi006.stdout:9/27: chown l5 640830 1 2024-03-22T06:14:48.082 INFO:tasks.workunit.client.0.smithi006.stdout:7/56: dwrite d5/f7 [0,4194304] 0 2024-03-22T06:14:48.084 INFO:tasks.workunit.client.0.smithi006.stdout:5/30: getdents d0 0 2024-03-22T06:14:48.088 INFO:tasks.workunit.client.0.smithi006.stdout:7/57: write f2 [3667182,48491] 0 2024-03-22T06:14:48.089 INFO:tasks.workunit.client.0.smithi006.stdout:9/28: mkdir d8 0 2024-03-22T06:14:48.089 INFO:tasks.workunit.client.0.smithi006.stdout:9/29: fdatasync f4 0 2024-03-22T06:14:48.092 INFO:tasks.workunit.client.0.smithi006.stdout:9/30: creat d8/f9 x:0 0 0 2024-03-22T06:14:48.092 INFO:tasks.workunit.client.0.smithi006.stdout:9/31: creat d8/fa x:0 0 0 2024-03-22T06:14:48.092 INFO:tasks.workunit.client.0.smithi006.stdout:9/32: creat d8/fb x:0 0 0 2024-03-22T06:14:48.094 INFO:tasks.workunit.client.0.smithi006.stdout:7/58: dread f1 [4194304,4194304] 0 2024-03-22T06:14:48.098 INFO:tasks.workunit.client.0.smithi006.stdout:9/33: read f4 [1162139,13117] 0 2024-03-22T06:14:48.098 INFO:tasks.workunit.client.0.smithi006.stdout:9/34: creat d8/fc x:0 0 0 2024-03-22T06:14:48.098 INFO:tasks.workunit.client.0.smithi006.stdout:9/35: creat d8/fd x:0 0 0 2024-03-22T06:14:48.098 INFO:tasks.workunit.client.0.smithi006.stdout:9/36: chown f4 4 1 2024-03-22T06:14:48.098 INFO:tasks.workunit.client.0.smithi006.stdout:9/37: dread - f3 zero size 2024-03-22T06:14:48.098 INFO:tasks.workunit.client.0.smithi006.stdout:9/38: dread - d8/fa zero size 2024-03-22T06:14:48.098 INFO:tasks.workunit.client.0.smithi006.stdout:9/39: dread - d8/fa zero size 2024-03-22T06:14:48.098 INFO:tasks.workunit.client.0.smithi006.stdout:9/40: chown f6 43 1 2024-03-22T06:14:48.099 INFO:tasks.workunit.client.0.smithi006.stdout:7/59: mknod d5/cf 0 2024-03-22T06:14:48.099 INFO:tasks.workunit.client.0.smithi006.stdout:7/60: write f0 [2962733,55797] 0 2024-03-22T06:14:48.106 INFO:tasks.workunit.client.0.smithi006.stdout:9/41: mknod d8/ce 0 2024-03-22T06:14:48.110 INFO:tasks.workunit.client.0.smithi006.stdout:5/31: dwrite d0/f5 [0,4194304] 0 2024-03-22T06:14:48.117 INFO:tasks.workunit.client.0.smithi006.stdout:5/32: symlink d0/l6 0 2024-03-22T06:14:48.117 INFO:tasks.workunit.client.0.smithi006.stdout:5/33: chown d0/c2 403 1 2024-03-22T06:14:48.123 INFO:tasks.workunit.client.0.smithi006.stdout:7/61: dwrite f1 [4194304,4194304] 0 2024-03-22T06:14:48.123 INFO:tasks.workunit.client.0.smithi006.stdout:9/42: dwrite d8/fa [0,4194304] 0 2024-03-22T06:14:48.125 INFO:tasks.workunit.client.0.smithi006.stdout:7/62: mkdir d5/db/d10 0 2024-03-22T06:14:48.128 INFO:tasks.workunit.client.0.smithi006.stdout:2/91: rename d13/d14 to d13/d20/d26 0 2024-03-22T06:14:48.134 INFO:tasks.workunit.client.0.smithi006.stdout:4/82: truncate d6/f8 4446434 0 2024-03-22T06:14:48.135 INFO:tasks.workunit.client.0.smithi006.stdout:7/63: creat d5/db/d10/f11 x:0 0 0 2024-03-22T06:14:48.137 INFO:tasks.workunit.client.0.smithi006.stdout:2/92: mknod d13/d20/d26/d19/c27 0 2024-03-22T06:14:48.137 INFO:tasks.workunit.client.0.smithi006.stdout:2/93: fsync f6 0 2024-03-22T06:14:48.141 INFO:tasks.workunit.client.0.smithi006.stdout:4/83: symlink d6/l19 0 2024-03-22T06:14:48.145 INFO:tasks.workunit.client.0.smithi006.stdout:2/94: mkdir d13/d20/d26/d19/d28 0 2024-03-22T06:14:48.148 INFO:tasks.workunit.client.0.smithi006.stdout:4/84: symlink d6/l1a 0 2024-03-22T06:14:48.150 INFO:tasks.workunit.client.0.smithi006.stdout:7/64: dwrite d5/f9 [4194304,4194304] 0 2024-03-22T06:14:48.152 INFO:tasks.workunit.client.0.smithi006.stdout:4/85: rmdir d6 39 2024-03-22T06:14:48.152 INFO:tasks.workunit.client.0.smithi006.stdout:4/86: write f0 [8947514,25411] 0 2024-03-22T06:14:48.159 INFO:tasks.workunit.client.0.smithi006.stdout:9/43: rmdir d8 39 2024-03-22T06:14:48.163 INFO:tasks.workunit.client.0.smithi006.stdout:4/87: mkdir d6/d1b 0 2024-03-22T06:14:48.167 INFO:tasks.workunit.client.0.smithi006.stdout:4/88: creat d6/f1c x:0 0 0 2024-03-22T06:14:48.170 INFO:tasks.workunit.client.0.smithi006.stdout:5/34: truncate d0/f5 202535 0 2024-03-22T06:14:48.177 INFO:tasks.workunit.client.0.smithi006.stdout:9/44: dwrite f6 [0,4194304] 0 2024-03-22T06:14:48.177 INFO:tasks.workunit.client.0.smithi006.stdout:9/45: write f7 [323513,8778] 0 2024-03-22T06:14:48.177 INFO:tasks.workunit.client.0.smithi006.stdout:9/46: truncate d8/f9 149672 0 2024-03-22T06:14:48.177 INFO:tasks.workunit.client.0.smithi006.stdout:9/47: chown d8/fd 25 1 2024-03-22T06:14:48.193 INFO:tasks.workunit.client.0.smithi006.stdout:9/48: dwrite f6 [0,4194304] 0 2024-03-22T06:14:48.195 INFO:tasks.workunit.client.0.smithi006.stdout:9/49: creat d8/ff x:0 0 0 2024-03-22T06:14:48.198 INFO:tasks.workunit.client.0.smithi006.stdout:9/50: mkdir d8/d10 0 2024-03-22T06:14:48.198 INFO:tasks.workunit.client.0.smithi006.stdout:9/51: dread - f3 zero size 2024-03-22T06:14:48.198 INFO:tasks.workunit.client.0.smithi006.stdout:2/95: truncate f9 1081676 0 2024-03-22T06:14:48.202 INFO:tasks.workunit.client.0.smithi006.stdout:2/96: mkdir d13/d20/d26/d19/d28/d29 0 2024-03-22T06:14:48.206 INFO:tasks.workunit.client.0.smithi006.stdout:2/97: mkdir d13/d20/d26/d2a 0 2024-03-22T06:14:48.213 INFO:tasks.workunit.client.0.smithi006.stdout:9/52: dwrite d8/fa [0,4194304] 0 2024-03-22T06:14:48.213 INFO:tasks.workunit.client.0.smithi006.stdout:9/53: stat d8/ce 0 2024-03-22T06:14:48.213 INFO:tasks.workunit.client.0.smithi006.stdout:9/54: dread - d8/ff zero size 2024-03-22T06:14:48.215 INFO:tasks.workunit.client.0.smithi006.stdout:7/65: truncate f4 1314223 0 2024-03-22T06:14:48.215 INFO:tasks.workunit.client.0.smithi006.stdout:7/66: chown d5/db/d10/f11 0 1 2024-03-22T06:14:48.220 INFO:tasks.workunit.client.0.smithi006.stdout:5/35: write d0/f5 [935762,78132] 0 2024-03-22T06:14:48.223 INFO:tasks.workunit.client.0.smithi006.stdout:9/55: dread f6 [0,4194304] 0 2024-03-22T06:14:48.223 INFO:tasks.workunit.client.0.smithi006.stdout:9/56: chown d8/ce 626639 1 2024-03-22T06:14:48.241 INFO:tasks.workunit.client.0.smithi006.stdout:5/36: dwrite d0/f5 [0,4194304] 0 2024-03-22T06:14:48.257 INFO:tasks.workunit.client.0.smithi006.stdout:2/98: write f6 [914203,91276] 0 2024-03-22T06:14:48.261 INFO:tasks.workunit.client.0.smithi006.stdout:2/99: mknod d13/d20/d26/d2a/c2b 0 2024-03-22T06:14:48.261 INFO:tasks.workunit.client.0.smithi006.stdout:2/100: stat d13/d20/d26/d19/c27 0 2024-03-22T06:14:48.261 INFO:tasks.workunit.client.0.smithi006.stdout:2/101: chown l5 85996870 1 2024-03-22T06:14:48.265 INFO:tasks.workunit.client.0.smithi006.stdout:5/37: dwrite d0/f3 [0,4194304] 0 2024-03-22T06:14:48.284 INFO:tasks.workunit.client.0.smithi006.stdout:2/102: dwrite fc [0,4194304] 0 2024-03-22T06:14:48.288 INFO:tasks.workunit.client.0.smithi006.stdout:2/103: symlink d13/d20/d26/d2a/l2c 0 2024-03-22T06:14:48.288 INFO:tasks.workunit.client.0.smithi006.stdout:2/104: chown cf 203882 1 2024-03-22T06:14:48.289 INFO:tasks.workunit.client.0.smithi006.stdout:2/105: mknod d13/d20/d26/d19/d28/c2d 0 2024-03-22T06:14:48.291 INFO:tasks.workunit.client.0.smithi006.stdout:5/38: dwrite d0/f4 [0,4194304] 0 2024-03-22T06:14:48.293 INFO:tasks.workunit.client.0.smithi006.stdout:5/39: symlink d0/l7 0 2024-03-22T06:14:48.294 INFO:tasks.workunit.client.0.smithi006.stdout:7/67: dwrite f1 [0,4194304] 0 2024-03-22T06:14:48.295 INFO:tasks.workunit.client.0.smithi006.stdout:5/40: symlink d0/l8 0 2024-03-22T06:14:48.308 INFO:tasks.workunit.client.0.smithi006.stdout:5/41: dread d0/f3 [4194304,4194304] 0 2024-03-22T06:14:48.309 INFO:tasks.workunit.client.0.smithi006.stdout:7/68: dwrite d5/fe [0,4194304] 0 2024-03-22T06:14:48.309 INFO:tasks.workunit.client.0.smithi006.stdout:7/69: chown d5/db/d10 0 1 2024-03-22T06:14:48.309 INFO:tasks.workunit.client.0.smithi006.stdout:7/70: write f2 [4466567,74780] 0 2024-03-22T06:14:48.309 INFO:tasks.workunit.client.0.smithi006.stdout:7/71: fdatasync d5/f7 0 2024-03-22T06:14:48.309 INFO:tasks.workunit.client.0.smithi006.stdout:7/72: write d5/fe [4510797,43091] 0 2024-03-22T06:14:48.309 INFO:tasks.workunit.client.0.smithi006.stdout:7/73: write d5/fd [1129315,121692] 0 2024-03-22T06:14:48.310 INFO:tasks.workunit.client.0.smithi006.stdout:5/42: creat d0/f9 x:0 0 0 2024-03-22T06:14:48.324 INFO:tasks.workunit.client.0.smithi006.stdout:5/43: dwrite d0/f4 [0,4194304] 0 2024-03-22T06:14:48.333 INFO:tasks.workunit.client.0.smithi006.stdout:5/44: dread d0/f4 [0,4194304] 0 2024-03-22T06:14:48.335 INFO:tasks.workunit.client.0.smithi006.stdout:5/45: mknod d0/ca 0 2024-03-22T06:14:48.335 INFO:tasks.workunit.client.0.smithi006.stdout:5/46: readlink d0/l6 0 2024-03-22T06:14:48.335 INFO:tasks.workunit.client.0.smithi006.stdout:5/47: write d0/f1 [1584042,96655] 0 2024-03-22T06:14:48.337 INFO:tasks.workunit.client.0.smithi006.stdout:5/48: mkdir d0/db 0 2024-03-22T06:14:48.341 INFO:tasks.workunit.client.0.smithi006.stdout:5/49: truncate d0/f3 3113895 0 2024-03-22T06:14:48.341 INFO:tasks.workunit.client.0.smithi006.stdout:5/50: write d0/f1 [2583436,13760] 0 2024-03-22T06:14:48.350 INFO:tasks.workunit.client.0.smithi006.stdout:5/51: dread d0/f5 [0,4194304] 0 2024-03-22T06:14:48.350 INFO:tasks.workunit.client.0.smithi006.stdout:5/52: chown d0/ca 27147 1 2024-03-22T06:14:48.379 INFO:tasks.workunit.client.0.smithi006.stdout:7/74: truncate d5/fd 639686 0 2024-03-22T06:14:48.389 INFO:tasks.workunit.client.0.smithi006.stdout:7/75: dread f4 [0,4194304] 0 2024-03-22T06:14:48.410 INFO:tasks.workunit.client.0.smithi006.stdout:7/76: truncate f2 1456957 0 2024-03-22T06:14:48.410 INFO:tasks.workunit.client.0.smithi006.stdout:7/77: chown d5/db/d10/f11 380267874 1 2024-03-22T06:14:48.410 INFO:tasks.workunit.client.0.smithi006.stdout:7/78: chown d5/la 651911 1 2024-03-22T06:14:48.410 INFO:tasks.workunit.client.0.smithi006.stdout:7/79: truncate f4 4339829 0 2024-03-22T06:14:48.476 INFO:tasks.workunit.client.0.smithi006.stdout:7/80: dwrite f0 [0,4194304] 0 2024-03-22T06:14:48.476 INFO:tasks.workunit.client.0.smithi006.stdout:7/81: rename d5/db/d10 to d5/db/d10/d12 22 2024-03-22T06:14:48.478 INFO:tasks.workunit.client.0.smithi006.stdout:7/82: rename f1 to d5/db/f13 0 2024-03-22T06:14:48.478 INFO:tasks.workunit.client.0.smithi006.stdout:7/83: chown d5/la 2198387 1 2024-03-22T06:14:48.478 INFO:tasks.workunit.client.0.smithi006.stdout:7/84: chown d5/db 0 1 2024-03-22T06:14:48.485 INFO:tasks.workunit.client.0.smithi006.stdout:7/85: dread d5/fe [0,4194304] 0 2024-03-22T06:14:48.485 INFO:tasks.workunit.client.0.smithi006.stdout:7/86: chown f2 21 1 2024-03-22T06:14:48.489 INFO:tasks.workunit.client.0.smithi006.stdout:7/87: dread f4 [0,4194304] 0 2024-03-22T06:14:48.497 INFO:tasks.workunit.client.0.smithi006.stdout:7/88: dread f0 [0,4194304] 0 2024-03-22T06:14:48.500 INFO:tasks.workunit.client.0.smithi006.stdout:7/89: read d5/fe [2793740,111093] 0 2024-03-22T06:14:48.502 INFO:tasks.workunit.client.0.smithi006.stdout:7/90: symlink d5/db/l14 0 2024-03-22T06:14:48.504 INFO:tasks.workunit.client.0.smithi006.stdout:7/91: mknod d5/db/d10/c15 0 2024-03-22T06:14:48.509 INFO:tasks.workunit.client.0.smithi006.stdout:7/92: write d5/db/f13 [683938,15681] 0 2024-03-22T06:14:48.509 INFO:tasks.workunit.client.0.smithi006.stdout:7/93: readlink d5/db/l14 0 2024-03-22T06:14:48.511 INFO:tasks.workunit.client.0.smithi006.stdout:7/94: mknod d5/c16 0 2024-03-22T06:14:48.566 INFO:tasks.workunit.client.0.smithi006.stdout:7/95: truncate d5/f9 32062 0 2024-03-22T06:14:48.638 INFO:tasks.workunit.client.0.smithi006.stdout:7/96: truncate f2 3499006 0 2024-03-22T06:14:48.641 INFO:tasks.workunit.client.0.smithi006.stdout:7/97: mknod d5/db/c17 0 2024-03-22T06:14:48.643 INFO:tasks.workunit.client.0.smithi006.stdout:7/98: getdents d5 0 2024-03-22T06:14:48.644 INFO:tasks.workunit.client.0.smithi006.stdout:7/99: mknod d5/db/c18 0 2024-03-22T06:14:48.713 INFO:tasks.workunit.client.0.smithi006.stdout:7/100: getdents d5/db/d10 0 2024-03-22T06:14:48.716 INFO:tasks.workunit.client.0.smithi006.stdout:7/101: symlink d5/l19 0 2024-03-22T06:14:48.716 INFO:tasks.workunit.client.0.smithi006.stdout:7/102: readlink l3 0 2024-03-22T06:14:48.953 INFO:tasks.workunit.client.0.smithi006.stdout:6/118: sync 2024-03-22T06:14:48.953 INFO:tasks.workunit.client.0.smithi006.stdout:6/119: write d6/fc [2006571,57741] 0 2024-03-22T06:14:48.953 INFO:tasks.workunit.client.0.smithi006.stdout:6/120: fsync d6/d1d/f26 0 2024-03-22T06:14:48.978 INFO:tasks.workunit.client.0.smithi006.stdout:6/121: dwrite d6/f1e [0,4194304] 0 2024-03-22T06:14:48.978 INFO:tasks.workunit.client.0.smithi006.stdout:6/122: chown d6/d1d/f2b 208940 1 2024-03-22T06:14:48.978 INFO:tasks.workunit.client.0.smithi006.stdout:6/123: creat d6/d1b/f30 x:0 0 0 2024-03-22T06:14:48.987 INFO:tasks.workunit.client.0.smithi006.stdout:6/124: dread d6/f13 [0,4194304] 0 2024-03-22T06:14:48.989 INFO:tasks.workunit.client.0.smithi006.stdout:6/125: creat d6/d24/f31 x:0 0 0 2024-03-22T06:14:49.003 INFO:tasks.workunit.client.0.smithi006.stdout:6/126: dwrite d6/d24/f31 [0,4194304] 0 2024-03-22T06:14:49.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:48 smithi117 ceph-mon[36514]: mgrmap e24: smithi117.bqeskk(active, since 9s), standbys: smithi006.sfpxdy 2024-03-22T06:14:49.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:48 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mgr metadata", "who": "smithi006.sfpxdy", "id": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:14:49.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:48 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:49.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:48 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:49.011 INFO:tasks.workunit.client.0.smithi006.stdout:6/127: dread d6/f1e [0,4194304] 0 2024-03-22T06:14:49.023 INFO:tasks.workunit.client.0.smithi006.stdout:6/128: mkdir d6/d1d/d32 0 2024-03-22T06:14:49.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:48 smithi006 ceph-mon[29301]: mgrmap e24: smithi117.bqeskk(active, since 9s), standbys: smithi006.sfpxdy 2024-03-22T06:14:49.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:48 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mgr metadata", "who": "smithi006.sfpxdy", "id": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:14:49.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:48 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:49.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:48 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:49.040 INFO:tasks.workunit.client.0.smithi006.stdout:6/129: dwrite d6/f11 [0,4194304] 0 2024-03-22T06:14:49.040 INFO:tasks.workunit.client.0.smithi006.stdout:6/130: rename d6 to d6/d24/d33 22 2024-03-22T06:14:49.046 INFO:tasks.workunit.client.0.smithi006.stdout:6/131: dread d6/f16 [0,4194304] 0 2024-03-22T06:14:49.144 INFO:tasks.workunit.client.0.smithi006.stdout:6/132: getdents d6/d1b 0 2024-03-22T06:14:49.144 INFO:tasks.workunit.client.0.smithi006.stdout:6/133: write d6/d1b/d2c/f2f [544408,49553] 0 2024-03-22T06:14:49.144 INFO:tasks.workunit.client.0.smithi006.stdout:6/134: chown d6/d1b/f30 2 1 2024-03-22T06:14:49.151 INFO:tasks.workunit.client.0.smithi006.stdout:6/135: dread d6/f11 [0,4194304] 0 2024-03-22T06:14:49.151 INFO:tasks.workunit.client.0.smithi006.stdout:6/136: fsync d6/f11 0 2024-03-22T06:14:49.151 INFO:tasks.workunit.client.0.smithi006.stdout:6/137: stat d6/d24/l25 0 2024-03-22T06:14:49.169 INFO:tasks.workunit.client.0.smithi006.stdout:6/138: dwrite d6/f18 [0,4194304] 0 2024-03-22T06:14:49.170 INFO:tasks.workunit.client.0.smithi006.stdout:1/18: sync 2024-03-22T06:14:49.170 INFO:tasks.workunit.client.0.smithi006.stdout:7/103: sync 2024-03-22T06:14:49.171 INFO:tasks.workunit.client.0.smithi006.stdout:4/89: sync 2024-03-22T06:14:49.171 INFO:tasks.workunit.client.0.smithi006.stdout:5/53: sync 2024-03-22T06:14:49.171 INFO:tasks.workunit.client.0.smithi006.stdout:4/90: readlink d6/l1a 0 2024-03-22T06:14:49.171 INFO:tasks.workunit.client.0.smithi006.stdout:9/57: sync 2024-03-22T06:14:49.171 INFO:tasks.workunit.client.0.smithi006.stdout:2/106: sync 2024-03-22T06:14:49.171 INFO:tasks.workunit.client.0.smithi006.stdout:8/45: sync 2024-03-22T06:14:49.171 INFO:tasks.workunit.client.0.smithi006.stdout:3/19: sync 2024-03-22T06:14:49.171 INFO:tasks.workunit.client.0.smithi006.stdout:0/50: sync 2024-03-22T06:14:49.171 INFO:tasks.workunit.client.0.smithi006.stdout:0/51: write f4 [2386282,87128] 0 2024-03-22T06:14:49.173 INFO:tasks.workunit.client.0.smithi006.stdout:6/139: link d6/d1b/f2d d6/d1b/d2c/f34 0 2024-03-22T06:14:49.174 INFO:tasks.workunit.client.0.smithi006.stdout:7/104: truncate d5/db/f13 4258180 0 2024-03-22T06:14:49.177 INFO:tasks.workunit.client.0.smithi006.stdout:0/52: dread f9 [0,4194304] 0 2024-03-22T06:14:49.178 INFO:tasks.workunit.client.0.smithi006.stdout:1/19: truncate f1 2331199 0 2024-03-22T06:14:49.179 INFO:tasks.workunit.client.0.smithi006.stdout:3/20: dread d0/f3 [4194304,4194304] 0 2024-03-22T06:14:49.180 INFO:tasks.workunit.client.0.smithi006.stdout:5/54: truncate d0/f4 2210112 0 2024-03-22T06:14:49.182 INFO:tasks.workunit.client.0.smithi006.stdout:2/107: mknod d13/d20/d26/d19/d28/d29/c2e 0 2024-03-22T06:14:49.184 INFO:tasks.workunit.client.0.smithi006.stdout:8/46: write d9/db/dc/fd [979637,10863] 0 2024-03-22T06:14:49.188 INFO:tasks.workunit.client.0.smithi006.stdout:6/140: mknod d6/c35 0 2024-03-22T06:14:49.190 INFO:tasks.workunit.client.0.smithi006.stdout:7/105: link d5/db/l14 d5/db/l1a 0 2024-03-22T06:14:49.191 INFO:tasks.workunit.client.0.smithi006.stdout:9/58: dwrite d8/fd [0,4194304] 0 2024-03-22T06:14:49.192 INFO:tasks.workunit.client.0.smithi006.stdout:6/141: dread d6/d1b/d2c/f34 [0,4194304] 0 2024-03-22T06:14:49.195 INFO:tasks.workunit.client.0.smithi006.stdout:0/53: creat db/ff x:0 0 0 2024-03-22T06:14:49.196 INFO:tasks.workunit.client.0.smithi006.stdout:0/54: stat c8 0 2024-03-22T06:14:49.196 INFO:tasks.workunit.client.0.smithi006.stdout:0/55: readlink l1 0 2024-03-22T06:14:49.199 INFO:tasks.workunit.client.0.smithi006.stdout:1/20: link l0 l5 0 2024-03-22T06:14:49.200 INFO:tasks.workunit.client.0.smithi006.stdout:3/21: rmdir d0/d4 39 2024-03-22T06:14:49.203 INFO:tasks.workunit.client.0.smithi006.stdout:5/55: mkdir d0/db/dc 0 2024-03-22T06:14:49.203 INFO:tasks.workunit.client.0.smithi006.stdout:5/56: fsync d0/f5 0 2024-03-22T06:14:49.203 INFO:tasks.workunit.client.0.smithi006.stdout:5/57: chown d0/l8 25421 1 2024-03-22T06:14:49.205 INFO:tasks.workunit.client.0.smithi006.stdout:2/108: creat d13/d20/d26/f2f x:0 0 0 2024-03-22T06:14:49.205 INFO:tasks.workunit.client.0.smithi006.stdout:2/109: read - f7 zero size 2024-03-22T06:14:49.207 INFO:tasks.workunit.client.0.smithi006.stdout:8/47: mkdir d9/d14 0 2024-03-22T06:14:49.212 INFO:tasks.workunit.client.0.smithi006.stdout:7/106: dwrite f0 [0,4194304] 0 2024-03-22T06:14:49.212 INFO:tasks.workunit.client.0.smithi006.stdout:9/59: mkdir d8/d10/d11 0 2024-03-22T06:14:49.215 INFO:tasks.workunit.client.0.smithi006.stdout:2/110: write f3 [3284644,109483] 0 2024-03-22T06:14:49.215 INFO:tasks.workunit.client.0.smithi006.stdout:6/142: rename d6/d1b/l29 to d6/d1b/d2c/l36 0 2024-03-22T06:14:49.217 INFO:tasks.workunit.client.0.smithi006.stdout:0/56: mkdir db/d10 0 2024-03-22T06:14:49.219 INFO:tasks.workunit.client.0.smithi006.stdout:1/21: truncate f3 2682762 0 2024-03-22T06:14:49.220 INFO:tasks.workunit.client.0.smithi006.stdout:9/60: dread f4 [0,4194304] 0 2024-03-22T06:14:49.221 INFO:tasks.workunit.client.0.smithi006.stdout:3/22: creat d0/d4/f7 x:0 0 0 2024-03-22T06:14:49.222 INFO:tasks.workunit.client.0.smithi006.stdout:3/23: read - d0/d4/f7 zero size 2024-03-22T06:14:49.223 INFO:tasks.workunit.client.0.smithi006.stdout:8/48: dwrite d9/fa [0,4194304] 0 2024-03-22T06:14:49.223 INFO:tasks.workunit.client.0.smithi006.stdout:5/58: creat d0/db/fd x:0 0 0 2024-03-22T06:14:49.224 INFO:tasks.workunit.client.0.smithi006.stdout:5/59: creat d0/fe x:0 0 0 2024-03-22T06:14:49.225 INFO:tasks.workunit.client.0.smithi006.stdout:7/107: rmdir d5/db 39 2024-03-22T06:14:49.228 INFO:tasks.workunit.client.0.smithi006.stdout:2/111: creat d13/d20/d26/d19/f30 x:0 0 0 2024-03-22T06:14:49.233 INFO:tasks.workunit.client.0.smithi006.stdout:0/57: mknod db/d10/c11 0 2024-03-22T06:14:49.239 INFO:tasks.workunit.client.0.smithi006.stdout:8/49: dwrite d9/f11 [0,4194304] 0 2024-03-22T06:14:49.240 INFO:tasks.workunit.client.0.smithi006.stdout:8/50: chown d9/db/d12 86649 1 2024-03-22T06:14:49.240 INFO:tasks.workunit.client.0.smithi006.stdout:8/51: chown f5 12171 1 2024-03-22T06:14:49.240 INFO:tasks.workunit.client.0.smithi006.stdout:8/52: getdents d9/db/d12 0 2024-03-22T06:14:49.240 INFO:tasks.workunit.client.0.smithi006.stdout:9/61: creat d8/d10/f12 x:0 0 0 2024-03-22T06:14:49.240 INFO:tasks.workunit.client.0.smithi006.stdout:9/62: fdatasync f6 0 2024-03-22T06:14:49.240 INFO:tasks.workunit.client.0.smithi006.stdout:9/63: getdents d8/d10/d11 0 2024-03-22T06:14:49.240 INFO:tasks.workunit.client.0.smithi006.stdout:8/53: fdatasync d9/db/dc/fd 0 2024-03-22T06:14:49.241 INFO:tasks.workunit.client.0.smithi006.stdout:3/24: mkdir d0/d8 0 2024-03-22T06:14:49.247 INFO:tasks.workunit.client.0.smithi006.stdout:7/108: symlink d5/db/d10/l1b 0 2024-03-22T06:14:49.250 INFO:tasks.workunit.client.0.smithi006.stdout:2/112: symlink d13/d20/d26/l31 0 2024-03-22T06:14:49.252 INFO:tasks.workunit.client.0.smithi006.stdout:2/113: write d13/d20/d26/f2f [150682,73949] 0 2024-03-22T06:14:49.252 INFO:tasks.workunit.client.0.smithi006.stdout:2/114: write d13/d20/d26/f15 [726684,76196] 0 2024-03-22T06:14:49.252 INFO:tasks.workunit.client.0.smithi006.stdout:0/58: symlink db/l12 0 2024-03-22T06:14:49.257 INFO:tasks.workunit.client.0.smithi006.stdout:9/64: creat d8/d10/d11/f13 x:0 0 0 2024-03-22T06:14:49.257 INFO:tasks.workunit.client.0.smithi006.stdout:0/59: dread f4 [0,4194304] 0 2024-03-22T06:14:49.258 INFO:tasks.workunit.client.0.smithi006.stdout:8/54: symlink d9/db/dc/de/l15 0 2024-03-22T06:14:49.259 INFO:tasks.workunit.client.0.smithi006.stdout:8/55: fsync d9/db/dc/de/ff 0 2024-03-22T06:14:49.264 INFO:tasks.workunit.client.0.smithi006.stdout:7/109: symlink d5/db/d10/l1c 0 2024-03-22T06:14:49.266 INFO:tasks.workunit.client.0.smithi006.stdout:2/115: link fe d13/d20/d26/d19/d28/f32 0 2024-03-22T06:14:49.266 INFO:tasks.workunit.client.0.smithi006.stdout:2/116: write f7 [136803,58034] 0 2024-03-22T06:14:49.266 INFO:tasks.workunit.client.0.smithi006.stdout:2/117: truncate d13/f23 463043 0 2024-03-22T06:14:49.270 INFO:tasks.workunit.client.0.smithi006.stdout:2/118: write d13/f21 [191650,47449] 0 2024-03-22T06:14:49.270 INFO:tasks.workunit.client.0.smithi006.stdout:2/119: read d13/d20/d26/d19/f1c [913854,73293] 0 2024-03-22T06:14:49.270 INFO:tasks.workunit.client.0.smithi006.stdout:9/65: symlink d8/l14 0 2024-03-22T06:14:49.271 INFO:tasks.workunit.client.0.smithi006.stdout:8/56: rename d9/db/dc/de/ff to d9/db/dc/de/f16 0 2024-03-22T06:14:49.275 INFO:tasks.workunit.client.0.smithi006.stdout:9/66: dread f4 [0,4194304] 0 2024-03-22T06:14:49.279 INFO:tasks.workunit.client.0.smithi006.stdout:8/57: rename d9/f13 to d9/db/dc/de/f17 0 2024-03-22T06:14:49.279 INFO:tasks.workunit.client.0.smithi006.stdout:8/58: creat d9/f18 x:0 0 0 2024-03-22T06:14:49.282 INFO:tasks.workunit.client.0.smithi006.stdout:9/67: rename d8/l14 to d8/d10/l15 0 2024-03-22T06:14:49.282 INFO:tasks.workunit.client.0.smithi006.stdout:9/68: chown d8/d10/d11 31 1 2024-03-22T06:14:49.289 INFO:tasks.workunit.client.0.smithi006.stdout:7/110: dwrite d5/fd [0,4194304] 0 2024-03-22T06:14:49.290 INFO:tasks.workunit.client.0.smithi006.stdout:7/111: write d5/fe [3442722,44774] 0 2024-03-22T06:14:49.309 INFO:tasks.workunit.client.0.smithi006.stdout:8/59: dwrite d9/db/dc/fd [0,4194304] 0 2024-03-22T06:14:49.311 INFO:tasks.workunit.client.0.smithi006.stdout:8/60: mknod d9/c19 0 2024-03-22T06:14:49.316 INFO:tasks.workunit.client.0.smithi006.stdout:9/69: dwrite d8/fb [0,4194304] 0 2024-03-22T06:14:49.317 INFO:tasks.workunit.client.0.smithi006.stdout:2/120: dwrite d13/f21 [0,4194304] 0 2024-03-22T06:14:49.317 INFO:tasks.workunit.client.0.smithi006.stdout:2/121: read f9 [991811,32343] 0 2024-03-22T06:14:49.327 INFO:tasks.workunit.client.0.smithi006.stdout:2/122: rename d13/d20/d26/l1e to d13/l33 0 2024-03-22T06:14:49.329 INFO:tasks.workunit.client.0.smithi006.stdout:2/123: mkdir d13/d34 0 2024-03-22T06:14:49.330 INFO:tasks.workunit.client.0.smithi006.stdout:2/124: stat d13/l17 0 2024-03-22T06:14:49.330 INFO:tasks.workunit.client.0.smithi006.stdout:2/125: chown d13/c1a 14411 1 2024-03-22T06:14:49.332 INFO:tasks.workunit.client.0.smithi006.stdout:2/126: mkdir d13/d20/d35 0 2024-03-22T06:14:49.332 INFO:tasks.workunit.client.0.smithi006.stdout:2/127: write d13/d20/d26/f2f [331947,80060] 0 2024-03-22T06:14:49.332 INFO:tasks.workunit.client.0.smithi006.stdout:2/128: fsync fb 0 2024-03-22T06:14:49.334 INFO:tasks.workunit.client.0.smithi006.stdout:9/70: dwrite d8/d10/f12 [0,4194304] 0 2024-03-22T06:14:49.334 INFO:tasks.workunit.client.0.smithi006.stdout:8/61: dwrite f2 [0,4194304] 0 2024-03-22T06:14:49.334 INFO:tasks.workunit.client.0.smithi006.stdout:9/71: creat d8/d10/d11/f16 x:0 0 0 2024-03-22T06:14:49.334 INFO:tasks.workunit.client.0.smithi006.stdout:9/72: creat d8/d10/f17 x:0 0 0 2024-03-22T06:14:49.335 INFO:tasks.workunit.client.0.smithi006.stdout:2/129: rename l5 to d13/d34/l36 0 2024-03-22T06:14:49.340 INFO:tasks.workunit.client.0.smithi006.stdout:8/62: creat d9/db/f1a x:0 0 0 2024-03-22T06:14:49.347 INFO:tasks.workunit.client.0.smithi006.stdout:9/73: symlink d8/d10/d11/l18 0 2024-03-22T06:14:49.350 INFO:tasks.workunit.client.0.smithi006.stdout:2/130: symlink d13/d20/d26/d2a/l37 0 2024-03-22T06:14:49.352 INFO:tasks.workunit.client.0.smithi006.stdout:9/74: link d8/fb d8/f19 0 2024-03-22T06:14:49.354 INFO:tasks.workunit.client.0.smithi006.stdout:8/63: dwrite f7 [4194304,4194304] 0 2024-03-22T06:14:49.354 INFO:tasks.workunit.client.0.smithi006.stdout:2/131: creat d13/d20/d26/d19/d28/f38 x:0 0 0 2024-03-22T06:14:49.354 INFO:tasks.workunit.client.0.smithi006.stdout:2/132: fdatasync f6 0 2024-03-22T06:14:49.356 INFO:tasks.workunit.client.0.smithi006.stdout:9/75: mknod d8/d10/c1a 0 2024-03-22T06:14:49.360 INFO:tasks.workunit.client.0.smithi006.stdout:8/64: link l0 d9/db/dc/l1b 0 2024-03-22T06:14:49.362 INFO:tasks.workunit.client.0.smithi006.stdout:9/76: mkdir d8/d10/d1b 0 2024-03-22T06:14:49.362 INFO:tasks.workunit.client.0.smithi006.stdout:9/77: creat d8/d10/d11/f1c x:0 0 0 2024-03-22T06:14:49.362 INFO:tasks.workunit.client.0.smithi006.stdout:9/78: truncate d8/d10/d11/f1c 667242 0 2024-03-22T06:14:49.371 INFO:tasks.workunit.client.0.smithi006.stdout:1/22: unlink f1 0 2024-03-22T06:14:49.381 INFO:tasks.workunit.client.0.smithi006.stdout:2/133: dwrite fc [0,4194304] 0 2024-03-22T06:14:49.382 INFO:tasks.workunit.client.0.smithi006.stdout:2/134: creat d13/d20/d26/d19/f39 x:0 0 0 2024-03-22T06:14:49.382 INFO:tasks.workunit.client.0.smithi006.stdout:2/135: stat f7 0 2024-03-22T06:14:49.382 INFO:tasks.workunit.client.0.smithi006.stdout:2/136: dread - d13/d20/d26/d19/d28/f38 zero size 2024-03-22T06:14:49.382 INFO:tasks.workunit.client.0.smithi006.stdout:8/65: dwrite f2 [0,4194304] 0 2024-03-22T06:14:49.387 INFO:tasks.workunit.client.0.smithi006.stdout:2/137: stat d13/l33 0 2024-03-22T06:14:49.390 INFO:tasks.workunit.client.0.smithi006.stdout:2/138: symlink d13/l3a 0 2024-03-22T06:14:49.393 INFO:tasks.workunit.client.0.smithi006.stdout:8/66: write d9/fa [1732618,80535] 0 2024-03-22T06:14:49.393 INFO:tasks.workunit.client.0.smithi006.stdout:2/139: mknod d13/d20/d26/d2a/c3b 0 2024-03-22T06:14:49.394 INFO:tasks.workunit.client.0.smithi006.stdout:2/140: write fe [425018,70209] 0 2024-03-22T06:14:49.394 INFO:tasks.workunit.client.0.smithi006.stdout:2/141: readlink d13/d20/d26/d2a/l2c 0 2024-03-22T06:14:49.397 INFO:tasks.workunit.client.0.smithi006.stdout:2/142: symlink d13/d20/d35/l3c 0 2024-03-22T06:14:49.397 INFO:tasks.workunit.client.0.smithi006.stdout:2/143: write fb [4868479,68780] 0 2024-03-22T06:14:49.399 INFO:tasks.workunit.client.0.smithi006.stdout:8/67: dread f7 [0,4194304] 0 2024-03-22T06:14:49.401 INFO:tasks.workunit.client.0.smithi006.stdout:2/144: rename d13/d20/d26/d19/f30 to d13/d20/f3d 0 2024-03-22T06:14:49.404 INFO:tasks.workunit.client.0.smithi006.stdout:8/68: creat d9/d14/f1c x:0 0 0 2024-03-22T06:14:49.404 INFO:tasks.workunit.client.0.smithi006.stdout:8/69: rename d9/d14 to d9/d14/d1d 22 2024-03-22T06:14:49.406 INFO:tasks.workunit.client.0.smithi006.stdout:6/143: getdents d6/d1b/d2c 0 2024-03-22T06:14:49.410 INFO:tasks.workunit.client.0.smithi006.stdout:6/144: dread d6/f11 [0,4194304] 0 2024-03-22T06:14:49.411 INFO:tasks.workunit.client.0.smithi006.stdout:8/70: mkdir d9/db/d12/d1e 0 2024-03-22T06:14:49.411 INFO:tasks.workunit.client.0.smithi006.stdout:1/23: write f3 [2991461,39916] 0 2024-03-22T06:14:49.414 INFO:tasks.workunit.client.0.smithi006.stdout:6/145: link d6/c8 d6/d1b/c37 0 2024-03-22T06:14:49.414 INFO:tasks.workunit.client.0.smithi006.stdout:3/25: rmdir d0/d4 39 2024-03-22T06:14:49.415 INFO:tasks.workunit.client.0.smithi006.stdout:8/71: truncate d9/db/f10 181799 0 2024-03-22T06:14:49.419 INFO:tasks.workunit.client.0.smithi006.stdout:6/146: unlink d6/d1b/d2c/f34 0 2024-03-22T06:14:49.419 INFO:tasks.workunit.client.0.smithi006.stdout:6/147: fdatasync d6/d1d/f26 0 2024-03-22T06:14:49.419 INFO:tasks.workunit.client.0.smithi006.stdout:6/148: fdatasync d6/d1d/f2b 0 2024-03-22T06:14:49.420 INFO:tasks.workunit.client.0.smithi006.stdout:3/26: mkdir d0/d8/d9 0 2024-03-22T06:14:49.421 INFO:tasks.workunit.client.0.smithi006.stdout:3/27: creat d0/fa x:0 0 0 2024-03-22T06:14:49.421 INFO:tasks.workunit.client.0.smithi006.stdout:3/28: truncate d0/d4/f7 958263 0 2024-03-22T06:14:49.421 INFO:tasks.workunit.client.0.smithi006.stdout:3/29: stat d0/d4 0 2024-03-22T06:14:49.421 INFO:tasks.workunit.client.0.smithi006.stdout:3/30: dread - d0/fa zero size 2024-03-22T06:14:49.421 INFO:tasks.workunit.client.0.smithi006.stdout:3/31: stat d0/d8 0 2024-03-22T06:14:49.423 INFO:tasks.workunit.client.0.smithi006.stdout:8/72: rename d9/db/f10 to d9/db/dc/f1f 0 2024-03-22T06:14:49.425 INFO:tasks.workunit.client.0.smithi006.stdout:8/73: dread f1 [0,4194304] 0 2024-03-22T06:14:49.426 INFO:tasks.workunit.client.0.smithi006.stdout:3/32: creat d0/d4/fb x:0 0 0 2024-03-22T06:14:49.428 INFO:tasks.workunit.client.0.smithi006.stdout:1/24: dwrite f3 [0,4194304] 0 2024-03-22T06:14:49.428 INFO:tasks.workunit.client.0.smithi006.stdout:6/149: rename d6/d1b/d2c/f2f to d6/d24/f38 0 2024-03-22T06:14:49.430 INFO:tasks.workunit.client.0.smithi006.stdout:8/74: creat d9/db/f20 x:0 0 0 2024-03-22T06:14:49.430 INFO:tasks.workunit.client.0.smithi006.stdout:8/75: read - d9/f18 zero size 2024-03-22T06:14:49.431 INFO:tasks.workunit.client.0.smithi006.stdout:8/76: dread f1 [0,4194304] 0 2024-03-22T06:14:49.431 INFO:tasks.workunit.client.0.smithi006.stdout:8/77: creat d9/f21 x:0 0 0 2024-03-22T06:14:49.431 INFO:tasks.workunit.client.0.smithi006.stdout:8/78: read - d9/db/f1a zero size 2024-03-22T06:14:49.432 INFO:tasks.workunit.client.0.smithi006.stdout:3/33: unlink d0/l6 0 2024-03-22T06:14:49.433 INFO:tasks.workunit.client.0.smithi006.stdout:5/60: fsync d0/fe 0 2024-03-22T06:14:49.433 INFO:tasks.workunit.client.0.smithi006.stdout:5/61: creat d0/ff x:0 0 0 2024-03-22T06:14:49.433 INFO:tasks.workunit.client.0.smithi006.stdout:1/25: mknod c6 0 2024-03-22T06:14:49.433 INFO:tasks.workunit.client.0.smithi006.stdout:5/62: read d0/f1 [1125187,122767] 0 2024-03-22T06:14:49.436 INFO:tasks.workunit.client.0.smithi006.stdout:6/150: mknod d6/d1b/c39 0 2024-03-22T06:14:49.436 INFO:tasks.workunit.client.0.smithi006.stdout:6/151: stat d6/d1b/l22 0 2024-03-22T06:14:49.436 INFO:tasks.workunit.client.0.smithi006.stdout:6/152: chown d6/c8 20189891 1 2024-03-22T06:14:49.436 INFO:tasks.workunit.client.0.smithi006.stdout:6/153: truncate d6/d1d/f26 281385 0 2024-03-22T06:14:49.436 INFO:tasks.workunit.client.0.smithi006.stdout:6/154: readlink d6/d1b/d2c/l36 0 2024-03-22T06:14:49.439 INFO:tasks.workunit.client.0.smithi006.stdout:6/155: write d6/fc [2452485,25296] 0 2024-03-22T06:14:49.439 INFO:tasks.workunit.client.0.smithi006.stdout:8/79: truncate f5 210993 0 2024-03-22T06:14:49.440 INFO:tasks.workunit.client.0.smithi006.stdout:8/80: dread - d9/db/dc/de/f17 zero size 2024-03-22T06:14:49.441 INFO:tasks.workunit.client.0.smithi006.stdout:3/34: link d0/d4/c5 d0/cc 0 2024-03-22T06:14:49.445 INFO:tasks.workunit.client.0.smithi006.stdout:8/81: dread d9/fa [0,4194304] 0 2024-03-22T06:14:49.446 INFO:tasks.workunit.client.0.smithi006.stdout:1/26: link l5 l7 0 2024-03-22T06:14:49.452 INFO:tasks.workunit.client.0.smithi006.stdout:6/156: mknod d6/d24/c3a 0 2024-03-22T06:14:49.454 INFO:tasks.workunit.client.0.smithi006.stdout:5/63: dwrite d0/fe [0,4194304] 0 2024-03-22T06:14:49.457 INFO:tasks.workunit.client.0.smithi006.stdout:3/35: unlink d0/d4/c5 0 2024-03-22T06:14:49.468 INFO:tasks.workunit.client.0.smithi006.stdout:5/64: rename d0/f1 to d0/db/dc/f10 0 2024-03-22T06:14:49.468 INFO:tasks.workunit.client.0.smithi006.stdout:5/65: chown d0/ff 226974082 1 2024-03-22T06:14:49.476 INFO:tasks.workunit.client.0.smithi006.stdout:1/27: dwrite f3 [0,4194304] 0 2024-03-22T06:14:49.477 INFO:tasks.workunit.client.0.smithi006.stdout:8/82: dwrite d9/db/f1a [0,4194304] 0 2024-03-22T06:14:49.481 INFO:tasks.workunit.client.0.smithi006.stdout:1/28: unlink l5 0 2024-03-22T06:14:49.481 INFO:tasks.workunit.client.0.smithi006.stdout:1/29: fdatasync f3 0 2024-03-22T06:14:49.481 INFO:tasks.workunit.client.0.smithi006.stdout:3/36: dwrite d0/f3 [4194304,4194304] 0 2024-03-22T06:14:49.506 INFO:tasks.workunit.client.0.smithi006.stdout:8/83: dwrite d9/db/dc/de/f17 [0,4194304] 0 2024-03-22T06:14:49.507 INFO:tasks.workunit.client.0.smithi006.stdout:8/84: getdents d9/db/d12/d1e 0 2024-03-22T06:14:49.507 INFO:tasks.workunit.client.0.smithi006.stdout:3/37: dwrite d0/f3 [0,4194304] 0 2024-03-22T06:14:49.515 INFO:tasks.workunit.client.0.smithi006.stdout:8/85: rename d9/f18 to d9/db/dc/de/f22 0 2024-03-22T06:14:49.515 INFO:tasks.workunit.client.0.smithi006.stdout:8/86: dread - d9/db/f20 zero size 2024-03-22T06:14:49.518 INFO:tasks.workunit.client.0.smithi006.stdout:3/38: dwrite d0/d4/fb [0,4194304] 0 2024-03-22T06:14:49.526 INFO:tasks.workunit.client.0.smithi006.stdout:3/39: unlink d0/d4/f7 0 2024-03-22T06:14:49.526 INFO:tasks.workunit.client.0.smithi006.stdout:3/40: fdatasync d0/f3 0 2024-03-22T06:14:49.537 INFO:tasks.workunit.client.0.smithi006.stdout:9/79: fsync d8/d10/d11/f1c 0 2024-03-22T06:14:49.543 INFO:tasks.workunit.client.0.smithi006.stdout:3/41: mknod d0/cd 0 2024-03-22T06:14:49.545 INFO:tasks.workunit.client.0.smithi006.stdout:9/80: link d8/fb d8/d10/d1b/f1d 0 2024-03-22T06:14:49.546 INFO:tasks.workunit.client.0.smithi006.stdout:3/42: symlink d0/d8/le 0 2024-03-22T06:14:49.551 INFO:tasks.workunit.client.0.smithi006.stdout:9/81: symlink d8/d10/d1b/l1e 0 2024-03-22T06:14:49.554 INFO:tasks.workunit.client.0.smithi006.stdout:9/82: mkdir d8/d1f 0 2024-03-22T06:14:49.561 INFO:tasks.workunit.client.0.smithi006.stdout:3/43: dwrite d0/d4/fb [0,4194304] 0 2024-03-22T06:14:49.567 INFO:tasks.workunit.client.0.smithi006.stdout:3/44: mknod d0/d8/cf 0 2024-03-22T06:14:49.570 INFO:tasks.workunit.client.0.smithi006.stdout:3/45: mkdir d0/d4/d10 0 2024-03-22T06:14:49.570 INFO:tasks.workunit.client.0.smithi006.stdout:3/46: truncate d0/fa 328360 0 2024-03-22T06:14:49.571 INFO:tasks.workunit.client.0.smithi006.stdout:9/83: dwrite d8/ff [0,4194304] 0 2024-03-22T06:14:49.579 INFO:tasks.workunit.client.0.smithi006.stdout:9/84: mkdir d8/d10/d20 0 2024-03-22T06:14:49.584 INFO:tasks.workunit.client.0.smithi006.stdout:9/85: link f6 d8/f21 0 2024-03-22T06:14:49.584 INFO:tasks.workunit.client.0.smithi006.stdout:9/86: write d8/d10/d11/f16 [303512,97637] 0 2024-03-22T06:14:49.599 INFO:tasks.workunit.client.0.smithi006.stdout:6/157: fsync d6/d1b/f2d 0 2024-03-22T06:14:49.616 INFO:tasks.workunit.client.0.smithi006.stdout:6/158: mknod d6/d1d/d32/c3b 0 2024-03-22T06:14:49.618 INFO:tasks.workunit.client.0.smithi006.stdout:9/87: dwrite d8/fb [0,4194304] 0 2024-03-22T06:14:49.619 INFO:tasks.workunit.client.0.smithi006.stdout:6/159: creat d6/d1b/f3c x:0 0 0 2024-03-22T06:14:49.622 INFO:tasks.workunit.client.0.smithi006.stdout:6/160: mknod d6/c3d 0 2024-03-22T06:14:49.622 INFO:tasks.workunit.client.0.smithi006.stdout:6/161: chown d6/d1d/f20 41 1 2024-03-22T06:14:49.626 INFO:tasks.workunit.client.0.smithi006.stdout:9/88: dread d8/d10/f12 [0,4194304] 0 2024-03-22T06:14:49.626 INFO:tasks.workunit.client.0.smithi006.stdout:9/89: creat d8/d10/f22 x:0 0 0 2024-03-22T06:14:49.628 INFO:tasks.workunit.client.0.smithi006.stdout:6/162: link d6/d1d/l2e d6/d1b/l3e 0 2024-03-22T06:14:49.628 INFO:tasks.workunit.client.0.smithi006.stdout:6/163: write d6/f1e [5153933,40286] 0 2024-03-22T06:14:49.628 INFO:tasks.workunit.client.0.smithi006.stdout:6/164: write d6/d1d/f20 [4991976,45261] 0 2024-03-22T06:14:49.628 INFO:tasks.workunit.client.0.smithi006.stdout:6/165: truncate d6/d1d/f26 643476 0 2024-03-22T06:14:49.636 INFO:tasks.workunit.client.0.smithi006.stdout:9/90: mkdir d8/d10/d23 0 2024-03-22T06:14:49.636 INFO:tasks.workunit.client.0.smithi006.stdout:9/91: fsync f3 0 2024-03-22T06:14:49.637 INFO:tasks.workunit.client.0.smithi006.stdout:6/166: truncate d6/d1b/f23 2634613 0 2024-03-22T06:14:49.641 INFO:tasks.workunit.client.0.smithi006.stdout:9/92: truncate d8/d10/f12 2767177 0 2024-03-22T06:14:49.652 INFO:tasks.workunit.client.0.smithi006.stdout:9/93: symlink d8/d10/l24 0 2024-03-22T06:14:49.657 INFO:tasks.workunit.client.0.smithi006.stdout:9/94: write f4 [2939884,10181] 0 2024-03-22T06:14:49.657 INFO:tasks.workunit.client.0.smithi006.stdout:9/95: write f1 [738172,114680] 0 2024-03-22T06:14:49.658 INFO:tasks.workunit.client.0.smithi006.stdout:9/96: creat d8/d10/d1b/f25 x:0 0 0 2024-03-22T06:14:49.667 INFO:tasks.workunit.client.0.smithi006.stdout:9/97: dread d8/f21 [0,4194304] 0 2024-03-22T06:14:49.696 INFO:tasks.workunit.client.0.smithi006.stdout:9/98: symlink d8/d10/d1b/l26 0 2024-03-22T06:14:49.696 INFO:tasks.workunit.client.0.smithi006.stdout:9/99: write f3 [492473,96080] 0 2024-03-22T06:14:49.696 INFO:tasks.workunit.client.0.smithi006.stdout:9/100: read f4 [2886770,70679] 0 2024-03-22T06:14:49.720 INFO:tasks.workunit.client.0.smithi006.stdout:9/101: creat d8/d10/d23/f27 x:0 0 0 2024-03-22T06:14:49.735 INFO:tasks.workunit.client.0.smithi006.stdout:7/112: truncate d5/db/f13 674673 0 2024-03-22T06:14:49.740 INFO:tasks.workunit.client.0.smithi006.stdout:7/113: creat d5/f1d x:0 0 0 2024-03-22T06:14:49.745 INFO:tasks.workunit.client.0.smithi006.stdout:7/114: mkdir d5/d1e 0 2024-03-22T06:14:49.745 INFO:tasks.workunit.client.0.smithi006.stdout:7/115: getdents d5/d1e 0 2024-03-22T06:14:49.755 INFO:tasks.workunit.client.0.smithi006.stdout:8/87: truncate f2 1128074 0 2024-03-22T06:14:49.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:49 smithi117 ceph-mon[36514]: pgmap v8: 65 pgs: 65 active+clean; 214 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 9.5 MiB/s rd, 19 MiB/s wr, 432 op/s 2024-03-22T06:14:49.760 INFO:tasks.workunit.client.0.smithi006.stdout:8/88: mkdir d9/d14/d23 0 2024-03-22T06:14:49.760 INFO:tasks.workunit.client.0.smithi006.stdout:8/89: stat d9 0 2024-03-22T06:14:49.768 INFO:tasks.workunit.client.0.smithi006.stdout:7/116: dwrite d5/f1d [0,4194304] 0 2024-03-22T06:14:49.768 INFO:tasks.workunit.client.0.smithi006.stdout:7/117: readlink d5/db/l1a 0 2024-03-22T06:14:49.773 INFO:tasks.workunit.client.0.smithi006.stdout:7/118: mkdir d5/db/d1f 0 2024-03-22T06:14:49.774 INFO:tasks.workunit.client.0.smithi006.stdout:8/90: dwrite d9/db/dc/de/f22 [0,4194304] 0 2024-03-22T06:14:49.782 INFO:tasks.workunit.client.0.smithi006.stdout:7/119: write f0 [77843,44940] 0 2024-03-22T06:14:49.809 INFO:tasks.workunit.client.0.smithi006.stdout:3/47: truncate d0/d4/fb 3498687 0 2024-03-22T06:14:49.811 INFO:tasks.workunit.client.0.smithi006.stdout:3/48: chown d0/fa 196 1 2024-03-22T06:14:49.821 INFO:tasks.workunit.client.0.smithi006.stdout:3/49: dread d0/f3 [4194304,4194304] 0 2024-03-22T06:14:49.841 INFO:tasks.workunit.client.0.smithi006.stdout:3/50: dwrite d0/fa [0,4194304] 0 2024-03-22T06:14:49.849 INFO:tasks.workunit.client.0.smithi006.stdout:3/51: write d0/fa [687970,47001] 0 2024-03-22T06:14:49.874 INFO:tasks.workunit.client.0.smithi006.stdout:9/102: dwrite d8/d10/f12 [0,4194304] 0 2024-03-22T06:14:49.906 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:49 smithi006 ceph-mon[29301]: pgmap v8: 65 pgs: 65 active+clean; 214 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 9.5 MiB/s rd, 19 MiB/s wr, 432 op/s 2024-03-22T06:14:49.915 INFO:tasks.workunit.client.0.smithi006.stdout:9/103: rename d8/d10/d1b to d8/d10/d28 0 2024-03-22T06:14:49.919 INFO:tasks.workunit.client.0.smithi006.stdout:9/104: mknod d8/d10/d28/c29 0 2024-03-22T06:14:49.919 INFO:tasks.workunit.client.0.smithi006.stdout:9/105: write f1 [1794857,49471] 0 2024-03-22T06:14:49.920 INFO:tasks.workunit.client.0.smithi006.stdout:9/106: write d8/d10/d23/f27 [987242,70472] 0 2024-03-22T06:14:49.922 INFO:tasks.workunit.client.0.smithi006.stdout:9/107: mknod d8/d10/d28/c2a 0 2024-03-22T06:14:49.924 INFO:tasks.workunit.client.0.smithi006.stdout:9/108: mknod d8/d10/d28/c2b 0 2024-03-22T06:14:49.991 INFO:tasks.workunit.client.0.smithi006.stdout:8/91: write f5 [401462,27028] 0 2024-03-22T06:14:50.046 INFO:tasks.workunit.client.0.smithi006.stdout:3/52: getdents d0 0 2024-03-22T06:14:50.055 INFO:tasks.workunit.client.0.smithi006.stdout:3/53: dread d0/d4/fb [0,4194304] 0 2024-03-22T06:14:50.093 INFO:tasks.workunit.client.0.smithi006.stdout:3/54: truncate d0/fa 1581581 0 2024-03-22T06:14:50.096 INFO:tasks.workunit.client.0.smithi006.stdout:3/55: symlink d0/d8/d9/l11 0 2024-03-22T06:14:50.099 INFO:tasks.workunit.client.0.smithi006.stdout:3/56: creat d0/f12 x:0 0 0 2024-03-22T06:14:50.111 INFO:tasks.workunit.client.0.smithi006.stdout:4/91: sync 2024-03-22T06:14:50.111 INFO:tasks.workunit.client.0.smithi006.stdout:4/92: stat d6/lc 0 2024-03-22T06:14:50.126 INFO:tasks.workunit.client.0.smithi006.stdout:4/93: dwrite d6/f7 [0,4194304] 0 2024-03-22T06:14:50.127 INFO:tasks.workunit.client.0.smithi006.stdout:4/94: chown f1 58 1 2024-03-22T06:14:50.127 INFO:tasks.workunit.client.0.smithi006.stdout:4/95: fsync f2 0 2024-03-22T06:14:50.127 INFO:tasks.workunit.client.0.smithi006.stdout:4/96: fsync f0 0 2024-03-22T06:14:50.130 INFO:tasks.workunit.client.0.smithi006.stdout:4/97: symlink d6/d1b/l1d 0 2024-03-22T06:14:50.132 INFO:tasks.workunit.client.0.smithi006.stdout:4/98: mknod d6/d13/c1e 0 2024-03-22T06:14:50.133 INFO:tasks.workunit.client.0.smithi006.stdout:4/99: creat d6/f1f x:0 0 0 2024-03-22T06:14:50.136 INFO:tasks.workunit.client.0.smithi006.stdout:4/100: mkdir d6/d1b/d20 0 2024-03-22T06:14:50.140 INFO:tasks.workunit.client.0.smithi006.stdout:4/101: mkdir d6/d1b/d20/d21 0 2024-03-22T06:14:50.143 INFO:tasks.workunit.client.0.smithi006.stdout:4/102: creat d6/d1b/d20/f22 x:0 0 0 2024-03-22T06:14:50.143 INFO:tasks.workunit.client.0.smithi006.stdout:4/103: stat d6/f1f 0 2024-03-22T06:14:50.146 INFO:tasks.workunit.client.0.smithi006.stdout:4/104: mknod d6/c23 0 2024-03-22T06:14:50.146 INFO:tasks.workunit.client.0.smithi006.stdout:4/105: creat d6/d1b/d20/f24 x:0 0 0 2024-03-22T06:14:50.152 INFO:tasks.workunit.client.0.smithi006.stdout:4/106: unlink d6/l10 0 2024-03-22T06:14:50.152 INFO:tasks.workunit.client.0.smithi006.stdout:4/107: chown d6 28 1 2024-03-22T06:14:50.153 INFO:tasks.workunit.client.0.smithi006.stdout:4/108: chown d6/d1b/d20 667 1 2024-03-22T06:14:50.172 INFO:tasks.workunit.client.0.smithi006.stdout:4/109: dwrite f2 [0,4194304] 0 2024-03-22T06:14:50.178 INFO:tasks.workunit.client.0.smithi006.stdout:4/110: symlink d6/d1b/d20/l25 0 2024-03-22T06:14:50.181 INFO:tasks.workunit.client.0.smithi006.stdout:4/111: mkdir d6/d1b/d20/d21/d26 0 2024-03-22T06:14:50.181 INFO:tasks.workunit.client.0.smithi006.stdout:4/112: truncate d6/d1b/d20/f24 566467 0 2024-03-22T06:14:50.230 INFO:tasks.workunit.client.0.smithi006.stdout:2/145: sync 2024-03-22T06:14:50.230 INFO:tasks.workunit.client.0.smithi006.stdout:0/60: sync 2024-03-22T06:14:50.230 INFO:tasks.workunit.client.0.smithi006.stdout:0/61: truncate f4 3158061 0 2024-03-22T06:14:50.231 INFO:tasks.workunit.client.0.smithi006.stdout:2/146: dread f7 [0,4194304] 0 2024-03-22T06:14:50.232 INFO:tasks.workunit.client.0.smithi006.stdout:0/62: rename c5 to db/dc/c13 0 2024-03-22T06:14:50.232 INFO:tasks.workunit.client.0.smithi006.stdout:2/147: dread d13/f23 [0,4194304] 0 2024-03-22T06:14:50.235 INFO:tasks.workunit.client.0.smithi006.stdout:0/63: read f9 [221891,119446] 0 2024-03-22T06:14:50.236 INFO:tasks.workunit.client.0.smithi006.stdout:2/148: dread d13/d20/d26/f15 [0,4194304] 0 2024-03-22T06:14:50.236 INFO:tasks.workunit.client.0.smithi006.stdout:2/149: read - f12 zero size 2024-03-22T06:14:50.236 INFO:tasks.workunit.client.0.smithi006.stdout:2/150: creat d13/d20/d26/d19/f3e x:0 0 0 2024-03-22T06:14:50.236 INFO:tasks.workunit.client.0.smithi006.stdout:2/151: write f3 [1407648,68211] 0 2024-03-22T06:14:50.236 INFO:tasks.workunit.client.0.smithi006.stdout:2/152: write d13/d20/d26/d19/f3e [857904,44534] 0 2024-03-22T06:14:50.236 INFO:tasks.workunit.client.0.smithi006.stdout:2/153: truncate d13/f18 888158 0 2024-03-22T06:14:50.237 INFO:tasks.workunit.client.0.smithi006.stdout:0/64: creat db/d10/f14 x:0 0 0 2024-03-22T06:14:50.237 INFO:tasks.workunit.client.0.smithi006.stdout:0/65: dread - db/ff zero size 2024-03-22T06:14:50.242 INFO:tasks.workunit.client.0.smithi006.stdout:2/154: link d13/l33 d13/l3f 0 2024-03-22T06:14:50.245 INFO:tasks.workunit.client.0.smithi006.stdout:2/155: rmdir d13 39 2024-03-22T06:14:50.246 INFO:tasks.workunit.client.0.smithi006.stdout:2/156: symlink d13/d20/d26/d2a/l40 0 2024-03-22T06:14:50.247 INFO:tasks.workunit.client.0.smithi006.stdout:2/157: stat d13/d20 0 2024-03-22T06:14:50.249 INFO:tasks.workunit.client.0.smithi006.stdout:0/66: dwrite db/d10/f14 [0,4194304] 0 2024-03-22T06:14:50.249 INFO:tasks.workunit.client.0.smithi006.stdout:0/67: stat l3 0 2024-03-22T06:14:50.250 INFO:tasks.workunit.client.0.smithi006.stdout:2/158: rename d13/l3f to d13/d20/d26/d2a/l41 0 2024-03-22T06:14:50.255 INFO:tasks.workunit.client.0.smithi006.stdout:0/68: rename l1 to db/dc/l15 0 2024-03-22T06:14:50.255 INFO:tasks.workunit.client.0.smithi006.stdout:0/69: chown f4 969052 1 2024-03-22T06:14:50.256 INFO:tasks.workunit.client.0.smithi006.stdout:0/70: dread - db/ff zero size 2024-03-22T06:14:50.256 INFO:tasks.workunit.client.0.smithi006.stdout:0/71: chown db/dc/c13 464 1 2024-03-22T06:14:50.258 INFO:tasks.workunit.client.0.smithi006.stdout:0/72: symlink db/dc/l16 0 2024-03-22T06:14:50.258 INFO:tasks.workunit.client.0.smithi006.stdout:2/159: dread fb [0,4194304] 0 2024-03-22T06:14:50.259 INFO:tasks.workunit.client.0.smithi006.stdout:2/160: fsync d13/f18 0 2024-03-22T06:14:50.262 INFO:tasks.workunit.client.0.smithi006.stdout:0/73: mknod db/d10/c17 0 2024-03-22T06:14:50.265 INFO:tasks.workunit.client.0.smithi006.stdout:2/161: rename d13/d20/d26/d19/f24 to d13/d20/f42 0 2024-03-22T06:14:50.265 INFO:tasks.workunit.client.0.smithi006.stdout:2/162: truncate d13/d20/d26/f2f 1148612 0 2024-03-22T06:14:50.266 INFO:tasks.workunit.client.0.smithi006.stdout:2/163: write d13/d20/d26/d19/f39 [360219,44130] 0 2024-03-22T06:14:50.266 INFO:tasks.workunit.client.0.smithi006.stdout:2/164: readlink d13/d20/d35/l3c 0 2024-03-22T06:14:50.266 INFO:tasks.workunit.client.0.smithi006.stdout:2/165: chown d13/d20/d26/d19/f3e 535389 1 2024-03-22T06:14:50.267 INFO:tasks.workunit.client.0.smithi006.stdout:0/74: symlink db/d10/l18 0 2024-03-22T06:14:50.269 INFO:tasks.workunit.client.0.smithi006.stdout:2/166: rename d13/l33 to d13/l43 0 2024-03-22T06:14:50.269 INFO:tasks.workunit.client.0.smithi006.stdout:2/167: chown fc 9264 1 2024-03-22T06:14:50.272 INFO:tasks.workunit.client.0.smithi006.stdout:0/75: mkdir db/dc/d19 0 2024-03-22T06:14:50.272 INFO:tasks.workunit.client.0.smithi006.stdout:0/76: chown db/d10 9079378 1 2024-03-22T06:14:50.272 INFO:tasks.workunit.client.0.smithi006.stdout:0/77: creat db/d10/f1a x:0 0 0 2024-03-22T06:14:50.276 INFO:tasks.workunit.client.0.smithi006.stdout:0/78: rmdir db/dc/d19 0 2024-03-22T06:14:50.276 INFO:tasks.workunit.client.0.smithi006.stdout:0/79: read f9 [768701,23582] 0 2024-03-22T06:14:50.280 INFO:tasks.workunit.client.0.smithi006.stdout:0/80: rename f9 to db/d10/f1b 0 2024-03-22T06:14:50.286 INFO:tasks.workunit.client.0.smithi006.stdout:2/168: dwrite d13/d20/d26/d19/f39 [0,4194304] 0 2024-03-22T06:14:50.309 INFO:tasks.workunit.client.0.smithi006.stdout:8/92: rmdir d9 39 2024-03-22T06:14:50.312 INFO:tasks.workunit.client.0.smithi006.stdout:8/93: mknod d9/db/dc/de/c24 0 2024-03-22T06:14:50.317 INFO:tasks.workunit.client.0.smithi006.stdout:8/94: rename d9/db/f1a to d9/db/f25 0 2024-03-22T06:14:50.317 INFO:tasks.workunit.client.0.smithi006.stdout:8/95: truncate d9/db/dc/f1f 553217 0 2024-03-22T06:14:50.321 INFO:tasks.workunit.client.0.smithi006.stdout:8/96: creat d9/db/d12/f26 x:0 0 0 2024-03-22T06:14:50.324 INFO:tasks.workunit.client.0.smithi006.stdout:8/97: creat d9/f27 x:0 0 0 2024-03-22T06:14:50.327 INFO:tasks.workunit.client.0.smithi006.stdout:8/98: mknod d9/db/d12/c28 0 2024-03-22T06:14:50.330 INFO:tasks.workunit.client.0.smithi006.stdout:8/99: creat d9/db/dc/de/f29 x:0 0 0 2024-03-22T06:14:50.337 INFO:tasks.workunit.client.0.smithi006.stdout:8/100: dread d9/db/dc/de/f17 [0,4194304] 0 2024-03-22T06:14:50.339 INFO:tasks.workunit.client.0.smithi006.stdout:8/101: rename d9/d14/f1c to d9/f2a 0 2024-03-22T06:14:50.342 INFO:tasks.workunit.client.0.smithi006.stdout:4/113: truncate d6/f7 977335 0 2024-03-22T06:14:50.345 INFO:tasks.workunit.client.0.smithi006.stdout:4/114: symlink d6/d1b/l27 0 2024-03-22T06:14:50.345 INFO:tasks.workunit.client.0.smithi006.stdout:4/115: readlink d6/l19 0 2024-03-22T06:14:50.359 INFO:tasks.workunit.client.0.smithi006.stdout:7/120: sync 2024-03-22T06:14:50.359 INFO:tasks.workunit.client.0.smithi006.stdout:5/66: sync 2024-03-22T06:14:50.362 INFO:tasks.workunit.client.0.smithi006.stdout:5/67: mkdir d0/db/d11 0 2024-03-22T06:14:50.362 INFO:tasks.workunit.client.0.smithi006.stdout:5/68: creat d0/db/f12 x:0 0 0 2024-03-22T06:14:50.365 INFO:tasks.workunit.client.0.smithi006.stdout:5/69: mknod d0/db/dc/c13 0 2024-03-22T06:14:50.365 INFO:tasks.workunit.client.0.smithi006.stdout:5/70: chown d0/c2 57 1 2024-03-22T06:14:50.367 INFO:tasks.workunit.client.0.smithi006.stdout:5/71: mkdir d0/db/dc/d14 0 2024-03-22T06:14:50.367 INFO:tasks.workunit.client.0.smithi006.stdout:5/72: write d0/db/dc/f10 [2728168,98327] 0 2024-03-22T06:14:50.376 INFO:tasks.workunit.client.0.smithi006.stdout:5/73: write d0/fe [1954696,26562] 0 2024-03-22T06:14:50.381 INFO:tasks.workunit.client.0.smithi006.stdout:5/74: link d0/db/fd d0/db/dc/d14/f15 0 2024-03-22T06:14:50.455 INFO:tasks.workunit.client.0.smithi006.stdout:5/75: dread d0/f4 [0,4194304] 0 2024-03-22T06:14:50.456 INFO:tasks.workunit.client.0.smithi006.stdout:2/169: dwrite f6 [0,4194304] 0 2024-03-22T06:14:50.466 INFO:tasks.workunit.client.0.smithi006.stdout:2/170: dread d13/d20/d26/d19/f39 [0,4194304] 0 2024-03-22T06:14:50.468 INFO:tasks.workunit.client.0.smithi006.stdout:2/171: symlink d13/l44 0 2024-03-22T06:14:50.470 INFO:tasks.workunit.client.0.smithi006.stdout:7/121: dwrite d5/f1d [0,4194304] 0 2024-03-22T06:14:50.476 INFO:tasks.workunit.client.0.smithi006.stdout:5/76: dwrite d0/db/fd [0,4194304] 0 2024-03-22T06:14:50.481 INFO:tasks.workunit.client.0.smithi006.stdout:2/172: dwrite d13/d20/d26/f2f [0,4194304] 0 2024-03-22T06:14:50.482 INFO:tasks.workunit.client.0.smithi006.stdout:2/173: dread - f12 zero size 2024-03-22T06:14:50.498 INFO:tasks.workunit.client.0.smithi006.stdout:7/122: mknod d5/d1e/c20 0 2024-03-22T06:14:50.498 INFO:tasks.workunit.client.0.smithi006.stdout:7/123: chown f4 12002 1 2024-03-22T06:14:50.498 INFO:tasks.workunit.client.0.smithi006.stdout:7/124: stat d5/fd 0 2024-03-22T06:14:50.498 INFO:tasks.workunit.client.0.smithi006.stdout:5/77: dwrite d0/db/fd [0,4194304] 0 2024-03-22T06:14:50.498 INFO:tasks.workunit.client.0.smithi006.stdout:5/78: fsync d0/f9 0 2024-03-22T06:14:50.505 INFO:tasks.workunit.client.0.smithi006.stdout:7/125: creat d5/d1e/f21 x:0 0 0 2024-03-22T06:14:50.505 INFO:tasks.workunit.client.0.smithi006.stdout:7/126: chown d5/db/f13 59001 1 2024-03-22T06:14:50.516 INFO:tasks.workunit.client.0.smithi006.stdout:5/79: dwrite d0/db/dc/d14/f15 [0,4194304] 0 2024-03-22T06:14:50.519 INFO:tasks.workunit.client.0.smithi006.stdout:5/80: mkdir d0/db/d16 0 2024-03-22T06:14:50.519 INFO:tasks.workunit.client.0.smithi006.stdout:5/81: fdatasync d0/db/dc/d14/f15 0 2024-03-22T06:14:50.522 INFO:tasks.workunit.client.0.smithi006.stdout:5/82: symlink d0/db/d16/l17 0 2024-03-22T06:14:50.529 INFO:tasks.workunit.client.0.smithi006.stdout:5/83: write d0/f5 [635022,61300] 0 2024-03-22T06:14:50.535 INFO:tasks.workunit.client.0.smithi006.stdout:5/84: rename d0/db/dc/c13 to d0/db/c18 0 2024-03-22T06:14:50.535 INFO:tasks.workunit.client.0.smithi006.stdout:5/85: truncate d0/f9 662032 0 2024-03-22T06:14:50.538 INFO:tasks.workunit.client.0.smithi006.stdout:5/86: unlink d0/f4 0 2024-03-22T06:14:50.538 INFO:tasks.workunit.client.0.smithi006.stdout:5/87: readlink d0/db/d16/l17 0 2024-03-22T06:14:50.538 INFO:tasks.workunit.client.0.smithi006.stdout:5/88: fsync d0/fe 0 2024-03-22T06:14:50.541 INFO:tasks.workunit.client.0.smithi006.stdout:5/89: mknod d0/db/d16/c19 0 2024-03-22T06:14:50.548 INFO:tasks.workunit.client.0.smithi006.stdout:5/90: write d0/db/fd [195171,82087] 0 2024-03-22T06:14:50.549 INFO:tasks.workunit.client.0.smithi006.stdout:5/91: creat d0/f1a x:0 0 0 2024-03-22T06:14:50.549 INFO:tasks.workunit.client.0.smithi006.stdout:5/92: chown d0/db/d11 29 1 2024-03-22T06:14:50.549 INFO:tasks.workunit.client.0.smithi006.stdout:5/93: truncate d0/f5 4661600 0 2024-03-22T06:14:50.553 INFO:tasks.workunit.client.0.smithi006.stdout:5/94: rmdir d0/db/dc 39 2024-03-22T06:14:50.556 INFO:tasks.workunit.client.0.smithi006.stdout:5/95: link d0/fe d0/db/f1b 0 2024-03-22T06:14:50.698 INFO:tasks.workunit.client.0.smithi006.stdout:6/167: sync 2024-03-22T06:14:50.698 INFO:tasks.workunit.client.0.smithi006.stdout:9/109: sync 2024-03-22T06:14:50.698 INFO:tasks.workunit.client.0.smithi006.stdout:1/30: sync 2024-03-22T06:14:50.698 INFO:tasks.workunit.client.0.smithi006.stdout:3/57: sync 2024-03-22T06:14:50.698 INFO:tasks.workunit.client.0.smithi006.stdout:3/58: chown d0/d8/d9 27221470 1 2024-03-22T06:14:50.698 INFO:tasks.workunit.client.0.smithi006.stdout:3/59: write d0/f3 [8453942,5946] 0 2024-03-22T06:14:50.700 INFO:tasks.workunit.client.0.smithi006.stdout:1/31: mkdir d8 0 2024-03-22T06:14:50.700 INFO:tasks.workunit.client.0.smithi006.stdout:1/32: fsync f3 0 2024-03-22T06:14:50.700 INFO:tasks.workunit.client.0.smithi006.stdout:1/33: stat c6 0 2024-03-22T06:14:50.703 INFO:tasks.workunit.client.0.smithi006.stdout:3/60: creat d0/d8/f13 x:0 0 0 2024-03-22T06:14:50.709 INFO:tasks.workunit.client.0.smithi006.stdout:1/34: dread f3 [0,4194304] 0 2024-03-22T06:14:50.710 INFO:tasks.workunit.client.0.smithi006.stdout:3/61: rename d0/d8/le to d0/l14 0 2024-03-22T06:14:50.711 INFO:tasks.workunit.client.0.smithi006.stdout:9/110: dwrite d8/d10/d28/f1d [0,4194304] 0 2024-03-22T06:14:50.712 INFO:tasks.workunit.client.0.smithi006.stdout:1/35: mkdir d8/d9 0 2024-03-22T06:14:50.715 INFO:tasks.workunit.client.0.smithi006.stdout:3/62: symlink d0/d8/d9/l15 0 2024-03-22T06:14:50.715 INFO:tasks.workunit.client.0.smithi006.stdout:3/63: fsync d0/f12 0 2024-03-22T06:14:50.715 INFO:tasks.workunit.client.0.smithi006.stdout:3/64: chown d0/d8/d9 53740 1 2024-03-22T06:14:50.715 INFO:tasks.workunit.client.0.smithi006.stdout:3/65: write d0/fa [2073911,9426] 0 2024-03-22T06:14:50.717 INFO:tasks.workunit.client.0.smithi006.stdout:1/36: write f3 [1570722,91841] 0 2024-03-22T06:14:50.719 INFO:tasks.workunit.client.0.smithi006.stdout:3/66: link d0/f12 d0/d8/d9/f16 0 2024-03-22T06:14:50.721 INFO:tasks.workunit.client.0.smithi006.stdout:1/37: mknod d8/d9/ca 0 2024-03-22T06:14:50.721 INFO:tasks.workunit.client.0.smithi006.stdout:1/38: write f3 [1994433,27092] 0 2024-03-22T06:14:50.721 INFO:tasks.workunit.client.0.smithi006.stdout:9/111: dread d8/ff [0,4194304] 0 2024-03-22T06:14:50.723 INFO:tasks.workunit.client.0.smithi006.stdout:9/112: dread d8/d10/d11/f16 [0,4194304] 0 2024-03-22T06:14:50.723 INFO:tasks.workunit.client.0.smithi006.stdout:3/67: link d0/d8/f13 d0/d8/f17 0 2024-03-22T06:14:50.723 INFO:tasks.workunit.client.0.smithi006.stdout:3/68: dread - d0/d8/d9/f16 zero size 2024-03-22T06:14:50.723 INFO:tasks.workunit.client.0.smithi006.stdout:3/69: dread - d0/d8/f17 zero size 2024-03-22T06:14:50.723 INFO:tasks.workunit.client.0.smithi006.stdout:9/113: write d8/d10/d11/f16 [239165,72224] 0 2024-03-22T06:14:50.725 INFO:tasks.workunit.client.0.smithi006.stdout:1/39: link l2 d8/d9/lb 0 2024-03-22T06:14:50.725 INFO:tasks.workunit.client.0.smithi006.stdout:1/40: readlink l2 0 2024-03-22T06:14:50.739 INFO:tasks.workunit.client.0.smithi006.stdout:1/41: dread f3 [0,4194304] 0 2024-03-22T06:14:50.740 INFO:tasks.workunit.client.0.smithi006.stdout:9/114: dwrite d8/d10/d23/f27 [0,4194304] 0 2024-03-22T06:14:50.743 INFO:tasks.workunit.client.0.smithi006.stdout:1/42: creat d8/d9/fc x:0 0 0 2024-03-22T06:14:50.743 INFO:tasks.workunit.client.0.smithi006.stdout:1/43: write f3 [885285,128179] 0 2024-03-22T06:14:50.743 INFO:tasks.workunit.client.0.smithi006.stdout:1/44: read - d8/d9/fc zero size 2024-03-22T06:14:50.743 INFO:tasks.workunit.client.0.smithi006.stdout:1/45: read f3 [1208636,105615] 0 2024-03-22T06:14:50.743 INFO:tasks.workunit.client.0.smithi006.stdout:1/46: chown d8/d9/ca 3282 1 2024-03-22T06:14:50.743 INFO:tasks.workunit.client.0.smithi006.stdout:1/47: creat d8/d9/fd x:0 0 0 2024-03-22T06:14:50.743 INFO:tasks.workunit.client.0.smithi006.stdout:1/48: write d8/d9/fc [846642,50279] 0 2024-03-22T06:14:50.748 INFO:tasks.workunit.client.0.smithi006.stdout:9/115: creat d8/d10/d20/f2c x:0 0 0 2024-03-22T06:14:50.754 INFO:tasks.workunit.client.0.smithi006.stdout:9/116: link d8/d10/d11/f1c d8/d10/d20/f2d 0 2024-03-22T06:14:50.756 INFO:tasks.workunit.client.0.smithi006.stdout:9/117: fsync d8/d10/d11/f16 0 2024-03-22T06:14:50.759 INFO:tasks.workunit.client.0.smithi006.stdout:5/96: rmdir d0 39 2024-03-22T06:14:50.759 INFO:tasks.workunit.client.0.smithi006.stdout:5/97: readlink d0/l6 0 2024-03-22T06:14:50.767 INFO:tasks.workunit.client.0.smithi006.stdout:5/98: mkdir d0/db/d16/d1c 0 2024-03-22T06:14:50.767 INFO:tasks.workunit.client.0.smithi006.stdout:5/99: write d0/fe [708474,59479] 0 2024-03-22T06:14:50.781 INFO:tasks.workunit.client.0.smithi006.stdout:5/100: dwrite d0/f9 [0,4194304] 0 2024-03-22T06:14:50.796 INFO:tasks.workunit.client.0.smithi006.stdout:5/101: dwrite d0/f5 [4194304,4194304] 0 2024-03-22T06:14:50.799 INFO:tasks.workunit.client.0.smithi006.stdout:5/102: link d0/f9 d0/db/dc/d14/f1d 0 2024-03-22T06:14:50.799 INFO:tasks.workunit.client.0.smithi006.stdout:5/103: stat d0/f9 0 2024-03-22T06:14:50.869 INFO:tasks.workunit.client.0.smithi006.stdout:3/70: rmdir d0 39 2024-03-22T06:14:50.869 INFO:tasks.workunit.client.0.smithi006.stdout:3/71: write d0/d8/f13 [969055,97143] 0 2024-03-22T06:14:50.869 INFO:tasks.workunit.client.0.smithi006.stdout:3/72: chown d0/d8/d9/f16 8 1 2024-03-22T06:14:50.870 INFO:tasks.workunit.client.0.smithi006.stdout:3/73: getdents d0/d8 0 2024-03-22T06:14:50.870 INFO:tasks.workunit.client.0.smithi006.stdout:3/74: stat d0 0 2024-03-22T06:14:50.873 INFO:tasks.workunit.client.0.smithi006.stdout:3/75: link d0/d4/fb d0/d8/f18 0 2024-03-22T06:14:50.873 INFO:tasks.workunit.client.0.smithi006.stdout:3/76: stat d0/d4/d10 0 2024-03-22T06:14:50.886 INFO:tasks.workunit.client.0.smithi006.stdout:1/49: rmdir d8/d9 39 2024-03-22T06:14:50.887 INFO:tasks.workunit.client.0.smithi006.stdout:1/50: symlink d8/le 0 2024-03-22T06:14:50.888 INFO:tasks.workunit.client.0.smithi006.stdout:1/51: stat d8/d9 0 2024-03-22T06:14:50.891 INFO:tasks.workunit.client.0.smithi006.stdout:3/77: dwrite d0/d8/d9/f16 [0,4194304] 0 2024-03-22T06:14:50.893 INFO:tasks.workunit.client.0.smithi006.stdout:1/52: truncate d8/d9/fc 94842 0 2024-03-22T06:14:50.908 INFO:tasks.workunit.client.0.smithi006.stdout:1/53: dwrite d8/d9/fd [0,4194304] 0 2024-03-22T06:14:50.916 INFO:tasks.workunit.client.0.smithi006.stdout:1/54: dread d8/d9/fd [0,4194304] 0 2024-03-22T06:14:50.931 INFO:tasks.workunit.client.0.smithi006.stdout:5/104: truncate d0/f5 4408634 0 2024-03-22T06:14:50.933 INFO:tasks.workunit.client.0.smithi006.stdout:1/55: dwrite d8/d9/fd [0,4194304] 0 2024-03-22T06:14:50.935 INFO:tasks.workunit.client.0.smithi006.stdout:5/105: rename d0/db/dc/f10 to d0/db/f1e 0 2024-03-22T06:14:50.935 INFO:tasks.workunit.client.0.smithi006.stdout:5/106: chown d0/ca 58493 1 2024-03-22T06:14:50.938 INFO:tasks.workunit.client.0.smithi006.stdout:1/56: rename c6 to d8/cf 0 2024-03-22T06:14:50.941 INFO:tasks.workunit.client.0.smithi006.stdout:1/57: mkdir d8/d9/d10 0 2024-03-22T06:14:50.941 INFO:tasks.workunit.client.0.smithi006.stdout:1/58: write f3 [3836422,128865] 0 2024-03-22T06:14:50.943 INFO:tasks.workunit.client.0.smithi006.stdout:1/59: rmdir d8 39 2024-03-22T06:14:50.945 INFO:tasks.workunit.client.0.smithi006.stdout:1/60: mknod d8/d9/d10/c11 0 2024-03-22T06:14:50.958 INFO:tasks.workunit.client.0.smithi006.stdout:5/107: dwrite d0/db/f1b [0,4194304] 0 2024-03-22T06:14:50.962 INFO:tasks.workunit.client.0.smithi006.stdout:5/108: mknod d0/db/dc/c1f 0 2024-03-22T06:14:50.962 INFO:tasks.workunit.client.0.smithi006.stdout:5/109: stat d0/l7 0 2024-03-22T06:14:50.965 INFO:tasks.workunit.client.0.smithi006.stdout:5/110: rename d0/f3 to d0/db/dc/d14/f20 0 2024-03-22T06:14:50.981 INFO:tasks.workunit.client.0.smithi006.stdout:5/111: dwrite d0/f9 [0,4194304] 0 2024-03-22T06:14:50.981 INFO:tasks.workunit.client.0.smithi006.stdout:5/112: readlink d0/l8 0 2024-03-22T06:14:50.982 INFO:tasks.workunit.client.0.smithi006.stdout:5/113: chown d0/db/c18 21 1 2024-03-22T06:14:50.982 INFO:tasks.workunit.client.0.smithi006.stdout:5/114: write d0/db/f1e [886148,102292] 0 2024-03-22T06:14:50.987 INFO:tasks.workunit.client.0.smithi006.stdout:5/115: creat d0/db/d16/f21 x:0 0 0 2024-03-22T06:14:50.988 INFO:tasks.workunit.client.0.smithi006.stdout:5/116: chown d0/f5 5743 1 2024-03-22T06:14:50.988 INFO:tasks.workunit.client.0.smithi006.stdout:5/117: write d0/db/fd [3901120,96588] 0 2024-03-22T06:14:50.990 INFO:tasks.workunit.client.0.smithi006.stdout:5/118: rename d0/db/f1b to d0/f22 0 2024-03-22T06:14:51.034 INFO:tasks.workunit.client.0.smithi006.stdout:1/61: dwrite d8/d9/fc [0,4194304] 0 2024-03-22T06:14:51.062 INFO:tasks.workunit.client.0.smithi006.stdout:7/127: sync 2024-03-22T06:14:51.062 INFO:tasks.workunit.client.0.smithi006.stdout:2/174: sync 2024-03-22T06:14:51.062 INFO:tasks.workunit.client.0.smithi006.stdout:4/116: sync 2024-03-22T06:14:51.062 INFO:tasks.workunit.client.0.smithi006.stdout:0/81: sync 2024-03-22T06:14:51.062 INFO:tasks.workunit.client.0.smithi006.stdout:7/128: creat d5/d1e/f22 x:0 0 0 2024-03-22T06:14:51.062 INFO:tasks.workunit.client.0.smithi006.stdout:8/102: sync 2024-03-22T06:14:51.062 INFO:tasks.workunit.client.0.smithi006.stdout:8/103: getdents d9/db/d12/d1e 0 2024-03-22T06:14:51.068 INFO:tasks.workunit.client.0.smithi006.stdout:4/117: link d6/l14 d6/d1b/d20/d21/l28 0 2024-03-22T06:14:51.072 INFO:tasks.workunit.client.0.smithi006.stdout:0/82: read db/d10/f14 [433850,76570] 0 2024-03-22T06:14:51.072 INFO:tasks.workunit.client.0.smithi006.stdout:7/129: rename l3 to d5/l23 0 2024-03-22T06:14:51.073 INFO:tasks.workunit.client.0.smithi006.stdout:8/104: read d9/db/dc/de/f22 [1707379,607] 0 2024-03-22T06:14:51.073 INFO:tasks.workunit.client.0.smithi006.stdout:8/105: truncate f1 781876 0 2024-03-22T06:14:51.075 INFO:tasks.workunit.client.0.smithi006.stdout:4/118: creat d6/d1b/f29 x:0 0 0 2024-03-22T06:14:51.078 INFO:tasks.workunit.client.0.smithi006.stdout:0/83: link db/dc/l15 db/l1c 0 2024-03-22T06:14:51.081 INFO:tasks.workunit.client.0.smithi006.stdout:7/130: symlink d5/db/d1f/l24 0 2024-03-22T06:14:51.082 INFO:tasks.workunit.client.0.smithi006.stdout:7/131: read d5/f9 [21291,15167] 0 2024-03-22T06:14:51.082 INFO:tasks.workunit.client.0.smithi006.stdout:0/84: dread f4 [0,4194304] 0 2024-03-22T06:14:51.082 INFO:tasks.workunit.client.0.smithi006.stdout:7/132: readlink d5/l23 0 2024-03-22T06:14:51.082 INFO:tasks.workunit.client.0.smithi006.stdout:1/62: truncate d8/d9/fd 113199 0 2024-03-22T06:14:51.084 INFO:tasks.workunit.client.0.smithi006.stdout:4/119: read d6/f8 [4180500,78625] 0 2024-03-22T06:14:51.085 INFO:tasks.workunit.client.0.smithi006.stdout:5/119: dwrite d0/f5 [0,4194304] 0 2024-03-22T06:14:51.086 INFO:tasks.workunit.client.0.smithi006.stdout:8/106: creat d9/d14/f2b x:0 0 0 2024-03-22T06:14:51.087 INFO:tasks.workunit.client.0.smithi006.stdout:2/175: dwrite f6 [0,4194304] 0 2024-03-22T06:14:51.088 INFO:tasks.workunit.client.0.smithi006.stdout:0/85: link db/dc/l16 db/dc/l1d 0 2024-03-22T06:14:51.090 INFO:tasks.workunit.client.0.smithi006.stdout:7/133: creat d5/db/d10/f25 x:0 0 0 2024-03-22T06:14:51.092 INFO:tasks.workunit.client.0.smithi006.stdout:4/120: mkdir d6/d1b/d20/d21/d2a 0 2024-03-22T06:14:51.092 INFO:tasks.workunit.client.0.smithi006.stdout:4/121: truncate d6/f11 833877 0 2024-03-22T06:14:51.096 INFO:tasks.workunit.client.0.smithi006.stdout:5/120: link d0/db/dc/c1f d0/db/dc/d14/c23 0 2024-03-22T06:14:51.099 INFO:tasks.workunit.client.0.smithi006.stdout:8/107: rmdir d9/d14/d23 0 2024-03-22T06:14:51.103 INFO:tasks.workunit.client.0.smithi006.stdout:2/176: creat d13/d20/d35/f45 x:0 0 0 2024-03-22T06:14:51.106 INFO:tasks.workunit.client.0.smithi006.stdout:7/134: mknod d5/db/c26 0 2024-03-22T06:14:51.111 INFO:tasks.workunit.client.0.smithi006.stdout:4/122: rename d6/d1b/l27 to d6/d1b/d20/d21/l2b 0 2024-03-22T06:14:51.111 INFO:tasks.workunit.client.0.smithi006.stdout:4/123: chown d6/d1b/d20 0 1 2024-03-22T06:14:51.111 INFO:tasks.workunit.client.0.smithi006.stdout:4/124: dread d6/f11 [0,4194304] 0 2024-03-22T06:14:51.111 INFO:tasks.workunit.client.0.smithi006.stdout:4/125: dread - d6/f1f zero size 2024-03-22T06:14:51.113 INFO:tasks.workunit.client.0.smithi006.stdout:5/121: mknod d0/db/d16/d1c/c24 0 2024-03-22T06:14:51.113 INFO:tasks.workunit.client.0.smithi006.stdout:5/122: fsync d0/db/dc/d14/f20 0 2024-03-22T06:14:51.117 INFO:tasks.workunit.client.0.smithi006.stdout:0/86: getdents db/d10 0 2024-03-22T06:14:51.124 INFO:tasks.workunit.client.0.smithi006.stdout:8/108: rename l0 to d9/db/d12/l2c 0 2024-03-22T06:14:51.126 INFO:tasks.workunit.client.0.smithi006.stdout:5/123: creat d0/db/f25 x:0 0 0 2024-03-22T06:14:51.126 INFO:tasks.workunit.client.0.smithi006.stdout:5/124: stat d0/db/dc/d14/f15 0 2024-03-22T06:14:51.127 INFO:tasks.workunit.client.0.smithi006.stdout:4/126: dwrite d6/f11 [0,4194304] 0 2024-03-22T06:14:51.130 INFO:tasks.workunit.client.0.smithi006.stdout:0/87: symlink db/d10/l1e 0 2024-03-22T06:14:51.133 INFO:tasks.workunit.client.0.smithi006.stdout:8/109: creat d9/db/d12/d1e/f2d x:0 0 0 2024-03-22T06:14:51.136 INFO:tasks.workunit.client.0.smithi006.stdout:5/125: rmdir d0/db/dc/d14 39 2024-03-22T06:14:51.141 INFO:tasks.workunit.client.0.smithi006.stdout:8/110: write d9/f11 [4151517,126788] 0 2024-03-22T06:14:51.142 INFO:tasks.workunit.client.0.smithi006.stdout:4/127: mkdir d6/d1b/d20/d2c 0 2024-03-22T06:14:51.144 INFO:tasks.workunit.client.0.smithi006.stdout:0/88: creat db/d10/f1f x:0 0 0 2024-03-22T06:14:51.147 INFO:tasks.workunit.client.0.smithi006.stdout:5/126: symlink d0/db/l26 0 2024-03-22T06:14:51.153 INFO:tasks.workunit.client.0.smithi006.stdout:8/111: creat d9/db/f2e x:0 0 0 2024-03-22T06:14:51.153 INFO:tasks.workunit.client.0.smithi006.stdout:8/112: creat d9/d14/f2f x:0 0 0 2024-03-22T06:14:51.153 INFO:tasks.workunit.client.0.smithi006.stdout:8/113: fsync d9/db/dc/fd 0 2024-03-22T06:14:51.155 INFO:tasks.workunit.client.0.smithi006.stdout:0/89: symlink db/dc/l20 0 2024-03-22T06:14:51.162 INFO:tasks.workunit.client.0.smithi006.stdout:8/114: write d9/db/dc/de/f16 [1123060,42305] 0 2024-03-22T06:14:51.165 INFO:tasks.workunit.client.0.smithi006.stdout:4/128: dwrite d6/f9 [0,4194304] 0 2024-03-22T06:14:51.165 INFO:tasks.workunit.client.0.smithi006.stdout:4/129: write d6/ff [507409,38262] 0 2024-03-22T06:14:51.168 INFO:tasks.workunit.client.0.smithi006.stdout:4/130: dread d6/d13/f17 [0,4194304] 0 2024-03-22T06:14:51.169 INFO:tasks.workunit.client.0.smithi006.stdout:5/127: dwrite d0/db/fd [4194304,4194304] 0 2024-03-22T06:14:51.181 INFO:tasks.workunit.client.0.smithi006.stdout:5/128: dread d0/fe [0,4194304] 0 2024-03-22T06:14:51.181 INFO:tasks.workunit.client.0.smithi006.stdout:5/129: chown d0/db/l26 355689694 1 2024-03-22T06:14:51.181 INFO:tasks.workunit.client.0.smithi006.stdout:5/130: stat d0/f22 0 2024-03-22T06:14:51.184 INFO:tasks.workunit.client.0.smithi006.stdout:4/131: dwrite f0 [8388608,4194304] 0 2024-03-22T06:14:51.184 INFO:tasks.workunit.client.0.smithi006.stdout:4/132: chown d6 1779 1 2024-03-22T06:14:51.185 INFO:tasks.workunit.client.0.smithi006.stdout:5/131: mknod d0/db/d16/c27 0 2024-03-22T06:14:51.206 INFO:tasks.workunit.client.0.smithi006.stdout:5/132: dwrite d0/db/fd [4194304,4194304] 0 2024-03-22T06:14:51.207 INFO:tasks.workunit.client.0.smithi006.stdout:5/133: truncate d0/db/f12 229302 0 2024-03-22T06:14:51.207 INFO:tasks.workunit.client.0.smithi006.stdout:5/134: readlink d0/l6 0 2024-03-22T06:14:51.207 INFO:tasks.workunit.client.0.smithi006.stdout:5/135: write d0/fe [4317524,79844] 0 2024-03-22T06:14:51.208 INFO:tasks.workunit.client.0.smithi006.stdout:4/133: dwrite d6/fb [4194304,4194304] 0 2024-03-22T06:14:51.208 INFO:tasks.workunit.client.0.smithi006.stdout:4/134: creat d6/d1b/f2d x:0 0 0 2024-03-22T06:14:51.209 INFO:tasks.workunit.client.0.smithi006.stdout:4/135: dread - d6/d1b/f2d zero size 2024-03-22T06:14:51.210 INFO:tasks.workunit.client.0.smithi006.stdout:5/136: mknod d0/c28 0 2024-03-22T06:14:51.213 INFO:tasks.workunit.client.0.smithi006.stdout:4/136: mknod d6/d1b/d20/d21/c2e 0 2024-03-22T06:14:51.214 INFO:tasks.workunit.client.0.smithi006.stdout:5/137: dread d0/db/fd [0,4194304] 0 2024-03-22T06:14:51.219 INFO:tasks.workunit.client.0.smithi006.stdout:5/138: unlink d0/db/f12 0 2024-03-22T06:14:51.219 INFO:tasks.workunit.client.0.smithi006.stdout:5/139: chown d0/c28 308156571 1 2024-03-22T06:14:51.219 INFO:tasks.workunit.client.0.smithi006.stdout:7/135: rmdir d5/d1e 39 2024-03-22T06:14:51.223 INFO:tasks.workunit.client.0.smithi006.stdout:5/140: rename d0/fe to d0/db/f29 0 2024-03-22T06:14:51.225 INFO:tasks.workunit.client.0.smithi006.stdout:6/168: sync 2024-03-22T06:14:51.225 INFO:tasks.workunit.client.0.smithi006.stdout:9/118: sync 2024-03-22T06:14:51.225 INFO:tasks.workunit.client.0.smithi006.stdout:3/78: sync 2024-03-22T06:14:51.225 INFO:tasks.workunit.client.0.smithi006.stdout:3/79: readlink d0/l14 0 2024-03-22T06:14:51.226 INFO:tasks.workunit.client.0.smithi006.stdout:3/80: chown d0/d4/fb 1 1 2024-03-22T06:14:51.226 INFO:tasks.workunit.client.0.smithi006.stdout:3/81: chown d0/d4/d10 0 1 2024-03-22T06:14:51.226 INFO:tasks.workunit.client.0.smithi006.stdout:3/82: fdatasync d0/d8/f18 0 2024-03-22T06:14:51.226 INFO:tasks.workunit.client.0.smithi006.stdout:7/136: symlink d5/db/d1f/l27 0 2024-03-22T06:14:51.226 INFO:tasks.workunit.client.0.smithi006.stdout:7/137: creat d5/f28 x:0 0 0 2024-03-22T06:14:51.226 INFO:tasks.workunit.client.0.smithi006.stdout:7/138: truncate d5/f9 427028 0 2024-03-22T06:14:51.227 INFO:tasks.workunit.client.0.smithi006.stdout:9/119: truncate d8/d10/d11/f1c 385995 0 2024-03-22T06:14:51.230 INFO:tasks.workunit.client.0.smithi006.stdout:5/141: dread d0/db/f29 [0,4194304] 0 2024-03-22T06:14:51.230 INFO:tasks.workunit.client.0.smithi006.stdout:6/169: symlink d6/d1d/d32/l3f 0 2024-03-22T06:14:51.232 INFO:tasks.workunit.client.0.smithi006.stdout:3/83: unlink d0/d8/d9/f16 0 2024-03-22T06:14:51.237 INFO:tasks.workunit.client.0.smithi006.stdout:9/120: truncate d8/fb 2803764 0 2024-03-22T06:14:51.237 INFO:tasks.workunit.client.0.smithi006.stdout:9/121: creat d8/d10/f2e x:0 0 0 2024-03-22T06:14:51.238 INFO:tasks.workunit.client.0.smithi006.stdout:5/142: symlink d0/db/d16/l2a 0 2024-03-22T06:14:51.240 INFO:tasks.workunit.client.0.smithi006.stdout:6/170: link d6/d1b/f3c d6/f40 0 2024-03-22T06:14:51.240 INFO:tasks.workunit.client.0.smithi006.stdout:6/171: write d6/f11 [4367004,81071] 0 2024-03-22T06:14:51.240 INFO:tasks.workunit.client.0.smithi006.stdout:3/84: fsync d0/d8/f17 0 2024-03-22T06:14:51.245 INFO:tasks.workunit.client.0.smithi006.stdout:3/85: rmdir d0 39 2024-03-22T06:14:51.251 INFO:tasks.workunit.client.0.smithi006.stdout:2/177: truncate d13/d20/d26/d19/f1c 3314618 0 2024-03-22T06:14:51.251 INFO:tasks.workunit.client.0.smithi006.stdout:2/178: creat d13/d20/d26/d19/f46 x:0 0 0 2024-03-22T06:14:51.254 INFO:tasks.workunit.client.0.smithi006.stdout:3/86: rename d0/l14 to d0/l19 0 2024-03-22T06:14:51.259 INFO:tasks.workunit.client.0.smithi006.stdout:3/87: write d0/f3 [8269039,11238] 0 2024-03-22T06:14:51.259 INFO:tasks.workunit.client.0.smithi006.stdout:9/122: dwrite d8/f9 [0,4194304] 0 2024-03-22T06:14:51.264 INFO:tasks.workunit.client.0.smithi006.stdout:3/88: unlink d0/f12 0 2024-03-22T06:14:51.264 INFO:tasks.workunit.client.0.smithi006.stdout:3/89: read d0/d8/f17 [61766,45932] 0 2024-03-22T06:14:51.264 INFO:tasks.workunit.client.0.smithi006.stdout:3/90: chown d0 1 1 2024-03-22T06:14:51.269 INFO:tasks.workunit.client.0.smithi006.stdout:9/123: mkdir d8/d10/d11/d2f 0 2024-03-22T06:14:51.271 INFO:tasks.workunit.client.0.smithi006.stdout:3/91: creat d0/d8/f1a x:0 0 0 2024-03-22T06:14:51.271 INFO:tasks.workunit.client.0.smithi006.stdout:3/92: read - d0/d8/f1a zero size 2024-03-22T06:14:51.272 INFO:tasks.workunit.client.0.smithi006.stdout:2/179: dwrite d13/f18 [0,4194304] 0 2024-03-22T06:14:51.272 INFO:tasks.workunit.client.0.smithi006.stdout:5/143: dwrite d0/db/dc/d14/f20 [0,4194304] 0 2024-03-22T06:14:51.278 INFO:tasks.workunit.client.0.smithi006.stdout:3/93: symlink d0/d8/d9/l1b 0 2024-03-22T06:14:51.278 INFO:tasks.workunit.client.0.smithi006.stdout:3/94: readlink d0/d8/d9/l15 0 2024-03-22T06:14:51.279 INFO:tasks.workunit.client.0.smithi006.stdout:2/180: write fc [2569566,98784] 0 2024-03-22T06:14:51.282 INFO:tasks.workunit.client.0.smithi006.stdout:8/115: getdents d9/db/d12/d1e 0 2024-03-22T06:14:51.286 INFO:tasks.workunit.client.0.smithi006.stdout:5/144: rmdir d0/db/dc/d14 39 2024-03-22T06:14:51.286 INFO:tasks.workunit.client.0.smithi006.stdout:5/145: read d0/db/fd [2893966,74994] 0 2024-03-22T06:14:51.289 INFO:tasks.workunit.client.0.smithi006.stdout:3/95: dread d0/d8/f13 [0,4194304] 0 2024-03-22T06:14:51.291 INFO:tasks.workunit.client.0.smithi006.stdout:0/90: fsync db/d10/f1f 0 2024-03-22T06:14:51.291 INFO:tasks.workunit.client.0.smithi006.stdout:0/91: stat db/d10/l1e 0 2024-03-22T06:14:51.292 INFO:tasks.workunit.client.0.smithi006.stdout:2/181: link d13/d20/d26/d19/f39 d13/d20/d26/d19/d28/d29/f47 0 2024-03-22T06:14:51.293 INFO:tasks.workunit.client.0.smithi006.stdout:2/182: truncate f12 726331 0 2024-03-22T06:14:51.295 INFO:tasks.workunit.client.0.smithi006.stdout:5/146: write d0/db/dc/d14/f15 [4652858,16358] 0 2024-03-22T06:14:51.295 INFO:tasks.workunit.client.0.smithi006.stdout:5/147: read - d0/db/f25 zero size 2024-03-22T06:14:51.295 INFO:tasks.workunit.client.0.smithi006.stdout:5/148: chown d0/db/fd 673079893 1 2024-03-22T06:14:51.299 INFO:tasks.workunit.client.0.smithi006.stdout:2/183: getdents d13/d20 0 2024-03-22T06:14:51.299 INFO:tasks.workunit.client.0.smithi006.stdout:2/184: dread - d13/d20/d26/d19/d28/f38 zero size 2024-03-22T06:14:51.301 INFO:tasks.workunit.client.0.smithi006.stdout:5/149: creat d0/db/dc/d14/f2b x:0 0 0 2024-03-22T06:14:51.303 INFO:tasks.workunit.client.0.smithi006.stdout:3/96: mkdir d0/d8/d1c 0 2024-03-22T06:14:51.305 INFO:tasks.workunit.client.0.smithi006.stdout:2/185: rmdir d13/d20/d26/d19/d28/d29 39 2024-03-22T06:14:51.308 INFO:tasks.workunit.client.0.smithi006.stdout:3/97: dread d0/d4/fb [0,4194304] 0 2024-03-22T06:14:51.308 INFO:tasks.workunit.client.0.smithi006.stdout:3/98: stat d0 0 2024-03-22T06:14:51.308 INFO:tasks.workunit.client.0.smithi006.stdout:3/99: chown d0/d8/d9/l1b 19460 1 2024-03-22T06:14:51.308 INFO:tasks.workunit.client.0.smithi006.stdout:5/150: mknod d0/db/d16/c2c 0 2024-03-22T06:14:51.314 INFO:tasks.workunit.client.0.smithi006.stdout:3/100: write d0/d4/fb [786067,89211] 0 2024-03-22T06:14:51.314 INFO:tasks.workunit.client.0.smithi006.stdout:0/92: dwrite db/d10/f1a [0,4194304] 0 2024-03-22T06:14:51.314 INFO:tasks.workunit.client.0.smithi006.stdout:3/101: dread d0/d8/f17 [0,4194304] 0 2024-03-22T06:14:51.319 INFO:tasks.workunit.client.0.smithi006.stdout:3/102: dread d0/f3 [4194304,4194304] 0 2024-03-22T06:14:51.319 INFO:tasks.workunit.client.0.smithi006.stdout:3/103: dread - d0/d8/f1a zero size 2024-03-22T06:14:51.319 INFO:tasks.workunit.client.0.smithi006.stdout:3/104: write d0/fa [1197863,63550] 0 2024-03-22T06:14:51.322 INFO:tasks.workunit.client.0.smithi006.stdout:2/186: dwrite d13/d20/d26/d19/d28/d29/f47 [0,4194304] 0 2024-03-22T06:14:51.322 INFO:tasks.workunit.client.0.smithi006.stdout:2/187: readlink d13/l44 0 2024-03-22T06:14:51.322 INFO:tasks.workunit.client.0.smithi006.stdout:2/188: chown d13/d20/d26/d19 70 1 2024-03-22T06:14:51.322 INFO:tasks.workunit.client.0.smithi006.stdout:2/189: creat d13/d20/d26/d19/f48 x:0 0 0 2024-03-22T06:14:51.322 INFO:tasks.workunit.client.0.smithi006.stdout:3/105: creat d0/d4/f1d x:0 0 0 2024-03-22T06:14:51.327 INFO:tasks.workunit.client.0.smithi006.stdout:2/190: mknod d13/d20/d26/c49 0 2024-03-22T06:14:51.327 INFO:tasks.workunit.client.0.smithi006.stdout:2/191: creat d13/d20/d26/d19/d28/f4a x:0 0 0 2024-03-22T06:14:51.329 INFO:tasks.workunit.client.0.smithi006.stdout:3/106: creat d0/d8/f1e x:0 0 0 2024-03-22T06:14:51.331 INFO:tasks.workunit.client.0.smithi006.stdout:0/93: dwrite db/d10/f1f [0,4194304] 0 2024-03-22T06:14:51.339 INFO:tasks.workunit.client.0.smithi006.stdout:0/94: write f4 [2019243,109489] 0 2024-03-22T06:14:51.352 INFO:tasks.workunit.client.0.smithi006.stdout:2/192: dwrite fc [0,4194304] 0 2024-03-22T06:14:51.355 INFO:tasks.workunit.client.0.smithi006.stdout:0/95: creat db/dc/f21 x:0 0 0 2024-03-22T06:14:51.356 INFO:tasks.workunit.client.0.smithi006.stdout:4/137: getdents d6/d1b/d20/d21 0 2024-03-22T06:14:51.357 INFO:tasks.workunit.client.0.smithi006.stdout:4/138: dread - d6/d1b/d20/f22 zero size 2024-03-22T06:14:51.359 INFO:tasks.workunit.client.0.smithi006.stdout:0/96: mknod db/dc/c22 0 2024-03-22T06:14:51.359 INFO:tasks.workunit.client.0.smithi006.stdout:0/97: stat f4 0 2024-03-22T06:14:51.359 INFO:tasks.workunit.client.0.smithi006.stdout:0/98: dread - db/ff zero size 2024-03-22T06:14:51.361 INFO:tasks.workunit.client.0.smithi006.stdout:4/139: symlink d6/d1b/l2f 0 2024-03-22T06:14:51.361 INFO:tasks.workunit.client.0.smithi006.stdout:4/140: creat d6/f30 x:0 0 0 2024-03-22T06:14:51.370 INFO:tasks.workunit.client.0.smithi006.stdout:7/139: rename d5/db to d5/d1e/d29 0 2024-03-22T06:14:51.375 INFO:tasks.workunit.client.0.smithi006.stdout:6/172: rename d6/fe to d6/d1d/d32/f41 0 2024-03-22T06:14:51.375 INFO:tasks.workunit.client.0.smithi006.stdout:6/173: readlink d6/d1d/l2e 0 2024-03-22T06:14:51.382 INFO:tasks.workunit.client.0.smithi006.stdout:8/116: rename d9/db/f25 to d9/db/d12/d1e/f30 0 2024-03-22T06:14:51.385 INFO:tasks.workunit.client.0.smithi006.stdout:6/174: link d6/f13 d6/f42 0 2024-03-22T06:14:51.385 INFO:tasks.workunit.client.0.smithi006.stdout:6/175: chown d6/d24/c3a 0 1 2024-03-22T06:14:51.387 INFO:tasks.workunit.client.0.smithi006.stdout:8/117: dread d9/f11 [0,4194304] 0 2024-03-22T06:14:51.388 INFO:tasks.workunit.client.0.smithi006.stdout:5/151: rename d0/ff to d0/db/d11/f2d 0 2024-03-22T06:14:51.393 INFO:tasks.workunit.client.0.smithi006.stdout:6/176: creat d6/d1b/d2c/f43 x:0 0 0 2024-03-22T06:14:51.394 INFO:tasks.workunit.client.0.smithi006.stdout:4/141: dwrite d6/ff [0,4194304] 0 2024-03-22T06:14:51.396 INFO:tasks.workunit.client.0.smithi006.stdout:2/193: dwrite d13/d20/f3d [0,4194304] 0 2024-03-22T06:14:51.397 INFO:tasks.workunit.client.0.smithi006.stdout:8/118: mknod d9/c31 0 2024-03-22T06:14:51.397 INFO:tasks.workunit.client.0.smithi006.stdout:7/140: dwrite d5/f9 [0,4194304] 0 2024-03-22T06:14:51.397 INFO:tasks.workunit.client.0.smithi006.stdout:7/141: stat d5/d1e/c20 0 2024-03-22T06:14:51.401 INFO:tasks.workunit.client.0.smithi006.stdout:5/152: creat d0/f2e x:0 0 0 2024-03-22T06:14:51.401 INFO:tasks.workunit.client.0.smithi006.stdout:5/153: fdatasync d0/f9 0 2024-03-22T06:14:51.402 INFO:tasks.workunit.client.0.smithi006.stdout:7/142: dread f0 [0,4194304] 0 2024-03-22T06:14:51.403 INFO:tasks.workunit.client.0.smithi006.stdout:6/177: mkdir d6/d24/d44 0 2024-03-22T06:14:51.403 INFO:tasks.workunit.client.0.smithi006.stdout:6/178: write d6/f19 [760699,105873] 0 2024-03-22T06:14:51.408 INFO:tasks.workunit.client.0.smithi006.stdout:4/142: creat d6/d1b/d20/d21/d2a/f31 x:0 0 0 2024-03-22T06:14:51.418 INFO:tasks.workunit.client.0.smithi006.stdout:8/119: mknod d9/c32 0 2024-03-22T06:14:51.421 INFO:tasks.workunit.client.0.smithi006.stdout:0/99: rename f6 to db/d10/f23 0 2024-03-22T06:14:51.424 INFO:tasks.workunit.client.0.smithi006.stdout:5/154: symlink d0/db/l2f 0 2024-03-22T06:14:51.426 INFO:tasks.workunit.client.0.smithi006.stdout:6/179: creat d6/d1d/d32/f45 x:0 0 0 2024-03-22T06:14:51.427 INFO:tasks.workunit.client.0.smithi006.stdout:6/180: write d6/d1d/f26 [600188,42275] 0 2024-03-22T06:14:51.430 INFO:tasks.workunit.client.0.smithi006.stdout:4/143: symlink d6/d1b/d20/d21/d2a/l32 0 2024-03-22T06:14:51.433 INFO:tasks.workunit.client.0.smithi006.stdout:2/194: link f9 d13/d20/f4b 0 2024-03-22T06:14:51.435 INFO:tasks.workunit.client.0.smithi006.stdout:8/120: truncate d9/db/d12/d1e/f30 697057 0 2024-03-22T06:14:51.438 INFO:tasks.workunit.client.0.smithi006.stdout:4/144: write d6/ff [4010487,127700] 0 2024-03-22T06:14:51.439 INFO:tasks.workunit.client.0.smithi006.stdout:5/155: link d0/db/fd d0/db/d16/f30 0 2024-03-22T06:14:51.440 INFO:tasks.workunit.client.0.smithi006.stdout:5/156: creat d0/f31 x:0 0 0 2024-03-22T06:14:51.440 INFO:tasks.workunit.client.0.smithi006.stdout:0/100: dwrite db/d10/f23 [0,4194304] 0 2024-03-22T06:14:51.440 INFO:tasks.workunit.client.0.smithi006.stdout:0/101: dread - db/dc/f21 zero size 2024-03-22T06:14:51.444 INFO:tasks.workunit.client.0.smithi006.stdout:2/195: write d13/d20/f4b [821870,13991] 0 2024-03-22T06:14:51.448 INFO:tasks.workunit.client.0.smithi006.stdout:0/102: dread db/d10/f23 [0,4194304] 0 2024-03-22T06:14:51.453 INFO:tasks.workunit.client.0.smithi006.stdout:5/157: creat d0/db/dc/f32 x:0 0 0 2024-03-22T06:14:51.458 INFO:tasks.workunit.client.0.smithi006.stdout:3/107: fsync d0/d4/f1d 0 2024-03-22T06:14:51.458 INFO:tasks.workunit.client.0.smithi006.stdout:2/196: mknod d13/d20/d26/d19/d28/c4c 0 2024-03-22T06:14:51.460 INFO:tasks.workunit.client.0.smithi006.stdout:4/145: dwrite d6/d13/f17 [0,4194304] 0 2024-03-22T06:14:51.471 INFO:tasks.workunit.client.0.smithi006.stdout:4/146: write d6/f8 [3328291,108104] 0 2024-03-22T06:14:51.471 INFO:tasks.workunit.client.0.smithi006.stdout:4/147: write d6/d1b/d20/f22 [716812,31130] 0 2024-03-22T06:14:51.472 INFO:tasks.workunit.client.0.smithi006.stdout:5/158: symlink d0/db/l33 0 2024-03-22T06:14:51.476 INFO:tasks.workunit.client.0.smithi006.stdout:5/159: write d0/f22 [2274918,47364] 0 2024-03-22T06:14:51.478 INFO:tasks.workunit.client.0.smithi006.stdout:3/108: dwrite d0/d8/f18 [0,4194304] 0 2024-03-22T06:14:51.478 INFO:tasks.workunit.client.0.smithi006.stdout:2/197: dwrite d13/d20/d26/d19/d28/d29/f47 [0,4194304] 0 2024-03-22T06:14:51.482 INFO:tasks.workunit.client.0.smithi006.stdout:5/160: mkdir d0/db/d16/d1c/d34 0 2024-03-22T06:14:51.482 INFO:tasks.workunit.client.0.smithi006.stdout:5/161: chown d0/db/d16/l2a 2 1 2024-03-22T06:14:51.488 INFO:tasks.workunit.client.0.smithi006.stdout:2/198: mkdir d13/d34/d4d 0 2024-03-22T06:14:51.490 INFO:tasks.workunit.client.0.smithi006.stdout:3/109: mknod d0/c1f 0 2024-03-22T06:14:51.493 INFO:tasks.workunit.client.0.smithi006.stdout:2/199: mkdir d13/d34/d4d/d4e 0 2024-03-22T06:14:51.501 INFO:tasks.workunit.client.0.smithi006.stdout:3/110: write d0/d8/f18 [2864230,26677] 0 2024-03-22T06:14:51.502 INFO:tasks.workunit.client.0.smithi006.stdout:2/200: read d13/f21 [3825188,23718] 0 2024-03-22T06:14:51.505 INFO:tasks.workunit.client.0.smithi006.stdout:5/162: dwrite d0/db/d16/f30 [0,4194304] 0 2024-03-22T06:14:51.505 INFO:tasks.workunit.client.0.smithi006.stdout:2/201: symlink d13/d34/l4f 0 2024-03-22T06:14:51.505 INFO:tasks.workunit.client.0.smithi006.stdout:2/202: chown d13/d34 1257096 1 2024-03-22T06:14:51.510 INFO:tasks.workunit.client.0.smithi006.stdout:3/111: dread d0/d4/fb [0,4194304] 0 2024-03-22T06:14:51.518 INFO:tasks.workunit.client.0.smithi006.stdout:2/203: rename f3 to d13/d20/d26/d19/f50 0 2024-03-22T06:14:51.524 INFO:tasks.workunit.client.0.smithi006.stdout:5/163: dwrite d0/db/dc/d14/f2b [0,4194304] 0 2024-03-22T06:14:51.534 INFO:tasks.workunit.client.0.smithi006.stdout:3/112: rename d0/d8/cf to d0/c20 0 2024-03-22T06:14:51.534 INFO:tasks.workunit.client.0.smithi006.stdout:2/204: dwrite d13/d20/d26/d19/d28/d29/f47 [0,4194304] 0 2024-03-22T06:14:51.534 INFO:tasks.workunit.client.0.smithi006.stdout:2/205: write d13/d20/d26/f15 [1326584,104441] 0 2024-03-22T06:14:51.534 INFO:tasks.workunit.client.0.smithi006.stdout:2/206: creat d13/d20/d26/d19/f51 x:0 0 0 2024-03-22T06:14:51.537 INFO:tasks.workunit.client.0.smithi006.stdout:3/113: rename d0/cc to d0/d4/d10/c21 0 2024-03-22T06:14:51.542 INFO:tasks.workunit.client.0.smithi006.stdout:5/164: dwrite d0/f1a [0,4194304] 0 2024-03-22T06:14:51.543 INFO:tasks.workunit.client.0.smithi006.stdout:2/207: creat d13/d20/f52 x:0 0 0 2024-03-22T06:14:51.551 INFO:tasks.workunit.client.0.smithi006.stdout:5/165: read d0/db/dc/d14/f20 [3411933,75371] 0 2024-03-22T06:14:51.551 INFO:tasks.workunit.client.0.smithi006.stdout:1/63: sync 2024-03-22T06:14:51.557 INFO:tasks.workunit.client.0.smithi006.stdout:3/114: mknod d0/d8/c22 0 2024-03-22T06:14:51.558 INFO:tasks.workunit.client.0.smithi006.stdout:5/166: mkdir d0/db/d16/d1c/d34/d35 0 2024-03-22T06:14:51.558 INFO:tasks.workunit.client.0.smithi006.stdout:5/167: fdatasync d0/f1a 0 2024-03-22T06:14:51.560 INFO:tasks.workunit.client.0.smithi006.stdout:8/121: getdents d9 0 2024-03-22T06:14:51.560 INFO:tasks.workunit.client.0.smithi006.stdout:8/122: creat d9/db/d12/f33 x:0 0 0 2024-03-22T06:14:51.561 INFO:tasks.workunit.client.0.smithi006.stdout:1/64: symlink d8/d9/l12 0 2024-03-22T06:14:51.571 INFO:tasks.workunit.client.0.smithi006.stdout:4/148: rmdir d6/d1b/d20 39 2024-03-22T06:14:51.572 INFO:tasks.workunit.client.0.smithi006.stdout:1/65: mkdir d8/d9/d13 0 2024-03-22T06:14:51.573 INFO:tasks.workunit.client.0.smithi006.stdout:4/149: write d6/f7 [667287,48491] 0 2024-03-22T06:14:51.573 INFO:tasks.workunit.client.0.smithi006.stdout:4/150: fdatasync d6/f8 0 2024-03-22T06:14:51.574 INFO:tasks.workunit.client.0.smithi006.stdout:3/115: mkdir d0/d23 0 2024-03-22T06:14:51.579 INFO:tasks.workunit.client.0.smithi006.stdout:1/66: mkdir d8/d9/d10/d14 0 2024-03-22T06:14:51.582 INFO:tasks.workunit.client.0.smithi006.stdout:4/151: rmdir d6/d1b/d20/d21/d26 0 2024-03-22T06:14:51.584 INFO:tasks.workunit.client.0.smithi006.stdout:3/116: mkdir d0/d8/d9/d24 0 2024-03-22T06:14:51.590 INFO:tasks.workunit.client.0.smithi006.stdout:4/152: dread f1 [0,4194304] 0 2024-03-22T06:14:51.590 INFO:tasks.workunit.client.0.smithi006.stdout:4/153: write d6/f30 [769553,122678] 0 2024-03-22T06:14:51.594 INFO:tasks.workunit.client.0.smithi006.stdout:5/168: getdents d0/db/dc 0 2024-03-22T06:14:51.595 INFO:tasks.workunit.client.0.smithi006.stdout:1/67: mknod d8/c15 0 2024-03-22T06:14:51.595 INFO:tasks.workunit.client.0.smithi006.stdout:1/68: fdatasync f3 0 2024-03-22T06:14:51.597 INFO:tasks.workunit.client.0.smithi006.stdout:0/103: write db/d10/f23 [4347534,45286] 0 2024-03-22T06:14:51.597 INFO:tasks.workunit.client.0.smithi006.stdout:4/154: creat d6/d1b/d20/f33 x:0 0 0 2024-03-22T06:14:51.599 INFO:tasks.workunit.client.0.smithi006.stdout:1/69: link d8/d9/l12 d8/d9/l16 0 2024-03-22T06:14:51.602 INFO:tasks.workunit.client.0.smithi006.stdout:3/117: dwrite d0/d4/f1d [0,4194304] 0 2024-03-22T06:14:51.604 INFO:tasks.workunit.client.0.smithi006.stdout:5/169: write d0/f1a [1705889,49193] 0 2024-03-22T06:14:51.605 INFO:tasks.workunit.client.0.smithi006.stdout:0/104: unlink f4 0 2024-03-22T06:14:51.605 INFO:tasks.workunit.client.0.smithi006.stdout:0/105: write db/dc/f21 [567922,117307] 0 2024-03-22T06:14:51.607 INFO:tasks.workunit.client.0.smithi006.stdout:4/155: mknod d6/d1b/c34 0 2024-03-22T06:14:51.611 INFO:tasks.workunit.client.0.smithi006.stdout:2/208: rename d13/d20/d26/d19/d28 to d13/d53 0 2024-03-22T06:14:51.616 INFO:tasks.workunit.client.0.smithi006.stdout:5/170: write d0/f5 [3557090,6745] 0 2024-03-22T06:14:51.616 INFO:tasks.workunit.client.0.smithi006.stdout:5/171: chown d0/db/d16/f21 1 1 2024-03-22T06:14:51.618 INFO:tasks.workunit.client.0.smithi006.stdout:1/70: dwrite d8/d9/fc [0,4194304] 0 2024-03-22T06:14:51.622 INFO:tasks.workunit.client.0.smithi006.stdout:1/71: dread d8/d9/fd [0,4194304] 0 2024-03-22T06:14:51.622 INFO:tasks.workunit.client.0.smithi006.stdout:1/72: write d8/d9/fc [4882119,107890] 0 2024-03-22T06:14:51.622 INFO:tasks.workunit.client.0.smithi006.stdout:3/118: creat d0/d23/f25 x:0 0 0 2024-03-22T06:14:51.624 INFO:tasks.workunit.client.0.smithi006.stdout:0/106: chown db/d10/l18 21 1 2024-03-22T06:14:51.626 INFO:tasks.workunit.client.0.smithi006.stdout:4/156: dwrite d6/d1b/d20/f24 [0,4194304] 0 2024-03-22T06:14:51.628 INFO:tasks.workunit.client.0.smithi006.stdout:8/123: rename d9/f2a to d9/d14/f34 0 2024-03-22T06:14:51.631 INFO:tasks.workunit.client.0.smithi006.stdout:1/73: dread f3 [0,4194304] 0 2024-03-22T06:14:51.635 INFO:tasks.workunit.client.0.smithi006.stdout:5/172: link d0/f9 d0/db/d16/d1c/d34/d35/f36 0 2024-03-22T06:14:51.635 INFO:tasks.workunit.client.0.smithi006.stdout:5/173: dread - d0/db/dc/f32 zero size 2024-03-22T06:14:51.638 INFO:tasks.workunit.client.0.smithi006.stdout:0/107: dwrite db/ff [0,4194304] 0 2024-03-22T06:14:51.647 INFO:tasks.workunit.client.0.smithi006.stdout:2/209: rename d13/d20/l25 to d13/d34/d4d/l54 0 2024-03-22T06:14:51.647 INFO:tasks.workunit.client.0.smithi006.stdout:3/119: rename d0 to d0/d23/d26 22 2024-03-22T06:14:51.647 INFO:tasks.workunit.client.0.smithi006.stdout:2/210: chown ld 1636732 1 2024-03-22T06:14:51.647 INFO:tasks.workunit.client.0.smithi006.stdout:2/211: truncate d13/d20/f52 899028 0 2024-03-22T06:14:51.647 INFO:tasks.workunit.client.0.smithi006.stdout:2/212: read - d13/d53/f4a zero size 2024-03-22T06:14:51.648 INFO:tasks.workunit.client.0.smithi006.stdout:2/213: write d13/d20/f42 [511876,9877] 0 2024-03-22T06:14:51.667 INFO:tasks.workunit.client.0.smithi006.stdout:1/74: rename l0 to d8/d9/l17 0 2024-03-22T06:14:51.669 INFO:tasks.workunit.client.0.smithi006.stdout:3/120: mkdir d0/d23/d27 0 2024-03-22T06:14:51.670 INFO:tasks.workunit.client.0.smithi006.stdout:3/121: dread - d0/d8/f1e zero size 2024-03-22T06:14:51.672 INFO:tasks.workunit.client.0.smithi006.stdout:2/214: symlink d13/d53/d29/l55 0 2024-03-22T06:14:51.674 INFO:tasks.workunit.client.0.smithi006.stdout:5/174: getdents d0/db/dc/d14 0 2024-03-22T06:14:51.676 INFO:tasks.workunit.client.0.smithi006.stdout:0/108: getdents db/dc 0 2024-03-22T06:14:51.679 INFO:tasks.workunit.client.0.smithi006.stdout:5/175: dread d0/db/f29 [0,4194304] 0 2024-03-22T06:14:51.679 INFO:tasks.workunit.client.0.smithi006.stdout:5/176: chown d0/db/d16/d1c/d34/d35 11056 1 2024-03-22T06:14:51.680 INFO:tasks.workunit.client.0.smithi006.stdout:1/75: mkdir d8/d9/d10/d14/d18 0 2024-03-22T06:14:51.682 INFO:tasks.workunit.client.0.smithi006.stdout:3/122: mknod d0/d8/d1c/c28 0 2024-03-22T06:14:51.688 INFO:tasks.workunit.client.0.smithi006.stdout:0/109: symlink db/l24 0 2024-03-22T06:14:51.692 INFO:tasks.workunit.client.0.smithi006.stdout:1/76: creat d8/d9/d10/f19 x:0 0 0 2024-03-22T06:14:51.696 INFO:tasks.workunit.client.0.smithi006.stdout:0/110: write db/ff [1196264,34115] 0 2024-03-22T06:14:51.700 INFO:tasks.workunit.client.0.smithi006.stdout:5/177: dwrite d0/db/d16/d1c/d34/d35/f36 [0,4194304] 0 2024-03-22T06:14:51.701 INFO:tasks.workunit.client.0.smithi006.stdout:5/178: fdatasync d0/db/dc/d14/f15 0 2024-03-22T06:14:51.702 INFO:tasks.workunit.client.0.smithi006.stdout:2/215: truncate d13/d53/d29/f47 874368 0 2024-03-22T06:14:51.702 INFO:tasks.workunit.client.0.smithi006.stdout:2/216: dread - d13/d53/f4a zero size 2024-03-22T06:14:51.704 INFO:tasks.workunit.client.0.smithi006.stdout:1/77: mknod d8/d9/d10/d14/c1a 0 2024-03-22T06:14:51.708 INFO:tasks.workunit.client.0.smithi006.stdout:3/123: dwrite d0/d8/f1a [0,4194304] 0 2024-03-22T06:14:51.715 INFO:tasks.workunit.client.0.smithi006.stdout:2/217: mknod d13/d53/c56 0 2024-03-22T06:14:51.717 INFO:tasks.workunit.client.0.smithi006.stdout:8/124: rmdir d9/db/d12 39 2024-03-22T06:14:51.717 INFO:tasks.workunit.client.0.smithi006.stdout:8/125: write d9/f27 [468308,102709] 0 2024-03-22T06:14:51.718 INFO:tasks.workunit.client.0.smithi006.stdout:8/126: write d9/fa [1296829,10032] 0 2024-03-22T06:14:51.726 INFO:tasks.workunit.client.0.smithi006.stdout:3/124: mknod d0/d8/d9/d24/c29 0 2024-03-22T06:14:51.733 INFO:tasks.workunit.client.0.smithi006.stdout:0/111: dwrite db/d10/f23 [0,4194304] 0 2024-03-22T06:14:51.735 INFO:tasks.workunit.client.0.smithi006.stdout:8/127: mkdir d9/db/d35 0 2024-03-22T06:14:51.735 INFO:tasks.workunit.client.0.smithi006.stdout:5/179: dwrite d0/f9 [4194304,4194304] 0 2024-03-22T06:14:51.735 INFO:tasks.workunit.client.0.smithi006.stdout:1/78: dwrite d8/d9/d10/f19 [0,4194304] 0 2024-03-22T06:14:51.735 INFO:tasks.workunit.client.0.smithi006.stdout:1/79: stat d8/d9/d13 0 2024-03-22T06:14:51.742 INFO:tasks.workunit.client.0.smithi006.stdout:8/128: mknod d9/db/dc/c36 0 2024-03-22T06:14:51.743 INFO:tasks.workunit.client.0.smithi006.stdout:5/180: creat d0/db/d16/d1c/d34/d35/f37 x:0 0 0 2024-03-22T06:14:51.745 INFO:tasks.workunit.client.0.smithi006.stdout:2/218: dwrite d13/f23 [0,4194304] 0 2024-03-22T06:14:51.745 INFO:tasks.workunit.client.0.smithi006.stdout:3/125: dwrite d0/f3 [4194304,4194304] 0 2024-03-22T06:14:51.745 INFO:tasks.workunit.client.0.smithi006.stdout:3/126: readlink d0/d8/d9/l1b 0 2024-03-22T06:14:51.750 INFO:tasks.workunit.client.0.smithi006.stdout:1/80: rename d8/d9/fc to d8/d9/d10/d14/d18/f1b 0 2024-03-22T06:14:51.753 INFO:tasks.workunit.client.0.smithi006.stdout:0/112: mknod db/d10/c25 0 2024-03-22T06:14:51.754 INFO:tasks.workunit.client.0.smithi006.stdout:3/127: write d0/d8/f1a [3700856,85663] 0 2024-03-22T06:14:51.756 INFO:tasks.workunit.client.0.smithi006.stdout:9/124: sync 2024-03-22T06:14:51.756 INFO:tasks.workunit.client.0.smithi006.stdout:7/143: sync 2024-03-22T06:14:51.756 INFO:tasks.workunit.client.0.smithi006.stdout:7/144: chown d5/d1e/d29/d10/f11 1224745 1 2024-03-22T06:14:51.756 INFO:tasks.workunit.client.0.smithi006.stdout:5/181: mknod d0/db/d16/d1c/d34/c38 0 2024-03-22T06:14:51.757 INFO:tasks.workunit.client.0.smithi006.stdout:6/181: sync 2024-03-22T06:14:51.757 INFO:tasks.workunit.client.0.smithi006.stdout:6/182: fdatasync d6/fc 0 2024-03-22T06:14:51.764 INFO:tasks.workunit.client.0.smithi006.stdout:1/81: stat l7 0 2024-03-22T06:14:51.767 INFO:tasks.workunit.client.0.smithi006.stdout:2/219: rename f9 to d13/d20/f57 0 2024-03-22T06:14:51.770 INFO:tasks.workunit.client.0.smithi006.stdout:3/128: mkdir d0/d23/d2a 0 2024-03-22T06:14:51.770 INFO:tasks.workunit.client.0.smithi006.stdout:3/129: truncate d0/d4/fb 4389147 0 2024-03-22T06:14:51.771 INFO:tasks.workunit.client.0.smithi006.stdout:7/145: dwrite d5/fe [0,4194304] 0 2024-03-22T06:14:51.774 INFO:tasks.workunit.client.0.smithi006.stdout:8/129: mkdir d9/db/d37 0 2024-03-22T06:14:51.775 INFO:tasks.workunit.client.0.smithi006.stdout:1/82: dread d8/d9/d10/f19 [0,4194304] 0 2024-03-22T06:14:51.775 INFO:tasks.workunit.client.0.smithi006.stdout:1/83: chown d8/d9/l17 13564781 1 2024-03-22T06:14:51.776 INFO:tasks.workunit.client.0.smithi006.stdout:9/125: symlink d8/d10/l30 0 2024-03-22T06:14:51.776 INFO:tasks.workunit.client.0.smithi006.stdout:9/126: readlink d8/d10/d28/l26 0 2024-03-22T06:14:51.785 INFO:tasks.workunit.client.0.smithi006.stdout:5/182: rename d0/db/dc/d14/f15 to d0/f39 0 2024-03-22T06:14:51.785 INFO:tasks.workunit.client.0.smithi006.stdout:5/183: write d0/f1a [136378,51169] 0 2024-03-22T06:14:51.800 INFO:tasks.workunit.client.0.smithi006.stdout:3/130: dwrite d0/d23/f25 [0,4194304] 0 2024-03-22T06:14:51.800 INFO:tasks.workunit.client.0.smithi006.stdout:3/131: write d0/d4/fb [5182611,26383] 0 2024-03-22T06:14:51.800 INFO:tasks.workunit.client.0.smithi006.stdout:3/132: write d0/d8/f18 [5509185,86636] 0 2024-03-22T06:14:51.803 INFO:tasks.workunit.client.0.smithi006.stdout:1/84: dwrite f3 [4194304,4194304] 0 2024-03-22T06:14:51.804 INFO:tasks.workunit.client.0.smithi006.stdout:1/85: read d8/d9/fd [14410,63680] 0 2024-03-22T06:14:51.804 INFO:tasks.workunit.client.0.smithi006.stdout:2/220: creat d13/d34/f58 x:0 0 0 2024-03-22T06:14:51.804 INFO:tasks.workunit.client.0.smithi006.stdout:2/221: write d13/d34/f58 [987178,68542] 0 2024-03-22T06:14:51.804 INFO:tasks.workunit.client.0.smithi006.stdout:2/222: readlink d13/l43 0 2024-03-22T06:14:51.804 INFO:tasks.workunit.client.0.smithi006.stdout:2/223: creat d13/d20/d26/d19/f59 x:0 0 0 2024-03-22T06:14:51.804 INFO:tasks.workunit.client.0.smithi006.stdout:2/224: creat d13/d20/f5a x:0 0 0 2024-03-22T06:14:51.807 INFO:tasks.workunit.client.0.smithi006.stdout:8/130: dwrite d9/db/dc/fd [0,4194304] 0 2024-03-22T06:14:51.807 INFO:tasks.workunit.client.0.smithi006.stdout:8/131: fsync f5 0 2024-03-22T06:14:51.807 INFO:tasks.workunit.client.0.smithi006.stdout:0/113: rename db/d10/l18 to db/d10/l26 0 2024-03-22T06:14:51.808 INFO:tasks.workunit.client.0.smithi006.stdout:1/86: write d8/d9/d10/f19 [781754,18915] 0 2024-03-22T06:14:51.809 INFO:tasks.workunit.client.0.smithi006.stdout:7/146: creat d5/d1e/d29/f2a x:0 0 0 2024-03-22T06:14:51.809 INFO:tasks.workunit.client.0.smithi006.stdout:7/147: fsync d5/d1e/f21 0 2024-03-22T06:14:51.812 INFO:tasks.workunit.client.0.smithi006.stdout:6/183: getdents d6/d24 0 2024-03-22T06:14:51.819 INFO:tasks.workunit.client.0.smithi006.stdout:3/133: mknod d0/d23/d27/c2b 0 2024-03-22T06:14:51.828 INFO:tasks.workunit.client.0.smithi006.stdout:2/225: getdents d13/d34/d4d 0 2024-03-22T06:14:51.843 INFO:tasks.workunit.client.0.smithi006.stdout:8/132: symlink d9/db/dc/l38 0 2024-03-22T06:14:51.850 INFO:tasks.workunit.client.0.smithi006.stdout:1/87: rename l2 to d8/l1c 0 2024-03-22T06:14:51.851 INFO:tasks.workunit.client.0.smithi006.stdout:1/88: readlink d8/le 0 2024-03-22T06:14:51.851 INFO:tasks.workunit.client.0.smithi006.stdout:6/184: creat d6/d24/d44/f46 x:0 0 0 2024-03-22T06:14:51.852 INFO:tasks.workunit.client.0.smithi006.stdout:3/134: creat d0/d8/d1c/f2c x:0 0 0 2024-03-22T06:14:51.852 INFO:tasks.workunit.client.0.smithi006.stdout:3/135: chown d0/c20 6 1 2024-03-22T06:14:51.852 INFO:tasks.workunit.client.0.smithi006.stdout:3/136: readlink d0/d8/d9/l11 0 2024-03-22T06:14:51.853 INFO:tasks.workunit.client.0.smithi006.stdout:1/89: dread d8/d9/d10/f19 [0,4194304] 0 2024-03-22T06:14:51.854 INFO:tasks.workunit.client.0.smithi006.stdout:2/226: link d13/d20/d26/d2a/l41 d13/d20/d26/l5b 0 2024-03-22T06:14:51.855 INFO:tasks.workunit.client.0.smithi006.stdout:6/185: write d6/d1b/f23 [2137185,17313] 0 2024-03-22T06:14:51.855 INFO:tasks.workunit.client.0.smithi006.stdout:6/186: write d6/f19 [1130451,109386] 0 2024-03-22T06:14:51.856 INFO:tasks.workunit.client.0.smithi006.stdout:8/133: creat d9/db/d12/d1e/f39 x:0 0 0 2024-03-22T06:14:51.858 INFO:tasks.workunit.client.0.smithi006.stdout:0/114: truncate db/dc/f21 416454 0 2024-03-22T06:14:51.860 INFO:tasks.workunit.client.0.smithi006.stdout:3/137: unlink d0/d8/f18 0 2024-03-22T06:14:51.860 INFO:tasks.workunit.client.0.smithi006.stdout:5/184: fsync d0/db/d16/d1c/d34/d35/f37 0 2024-03-22T06:14:51.862 INFO:tasks.workunit.client.0.smithi006.stdout:1/90: mknod d8/d9/d10/c1d 0 2024-03-22T06:14:51.863 INFO:tasks.workunit.client.0.smithi006.stdout:8/134: write d9/db/dc/de/f17 [1510507,62701] 0 2024-03-22T06:14:51.863 INFO:tasks.workunit.client.0.smithi006.stdout:8/135: dread - d9/db/d12/d1e/f39 zero size 2024-03-22T06:14:51.863 INFO:tasks.workunit.client.0.smithi006.stdout:8/136: chown d9/c31 117577783 1 2024-03-22T06:14:51.865 INFO:tasks.workunit.client.0.smithi006.stdout:2/227: creat d13/d34/d4d/f5c x:0 0 0 2024-03-22T06:14:51.866 INFO:tasks.workunit.client.0.smithi006.stdout:6/187: symlink d6/d1b/d2c/l47 0 2024-03-22T06:14:51.868 INFO:tasks.workunit.client.0.smithi006.stdout:0/115: mknod db/dc/c27 0 2024-03-22T06:14:51.871 INFO:tasks.workunit.client.0.smithi006.stdout:3/138: stat d0/cd 0 2024-03-22T06:14:51.873 INFO:tasks.workunit.client.0.smithi006.stdout:5/185: unlink d0/f22 0 2024-03-22T06:14:51.877 INFO:tasks.workunit.client.0.smithi006.stdout:8/137: creat d9/db/dc/f3a x:0 0 0 2024-03-22T06:14:51.877 INFO:tasks.workunit.client.0.smithi006.stdout:8/138: fdatasync d9/db/dc/f3a 0 2024-03-22T06:14:51.879 INFO:tasks.workunit.client.0.smithi006.stdout:2/228: truncate d13/d20/f57 1312936 0 2024-03-22T06:14:51.880 INFO:tasks.workunit.client.0.smithi006.stdout:1/91: dwrite d8/d9/d10/f19 [0,4194304] 0 2024-03-22T06:14:51.882 INFO:tasks.workunit.client.0.smithi006.stdout:6/188: rename d6/f19 to d6/d1b/f48 0 2024-03-22T06:14:51.884 INFO:tasks.workunit.client.0.smithi006.stdout:0/116: symlink db/l28 0 2024-03-22T06:14:51.884 INFO:tasks.workunit.client.0.smithi006.stdout:0/117: chown db/dc/l20 1 1 2024-03-22T06:14:51.886 INFO:tasks.workunit.client.0.smithi006.stdout:3/139: symlink d0/d4/d10/l2d 0 2024-03-22T06:14:51.887 INFO:tasks.workunit.client.0.smithi006.stdout:5/186: truncate d0/f5 1720371 0 2024-03-22T06:14:51.889 INFO:tasks.workunit.client.0.smithi006.stdout:8/139: creat d9/db/d37/f3b x:0 0 0 2024-03-22T06:14:51.894 INFO:tasks.workunit.client.0.smithi006.stdout:2/229: dread d13/f18 [0,4194304] 0 2024-03-22T06:14:51.894 INFO:tasks.workunit.client.0.smithi006.stdout:2/230: creat d13/d34/d4d/f5d x:0 0 0 2024-03-22T06:14:51.896 INFO:tasks.workunit.client.0.smithi006.stdout:6/189: mknod d6/d1d/d32/c49 0 2024-03-22T06:14:51.899 INFO:tasks.workunit.client.0.smithi006.stdout:0/118: mknod db/c29 0 2024-03-22T06:14:51.906 INFO:tasks.workunit.client.0.smithi006.stdout:2/231: creat d13/d53/f5e x:0 0 0 2024-03-22T06:14:51.908 INFO:tasks.workunit.client.0.smithi006.stdout:6/190: unlink d6/fc 0 2024-03-22T06:14:51.912 INFO:tasks.workunit.client.0.smithi006.stdout:3/140: getdents d0/d23/d27 0 2024-03-22T06:14:51.912 INFO:tasks.workunit.client.0.smithi006.stdout:5/187: dwrite d0/db/dc/f32 [0,4194304] 0 2024-03-22T06:14:51.912 INFO:tasks.workunit.client.0.smithi006.stdout:8/140: dwrite f2 [0,4194304] 0 2024-03-22T06:14:51.913 INFO:tasks.workunit.client.0.smithi006.stdout:8/141: truncate d9/db/d12/d1e/f30 1202985 0 2024-03-22T06:14:51.913 INFO:tasks.workunit.client.0.smithi006.stdout:8/142: chown d9/db/dc 122 1 2024-03-22T06:14:51.916 INFO:tasks.workunit.client.0.smithi006.stdout:6/191: write d6/f11 [2528631,66740] 0 2024-03-22T06:14:51.916 INFO:tasks.workunit.client.0.smithi006.stdout:6/192: fdatasync d6/d1d/f26 0 2024-03-22T06:14:51.918 INFO:tasks.workunit.client.0.smithi006.stdout:0/119: mkdir db/d10/d2a 0 2024-03-22T06:14:51.933 INFO:tasks.workunit.client.0.smithi006.stdout:6/193: mkdir d6/d1b/d2c/d4a 0 2024-03-22T06:14:51.935 INFO:tasks.workunit.client.0.smithi006.stdout:8/143: dwrite d9/db/f2e [0,4194304] 0 2024-03-22T06:14:51.935 INFO:tasks.workunit.client.0.smithi006.stdout:5/188: dwrite d0/db/d16/f30 [4194304,4194304] 0 2024-03-22T06:14:51.940 INFO:tasks.workunit.client.0.smithi006.stdout:4/157: sync 2024-03-22T06:14:51.940 INFO:tasks.workunit.client.0.smithi006.stdout:4/158: getdents d6/d1b/d20/d2c 0 2024-03-22T06:14:51.940 INFO:tasks.workunit.client.0.smithi006.stdout:4/159: write f3 [2752831,27847] 0 2024-03-22T06:14:51.940 INFO:tasks.workunit.client.0.smithi006.stdout:4/160: fdatasync d6/f9 0 2024-03-22T06:14:51.940 INFO:tasks.workunit.client.0.smithi006.stdout:4/161: read - d6/d1b/d20/f33 zero size 2024-03-22T06:14:51.942 INFO:tasks.workunit.client.0.smithi006.stdout:8/144: dread d9/fa [0,4194304] 0 2024-03-22T06:14:51.944 INFO:tasks.workunit.client.0.smithi006.stdout:4/162: symlink d6/d13/l35 0 2024-03-22T06:14:51.946 INFO:tasks.workunit.client.0.smithi006.stdout:8/145: rename d9/db/d37/f3b to d9/db/d35/f3c 0 2024-03-22T06:14:51.948 INFO:tasks.workunit.client.0.smithi006.stdout:1/92: getdents d8 0 2024-03-22T06:14:51.950 INFO:tasks.workunit.client.0.smithi006.stdout:4/163: mknod d6/d13/c36 0 2024-03-22T06:14:51.953 INFO:tasks.workunit.client.0.smithi006.stdout:8/146: link d9/db/dc/fd d9/db/d12/f3d 0 2024-03-22T06:14:51.961 INFO:tasks.workunit.client.0.smithi006.stdout:0/120: write db/dc/f21 [681506,30695] 0 2024-03-22T06:14:51.966 INFO:tasks.workunit.client.0.smithi006.stdout:0/121: link db/d10/c11 db/dc/c2b 0 2024-03-22T06:14:51.974 INFO:tasks.workunit.client.0.smithi006.stdout:0/122: dread db/d10/f1b [0,4194304] 0 2024-03-22T06:14:51.977 INFO:tasks.workunit.client.0.smithi006.stdout:0/123: creat db/d10/d2a/f2c x:0 0 0 2024-03-22T06:14:51.985 INFO:tasks.workunit.client.0.smithi006.stdout:0/124: dread db/d10/f1f [0,4194304] 0 2024-03-22T06:14:51.985 INFO:tasks.workunit.client.0.smithi006.stdout:0/125: stat db/l12 0 2024-03-22T06:14:51.990 INFO:tasks.workunit.client.0.smithi006.stdout:3/141: rmdir d0/d4 39 2024-03-22T06:14:51.993 INFO:tasks.workunit.client.0.smithi006.stdout:0/126: truncate db/ff 421474 0 2024-03-22T06:14:51.993 INFO:tasks.workunit.client.0.smithi006.stdout:0/127: readlink la 0 2024-03-22T06:14:51.995 INFO:tasks.workunit.client.0.smithi006.stdout:3/142: symlink d0/d8/d9/d24/l2e 0 2024-03-22T06:14:52.003 INFO:tasks.workunit.client.0.smithi006.stdout:4/164: dwrite d6/f12 [0,4194304] 0 2024-03-22T06:14:52.003 INFO:tasks.workunit.client.0.smithi006.stdout:1/93: dwrite d8/d9/d10/f19 [0,4194304] 0 2024-03-22T06:14:52.005 INFO:tasks.workunit.client.0.smithi006.stdout:2/232: getdents d13/d53 0 2024-03-22T06:14:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:51 smithi117 ceph-mon[36514]: pgmap v9: 65 pgs: 65 active+clean; 350 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 25 MiB/s rd, 44 MiB/s wr, 403 op/s 2024-03-22T06:14:52.007 INFO:tasks.workunit.client.0.smithi006.stdout:2/233: dread d13/d53/f32 [0,4194304] 0 2024-03-22T06:14:52.007 INFO:tasks.workunit.client.0.smithi006.stdout:2/234: chown d13/d20/d26/c1f 292246 1 2024-03-22T06:14:52.007 INFO:tasks.workunit.client.0.smithi006.stdout:2/235: fdatasync fb 0 2024-03-22T06:14:52.009 INFO:tasks.workunit.client.0.smithi006.stdout:4/165: creat d6/d1b/d20/d2c/f37 x:0 0 0 2024-03-22T06:14:52.012 INFO:tasks.workunit.client.0.smithi006.stdout:1/94: creat d8/d9/f1e x:0 0 0 2024-03-22T06:14:52.012 INFO:tasks.workunit.client.0.smithi006.stdout:1/95: creat d8/d9/f1f x:0 0 0 2024-03-22T06:14:52.012 INFO:tasks.workunit.client.0.smithi006.stdout:1/96: write d8/d9/fd [527724,13667] 0 2024-03-22T06:14:52.012 INFO:tasks.workunit.client.0.smithi006.stdout:1/97: readlink l4 0 2024-03-22T06:14:52.013 INFO:tasks.workunit.client.0.smithi006.stdout:8/147: dwrite d9/f27 [0,4194304] 0 2024-03-22T06:14:52.015 INFO:tasks.workunit.client.0.smithi006.stdout:3/143: dread d0/d4/fb [0,4194304] 0 2024-03-22T06:14:52.015 INFO:tasks.workunit.client.0.smithi006.stdout:0/128: dwrite db/d10/f14 [0,4194304] 0 2024-03-22T06:14:52.016 INFO:tasks.workunit.client.0.smithi006.stdout:0/129: chown db/l24 2587 1 2024-03-22T06:14:52.016 INFO:tasks.workunit.client.0.smithi006.stdout:4/166: link d6/l18 d6/d1b/d20/d2c/l38 0 2024-03-22T06:14:52.021 INFO:tasks.workunit.client.0.smithi006.stdout:1/98: rename f3 to d8/d9/d10/d14/f20 0 2024-03-22T06:14:52.028 INFO:tasks.workunit.client.0.smithi006.stdout:8/148: mkdir d9/db/dc/de/d3e 0 2024-03-22T06:14:52.029 INFO:tasks.workunit.client.0.smithi006.stdout:8/149: read - d9/d14/f2f zero size 2024-03-22T06:14:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:51 smithi006 ceph-mon[29301]: pgmap v9: 65 pgs: 65 active+clean; 350 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 25 MiB/s rd, 44 MiB/s wr, 403 op/s 2024-03-22T06:14:52.034 INFO:tasks.workunit.client.0.smithi006.stdout:0/130: mknod db/c2d 0 2024-03-22T06:14:52.039 INFO:tasks.workunit.client.0.smithi006.stdout:3/144: dwrite d0/d8/f1a [0,4194304] 0 2024-03-22T06:14:52.040 INFO:tasks.workunit.client.0.smithi006.stdout:8/150: creat d9/db/dc/de/d3e/f3f x:0 0 0 2024-03-22T06:14:52.040 INFO:tasks.workunit.client.0.smithi006.stdout:8/151: creat d9/db/dc/de/d3e/f40 x:0 0 0 2024-03-22T06:14:52.040 INFO:tasks.workunit.client.0.smithi006.stdout:8/152: creat d9/db/d37/f41 x:0 0 0 2024-03-22T06:14:52.040 INFO:tasks.workunit.client.0.smithi006.stdout:8/153: write d9/db/dc/de/f16 [3914813,85792] 0 2024-03-22T06:14:52.056 INFO:tasks.workunit.client.0.smithi006.stdout:8/154: dwrite d9/fa [4194304,4194304] 0 2024-03-22T06:14:52.057 INFO:tasks.workunit.client.0.smithi006.stdout:3/145: rmdir d0/d23 39 2024-03-22T06:14:52.061 INFO:tasks.workunit.client.0.smithi006.stdout:0/131: link db/d10/c17 db/d10/c2e 0 2024-03-22T06:14:52.065 INFO:tasks.workunit.client.0.smithi006.stdout:8/155: rename d9/db/dc/de/f29 to d9/d14/f42 0 2024-03-22T06:14:52.067 INFO:tasks.workunit.client.0.smithi006.stdout:3/146: link d0/d23/f25 d0/d4/f2f 0 2024-03-22T06:14:52.069 INFO:tasks.workunit.client.0.smithi006.stdout:0/132: creat db/d10/f2f x:0 0 0 2024-03-22T06:14:52.069 INFO:tasks.workunit.client.0.smithi006.stdout:0/133: creat db/f30 x:0 0 0 2024-03-22T06:14:52.071 INFO:tasks.workunit.client.0.smithi006.stdout:8/156: creat d9/db/d12/f43 x:0 0 0 2024-03-22T06:14:52.073 INFO:tasks.workunit.client.0.smithi006.stdout:3/147: creat d0/d4/d10/f30 x:0 0 0 2024-03-22T06:14:52.073 INFO:tasks.workunit.client.0.smithi006.stdout:3/148: chown d0/d4 2511 1 2024-03-22T06:14:52.085 INFO:tasks.workunit.client.0.smithi006.stdout:0/134: unlink db/ff 0 2024-03-22T06:14:52.089 INFO:tasks.workunit.client.0.smithi006.stdout:3/149: dwrite d0/d4/d10/f30 [0,4194304] 0 2024-03-22T06:14:52.095 INFO:tasks.workunit.client.0.smithi006.stdout:3/150: mkdir d0/d4/d31 0 2024-03-22T06:14:52.096 INFO:tasks.workunit.client.0.smithi006.stdout:0/135: mknod db/d10/c31 0 2024-03-22T06:14:52.097 INFO:tasks.workunit.client.0.smithi006.stdout:4/167: rmdir d6/d1b 39 2024-03-22T06:14:52.098 INFO:tasks.workunit.client.0.smithi006.stdout:4/168: fdatasync d6/f7 0 2024-03-22T06:14:52.098 INFO:tasks.workunit.client.0.smithi006.stdout:3/151: creat d0/d8/d9/f32 x:0 0 0 2024-03-22T06:14:52.099 INFO:tasks.workunit.client.0.smithi006.stdout:1/99: fsync d8/d9/f1f 0 2024-03-22T06:14:52.106 INFO:tasks.workunit.client.0.smithi006.stdout:1/100: mknod d8/d9/d10/d14/d18/c21 0 2024-03-22T06:14:52.116 INFO:tasks.workunit.client.0.smithi006.stdout:1/101: dread d8/d9/d10/d14/d18/f1b [0,4194304] 0 2024-03-22T06:14:52.117 INFO:tasks.workunit.client.0.smithi006.stdout:3/152: dwrite d0/d8/d9/f32 [0,4194304] 0 2024-03-22T06:14:52.118 INFO:tasks.workunit.client.0.smithi006.stdout:1/102: stat d8/d9/lb 0 2024-03-22T06:14:52.120 INFO:tasks.workunit.client.0.smithi006.stdout:4/169: dwrite f2 [0,4194304] 0 2024-03-22T06:14:52.121 INFO:tasks.workunit.client.0.smithi006.stdout:3/153: mkdir d0/d23/d33 0 2024-03-22T06:14:52.123 INFO:tasks.workunit.client.0.smithi006.stdout:1/103: read d8/d9/d10/d14/d18/f1b [585555,7018] 0 2024-03-22T06:14:52.129 INFO:tasks.workunit.client.0.smithi006.stdout:4/170: unlink d6/d1b/d20/d21/d2a/f31 0 2024-03-22T06:14:52.133 INFO:tasks.workunit.client.0.smithi006.stdout:4/171: symlink d6/d1b/d20/l39 0 2024-03-22T06:14:52.133 INFO:tasks.workunit.client.0.smithi006.stdout:4/172: creat d6/f3a x:0 0 0 2024-03-22T06:14:52.133 INFO:tasks.workunit.client.0.smithi006.stdout:4/173: fsync d6/d1b/d20/d2c/f37 0 2024-03-22T06:14:52.133 INFO:tasks.workunit.client.0.smithi006.stdout:4/174: chown d6/f11 361 1 2024-03-22T06:14:52.136 INFO:tasks.workunit.client.0.smithi006.stdout:4/175: mkdir d6/d13/d3b 0 2024-03-22T06:14:52.137 INFO:tasks.workunit.client.0.smithi006.stdout:4/176: chown d6/l14 0 1 2024-03-22T06:14:52.139 INFO:tasks.workunit.client.0.smithi006.stdout:8/157: rmdir d9/db/dc 39 2024-03-22T06:14:52.142 INFO:tasks.workunit.client.0.smithi006.stdout:1/104: dwrite d8/d9/f1f [0,4194304] 0 2024-03-22T06:14:52.143 INFO:tasks.workunit.client.0.smithi006.stdout:3/154: dwrite d0/f3 [4194304,4194304] 0 2024-03-22T06:14:52.143 INFO:tasks.workunit.client.0.smithi006.stdout:3/155: write d0/fa [246319,128520] 0 2024-03-22T06:14:52.150 INFO:tasks.workunit.client.0.smithi006.stdout:1/105: mkdir d8/d9/d10/d14/d22 0 2024-03-22T06:14:52.155 INFO:tasks.workunit.client.0.smithi006.stdout:1/106: mknod d8/d9/c23 0 2024-03-22T06:14:52.157 INFO:tasks.workunit.client.0.smithi006.stdout:1/107: mkdir d8/d9/d13/d24 0 2024-03-22T06:14:52.165 INFO:tasks.workunit.client.0.smithi006.stdout:4/177: dwrite d6/f30 [0,4194304] 0 2024-03-22T06:14:52.165 INFO:tasks.workunit.client.0.smithi006.stdout:4/178: rename d6/d1b/d20/d21/d2a to d6/d1b/d20/d21/d2a/d3c 22 2024-03-22T06:14:52.168 INFO:tasks.workunit.client.0.smithi006.stdout:8/158: dwrite d9/f11 [0,4194304] 0 2024-03-22T06:14:52.169 INFO:tasks.workunit.client.0.smithi006.stdout:3/156: dwrite d0/d23/f25 [0,4194304] 0 2024-03-22T06:14:52.169 INFO:tasks.workunit.client.0.smithi006.stdout:3/157: getdents d0/d23/d33 0 2024-03-22T06:14:52.169 INFO:tasks.workunit.client.0.smithi006.stdout:0/136: rmdir db 39 2024-03-22T06:14:52.169 INFO:tasks.workunit.client.0.smithi006.stdout:4/179: dread d6/f7 [0,4194304] 0 2024-03-22T06:14:52.179 INFO:tasks.workunit.client.0.smithi006.stdout:0/137: symlink db/l32 0 2024-03-22T06:14:52.180 INFO:tasks.workunit.client.0.smithi006.stdout:3/158: read d0/d4/d10/f30 [1137078,19717] 0 2024-03-22T06:14:52.180 INFO:tasks.workunit.client.0.smithi006.stdout:4/180: rename d6/d1b/d20/d2c/f37 to d6/f3d 0 2024-03-22T06:14:52.183 INFO:tasks.workunit.client.0.smithi006.stdout:8/159: rmdir d9/db/d12/d1e 39 2024-03-22T06:14:52.183 INFO:tasks.workunit.client.0.smithi006.stdout:8/160: dread - d9/db/dc/f3a zero size 2024-03-22T06:14:52.189 INFO:tasks.workunit.client.0.smithi006.stdout:8/161: mkdir d9/d14/d44 0 2024-03-22T06:14:52.190 INFO:tasks.workunit.client.0.smithi006.stdout:0/138: mkdir db/d10/d33 0 2024-03-22T06:14:52.193 INFO:tasks.workunit.client.0.smithi006.stdout:8/162: unlink d9/d14/f42 0 2024-03-22T06:14:52.194 INFO:tasks.workunit.client.0.smithi006.stdout:0/139: creat db/d10/d2a/f34 x:0 0 0 2024-03-22T06:14:52.195 INFO:tasks.workunit.client.0.smithi006.stdout:9/127: sync 2024-03-22T06:14:52.199 INFO:tasks.workunit.client.0.smithi006.stdout:4/181: dwrite d6/f12 [4194304,4194304] 0 2024-03-22T06:14:52.199 INFO:tasks.workunit.client.0.smithi006.stdout:3/159: dwrite d0/f3 [4194304,4194304] 0 2024-03-22T06:14:52.200 INFO:tasks.workunit.client.0.smithi006.stdout:9/128: mknod d8/d10/c31 0 2024-03-22T06:14:52.200 INFO:tasks.workunit.client.0.smithi006.stdout:9/129: chown d8/f19 17133062 1 2024-03-22T06:14:52.202 INFO:tasks.workunit.client.0.smithi006.stdout:8/163: rmdir d9/db/d37 39 2024-03-22T06:14:52.204 INFO:tasks.workunit.client.0.smithi006.stdout:0/140: mkdir db/d10/d35 0 2024-03-22T06:14:52.204 INFO:tasks.workunit.client.0.smithi006.stdout:0/141: fsync db/d10/f1a 0 2024-03-22T06:14:52.213 INFO:tasks.workunit.client.0.smithi006.stdout:9/130: mknod d8/d10/d11/c32 0 2024-03-22T06:14:52.214 INFO:tasks.workunit.client.0.smithi006.stdout:3/160: write d0/d8/f1a [4072028,39340] 0 2024-03-22T06:14:52.215 INFO:tasks.workunit.client.0.smithi006.stdout:9/131: symlink d8/d10/d11/d2f/l33 0 2024-03-22T06:14:52.219 INFO:tasks.workunit.client.0.smithi006.stdout:9/132: unlink d8/d10/f22 0 2024-03-22T06:14:52.222 INFO:tasks.workunit.client.0.smithi006.stdout:1/108: getdents d8/d9 0 2024-03-22T06:14:52.224 INFO:tasks.workunit.client.0.smithi006.stdout:1/109: symlink d8/d9/d13/l25 0 2024-03-22T06:14:52.224 INFO:tasks.workunit.client.0.smithi006.stdout:3/161: dread d0/d8/d9/f32 [0,4194304] 0 2024-03-22T06:14:52.226 INFO:tasks.workunit.client.0.smithi006.stdout:4/182: dwrite d6/f30 [0,4194304] 0 2024-03-22T06:14:52.227 INFO:tasks.workunit.client.0.smithi006.stdout:1/110: creat d8/d9/d13/f26 x:0 0 0 2024-03-22T06:14:52.227 INFO:tasks.workunit.client.0.smithi006.stdout:1/111: write d8/d9/f1f [4545551,29114] 0 2024-03-22T06:14:52.228 INFO:tasks.workunit.client.0.smithi006.stdout:3/162: mknod d0/d4/d31/c34 0 2024-03-22T06:14:52.230 INFO:tasks.workunit.client.0.smithi006.stdout:8/164: dwrite d9/db/dc/de/d3e/f40 [0,4194304] 0 2024-03-22T06:14:52.230 INFO:tasks.workunit.client.0.smithi006.stdout:8/165: write d9/db/d37/f41 [72829,24926] 0 2024-03-22T06:14:52.232 INFO:tasks.workunit.client.0.smithi006.stdout:0/142: dwrite db/d10/f2f [0,4194304] 0 2024-03-22T06:14:52.233 INFO:tasks.workunit.client.0.smithi006.stdout:0/143: chown db/f30 351387171 1 2024-03-22T06:14:52.235 INFO:tasks.workunit.client.0.smithi006.stdout:1/112: read d8/d9/d10/d14/f20 [7382759,118287] 0 2024-03-22T06:14:52.236 INFO:tasks.workunit.client.0.smithi006.stdout:3/163: link d0/d23/f25 d0/d4/d10/f35 0 2024-03-22T06:14:52.240 INFO:tasks.workunit.client.0.smithi006.stdout:8/166: dread d9/f27 [0,4194304] 0 2024-03-22T06:14:52.241 INFO:tasks.workunit.client.0.smithi006.stdout:4/183: dwrite d6/f9 [4194304,4194304] 0 2024-03-22T06:14:52.241 INFO:tasks.workunit.client.0.smithi006.stdout:4/184: dread - d6/d1b/f2d zero size 2024-03-22T06:14:52.247 INFO:tasks.workunit.client.0.smithi006.stdout:3/164: rmdir d0/d23/d2a 0 2024-03-22T06:14:52.249 INFO:tasks.workunit.client.0.smithi006.stdout:8/167: rmdir d9/db/dc/de 39 2024-03-22T06:14:52.253 INFO:tasks.workunit.client.0.smithi006.stdout:4/185: symlink d6/d13/d3b/l3e 0 2024-03-22T06:14:52.257 INFO:tasks.workunit.client.0.smithi006.stdout:8/168: dread d9/db/d12/f3d [0,4194304] 0 2024-03-22T06:14:52.257 INFO:tasks.workunit.client.0.smithi006.stdout:3/165: truncate d0/d4/d10/f30 1640612 0 2024-03-22T06:14:52.259 INFO:tasks.workunit.client.0.smithi006.stdout:4/186: creat d6/d1b/d20/d21/d2a/f3f x:0 0 0 2024-03-22T06:14:52.259 INFO:tasks.workunit.client.0.smithi006.stdout:4/187: truncate d6/d1b/f29 271328 0 2024-03-22T06:14:52.260 INFO:tasks.workunit.client.0.smithi006.stdout:1/113: dwrite d8/d9/f1e [0,4194304] 0 2024-03-22T06:14:52.260 INFO:tasks.workunit.client.0.smithi006.stdout:1/114: truncate d8/d9/fd 1175092 0 2024-03-22T06:14:52.260 INFO:tasks.workunit.client.0.smithi006.stdout:1/115: chown d8/d9/d10/d14/d22 162747 1 2024-03-22T06:14:52.261 INFO:tasks.workunit.client.0.smithi006.stdout:8/169: symlink d9/l45 0 2024-03-22T06:14:52.261 INFO:tasks.workunit.client.0.smithi006.stdout:8/170: write d9/db/d12/f33 [957506,72210] 0 2024-03-22T06:14:52.261 INFO:tasks.workunit.client.0.smithi006.stdout:8/171: fdatasync f5 0 2024-03-22T06:14:52.263 INFO:tasks.workunit.client.0.smithi006.stdout:3/166: mkdir d0/d23/d27/d36 0 2024-03-22T06:14:52.266 INFO:tasks.workunit.client.0.smithi006.stdout:4/188: rename d6/d1b/d20/d21/d2a/l32 to d6/d13/d3b/l40 0 2024-03-22T06:14:52.266 INFO:tasks.workunit.client.0.smithi006.stdout:4/189: fsync d6/d1b/f29 0 2024-03-22T06:14:52.272 INFO:tasks.workunit.client.0.smithi006.stdout:1/116: truncate d8/d9/d10/d14/f20 8111064 0 2024-03-22T06:14:52.274 INFO:tasks.workunit.client.0.smithi006.stdout:8/172: mknod d9/db/d37/c46 0 2024-03-22T06:14:52.274 INFO:tasks.workunit.client.0.smithi006.stdout:8/173: stat d9/db/dc/l38 0 2024-03-22T06:14:52.274 INFO:tasks.workunit.client.0.smithi006.stdout:8/174: write f1 [1154262,7802] 0 2024-03-22T06:14:52.275 INFO:tasks.workunit.client.0.smithi006.stdout:8/175: dread d9/db/dc/f1f [0,4194304] 0 2024-03-22T06:14:52.276 INFO:tasks.workunit.client.0.smithi006.stdout:4/190: getdents d6/d13 0 2024-03-22T06:14:52.277 INFO:tasks.workunit.client.0.smithi006.stdout:4/191: chown d6/d1b/f2d 331226 1 2024-03-22T06:14:52.278 INFO:tasks.workunit.client.0.smithi006.stdout:3/167: dwrite d0/d4/f2f [4194304,4194304] 0 2024-03-22T06:14:52.278 INFO:tasks.workunit.client.0.smithi006.stdout:3/168: stat d0/d23/d27 0 2024-03-22T06:14:52.278 INFO:tasks.workunit.client.0.smithi006.stdout:3/169: readlink d0/d8/d9/l11 0 2024-03-22T06:14:52.278 INFO:tasks.workunit.client.0.smithi006.stdout:1/117: truncate d8/d9/d10/f19 314713 0 2024-03-22T06:14:52.281 INFO:tasks.workunit.client.0.smithi006.stdout:4/192: symlink d6/d1b/d20/d21/l41 0 2024-03-22T06:14:52.288 INFO:tasks.workunit.client.0.smithi006.stdout:4/193: truncate f0 2426549 0 2024-03-22T06:14:52.293 INFO:tasks.workunit.client.0.smithi006.stdout:8/176: dwrite d9/db/f20 [0,4194304] 0 2024-03-22T06:14:52.301 INFO:tasks.workunit.client.0.smithi006.stdout:4/194: write d6/f30 [2528371,56337] 0 2024-03-22T06:14:52.303 INFO:tasks.workunit.client.0.smithi006.stdout:1/118: dwrite d8/d9/f1e [0,4194304] 0 2024-03-22T06:14:52.304 INFO:tasks.workunit.client.0.smithi006.stdout:1/119: fsync d8/d9/d10/d14/d18/f1b 0 2024-03-22T06:14:52.319 INFO:tasks.workunit.client.0.smithi006.stdout:4/195: dwrite d6/f1f [0,4194304] 0 2024-03-22T06:14:52.319 INFO:tasks.workunit.client.0.smithi006.stdout:4/196: creat d6/f42 x:0 0 0 2024-03-22T06:14:52.334 INFO:tasks.workunit.client.0.smithi006.stdout:4/197: dwrite d6/f9 [8388608,4194304] 0 2024-03-22T06:14:52.352 INFO:tasks.workunit.client.0.smithi006.stdout:4/198: dwrite d6/f8 [0,4194304] 0 2024-03-22T06:14:52.384 INFO:tasks.workunit.client.0.smithi006.stdout:8/177: mkdir d9/d14/d47 0 2024-03-22T06:14:52.389 INFO:tasks.workunit.client.0.smithi006.stdout:1/120: rename d8/l1c to d8/d9/l27 0 2024-03-22T06:14:52.389 INFO:tasks.workunit.client.0.smithi006.stdout:1/121: stat d8/d9/fd 0 2024-03-22T06:14:52.394 INFO:tasks.workunit.client.0.smithi006.stdout:4/199: mkdir d6/d1b/d20/d21/d2a/d43 0 2024-03-22T06:14:52.396 INFO:tasks.workunit.client.0.smithi006.stdout:8/178: symlink d9/db/d12/l48 0 2024-03-22T06:14:52.396 INFO:tasks.workunit.client.0.smithi006.stdout:8/179: stat d9/db/dc/de/f17 0 2024-03-22T06:14:52.396 INFO:tasks.workunit.client.0.smithi006.stdout:8/180: dread - d9/db/d12/f26 zero size 2024-03-22T06:14:52.399 INFO:tasks.workunit.client.0.smithi006.stdout:4/200: rename d6/d1b/d20/d2c/l38 to d6/d1b/d20/l44 0 2024-03-22T06:14:52.403 INFO:tasks.workunit.client.0.smithi006.stdout:4/201: mkdir d6/d1b/d20/d45 0 2024-03-22T06:14:52.405 INFO:tasks.workunit.client.0.smithi006.stdout:8/181: mknod d9/db/c49 0 2024-03-22T06:14:52.405 INFO:tasks.workunit.client.0.smithi006.stdout:8/182: readlink d9/db/dc/de/l15 0 2024-03-22T06:14:52.406 INFO:tasks.workunit.client.0.smithi006.stdout:4/202: mkdir d6/d1b/d46 0 2024-03-22T06:14:52.408 INFO:tasks.workunit.client.0.smithi006.stdout:8/183: mkdir d9/db/d37/d4a 0 2024-03-22T06:14:52.411 INFO:tasks.workunit.client.0.smithi006.stdout:4/203: creat d6/d1b/f47 x:0 0 0 2024-03-22T06:14:52.411 INFO:tasks.workunit.client.0.smithi006.stdout:4/204: chown d6/d1b/d20/f33 126797714 1 2024-03-22T06:14:52.418 INFO:tasks.workunit.client.0.smithi006.stdout:4/205: write d6/d13/f17 [1376221,57773] 0 2024-03-22T06:14:52.421 INFO:tasks.workunit.client.0.smithi006.stdout:4/206: write f1 [778759,124513] 0 2024-03-22T06:14:52.436 INFO:tasks.workunit.client.0.smithi006.stdout:8/184: getdents d9/d14 0 2024-03-22T06:14:52.436 INFO:tasks.workunit.client.0.smithi006.stdout:8/185: write d9/db/f2e [5177893,81757] 0 2024-03-22T06:14:52.439 INFO:tasks.workunit.client.0.smithi006.stdout:8/186: dread d9/db/dc/fd [0,4194304] 0 2024-03-22T06:14:52.440 INFO:tasks.workunit.client.0.smithi006.stdout:8/187: fdatasync f1 0 2024-03-22T06:14:52.440 INFO:tasks.workunit.client.0.smithi006.stdout:8/188: chown f2 2 1 2024-03-22T06:14:52.444 INFO:tasks.workunit.client.0.smithi006.stdout:8/189: creat d9/db/d37/f4b x:0 0 0 2024-03-22T06:14:52.445 INFO:tasks.workunit.client.0.smithi006.stdout:8/190: read - d9/db/d37/f4b zero size 2024-03-22T06:14:52.445 INFO:tasks.workunit.client.0.smithi006.stdout:8/191: dread - d9/d14/f2f zero size 2024-03-22T06:14:52.447 INFO:tasks.workunit.client.0.smithi006.stdout:8/192: mknod d9/db/d35/c4c 0 2024-03-22T06:14:52.450 INFO:tasks.workunit.client.0.smithi006.stdout:4/207: dwrite f2 [0,4194304] 0 2024-03-22T06:14:52.459 INFO:tasks.workunit.client.0.smithi006.stdout:4/208: mknod d6/d13/c48 0 2024-03-22T06:14:52.477 INFO:tasks.workunit.client.0.smithi006.stdout:4/209: dwrite d6/fb [4194304,4194304] 0 2024-03-22T06:14:52.482 INFO:tasks.workunit.client.0.smithi006.stdout:4/210: dread d6/f30 [0,4194304] 0 2024-03-22T06:14:52.485 INFO:tasks.workunit.client.0.smithi006.stdout:4/211: truncate d6/fa 3430829 0 2024-03-22T06:14:52.487 INFO:tasks.workunit.client.0.smithi006.stdout:4/212: creat d6/d1b/d20/d2c/f49 x:0 0 0 2024-03-22T06:14:52.488 INFO:tasks.workunit.client.0.smithi006.stdout:7/148: sync 2024-03-22T06:14:52.489 INFO:tasks.workunit.client.0.smithi006.stdout:5/189: sync 2024-03-22T06:14:52.489 INFO:tasks.workunit.client.0.smithi006.stdout:6/194: sync 2024-03-22T06:14:52.489 INFO:tasks.workunit.client.0.smithi006.stdout:6/195: creat d6/d1b/d2c/f4b x:0 0 0 2024-03-22T06:14:52.489 INFO:tasks.workunit.client.0.smithi006.stdout:6/196: write d6/d1b/f30 [533709,30665] 0 2024-03-22T06:14:52.489 INFO:tasks.workunit.client.0.smithi006.stdout:6/197: stat d6/d1d 0 2024-03-22T06:14:52.489 INFO:tasks.workunit.client.0.smithi006.stdout:6/198: creat d6/d1b/d2c/f4c x:0 0 0 2024-03-22T06:14:52.490 INFO:tasks.workunit.client.0.smithi006.stdout:4/213: rmdir d6/d1b/d20/d45 0 2024-03-22T06:14:52.490 INFO:tasks.workunit.client.0.smithi006.stdout:4/214: chown d6/l19 701948 1 2024-03-22T06:14:52.490 INFO:tasks.workunit.client.0.smithi006.stdout:4/215: creat d6/f4a x:0 0 0 2024-03-22T06:14:52.490 INFO:tasks.workunit.client.0.smithi006.stdout:7/149: dread f4 [0,4194304] 0 2024-03-22T06:14:52.490 INFO:tasks.workunit.client.0.smithi006.stdout:7/150: write d5/f28 [334312,114991] 0 2024-03-22T06:14:52.499 INFO:tasks.workunit.client.0.smithi006.stdout:4/216: stat d6/d13/d3b/l40 0 2024-03-22T06:14:52.499 INFO:tasks.workunit.client.0.smithi006.stdout:4/217: stat d6/fa 0 2024-03-22T06:14:52.499 INFO:tasks.workunit.client.0.smithi006.stdout:4/218: chown d6/d1b/d20/d21 804064 1 2024-03-22T06:14:52.501 INFO:tasks.workunit.client.0.smithi006.stdout:7/151: unlink d5/d1e/d29/c17 0 2024-03-22T06:14:52.505 INFO:tasks.workunit.client.0.smithi006.stdout:5/190: rmdir d0/db 39 2024-03-22T06:14:52.508 INFO:tasks.workunit.client.0.smithi006.stdout:6/199: getdents d6/d1d 0 2024-03-22T06:14:52.508 INFO:tasks.workunit.client.0.smithi006.stdout:5/191: dread d0/f5 [0,4194304] 0 2024-03-22T06:14:52.508 INFO:tasks.workunit.client.0.smithi006.stdout:5/192: readlink d0/db/l33 0 2024-03-22T06:14:52.508 INFO:tasks.workunit.client.0.smithi006.stdout:6/200: creat d6/d24/d44/f4d x:0 0 0 2024-03-22T06:14:52.510 INFO:tasks.workunit.client.0.smithi006.stdout:4/219: mkdir d6/d1b/d20/d21/d2a/d43/d4b 0 2024-03-22T06:14:52.512 INFO:tasks.workunit.client.0.smithi006.stdout:6/201: dread d6/f11 [0,4194304] 0 2024-03-22T06:14:52.513 INFO:tasks.workunit.client.0.smithi006.stdout:6/202: read - d6/f40 zero size 2024-03-22T06:14:52.517 INFO:tasks.workunit.client.0.smithi006.stdout:5/193: truncate d0/db/dc/d14/f1d 4227779 0 2024-03-22T06:14:52.522 INFO:tasks.workunit.client.0.smithi006.stdout:4/220: rename d6/f3a to d6/d1b/d20/d21/d2a/d43/f4c 0 2024-03-22T06:14:52.528 INFO:tasks.workunit.client.0.smithi006.stdout:6/203: mknod d6/d1d/d32/c4e 0 2024-03-22T06:14:52.528 INFO:tasks.workunit.client.0.smithi006.stdout:6/204: chown d6/l1a 6527 1 2024-03-22T06:14:52.528 INFO:tasks.workunit.client.0.smithi006.stdout:6/205: truncate d6/d1d/d32/f45 377190 0 2024-03-22T06:14:52.528 INFO:tasks.workunit.client.0.smithi006.stdout:6/206: chown d6/c8 0 1 2024-03-22T06:14:52.529 INFO:tasks.workunit.client.0.smithi006.stdout:5/194: unlink d0/f2e 0 2024-03-22T06:14:52.531 INFO:tasks.workunit.client.0.smithi006.stdout:5/195: truncate d0/db/d16/f30 711397 0 2024-03-22T06:14:52.538 INFO:tasks.workunit.client.0.smithi006.stdout:7/152: rmdir d5 39 2024-03-22T06:14:52.538 INFO:tasks.workunit.client.0.smithi006.stdout:7/153: write f4 [753170,57277] 0 2024-03-22T06:14:52.538 INFO:tasks.workunit.client.0.smithi006.stdout:7/154: rename d5/d1e/d29/l1a to d5/d1e/d29/l2b 0 2024-03-22T06:14:52.545 INFO:tasks.workunit.client.0.smithi006.stdout:7/155: dread d5/fd [0,4194304] 0 2024-03-22T06:14:52.547 INFO:tasks.workunit.client.0.smithi006.stdout:7/156: mknod d5/d1e/c2c 0 2024-03-22T06:14:52.549 INFO:tasks.workunit.client.0.smithi006.stdout:5/196: unlink d0/db/d16/d1c/d34/d35/f36 0 2024-03-22T06:14:52.550 INFO:tasks.workunit.client.0.smithi006.stdout:7/157: mkdir d5/d1e/d29/d1f/d2d 0 2024-03-22T06:14:52.552 INFO:tasks.workunit.client.0.smithi006.stdout:5/197: symlink d0/db/d16/d1c/d34/l3a 0 2024-03-22T06:14:52.554 INFO:tasks.workunit.client.0.smithi006.stdout:7/158: symlink d5/d1e/d29/d10/l2e 0 2024-03-22T06:14:52.556 INFO:tasks.workunit.client.0.smithi006.stdout:5/198: creat d0/db/dc/f3b x:0 0 0 2024-03-22T06:14:52.564 INFO:tasks.workunit.client.0.smithi006.stdout:5/199: mkdir d0/db/d16/d1c/d34/d3c 0 2024-03-22T06:14:52.566 INFO:tasks.workunit.client.0.smithi006.stdout:5/200: creat d0/db/d11/f3d x:0 0 0 2024-03-22T06:14:52.566 INFO:tasks.workunit.client.0.smithi006.stdout:5/201: chown d0/db/dc/c1f 71623837 1 2024-03-22T06:14:52.569 INFO:tasks.workunit.client.0.smithi006.stdout:5/202: truncate d0/f9 4809210 0 2024-03-22T06:14:52.569 INFO:tasks.workunit.client.0.smithi006.stdout:5/203: write d0/db/d16/f21 [496490,101017] 0 2024-03-22T06:14:52.569 INFO:tasks.workunit.client.0.smithi006.stdout:5/204: creat d0/f3e x:0 0 0 2024-03-22T06:14:52.571 INFO:tasks.workunit.client.0.smithi006.stdout:5/205: creat d0/db/dc/d14/f3f x:0 0 0 2024-03-22T06:14:52.574 INFO:tasks.workunit.client.0.smithi006.stdout:5/206: creat d0/db/d16/d1c/f40 x:0 0 0 2024-03-22T06:14:52.574 INFO:tasks.workunit.client.0.smithi006.stdout:5/207: chown d0/f31 1 1 2024-03-22T06:14:52.574 INFO:tasks.workunit.client.0.smithi006.stdout:5/208: dread - d0/db/dc/f3b zero size 2024-03-22T06:14:52.576 INFO:tasks.workunit.client.0.smithi006.stdout:7/159: dwrite d5/fd [0,4194304] 0 2024-03-22T06:14:52.581 INFO:tasks.workunit.client.0.smithi006.stdout:7/160: creat d5/d1e/d29/d1f/f2f x:0 0 0 2024-03-22T06:14:52.584 INFO:tasks.workunit.client.0.smithi006.stdout:7/161: link d5/d1e/d29/d1f/l27 d5/d1e/d29/l30 0 2024-03-22T06:14:52.584 INFO:tasks.workunit.client.0.smithi006.stdout:7/162: readlink d5/d1e/d29/d10/l2e 0 2024-03-22T06:14:52.586 INFO:tasks.workunit.client.0.smithi006.stdout:7/163: dread d5/d1e/d29/f13 [0,4194304] 0 2024-03-22T06:14:52.588 INFO:tasks.workunit.client.0.smithi006.stdout:7/164: creat d5/d1e/f31 x:0 0 0 2024-03-22T06:14:52.590 INFO:tasks.workunit.client.0.smithi006.stdout:2/236: sync 2024-03-22T06:14:52.591 INFO:tasks.workunit.client.0.smithi006.stdout:2/237: read f12 [530491,91582] 0 2024-03-22T06:14:52.591 INFO:tasks.workunit.client.0.smithi006.stdout:2/238: read - d13/d53/f38 zero size 2024-03-22T06:14:52.591 INFO:tasks.workunit.client.0.smithi006.stdout:2/239: chown d13/d34/d4d/l54 26 1 2024-03-22T06:14:52.591 INFO:tasks.workunit.client.0.smithi006.stdout:2/240: write d13/d20/f4b [136222,29382] 0 2024-03-22T06:14:52.592 INFO:tasks.workunit.client.0.smithi006.stdout:7/165: link d5/d1e/d29/d10/l1b d5/d1e/d29/d10/l32 0 2024-03-22T06:14:52.594 INFO:tasks.workunit.client.0.smithi006.stdout:2/241: truncate d13/f18 2821026 0 2024-03-22T06:14:52.594 INFO:tasks.workunit.client.0.smithi006.stdout:2/242: write d13/d20/d26/d19/f46 [557608,82441] 0 2024-03-22T06:14:52.595 INFO:tasks.workunit.client.0.smithi006.stdout:7/166: mknod d5/d1e/d29/d1f/c33 0 2024-03-22T06:14:52.595 INFO:tasks.workunit.client.0.smithi006.stdout:5/209: dwrite d0/db/fd [0,4194304] 0 2024-03-22T06:14:52.596 INFO:tasks.workunit.client.0.smithi006.stdout:2/243: rename d13/d20/d26/d19/f50 to d13/d53/d29/f5f 0 2024-03-22T06:14:52.600 INFO:tasks.workunit.client.0.smithi006.stdout:7/167: rename d5/d1e/d29/c18 to d5/c34 0 2024-03-22T06:14:52.600 INFO:tasks.workunit.client.0.smithi006.stdout:7/168: chown d5/d1e/d29/l2b 1354 1 2024-03-22T06:14:52.603 INFO:tasks.workunit.client.0.smithi006.stdout:5/210: rename d0/db/dc/d14/f20 to d0/db/d11/f41 0 2024-03-22T06:14:52.607 INFO:tasks.workunit.client.0.smithi006.stdout:7/169: read d5/f1d [3693820,75371] 0 2024-03-22T06:14:52.607 INFO:tasks.workunit.client.0.smithi006.stdout:2/244: rename d13/d53/f32 to d13/d34/f60 0 2024-03-22T06:14:52.607 INFO:tasks.workunit.client.0.smithi006.stdout:2/245: write d13/d53/f5e [45492,23767] 0 2024-03-22T06:14:52.609 INFO:tasks.workunit.client.0.smithi006.stdout:5/211: symlink d0/db/dc/l42 0 2024-03-22T06:14:52.609 INFO:tasks.workunit.client.0.smithi006.stdout:5/212: truncate d0/db/d16/d1c/f40 343438 0 2024-03-22T06:14:52.610 INFO:tasks.workunit.client.0.smithi006.stdout:7/170: mknod d5/d1e/d29/d1f/d2d/c35 0 2024-03-22T06:14:52.610 INFO:tasks.workunit.client.0.smithi006.stdout:7/171: creat d5/d1e/f36 x:0 0 0 2024-03-22T06:14:52.618 INFO:tasks.workunit.client.0.smithi006.stdout:5/213: dread d0/db/f29 [0,4194304] 0 2024-03-22T06:14:52.619 INFO:tasks.workunit.client.0.smithi006.stdout:2/246: getdents d13/d20/d26 0 2024-03-22T06:14:52.619 INFO:tasks.workunit.client.0.smithi006.stdout:2/247: write d13/d20/d26/d19/f59 [463014,65827] 0 2024-03-22T06:14:52.622 INFO:tasks.workunit.client.0.smithi006.stdout:7/172: rename d5/d1e/d29/d1f/f2f to d5/d1e/d29/d1f/d2d/f37 0 2024-03-22T06:14:52.624 INFO:tasks.workunit.client.0.smithi006.stdout:5/214: mkdir d0/db/d16/d43 0 2024-03-22T06:14:52.625 INFO:tasks.workunit.client.0.smithi006.stdout:7/173: truncate d5/f7 3143631 0 2024-03-22T06:14:52.628 INFO:tasks.workunit.client.0.smithi006.stdout:5/215: rename d0/db/d16/d1c/d34/d35/f37 to d0/db/dc/d14/f44 0 2024-03-22T06:14:52.638 INFO:tasks.workunit.client.0.smithi006.stdout:2/248: dwrite fb [0,4194304] 0 2024-03-22T06:14:52.641 INFO:tasks.workunit.client.0.smithi006.stdout:2/249: rename d13/d20/d26/d19/f51 to d13/f61 0 2024-03-22T06:14:52.649 INFO:tasks.workunit.client.0.smithi006.stdout:5/216: dwrite d0/db/f1e [0,4194304] 0 2024-03-22T06:14:52.649 INFO:tasks.workunit.client.0.smithi006.stdout:5/217: stat d0/db/d16/d1c/d34/d35 0 2024-03-22T06:14:52.649 INFO:tasks.workunit.client.0.smithi006.stdout:5/218: creat d0/db/dc/d14/f45 x:0 0 0 2024-03-22T06:14:52.649 INFO:tasks.workunit.client.0.smithi006.stdout:5/219: readlink d0/l8 0 2024-03-22T06:14:52.649 INFO:tasks.workunit.client.0.smithi006.stdout:5/220: chown d0/db/dc/d14/c23 61 1 2024-03-22T06:14:52.654 INFO:tasks.workunit.client.0.smithi006.stdout:5/221: creat d0/db/d16/f46 x:0 0 0 2024-03-22T06:14:52.669 INFO:tasks.workunit.client.0.smithi006.stdout:7/174: link d5/d1e/d29/d1f/d2d/f37 d5/d1e/d29/d1f/f38 0 2024-03-22T06:14:52.669 INFO:tasks.workunit.client.0.smithi006.stdout:7/175: rename d5/d1e to d5/d1e/d39 22 2024-03-22T06:14:52.676 INFO:tasks.workunit.client.0.smithi006.stdout:2/250: dwrite f6 [0,4194304] 0 2024-03-22T06:14:52.676 INFO:tasks.workunit.client.0.smithi006.stdout:5/222: dwrite d0/db/d16/d1c/f40 [0,4194304] 0 2024-03-22T06:14:52.676 INFO:tasks.workunit.client.0.smithi006.stdout:5/223: chown d0/db/dc/f3b 1 1 2024-03-22T06:14:52.680 INFO:tasks.workunit.client.0.smithi006.stdout:7/176: write d5/fd [3819280,9782] 0 2024-03-22T06:14:52.686 INFO:tasks.workunit.client.0.smithi006.stdout:7/177: rename d5/f7 to d5/d1e/f3a 0 2024-03-22T06:14:52.686 INFO:tasks.workunit.client.0.smithi006.stdout:7/178: write d5/d1e/f31 [233497,126984] 0 2024-03-22T06:14:52.690 INFO:tasks.workunit.client.0.smithi006.stdout:7/179: dread d5/f1d [0,4194304] 0 2024-03-22T06:14:52.693 INFO:tasks.workunit.client.0.smithi006.stdout:5/224: dread d0/db/d11/f41 [0,4194304] 0 2024-03-22T06:14:52.693 INFO:tasks.workunit.client.0.smithi006.stdout:5/225: fsync d0/db/d11/f3d 0 2024-03-22T06:14:52.694 INFO:tasks.workunit.client.0.smithi006.stdout:7/180: rename d5/d1e/c20 to d5/c3b 0 2024-03-22T06:14:52.697 INFO:tasks.workunit.client.0.smithi006.stdout:7/181: link d5/fd d5/d1e/d29/d1f/d2d/f3c 0 2024-03-22T06:14:52.699 INFO:tasks.workunit.client.0.smithi006.stdout:7/182: truncate f0 441705 0 2024-03-22T06:14:52.700 INFO:tasks.workunit.client.0.smithi006.stdout:2/251: dwrite d13/f23 [0,4194304] 0 2024-03-22T06:14:52.700 INFO:tasks.workunit.client.0.smithi006.stdout:2/252: fdatasync d13/d34/f60 0 2024-03-22T06:14:52.700 INFO:tasks.workunit.client.0.smithi006.stdout:2/253: fdatasync d13/d34/d4d/f5c 0 2024-03-22T06:14:52.711 INFO:tasks.workunit.client.0.smithi006.stdout:2/254: mkdir d13/d20/d26/d2a/d62 0 2024-03-22T06:14:52.711 INFO:tasks.workunit.client.0.smithi006.stdout:2/255: fsync f12 0 2024-03-22T06:14:52.711 INFO:tasks.workunit.client.0.smithi006.stdout:2/256: chown d13 13140 1 2024-03-22T06:14:52.717 INFO:tasks.workunit.client.0.smithi006.stdout:5/226: dwrite d0/f9 [0,4194304] 0 2024-03-22T06:14:52.717 INFO:tasks.workunit.client.0.smithi006.stdout:5/227: stat d0/db/d16/d1c/d34/d35 0 2024-03-22T06:14:52.717 INFO:tasks.workunit.client.0.smithi006.stdout:7/183: dwrite f4 [0,4194304] 0 2024-03-22T06:14:52.732 INFO:tasks.workunit.client.0.smithi006.stdout:7/184: unlink d5/c3b 0 2024-03-22T06:14:52.732 INFO:tasks.workunit.client.0.smithi006.stdout:7/185: fsync d5/d1e/d29/d10/f25 0 2024-03-22T06:14:52.733 INFO:tasks.workunit.client.0.smithi006.stdout:5/228: getdents d0/db/dc/d14 0 2024-03-22T06:14:52.734 INFO:tasks.workunit.client.0.smithi006.stdout:2/257: dwrite d13/d20/f4b [0,4194304] 0 2024-03-22T06:14:52.736 INFO:tasks.workunit.client.0.smithi006.stdout:7/186: creat d5/d1e/d29/d10/f3d x:0 0 0 2024-03-22T06:14:52.736 INFO:tasks.workunit.client.0.smithi006.stdout:7/187: write d5/fd [1733981,128775] 0 2024-03-22T06:14:52.745 INFO:tasks.workunit.client.0.smithi006.stdout:5/229: link d0/f1a d0/db/d16/d1c/d34/d3c/f47 0 2024-03-22T06:14:52.745 INFO:tasks.workunit.client.0.smithi006.stdout:7/188: dread d5/d1e/f31 [0,4194304] 0 2024-03-22T06:14:52.745 INFO:tasks.workunit.client.0.smithi006.stdout:7/189: chown f2 160 1 2024-03-22T06:14:52.746 INFO:tasks.workunit.client.0.smithi006.stdout:7/190: readlink d5/l19 0 2024-03-22T06:14:52.746 INFO:tasks.workunit.client.0.smithi006.stdout:7/191: creat d5/d1e/d29/d1f/f3e x:0 0 0 2024-03-22T06:14:52.747 INFO:tasks.workunit.client.0.smithi006.stdout:9/133: sync 2024-03-22T06:14:52.747 INFO:tasks.workunit.client.0.smithi006.stdout:0/144: sync 2024-03-22T06:14:52.747 INFO:tasks.workunit.client.0.smithi006.stdout:3/170: sync 2024-03-22T06:14:52.748 INFO:tasks.workunit.client.0.smithi006.stdout:2/258: dwrite d13/d34/d4d/f5d [0,4194304] 0 2024-03-22T06:14:52.748 INFO:tasks.workunit.client.0.smithi006.stdout:2/259: readlink ld 0 2024-03-22T06:14:52.748 INFO:tasks.workunit.client.0.smithi006.stdout:2/260: fsync d13/f61 0 2024-03-22T06:14:52.751 INFO:tasks.workunit.client.0.smithi006.stdout:5/230: write d0/db/dc/d14/f1d [1016135,89618] 0 2024-03-22T06:14:52.752 INFO:tasks.workunit.client.0.smithi006.stdout:0/145: link db/l28 db/dc/l36 0 2024-03-22T06:14:52.752 INFO:tasks.workunit.client.0.smithi006.stdout:0/146: chown db/dc/f21 1449836389 1 2024-03-22T06:14:52.754 INFO:tasks.workunit.client.0.smithi006.stdout:9/134: mkdir d8/d1f/d34 0 2024-03-22T06:14:52.756 INFO:tasks.workunit.client.0.smithi006.stdout:8/193: sync 2024-03-22T06:14:52.756 INFO:tasks.workunit.client.0.smithi006.stdout:1/122: sync 2024-03-22T06:14:52.756 INFO:tasks.workunit.client.0.smithi006.stdout:6/207: sync 2024-03-22T06:14:52.756 INFO:tasks.workunit.client.0.smithi006.stdout:4/221: sync 2024-03-22T06:14:52.756 INFO:tasks.workunit.client.0.smithi006.stdout:1/123: creat d8/d9/d13/f28 x:0 0 0 2024-03-22T06:14:52.758 INFO:tasks.workunit.client.0.smithi006.stdout:7/192: dwrite d5/d1e/d29/d10/f25 [0,4194304] 0 2024-03-22T06:14:52.758 INFO:tasks.workunit.client.0.smithi006.stdout:2/261: mknod d13/c63 0 2024-03-22T06:14:52.758 INFO:tasks.workunit.client.0.smithi006.stdout:7/193: chown d5/d1e/d29/c26 127 1 2024-03-22T06:14:52.758 INFO:tasks.workunit.client.0.smithi006.stdout:2/262: write d13/d20/d26/f15 [1893006,20749] 0 2024-03-22T06:14:52.759 INFO:tasks.workunit.client.0.smithi006.stdout:9/135: dread d8/ff [0,4194304] 0 2024-03-22T06:14:52.760 INFO:tasks.workunit.client.0.smithi006.stdout:9/136: truncate d8/fc 567316 0 2024-03-22T06:14:52.760 INFO:tasks.workunit.client.0.smithi006.stdout:9/137: dread - d8/d10/d20/f2c zero size 2024-03-22T06:14:52.762 INFO:tasks.workunit.client.0.smithi006.stdout:5/231: rename d0/f31 to d0/f48 0 2024-03-22T06:14:52.762 INFO:tasks.workunit.client.0.smithi006.stdout:5/232: stat d0/db/d11/f2d 0 2024-03-22T06:14:52.763 INFO:tasks.workunit.client.0.smithi006.stdout:5/233: creat d0/db/d16/f49 x:0 0 0 2024-03-22T06:14:52.764 INFO:tasks.workunit.client.0.smithi006.stdout:0/147: mkdir db/dc/d37 0 2024-03-22T06:14:52.766 INFO:tasks.workunit.client.0.smithi006.stdout:6/208: mkdir d6/d1d/d32/d4f 0 2024-03-22T06:14:52.769 INFO:tasks.workunit.client.0.smithi006.stdout:4/222: rename d6/d1b/d20/d21/c2e to d6/d13/d3b/c4d 0 2024-03-22T06:14:52.773 INFO:tasks.workunit.client.0.smithi006.stdout:1/124: symlink d8/d9/d10/d14/d22/l29 0 2024-03-22T06:14:52.782 INFO:tasks.workunit.client.0.smithi006.stdout:9/138: creat d8/d10/d28/f35 x:0 0 0 2024-03-22T06:14:52.782 INFO:tasks.workunit.client.0.smithi006.stdout:9/139: write d8/d10/f17 [112172,100350] 0 2024-03-22T06:14:52.782 INFO:tasks.workunit.client.0.smithi006.stdout:9/140: chown d8 1663817 1 2024-03-22T06:14:52.785 INFO:tasks.workunit.client.0.smithi006.stdout:4/223: truncate d6/f7 32928 0 2024-03-22T06:14:52.785 INFO:tasks.workunit.client.0.smithi006.stdout:4/224: fdatasync d6/d1b/d20/d21/d2a/f3f 0 2024-03-22T06:14:52.787 INFO:tasks.workunit.client.0.smithi006.stdout:0/148: write db/d10/f1a [213806,7515] 0 2024-03-22T06:14:52.787 INFO:tasks.workunit.client.0.smithi006.stdout:1/125: mkdir d8/d9/d10/d14/d18/d2a 0 2024-03-22T06:14:52.787 INFO:tasks.workunit.client.0.smithi006.stdout:1/126: chown d8 3940429 1 2024-03-22T06:14:52.788 INFO:tasks.workunit.client.0.smithi006.stdout:4/225: creat d6/d13/d3b/f4e x:0 0 0 2024-03-22T06:14:52.788 INFO:tasks.workunit.client.0.smithi006.stdout:4/226: read - d6/d1b/d20/d21/d2a/f3f zero size 2024-03-22T06:14:52.790 INFO:tasks.workunit.client.0.smithi006.stdout:0/149: stat db/dc/l1d 0 2024-03-22T06:14:52.792 INFO:tasks.workunit.client.0.smithi006.stdout:1/127: mknod d8/d9/d10/d14/d18/d2a/c2b 0 2024-03-22T06:14:52.792 INFO:tasks.workunit.client.0.smithi006.stdout:1/128: write d8/d9/d13/f26 [986694,45779] 0 2024-03-22T06:14:52.792 INFO:tasks.workunit.client.0.smithi006.stdout:1/129: write d8/d9/f1e [4931568,77244] 0 2024-03-22T06:14:52.792 INFO:tasks.workunit.client.0.smithi006.stdout:1/130: fdatasync d8/d9/f1f 0 2024-03-22T06:14:52.793 INFO:tasks.workunit.client.0.smithi006.stdout:1/131: mknod d8/d9/d10/d14/d18/c2c 0 2024-03-22T06:14:52.794 INFO:tasks.workunit.client.0.smithi006.stdout:0/150: dread db/dc/f21 [0,4194304] 0 2024-03-22T06:14:52.799 INFO:tasks.workunit.client.0.smithi006.stdout:4/227: dread d6/f8 [0,4194304] 0 2024-03-22T06:14:52.799 INFO:tasks.workunit.client.0.smithi006.stdout:4/228: creat d6/d13/d3b/f4f x:0 0 0 2024-03-22T06:14:52.805 INFO:tasks.workunit.client.0.smithi006.stdout:8/194: dwrite d9/db/dc/de/d3e/f3f [0,4194304] 0 2024-03-22T06:14:52.818 INFO:tasks.workunit.client.0.smithi006.stdout:8/195: dread d9/fa [0,4194304] 0 2024-03-22T06:14:52.820 INFO:tasks.workunit.client.0.smithi006.stdout:8/196: mkdir d9/db/d37/d4a/d4d 0 2024-03-22T06:14:52.821 INFO:tasks.workunit.client.0.smithi006.stdout:8/197: symlink d9/d14/d47/l4e 0 2024-03-22T06:14:52.822 INFO:tasks.workunit.client.0.smithi006.stdout:8/198: creat d9/db/f4f x:0 0 0 2024-03-22T06:14:52.824 INFO:tasks.workunit.client.0.smithi006.stdout:8/199: link d9/db/d12/l2c d9/db/dc/de/d3e/l50 0 2024-03-22T06:14:52.827 INFO:tasks.workunit.client.0.smithi006.stdout:0/151: dwrite db/dc/f21 [0,4194304] 0 2024-03-22T06:14:52.827 INFO:tasks.workunit.client.0.smithi006.stdout:5/234: dwrite d0/f39 [4194304,4194304] 0 2024-03-22T06:14:52.827 INFO:tasks.workunit.client.0.smithi006.stdout:7/194: dwrite d5/d1e/f36 [0,4194304] 0 2024-03-22T06:14:52.827 INFO:tasks.workunit.client.0.smithi006.stdout:5/235: read - d0/db/d16/f46 zero size 2024-03-22T06:14:52.827 INFO:tasks.workunit.client.0.smithi006.stdout:7/195: readlink d5/d1e/d29/l14 0 2024-03-22T06:14:52.827 INFO:tasks.workunit.client.0.smithi006.stdout:7/196: readlink d5/l19 0 2024-03-22T06:14:52.828 INFO:tasks.workunit.client.0.smithi006.stdout:5/236: write d0/db/dc/d14/f45 [706409,10356] 0 2024-03-22T06:14:52.828 INFO:tasks.workunit.client.0.smithi006.stdout:4/229: dwrite d6/f12 [0,4194304] 0 2024-03-22T06:14:52.828 INFO:tasks.workunit.client.0.smithi006.stdout:4/230: fdatasync d6/d1b/f29 0 2024-03-22T06:14:52.828 INFO:tasks.workunit.client.0.smithi006.stdout:6/209: dwrite d6/d1b/f21 [0,4194304] 0 2024-03-22T06:14:52.828 INFO:tasks.workunit.client.0.smithi006.stdout:9/141: dwrite d8/d10/d20/f2d [0,4194304] 0 2024-03-22T06:14:52.828 INFO:tasks.workunit.client.0.smithi006.stdout:9/142: dread - d8/d10/d20/f2c zero size 2024-03-22T06:14:52.828 INFO:tasks.workunit.client.0.smithi006.stdout:9/143: chown d8/d10/d11/f16 1 1 2024-03-22T06:14:52.828 INFO:tasks.workunit.client.0.smithi006.stdout:9/144: chown d8/fd 506 1 2024-03-22T06:14:52.828 INFO:tasks.workunit.client.0.smithi006.stdout:8/200: rename d9/d14/f2f to d9/db/d12/d1e/f51 0 2024-03-22T06:14:52.828 INFO:tasks.workunit.client.0.smithi006.stdout:8/201: chown d9/db/d37/c46 65638885 1 2024-03-22T06:14:52.830 INFO:tasks.workunit.client.0.smithi006.stdout:4/231: dread d6/d1b/d20/f22 [0,4194304] 0 2024-03-22T06:14:52.832 INFO:tasks.workunit.client.0.smithi006.stdout:5/237: dread d0/db/dc/d14/f45 [0,4194304] 0 2024-03-22T06:14:52.832 INFO:tasks.workunit.client.0.smithi006.stdout:5/238: write d0/db/d16/f46 [203407,26011] 0 2024-03-22T06:14:52.832 INFO:tasks.workunit.client.0.smithi006.stdout:0/152: mkdir db/dc/d38 0 2024-03-22T06:14:52.841 INFO:tasks.workunit.client.0.smithi006.stdout:1/132: dread d8/d9/f1e [0,4194304] 0 2024-03-22T06:14:52.841 INFO:tasks.workunit.client.0.smithi006.stdout:4/232: dread d6/fa [0,4194304] 0 2024-03-22T06:14:52.842 INFO:tasks.workunit.client.0.smithi006.stdout:8/202: dread d9/db/dc/de/d3e/f3f [0,4194304] 0 2024-03-22T06:14:52.843 INFO:tasks.workunit.client.0.smithi006.stdout:7/197: mkdir d5/d1e/d3f 0 2024-03-22T06:14:52.843 INFO:tasks.workunit.client.0.smithi006.stdout:7/198: fsync d5/d1e/f21 0 2024-03-22T06:14:52.850 INFO:tasks.workunit.client.0.smithi006.stdout:9/145: truncate d8/fa 272459 0 2024-03-22T06:14:52.851 INFO:tasks.workunit.client.0.smithi006.stdout:5/239: dwrite d0/db/dc/d14/f3f [0,4194304] 0 2024-03-22T06:14:52.852 INFO:tasks.workunit.client.0.smithi006.stdout:6/210: rename d6/cb to d6/d1b/d2c/c50 0 2024-03-22T06:14:52.861 INFO:tasks.workunit.client.0.smithi006.stdout:1/133: truncate d8/d9/d10/d14/d18/f1b 2122377 0 2024-03-22T06:14:52.864 INFO:tasks.workunit.client.0.smithi006.stdout:6/211: dwrite d6/d1d/f28 [0,4194304] 0 2024-03-22T06:14:52.864 INFO:tasks.workunit.client.0.smithi006.stdout:6/212: creat d6/f51 x:0 0 0 2024-03-22T06:14:52.864 INFO:tasks.workunit.client.0.smithi006.stdout:4/233: mknod d6/d1b/d20/d2c/c50 0 2024-03-22T06:14:52.866 INFO:tasks.workunit.client.0.smithi006.stdout:8/203: creat d9/d14/d44/f52 x:0 0 0 2024-03-22T06:14:52.876 INFO:tasks.workunit.client.0.smithi006.stdout:9/146: rename d8/ce to d8/d10/d11/c36 0 2024-03-22T06:14:52.876 INFO:tasks.workunit.client.0.smithi006.stdout:9/147: rename d8/d1f to d8/d1f/d37 22 2024-03-22T06:14:52.876 INFO:tasks.workunit.client.0.smithi006.stdout:9/148: fdatasync d8/d10/d11/f16 0 2024-03-22T06:14:52.876 INFO:tasks.workunit.client.0.smithi006.stdout:9/149: chown d8 93914 1 2024-03-22T06:14:52.879 INFO:tasks.workunit.client.0.smithi006.stdout:1/134: creat d8/d9/f2d x:0 0 0 2024-03-22T06:14:52.893 INFO:tasks.workunit.client.0.smithi006.stdout:9/150: dwrite f6 [0,4194304] 0 2024-03-22T06:14:52.893 INFO:tasks.workunit.client.0.smithi006.stdout:9/151: chown d8/f19 221 1 2024-03-22T06:14:52.893 INFO:tasks.workunit.client.0.smithi006.stdout:9/152: write d8/d10/f17 [967131,21338] 0 2024-03-22T06:14:52.893 INFO:tasks.workunit.client.0.smithi006.stdout:4/234: unlink d6/d1b/d20/d21/l28 0 2024-03-22T06:14:52.893 INFO:tasks.workunit.client.0.smithi006.stdout:4/235: chown d6/fa 1280 1 2024-03-22T06:14:52.894 INFO:tasks.workunit.client.0.smithi006.stdout:8/204: creat d9/db/dc/de/f53 x:0 0 0 2024-03-22T06:14:52.894 INFO:tasks.workunit.client.0.smithi006.stdout:8/205: fdatasync d9/f11 0 2024-03-22T06:14:52.902 INFO:tasks.workunit.client.0.smithi006.stdout:5/240: getdents d0/db/d16 0 2024-03-22T06:14:52.902 INFO:tasks.workunit.client.0.smithi006.stdout:5/241: readlink d0/l7 0 2024-03-22T06:14:52.902 INFO:tasks.workunit.client.0.smithi006.stdout:5/242: dread - d0/db/d11/f3d zero size 2024-03-22T06:14:52.902 INFO:tasks.workunit.client.0.smithi006.stdout:5/243: getdents d0/db/d16/d43 0 2024-03-22T06:14:52.902 INFO:tasks.workunit.client.0.smithi006.stdout:5/244: chown d0/db/d16/c2c 1 1 2024-03-22T06:14:52.904 INFO:tasks.workunit.client.0.smithi006.stdout:1/135: symlink d8/d9/l2e 0 2024-03-22T06:14:52.906 INFO:tasks.workunit.client.0.smithi006.stdout:9/153: mkdir d8/d10/d11/d2f/d38 0 2024-03-22T06:14:52.909 INFO:tasks.workunit.client.0.smithi006.stdout:4/236: unlink d6/d1b/d20/d2c/f49 0 2024-03-22T06:14:52.909 INFO:tasks.workunit.client.0.smithi006.stdout:4/237: chown d6/l1a 361 1 2024-03-22T06:14:52.909 INFO:tasks.workunit.client.0.smithi006.stdout:4/238: creat d6/d13/d3b/f51 x:0 0 0 2024-03-22T06:14:52.912 INFO:tasks.workunit.client.0.smithi006.stdout:8/206: mkdir d9/db/d35/d54 0 2024-03-22T06:14:52.915 INFO:tasks.workunit.client.0.smithi006.stdout:1/136: truncate d8/d9/d10/d14/f20 6838952 0 2024-03-22T06:14:52.916 INFO:tasks.workunit.client.0.smithi006.stdout:1/137: chown d8/d9/d10/d14/d18 13942 1 2024-03-22T06:14:52.917 INFO:tasks.workunit.client.0.smithi006.stdout:9/154: mkdir d8/d10/d28/d39 0 2024-03-22T06:14:52.922 INFO:tasks.workunit.client.0.smithi006.stdout:4/239: mknod d6/d1b/d20/d21/d2a/d43/d4b/c52 0 2024-03-22T06:14:52.924 INFO:tasks.workunit.client.0.smithi006.stdout:1/138: symlink d8/d9/d10/d14/d22/l2f 0 2024-03-22T06:14:52.928 INFO:tasks.workunit.client.0.smithi006.stdout:5/245: dwrite d0/db/d11/f2d [0,4194304] 0 2024-03-22T06:14:52.928 INFO:tasks.workunit.client.0.smithi006.stdout:1/139: rename d8/d9/d10/d14/d18/d2a/c2b to d8/d9/d10/d14/d18/d2a/c30 0 2024-03-22T06:14:52.928 INFO:tasks.workunit.client.0.smithi006.stdout:1/140: write d8/d9/d13/f26 [1170031,115789] 0 2024-03-22T06:14:52.930 INFO:tasks.workunit.client.0.smithi006.stdout:7/199: rmdir d5/d1e 39 2024-03-22T06:14:52.935 INFO:tasks.workunit.client.0.smithi006.stdout:1/141: rename d8/d9/d10/c11 to d8/c31 0 2024-03-22T06:14:52.950 INFO:tasks.workunit.client.0.smithi006.stdout:8/207: dwrite d9/db/dc/de/d3e/f40 [0,4194304] 0 2024-03-22T06:14:52.950 INFO:tasks.workunit.client.0.smithi006.stdout:8/208: readlink d9/db/d12/l2c 0 2024-03-22T06:14:52.955 INFO:tasks.workunit.client.0.smithi006.stdout:8/209: creat d9/db/d35/f55 x:0 0 0 2024-03-22T06:14:52.955 INFO:tasks.workunit.client.0.smithi006.stdout:8/210: dread - d9/db/d12/f26 zero size 2024-03-22T06:14:52.958 INFO:tasks.workunit.client.0.smithi006.stdout:8/211: dread f1 [0,4194304] 0 2024-03-22T06:14:52.959 INFO:tasks.workunit.client.0.smithi006.stdout:9/155: dwrite d8/d10/f12 [0,4194304] 0 2024-03-22T06:14:52.960 INFO:tasks.workunit.client.0.smithi006.stdout:5/246: dwrite d0/db/d16/f21 [0,4194304] 0 2024-03-22T06:14:52.960 INFO:tasks.workunit.client.0.smithi006.stdout:5/247: fdatasync d0/db/dc/f3b 0 2024-03-22T06:14:52.960 INFO:tasks.workunit.client.0.smithi006.stdout:7/200: dwrite d5/d1e/d29/f2a [0,4194304] 0 2024-03-22T06:14:52.960 INFO:tasks.workunit.client.0.smithi006.stdout:9/156: stat d8/d10/l15 0 2024-03-22T06:14:52.961 INFO:tasks.workunit.client.0.smithi006.stdout:8/212: write d9/db/d12/d1e/f30 [715349,14429] 0 2024-03-22T06:14:52.963 INFO:tasks.workunit.client.0.smithi006.stdout:1/142: dread d8/d9/d10/d14/d18/f1b [0,4194304] 0 2024-03-22T06:14:52.963 INFO:tasks.workunit.client.0.smithi006.stdout:1/143: fdatasync d8/d9/d13/f28 0 2024-03-22T06:14:52.965 INFO:tasks.workunit.client.0.smithi006.stdout:5/248: dread d0/db/d11/f41 [0,4194304] 0 2024-03-22T06:14:52.972 INFO:tasks.workunit.client.0.smithi006.stdout:7/201: link d5/d1e/d29/d1f/l24 d5/d1e/d29/d10/l40 0 2024-03-22T06:14:52.972 INFO:tasks.workunit.client.0.smithi006.stdout:7/202: chown d5/la 8012243 1 2024-03-22T06:14:52.977 INFO:tasks.workunit.client.0.smithi006.stdout:9/157: rename d8/d10/d11/f1c to d8/d1f/d34/f3a 0 2024-03-22T06:14:52.980 INFO:tasks.workunit.client.0.smithi006.stdout:8/213: creat d9/db/dc/de/d3e/f56 x:0 0 0 2024-03-22T06:14:52.982 INFO:tasks.workunit.client.0.smithi006.stdout:1/144: mkdir d8/d9/d13/d24/d32 0 2024-03-22T06:14:52.982 INFO:tasks.workunit.client.0.smithi006.stdout:1/145: dread - d8/d9/f2d zero size 2024-03-22T06:14:52.986 INFO:tasks.workunit.client.0.smithi006.stdout:5/249: creat d0/db/d16/d1c/d34/f4a x:0 0 0 2024-03-22T06:14:52.987 INFO:tasks.workunit.client.0.smithi006.stdout:8/214: write d9/db/f20 [4017454,104896] 0 2024-03-22T06:14:52.987 INFO:tasks.workunit.client.0.smithi006.stdout:9/158: write f6 [3064494,75210] 0 2024-03-22T06:14:52.990 INFO:tasks.workunit.client.0.smithi006.stdout:7/203: mkdir d5/d1e/d29/d1f/d2d/d41 0 2024-03-22T06:14:52.990 INFO:tasks.workunit.client.0.smithi006.stdout:7/204: dread - d5/d1e/d29/d1f/f38 zero size 2024-03-22T06:14:52.991 INFO:tasks.workunit.client.0.smithi006.stdout:7/205: dread d5/fe [4194304,4194304] 0 2024-03-22T06:14:52.993 INFO:tasks.workunit.client.0.smithi006.stdout:5/250: unlink d0/ca 0 2024-03-22T06:14:52.997 INFO:tasks.workunit.client.0.smithi006.stdout:8/215: mknod d9/db/d37/d4a/c57 0 2024-03-22T06:14:53.003 INFO:tasks.workunit.client.0.smithi006.stdout:8/216: dread d9/db/dc/de/f17 [0,4194304] 0 2024-03-22T06:14:53.005 INFO:tasks.workunit.client.0.smithi006.stdout:1/146: write d8/d9/d10/d14/f20 [7575666,43766] 0 2024-03-22T06:14:53.008 INFO:tasks.workunit.client.0.smithi006.stdout:8/217: dread d9/db/d12/f33 [0,4194304] 0 2024-03-22T06:14:53.011 INFO:tasks.workunit.client.0.smithi006.stdout:8/218: symlink d9/d14/d47/l58 0 2024-03-22T06:14:53.014 INFO:tasks.workunit.client.0.smithi006.stdout:8/219: creat d9/db/f59 x:0 0 0 2024-03-22T06:14:53.014 INFO:tasks.workunit.client.0.smithi006.stdout:8/220: fdatasync d9/db/dc/fd 0 2024-03-22T06:14:53.014 INFO:tasks.workunit.client.0.smithi006.stdout:8/221: write d9/db/f20 [1002305,61027] 0 2024-03-22T06:14:53.015 INFO:tasks.workunit.client.0.smithi006.stdout:5/251: dwrite d0/db/f25 [0,4194304] 0 2024-03-22T06:14:53.015 INFO:tasks.workunit.client.0.smithi006.stdout:5/252: write d0/db/dc/d14/f44 [314744,106225] 0 2024-03-22T06:14:53.022 INFO:tasks.workunit.client.0.smithi006.stdout:8/222: rename d9/db/f59 to d9/db/d12/d1e/f5a 0 2024-03-22T06:14:53.022 INFO:tasks.workunit.client.0.smithi006.stdout:8/223: write d9/db/d12/d1e/f5a [805913,105091] 0 2024-03-22T06:14:53.030 INFO:tasks.workunit.client.0.smithi006.stdout:1/147: dwrite d8/d9/d13/f28 [0,4194304] 0 2024-03-22T06:14:53.030 INFO:tasks.workunit.client.0.smithi006.stdout:5/253: dwrite d0/db/d16/f46 [0,4194304] 0 2024-03-22T06:14:53.030 INFO:tasks.workunit.client.0.smithi006.stdout:5/254: readlink d0/db/d16/l17 0 2024-03-22T06:14:53.043 INFO:tasks.workunit.client.0.smithi006.stdout:5/255: truncate d0/f5 1453396 0 2024-03-22T06:14:53.043 INFO:tasks.workunit.client.0.smithi006.stdout:5/256: getdents d0/db/d16/d43 0 2024-03-22T06:14:53.043 INFO:tasks.workunit.client.0.smithi006.stdout:5/257: fdatasync d0/db/f25 0 2024-03-22T06:14:53.047 INFO:tasks.workunit.client.0.smithi006.stdout:8/224: dwrite d9/db/d35/f55 [0,4194304] 0 2024-03-22T06:14:53.047 INFO:tasks.workunit.client.0.smithi006.stdout:8/225: write d9/db/d12/d1e/f30 [902978,1748] 0 2024-03-22T06:14:53.047 INFO:tasks.workunit.client.0.smithi006.stdout:8/226: truncate d9/db/d12/f43 85290 0 2024-03-22T06:14:53.047 INFO:tasks.workunit.client.0.smithi006.stdout:1/148: dwrite d8/d9/f1f [0,4194304] 0 2024-03-22T06:14:53.047 INFO:tasks.workunit.client.0.smithi006.stdout:1/149: fdatasync d8/d9/d13/f28 0 2024-03-22T06:14:53.048 INFO:tasks.workunit.client.0.smithi006.stdout:5/258: getdents d0/db/d16/d1c 0 2024-03-22T06:14:53.048 INFO:tasks.workunit.client.0.smithi006.stdout:5/259: creat d0/db/d16/d1c/d34/f4b x:0 0 0 2024-03-22T06:14:53.048 INFO:tasks.workunit.client.0.smithi006.stdout:5/260: fsync d0/db/f29 0 2024-03-22T06:14:53.058 INFO:tasks.workunit.client.0.smithi006.stdout:8/227: creat d9/db/d37/d4a/f5b x:0 0 0 2024-03-22T06:14:53.058 INFO:tasks.workunit.client.0.smithi006.stdout:8/228: fsync d9/db/d12/f3d 0 2024-03-22T06:14:53.058 INFO:tasks.workunit.client.0.smithi006.stdout:8/229: fsync d9/db/d12/f33 0 2024-03-22T06:14:53.059 INFO:tasks.workunit.client.0.smithi006.stdout:5/261: dread d0/db/d16/d1c/f40 [0,4194304] 0 2024-03-22T06:14:53.059 INFO:tasks.workunit.client.0.smithi006.stdout:5/262: write d0/f48 [894548,119873] 0 2024-03-22T06:14:53.065 INFO:tasks.workunit.client.0.smithi006.stdout:1/150: dwrite d8/d9/fd [0,4194304] 0 2024-03-22T06:14:53.068 INFO:tasks.workunit.client.0.smithi006.stdout:5/263: write d0/db/d16/f30 [3950814,67059] 0 2024-03-22T06:14:53.117 INFO:tasks.workunit.client.0.smithi006.stdout:8/230: creat d9/d14/d47/f5c x:0 0 0 2024-03-22T06:14:53.117 INFO:tasks.workunit.client.0.smithi006.stdout:8/231: creat d9/db/d12/f5d x:0 0 0 2024-03-22T06:14:53.118 INFO:tasks.workunit.client.0.smithi006.stdout:8/232: readlink d9/d14/d47/l58 0 2024-03-22T06:14:53.124 INFO:tasks.workunit.client.0.smithi006.stdout:1/151: mkdir d8/d33 0 2024-03-22T06:14:53.127 INFO:tasks.workunit.client.0.smithi006.stdout:5/264: rename d0/db/dc/d14/f44 to d0/db/d16/d1c/f4c 0 2024-03-22T06:14:53.127 INFO:tasks.workunit.client.0.smithi006.stdout:5/265: fdatasync d0/db/d16/d1c/f4c 0 2024-03-22T06:14:53.127 INFO:tasks.workunit.client.0.smithi006.stdout:5/266: chown d0/db/d16/c19 50635638 1 2024-03-22T06:14:53.131 INFO:tasks.workunit.client.0.smithi006.stdout:8/233: mkdir d9/d5e 0 2024-03-22T06:14:53.134 INFO:tasks.workunit.client.0.smithi006.stdout:5/267: mkdir d0/db/d16/d1c/d34/d4d 0 2024-03-22T06:14:53.141 INFO:tasks.workunit.client.0.smithi006.stdout:5/268: dread d0/db/d16/d1c/d34/d3c/f47 [0,4194304] 0 2024-03-22T06:14:53.141 INFO:tasks.workunit.client.0.smithi006.stdout:5/269: fsync d0/db/d16/d1c/f40 0 2024-03-22T06:14:53.154 INFO:tasks.workunit.client.0.smithi006.stdout:8/234: unlink d9/db/d12/d1e/f5a 0 2024-03-22T06:14:53.154 INFO:tasks.workunit.client.0.smithi006.stdout:8/235: chown d9/db/d12/f26 59564 1 2024-03-22T06:14:53.154 INFO:tasks.workunit.client.0.smithi006.stdout:8/236: creat d9/d5e/f5f x:0 0 0 2024-03-22T06:14:53.155 INFO:tasks.workunit.client.0.smithi006.stdout:8/237: chown f1 17647 1 2024-03-22T06:14:53.156 INFO:tasks.workunit.client.0.smithi006.stdout:8/238: creat d9/db/f60 x:0 0 0 2024-03-22T06:14:53.156 INFO:tasks.workunit.client.0.smithi006.stdout:8/239: readlink d9/d14/d47/l4e 0 2024-03-22T06:14:53.158 INFO:tasks.workunit.client.0.smithi006.stdout:8/240: mknod d9/d14/d44/c61 0 2024-03-22T06:14:53.161 INFO:tasks.workunit.client.0.smithi006.stdout:8/241: truncate d9/db/dc/de/f22 2444249 0 2024-03-22T06:14:53.161 INFO:tasks.workunit.client.0.smithi006.stdout:8/242: fsync d9/db/d12/f43 0 2024-03-22T06:14:53.161 INFO:tasks.workunit.client.0.smithi006.stdout:8/243: write d9/d14/d47/f5c [172182,52833] 0 2024-03-22T06:14:53.170 INFO:tasks.workunit.client.0.smithi006.stdout:8/244: dread d9/fa [4194304,4194304] 0 2024-03-22T06:14:53.202 INFO:tasks.workunit.client.0.smithi006.stdout:5/270: getdents d0/db/d16/d1c/d34 0 2024-03-22T06:14:53.205 INFO:tasks.workunit.client.0.smithi006.stdout:5/271: link d0/db/dc/d14/f3f d0/db/d16/d1c/d34/d35/f4e 0 2024-03-22T06:14:53.230 INFO:tasks.workunit.client.0.smithi006.stdout:5/272: dwrite d0/f48 [0,4194304] 0 2024-03-22T06:14:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:5/273: mknod d0/db/d16/d1c/d34/d3c/c4f 0 2024-03-22T06:14:53.240 INFO:tasks.workunit.client.0.smithi006.stdout:5/274: mkdir d0/db/d50 0 2024-03-22T06:14:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:5/275: mknod d0/db/d16/d1c/d34/c51 0 2024-03-22T06:14:53.264 INFO:tasks.workunit.client.0.smithi006.stdout:5/276: dwrite d0/f39 [4194304,4194304] 0 2024-03-22T06:14:53.264 INFO:tasks.workunit.client.0.smithi006.stdout:5/277: chown d0/db/dc 7 1 2024-03-22T06:14:53.266 INFO:tasks.workunit.client.0.smithi006.stdout:5/278: mkdir d0/db/d11/d52 0 2024-03-22T06:14:53.266 INFO:tasks.workunit.client.0.smithi006.stdout:5/279: read d0/f9 [80726,127336] 0 2024-03-22T06:14:53.270 INFO:tasks.workunit.client.0.smithi006.stdout:5/280: rename d0/db/d16/d1c/d34/f4a to d0/db/d16/d1c/d34/f53 0 2024-03-22T06:14:53.270 INFO:tasks.workunit.client.0.smithi006.stdout:5/281: chown d0/l6 4035 1 2024-03-22T06:14:53.273 INFO:tasks.workunit.client.0.smithi006.stdout:5/282: mkdir d0/db/d16/d1c/d34/d3c/d54 0 2024-03-22T06:14:53.275 INFO:tasks.workunit.client.0.smithi006.stdout:5/283: creat d0/db/d16/d1c/d34/d3c/f55 x:0 0 0 2024-03-22T06:14:53.276 INFO:tasks.workunit.client.0.smithi006.stdout:5/284: readlink d0/db/d16/l2a 0 2024-03-22T06:14:53.290 INFO:tasks.workunit.client.0.smithi006.stdout:2/263: sync 2024-03-22T06:14:53.290 INFO:tasks.workunit.client.0.smithi006.stdout:3/171: sync 2024-03-22T06:14:53.290 INFO:tasks.workunit.client.0.smithi006.stdout:3/172: chown d0/d8/d9/d24 55 1 2024-03-22T06:14:53.291 INFO:tasks.workunit.client.0.smithi006.stdout:3/173: creat d0/d8/d1c/f37 x:0 0 0 2024-03-22T06:14:53.292 INFO:tasks.workunit.client.0.smithi006.stdout:2/264: unlink d13/d20/d26/d19/f39 0 2024-03-22T06:14:53.295 INFO:tasks.workunit.client.0.smithi006.stdout:5/285: dwrite d0/db/d11/f2d [4194304,4194304] 0 2024-03-22T06:14:53.296 INFO:tasks.workunit.client.0.smithi006.stdout:3/174: mknod d0/d4/d31/c38 0 2024-03-22T06:14:53.299 INFO:tasks.workunit.client.0.smithi006.stdout:2/265: link d13/d20/f5a d13/d53/f64 0 2024-03-22T06:14:53.306 INFO:tasks.workunit.client.0.smithi006.stdout:2/266: rmdir d13/d20/d26/d2a 39 2024-03-22T06:14:53.313 INFO:tasks.workunit.client.0.smithi006.stdout:5/286: dwrite d0/db/dc/d14/f2b [0,4194304] 0 2024-03-22T06:14:53.313 INFO:tasks.workunit.client.0.smithi006.stdout:5/287: stat d0/db/d16/f30 0 2024-03-22T06:14:53.313 INFO:tasks.workunit.client.0.smithi006.stdout:5/288: stat d0/l8 0 2024-03-22T06:14:53.328 INFO:tasks.workunit.client.0.smithi006.stdout:5/289: dwrite d0/db/d16/f46 [0,4194304] 0 2024-03-22T06:14:53.332 INFO:tasks.workunit.client.0.smithi006.stdout:5/290: rename d0/c2 to d0/db/d11/c56 0 2024-03-22T06:14:53.332 INFO:tasks.workunit.client.0.smithi006.stdout:5/291: fsync d0/db/d16/d1c/f40 0 2024-03-22T06:14:53.338 INFO:tasks.workunit.client.0.smithi006.stdout:5/292: dread d0/db/d16/d1c/d34/d3c/f47 [0,4194304] 0 2024-03-22T06:14:53.342 INFO:tasks.workunit.client.0.smithi006.stdout:3/175: getdents d0/d8/d1c 0 2024-03-22T06:14:53.347 INFO:tasks.workunit.client.0.smithi006.stdout:5/293: write d0/db/dc/d14/f2b [1609194,2435] 0 2024-03-22T06:14:53.347 INFO:tasks.workunit.client.0.smithi006.stdout:5/294: chown d0/db/dc 1271 1 2024-03-22T06:14:53.349 INFO:tasks.workunit.client.0.smithi006.stdout:3/176: write d0/f3 [5142896,65712] 0 2024-03-22T06:14:53.352 INFO:tasks.workunit.client.0.smithi006.stdout:5/295: rename d0/f9 to d0/db/f57 0 2024-03-22T06:14:53.352 INFO:tasks.workunit.client.0.smithi006.stdout:5/296: fdatasync d0/db/d16/d1c/d34/d35/f4e 0 2024-03-22T06:14:53.354 INFO:tasks.workunit.client.0.smithi006.stdout:5/297: getdents d0/db/dc/d14 0 2024-03-22T06:14:53.358 INFO:tasks.workunit.client.0.smithi006.stdout:3/177: rmdir d0/d8 39 2024-03-22T06:14:53.358 INFO:tasks.workunit.client.0.smithi006.stdout:0/153: sync 2024-03-22T06:14:53.359 INFO:tasks.workunit.client.0.smithi006.stdout:6/213: sync 2024-03-22T06:14:53.359 INFO:tasks.workunit.client.0.smithi006.stdout:6/214: creat d6/d1b/d2c/f52 x:0 0 0 2024-03-22T06:14:53.361 INFO:tasks.workunit.client.0.smithi006.stdout:5/298: dread d0/db/dc/d14/f3f [0,4194304] 0 2024-03-22T06:14:53.361 INFO:tasks.workunit.client.0.smithi006.stdout:5/299: chown d0/db/dc 9947377 1 2024-03-22T06:14:53.361 INFO:tasks.workunit.client.0.smithi006.stdout:5/300: chown d0/db/d16/l2a 113112951 1 2024-03-22T06:14:53.361 INFO:tasks.workunit.client.0.smithi006.stdout:5/301: fsync d0/db/d16/d1c/d34/f53 0 2024-03-22T06:14:53.361 INFO:tasks.workunit.client.0.smithi006.stdout:5/302: chown d0/db/d16/f21 37807475 1 2024-03-22T06:14:53.362 INFO:tasks.workunit.client.0.smithi006.stdout:0/154: symlink db/l39 0 2024-03-22T06:14:53.363 INFO:tasks.workunit.client.0.smithi006.stdout:0/155: dread - db/f30 zero size 2024-03-22T06:14:53.370 INFO:tasks.workunit.client.0.smithi006.stdout:5/303: mkdir d0/db/d50/d58 0 2024-03-22T06:14:53.372 INFO:tasks.workunit.client.0.smithi006.stdout:0/156: stat db/l28 0 2024-03-22T06:14:53.372 INFO:tasks.workunit.client.0.smithi006.stdout:0/157: write db/d10/d2a/f2c [336256,52246] 0 2024-03-22T06:14:53.374 INFO:tasks.workunit.client.0.smithi006.stdout:3/178: rename d0/d4/d10 to d0/d23/d27/d36/d39 0 2024-03-22T06:14:53.378 INFO:tasks.workunit.client.0.smithi006.stdout:6/215: rename d6/d1b/c39 to d6/c53 0 2024-03-22T06:14:53.380 INFO:tasks.workunit.client.0.smithi006.stdout:3/179: mkdir d0/d4/d3a 0 2024-03-22T06:14:53.385 INFO:tasks.workunit.client.0.smithi006.stdout:0/158: rename db/d10/f1f to db/dc/d38/f3a 0 2024-03-22T06:14:53.385 INFO:tasks.workunit.client.0.smithi006.stdout:0/159: fdatasync db/dc/d38/f3a 0 2024-03-22T06:14:53.388 INFO:tasks.workunit.client.0.smithi006.stdout:0/160: mknod db/dc/d37/c3b 0 2024-03-22T06:14:53.391 INFO:tasks.workunit.client.0.smithi006.stdout:0/161: mknod db/d10/d33/c3c 0 2024-03-22T06:14:53.394 INFO:tasks.workunit.client.0.smithi006.stdout:0/162: mkdir db/d3d 0 2024-03-22T06:14:53.396 INFO:tasks.workunit.client.0.smithi006.stdout:0/163: mknod db/dc/d37/c3e 0 2024-03-22T06:14:53.401 INFO:tasks.workunit.client.0.smithi006.stdout:6/216: dwrite d6/d24/d44/f4d [0,4194304] 0 2024-03-22T06:14:53.401 INFO:tasks.workunit.client.0.smithi006.stdout:0/164: rename db/dc/l1d to db/d3d/l3f 0 2024-03-22T06:14:53.401 INFO:tasks.workunit.client.0.smithi006.stdout:0/165: dread - db/d10/d2a/f34 zero size 2024-03-22T06:14:53.401 INFO:tasks.workunit.client.0.smithi006.stdout:0/166: chown db/dc/l16 0 1 2024-03-22T06:14:53.401 INFO:tasks.workunit.client.0.smithi006.stdout:3/180: dwrite d0/d4/f2f [0,4194304] 0 2024-03-22T06:14:53.401 INFO:tasks.workunit.client.0.smithi006.stdout:0/167: write db/d10/f14 [5225755,130602] 0 2024-03-22T06:14:53.401 INFO:tasks.workunit.client.0.smithi006.stdout:0/168: chown db/dc 13311913 1 2024-03-22T06:14:53.407 INFO:tasks.workunit.client.0.smithi006.stdout:6/217: creat d6/d24/f54 x:0 0 0 2024-03-22T06:14:53.408 INFO:tasks.workunit.client.0.smithi006.stdout:0/169: symlink db/d10/d2a/l40 0 2024-03-22T06:14:53.415 INFO:tasks.workunit.client.0.smithi006.stdout:6/218: link d6/c35 d6/d1d/d32/d4f/c55 0 2024-03-22T06:14:53.416 INFO:tasks.workunit.client.0.smithi006.stdout:0/170: write db/d10/f14 [1415573,127155] 0 2024-03-22T06:14:53.424 INFO:tasks.workunit.client.0.smithi006.stdout:6/219: dread d6/d1d/f28 [0,4194304] 0 2024-03-22T06:14:53.427 INFO:tasks.workunit.client.0.smithi006.stdout:6/220: mkdir d6/d1d/d32/d4f/d56 0 2024-03-22T06:14:53.427 INFO:tasks.workunit.client.0.smithi006.stdout:6/221: stat d6/d1d/f2b 0 2024-03-22T06:14:53.429 INFO:tasks.workunit.client.0.smithi006.stdout:6/222: mknod d6/d24/c57 0 2024-03-22T06:14:53.432 INFO:tasks.workunit.client.0.smithi006.stdout:6/223: rename d6/d1b/d2c/f52 to d6/d24/f58 0 2024-03-22T06:14:53.433 INFO:tasks.workunit.client.0.smithi006.stdout:0/171: dwrite db/d10/f14 [0,4194304] 0 2024-03-22T06:14:53.433 INFO:tasks.workunit.client.0.smithi006.stdout:0/172: chown db/c29 501452 1 2024-03-22T06:14:53.433 INFO:tasks.workunit.client.0.smithi006.stdout:0/173: write db/d10/d2a/f2c [57485,127841] 0 2024-03-22T06:14:53.435 INFO:tasks.workunit.client.0.smithi006.stdout:6/224: link d6/d1b/f2d d6/d1b/d2c/f59 0 2024-03-22T06:14:53.435 INFO:tasks.workunit.client.0.smithi006.stdout:6/225: dread - d6/d1b/d2c/f4c zero size 2024-03-22T06:14:53.436 INFO:tasks.workunit.client.0.smithi006.stdout:0/174: mkdir db/d10/d2a/d41 0 2024-03-22T06:14:53.438 INFO:tasks.workunit.client.0.smithi006.stdout:6/226: creat d6/d1b/d2c/d4a/f5a x:0 0 0 2024-03-22T06:14:53.439 INFO:tasks.workunit.client.0.smithi006.stdout:6/227: fsync d6/d1d/d32/f45 0 2024-03-22T06:14:53.439 INFO:tasks.workunit.client.0.smithi006.stdout:6/228: fdatasync d6/f40 0 2024-03-22T06:14:53.439 INFO:tasks.workunit.client.0.smithi006.stdout:6/229: read d6/f11 [4140900,59533] 0 2024-03-22T06:14:53.441 INFO:tasks.workunit.client.0.smithi006.stdout:0/175: rename db/d10/d2a/f2c to db/d10/d33/f42 0 2024-03-22T06:14:53.444 INFO:tasks.workunit.client.0.smithi006.stdout:0/176: stat db/d10/l26 0 2024-03-22T06:14:53.452 INFO:tasks.workunit.client.0.smithi006.stdout:6/230: dwrite d6/f51 [0,4194304] 0 2024-03-22T06:14:53.456 INFO:tasks.workunit.client.0.smithi006.stdout:6/231: symlink d6/d1b/d2c/d4a/l5b 0 2024-03-22T06:14:53.458 INFO:tasks.workunit.client.0.smithi006.stdout:6/232: mkdir d6/d1d/d5c 0 2024-03-22T06:14:53.458 INFO:tasks.workunit.client.0.smithi006.stdout:6/233: write d6/d1b/f2d [808934,33607] 0 2024-03-22T06:14:53.461 INFO:tasks.workunit.client.0.smithi006.stdout:0/177: dwrite db/d10/f23 [0,4194304] 0 2024-03-22T06:14:53.461 INFO:tasks.workunit.client.0.smithi006.stdout:0/178: write db/f30 [784388,39840] 0 2024-03-22T06:14:53.461 INFO:tasks.workunit.client.0.smithi006.stdout:0/179: readlink db/d10/d2a/l40 0 2024-03-22T06:14:53.465 INFO:tasks.workunit.client.0.smithi006.stdout:0/180: creat db/d10/d35/f43 x:0 0 0 2024-03-22T06:14:53.465 INFO:tasks.workunit.client.0.smithi006.stdout:0/181: chown db/dc/d37/c3b 196460 1 2024-03-22T06:14:53.476 INFO:tasks.workunit.client.0.smithi006.stdout:0/182: creat db/dc/f44 x:0 0 0 2024-03-22T06:14:53.492 INFO:tasks.workunit.client.0.smithi006.stdout:0/183: dwrite db/d10/d2a/f34 [0,4194304] 0 2024-03-22T06:14:53.493 INFO:tasks.workunit.client.0.smithi006.stdout:0/184: truncate db/d10/d33/f42 1177332 0 2024-03-22T06:14:53.493 INFO:tasks.workunit.client.0.smithi006.stdout:0/185: chown db/d10/c11 4030026 1 2024-03-22T06:14:53.497 INFO:tasks.workunit.client.0.smithi006.stdout:0/186: dread db/dc/d38/f3a [0,4194304] 0 2024-03-22T06:14:53.497 INFO:tasks.workunit.client.0.smithi006.stdout:0/187: chown db/dc/l20 5018828 1 2024-03-22T06:14:53.498 INFO:tasks.workunit.client.0.smithi006.stdout:0/188: write db/d10/f23 [4328663,15220] 0 2024-03-22T06:14:53.498 INFO:tasks.workunit.client.0.smithi006.stdout:0/189: dread - db/d10/d35/f43 zero size 2024-03-22T06:14:53.503 INFO:tasks.workunit.client.0.smithi006.stdout:0/190: dread db/dc/d38/f3a [0,4194304] 0 2024-03-22T06:14:53.504 INFO:tasks.workunit.client.0.smithi006.stdout:6/234: rmdir d6/d1b 39 2024-03-22T06:14:53.505 INFO:tasks.workunit.client.0.smithi006.stdout:6/235: read d6/d24/f38 [481471,48167] 0 2024-03-22T06:14:53.506 INFO:tasks.workunit.client.0.smithi006.stdout:0/191: creat db/f45 x:0 0 0 2024-03-22T06:14:53.506 INFO:tasks.workunit.client.0.smithi006.stdout:0/192: dread - db/f45 zero size 2024-03-22T06:14:53.508 INFO:tasks.workunit.client.0.smithi006.stdout:6/236: creat d6/d1d/d32/d4f/f5d x:0 0 0 2024-03-22T06:14:53.510 INFO:tasks.workunit.client.0.smithi006.stdout:0/193: symlink db/l46 0 2024-03-22T06:14:53.510 INFO:tasks.workunit.client.0.smithi006.stdout:0/194: readlink db/l1c 0 2024-03-22T06:14:53.513 INFO:tasks.workunit.client.0.smithi006.stdout:6/237: getdents d6/d1b/d2c 0 2024-03-22T06:14:53.516 INFO:tasks.workunit.client.0.smithi006.stdout:6/238: creat d6/d1d/d5c/f5e x:0 0 0 2024-03-22T06:14:53.518 INFO:tasks.workunit.client.0.smithi006.stdout:6/239: rename d6/d1b/f30 to d6/d24/d44/f5f 0 2024-03-22T06:14:53.519 INFO:tasks.workunit.client.0.smithi006.stdout:6/240: write d6/d1b/f2a [855197,15246] 0 2024-03-22T06:14:53.519 INFO:tasks.workunit.client.0.smithi006.stdout:6/241: write d6/d1b/f2d [674725,91487] 0 2024-03-22T06:14:53.521 INFO:tasks.workunit.client.0.smithi006.stdout:6/242: mknod d6/d1d/d32/d4f/c60 0 2024-03-22T06:14:53.524 INFO:tasks.workunit.client.0.smithi006.stdout:6/243: mknod d6/d1b/d2c/d4a/c61 0 2024-03-22T06:14:53.529 INFO:tasks.workunit.client.0.smithi006.stdout:6/244: dread d6/f42 [0,4194304] 0 2024-03-22T06:14:53.529 INFO:tasks.workunit.client.0.smithi006.stdout:6/245: creat d6/d24/d44/f62 x:0 0 0 2024-03-22T06:14:53.532 INFO:tasks.workunit.client.0.smithi006.stdout:6/246: rmdir d6/d1b/d2c 39 2024-03-22T06:14:53.535 INFO:tasks.workunit.client.0.smithi006.stdout:6/247: mknod d6/c63 0 2024-03-22T06:14:53.535 INFO:tasks.workunit.client.0.smithi006.stdout:6/248: readlink d6/d24/l25 0 2024-03-22T06:14:53.544 INFO:tasks.workunit.client.0.smithi006.stdout:6/249: dread d6/f16 [0,4194304] 0 2024-03-22T06:14:53.546 INFO:tasks.workunit.client.0.smithi006.stdout:6/250: mkdir d6/d1d/d64 0 2024-03-22T06:14:53.546 INFO:tasks.workunit.client.0.smithi006.stdout:6/251: write d6/d1b/f2d [23439,72997] 0 2024-03-22T06:14:53.546 INFO:tasks.workunit.client.0.smithi006.stdout:6/252: truncate d6/d1b/d2c/f43 855763 0 2024-03-22T06:14:53.549 INFO:tasks.workunit.client.0.smithi006.stdout:0/195: dwrite db/d10/f2f [0,4194304] 0 2024-03-22T06:14:53.549 INFO:tasks.workunit.client.0.smithi006.stdout:0/196: chown db/dc/d37/c3b 7106 1 2024-03-22T06:14:53.549 INFO:tasks.workunit.client.0.smithi006.stdout:0/197: getdents db/d10/d2a/d41 0 2024-03-22T06:14:53.549 INFO:tasks.workunit.client.0.smithi006.stdout:0/198: fsync db/d10/f14 0 2024-03-22T06:14:53.549 INFO:tasks.workunit.client.0.smithi006.stdout:6/253: truncate d6/d24/f31 3968717 0 2024-03-22T06:14:53.549 INFO:tasks.workunit.client.0.smithi006.stdout:6/254: creat d6/d1d/d32/d4f/f65 x:0 0 0 2024-03-22T06:14:53.552 INFO:tasks.workunit.client.0.smithi006.stdout:0/199: mknod db/dc/d37/c47 0 2024-03-22T06:14:53.558 INFO:tasks.workunit.client.0.smithi006.stdout:6/255: link d6/d1d/f20 d6/f66 0 2024-03-22T06:14:53.558 INFO:tasks.workunit.client.0.smithi006.stdout:6/256: write d6/f13 [5241011,98252] 0 2024-03-22T06:14:53.558 INFO:tasks.workunit.client.0.smithi006.stdout:6/257: write d6/f11 [2396402,92785] 0 2024-03-22T06:14:53.558 INFO:tasks.workunit.client.0.smithi006.stdout:6/258: creat d6/d1d/d32/f67 x:0 0 0 2024-03-22T06:14:53.559 INFO:tasks.workunit.client.0.smithi006.stdout:6/259: write d6/d1b/f48 [1428239,116222] 0 2024-03-22T06:14:53.562 INFO:tasks.workunit.client.0.smithi006.stdout:0/200: write db/d10/d2a/f34 [3221431,111153] 0 2024-03-22T06:14:53.574 INFO:tasks.workunit.client.0.smithi006.stdout:6/260: symlink d6/d1d/d32/d4f/l68 0 2024-03-22T06:14:53.585 INFO:tasks.workunit.client.0.smithi006.stdout:0/201: symlink db/d10/l48 0 2024-03-22T06:14:53.585 INFO:tasks.workunit.client.0.smithi006.stdout:0/202: fsync db/dc/d38/f3a 0 2024-03-22T06:14:53.588 INFO:tasks.workunit.client.0.smithi006.stdout:0/203: mknod db/dc/d37/c49 0 2024-03-22T06:14:53.591 INFO:tasks.workunit.client.0.smithi006.stdout:0/204: getdents db/dc/d38 0 2024-03-22T06:14:53.594 INFO:tasks.workunit.client.0.smithi006.stdout:0/205: mkdir db/d4a 0 2024-03-22T06:14:53.594 INFO:tasks.workunit.client.0.smithi006.stdout:0/206: write db/d10/f1a [3011547,126072] 0 2024-03-22T06:14:53.653 INFO:tasks.workunit.client.0.smithi006.stdout:9/159: sync 2024-03-22T06:14:53.654 INFO:tasks.workunit.client.0.smithi006.stdout:1/152: sync 2024-03-22T06:14:53.654 INFO:tasks.workunit.client.0.smithi006.stdout:7/206: sync 2024-03-22T06:14:53.654 INFO:tasks.workunit.client.0.smithi006.stdout:4/240: sync 2024-03-22T06:14:53.654 INFO:tasks.workunit.client.0.smithi006.stdout:7/207: read - d5/d1e/d29/d1f/f38 zero size 2024-03-22T06:14:53.654 INFO:tasks.workunit.client.0.smithi006.stdout:7/208: stat d5/d1e/d29/d10 0 2024-03-22T06:14:53.658 INFO:tasks.workunit.client.0.smithi006.stdout:9/160: dread f4 [0,4194304] 0 2024-03-22T06:14:53.659 INFO:tasks.workunit.client.0.smithi006.stdout:1/153: mkdir d8/d9/d13/d24/d32/d34 0 2024-03-22T06:14:53.662 INFO:tasks.workunit.client.0.smithi006.stdout:7/209: write d5/d1e/f36 [1495570,130452] 0 2024-03-22T06:14:53.670 INFO:tasks.workunit.client.0.smithi006.stdout:7/210: dread d5/d1e/d29/f2a [0,4194304] 0 2024-03-22T06:14:53.670 INFO:tasks.workunit.client.0.smithi006.stdout:7/211: fdatasync f2 0 2024-03-22T06:14:53.670 INFO:tasks.workunit.client.0.smithi006.stdout:7/212: chown f0 1288903 1 2024-03-22T06:14:53.766 INFO:tasks.workunit.client.0.smithi006.stdout:4/241: mknod d6/d1b/d20/d2c/c53 0 2024-03-22T06:14:53.766 INFO:tasks.workunit.client.0.smithi006.stdout:1/154: link d8/d9/d10/d14/f20 d8/d9/f35 0 2024-03-22T06:14:53.768 INFO:tasks.workunit.client.0.smithi006.stdout:7/213: creat d5/d1e/d29/d1f/d2d/d41/f42 x:0 0 0 2024-03-22T06:14:53.769 INFO:tasks.workunit.client.0.smithi006.stdout:4/242: rmdir d6/d1b/d20 39 2024-03-22T06:14:53.772 INFO:tasks.workunit.client.0.smithi006.stdout:7/214: mknod d5/d1e/d29/d1f/d2d/c43 0 2024-03-22T06:14:53.773 INFO:tasks.workunit.client.0.smithi006.stdout:7/215: chown d5/c16 3 1 2024-03-22T06:14:53.773 INFO:tasks.workunit.client.0.smithi006.stdout:7/216: creat d5/d1e/d29/d1f/d2d/d41/f44 x:0 0 0 2024-03-22T06:14:53.773 INFO:tasks.workunit.client.0.smithi006.stdout:4/243: stat d6/l14 0 2024-03-22T06:14:53.775 INFO:tasks.workunit.client.0.smithi006.stdout:1/155: getdents d8/d9/d13 0 2024-03-22T06:14:53.775 INFO:tasks.workunit.client.0.smithi006.stdout:1/156: write d8/d9/d10/d14/d18/f1b [1155128,121245] 0 2024-03-22T06:14:53.778 INFO:tasks.workunit.client.0.smithi006.stdout:7/217: creat d5/d1e/d3f/f45 x:0 0 0 2024-03-22T06:14:53.780 INFO:tasks.workunit.client.0.smithi006.stdout:4/244: link d6/d1b/d20/f22 d6/d1b/f54 0 2024-03-22T06:14:53.780 INFO:tasks.workunit.client.0.smithi006.stdout:4/245: chown d6/d1b/d20/d2c/c53 1791775 1 2024-03-22T06:14:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:53 smithi006 ceph-mon[29301]: pgmap v10: 65 pgs: 65 active+clean; 350 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 23 MiB/s rd, 41 MiB/s wr, 369 op/s 2024-03-22T06:14:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:53 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:53 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:53 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:53 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:53 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:53 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:53 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:53 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:53.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:53 smithi006 ceph-mon[29301]: Adjusting osd_memory_target on smithi117 to 2581M 2024-03-22T06:14:53.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:53 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:53.783 INFO:tasks.workunit.client.0.smithi006.stdout:7/218: rename d5/d1e/d29/d10/c15 to d5/d1e/d29/c46 0 2024-03-22T06:14:53.783 INFO:tasks.workunit.client.0.smithi006.stdout:7/219: readlink d5/l23 0 2024-03-22T06:14:53.788 INFO:tasks.workunit.client.0.smithi006.stdout:7/220: rename f4 to d5/d1e/d29/d1f/d2d/f47 0 2024-03-22T06:14:53.788 INFO:tasks.workunit.client.0.smithi006.stdout:7/221: read d5/fd [3316332,25470] 0 2024-03-22T06:14:53.788 INFO:tasks.workunit.client.0.smithi006.stdout:7/222: write d5/d1e/f21 [356356,110120] 0 2024-03-22T06:14:53.791 INFO:tasks.workunit.client.0.smithi006.stdout:7/223: link d5/d1e/f31 d5/d1e/d3f/f48 0 2024-03-22T06:14:53.791 INFO:tasks.workunit.client.0.smithi006.stdout:7/224: stat d5/cc 0 2024-03-22T06:14:53.793 INFO:tasks.workunit.client.0.smithi006.stdout:4/246: dwrite d6/f7 [0,4194304] 0 2024-03-22T06:14:53.795 INFO:tasks.workunit.client.0.smithi006.stdout:7/225: creat d5/f49 x:0 0 0 2024-03-22T06:14:53.799 INFO:tasks.workunit.client.0.smithi006.stdout:4/247: symlink d6/d1b/d20/d21/d2a/d43/d4b/l55 0 2024-03-22T06:14:53.803 INFO:tasks.workunit.client.0.smithi006.stdout:4/248: symlink d6/d13/l56 0 2024-03-22T06:14:53.803 INFO:tasks.workunit.client.0.smithi006.stdout:4/249: truncate d6/d1b/f54 1510625 0 2024-03-22T06:14:53.806 INFO:tasks.workunit.client.0.smithi006.stdout:1/157: getdents d8/d9/d13/d24/d32 0 2024-03-22T06:14:53.809 INFO:tasks.workunit.client.0.smithi006.stdout:4/250: dread d6/fa [0,4194304] 0 2024-03-22T06:14:53.810 INFO:tasks.workunit.client.0.smithi006.stdout:1/158: rename d8/d9/fd to d8/d9/d13/d24/d32/d34/f36 0 2024-03-22T06:14:53.812 INFO:tasks.workunit.client.0.smithi006.stdout:7/226: dwrite d5/d1e/d29/f13 [0,4194304] 0 2024-03-22T06:14:53.812 INFO:tasks.workunit.client.0.smithi006.stdout:7/227: write d5/d1e/d29/d1f/f3e [404937,67607] 0 2024-03-22T06:14:53.815 INFO:tasks.workunit.client.0.smithi006.stdout:1/159: creat d8/d9/d10/d14/d18/f37 x:0 0 0 2024-03-22T06:14:53.815 INFO:tasks.workunit.client.0.smithi006.stdout:1/160: chown d8/d9/l2e 33862405 1 2024-03-22T06:14:53.815 INFO:tasks.workunit.client.0.smithi006.stdout:1/161: stat d8/d9/d10/d14/d22/l2f 0 2024-03-22T06:14:53.823 INFO:tasks.workunit.client.0.smithi006.stdout:1/162: dread d8/d9/d13/d24/d32/d34/f36 [0,4194304] 0 2024-03-22T06:14:53.823 INFO:tasks.workunit.client.0.smithi006.stdout:1/163: getdents d8/d33 0 2024-03-22T06:14:53.823 INFO:tasks.workunit.client.0.smithi006.stdout:1/164: fdatasync d8/d9/d13/d24/d32/d34/f36 0 2024-03-22T06:14:53.824 INFO:tasks.workunit.client.0.smithi006.stdout:1/165: chown d8/c31 3 1 2024-03-22T06:14:53.824 INFO:tasks.workunit.client.0.smithi006.stdout:1/166: stat d8/d9/c23 0 2024-03-22T06:14:53.824 INFO:tasks.workunit.client.0.smithi006.stdout:7/228: creat d5/d1e/d29/f4a x:0 0 0 2024-03-22T06:14:53.835 INFO:tasks.workunit.client.0.smithi006.stdout:7/229: dread d5/d1e/f3a [0,4194304] 0 2024-03-22T06:14:53.836 INFO:tasks.workunit.client.0.smithi006.stdout:4/251: getdents d6/d13 0 2024-03-22T06:14:53.836 INFO:tasks.workunit.client.0.smithi006.stdout:4/252: write d6/d13/d3b/f4e [84020,50339] 0 2024-03-22T06:14:53.838 INFO:tasks.workunit.client.0.smithi006.stdout:7/230: rename d5/d1e/f36 to d5/d1e/d3f/f4b 0 2024-03-22T06:14:53.840 INFO:tasks.workunit.client.0.smithi006.stdout:4/253: link d6/d13/l16 d6/d1b/d20/d21/d2a/l57 0 2024-03-22T06:14:53.842 INFO:tasks.workunit.client.0.smithi006.stdout:4/254: creat d6/d1b/d46/f58 x:0 0 0 2024-03-22T06:14:53.842 INFO:tasks.workunit.client.0.smithi006.stdout:4/255: creat d6/f59 x:0 0 0 2024-03-22T06:14:53.844 INFO:tasks.workunit.client.0.smithi006.stdout:1/167: dwrite d8/d9/d13/d24/d32/d34/f36 [0,4194304] 0 2024-03-22T06:14:53.844 INFO:tasks.workunit.client.0.smithi006.stdout:1/168: write d8/d9/d10/d14/d18/f1b [2334793,57327] 0 2024-03-22T06:14:53.844 INFO:tasks.workunit.client.0.smithi006.stdout:1/169: dread - d8/d9/f2d zero size 2024-03-22T06:14:53.848 INFO:tasks.workunit.client.0.smithi006.stdout:7/231: write d5/f9 [3152725,94773] 0 2024-03-22T06:14:53.854 INFO:tasks.workunit.client.0.smithi006.stdout:1/170: dread d8/d9/f1e [0,4194304] 0 2024-03-22T06:14:53.869 INFO:tasks.workunit.client.0.smithi006.stdout:4/256: dwrite d6/f42 [0,4194304] 0 2024-03-22T06:14:53.869 INFO:tasks.workunit.client.0.smithi006.stdout:4/257: creat d6/d1b/d46/f5a x:0 0 0 2024-03-22T06:14:53.869 INFO:tasks.workunit.client.0.smithi006.stdout:4/258: creat d6/d1b/f5b x:0 0 0 2024-03-22T06:14:53.869 INFO:tasks.workunit.client.0.smithi006.stdout:7/232: dwrite d5/d1e/d3f/f45 [0,4194304] 0 2024-03-22T06:14:53.869 INFO:tasks.workunit.client.0.smithi006.stdout:7/233: fsync d5/d1e/d3f/f45 0 2024-03-22T06:14:53.869 INFO:tasks.workunit.client.0.smithi006.stdout:7/234: chown d5/d1e/d29/d1f/d2d/d41/f44 48538287 1 2024-03-22T06:14:53.869 INFO:tasks.workunit.client.0.smithi006.stdout:7/235: write d5/fd [4363824,108012] 0 2024-03-22T06:14:53.869 INFO:tasks.workunit.client.0.smithi006.stdout:7/236: chown d5/d1e 5 1 2024-03-22T06:14:53.871 INFO:tasks.workunit.client.0.smithi006.stdout:8/245: sync 2024-03-22T06:14:53.873 INFO:tasks.workunit.client.0.smithi006.stdout:4/259: write d6/fa [4067796,26985] 0 2024-03-22T06:14:53.874 INFO:tasks.workunit.client.0.smithi006.stdout:8/246: dread d9/db/d37/f41 [0,4194304] 0 2024-03-22T06:14:53.875 INFO:tasks.workunit.client.0.smithi006.stdout:8/247: fdatasync f7 0 2024-03-22T06:14:53.875 INFO:tasks.workunit.client.0.smithi006.stdout:8/248: stat d9/d14/d47/l4e 0 2024-03-22T06:14:53.877 INFO:tasks.workunit.client.0.smithi006.stdout:8/249: dread d9/db/dc/de/f22 [0,4194304] 0 2024-03-22T06:14:53.883 INFO:tasks.workunit.client.0.smithi006.stdout:7/237: creat d5/d1e/d29/d1f/d2d/f4c x:0 0 0 2024-03-22T06:14:53.885 INFO:tasks.workunit.client.0.smithi006.stdout:4/260: mkdir d6/d1b/d20/d21/d2a/d5c 0 2024-03-22T06:14:53.886 INFO:tasks.workunit.client.0.smithi006.stdout:1/171: dwrite d8/d9/d10/d14/d18/f1b [0,4194304] 0 2024-03-22T06:14:53.887 INFO:tasks.workunit.client.0.smithi006.stdout:1/172: write d8/d9/d10/d14/f20 [7421956,72971] 0 2024-03-22T06:14:53.892 INFO:tasks.workunit.client.0.smithi006.stdout:8/250: rename d9/db/dc/de/f53 to d9/db/dc/f62 0 2024-03-22T06:14:53.899 INFO:tasks.workunit.client.0.smithi006.stdout:7/238: symlink d5/d1e/l4d 0 2024-03-22T06:14:53.901 INFO:tasks.workunit.client.0.smithi006.stdout:4/261: symlink d6/d1b/d20/d21/d2a/d43/d4b/l5d 0 2024-03-22T06:14:53.901 INFO:tasks.workunit.client.0.smithi006.stdout:4/262: fsync d6/f4a 0 2024-03-22T06:14:53.901 INFO:tasks.workunit.client.0.smithi006.stdout:4/263: fdatasync d6/d1b/d20/d21/d2a/f3f 0 2024-03-22T06:14:53.906 INFO:tasks.workunit.client.0.smithi006.stdout:8/251: creat d9/db/d35/d54/f63 x:0 0 0 2024-03-22T06:14:53.912 INFO:tasks.workunit.client.0.smithi006.stdout:4/264: mkdir d6/d1b/d5e 0 2024-03-22T06:14:53.912 INFO:tasks.workunit.client.0.smithi006.stdout:1/173: getdents d8/d9/d10 0 2024-03-22T06:14:53.916 INFO:tasks.workunit.client.0.smithi006.stdout:4/265: rename d6/d1b/d20/d21/l2b to d6/d1b/l5f 0 2024-03-22T06:14:53.919 INFO:tasks.workunit.client.0.smithi006.stdout:8/252: dwrite d9/d14/d47/f5c [0,4194304] 0 2024-03-22T06:14:53.919 INFO:tasks.workunit.client.0.smithi006.stdout:8/253: write d9/db/dc/de/d3e/f56 [654756,64479] 0 2024-03-22T06:14:53.919 INFO:tasks.workunit.client.0.smithi006.stdout:8/254: creat d9/d14/d47/f64 x:0 0 0 2024-03-22T06:14:53.919 INFO:tasks.workunit.client.0.smithi006.stdout:8/255: write d9/db/dc/f1f [580697,126731] 0 2024-03-22T06:14:53.919 INFO:tasks.workunit.client.0.smithi006.stdout:4/266: mkdir d6/d1b/d5e/d60 0 2024-03-22T06:14:53.923 INFO:tasks.workunit.client.0.smithi006.stdout:1/174: dread d8/d9/f1f [0,4194304] 0 2024-03-22T06:14:53.926 INFO:tasks.workunit.client.0.smithi006.stdout:8/256: truncate d9/db/dc/de/f16 914557 0 2024-03-22T06:14:53.927 INFO:tasks.workunit.client.0.smithi006.stdout:1/175: mknod d8/d9/d13/d24/c38 0 2024-03-22T06:14:53.929 INFO:tasks.workunit.client.0.smithi006.stdout:8/257: mknod d9/d5e/c65 0 2024-03-22T06:14:53.938 INFO:tasks.workunit.client.0.smithi006.stdout:4/267: dwrite d6/fb [4194304,4194304] 0 2024-03-22T06:14:53.938 INFO:tasks.workunit.client.0.smithi006.stdout:4/268: write d6/f4a [860081,90049] 0 2024-03-22T06:14:53.938 INFO:tasks.workunit.client.0.smithi006.stdout:4/269: read - d6/d1b/f5b zero size 2024-03-22T06:14:53.938 INFO:tasks.workunit.client.0.smithi006.stdout:4/270: truncate d6/d1b/d46/f5a 410377 0 2024-03-22T06:14:53.938 INFO:tasks.workunit.client.0.smithi006.stdout:7/239: truncate d5/fd 1413770 0 2024-03-22T06:14:53.939 INFO:tasks.workunit.client.0.smithi006.stdout:8/258: dread d9/d14/d47/f5c [0,4194304] 0 2024-03-22T06:14:53.940 INFO:tasks.workunit.client.0.smithi006.stdout:4/271: symlink d6/l61 0 2024-03-22T06:14:53.945 INFO:tasks.workunit.client.0.smithi006.stdout:4/272: rename d6/f3d to d6/f62 0 2024-03-22T06:14:53.945 INFO:tasks.workunit.client.0.smithi006.stdout:4/273: chown d6/d1b/f47 611 1 2024-03-22T06:14:53.947 INFO:tasks.workunit.client.0.smithi006.stdout:8/259: getdents d9/db/dc/de/d3e 0 2024-03-22T06:14:53.949 INFO:tasks.workunit.client.0.smithi006.stdout:8/260: mkdir d9/db/d12/d66 0 2024-03-22T06:14:53.951 INFO:tasks.workunit.client.0.smithi006.stdout:1/176: dwrite d8/d9/d10/d14/f20 [4194304,4194304] 0 2024-03-22T06:14:53.951 INFO:tasks.workunit.client.0.smithi006.stdout:1/177: write d8/d9/d10/d14/f20 [2919201,89437] 0 2024-03-22T06:14:53.953 INFO:tasks.workunit.client.0.smithi006.stdout:8/261: getdents d9/db/d12/d1e 0 2024-03-22T06:14:53.956 INFO:tasks.workunit.client.0.smithi006.stdout:4/274: write f2 [1717195,53167] 0 2024-03-22T06:14:53.969 INFO:tasks.workunit.client.0.smithi006.stdout:1/178: mkdir d8/d9/d13/d39 0 2024-03-22T06:14:53.974 INFO:tasks.workunit.client.0.smithi006.stdout:1/179: rename d8/d9/d10/d14 to d8/d9/d3a 0 2024-03-22T06:14:53.976 INFO:tasks.workunit.client.0.smithi006.stdout:8/262: dwrite d9/db/d12/f26 [0,4194304] 0 2024-03-22T06:14:53.976 INFO:tasks.workunit.client.0.smithi006.stdout:4/275: dwrite d6/d1b/f5b [0,4194304] 0 2024-03-22T06:14:53.978 INFO:tasks.workunit.client.0.smithi006.stdout:1/180: write d8/d9/d10/f19 [50758,107405] 0 2024-03-22T06:14:53.981 INFO:tasks.workunit.client.0.smithi006.stdout:8/263: link d9/c32 d9/db/d12/d66/c67 0 2024-03-22T06:14:53.992 INFO:tasks.workunit.client.0.smithi006.stdout:4/276: mkdir d6/d1b/d20/d63 0 2024-03-22T06:14:53.995 INFO:tasks.workunit.client.0.smithi006.stdout:8/264: dwrite f7 [4194304,4194304] 0 2024-03-22T06:14:53.996 INFO:tasks.workunit.client.0.smithi006.stdout:1/181: rename d8/d9/lb to d8/d9/l3b 0 2024-03-22T06:14:53.999 INFO:tasks.workunit.client.0.smithi006.stdout:7/240: unlink d5/d1e/d29/d1f/d2d/f3c 0 2024-03-22T06:14:54.000 INFO:tasks.workunit.client.0.smithi006.stdout:4/277: rename d6/d1b/f29 to d6/d1b/d5e/f64 0 2024-03-22T06:14:54.004 INFO:tasks.workunit.client.0.smithi006.stdout:1/182: dread d8/d9/d3a/f20 [0,4194304] 0 2024-03-22T06:14:54.005 INFO:tasks.workunit.client.0.smithi006.stdout:7/241: symlink d5/d1e/d29/d1f/l4e 0 2024-03-22T06:14:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:53 smithi117 ceph-mon[36514]: pgmap v10: 65 pgs: 65 active+clean; 350 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 23 MiB/s rd, 41 MiB/s wr, 369 op/s 2024-03-22T06:14:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:53 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:53 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:53 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:53 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:53 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:53 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:53 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:53 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:14:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:53 smithi117 ceph-mon[36514]: Adjusting osd_memory_target on smithi117 to 2581M 2024-03-22T06:14:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:53 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:14:54.008 INFO:tasks.workunit.client.0.smithi006.stdout:8/265: rename d9/db/d12/l48 to d9/db/dc/l68 0 2024-03-22T06:14:54.008 INFO:tasks.workunit.client.0.smithi006.stdout:8/266: creat d9/d5e/f69 x:0 0 0 2024-03-22T06:14:54.009 INFO:tasks.workunit.client.0.smithi006.stdout:4/278: write d6/d1b/d20/f24 [1228283,108057] 0 2024-03-22T06:14:54.010 INFO:tasks.workunit.client.0.smithi006.stdout:7/242: mknod d5/c4f 0 2024-03-22T06:14:54.010 INFO:tasks.workunit.client.0.smithi006.stdout:7/243: chown f2 14 1 2024-03-22T06:14:54.012 INFO:tasks.workunit.client.0.smithi006.stdout:1/183: rename d8/d9/ca to d8/d9/d3a/d18/c3c 0 2024-03-22T06:14:54.012 INFO:tasks.workunit.client.0.smithi006.stdout:1/184: chown d8/d9/d3a/d22/l2f 530 1 2024-03-22T06:14:54.017 INFO:tasks.workunit.client.0.smithi006.stdout:7/244: read d5/d1e/d29/d10/f25 [3476974,53242] 0 2024-03-22T06:14:54.017 INFO:tasks.workunit.client.0.smithi006.stdout:7/245: write d5/f28 [695456,82143] 0 2024-03-22T06:14:54.017 INFO:tasks.workunit.client.0.smithi006.stdout:7/246: write d5/d1e/d29/d1f/d2d/d41/f42 [786453,100563] 0 2024-03-22T06:14:54.018 INFO:tasks.workunit.client.0.smithi006.stdout:8/267: unlink d9/d14/f2b 0 2024-03-22T06:14:54.025 INFO:tasks.workunit.client.0.smithi006.stdout:4/279: symlink d6/d1b/d20/l65 0 2024-03-22T06:14:54.028 INFO:tasks.workunit.client.0.smithi006.stdout:8/268: symlink d9/db/dc/de/l6a 0 2024-03-22T06:14:54.028 INFO:tasks.workunit.client.0.smithi006.stdout:8/269: chown d9/db/dc/de/d3e/f3f 2364973 1 2024-03-22T06:14:54.029 INFO:tasks.workunit.client.0.smithi006.stdout:4/280: creat d6/d1b/d20/d21/d2a/d5c/f66 x:0 0 0 2024-03-22T06:14:54.031 INFO:tasks.workunit.client.0.smithi006.stdout:8/270: creat d9/f6b x:0 0 0 2024-03-22T06:14:54.035 INFO:tasks.workunit.client.0.smithi006.stdout:1/185: dwrite d8/d9/f1f [4194304,4194304] 0 2024-03-22T06:14:54.035 INFO:tasks.workunit.client.0.smithi006.stdout:8/271: truncate d9/f27 2337900 0 2024-03-22T06:14:54.035 INFO:tasks.workunit.client.0.smithi006.stdout:7/247: dwrite d5/f1d [0,4194304] 0 2024-03-22T06:14:54.039 INFO:tasks.workunit.client.0.smithi006.stdout:1/186: truncate d8/d9/d13/d24/d32/d34/f36 28794 0 2024-03-22T06:14:54.040 INFO:tasks.workunit.client.0.smithi006.stdout:1/187: chown l4 3803223 1 2024-03-22T06:14:54.045 INFO:tasks.workunit.client.0.smithi006.stdout:7/248: mknod d5/d1e/c50 0 2024-03-22T06:14:54.045 INFO:tasks.workunit.client.0.smithi006.stdout:7/249: write d5/d1e/d29/d1f/d2d/d41/f42 [318166,84764] 0 2024-03-22T06:14:54.047 INFO:tasks.workunit.client.0.smithi006.stdout:1/188: symlink d8/d9/d3a/d18/d2a/l3d 0 2024-03-22T06:14:54.048 INFO:tasks.workunit.client.0.smithi006.stdout:1/189: fsync d8/d9/f1e 0 2024-03-22T06:14:54.048 INFO:tasks.workunit.client.0.smithi006.stdout:1/190: chown d8/d9/d3a/d22/l29 248 1 2024-03-22T06:14:54.048 INFO:tasks.workunit.client.0.smithi006.stdout:1/191: chown d8/d9/d3a/d22 1 1 2024-03-22T06:14:54.057 INFO:tasks.workunit.client.0.smithi006.stdout:8/272: rename d9/db/d35/d54 to d9/db/d12/d66/d6c 0 2024-03-22T06:14:54.060 INFO:tasks.workunit.client.0.smithi006.stdout:1/192: dwrite d8/d9/f2d [0,4194304] 0 2024-03-22T06:14:54.069 INFO:tasks.workunit.client.0.smithi006.stdout:8/273: creat d9/db/d12/d1e/f6d x:0 0 0 2024-03-22T06:14:54.072 INFO:tasks.workunit.client.0.smithi006.stdout:4/281: rmdir d6/d1b/d20/d21/d2a/d5c 39 2024-03-22T06:14:54.076 INFO:tasks.workunit.client.0.smithi006.stdout:1/193: mknod d8/d9/d13/d24/d32/d34/c3e 0 2024-03-22T06:14:54.078 INFO:tasks.workunit.client.0.smithi006.stdout:8/274: symlink d9/db/d12/d66/d6c/l6e 0 2024-03-22T06:14:54.078 INFO:tasks.workunit.client.0.smithi006.stdout:8/275: creat d9/db/d35/f6f x:0 0 0 2024-03-22T06:14:54.078 INFO:tasks.workunit.client.0.smithi006.stdout:8/276: creat d9/d14/d47/f70 x:0 0 0 2024-03-22T06:14:54.083 INFO:tasks.workunit.client.0.smithi006.stdout:4/282: creat d6/d1b/d5e/d60/f67 x:0 0 0 2024-03-22T06:14:54.083 INFO:tasks.workunit.client.0.smithi006.stdout:4/283: write f1 [400948,124848] 0 2024-03-22T06:14:54.083 INFO:tasks.workunit.client.0.smithi006.stdout:4/284: creat d6/d1b/d5e/d60/f68 x:0 0 0 2024-03-22T06:14:54.083 INFO:tasks.workunit.client.0.smithi006.stdout:1/194: link d8/d9/d13/l25 d8/d9/d13/l3f 0 2024-03-22T06:14:54.088 INFO:tasks.workunit.client.0.smithi006.stdout:8/277: mkdir d9/db/d12/d66/d6c/d71 0 2024-03-22T06:14:54.088 INFO:tasks.workunit.client.0.smithi006.stdout:8/278: read - d9/d14/f34 zero size 2024-03-22T06:14:54.091 INFO:tasks.workunit.client.0.smithi006.stdout:7/250: rmdir d5 39 2024-03-22T06:14:54.093 INFO:tasks.workunit.client.0.smithi006.stdout:4/285: rmdir d6/d1b/d20/d21/d2a/d5c 39 2024-03-22T06:14:54.093 INFO:tasks.workunit.client.0.smithi006.stdout:4/286: fsync d6/f9 0 2024-03-22T06:14:54.096 INFO:tasks.workunit.client.0.smithi006.stdout:7/251: unlink d5/d1e/d29/d10/l32 0 2024-03-22T06:14:54.100 INFO:tasks.workunit.client.0.smithi006.stdout:7/252: creat d5/d1e/d29/d1f/d2d/f51 x:0 0 0 2024-03-22T06:14:54.104 INFO:tasks.workunit.client.0.smithi006.stdout:7/253: unlink d5/d1e/f21 0 2024-03-22T06:14:54.106 INFO:tasks.workunit.client.0.smithi006.stdout:1/195: dwrite d8/d9/d13/f26 [0,4194304] 0 2024-03-22T06:14:54.114 INFO:tasks.workunit.client.0.smithi006.stdout:8/279: dwrite d9/db/dc/f1f [0,4194304] 0 2024-03-22T06:14:54.114 INFO:tasks.workunit.client.0.smithi006.stdout:4/287: dwrite d6/f30 [0,4194304] 0 2024-03-22T06:14:54.118 INFO:tasks.workunit.client.0.smithi006.stdout:1/196: dread d8/d9/d10/f19 [0,4194304] 0 2024-03-22T06:14:54.121 INFO:tasks.workunit.client.0.smithi006.stdout:4/288: rename d6/d13/l35 to d6/d1b/d5e/d60/l69 0 2024-03-22T06:14:54.122 INFO:tasks.workunit.client.0.smithi006.stdout:7/254: dwrite d5/d1e/d29/d1f/d2d/f4c [0,4194304] 0 2024-03-22T06:14:54.126 INFO:tasks.workunit.client.0.smithi006.stdout:4/289: creat d6/d13/f6a x:0 0 0 2024-03-22T06:14:54.127 INFO:tasks.workunit.client.0.smithi006.stdout:4/290: read - d6/f59 zero size 2024-03-22T06:14:54.143 INFO:tasks.workunit.client.0.smithi006.stdout:1/197: dwrite d8/d9/d13/d24/d32/d34/f36 [0,4194304] 0 2024-03-22T06:14:54.143 INFO:tasks.workunit.client.0.smithi006.stdout:7/255: dwrite d5/d1e/d29/f13 [0,4194304] 0 2024-03-22T06:14:54.143 INFO:tasks.workunit.client.0.smithi006.stdout:1/198: getdents d8/d33 0 2024-03-22T06:14:54.144 INFO:tasks.workunit.client.0.smithi006.stdout:7/256: read d5/d1e/d3f/f4b [2669735,114542] 0 2024-03-22T06:14:54.144 INFO:tasks.workunit.client.0.smithi006.stdout:1/199: mkdir d8/d9/d3a/d40 0 2024-03-22T06:14:54.145 INFO:tasks.workunit.client.0.smithi006.stdout:7/257: mknod d5/c52 0 2024-03-22T06:14:54.152 INFO:tasks.workunit.client.0.smithi006.stdout:1/200: read d8/d9/f2d [112236,103508] 0 2024-03-22T06:14:54.152 INFO:tasks.workunit.client.0.smithi006.stdout:1/201: truncate d8/d9/d10/f19 1079688 0 2024-03-22T06:14:54.154 INFO:tasks.workunit.client.0.smithi006.stdout:1/202: creat d8/d9/d13/d24/d32/f41 x:0 0 0 2024-03-22T06:14:54.156 INFO:tasks.workunit.client.0.smithi006.stdout:5/304: sync 2024-03-22T06:14:54.156 INFO:tasks.workunit.client.0.smithi006.stdout:2/267: sync 2024-03-22T06:14:54.156 INFO:tasks.workunit.client.0.smithi006.stdout:9/161: sync 2024-03-22T06:14:54.156 INFO:tasks.workunit.client.0.smithi006.stdout:3/181: sync 2024-03-22T06:14:54.157 INFO:tasks.workunit.client.0.smithi006.stdout:0/207: sync 2024-03-22T06:14:54.157 INFO:tasks.workunit.client.0.smithi006.stdout:6/261: sync 2024-03-22T06:14:54.157 INFO:tasks.workunit.client.0.smithi006.stdout:6/262: chown d6/d1d/d32/l3f 75 1 2024-03-22T06:14:54.157 INFO:tasks.workunit.client.0.smithi006.stdout:4/291: rename d6/d1b/d5e to d6/d1b/d20/d6b 0 2024-03-22T06:14:54.157 INFO:tasks.workunit.client.0.smithi006.stdout:4/292: chown d6/f11 46 1 2024-03-22T06:14:54.157 INFO:tasks.workunit.client.0.smithi006.stdout:4/293: write d6/d1b/d20/d6b/d60/f68 [444950,67518] 0 2024-03-22T06:14:54.157 INFO:tasks.workunit.client.0.smithi006.stdout:4/294: chown d6/l1a 109966752 1 2024-03-22T06:14:54.158 INFO:tasks.workunit.client.0.smithi006.stdout:4/295: chown d6/d1b/l5f 4302 1 2024-03-22T06:14:54.158 INFO:tasks.workunit.client.0.smithi006.stdout:4/296: stat d6/lc 0 2024-03-22T06:14:54.160 INFO:tasks.workunit.client.0.smithi006.stdout:1/203: read d8/d9/f1e [1580729,16787] 0 2024-03-22T06:14:54.161 INFO:tasks.workunit.client.0.smithi006.stdout:8/280: truncate f7 5994697 0 2024-03-22T06:14:54.161 INFO:tasks.workunit.client.0.smithi006.stdout:8/281: fdatasync d9/f11 0 2024-03-22T06:14:54.162 INFO:tasks.workunit.client.0.smithi006.stdout:3/182: link d0/f3 d0/d4/d31/f3b 0 2024-03-22T06:14:54.162 INFO:tasks.workunit.client.0.smithi006.stdout:7/258: dwrite d5/d1e/d29/d10/f3d [0,4194304] 0 2024-03-22T06:14:54.165 INFO:tasks.workunit.client.0.smithi006.stdout:2/268: mkdir d13/d20/d35/d65 0 2024-03-22T06:14:54.167 INFO:tasks.workunit.client.0.smithi006.stdout:0/208: creat db/dc/d38/f4b x:0 0 0 2024-03-22T06:14:54.169 INFO:tasks.workunit.client.0.smithi006.stdout:5/305: mkdir d0/db/dc/d59 0 2024-03-22T06:14:54.171 INFO:tasks.workunit.client.0.smithi006.stdout:6/263: rename d6/d1d/l2e to d6/d1d/d5c/l69 0 2024-03-22T06:14:54.172 INFO:tasks.workunit.client.0.smithi006.stdout:6/264: write d6/fd [1623030,53716] 0 2024-03-22T06:14:54.172 INFO:tasks.workunit.client.0.smithi006.stdout:9/162: dwrite f1 [0,4194304] 0 2024-03-22T06:14:54.174 INFO:tasks.workunit.client.0.smithi006.stdout:9/163: dread f7 [0,4194304] 0 2024-03-22T06:14:54.174 INFO:tasks.workunit.client.0.smithi006.stdout:5/306: read d0/db/d16/d1c/f40 [2510582,9529] 0 2024-03-22T06:14:54.174 INFO:tasks.workunit.client.0.smithi006.stdout:5/307: chown d0/l8 465 1 2024-03-22T06:14:54.174 INFO:tasks.workunit.client.0.smithi006.stdout:4/297: creat d6/d1b/d20/d21/d2a/d5c/f6c x:0 0 0 2024-03-22T06:14:54.176 INFO:tasks.workunit.client.0.smithi006.stdout:1/204: symlink d8/d9/d13/d39/l42 0 2024-03-22T06:14:54.178 INFO:tasks.workunit.client.0.smithi006.stdout:0/209: dread db/d10/f2f [0,4194304] 0 2024-03-22T06:14:54.179 INFO:tasks.workunit.client.0.smithi006.stdout:8/282: symlink d9/d14/d44/l72 0 2024-03-22T06:14:54.183 INFO:tasks.workunit.client.0.smithi006.stdout:3/183: mknod d0/d4/d3a/c3c 0 2024-03-22T06:14:54.183 INFO:tasks.workunit.client.0.smithi006.stdout:3/184: write d0/d8/f1e [437052,12132] 0 2024-03-22T06:14:54.184 INFO:tasks.workunit.client.0.smithi006.stdout:7/259: mknod d5/d1e/d29/d1f/c53 0 2024-03-22T06:14:54.185 INFO:tasks.workunit.client.0.smithi006.stdout:7/260: dread - d5/d1e/d29/f4a zero size 2024-03-22T06:14:54.186 INFO:tasks.workunit.client.0.smithi006.stdout:2/269: write d13/d53/d29/f5f [1856203,90043] 0 2024-03-22T06:14:54.188 INFO:tasks.workunit.client.0.smithi006.stdout:6/265: truncate d6/d24/d44/f4d 3250040 0 2024-03-22T06:14:54.193 INFO:tasks.workunit.client.0.smithi006.stdout:2/270: dread d13/d53/d29/f5f [0,4194304] 0 2024-03-22T06:14:54.194 INFO:tasks.workunit.client.0.smithi006.stdout:3/185: dread d0/d23/f25 [4194304,4194304] 0 2024-03-22T06:14:54.197 INFO:tasks.workunit.client.0.smithi006.stdout:9/164: mknod d8/d10/d11/d2f/d38/c3b 0 2024-03-22T06:14:54.197 INFO:tasks.workunit.client.0.smithi006.stdout:9/165: readlink d8/d10/d11/d2f/l33 0 2024-03-22T06:14:54.197 INFO:tasks.workunit.client.0.smithi006.stdout:9/166: creat d8/d10/d11/f3c x:0 0 0 2024-03-22T06:14:54.200 INFO:tasks.workunit.client.0.smithi006.stdout:5/308: rename d0/db/d16/d1c/d34/d35/f4e to d0/db/dc/d14/f5a 0 2024-03-22T06:14:54.200 INFO:tasks.workunit.client.0.smithi006.stdout:5/309: dread - d0/f3e zero size 2024-03-22T06:14:54.200 INFO:tasks.workunit.client.0.smithi006.stdout:5/310: readlink d0/db/l33 0 2024-03-22T06:14:54.200 INFO:tasks.workunit.client.0.smithi006.stdout:5/311: dread - d0/db/d11/f3d zero size 2024-03-22T06:14:54.200 INFO:tasks.workunit.client.0.smithi006.stdout:5/312: write d0/db/dc/f3b [323105,9795] 0 2024-03-22T06:14:54.203 INFO:tasks.workunit.client.0.smithi006.stdout:4/298: symlink d6/d1b/d46/l6d 0 2024-03-22T06:14:54.203 INFO:tasks.workunit.client.0.smithi006.stdout:4/299: truncate d6/f4a 1964002 0 2024-03-22T06:14:54.205 INFO:tasks.workunit.client.0.smithi006.stdout:1/205: symlink d8/d9/d13/d39/l43 0 2024-03-22T06:14:54.207 INFO:tasks.workunit.client.0.smithi006.stdout:8/283: creat d9/db/dc/de/d3e/f73 x:0 0 0 2024-03-22T06:14:54.207 INFO:tasks.workunit.client.0.smithi006.stdout:8/284: truncate f5 957077 0 2024-03-22T06:14:54.208 INFO:tasks.workunit.client.0.smithi006.stdout:8/285: creat d9/db/d37/d4a/f74 x:0 0 0 2024-03-22T06:14:54.208 INFO:tasks.workunit.client.0.smithi006.stdout:8/286: dread - d9/db/d12/d1e/f2d zero size 2024-03-22T06:14:54.210 INFO:tasks.workunit.client.0.smithi006.stdout:7/261: link d5/d1e/d29/d1f/f3e d5/d1e/d29/f54 0 2024-03-22T06:14:54.210 INFO:tasks.workunit.client.0.smithi006.stdout:7/262: chown d5/cc 4795 1 2024-03-22T06:14:54.210 INFO:tasks.workunit.client.0.smithi006.stdout:7/263: readlink d5/d1e/d29/d10/l40 0 2024-03-22T06:14:54.213 INFO:tasks.workunit.client.0.smithi006.stdout:4/300: dread d6/d1b/f5b [0,4194304] 0 2024-03-22T06:14:54.213 INFO:tasks.workunit.client.0.smithi006.stdout:4/301: creat d6/d13/d3b/f6e x:0 0 0 2024-03-22T06:14:54.213 INFO:tasks.workunit.client.0.smithi006.stdout:9/167: dwrite d8/d10/d28/f1d [0,4194304] 0 2024-03-22T06:14:54.221 INFO:tasks.workunit.client.0.smithi006.stdout:2/271: creat d13/d20/d26/d2a/d62/f66 x:0 0 0 2024-03-22T06:14:54.224 INFO:tasks.workunit.client.0.smithi006.stdout:3/186: truncate d0/d4/f2f 4343467 0 2024-03-22T06:14:54.240 INFO:tasks.workunit.client.0.smithi006.stdout:0/210: rename db/dc/l36 to db/d10/d2a/d41/l4c 0 2024-03-22T06:14:54.245 INFO:tasks.workunit.client.0.smithi006.stdout:8/287: creat d9/db/d37/d4a/d4d/f75 x:0 0 0 2024-03-22T06:14:54.246 INFO:tasks.workunit.client.0.smithi006.stdout:7/264: mkdir d5/d1e/d29/d1f/d2d/d55 0 2024-03-22T06:14:54.254 INFO:tasks.workunit.client.0.smithi006.stdout:0/211: link db/d10/f1b db/d10/d33/f4d 0 2024-03-22T06:14:54.255 INFO:tasks.workunit.client.0.smithi006.stdout:8/288: mkdir d9/db/d37/d76 0 2024-03-22T06:14:54.257 INFO:tasks.workunit.client.0.smithi006.stdout:7/265: getdents d5/d1e/d29/d1f 0 2024-03-22T06:14:54.257 INFO:tasks.workunit.client.0.smithi006.stdout:7/266: write d5/d1e/f3a [4084731,63810] 0 2024-03-22T06:14:54.259 INFO:tasks.workunit.client.0.smithi006.stdout:8/289: symlink d9/db/d37/l77 0 2024-03-22T06:14:54.259 INFO:tasks.workunit.client.0.smithi006.stdout:8/290: truncate d9/db/d12/d1e/f39 100879 0 2024-03-22T06:14:54.259 INFO:tasks.workunit.client.0.smithi006.stdout:8/291: chown d9/d14/d47/l4e 912 1 2024-03-22T06:14:54.259 INFO:tasks.workunit.client.0.smithi006.stdout:8/292: chown d9/db/d12/d66/d6c/d71 418 1 2024-03-22T06:14:54.259 INFO:tasks.workunit.client.0.smithi006.stdout:8/293: chown d9/d5e/c65 24107 1 2024-03-22T06:14:54.259 INFO:tasks.workunit.client.0.smithi006.stdout:8/294: readlink d9/d14/d47/l4e 0 2024-03-22T06:14:54.259 INFO:tasks.workunit.client.0.smithi006.stdout:8/295: readlink d9/db/dc/l38 0 2024-03-22T06:14:54.259 INFO:tasks.workunit.client.0.smithi006.stdout:8/296: write d9/db/f4f [790223,33130] 0 2024-03-22T06:14:54.259 INFO:tasks.workunit.client.0.smithi006.stdout:4/302: dwrite d6/f12 [4194304,4194304] 0 2024-03-22T06:14:54.260 INFO:tasks.workunit.client.0.smithi006.stdout:2/272: dwrite d13/d20/f52 [0,4194304] 0 2024-03-22T06:14:54.260 INFO:tasks.workunit.client.0.smithi006.stdout:9/168: dwrite d8/f21 [0,4194304] 0 2024-03-22T06:14:54.260 INFO:tasks.workunit.client.0.smithi006.stdout:9/169: chown f4 0 1 2024-03-22T06:14:54.260 INFO:tasks.workunit.client.0.smithi006.stdout:2/273: chown ld 99140556 1 2024-03-22T06:14:54.260 INFO:tasks.workunit.client.0.smithi006.stdout:3/187: dwrite d0/f3 [4194304,4194304] 0 2024-03-22T06:14:54.261 INFO:tasks.workunit.client.0.smithi006.stdout:7/267: creat d5/f56 x:0 0 0 2024-03-22T06:14:54.261 INFO:tasks.workunit.client.0.smithi006.stdout:7/268: readlink d5/d1e/d29/d10/l1b 0 2024-03-22T06:14:54.262 INFO:tasks.workunit.client.0.smithi006.stdout:0/212: dread db/d10/d33/f4d [0,4194304] 0 2024-03-22T06:14:54.265 INFO:tasks.workunit.client.0.smithi006.stdout:8/297: symlink d9/db/d12/d66/d6c/l78 0 2024-03-22T06:14:54.276 INFO:tasks.workunit.client.0.smithi006.stdout:9/170: link d8/d10/f12 d8/d10/d28/f3d 0 2024-03-22T06:14:54.280 INFO:tasks.workunit.client.0.smithi006.stdout:3/188: dread d0/d23/f25 [0,4194304] 0 2024-03-22T06:14:54.280 INFO:tasks.workunit.client.0.smithi006.stdout:8/298: dread d9/db/d12/f26 [0,4194304] 0 2024-03-22T06:14:54.280 INFO:tasks.workunit.client.0.smithi006.stdout:8/299: fsync d9/db/d12/d1e/f51 0 2024-03-22T06:14:54.280 INFO:tasks.workunit.client.0.smithi006.stdout:2/274: creat d13/d53/d29/f67 x:0 0 0 2024-03-22T06:14:54.281 INFO:tasks.workunit.client.0.smithi006.stdout:4/303: dwrite d6/d1b/d20/d21/d2a/d5c/f66 [0,4194304] 0 2024-03-22T06:14:54.281 INFO:tasks.workunit.client.0.smithi006.stdout:4/304: write d6/f8 [2005131,121440] 0 2024-03-22T06:14:54.281 INFO:tasks.workunit.client.0.smithi006.stdout:0/213: rmdir db/d10/d2a/d41 39 2024-03-22T06:14:54.283 INFO:tasks.workunit.client.0.smithi006.stdout:9/171: mkdir d8/d10/d23/d3e 0 2024-03-22T06:14:54.283 INFO:tasks.workunit.client.0.smithi006.stdout:9/172: write d8/f21 [5002517,105890] 0 2024-03-22T06:14:54.285 INFO:tasks.workunit.client.0.smithi006.stdout:2/275: creat d13/d20/d35/d65/f68 x:0 0 0 2024-03-22T06:14:54.287 INFO:tasks.workunit.client.0.smithi006.stdout:7/269: dwrite d5/f56 [0,4194304] 0 2024-03-22T06:14:54.287 INFO:tasks.workunit.client.0.smithi006.stdout:7/270: stat d5/c52 0 2024-03-22T06:14:54.295 INFO:tasks.workunit.client.0.smithi006.stdout:4/305: symlink d6/d1b/d20/d21/l6f 0 2024-03-22T06:14:54.297 INFO:tasks.workunit.client.0.smithi006.stdout:3/189: dwrite d0/d4/f2f [0,4194304] 0 2024-03-22T06:14:54.297 INFO:tasks.workunit.client.0.smithi006.stdout:0/214: truncate db/d10/f1a 780646 0 2024-03-22T06:14:54.311 INFO:tasks.workunit.client.0.smithi006.stdout:4/306: dwrite f2 [4194304,4194304] 0 2024-03-22T06:14:54.312 INFO:tasks.workunit.client.0.smithi006.stdout:9/173: symlink d8/l3f 0 2024-03-22T06:14:54.312 INFO:tasks.workunit.client.0.smithi006.stdout:9/174: creat d8/d10/d23/f40 x:0 0 0 2024-03-22T06:14:54.315 INFO:tasks.workunit.client.0.smithi006.stdout:8/300: getdents d9/d14/d47 0 2024-03-22T06:14:54.316 INFO:tasks.workunit.client.0.smithi006.stdout:8/301: chown d9/d14/d44 3541349 1 2024-03-22T06:14:54.318 INFO:tasks.workunit.client.0.smithi006.stdout:2/276: mkdir d13/d69 0 2024-03-22T06:14:54.319 INFO:tasks.workunit.client.0.smithi006.stdout:2/277: creat d13/d20/d26/d2a/d62/f6a x:0 0 0 2024-03-22T06:14:54.324 INFO:tasks.workunit.client.0.smithi006.stdout:7/271: rename d5/d1e/d29/d1f/l4e to d5/d1e/d29/d1f/l57 0 2024-03-22T06:14:54.326 INFO:tasks.workunit.client.0.smithi006.stdout:3/190: mkdir d0/d23/d33/d3d 0 2024-03-22T06:14:54.326 INFO:tasks.workunit.client.0.smithi006.stdout:3/191: chown d0/d8/d1c/f2c 307 1 2024-03-22T06:14:54.327 INFO:tasks.workunit.client.0.smithi006.stdout:0/215: mkdir db/d3d/d4e 0 2024-03-22T06:14:54.330 INFO:tasks.workunit.client.0.smithi006.stdout:9/175: mknod d8/d10/d11/c41 0 2024-03-22T06:14:54.335 INFO:tasks.workunit.client.0.smithi006.stdout:4/307: dwrite d6/d13/f6a [0,4194304] 0 2024-03-22T06:14:54.336 INFO:tasks.workunit.client.0.smithi006.stdout:4/308: dread - d6/f59 zero size 2024-03-22T06:14:54.336 INFO:tasks.workunit.client.0.smithi006.stdout:4/309: chown d6/fa 110165 1 2024-03-22T06:14:54.336 INFO:tasks.workunit.client.0.smithi006.stdout:8/302: dwrite d9/db/dc/fd [0,4194304] 0 2024-03-22T06:14:54.336 INFO:tasks.workunit.client.0.smithi006.stdout:2/278: mkdir d13/d53/d6b 0 2024-03-22T06:14:54.337 INFO:tasks.workunit.client.0.smithi006.stdout:4/310: dread d6/d1b/d20/d6b/d60/f68 [0,4194304] 0 2024-03-22T06:14:54.341 INFO:tasks.workunit.client.0.smithi006.stdout:2/279: dread d13/f21 [0,4194304] 0 2024-03-22T06:14:54.342 INFO:tasks.workunit.client.0.smithi006.stdout:3/192: truncate d0/d8/f1a 2611251 0 2024-03-22T06:14:54.344 INFO:tasks.workunit.client.0.smithi006.stdout:0/216: mkdir db/d10/d2a/d41/d4f 0 2024-03-22T06:14:54.346 INFO:tasks.workunit.client.0.smithi006.stdout:0/217: dread db/d10/d33/f42 [0,4194304] 0 2024-03-22T06:14:54.346 INFO:tasks.workunit.client.0.smithi006.stdout:9/176: truncate d8/d10/d28/f3d 1865125 0 2024-03-22T06:14:54.356 INFO:tasks.workunit.client.0.smithi006.stdout:4/311: dwrite d6/d1b/d20/d6b/f64 [0,4194304] 0 2024-03-22T06:14:54.356 INFO:tasks.workunit.client.0.smithi006.stdout:8/303: truncate d9/db/f20 3989410 0 2024-03-22T06:14:54.356 INFO:tasks.workunit.client.0.smithi006.stdout:8/304: write d9/db/d37/f41 [800391,41901] 0 2024-03-22T06:14:54.356 INFO:tasks.workunit.client.0.smithi006.stdout:8/305: truncate d9/db/d12/f5d 735817 0 2024-03-22T06:14:54.359 INFO:tasks.workunit.client.0.smithi006.stdout:3/193: mkdir d0/d8/d1c/d3e 0 2024-03-22T06:14:54.361 INFO:tasks.workunit.client.0.smithi006.stdout:9/177: unlink d8/d10/d11/d2f/l33 0 2024-03-22T06:14:54.361 INFO:tasks.workunit.client.0.smithi006.stdout:7/272: rmdir d5/d1e 39 2024-03-22T06:14:54.362 INFO:tasks.workunit.client.0.smithi006.stdout:7/273: chown d5/d1e/d29/d1f/d2d/f4c 27709 1 2024-03-22T06:14:54.362 INFO:tasks.workunit.client.0.smithi006.stdout:7/274: chown d5/cc 0 1 2024-03-22T06:14:54.368 INFO:tasks.workunit.client.0.smithi006.stdout:4/312: mknod d6/d1b/d20/d6b/c70 0 2024-03-22T06:14:54.370 INFO:tasks.workunit.client.0.smithi006.stdout:2/280: mkdir d13/d53/d29/d6c 0 2024-03-22T06:14:54.370 INFO:tasks.workunit.client.0.smithi006.stdout:0/218: dwrite db/d10/d2a/f34 [0,4194304] 0 2024-03-22T06:14:54.371 INFO:tasks.workunit.client.0.smithi006.stdout:0/219: fdatasync db/d10/d35/f43 0 2024-03-22T06:14:54.372 INFO:tasks.workunit.client.0.smithi006.stdout:3/194: rename d0/d23/d27/c2b to d0/d23/d33/d3d/c3f 0 2024-03-22T06:14:54.373 INFO:tasks.workunit.client.0.smithi006.stdout:8/306: dwrite f1 [0,4194304] 0 2024-03-22T06:14:54.373 INFO:tasks.workunit.client.0.smithi006.stdout:8/307: chown d9/db/d12/f5d 104 1 2024-03-22T06:14:54.378 INFO:tasks.workunit.client.0.smithi006.stdout:7/275: link d5/d1e/d29/d10/f3d d5/d1e/d29/d1f/d2d/d55/f58 0 2024-03-22T06:14:54.378 INFO:tasks.workunit.client.0.smithi006.stdout:7/276: fsync d5/d1e/d29/f2a 0 2024-03-22T06:14:54.378 INFO:tasks.workunit.client.0.smithi006.stdout:2/281: mknod d13/d53/d6b/c6d 0 2024-03-22T06:14:54.380 INFO:tasks.workunit.client.0.smithi006.stdout:3/195: unlink d0/d4/f1d 0 2024-03-22T06:14:54.385 INFO:tasks.workunit.client.0.smithi006.stdout:9/178: dwrite f3 [0,4194304] 0 2024-03-22T06:14:54.386 INFO:tasks.workunit.client.0.smithi006.stdout:9/179: dread f7 [0,4194304] 0 2024-03-22T06:14:54.387 INFO:tasks.workunit.client.0.smithi006.stdout:9/180: chown d8/d10/f12 622 1 2024-03-22T06:14:54.389 INFO:tasks.workunit.client.0.smithi006.stdout:8/308: mkdir d9/db/d12/d66/d6c/d71/d79 0 2024-03-22T06:14:54.389 INFO:tasks.workunit.client.0.smithi006.stdout:8/309: chown d9/d14/d44/l72 379433756 1 2024-03-22T06:14:54.389 INFO:tasks.workunit.client.0.smithi006.stdout:8/310: chown d9/db/d12/d66 248 1 2024-03-22T06:14:54.390 INFO:tasks.workunit.client.0.smithi006.stdout:2/282: dread f6 [0,4194304] 0 2024-03-22T06:14:54.397 INFO:tasks.workunit.client.0.smithi006.stdout:3/196: mknod d0/d8/d9/d24/c40 0 2024-03-22T06:14:54.397 INFO:tasks.workunit.client.0.smithi006.stdout:3/197: write d0/d8/f1a [1652184,78629] 0 2024-03-22T06:14:54.397 INFO:tasks.workunit.client.0.smithi006.stdout:3/198: read - d0/d8/d1c/f2c zero size 2024-03-22T06:14:54.402 INFO:tasks.workunit.client.0.smithi006.stdout:8/311: rename l4 to d9/l7a 0 2024-03-22T06:14:54.402 INFO:tasks.workunit.client.0.smithi006.stdout:8/312: read - d9/f21 zero size 2024-03-22T06:14:54.403 INFO:tasks.workunit.client.0.smithi006.stdout:2/283: truncate d13/f18 2029723 0 2024-03-22T06:14:54.407 INFO:tasks.workunit.client.0.smithi006.stdout:7/277: getdents d5/d1e/d29/d1f/d2d/d55 0 2024-03-22T06:14:54.409 INFO:tasks.workunit.client.0.smithi006.stdout:3/199: dread d0/d23/d27/d36/d39/f35 [0,4194304] 0 2024-03-22T06:14:54.409 INFO:tasks.workunit.client.0.smithi006.stdout:3/200: readlink d0/d23/d27/d36/d39/l2d 0 2024-03-22T06:14:54.409 INFO:tasks.workunit.client.0.smithi006.stdout:2/284: mkdir d13/d34/d6e 0 2024-03-22T06:14:54.410 INFO:tasks.workunit.client.0.smithi006.stdout:2/285: stat d13/d20/f52 0 2024-03-22T06:14:54.412 INFO:tasks.workunit.client.0.smithi006.stdout:8/313: rename d9/db/dc/de/f16 to d9/db/f7b 0 2024-03-22T06:14:54.412 INFO:tasks.workunit.client.0.smithi006.stdout:8/314: dread - d9/db/d12/d1e/f6d zero size 2024-03-22T06:14:54.412 INFO:tasks.workunit.client.0.smithi006.stdout:8/315: truncate d9/db/d35/f6f 984020 0 2024-03-22T06:14:54.416 INFO:tasks.workunit.client.0.smithi006.stdout:3/201: rename d0/d8/f1e to d0/d8/d1c/f41 0 2024-03-22T06:14:54.416 INFO:tasks.workunit.client.0.smithi006.stdout:3/202: write d0/d8/f17 [316449,16853] 0 2024-03-22T06:14:54.417 INFO:tasks.workunit.client.0.smithi006.stdout:8/316: unlink d9/db/dc/de/d3e/f40 0 2024-03-22T06:14:54.432 INFO:tasks.workunit.client.0.smithi006.stdout:7/278: write d5/f1d [3520100,12592] 0 2024-03-22T06:14:54.432 INFO:tasks.workunit.client.0.smithi006.stdout:7/279: chown f0 167 1 2024-03-22T06:14:54.432 INFO:tasks.workunit.client.0.smithi006.stdout:7/280: dread - d5/d1e/d29/d10/f11 zero size 2024-03-22T06:14:54.432 INFO:tasks.workunit.client.0.smithi006.stdout:7/281: chown d5/d1e/d29/d10/f11 7004399 1 2024-03-22T06:14:54.435 INFO:tasks.workunit.client.0.smithi006.stdout:7/282: mkdir d5/d1e/d29/d59 0 2024-03-22T06:14:54.437 INFO:tasks.workunit.client.0.smithi006.stdout:7/283: creat d5/d1e/d3f/f5a x:0 0 0 2024-03-22T06:14:54.443 INFO:tasks.workunit.client.0.smithi006.stdout:7/284: dread d5/d1e/f3a [0,4194304] 0 2024-03-22T06:14:54.444 INFO:tasks.workunit.client.0.smithi006.stdout:7/285: dread d5/d1e/f31 [0,4194304] 0 2024-03-22T06:14:54.446 INFO:tasks.workunit.client.0.smithi006.stdout:8/317: dwrite d9/db/dc/fd [4194304,4194304] 0 2024-03-22T06:14:54.446 INFO:tasks.workunit.client.0.smithi006.stdout:8/318: fdatasync d9/db/f20 0 2024-03-22T06:14:54.447 INFO:tasks.workunit.client.0.smithi006.stdout:9/181: dwrite d8/d10/d28/f1d [4194304,4194304] 0 2024-03-22T06:14:54.447 INFO:tasks.workunit.client.0.smithi006.stdout:2/286: dwrite d13/f21 [0,4194304] 0 2024-03-22T06:14:54.447 INFO:tasks.workunit.client.0.smithi006.stdout:9/182: chown d8/d1f/d34 3380 1 2024-03-22T06:14:54.458 INFO:tasks.workunit.client.0.smithi006.stdout:8/319: dwrite d9/db/d37/d4a/d4d/f75 [0,4194304] 0 2024-03-22T06:14:54.459 INFO:tasks.workunit.client.0.smithi006.stdout:9/183: rmdir d8/d10/d23 39 2024-03-22T06:14:54.459 INFO:tasks.workunit.client.0.smithi006.stdout:9/184: creat d8/d10/d11/f42 x:0 0 0 2024-03-22T06:14:54.460 INFO:tasks.workunit.client.0.smithi006.stdout:9/185: fdatasync f1 0 2024-03-22T06:14:54.460 INFO:tasks.workunit.client.0.smithi006.stdout:3/203: rmdir d0 39 2024-03-22T06:14:54.467 INFO:tasks.workunit.client.0.smithi006.stdout:8/320: creat d9/db/d12/d66/d6c/f7c x:0 0 0 2024-03-22T06:14:54.469 INFO:tasks.workunit.client.0.smithi006.stdout:9/186: rename d8/d10/c1a to d8/d10/d28/d39/c43 0 2024-03-22T06:14:54.470 INFO:tasks.workunit.client.0.smithi006.stdout:9/187: fsync d8/d10/d28/f35 0 2024-03-22T06:14:54.470 INFO:tasks.workunit.client.0.smithi006.stdout:9/188: fdatasync d8/d10/d23/f27 0 2024-03-22T06:14:54.470 INFO:tasks.workunit.client.0.smithi006.stdout:8/321: symlink d9/db/d12/d66/d6c/d71/l7d 0 2024-03-22T06:14:54.475 INFO:tasks.workunit.client.0.smithi006.stdout:8/322: mkdir d9/db/d12/d7e 0 2024-03-22T06:14:54.475 INFO:tasks.workunit.client.0.smithi006.stdout:8/323: chown d9/f6b 27422686 1 2024-03-22T06:14:54.478 INFO:tasks.workunit.client.0.smithi006.stdout:8/324: creat d9/db/d12/d66/f7f x:0 0 0 2024-03-22T06:14:54.479 INFO:tasks.workunit.client.0.smithi006.stdout:3/204: dwrite d0/d8/d1c/f37 [0,4194304] 0 2024-03-22T06:14:54.479 INFO:tasks.workunit.client.0.smithi006.stdout:3/205: stat d0/d23 0 2024-03-22T06:14:54.479 INFO:tasks.workunit.client.0.smithi006.stdout:3/206: write d0/d23/d27/d36/d39/f35 [4878539,125666] 0 2024-03-22T06:14:54.480 INFO:tasks.workunit.client.0.smithi006.stdout:3/207: chown d0/d8/d9/l1b 92 1 2024-03-22T06:14:54.480 INFO:tasks.workunit.client.0.smithi006.stdout:3/208: chown d0/cd 0 1 2024-03-22T06:14:54.487 INFO:tasks.workunit.client.0.smithi006.stdout:3/209: link d0/d23/d27/d36/d39/l2d d0/l42 0 2024-03-22T06:14:54.493 INFO:tasks.workunit.client.0.smithi006.stdout:9/189: rmdir d8/d10/d11 39 2024-03-22T06:14:54.497 INFO:tasks.workunit.client.0.smithi006.stdout:3/210: write d0/f3 [2734071,94666] 0 2024-03-22T06:14:54.501 INFO:tasks.workunit.client.0.smithi006.stdout:8/325: dwrite d9/db/d37/d4a/f74 [0,4194304] 0 2024-03-22T06:14:54.505 INFO:tasks.workunit.client.0.smithi006.stdout:8/326: mkdir d9/db/d12/d66/d6c/d80 0 2024-03-22T06:14:54.509 INFO:tasks.workunit.client.0.smithi006.stdout:9/190: write d8/f21 [1599517,72045] 0 2024-03-22T06:14:54.515 INFO:tasks.workunit.client.0.smithi006.stdout:9/191: link d8/f9 d8/d10/d28/f44 0 2024-03-22T06:14:54.518 INFO:tasks.workunit.client.0.smithi006.stdout:9/192: mkdir d8/d10/d20/d45 0 2024-03-22T06:14:54.521 INFO:tasks.workunit.client.0.smithi006.stdout:9/193: link d8/fc d8/d10/d28/d39/f46 0 2024-03-22T06:14:54.521 INFO:tasks.workunit.client.0.smithi006.stdout:9/194: readlink d8/d10/l30 0 2024-03-22T06:14:54.523 INFO:tasks.workunit.client.0.smithi006.stdout:9/195: mkdir d8/d10/d47 0 2024-03-22T06:14:54.523 INFO:tasks.workunit.client.0.smithi006.stdout:9/196: stat d8/d10/f2e 0 2024-03-22T06:14:54.525 INFO:tasks.workunit.client.0.smithi006.stdout:3/211: dwrite d0/d8/f1a [0,4194304] 0 2024-03-22T06:14:54.525 INFO:tasks.workunit.client.0.smithi006.stdout:8/327: dwrite d9/db/dc/de/f17 [0,4194304] 0 2024-03-22T06:14:54.529 INFO:tasks.workunit.client.0.smithi006.stdout:3/212: mknod d0/d23/d27/d36/d39/c43 0 2024-03-22T06:14:54.534 INFO:tasks.workunit.client.0.smithi006.stdout:8/328: mknod d9/db/d37/d4a/c81 0 2024-03-22T06:14:54.534 INFO:tasks.workunit.client.0.smithi006.stdout:8/329: chown d9/db/d37/f41 61272 1 2024-03-22T06:14:54.546 INFO:tasks.workunit.client.0.smithi006.stdout:3/213: dwrite d0/d8/f13 [0,4194304] 0 2024-03-22T06:14:54.549 INFO:tasks.workunit.client.0.smithi006.stdout:8/330: dwrite d9/db/d35/f3c [0,4194304] 0 2024-03-22T06:14:54.554 INFO:tasks.workunit.client.0.smithi006.stdout:8/331: dread d9/f27 [0,4194304] 0 2024-03-22T06:14:54.554 INFO:tasks.workunit.client.0.smithi006.stdout:8/332: creat d9/db/d37/d4a/f82 x:0 0 0 2024-03-22T06:14:54.558 INFO:tasks.workunit.client.0.smithi006.stdout:3/214: creat d0/d23/d33/f44 x:0 0 0 2024-03-22T06:14:54.558 INFO:tasks.workunit.client.0.smithi006.stdout:3/215: truncate d0/d23/d27/d36/d39/f30 1975684 0 2024-03-22T06:14:54.564 INFO:tasks.workunit.client.0.smithi006.stdout:8/333: rename d9/db/f60 to d9/db/dc/de/f83 0 2024-03-22T06:14:54.564 INFO:tasks.workunit.client.0.smithi006.stdout:8/334: fsync d9/db/d12/d1e/f2d 0 2024-03-22T06:14:54.569 INFO:tasks.workunit.client.0.smithi006.stdout:3/216: link d0/l19 d0/l45 0 2024-03-22T06:14:54.571 INFO:tasks.workunit.client.0.smithi006.stdout:8/335: rmdir d9/db/d35 39 2024-03-22T06:14:54.574 INFO:tasks.workunit.client.0.smithi006.stdout:8/336: mknod d9/db/d12/d66/d6c/d80/c84 0 2024-03-22T06:14:54.576 INFO:tasks.workunit.client.0.smithi006.stdout:3/217: mkdir d0/d46 0 2024-03-22T06:14:54.576 INFO:tasks.workunit.client.0.smithi006.stdout:3/218: chown d0/l42 15 1 2024-03-22T06:14:54.591 INFO:tasks.workunit.client.0.smithi006.stdout:3/219: dwrite d0/d8/d1c/f37 [4194304,4194304] 0 2024-03-22T06:14:54.594 INFO:tasks.workunit.client.0.smithi006.stdout:3/220: stat d0/d23/d27/d36/d39/c21 0 2024-03-22T06:14:54.597 INFO:tasks.workunit.client.0.smithi006.stdout:3/221: unlink d0/d4/fb 0 2024-03-22T06:14:54.597 INFO:tasks.workunit.client.0.smithi006.stdout:3/222: truncate d0/d23/d27/d36/d39/f35 5592776 0 2024-03-22T06:14:54.597 INFO:tasks.workunit.client.0.smithi006.stdout:3/223: chown d0/d4/d3a/c3c 922 1 2024-03-22T06:14:54.597 INFO:tasks.workunit.client.0.smithi006.stdout:8/337: dwrite d9/db/f2e [4194304,4194304] 0 2024-03-22T06:14:54.597 INFO:tasks.workunit.client.0.smithi006.stdout:8/338: dread - d9/db/d12/d1e/f51 zero size 2024-03-22T06:14:54.607 INFO:tasks.workunit.client.0.smithi006.stdout:3/224: read d0/d8/d1c/f37 [8349762,57014] 0 2024-03-22T06:14:54.608 INFO:tasks.workunit.client.0.smithi006.stdout:8/339: read d9/db/dc/de/f17 [1297344,129610] 0 2024-03-22T06:14:54.610 INFO:tasks.workunit.client.0.smithi006.stdout:3/225: creat d0/d23/d27/f47 x:0 0 0 2024-03-22T06:14:54.617 INFO:tasks.workunit.client.0.smithi006.stdout:3/226: dread d0/d8/f13 [0,4194304] 0 2024-03-22T06:14:54.618 INFO:tasks.workunit.client.0.smithi006.stdout:3/227: stat d0/d8/d9/d24 0 2024-03-22T06:14:54.624 INFO:tasks.workunit.client.0.smithi006.stdout:3/228: read d0/d4/d31/f3b [5619617,40142] 0 2024-03-22T06:14:54.631 INFO:tasks.workunit.client.0.smithi006.stdout:3/229: dread d0/d8/d9/f32 [0,4194304] 0 2024-03-22T06:14:54.632 INFO:tasks.workunit.client.0.smithi006.stdout:3/230: creat d0/d23/d27/f48 x:0 0 0 2024-03-22T06:14:54.636 INFO:tasks.workunit.client.0.smithi006.stdout:3/231: dread d0/d23/f25 [0,4194304] 0 2024-03-22T06:14:54.638 INFO:tasks.workunit.client.0.smithi006.stdout:3/232: mkdir d0/d8/d1c/d49 0 2024-03-22T06:14:54.640 INFO:tasks.workunit.client.0.smithi006.stdout:3/233: rmdir d0/d4 39 2024-03-22T06:14:54.640 INFO:tasks.workunit.client.0.smithi006.stdout:3/234: chown d0/d8/d9/f32 39819 1 2024-03-22T06:14:54.642 INFO:tasks.workunit.client.0.smithi006.stdout:3/235: unlink d0/d4/d31/c34 0 2024-03-22T06:14:54.651 INFO:tasks.workunit.client.0.smithi006.stdout:3/236: dread d0/d8/f1a [0,4194304] 0 2024-03-22T06:14:54.651 INFO:tasks.workunit.client.0.smithi006.stdout:3/237: write d0/d8/d1c/f2c [412638,120111] 0 2024-03-22T06:14:54.651 INFO:tasks.workunit.client.0.smithi006.stdout:3/238: chown d0/d8/f17 2 1 2024-03-22T06:14:54.653 INFO:tasks.workunit.client.0.smithi006.stdout:3/239: symlink d0/d46/l4a 0 2024-03-22T06:14:54.654 INFO:tasks.workunit.client.0.smithi006.stdout:3/240: fdatasync d0/d4/f2f 0 2024-03-22T06:14:54.654 INFO:tasks.workunit.client.0.smithi006.stdout:3/241: write d0/d23/d27/d36/d39/f35 [6613398,90555] 0 2024-03-22T06:14:54.656 INFO:tasks.workunit.client.0.smithi006.stdout:3/242: symlink d0/d8/d1c/d3e/l4b 0 2024-03-22T06:14:54.661 INFO:tasks.workunit.client.0.smithi006.stdout:3/243: rename d0/d23/d27/d36/d39 to d0/d23/d4c 0 2024-03-22T06:14:54.668 INFO:tasks.workunit.client.0.smithi006.stdout:3/244: dread d0/d23/d4c/f35 [0,4194304] 0 2024-03-22T06:14:54.668 INFO:tasks.workunit.client.0.smithi006.stdout:3/245: read d0/d4/d31/f3b [7058358,1271] 0 2024-03-22T06:14:54.668 INFO:tasks.workunit.client.0.smithi006.stdout:3/246: stat d0/d8/d1c/d3e/l4b 0 2024-03-22T06:14:54.668 INFO:tasks.workunit.client.0.smithi006.stdout:3/247: stat d0/c1f 0 2024-03-22T06:14:54.668 INFO:tasks.workunit.client.0.smithi006.stdout:3/248: chown d0/d4/d31/c38 1232 1 2024-03-22T06:14:54.687 INFO:tasks.workunit.client.0.smithi006.stdout:3/249: dwrite d0/fa [0,4194304] 0 2024-03-22T06:14:54.687 INFO:tasks.workunit.client.0.smithi006.stdout:3/250: fsync d0/d23/d27/f47 0 2024-03-22T06:14:54.688 INFO:tasks.workunit.client.0.smithi006.stdout:3/251: creat d0/d23/d33/f4d x:0 0 0 2024-03-22T06:14:54.688 INFO:tasks.workunit.client.0.smithi006.stdout:3/252: write d0/d23/d4c/f30 [1852178,73061] 0 2024-03-22T06:14:54.782 INFO:tasks.workunit.client.0.smithi006.stdout:3/253: rmdir d0/d23/d33/d3d 39 2024-03-22T06:14:54.782 INFO:tasks.workunit.client.0.smithi006.stdout:3/254: truncate d0/d8/d1c/f2c 977673 0 2024-03-22T06:14:54.786 INFO:tasks.workunit.client.0.smithi006.stdout:3/255: dread d0/d23/d4c/f35 [4194304,4194304] 0 2024-03-22T06:14:54.800 INFO:tasks.workunit.client.0.smithi006.stdout:3/256: dwrite d0/d23/d27/f48 [0,4194304] 0 2024-03-22T06:14:54.803 INFO:tasks.workunit.client.0.smithi006.stdout:3/257: dread d0/d23/d4c/f35 [4194304,4194304] 0 2024-03-22T06:14:54.803 INFO:tasks.workunit.client.0.smithi006.stdout:3/258: fdatasync d0/d23/d33/f44 0 2024-03-22T06:14:54.842 INFO:tasks.workunit.client.0.smithi006.stdout:3/259: mknod d0/d8/d1c/d49/c4e 0 2024-03-22T06:14:54.845 INFO:tasks.workunit.client.0.smithi006.stdout:3/260: rename d0/d4/f2f to d0/d8/d1c/f4f 0 2024-03-22T06:14:54.848 INFO:tasks.workunit.client.0.smithi006.stdout:3/261: symlink d0/d23/d33/l50 0 2024-03-22T06:14:54.851 INFO:tasks.workunit.client.0.smithi006.stdout:3/262: link d0/d23/d33/d3d/c3f d0/d8/d1c/c51 0 2024-03-22T06:14:54.851 INFO:tasks.workunit.client.0.smithi006.stdout:3/263: fsync d0/d8/d1c/f41 0 2024-03-22T06:14:54.851 INFO:tasks.workunit.client.0.smithi006.stdout:3/264: write d0/d23/d33/f44 [473951,29606] 0 2024-03-22T06:14:54.856 INFO:tasks.workunit.client.0.smithi006.stdout:3/265: symlink d0/d8/l52 0 2024-03-22T06:14:54.859 INFO:tasks.workunit.client.0.smithi006.stdout:3/266: unlink d0/d8/d1c/f37 0 2024-03-22T06:14:54.859 INFO:tasks.workunit.client.0.smithi006.stdout:3/267: chown d0/d23/d4c/f35 26 1 2024-03-22T06:14:54.875 INFO:tasks.workunit.client.0.smithi006.stdout:3/268: dwrite d0/d8/d1c/f41 [0,4194304] 0 2024-03-22T06:14:54.875 INFO:tasks.workunit.client.0.smithi006.stdout:3/269: write d0/d23/d33/f4d [538022,47917] 0 2024-03-22T06:14:54.881 INFO:tasks.workunit.client.0.smithi006.stdout:3/270: dread d0/d8/f13 [0,4194304] 0 2024-03-22T06:14:54.898 INFO:tasks.workunit.client.0.smithi006.stdout:3/271: dwrite d0/d8/f13 [0,4194304] 0 2024-03-22T06:14:54.898 INFO:tasks.workunit.client.0.smithi006.stdout:3/272: readlink d0/l42 0 2024-03-22T06:14:54.899 INFO:tasks.workunit.client.0.smithi006.stdout:3/273: mkdir d0/d4/d53 0 2024-03-22T06:14:54.904 INFO:tasks.workunit.client.0.smithi006.stdout:3/274: symlink d0/d8/d9/d24/l54 0 2024-03-22T06:14:54.905 INFO:tasks.workunit.client.0.smithi006.stdout:3/275: symlink d0/d8/d1c/l55 0 2024-03-22T06:14:54.942 INFO:tasks.workunit.client.0.smithi006.stdout:3/276: creat d0/f56 x:0 0 0 2024-03-22T06:14:54.942 INFO:tasks.workunit.client.0.smithi006.stdout:3/277: mknod d0/d23/d33/d3d/c57 0 2024-03-22T06:14:54.949 INFO:tasks.workunit.client.0.smithi006.stdout:3/278: dread d0/d23/d27/f48 [0,4194304] 0 2024-03-22T06:14:54.951 INFO:tasks.workunit.client.0.smithi006.stdout:3/279: mkdir d0/d23/d33/d3d/d58 0 2024-03-22T06:14:54.951 INFO:tasks.workunit.client.0.smithi006.stdout:3/280: write d0/d8/d1c/f4f [7375831,99907] 0 2024-03-22T06:14:54.951 INFO:tasks.workunit.client.0.smithi006.stdout:3/281: creat d0/d23/d27/f59 x:0 0 0 2024-03-22T06:14:54.959 INFO:tasks.workunit.client.0.smithi006.stdout:3/282: write d0/fa [2973532,23387] 0 2024-03-22T06:14:54.963 INFO:tasks.workunit.client.0.smithi006.stdout:3/283: write d0/d23/d27/f48 [3415409,98304] 0 2024-03-22T06:14:54.963 INFO:tasks.workunit.client.0.smithi006.stdout:3/284: fdatasync d0/d8/f13 0 2024-03-22T06:14:54.963 INFO:tasks.workunit.client.0.smithi006.stdout:3/285: write d0/d4/d31/f3b [4721305,97399] 0 2024-03-22T06:14:54.964 INFO:tasks.workunit.client.0.smithi006.stdout:3/286: write d0/d23/d4c/f35 [4408849,54415] 0 2024-03-22T06:14:54.966 INFO:tasks.workunit.client.0.smithi006.stdout:3/287: symlink d0/d8/d9/l5a 0 2024-03-22T06:14:54.966 INFO:tasks.workunit.client.0.smithi006.stdout:3/288: fdatasync d0/d8/f1a 0 2024-03-22T06:14:54.967 INFO:tasks.workunit.client.0.smithi006.stdout:3/289: creat d0/d23/d33/d3d/f5b x:0 0 0 2024-03-22T06:14:54.984 INFO:tasks.workunit.client.0.smithi006.stdout:3/290: dwrite d0/d8/d1c/f41 [4194304,4194304] 0 2024-03-22T06:14:55.001 INFO:tasks.workunit.client.0.smithi006.stdout:3/291: dwrite d0/d4/d31/f3b [4194304,4194304] 0 2024-03-22T06:14:55.001 INFO:tasks.workunit.client.0.smithi006.stdout:3/292: chown d0/d8/d9/l5a 44120170 1 2024-03-22T06:14:55.003 INFO:tasks.workunit.client.0.smithi006.stdout:3/293: unlink d0/d8/d9/l1b 0 2024-03-22T06:14:55.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:54 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:14:55.008 INFO:tasks.workunit.client.0.smithi006.stdout:3/294: dread d0/d8/d9/f32 [0,4194304] 0 2024-03-22T06:14:55.017 INFO:tasks.workunit.client.0.smithi006.stdout:3/295: read d0/d8/f13 [2231567,122784] 0 2024-03-22T06:14:55.017 INFO:tasks.workunit.client.0.smithi006.stdout:3/296: write d0/d23/d27/f48 [4140094,90684] 0 2024-03-22T06:14:55.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:54 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:14:55.034 INFO:tasks.workunit.client.0.smithi006.stdout:3/297: dwrite d0/d23/d27/f48 [0,4194304] 0 2024-03-22T06:14:55.036 INFO:tasks.workunit.client.0.smithi006.stdout:3/298: mknod d0/d8/d1c/c5c 0 2024-03-22T06:14:55.036 INFO:tasks.workunit.client.0.smithi006.stdout:3/299: fsync d0/d8/f13 0 2024-03-22T06:14:55.039 INFO:tasks.workunit.client.0.smithi006.stdout:3/300: unlink d0/d8/c22 0 2024-03-22T06:14:55.039 INFO:tasks.workunit.client.0.smithi006.stdout:3/301: fdatasync d0/d23/d27/f48 0 2024-03-22T06:14:55.039 INFO:tasks.workunit.client.0.smithi006.stdout:3/302: write d0/d23/d33/d3d/f5b [825576,53244] 0 2024-03-22T06:14:55.081 INFO:tasks.workunit.client.0.smithi006.stdout:6/266: sync 2024-03-22T06:14:55.081 INFO:tasks.workunit.client.0.smithi006.stdout:6/267: truncate d6/d1d/d32/f45 1036875 0 2024-03-22T06:14:55.085 INFO:tasks.workunit.client.0.smithi006.stdout:6/268: rmdir d6/d1b/d2c/d4a 39 2024-03-22T06:14:55.086 INFO:tasks.workunit.client.0.smithi006.stdout:6/269: stat d6/f42 0 2024-03-22T06:14:55.086 INFO:tasks.workunit.client.0.smithi006.stdout:6/270: fdatasync d6/d1b/d2c/f59 0 2024-03-22T06:14:55.087 INFO:tasks.workunit.client.0.smithi006.stdout:6/271: creat d6/d1d/d64/f6a x:0 0 0 2024-03-22T06:14:55.090 INFO:tasks.workunit.client.0.smithi006.stdout:6/272: creat d6/d1d/d32/d4f/d56/f6b x:0 0 0 2024-03-22T06:14:55.090 INFO:tasks.workunit.client.0.smithi006.stdout:6/273: stat d6/d1d/d32 0 2024-03-22T06:14:55.090 INFO:tasks.workunit.client.0.smithi006.stdout:6/274: stat d6/d1d/f20 0 2024-03-22T06:14:55.091 INFO:tasks.workunit.client.0.smithi006.stdout:3/303: getdents d0/d8 0 2024-03-22T06:14:55.092 INFO:tasks.workunit.client.0.smithi006.stdout:3/304: chown d0/d46/l4a 659031625 1 2024-03-22T06:14:55.107 INFO:tasks.workunit.client.0.smithi006.stdout:6/275: dwrite d6/f42 [0,4194304] 0 2024-03-22T06:14:55.114 INFO:tasks.workunit.client.0.smithi006.stdout:6/276: dread d6/d1b/f23 [0,4194304] 0 2024-03-22T06:14:55.129 INFO:tasks.workunit.client.0.smithi006.stdout:3/305: truncate d0/d23/d27/f48 1978861 0 2024-03-22T06:14:55.130 INFO:tasks.workunit.client.0.smithi006.stdout:5/313: sync 2024-03-22T06:14:55.130 INFO:tasks.workunit.client.0.smithi006.stdout:5/314: dread - d0/f3e zero size 2024-03-22T06:14:55.130 INFO:tasks.workunit.client.0.smithi006.stdout:5/315: getdents d0/db/d50/d58 0 2024-03-22T06:14:55.133 INFO:tasks.workunit.client.0.smithi006.stdout:6/277: dwrite d6/f42 [0,4194304] 0 2024-03-22T06:14:55.133 INFO:tasks.workunit.client.0.smithi006.stdout:6/278: write d6/d24/f38 [618142,17751] 0 2024-03-22T06:14:55.133 INFO:tasks.workunit.client.0.smithi006.stdout:5/316: creat d0/db/d50/f5b x:0 0 0 2024-03-22T06:14:55.135 INFO:tasks.workunit.client.0.smithi006.stdout:6/279: creat d6/f6c x:0 0 0 2024-03-22T06:14:55.138 INFO:tasks.workunit.client.0.smithi006.stdout:6/280: unlink d6/c3d 0 2024-03-22T06:14:55.138 INFO:tasks.workunit.client.0.smithi006.stdout:6/281: readlink d6/d1b/d2c/l47 0 2024-03-22T06:14:55.141 INFO:tasks.workunit.client.0.smithi006.stdout:6/282: unlink d6/d1d/f2b 0 2024-03-22T06:14:55.145 INFO:tasks.workunit.client.0.smithi006.stdout:5/317: dread d0/db/f25 [0,4194304] 0 2024-03-22T06:14:55.162 INFO:tasks.workunit.client.0.smithi006.stdout:3/306: dwrite d0/d23/d27/f59 [0,4194304] 0 2024-03-22T06:14:55.167 INFO:tasks.workunit.client.0.smithi006.stdout:3/307: truncate d0/f3 6117089 0 2024-03-22T06:14:55.167 INFO:tasks.workunit.client.0.smithi006.stdout:3/308: creat d0/d23/d33/d3d/f5d x:0 0 0 2024-03-22T06:14:55.167 INFO:tasks.workunit.client.0.smithi006.stdout:6/283: dwrite d6/f6c [0,4194304] 0 2024-03-22T06:14:55.167 INFO:tasks.workunit.client.0.smithi006.stdout:5/318: dwrite d0/db/f1e [0,4194304] 0 2024-03-22T06:14:55.175 INFO:tasks.workunit.client.0.smithi006.stdout:6/284: dread d6/d1d/f20 [0,4194304] 0 2024-03-22T06:14:55.178 INFO:tasks.workunit.client.0.smithi006.stdout:0/220: sync 2024-03-22T06:14:55.178 INFO:tasks.workunit.client.0.smithi006.stdout:7/286: sync 2024-03-22T06:14:55.178 INFO:tasks.workunit.client.0.smithi006.stdout:2/287: sync 2024-03-22T06:14:55.178 INFO:tasks.workunit.client.0.smithi006.stdout:8/340: sync 2024-03-22T06:14:55.178 INFO:tasks.workunit.client.0.smithi006.stdout:9/197: sync 2024-03-22T06:14:55.179 INFO:tasks.workunit.client.0.smithi006.stdout:1/206: sync 2024-03-22T06:14:55.179 INFO:tasks.workunit.client.0.smithi006.stdout:4/313: sync 2024-03-22T06:14:55.179 INFO:tasks.workunit.client.0.smithi006.stdout:4/314: chown d6/d1b/d20/f24 904358105 1 2024-03-22T06:14:55.179 INFO:tasks.workunit.client.0.smithi006.stdout:1/207: dread - d8/d9/d3a/d18/f37 zero size 2024-03-22T06:14:55.179 INFO:tasks.workunit.client.0.smithi006.stdout:0/221: truncate db/d10/d35/f43 842937 0 2024-03-22T06:14:55.179 INFO:tasks.workunit.client.0.smithi006.stdout:8/341: write d9/db/f7b [693790,16291] 0 2024-03-22T06:14:55.179 INFO:tasks.workunit.client.0.smithi006.stdout:8/342: truncate d9/db/dc/f62 619911 0 2024-03-22T06:14:55.179 INFO:tasks.workunit.client.0.smithi006.stdout:4/315: write d6/d13/d3b/f51 [163257,126267] 0 2024-03-22T06:14:55.183 INFO:tasks.workunit.client.0.smithi006.stdout:9/198: mkdir d8/d10/d28/d48 0 2024-03-22T06:14:55.185 INFO:tasks.workunit.client.0.smithi006.stdout:2/288: creat d13/f6f x:0 0 0 2024-03-22T06:14:55.185 INFO:tasks.workunit.client.0.smithi006.stdout:2/289: chown d13/d34/l4f 15709 1 2024-03-22T06:14:55.185 INFO:tasks.workunit.client.0.smithi006.stdout:2/290: fsync d13/d20/d26/d2a/d62/f66 0 2024-03-22T06:14:55.186 INFO:tasks.workunit.client.0.smithi006.stdout:6/285: dread d6/f18 [0,4194304] 0 2024-03-22T06:14:55.187 INFO:tasks.workunit.client.0.smithi006.stdout:6/286: stat d6/f51 0 2024-03-22T06:14:55.188 INFO:tasks.workunit.client.0.smithi006.stdout:0/222: mkdir db/dc/d38/d50 0 2024-03-22T06:14:55.188 INFO:tasks.workunit.client.0.smithi006.stdout:0/223: fdatasync db/d10/f2f 0 2024-03-22T06:14:55.188 INFO:tasks.workunit.client.0.smithi006.stdout:0/224: creat db/d10/d2a/f51 x:0 0 0 2024-03-22T06:14:55.189 INFO:tasks.workunit.client.0.smithi006.stdout:8/343: mkdir d9/db/d12/d66/d6c/d80/d85 0 2024-03-22T06:14:55.192 INFO:tasks.workunit.client.0.smithi006.stdout:4/316: mkdir d6/d13/d71 0 2024-03-22T06:14:55.192 INFO:tasks.workunit.client.0.smithi006.stdout:4/317: write d6/d13/d3b/f4e [812164,126700] 0 2024-03-22T06:14:55.192 INFO:tasks.workunit.client.0.smithi006.stdout:4/318: chown d6/fb 0 1 2024-03-22T06:14:55.193 INFO:tasks.workunit.client.0.smithi006.stdout:5/319: dwrite d0/db/d16/f21 [4194304,4194304] 0 2024-03-22T06:14:55.194 INFO:tasks.workunit.client.0.smithi006.stdout:8/344: read d9/db/d12/f26 [3821657,119008] 0 2024-03-22T06:14:55.202 INFO:tasks.workunit.client.0.smithi006.stdout:7/287: dwrite d5/d1e/d29/f2a [0,4194304] 0 2024-03-22T06:14:55.202 INFO:tasks.workunit.client.0.smithi006.stdout:7/288: write d5/d1e/d29/d1f/f3e [637254,14794] 0 2024-03-22T06:14:55.202 INFO:tasks.workunit.client.0.smithi006.stdout:0/225: mknod db/d3d/d4e/c52 0 2024-03-22T06:14:55.206 INFO:tasks.workunit.client.0.smithi006.stdout:4/319: symlink d6/d13/d3b/l72 0 2024-03-22T06:14:55.211 INFO:tasks.workunit.client.0.smithi006.stdout:5/320: mkdir d0/db/d16/d1c/d34/d3c/d5c 0 2024-03-22T06:14:55.212 INFO:tasks.workunit.client.0.smithi006.stdout:0/226: write db/d10/d2a/f34 [1682364,49997] 0 2024-03-22T06:14:55.214 INFO:tasks.workunit.client.0.smithi006.stdout:1/208: dwrite d8/d9/d3a/f20 [0,4194304] 0 2024-03-22T06:14:55.214 INFO:tasks.workunit.client.0.smithi006.stdout:1/209: fdatasync d8/d9/d13/d24/d32/f41 0 2024-03-22T06:14:55.217 INFO:tasks.workunit.client.0.smithi006.stdout:9/199: dwrite d8/d10/f17 [0,4194304] 0 2024-03-22T06:14:55.217 INFO:tasks.workunit.client.0.smithi006.stdout:7/289: mknod d5/d1e/d29/d10/c5b 0 2024-03-22T06:14:55.219 INFO:tasks.workunit.client.0.smithi006.stdout:4/320: dread d6/f1f [0,4194304] 0 2024-03-22T06:14:55.219 INFO:tasks.workunit.client.0.smithi006.stdout:8/345: dwrite d9/db/dc/de/d3e/f73 [0,4194304] 0 2024-03-22T06:14:55.219 INFO:tasks.workunit.client.0.smithi006.stdout:8/346: readlink d9/d14/d47/l4e 0 2024-03-22T06:14:55.220 INFO:tasks.workunit.client.0.smithi006.stdout:5/321: rename d0/db/dc/d59 to d0/db/d16/d1c/d34/d35/d5d 0 2024-03-22T06:14:55.233 INFO:tasks.workunit.client.0.smithi006.stdout:8/347: dread d9/db/d35/f55 [0,4194304] 0 2024-03-22T06:14:55.233 INFO:tasks.workunit.client.0.smithi006.stdout:8/348: chown d9/db/d12/d66/d6c/l6e 63193 1 2024-03-22T06:14:55.233 INFO:tasks.workunit.client.0.smithi006.stdout:8/349: read d9/db/f20 [1527450,105538] 0 2024-03-22T06:14:55.239 INFO:tasks.workunit.client.0.smithi006.stdout:3/309: dwrite d0/d23/d27/f59 [4194304,4194304] 0 2024-03-22T06:14:55.240 INFO:tasks.workunit.client.0.smithi006.stdout:3/310: dread - d0/d23/d27/f47 zero size 2024-03-22T06:14:55.240 INFO:tasks.workunit.client.0.smithi006.stdout:1/210: rename d8/d9/d3a/d22 to d8/d9/d3a/d40/d44 0 2024-03-22T06:14:55.240 INFO:tasks.workunit.client.0.smithi006.stdout:1/211: dread - d8/d9/d3a/d18/f37 zero size 2024-03-22T06:14:55.241 INFO:tasks.workunit.client.0.smithi006.stdout:4/321: dwrite d6/f42 [4194304,4194304] 0 2024-03-22T06:14:55.241 INFO:tasks.workunit.client.0.smithi006.stdout:4/322: write d6/ff [4267082,22524] 0 2024-03-22T06:14:55.241 INFO:tasks.workunit.client.0.smithi006.stdout:4/323: fdatasync d6/f1f 0 2024-03-22T06:14:55.242 INFO:tasks.workunit.client.0.smithi006.stdout:5/322: getdents d0/db/d16/d1c 0 2024-03-22T06:14:55.243 INFO:tasks.workunit.client.0.smithi006.stdout:2/291: fsync d13/f6f 0 2024-03-22T06:14:55.248 INFO:tasks.workunit.client.0.smithi006.stdout:3/311: link d0/d8/d1c/f4f d0/d8/d1c/d49/f5e 0 2024-03-22T06:14:55.249 INFO:tasks.workunit.client.0.smithi006.stdout:8/350: dwrite d9/d14/d47/f5c [4194304,4194304] 0 2024-03-22T06:14:55.249 INFO:tasks.workunit.client.0.smithi006.stdout:3/312: write d0/d8/d1c/f2c [875377,8809] 0 2024-03-22T06:14:55.253 INFO:tasks.workunit.client.0.smithi006.stdout:1/212: truncate d8/d9/d13/f26 2423846 0 2024-03-22T06:14:55.262 INFO:tasks.workunit.client.0.smithi006.stdout:5/323: mkdir d0/db/d16/d1c/d34/d3c/d5e 0 2024-03-22T06:14:55.265 INFO:tasks.workunit.client.0.smithi006.stdout:2/292: link fe d13/d34/d6e/f70 0 2024-03-22T06:14:55.271 INFO:tasks.workunit.client.0.smithi006.stdout:5/324: creat d0/db/d50/d58/f5f x:0 0 0 2024-03-22T06:14:55.276 INFO:tasks.workunit.client.0.smithi006.stdout:8/351: dwrite d9/db/dc/de/f17 [0,4194304] 0 2024-03-22T06:14:55.278 INFO:tasks.workunit.client.0.smithi006.stdout:1/213: dwrite d8/d9/d13/f28 [4194304,4194304] 0 2024-03-22T06:14:55.278 INFO:tasks.workunit.client.0.smithi006.stdout:4/324: rename d6/d1b/d20/d2c to d6/d13/d73 0 2024-03-22T06:14:55.278 INFO:tasks.workunit.client.0.smithi006.stdout:4/325: write d6/d1b/d20/f33 [271809,21529] 0 2024-03-22T06:14:55.278 INFO:tasks.workunit.client.0.smithi006.stdout:8/352: write d9/db/d37/f41 [605192,114114] 0 2024-03-22T06:14:55.278 INFO:tasks.workunit.client.0.smithi006.stdout:8/353: chown d9/l7a 401852051 1 2024-03-22T06:14:55.279 INFO:tasks.workunit.client.0.smithi006.stdout:5/325: creat d0/db/f60 x:0 0 0 2024-03-22T06:14:55.279 INFO:tasks.workunit.client.0.smithi006.stdout:5/326: chown d0/db/d11 828 1 2024-03-22T06:14:55.280 INFO:tasks.workunit.client.0.smithi006.stdout:2/293: rmdir d13/d34/d4d 39 2024-03-22T06:14:55.282 INFO:tasks.workunit.client.0.smithi006.stdout:7/290: truncate d5/d1e/d29/d1f/f3e 523873 0 2024-03-22T06:14:55.284 INFO:tasks.workunit.client.0.smithi006.stdout:5/327: write d0/db/f25 [734387,129641] 0 2024-03-22T06:14:55.285 INFO:tasks.workunit.client.0.smithi006.stdout:3/313: rename d0/d46 to d0/d8/d1c/d3e/d5f 0 2024-03-22T06:14:55.288 INFO:tasks.workunit.client.0.smithi006.stdout:1/214: dread d8/d9/d13/f28 [0,4194304] 0 2024-03-22T06:14:55.288 INFO:tasks.workunit.client.0.smithi006.stdout:8/354: symlink d9/d14/d44/l86 0 2024-03-22T06:14:55.289 INFO:tasks.workunit.client.0.smithi006.stdout:2/294: dread d13/d20/d26/f15 [0,4194304] 0 2024-03-22T06:14:55.290 INFO:tasks.workunit.client.0.smithi006.stdout:7/291: mkdir d5/d1e/d29/d59/d5c 0 2024-03-22T06:14:55.296 INFO:tasks.workunit.client.0.smithi006.stdout:5/328: rmdir d0/db/d16/d1c 39 2024-03-22T06:14:55.296 INFO:tasks.workunit.client.0.smithi006.stdout:5/329: dread - d0/db/d16/f49 zero size 2024-03-22T06:14:55.298 INFO:tasks.workunit.client.0.smithi006.stdout:5/330: write d0/db/d16/d1c/f4c [233219,111889] 0 2024-03-22T06:14:55.299 INFO:tasks.workunit.client.0.smithi006.stdout:3/314: creat d0/d8/d1c/d3e/f60 x:0 0 0 2024-03-22T06:14:55.303 INFO:tasks.workunit.client.0.smithi006.stdout:1/215: link d8/d9/d13/d24/d32/d34/f36 d8/d9/d13/d24/d32/d34/f45 0 2024-03-22T06:14:55.304 INFO:tasks.workunit.client.0.smithi006.stdout:1/216: fsync d8/d9/f1f 0 2024-03-22T06:14:55.304 INFO:tasks.workunit.client.0.smithi006.stdout:8/355: symlink d9/db/d12/d66/d6c/d80/l87 0 2024-03-22T06:14:55.305 INFO:tasks.workunit.client.0.smithi006.stdout:8/356: stat d9/db/dc 0 2024-03-22T06:14:55.307 INFO:tasks.workunit.client.0.smithi006.stdout:2/295: rename d13/d34/l4f to d13/l71 0 2024-03-22T06:14:55.308 INFO:tasks.workunit.client.0.smithi006.stdout:2/296: write d13/d20/d35/f45 [679655,71019] 0 2024-03-22T06:14:55.309 INFO:tasks.workunit.client.0.smithi006.stdout:4/326: truncate d6/ff 3365690 0 2024-03-22T06:14:55.312 INFO:tasks.workunit.client.0.smithi006.stdout:1/217: dread d8/d9/d3a/d18/f1b [0,4194304] 0 2024-03-22T06:14:55.315 INFO:tasks.workunit.client.0.smithi006.stdout:5/331: mkdir d0/db/d11/d61 0 2024-03-22T06:14:55.316 INFO:tasks.workunit.client.0.smithi006.stdout:3/315: unlink d0/d8/d9/d24/c29 0 2024-03-22T06:14:55.317 INFO:tasks.workunit.client.0.smithi006.stdout:3/316: truncate d0/f56 322635 0 2024-03-22T06:14:55.318 INFO:tasks.workunit.client.0.smithi006.stdout:8/357: rename f5 to d9/db/d37/d76/f88 0 2024-03-22T06:14:55.320 INFO:tasks.workunit.client.0.smithi006.stdout:2/297: rmdir d13/d20/d26/d19 39 2024-03-22T06:14:55.322 INFO:tasks.workunit.client.0.smithi006.stdout:5/332: write d0/db/d16/f46 [3911385,23783] 0 2024-03-22T06:14:55.323 INFO:tasks.workunit.client.0.smithi006.stdout:5/333: chown d0/db/d16/d1c/d34/d3c/d54 1440010 1 2024-03-22T06:14:55.323 INFO:tasks.workunit.client.0.smithi006.stdout:1/218: mknod d8/d9/d13/d39/c46 0 2024-03-22T06:14:55.326 INFO:tasks.workunit.client.0.smithi006.stdout:3/317: creat d0/d23/d4c/f61 x:0 0 0 2024-03-22T06:14:55.329 INFO:tasks.workunit.client.0.smithi006.stdout:8/358: mkdir d9/db/d12/d66/d6c/d80/d85/d89 0 2024-03-22T06:14:55.331 INFO:tasks.workunit.client.0.smithi006.stdout:5/334: dread d0/db/dc/d14/f1d [0,4194304] 0 2024-03-22T06:14:55.331 INFO:tasks.workunit.client.0.smithi006.stdout:2/298: mkdir d13/d34/d6e/d72 0 2024-03-22T06:14:55.334 INFO:tasks.workunit.client.0.smithi006.stdout:4/327: getdents d6/d1b/d46 0 2024-03-22T06:14:55.334 INFO:tasks.workunit.client.0.smithi006.stdout:4/328: read - d6/d13/d3b/f6e zero size 2024-03-22T06:14:55.339 INFO:tasks.workunit.client.0.smithi006.stdout:8/359: write d9/db/d37/d4a/f74 [2253971,83188] 0 2024-03-22T06:14:55.339 INFO:tasks.workunit.client.0.smithi006.stdout:8/360: chown d9/db/d37/d4a/c57 18 1 2024-03-22T06:14:55.339 INFO:tasks.workunit.client.0.smithi006.stdout:8/361: write d9/db/d12/f33 [1909169,39888] 0 2024-03-22T06:14:55.339 INFO:tasks.workunit.client.0.smithi006.stdout:8/362: fdatasync d9/d14/d44/f52 0 2024-03-22T06:14:55.339 INFO:tasks.workunit.client.0.smithi006.stdout:8/363: chown d9/db/d12/d66/d6c 8 1 2024-03-22T06:14:55.340 INFO:tasks.workunit.client.0.smithi006.stdout:8/364: fsync d9/d14/d47/f64 0 2024-03-22T06:14:55.340 INFO:tasks.workunit.client.0.smithi006.stdout:8/365: dread - d9/d14/d47/f64 zero size 2024-03-22T06:14:55.340 INFO:tasks.workunit.client.0.smithi006.stdout:1/219: link d8/d9/d13/d39/l42 d8/d9/l47 0 2024-03-22T06:14:55.340 INFO:tasks.workunit.client.0.smithi006.stdout:1/220: stat d8/d9/d3a/d18 0 2024-03-22T06:14:55.340 INFO:tasks.workunit.client.0.smithi006.stdout:3/318: mknod d0/d8/c62 0 2024-03-22T06:14:55.341 INFO:tasks.workunit.client.0.smithi006.stdout:1/221: dread d8/d9/f2d [0,4194304] 0 2024-03-22T06:14:55.349 INFO:tasks.workunit.client.0.smithi006.stdout:1/222: write d8/d9/d3a/f20 [1397220,1569] 0 2024-03-22T06:14:55.358 INFO:tasks.workunit.client.0.smithi006.stdout:1/223: dread d8/d9/f35 [0,4194304] 0 2024-03-22T06:14:55.358 INFO:tasks.workunit.client.0.smithi006.stdout:1/224: chown d8/d9/l3b 248175 1 2024-03-22T06:14:55.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.397+0000 7f6703d1d700 1 -- 172.21.15.6:0/2739820704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66fc0fcf80 msgr2=0x7f66fc0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:55.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.397+0000 7f6703d1d700 1 --2- 172.21.15.6:0/2739820704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66fc0fcf80 0x7f66fc0fd3f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f66f0004940 tx=0x7f66f0009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:55.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.397+0000 7f6703d1d700 1 -- 172.21.15.6:0/2739820704 shutdown_connections 2024-03-22T06:14:55.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.397+0000 7f6703d1d700 1 --2- 172.21.15.6:0/2739820704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66fc0fcf80 0x7f66fc0fd3f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:55.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.397+0000 7f6703d1d700 1 --2- 172.21.15.6:0/2739820704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc0fbcd0 0x7f66fc0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:55.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.397+0000 7f6703d1d700 1 -- 172.21.15.6:0/2739820704 >> 172.21.15.6:0/2739820704 conn(0x7f66fc0f7450 msgr2=0x7f66fc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:55.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.397+0000 7f6703d1d700 1 -- 172.21.15.6:0/2739820704 shutdown_connections 2024-03-22T06:14:55.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.398+0000 7f6703d1d700 1 -- 172.21.15.6:0/2739820704 wait complete. 2024-03-22T06:14:55.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.398+0000 7f6703d1d700 1 Processor -- start 2024-03-22T06:14:55.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.398+0000 7f6703d1d700 1 -- start start 2024-03-22T06:14:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.398+0000 7f6703d1d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc0fbcd0 0x7f66fc0ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.398+0000 7f6703d1d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66fc0fcf80 0x7f66fc0fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.398+0000 7f6703d1d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1001c0 con 0x7f66fc0fcf80 2024-03-22T06:14:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.398+0000 7f6703d1d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0fe800 con 0x7f66fc0fbcd0 2024-03-22T06:14:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.398+0000 7f67012b8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66fc0fcf80 0x7f66fc0fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.398+0000 7f67012b8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66fc0fcf80 0x7f66fc0fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33104/0 (socket says 172.21.15.6:33104) 2024-03-22T06:14:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.398+0000 7f67012b8700 1 -- 172.21.15.6:0/3153500134 learned_addr learned my addr 172.21.15.6:0/3153500134 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.398+0000 7f6701ab9700 1 --2- 172.21.15.6:0/3153500134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc0fbcd0 0x7f66fc0ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.399+0000 7f67012b8700 1 -- 172.21.15.6:0/3153500134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc0fbcd0 msgr2=0x7f66fc0ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.399+0000 7f67012b8700 1 --2- 172.21.15.6:0/3153500134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc0fbcd0 0x7f66fc0ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:55.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.399+0000 7f67012b8700 1 -- 172.21.15.6:0/3153500134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0005040 con 0x7f66fc0fcf80 2024-03-22T06:14:55.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.399+0000 7f67012b8700 1 --2- 172.21.15.6:0/3153500134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66fc0fcf80 0x7f66fc0fe2c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f66f000b3b0 tx=0x7f66f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:55.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.399+0000 7f66ee7fc700 1 -- 172.21.15.6:0/3153500134 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66f000e040 con 0x7f66fc0fcf80 2024-03-22T06:14:55.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.399+0000 7f6703d1d700 1 -- 172.21.15.6:0/3153500134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc0fea80 con 0x7f66fc0fcf80 2024-03-22T06:14:55.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.399+0000 7f66ee7fc700 1 -- 172.21.15.6:0/3153500134 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66f0004290 con 0x7f66fc0fcf80 2024-03-22T06:14:55.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.399+0000 7f66ee7fc700 1 -- 172.21.15.6:0/3153500134 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66f001c3f0 con 0x7f66fc0fcf80 2024-03-22T06:14:55.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.399+0000 7f6703d1d700 1 -- 172.21.15.6:0/3153500134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc0fefc0 con 0x7f66fc0fcf80 2024-03-22T06:14:55.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.400+0000 7f6703d1d700 1 -- 172.21.15.6:0/3153500134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc101b10 con 0x7f66fc0fcf80 2024-03-22T06:14:55.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.400+0000 7f66ee7fc700 1 -- 172.21.15.6:0/3153500134 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f66f000bbf0 con 0x7f66fc0fcf80 2024-03-22T06:14:55.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.401+0000 7f66ee7fc700 1 --2- 172.21.15.6:0/3153500134 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f66e806bb20 0x7f66e806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:55.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.401+0000 7f66ee7fc700 1 -- 172.21.15.6:0/3153500134 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f66f008bb30 con 0x7f66fc0fcf80 2024-03-22T06:14:55.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.401+0000 7f6701ab9700 1 --2- 172.21.15.6:0/3153500134 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f66e806bb20 0x7f66e806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:55.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.402+0000 7f6701ab9700 1 --2- 172.21.15.6:0/3153500134 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f66e806bb20 0x7f66e806dfd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f66f8007fd0 tx=0x7f66f8009040 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:55.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.406+0000 7f66ee7fc700 1 -- 172.21.15.6:0/3153500134 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f66f0058e60 con 0x7f66fc0fcf80 2024-03-22T06:14:55.457 INFO:tasks.workunit.client.0.smithi006.stdout:5/335: link d0/db/d16/l2a d0/db/l62 0 2024-03-22T06:14:55.457 INFO:tasks.workunit.client.0.smithi006.stdout:5/336: creat d0/db/d11/f63 x:0 0 0 2024-03-22T06:14:55.461 INFO:tasks.workunit.client.0.smithi006.stdout:2/299: link d13/d53/f38 d13/d20/d26/d19/f73 0 2024-03-22T06:14:55.461 INFO:tasks.workunit.client.0.smithi006.stdout:2/300: truncate d13/d20/d26/d19/f73 177680 0 2024-03-22T06:14:55.461 INFO:tasks.workunit.client.0.smithi006.stdout:2/301: truncate d13/d34/d4d/f5c 162239 0 2024-03-22T06:14:55.462 INFO:tasks.workunit.client.0.smithi006.stdout:4/329: mknod d6/d13/d3b/c74 0 2024-03-22T06:14:55.469 INFO:tasks.workunit.client.0.smithi006.stdout:8/366: truncate d9/db/d37/d76/f88 215735 0 2024-03-22T06:14:55.469 INFO:tasks.workunit.client.0.smithi006.stdout:8/367: read - d9/d5e/f5f zero size 2024-03-22T06:14:55.477 INFO:tasks.workunit.client.0.smithi006.stdout:1/225: rename d8/d9/d3a/d18/c21 to d8/d33/c48 0 2024-03-22T06:14:55.479 INFO:tasks.workunit.client.0.smithi006.stdout:4/330: dwrite f0 [0,4194304] 0 2024-03-22T06:14:55.480 INFO:tasks.workunit.client.0.smithi006.stdout:4/331: write d6/f12 [2510416,27077] 0 2024-03-22T06:14:55.480 INFO:tasks.workunit.client.0.smithi006.stdout:4/332: fdatasync d6/d13/d3b/f4f 0 2024-03-22T06:14:55.480 INFO:tasks.workunit.client.0.smithi006.stdout:4/333: readlink d6/d1b/d20/d6b/d60/l69 0 2024-03-22T06:14:55.481 INFO:tasks.workunit.client.0.smithi006.stdout:1/226: dread d8/d9/f2d [0,4194304] 0 2024-03-22T06:14:55.487 INFO:tasks.workunit.client.0.smithi006.stdout:2/302: unlink d13/d20/d26/d2a/l2c 0 2024-03-22T06:14:55.489 INFO:tasks.workunit.client.0.smithi006.stdout:8/368: link d9/db/dc/de/f83 d9/db/d37/d4a/f8a 0 2024-03-22T06:14:55.489 INFO:tasks.workunit.client.0.smithi006.stdout:8/369: write d9/db/d35/f6f [583125,41171] 0 2024-03-22T06:14:55.490 INFO:tasks.workunit.client.0.smithi006.stdout:8/370: truncate d9/db/d12/d66/d6c/f63 60239 0 2024-03-22T06:14:55.490 INFO:tasks.workunit.client.0.smithi006.stdout:3/319: getdents d0/d8/d9 0 2024-03-22T06:14:55.492 INFO:tasks.workunit.client.0.smithi006.stdout:4/334: getdents d6/d1b/d20/d21/d2a/d43/d4b 0 2024-03-22T06:14:55.494 INFO:tasks.workunit.client.0.smithi006.stdout:1/227: getdents d8/d9/d10 0 2024-03-22T06:14:55.496 INFO:tasks.workunit.client.0.smithi006.stdout:2/303: creat d13/d20/f74 x:0 0 0 2024-03-22T06:14:55.501 INFO:tasks.workunit.client.0.smithi006.stdout:3/320: rename d0/d8/d1c/f41 to d0/d23/f63 0 2024-03-22T06:14:55.502 INFO:tasks.workunit.client.0.smithi006.stdout:4/335: rmdir d6/d1b/d20 39 2024-03-22T06:14:55.504 INFO:tasks.workunit.client.0.smithi006.stdout:1/228: symlink d8/d9/l49 0 2024-03-22T06:14:55.505 INFO:tasks.workunit.client.0.smithi006.stdout:2/304: truncate f6 2691711 0 2024-03-22T06:14:55.505 INFO:tasks.workunit.client.0.smithi006.stdout:8/371: dwrite d9/d14/f34 [0,4194304] 0 2024-03-22T06:14:55.509 INFO:tasks.workunit.client.0.smithi006.stdout:8/372: dread d9/db/d37/d4a/f74 [0,4194304] 0 2024-03-22T06:14:55.509 INFO:tasks.workunit.client.0.smithi006.stdout:4/336: creat d6/d1b/d20/d6b/f75 x:0 0 0 2024-03-22T06:14:55.511 INFO:tasks.workunit.client.0.smithi006.stdout:1/229: write d8/d9/d13/f26 [690713,126340] 0 2024-03-22T06:14:55.511 INFO:tasks.workunit.client.0.smithi006.stdout:1/230: write d8/d9/d10/f19 [2096317,17419] 0 2024-03-22T06:14:55.511 INFO:tasks.workunit.client.0.smithi006.stdout:1/231: chown d8/d9/d3a/d18/f1b 6 1 2024-03-22T06:14:55.517 INFO:tasks.workunit.client.0.smithi006.stdout:2/305: read d13/f23 [3162,106143] 0 2024-03-22T06:14:55.517 INFO:tasks.workunit.client.0.smithi006.stdout:2/306: write d13/d20/f42 [1508895,73382] 0 2024-03-22T06:14:55.517 INFO:tasks.workunit.client.0.smithi006.stdout:2/307: write d13/d20/d26/d19/f73 [1216282,100165] 0 2024-03-22T06:14:55.517 INFO:tasks.workunit.client.0.smithi006.stdout:2/308: write d13/d53/f4a [237795,29334] 0 2024-03-22T06:14:55.517 INFO:tasks.workunit.client.0.smithi006.stdout:3/321: symlink d0/l64 0 2024-03-22T06:14:55.517 INFO:tasks.workunit.client.0.smithi006.stdout:3/322: write d0/d23/d4c/f61 [705808,68606] 0 2024-03-22T06:14:55.517 INFO:tasks.workunit.client.0.smithi006.stdout:3/323: fsync d0/d23/d33/f44 0 2024-03-22T06:14:55.517 INFO:tasks.workunit.client.0.smithi006.stdout:3/324: creat d0/d23/d33/f65 x:0 0 0 2024-03-22T06:14:55.519 INFO:tasks.workunit.client.0.smithi006.stdout:8/373: truncate d9/db/dc/fd 2491909 0 2024-03-22T06:14:55.521 INFO:tasks.workunit.client.0.smithi006.stdout:4/337: rmdir d6/d1b/d46 39 2024-03-22T06:14:55.529 INFO:tasks.workunit.client.0.smithi006.stdout:1/232: dwrite d8/d9/d3a/d18/f1b [0,4194304] 0 2024-03-22T06:14:55.529 INFO:tasks.workunit.client.0.smithi006.stdout:1/233: chown d8/d9/d3a/d18/f1b 0 1 2024-03-22T06:14:55.532 INFO:tasks.workunit.client.0.smithi006.stdout:3/325: symlink d0/d23/d33/d3d/d58/l66 0 2024-03-22T06:14:55.536 INFO:tasks.workunit.client.0.smithi006.stdout:8/374: creat d9/db/d12/d7e/f8b x:0 0 0 2024-03-22T06:14:55.538 INFO:tasks.workunit.client.0.smithi006.stdout:4/338: symlink d6/d1b/d20/d21/l76 0 2024-03-22T06:14:55.540 INFO:tasks.workunit.client.0.smithi006.stdout:3/326: dread d0/d8/d9/f32 [0,4194304] 0 2024-03-22T06:14:55.540 INFO:tasks.workunit.client.0.smithi006.stdout:3/327: chown d0/d8/f13 206 1 2024-03-22T06:14:55.541 INFO:tasks.workunit.client.0.smithi006.stdout:4/339: dread d6/d13/d3b/f51 [0,4194304] 0 2024-03-22T06:14:55.545 INFO:tasks.workunit.client.0.smithi006.stdout:8/375: write d9/db/dc/f1f [3975359,91399] 0 2024-03-22T06:14:55.546 INFO:tasks.workunit.client.0.smithi006.stdout:1/234: mknod d8/d9/c4a 0 2024-03-22T06:14:55.554 INFO:tasks.workunit.client.0.smithi006.stdout:1/235: dread d8/d9/f35 [4194304,4194304] 0 2024-03-22T06:14:55.554 INFO:tasks.workunit.client.0.smithi006.stdout:1/236: fsync d8/d9/d3a/d18/f1b 0 2024-03-22T06:14:55.554 INFO:tasks.workunit.client.0.smithi006.stdout:1/237: creat d8/d9/f4b x:0 0 0 2024-03-22T06:14:55.582 INFO:tasks.workunit.client.0.smithi006.stdout:3/328: creat d0/d8/d1c/d49/f67 x:0 0 0 2024-03-22T06:14:55.583 INFO:tasks.workunit.client.0.smithi006.stdout:3/329: truncate d0/d8/d1c/d49/f67 627669 0 2024-03-22T06:14:55.584 INFO:tasks.workunit.client.0.smithi006.stdout:4/340: mkdir d6/d1b/d20/d21/d77 0 2024-03-22T06:14:55.584 INFO:tasks.workunit.client.0.smithi006.stdout:4/341: chown d6/d1b/d46 415242570 1 2024-03-22T06:14:55.584 INFO:tasks.workunit.client.0.smithi006.stdout:4/342: readlink d6/d1b/l2f 0 2024-03-22T06:14:55.586 INFO:tasks.workunit.client.0.smithi006.stdout:2/309: rename d13/d20/d26/d2a to d13/d34/d4d/d4e/d75 0 2024-03-22T06:14:55.588 INFO:tasks.workunit.client.0.smithi006.stdout:8/376: symlink d9/db/d37/d4a/l8c 0 2024-03-22T06:14:55.588 INFO:tasks.workunit.client.0.smithi006.stdout:8/377: write d9/db/f7b [649438,42821] 0 2024-03-22T06:14:55.589 INFO:tasks.workunit.client.0.smithi006.stdout:2/310: dread d13/d20/f4b [0,4194304] 0 2024-03-22T06:14:55.592 INFO:tasks.workunit.client.0.smithi006.stdout:1/238: symlink d8/d9/d13/d24/l4c 0 2024-03-22T06:14:55.592 INFO:tasks.workunit.client.0.smithi006.stdout:4/343: write d6/d13/f6a [653041,78856] 0 2024-03-22T06:14:55.595 INFO:tasks.workunit.client.0.smithi006.stdout:3/330: rename d0/d23/d27/d36 to d0/d8/d1c/d3e/d68 0 2024-03-22T06:14:55.595 INFO:tasks.workunit.client.0.smithi006.stdout:3/331: stat d0/d8/d1c/d49/f5e 0 2024-03-22T06:14:55.596 INFO:tasks.workunit.client.0.smithi006.stdout:2/311: creat d13/d53/d6b/f76 x:0 0 0 2024-03-22T06:14:55.596 INFO:tasks.workunit.client.0.smithi006.stdout:2/312: creat d13/d20/d35/d65/f77 x:0 0 0 2024-03-22T06:14:55.599 INFO:tasks.workunit.client.0.smithi006.stdout:3/332: dread d0/d23/d4c/f30 [0,4194304] 0 2024-03-22T06:14:55.599 INFO:tasks.workunit.client.0.smithi006.stdout:3/333: chown d0/d23/f63 2921577 1 2024-03-22T06:14:55.599 INFO:tasks.workunit.client.0.smithi006.stdout:1/239: getdents d8/d9/d13 0 2024-03-22T06:14:55.606 INFO:tasks.workunit.client.0.smithi006.stdout:4/344: link d6/d1b/d20/l39 d6/d1b/d20/d21/d77/l78 0 2024-03-22T06:14:55.606 INFO:tasks.workunit.client.0.smithi006.stdout:4/345: dread - d6/f1c zero size 2024-03-22T06:14:55.606 INFO:tasks.workunit.client.0.smithi006.stdout:1/240: dread d8/d9/f35 [0,4194304] 0 2024-03-22T06:14:55.607 INFO:tasks.workunit.client.0.smithi006.stdout:2/313: creat d13/d53/d29/d6c/f78 x:0 0 0 2024-03-22T06:14:55.607 INFO:tasks.workunit.client.0.smithi006.stdout:2/314: readlink d13/d34/d4d/l54 0 2024-03-22T06:14:55.607 INFO:tasks.workunit.client.0.smithi006.stdout:2/315: chown d13/d34 1811292112 1 2024-03-22T06:14:55.607 INFO:tasks.workunit.client.0.smithi006.stdout:2/316: stat d13/d34/d6e 0 2024-03-22T06:14:55.609 INFO:tasks.workunit.client.0.smithi006.stdout:3/334: unlink d0/d23/d33/d3d/f5d 0 2024-03-22T06:14:55.609 INFO:tasks.workunit.client.0.smithi006.stdout:3/335: chown d0/d8/d1c/d3e/d68 1 1 2024-03-22T06:14:55.610 INFO:tasks.workunit.client.0.smithi006.stdout:8/378: dwrite d9/db/d12/d1e/f30 [0,4194304] 0 2024-03-22T06:14:55.612 INFO:tasks.workunit.client.0.smithi006.stdout:4/346: rename d6/d1b/l2f to d6/d13/l79 0 2024-03-22T06:14:55.613 INFO:tasks.workunit.client.0.smithi006.stdout:4/347: creat d6/d13/d3b/f7a x:0 0 0 2024-03-22T06:14:55.615 INFO:tasks.workunit.client.0.smithi006.stdout:1/241: creat d8/d33/f4d x:0 0 0 2024-03-22T06:14:55.615 INFO:tasks.workunit.client.0.smithi006.stdout:1/242: readlink d8/d9/d3a/d40/d44/l29 0 2024-03-22T06:14:55.617 INFO:tasks.workunit.client.0.smithi006.stdout:3/336: symlink d0/d4/d53/l69 0 2024-03-22T06:14:55.625 INFO:tasks.workunit.client.0.smithi006.stdout:8/379: rename d9/f11 to d9/db/d37/d4a/d4d/f8d 0 2024-03-22T06:14:55.625 INFO:tasks.workunit.client.0.smithi006.stdout:8/380: creat d9/d14/d44/f8e x:0 0 0 2024-03-22T06:14:55.628 INFO:tasks.workunit.client.0.smithi006.stdout:1/243: creat d8/d9/d10/f4e x:0 0 0 2024-03-22T06:14:55.630 INFO:tasks.workunit.client.0.smithi006.stdout:8/381: mknod d9/db/dc/de/c8f 0 2024-03-22T06:14:55.630 INFO:tasks.workunit.client.0.smithi006.stdout:8/382: creat d9/db/d12/f90 x:0 0 0 2024-03-22T06:14:55.630 INFO:tasks.workunit.client.0.smithi006.stdout:8/383: chown d9/db/d12/d66/d6c/d71/d79 104 1 2024-03-22T06:14:55.631 INFO:tasks.workunit.client.0.smithi006.stdout:8/384: dread - d9/db/d12/f90 zero size 2024-03-22T06:14:55.632 INFO:tasks.workunit.client.0.smithi006.stdout:2/317: dwrite d13/d20/d26/d19/f1c [0,4194304] 0 2024-03-22T06:14:55.633 INFO:tasks.workunit.client.0.smithi006.stdout:1/244: link d8/d9/c4a d8/d9/c4f 0 2024-03-22T06:14:55.633 INFO:tasks.workunit.client.0.smithi006.stdout:1/245: creat d8/d9/f50 x:0 0 0 2024-03-22T06:14:55.637 INFO:tasks.workunit.client.0.smithi006.stdout:2/318: rename d13/d34/d6e/f70 to d13/d20/d26/f79 0 2024-03-22T06:14:55.641 INFO:tasks.workunit.client.0.smithi006.stdout:1/246: symlink d8/d9/d3a/d18/l51 0 2024-03-22T06:14:55.646 INFO:tasks.workunit.client.0.smithi006.stdout:8/385: dwrite d9/db/d35/f6f [0,4194304] 0 2024-03-22T06:14:55.646 INFO:tasks.workunit.client.0.smithi006.stdout:8/386: write d9/db/dc/de/f17 [4848513,14730] 0 2024-03-22T06:14:55.646 INFO:tasks.workunit.client.0.smithi006.stdout:8/387: dread - d9/db/d12/d1e/f6d zero size 2024-03-22T06:14:55.646 INFO:tasks.workunit.client.0.smithi006.stdout:2/319: truncate d13/d20/d26/d19/f73 517030 0 2024-03-22T06:14:55.647 INFO:tasks.workunit.client.0.smithi006.stdout:1/247: symlink d8/d9/d13/d39/l52 0 2024-03-22T06:14:55.653 INFO:tasks.workunit.client.0.smithi006.stdout:8/388: mknod d9/db/dc/de/d3e/c91 0 2024-03-22T06:14:55.653 INFO:tasks.workunit.client.0.smithi006.stdout:8/389: read - d9/db/d37/d4a/f5b zero size 2024-03-22T06:14:55.653 INFO:tasks.workunit.client.0.smithi006.stdout:8/390: truncate d9/db/f7b 1251289 0 2024-03-22T06:14:55.656 INFO:tasks.workunit.client.0.smithi006.stdout:2/320: dread fc [0,4194304] 0 2024-03-22T06:14:55.657 INFO:tasks.workunit.client.0.smithi006.stdout:1/248: creat d8/d9/d3a/d40/d44/f53 x:0 0 0 2024-03-22T06:14:55.657 INFO:tasks.workunit.client.0.smithi006.stdout:1/249: fdatasync d8/d9/d3a/d40/d44/f53 0 2024-03-22T06:14:55.657 INFO:tasks.workunit.client.0.smithi006.stdout:1/250: creat d8/d9/f54 x:0 0 0 2024-03-22T06:14:55.659 INFO:tasks.workunit.client.0.smithi006.stdout:8/391: rmdir d9/db/d12/d66/d6c/d80 39 2024-03-22T06:14:55.659 INFO:tasks.workunit.client.0.smithi006.stdout:8/392: chown d9/db/d37 15369 1 2024-03-22T06:14:55.664 INFO:tasks.workunit.client.0.smithi006.stdout:1/251: mknod d8/d9/d13/c55 0 2024-03-22T06:14:55.664 INFO:tasks.workunit.client.0.smithi006.stdout:4/348: getdents d6/d13 0 2024-03-22T06:14:55.669 INFO:tasks.workunit.client.0.smithi006.stdout:8/393: truncate d9/db/d35/f3c 3376917 0 2024-03-22T06:14:55.673 INFO:tasks.workunit.client.0.smithi006.stdout:1/252: creat d8/d9/d3a/d18/f56 x:0 0 0 2024-03-22T06:14:55.673 INFO:tasks.workunit.client.0.smithi006.stdout:1/253: chown d8/d9/d3a/d40/d44/l29 2 1 2024-03-22T06:14:55.673 INFO:tasks.workunit.client.0.smithi006.stdout:1/254: readlink d8/d9/d13/d39/l52 0 2024-03-22T06:14:55.675 INFO:tasks.workunit.client.0.smithi006.stdout:4/349: mkdir d6/d1b/d20/d6b/d7b 0 2024-03-22T06:14:55.676 INFO:tasks.workunit.client.0.smithi006.stdout:8/394: mkdir d9/db/d12/d66/d6c/d92 0 2024-03-22T06:14:55.678 INFO:tasks.workunit.client.0.smithi006.stdout:1/255: mknod d8/d9/d13/d24/d32/c57 0 2024-03-22T06:14:55.681 INFO:tasks.workunit.client.0.smithi006.stdout:8/395: dread d9/db/dc/fd [0,4194304] 0 2024-03-22T06:14:55.681 INFO:tasks.workunit.client.0.smithi006.stdout:4/350: mknod d6/d13/c7c 0 2024-03-22T06:14:55.684 INFO:tasks.workunit.client.0.smithi006.stdout:1/256: rename d8/d9/d13/l3f to d8/d9/d3a/d18/d2a/l58 0 2024-03-22T06:14:55.684 INFO:tasks.workunit.client.0.smithi006.stdout:1/257: dread - d8/d9/f50 zero size 2024-03-22T06:14:55.684 INFO:tasks.workunit.client.0.smithi006.stdout:1/258: chown d8/d9/d3a/d18/f37 16 1 2024-03-22T06:14:55.686 INFO:tasks.workunit.client.0.smithi006.stdout:8/396: creat d9/db/d12/d66/d6c/d71/f93 x:0 0 0 2024-03-22T06:14:55.688 INFO:tasks.workunit.client.0.smithi006.stdout:1/259: unlink d8/d9/c23 0 2024-03-22T06:14:55.688 INFO:tasks.workunit.client.0.smithi006.stdout:1/260: chown d8/d9/l12 2612 1 2024-03-22T06:14:55.690 INFO:tasks.workunit.client.0.smithi006.stdout:8/397: mkdir d9/d14/d47/d94 0 2024-03-22T06:14:55.692 INFO:tasks.workunit.client.0.smithi006.stdout:1/261: mkdir d8/d9/d3a/d40/d59 0 2024-03-22T06:14:55.692 INFO:tasks.workunit.client.0.smithi006.stdout:1/262: dread - d8/d9/d13/d24/d32/f41 zero size 2024-03-22T06:14:55.695 INFO:tasks.workunit.client.0.smithi006.stdout:8/398: creat d9/d14/f95 x:0 0 0 2024-03-22T06:14:55.696 INFO:tasks.workunit.client.0.smithi006.stdout:8/399: fdatasync d9/db/d35/f55 0 2024-03-22T06:14:55.696 INFO:tasks.workunit.client.0.smithi006.stdout:8/400: creat d9/db/d37/d4a/f96 x:0 0 0 2024-03-22T06:14:55.696 INFO:tasks.workunit.client.0.smithi006.stdout:8/401: creat d9/d14/d44/f97 x:0 0 0 2024-03-22T06:14:55.696 INFO:tasks.workunit.client.0.smithi006.stdout:8/402: dread - d9/f21 zero size 2024-03-22T06:14:55.696 INFO:tasks.workunit.client.0.smithi006.stdout:8/403: write d9/d14/f95 [38364,52715] 0 2024-03-22T06:14:55.697 INFO:tasks.workunit.client.0.smithi006.stdout:1/263: creat d8/d9/d13/f5a x:0 0 0 2024-03-22T06:14:55.704 INFO:tasks.workunit.client.0.smithi006.stdout:8/404: mknod d9/db/d12/d66/d6c/d71/c98 0 2024-03-22T06:14:55.704 INFO:tasks.workunit.client.0.smithi006.stdout:8/405: fdatasync d9/db/d12/f5d 0 2024-03-22T06:14:55.711 INFO:tasks.workunit.client.0.smithi006.stdout:1/264: rmdir d8 39 2024-03-22T06:14:55.711 INFO:tasks.workunit.client.0.smithi006.stdout:1/265: dread - d8/d33/f4d zero size 2024-03-22T06:14:55.712 INFO:tasks.workunit.client.0.smithi006.stdout:8/406: write d9/db/f2e [1561562,93563] 0 2024-03-22T06:14:55.715 INFO:tasks.workunit.client.0.smithi006.stdout:1/266: dread d8/d9/f2d [0,4194304] 0 2024-03-22T06:14:55.716 INFO:tasks.workunit.client.0.smithi006.stdout:1/267: truncate d8/d9/d10/f19 3098979 0 2024-03-22T06:14:55.716 INFO:tasks.workunit.client.0.smithi006.stdout:1/268: dread - d8/d33/f4d zero size 2024-03-22T06:14:55.719 INFO:tasks.workunit.client.0.smithi006.stdout:8/407: symlink d9/d14/d47/l99 0 2024-03-22T06:14:55.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.721+0000 7f6703d1d700 1 -- 172.21.15.6:0/3153500134 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66fc05e400 con 0x7f66e806bb20 2024-03-22T06:14:55.722 INFO:tasks.workunit.client.0.smithi006.stdout:1/269: truncate d8/d9/d13/d24/d32/d34/f36 3302149 0 2024-03-22T06:14:55.722 INFO:tasks.workunit.client.0.smithi006.stdout:1/270: creat d8/d33/f5b x:0 0 0 2024-03-22T06:14:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.722+0000 7f66ee7fc700 1 -- 172.21.15.6:0/3153500134 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f66fc05e400 con 0x7f66e806bb20 2024-03-22T06:14:55.724 INFO:tasks.workunit.client.0.smithi006.stdout:8/408: write d9/db/dc/de/d3e/f3f [3206757,19845] 0 2024-03-22T06:14:55.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.725+0000 7f6703d1d700 1 -- 172.21.15.6:0/3153500134 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f66e806bb20 msgr2=0x7f66e806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:55.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.725+0000 7f6703d1d700 1 --2- 172.21.15.6:0/3153500134 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f66e806bb20 0x7f66e806dfd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f66f8007fd0 tx=0x7f66f8009040 comp rx=0 tx=0).stop 2024-03-22T06:14:55.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.725+0000 7f6703d1d700 1 -- 172.21.15.6:0/3153500134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66fc0fcf80 msgr2=0x7f66fc0fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:55.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.725+0000 7f6703d1d700 1 --2- 172.21.15.6:0/3153500134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66fc0fcf80 0x7f66fc0fe2c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f66f000b3b0 tx=0x7f66f0009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:55.726 INFO:tasks.workunit.client.0.smithi006.stdout:1/271: symlink d8/d9/d3a/l5c 0 2024-03-22T06:14:55.726 INFO:tasks.workunit.client.0.smithi006.stdout:1/272: truncate d8/d9/f50 157406 0 2024-03-22T06:14:55.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.725+0000 7f6703d1d700 1 -- 172.21.15.6:0/3153500134 shutdown_connections 2024-03-22T06:14:55.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.725+0000 7f6703d1d700 1 --2- 172.21.15.6:0/3153500134 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f66e806bb20 0x7f66e806dfd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:55.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.725+0000 7f6703d1d700 1 --2- 172.21.15.6:0/3153500134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66fc0fbcd0 0x7f66fc0ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:55.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.725+0000 7f6703d1d700 1 --2- 172.21.15.6:0/3153500134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66fc0fcf80 0x7f66fc0fe2c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:55.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.725+0000 7f6703d1d700 1 -- 172.21.15.6:0/3153500134 >> 172.21.15.6:0/3153500134 conn(0x7f66fc0f7450 msgr2=0x7f66fc066810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:55.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.725+0000 7f6703d1d700 1 -- 172.21.15.6:0/3153500134 shutdown_connections 2024-03-22T06:14:55.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.725+0000 7f6703d1d700 1 -- 172.21.15.6:0/3153500134 wait complete. 2024-03-22T06:14:55.730 INFO:tasks.workunit.client.0.smithi006.stdout:8/409: unlink d9/f27 0 2024-03-22T06:14:55.733 INFO:tasks.workunit.client.0.smithi006.stdout:1/273: mknod d8/d9/d13/d24/c5d 0 2024-03-22T06:14:55.733 INFO:tasks.workunit.client.0.smithi006.stdout:1/274: write d8/d33/f5b [319769,66868] 0 2024-03-22T06:14:55.733 INFO:tasks.workunit.client.0.smithi006.stdout:1/275: write d8/d9/f50 [465893,105107] 0 2024-03-22T06:14:55.733 INFO:tasks.workunit.client.0.smithi006.stdout:1/276: chown d8/d9/d3a/d40/d44/l2f 111077 1 2024-03-22T06:14:55.738 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:14:55.742 INFO:tasks.workunit.client.0.smithi006.stdout:1/277: link d8/d9/l49 d8/d9/d3a/d18/l5e 0 2024-03-22T06:14:55.745 INFO:tasks.workunit.client.0.smithi006.stdout:8/410: mknod d9/db/c9a 0 2024-03-22T06:14:55.745 INFO:tasks.workunit.client.0.smithi006.stdout:8/411: creat d9/db/d12/d66/f9b x:0 0 0 2024-03-22T06:14:55.745 INFO:tasks.workunit.client.0.smithi006.stdout:8/412: write d9/db/dc/f1f [1432020,27475] 0 2024-03-22T06:14:55.746 INFO:tasks.workunit.client.0.smithi006.stdout:1/278: rmdir d8 39 2024-03-22T06:14:55.746 INFO:tasks.workunit.client.0.smithi006.stdout:1/279: chown d8/d9/d13/d24/d32/d34/c3e 241028 1 2024-03-22T06:14:55.749 INFO:tasks.workunit.client.0.smithi006.stdout:8/413: symlink d9/db/d37/l9c 0 2024-03-22T06:14:55.751 INFO:tasks.workunit.client.0.smithi006.stdout:1/280: symlink d8/d9/d3a/d18/l5f 0 2024-03-22T06:14:55.754 INFO:tasks.workunit.client.0.smithi006.stdout:1/281: mkdir d8/d9/d13/d39/d60 0 2024-03-22T06:14:55.754 INFO:tasks.workunit.client.0.smithi006.stdout:1/282: readlink d8/d9/l12 0 2024-03-22T06:14:55.775 INFO:tasks.workunit.client.0.smithi006.stdout:8/414: dwrite d9/db/d12/d1e/f39 [0,4194304] 0 2024-03-22T06:14:55.775 INFO:tasks.workunit.client.0.smithi006.stdout:8/415: fdatasync d9/db/d37/f4b 0 2024-03-22T06:14:55.775 INFO:tasks.workunit.client.0.smithi006.stdout:1/283: dwrite d8/d9/d3a/d18/f1b [0,4194304] 0 2024-03-22T06:14:55.775 INFO:tasks.workunit.client.0.smithi006.stdout:1/284: creat d8/d9/d3a/d18/f61 x:0 0 0 2024-03-22T06:14:55.775 INFO:tasks.workunit.client.0.smithi006.stdout:1/285: creat d8/d9/d3a/d18/f62 x:0 0 0 2024-03-22T06:14:55.780 INFO:tasks.workunit.client.0.smithi006.stdout:8/416: symlink d9/d14/d47/d94/l9d 0 2024-03-22T06:14:55.786 INFO:tasks.workunit.client.0.smithi006.stdout:6/287: sync 2024-03-22T06:14:55.786 INFO:tasks.workunit.client.0.smithi006.stdout:1/286: dread d8/d9/d3a/d18/f1b [0,4194304] 0 2024-03-22T06:14:55.789 INFO:tasks.workunit.client.0.smithi006.stdout:6/288: symlink d6/d1d/l6d 0 2024-03-22T06:14:55.793 INFO:tasks.workunit.client.0.smithi006.stdout:6/289: truncate d6/d1d/f20 1816067 0 2024-03-22T06:14:55.796 INFO:tasks.workunit.client.0.smithi006.stdout:6/290: truncate d6/d1b/f48 479901 0 2024-03-22T06:14:55.803 INFO:tasks.workunit.client.0.smithi006.stdout:6/291: getdents d6/d24/d44 0 2024-03-22T06:14:55.811 INFO:tasks.workunit.client.0.smithi006.stdout:8/417: dwrite d9/db/f2e [0,4194304] 0 2024-03-22T06:14:55.812 INFO:tasks.workunit.client.0.smithi006.stdout:8/418: chown d9/d14/d47 29405403 1 2024-03-22T06:14:55.812 INFO:tasks.workunit.client.0.smithi006.stdout:8/419: fsync d9/d14/d44/f52 0 2024-03-22T06:14:55.812 INFO:tasks.workunit.client.0.smithi006.stdout:8/420: chown d9/db/d12/d7e 21549416 1 2024-03-22T06:14:55.812 INFO:tasks.workunit.client.0.smithi006.stdout:8/421: chown d9/d14/d47/l58 1967 1 2024-03-22T06:14:55.812 INFO:tasks.workunit.client.0.smithi006.stdout:1/287: dwrite d8/d9/d3a/d18/f1b [0,4194304] 0 2024-03-22T06:14:55.812 INFO:tasks.workunit.client.0.smithi006.stdout:1/288: readlink d8/d9/l2e 0 2024-03-22T06:14:55.814 INFO:tasks.workunit.client.0.smithi006.stdout:8/422: link d9/db/d37/d4a/c57 d9/db/d12/d7e/c9e 0 2024-03-22T06:14:55.817 INFO:tasks.workunit.client.0.smithi006.stdout:1/289: link d8/d9/f35 d8/d9/d13/d39/f63 0 2024-03-22T06:14:55.817 INFO:tasks.workunit.client.0.smithi006.stdout:1/290: write d8/d9/d3a/d40/d44/f53 [1035532,97603] 0 2024-03-22T06:14:55.818 INFO:tasks.workunit.client.0.smithi006.stdout:6/292: dwrite d6/d1b/d2c/f43 [0,4194304] 0 2024-03-22T06:14:55.818 INFO:tasks.workunit.client.0.smithi006.stdout:6/293: readlink d6/d1d/d32/l3f 0 2024-03-22T06:14:55.818 INFO:tasks.workunit.client.0.smithi006.stdout:6/294: chown d6/c35 30 1 2024-03-22T06:14:55.818 INFO:tasks.workunit.client.0.smithi006.stdout:6/295: write d6/d1b/f2d [1330597,82761] 0 2024-03-22T06:14:55.822 INFO:tasks.workunit.client.0.smithi006.stdout:1/291: dread d8/d9/f35 [0,4194304] 0 2024-03-22T06:14:55.823 INFO:tasks.workunit.client.0.smithi006.stdout:8/423: link d9/c19 d9/db/dc/de/c9f 0 2024-03-22T06:14:55.830 INFO:tasks.workunit.client.0.smithi006.stdout:6/296: rename d6/d1d/f28 to d6/d1d/d32/d4f/d56/f6e 0 2024-03-22T06:14:55.840 INFO:tasks.workunit.client.0.smithi006.stdout:8/424: creat d9/db/d12/d66/d6c/d80/fa0 x:0 0 0 2024-03-22T06:14:55.844 INFO:tasks.workunit.client.0.smithi006.stdout:1/292: getdents d8/d9 0 2024-03-22T06:14:55.864 INFO:tasks.workunit.client.0.smithi006.stdout:1/293: dwrite d8/d9/d13/f26 [0,4194304] 0 2024-03-22T06:14:55.873 INFO:tasks.workunit.client.0.smithi006.stdout:1/294: stat d8/d9/d3a/d18/l5e 0 2024-03-22T06:14:55.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.883+0000 7f4942bf1700 1 -- 172.21.15.6:0/3742371361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f493c0fe040 msgr2=0x7f493c0fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:55.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.883+0000 7f4942bf1700 1 --2- 172.21.15.6:0/3742371361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f493c0fe040 0x7f493c0fe4b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4928004940 tx=0x7f4928009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:55.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.883+0000 7f4942bf1700 1 -- 172.21.15.6:0/3742371361 shutdown_connections 2024-03-22T06:14:55.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.883+0000 7f4942bf1700 1 --2- 172.21.15.6:0/3742371361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f493c0fe040 0x7f493c0fe4b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:55.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.883+0000 7f4942bf1700 1 --2- 172.21.15.6:0/3742371361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c0fce40 0x7f493c0fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:55.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.883+0000 7f4942bf1700 1 -- 172.21.15.6:0/3742371361 >> 172.21.15.6:0/3742371361 conn(0x7f493c0f8570 msgr2=0x7f493c0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:55.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.883+0000 7f4942bf1700 1 -- 172.21.15.6:0/3742371361 shutdown_connections 2024-03-22T06:14:55.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.884+0000 7f4942bf1700 1 -- 172.21.15.6:0/3742371361 wait complete. 2024-03-22T06:14:55.885 INFO:tasks.workunit.client.0.smithi006.stdout:6/297: rename d6/d1d/d32/d4f/d56 to d6/d6f 0 2024-03-22T06:14:55.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.884+0000 7f4942bf1700 1 Processor -- start 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.884+0000 7f4942bf1700 1 -- start start 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.884+0000 7f4942bf1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c0fce40 0x7f493c10d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.884+0000 7f4942bf1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f493c0fe040 0x7f493c10dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.884+0000 7f4942bf1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c10e4e0 con 0x7f493c0fe040 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.884+0000 7f4942bf1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c10ed30 con 0x7f493c0fce40 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.885+0000 7f493bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f493c0fe040 0x7f493c10dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.885+0000 7f493bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f493c0fe040 0x7f493c10dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33120/0 (socket says 172.21.15.6:33120) 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.885+0000 7f493bfff700 1 -- 172.21.15.6:0/813641365 learned_addr learned my addr 172.21.15.6:0/813641365 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.885+0000 7f494098d700 1 --2- 172.21.15.6:0/813641365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c0fce40 0x7f493c10d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.885+0000 7f493bfff700 1 -- 172.21.15.6:0/813641365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c0fce40 msgr2=0x7f493c10d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.885+0000 7f493bfff700 1 --2- 172.21.15.6:0/813641365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c0fce40 0x7f493c10d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.885+0000 7f493bfff700 1 -- 172.21.15.6:0/813641365 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928005040 con 0x7f493c0fe040 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.885+0000 7f493bfff700 1 --2- 172.21.15.6:0/813641365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f493c0fe040 0x7f493c10dee0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4928009150 tx=0x7f4928009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:55.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.885+0000 7f49397fa700 1 -- 172.21.15.6:0/813641365 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f492800e040 con 0x7f493c0fe040 2024-03-22T06:14:55.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.885+0000 7f49397fa700 1 -- 172.21.15.6:0/813641365 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f492800bcc0 con 0x7f493c0fe040 2024-03-22T06:14:55.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.885+0000 7f4942bf1700 1 -- 172.21.15.6:0/813641365 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c10efb0 con 0x7f493c0fe040 2024-03-22T06:14:55.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.886+0000 7f49397fa700 1 -- 172.21.15.6:0/813641365 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f492801c630 con 0x7f493c0fe040 2024-03-22T06:14:55.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.886+0000 7f4942bf1700 1 -- 172.21.15.6:0/813641365 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c10f4f0 con 0x7f493c0fe040 2024-03-22T06:14:55.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.887+0000 7f4942bf1700 1 -- 172.21.15.6:0/813641365 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c05f5f0 con 0x7f493c0fe040 2024-03-22T06:14:55.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.888+0000 7f49397fa700 1 -- 172.21.15.6:0/813641365 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f4928013420 con 0x7f493c0fe040 2024-03-22T06:14:55.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.888+0000 7f49397fa700 1 --2- 172.21.15.6:0/813641365 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f492c06bb10 0x7f492c06dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:55.889 INFO:tasks.workunit.client.0.smithi006.stdout:6/298: mknod d6/d1d/d32/c70 0 2024-03-22T06:14:55.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.888+0000 7f49397fa700 1 -- 172.21.15.6:0/813641365 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f492808b4f0 con 0x7f493c0fe040 2024-03-22T06:14:55.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.888+0000 7f494098d700 1 --2- 172.21.15.6:0/813641365 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f492c06bb10 0x7f492c06dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:55.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.889+0000 7f494098d700 1 --2- 172.21.15.6:0/813641365 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f492c06bb10 0x7f492c06dfc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f493c0fdbb0 tx=0x7f4930007ea0 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:55.894 INFO:tasks.workunit.client.0.smithi006.stdout:6/299: truncate d6/f1e 4658941 0 2024-03-22T06:14:55.894 INFO:tasks.workunit.client.0.smithi006.stdout:8/425: rmdir d9/db/d12/d66/d6c/d80 39 2024-03-22T06:14:55.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:55.893+0000 7f49397fa700 1 -- 172.21.15.6:0/813641365 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f4928058940 con 0x7f493c0fe040 2024-03-22T06:14:55.896 INFO:tasks.workunit.client.0.smithi006.stdout:6/300: creat d6/d1d/f71 x:0 0 0 2024-03-22T06:14:55.900 INFO:tasks.workunit.client.0.smithi006.stdout:8/426: mknod d9/d14/ca1 0 2024-03-22T06:14:55.900 INFO:tasks.workunit.client.0.smithi006.stdout:8/427: readlink d9/db/d37/l77 0 2024-03-22T06:14:55.912 INFO:tasks.workunit.client.0.smithi006.stdout:6/301: dwrite d6/d1b/f48 [0,4194304] 0 2024-03-22T06:14:55.915 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:55 smithi117 ceph-mon[36514]: pgmap v11: 65 pgs: 65 active+clean; 350 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 23 MiB/s rd, 41 MiB/s wr, 369 op/s 2024-03-22T06:14:55.920 INFO:tasks.workunit.client.0.smithi006.stdout:6/302: creat d6/d1b/d2c/f72 x:0 0 0 2024-03-22T06:14:55.923 INFO:tasks.workunit.client.0.smithi006.stdout:6/303: mknod d6/d1d/d64/c73 0 2024-03-22T06:14:55.923 INFO:tasks.workunit.client.0.smithi006.stdout:6/304: chown d6/d1d/d5c 13675 1 2024-03-22T06:14:55.923 INFO:tasks.workunit.client.0.smithi006.stdout:6/305: truncate d6/d1b/d2c/d4a/f5a 276720 0 2024-03-22T06:14:55.923 INFO:tasks.workunit.client.0.smithi006.stdout:6/306: creat d6/d1b/d2c/f74 x:0 0 0 2024-03-22T06:14:55.928 INFO:tasks.workunit.client.0.smithi006.stdout:6/307: getdents d6/d1d/d5c 0 2024-03-22T06:14:55.928 INFO:tasks.workunit.client.0.smithi006.stdout:6/308: stat d6/d1d/d32/d4f/f65 0 2024-03-22T06:14:55.928 INFO:tasks.workunit.client.0.smithi006.stdout:6/309: chown d6/d1b/f2a 22204772 1 2024-03-22T06:14:55.944 INFO:tasks.workunit.client.0.smithi006.stdout:6/310: dwrite d6/d1b/f2a [0,4194304] 0 2024-03-22T06:14:55.944 INFO:tasks.workunit.client.0.smithi006.stdout:6/311: dread - d6/d1d/d32/d4f/f65 zero size 2024-03-22T06:14:55.947 INFO:tasks.workunit.client.0.smithi006.stdout:9/200: sync 2024-03-22T06:14:55.948 INFO:tasks.workunit.client.0.smithi006.stdout:0/227: sync 2024-03-22T06:14:55.948 INFO:tasks.workunit.client.0.smithi006.stdout:7/292: sync 2024-03-22T06:14:55.948 INFO:tasks.workunit.client.0.smithi006.stdout:0/228: creat db/d10/d35/f53 x:0 0 0 2024-03-22T06:14:55.972 INFO:tasks.workunit.client.0.smithi006.stdout:9/201: dwrite d8/d10/f12 [0,4194304] 0 2024-03-22T06:14:55.972 INFO:tasks.workunit.client.0.smithi006.stdout:7/293: dwrite d5/d1e/d29/d10/f3d [4194304,4194304] 0 2024-03-22T06:14:55.979 INFO:tasks.workunit.client.0.smithi006.stdout:7/294: creat d5/d1e/d29/f5d x:0 0 0 2024-03-22T06:14:55.979 INFO:tasks.workunit.client.0.smithi006.stdout:7/295: dread - d5/f49 zero size 2024-03-22T06:14:55.983 INFO:tasks.workunit.client.0.smithi006.stdout:7/296: mknod d5/d1e/d29/d1f/d2d/d41/c5e 0 2024-03-22T06:14:55.986 INFO:tasks.workunit.client.0.smithi006.stdout:9/202: dwrite d8/f19 [4194304,4194304] 0 2024-03-22T06:14:56.000 INFO:tasks.workunit.client.0.smithi006.stdout:9/203: dwrite d8/fa [0,4194304] 0 2024-03-22T06:14:56.017 INFO:tasks.workunit.client.0.smithi006.stdout:9/204: dwrite d8/ff [0,4194304] 0 2024-03-22T06:14:56.019 INFO:tasks.workunit.client.0.smithi006.stdout:9/205: mkdir d8/d10/d20/d45/d49 0 2024-03-22T06:14:56.019 INFO:tasks.workunit.client.0.smithi006.stdout:9/206: read - d8/d10/d28/f35 zero size 2024-03-22T06:14:56.026 INFO:tasks.workunit.client.0.smithi006.stdout:9/207: write f1 [3376734,84413] 0 2024-03-22T06:14:56.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:55 smithi006 ceph-mon[29301]: pgmap v11: 65 pgs: 65 active+clean; 350 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 23 MiB/s rd, 41 MiB/s wr, 369 op/s 2024-03-22T06:14:56.042 INFO:tasks.workunit.client.0.smithi006.stdout:9/208: dwrite d8/ff [4194304,4194304] 0 2024-03-22T06:14:56.043 INFO:tasks.workunit.client.0.smithi006.stdout:9/209: truncate d8/d10/f2e 259829 0 2024-03-22T06:14:56.043 INFO:tasks.workunit.client.0.smithi006.stdout:9/210: fdatasync d8/d10/f2e 0 2024-03-22T06:14:56.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.188+0000 7f4942bf1700 1 -- 172.21.15.6:0/813641365 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f493c10f820 con 0x7f492c06bb10 2024-03-22T06:14:56.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.190+0000 7f49397fa700 1 -- 172.21.15.6:0/813641365 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f493c10f820 con 0x7f492c06bb10 2024-03-22T06:14:56.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.192+0000 7f4942bf1700 1 -- 172.21.15.6:0/813641365 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f492c06bb10 msgr2=0x7f492c06dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:56.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.192+0000 7f4942bf1700 1 --2- 172.21.15.6:0/813641365 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f492c06bb10 0x7f492c06dfc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f493c0fdbb0 tx=0x7f4930007ea0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.192+0000 7f4942bf1700 1 -- 172.21.15.6:0/813641365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f493c0fe040 msgr2=0x7f493c10dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:56.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.192+0000 7f4942bf1700 1 --2- 172.21.15.6:0/813641365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f493c0fe040 0x7f493c10dee0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4928009150 tx=0x7f4928009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:56.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.192+0000 7f4942bf1700 1 -- 172.21.15.6:0/813641365 shutdown_connections 2024-03-22T06:14:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.192+0000 7f4942bf1700 1 --2- 172.21.15.6:0/813641365 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f492c06bb10 0x7f492c06dfc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.192+0000 7f4942bf1700 1 --2- 172.21.15.6:0/813641365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c0fce40 0x7f493c10d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.192+0000 7f4942bf1700 1 --2- 172.21.15.6:0/813641365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f493c0fe040 0x7f493c10dee0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.192+0000 7f4942bf1700 1 -- 172.21.15.6:0/813641365 >> 172.21.15.6:0/813641365 conn(0x7f493c0f8570 msgr2=0x7f493c0f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.192+0000 7f4942bf1700 1 -- 172.21.15.6:0/813641365 shutdown_connections 2024-03-22T06:14:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.192+0000 7f4942bf1700 1 -- 172.21.15.6:0/813641365 wait complete. 2024-03-22T06:14:56.214 INFO:tasks.workunit.client.0.smithi006.stdout:9/211: truncate d8/f19 7990248 0 2024-03-22T06:14:56.218 INFO:tasks.workunit.client.0.smithi006.stdout:9/212: rename d8/d10/d28/c2a to d8/d10/d20/c4a 0 2024-03-22T06:14:56.219 INFO:tasks.workunit.client.0.smithi006.stdout:9/213: link d8/d10/l15 d8/d10/d23/d3e/l4b 0 2024-03-22T06:14:56.222 INFO:tasks.workunit.client.0.smithi006.stdout:9/214: mknod d8/d10/d20/c4c 0 2024-03-22T06:14:56.222 INFO:tasks.workunit.client.0.smithi006.stdout:9/215: chown d8/d10/d28/c2b 13 1 2024-03-22T06:14:56.222 INFO:tasks.workunit.client.0.smithi006.stdout:9/216: write d8/d10/d28/d39/f46 [869436,72631] 0 2024-03-22T06:14:56.239 INFO:tasks.workunit.client.0.smithi006.stdout:9/217: dwrite d8/d10/f17 [0,4194304] 0 2024-03-22T06:14:56.241 INFO:tasks.workunit.client.0.smithi006.stdout:9/218: truncate d8/d10/d11/f16 268050 0 2024-03-22T06:14:56.252 INFO:tasks.workunit.client.0.smithi006.stdout:9/219: write d8/ff [6324820,102115] 0 2024-03-22T06:14:56.252 INFO:tasks.workunit.client.0.smithi006.stdout:9/220: chown d8/d10/d11/c36 119143 1 2024-03-22T06:14:56.253 INFO:tasks.workunit.client.0.smithi006.stdout:5/337: sync 2024-03-22T06:14:56.256 INFO:tasks.workunit.client.0.smithi006.stdout:5/338: mknod d0/db/d16/c64 0 2024-03-22T06:14:56.256 INFO:tasks.workunit.client.0.smithi006.stdout:5/339: write d0/db/dc/d14/f45 [1300178,32619] 0 2024-03-22T06:14:56.259 INFO:tasks.workunit.client.0.smithi006.stdout:9/221: dread d8/fd [0,4194304] 0 2024-03-22T06:14:56.261 INFO:tasks.workunit.client.0.smithi006.stdout:5/340: truncate d0/db/dc/d14/f5a 3288477 0 2024-03-22T06:14:56.261 INFO:tasks.workunit.client.0.smithi006.stdout:5/341: write d0/db/dc/d14/f1d [363628,22208] 0 2024-03-22T06:14:56.263 INFO:tasks.workunit.client.0.smithi006.stdout:5/342: rmdir d0/db/dc/d14 39 2024-03-22T06:14:56.265 INFO:tasks.workunit.client.0.smithi006.stdout:3/337: sync 2024-03-22T06:14:56.268 INFO:tasks.workunit.client.0.smithi006.stdout:9/222: write f3 [2674633,127651] 0 2024-03-22T06:14:56.268 INFO:tasks.workunit.client.0.smithi006.stdout:9/223: write d8/d10/d20/f2c [928714,107621] 0 2024-03-22T06:14:56.268 INFO:tasks.workunit.client.0.smithi006.stdout:9/224: fdatasync f1 0 2024-03-22T06:14:56.268 INFO:tasks.workunit.client.0.smithi006.stdout:9/225: write d8/d10/f2e [283518,106324] 0 2024-03-22T06:14:56.268 INFO:tasks.workunit.client.0.smithi006.stdout:3/338: dread d0/d8/d1c/f2c [0,4194304] 0 2024-03-22T06:14:56.268 INFO:tasks.workunit.client.0.smithi006.stdout:5/343: mkdir d0/db/d16/d1c/d34/d3c/d54/d65 0 2024-03-22T06:14:56.268 INFO:tasks.workunit.client.0.smithi006.stdout:5/344: chown d0/f5 501755699 1 2024-03-22T06:14:56.268 INFO:tasks.workunit.client.0.smithi006.stdout:5/345: getdents d0/db/d11/d52 0 2024-03-22T06:14:56.270 INFO:tasks.workunit.client.0.smithi006.stdout:9/226: mkdir d8/d10/d20/d45/d4d 0 2024-03-22T06:14:56.271 INFO:tasks.workunit.client.0.smithi006.stdout:5/346: dread d0/db/d16/d1c/f4c [0,4194304] 0 2024-03-22T06:14:56.272 INFO:tasks.workunit.client.0.smithi006.stdout:3/339: write d0/f3 [4706950,105750] 0 2024-03-22T06:14:56.274 INFO:tasks.workunit.client.0.smithi006.stdout:9/227: mknod d8/d10/d23/c4e 0 2024-03-22T06:14:56.275 INFO:tasks.workunit.client.0.smithi006.stdout:3/340: mkdir d0/d8/d6a 0 2024-03-22T06:14:56.278 INFO:tasks.workunit.client.0.smithi006.stdout:9/228: link d8/d10/d28/f3d d8/d10/d28/d48/f4f 0 2024-03-22T06:14:56.281 INFO:tasks.workunit.client.0.smithi006.stdout:9/229: truncate f1 2310713 0 2024-03-22T06:14:56.285 INFO:tasks.workunit.client.0.smithi006.stdout:9/230: rename d8/d10/d11/d2f to d8/d10/d28/d48/d50 0 2024-03-22T06:14:56.285 INFO:tasks.workunit.client.0.smithi006.stdout:9/231: readlink d8/d10/l24 0 2024-03-22T06:14:56.287 INFO:tasks.workunit.client.0.smithi006.stdout:9/232: symlink d8/d10/d28/l51 0 2024-03-22T06:14:56.291 INFO:tasks.workunit.client.0.smithi006.stdout:9/233: rmdir d8/d10/d28/d48/d50/d38 39 2024-03-22T06:14:56.291 INFO:tasks.workunit.client.0.smithi006.stdout:9/234: chown d8/l3f 0 1 2024-03-22T06:14:56.291 INFO:tasks.workunit.client.0.smithi006.stdout:9/235: creat d8/d10/d28/f52 x:0 0 0 2024-03-22T06:14:56.294 INFO:tasks.workunit.client.0.smithi006.stdout:3/341: dwrite d0/d8/f1a [4194304,4194304] 0 2024-03-22T06:14:56.306 INFO:tasks.workunit.client.0.smithi006.stdout:9/236: dwrite d8/fd [4194304,4194304] 0 2024-03-22T06:14:56.314 INFO:tasks.workunit.client.0.smithi006.stdout:3/342: creat d0/d8/d1c/d3e/d68/f6b x:0 0 0 2024-03-22T06:14:56.317 INFO:tasks.workunit.client.0.smithi006.stdout:5/347: write d0/db/dc/d14/f3f [3954413,1525] 0 2024-03-22T06:14:56.332 INFO:tasks.workunit.client.0.smithi006.stdout:9/237: creat d8/d10/d28/d48/d50/f53 x:0 0 0 2024-03-22T06:14:56.335 INFO:tasks.workunit.client.0.smithi006.stdout:3/343: symlink d0/d4/d31/l6c 0 2024-03-22T06:14:56.335 INFO:tasks.workunit.client.0.smithi006.stdout:3/344: truncate d0/d23/d33/f65 320098 0 2024-03-22T06:14:56.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.335+0000 7ff75efdc700 1 -- 172.21.15.6:0/327136769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7580fcf80 msgr2=0x7ff7580fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:56.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.335+0000 7ff75efdc700 1 --2- 172.21.15.6:0/327136769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7580fcf80 0x7ff7580fd3f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff744004940 tx=0x7ff744009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:56.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.336+0000 7ff75efdc700 1 -- 172.21.15.6:0/327136769 shutdown_connections 2024-03-22T06:14:56.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.336+0000 7ff75efdc700 1 --2- 172.21.15.6:0/327136769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7580fcf80 0x7ff7580fd3f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.336+0000 7ff75efdc700 1 --2- 172.21.15.6:0/327136769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7580fbcd0 0x7ff7580fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.336+0000 7ff75efdc700 1 -- 172.21.15.6:0/327136769 >> 172.21.15.6:0/327136769 conn(0x7ff7580f7430 msgr2=0x7ff7580f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:56.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.336+0000 7ff75efdc700 1 -- 172.21.15.6:0/327136769 shutdown_connections 2024-03-22T06:14:56.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.336+0000 7ff75efdc700 1 -- 172.21.15.6:0/327136769 wait complete. 2024-03-22T06:14:56.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.337+0000 7ff75efdc700 1 Processor -- start 2024-03-22T06:14:56.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.337+0000 7ff75efdc700 1 -- start start 2024-03-22T06:14:56.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.337+0000 7ff75efdc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7580fbcd0 0x7ff75810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:56.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.338+0000 7ff75efdc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7580fcf80 0x7ff75810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:56.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.338+0000 7ff75efdc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75810e4c0 con 0x7ff7580fbcd0 2024-03-22T06:14:56.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.338+0000 7ff75cd78700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7580fbcd0 0x7ff75810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:56.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.338+0000 7ff75efdc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75810ed10 con 0x7ff7580fcf80 2024-03-22T06:14:56.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.338+0000 7ff75cd78700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7580fbcd0 0x7ff75810d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33134/0 (socket says 172.21.15.6:33134) 2024-03-22T06:14:56.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.338+0000 7ff75cd78700 1 -- 172.21.15.6:0/4169638570 learned_addr learned my addr 172.21.15.6:0/4169638570 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:56.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.338+0000 7ff757fff700 1 --2- 172.21.15.6:0/4169638570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7580fcf80 0x7ff75810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:56.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.338+0000 7ff75cd78700 1 -- 172.21.15.6:0/4169638570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7580fcf80 msgr2=0x7ff75810dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:56.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.338+0000 7ff75cd78700 1 --2- 172.21.15.6:0/4169638570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7580fcf80 0x7ff75810dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.338+0000 7ff75cd78700 1 -- 172.21.15.6:0/4169638570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff744005040 con 0x7ff7580fbcd0 2024-03-22T06:14:56.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.339+0000 7ff75cd78700 1 --2- 172.21.15.6:0/4169638570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7580fbcd0 0x7ff75810d980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff74c002a80 tx=0x7ff74c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:56.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.340+0000 7ff7557fa700 1 -- 172.21.15.6:0/4169638570 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74c007cc0 con 0x7ff7580fbcd0 2024-03-22T06:14:56.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.340+0000 7ff7557fa700 1 -- 172.21.15.6:0/4169638570 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff74c00f040 con 0x7ff7580fbcd0 2024-03-22T06:14:56.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.340+0000 7ff7557fa700 1 -- 172.21.15.6:0/4169638570 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74c00da70 con 0x7ff7580fbcd0 2024-03-22T06:14:56.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.340+0000 7ff75efdc700 1 -- 172.21.15.6:0/4169638570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75810eff0 con 0x7ff7580fbcd0 2024-03-22T06:14:56.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.341+0000 7ff75efdc700 1 -- 172.21.15.6:0/4169638570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75810f470 con 0x7ff7580fbcd0 2024-03-22T06:14:56.342 INFO:tasks.workunit.client.0.smithi006.stdout:5/348: mkdir d0/db/d16/d1c/d34/d3c/d5e/d66 0 2024-03-22T06:14:56.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.343+0000 7ff75efdc700 1 -- 172.21.15.6:0/4169638570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff758101b10 con 0x7ff7580fbcd0 2024-03-22T06:14:56.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.343+0000 7ff7557fa700 1 -- 172.21.15.6:0/4169638570 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7ff74c00dbd0 con 0x7ff7580fbcd0 2024-03-22T06:14:56.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.344+0000 7ff7557fa700 1 --2- 172.21.15.6:0/4169638570 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7ff74806bbd0 0x7ff74806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:56.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.344+0000 7ff757fff700 1 --2- 172.21.15.6:0/4169638570 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7ff74806bbd0 0x7ff74806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:56.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.345+0000 7ff7557fa700 1 -- 172.21.15.6:0/4169638570 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7ff74c089f70 con 0x7ff7580fbcd0 2024-03-22T06:14:56.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.345+0000 7ff757fff700 1 --2- 172.21.15.6:0/4169638570 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7ff74806bbd0 0x7ff74806e080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff744009fd0 tx=0x7ff744009e30 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:56.346 INFO:tasks.workunit.client.0.smithi006.stdout:9/238: symlink d8/d10/l54 0 2024-03-22T06:14:56.347 INFO:tasks.workunit.client.0.smithi006.stdout:5/349: creat d0/db/d16/d43/f67 x:0 0 0 2024-03-22T06:14:56.350 INFO:tasks.workunit.client.0.smithi006.stdout:9/239: link d8/d10/d28/f25 d8/d10/d28/f55 0 2024-03-22T06:14:56.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.349+0000 7ff7557fa700 1 -- 172.21.15.6:0/4169638570 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7ff74c057340 con 0x7ff7580fbcd0 2024-03-22T06:14:56.351 INFO:tasks.workunit.client.0.smithi006.stdout:3/345: dwrite d0/d8/d1c/d3e/f60 [0,4194304] 0 2024-03-22T06:14:56.357 INFO:tasks.workunit.client.0.smithi006.stdout:9/240: creat d8/d10/f56 x:0 0 0 2024-03-22T06:14:56.364 INFO:tasks.workunit.client.0.smithi006.stdout:9/241: rename d8/f19 to d8/d10/d28/f57 0 2024-03-22T06:14:56.366 INFO:tasks.workunit.client.0.smithi006.stdout:9/242: dread d8/d10/f2e [0,4194304] 0 2024-03-22T06:14:56.367 INFO:tasks.workunit.client.0.smithi006.stdout:9/243: dread d8/d10/f2e [0,4194304] 0 2024-03-22T06:14:56.370 INFO:tasks.workunit.client.0.smithi006.stdout:9/244: creat d8/d10/d28/d48/d50/d38/f58 x:0 0 0 2024-03-22T06:14:56.371 INFO:tasks.workunit.client.0.smithi006.stdout:9/245: write d8/d10/f56 [886807,47770] 0 2024-03-22T06:14:56.372 INFO:tasks.workunit.client.0.smithi006.stdout:4/351: sync 2024-03-22T06:14:56.372 INFO:tasks.workunit.client.0.smithi006.stdout:2/321: sync 2024-03-22T06:14:56.372 INFO:tasks.workunit.client.0.smithi006.stdout:4/352: readlink d6/d1b/d20/l39 0 2024-03-22T06:14:56.374 INFO:tasks.workunit.client.0.smithi006.stdout:9/246: getdents d8/d10/d20/d45 0 2024-03-22T06:14:56.382 INFO:tasks.workunit.client.0.smithi006.stdout:4/353: rename f0 to d6/d1b/d20/d21/d2a/d43/f7d 0 2024-03-22T06:14:56.384 INFO:tasks.workunit.client.0.smithi006.stdout:9/247: unlink d8/d10/d28/d48/d50/f53 0 2024-03-22T06:14:56.391 INFO:tasks.workunit.client.0.smithi006.stdout:2/322: getdents d13/d20 0 2024-03-22T06:14:56.392 INFO:tasks.workunit.client.0.smithi006.stdout:7/297: sync 2024-03-22T06:14:56.392 INFO:tasks.workunit.client.0.smithi006.stdout:7/298: chown d5/d1e/d29/d1f/c53 255936620 1 2024-03-22T06:14:56.392 INFO:tasks.workunit.client.0.smithi006.stdout:7/299: write d5/d1e/d29/f4a [346761,36121] 0 2024-03-22T06:14:56.393 INFO:tasks.workunit.client.0.smithi006.stdout:6/312: sync 2024-03-22T06:14:56.393 INFO:tasks.workunit.client.0.smithi006.stdout:1/295: sync 2024-03-22T06:14:56.394 INFO:tasks.workunit.client.0.smithi006.stdout:8/428: sync 2024-03-22T06:14:56.394 INFO:tasks.workunit.client.0.smithi006.stdout:0/229: sync 2024-03-22T06:14:56.395 INFO:tasks.workunit.client.0.smithi006.stdout:9/248: creat d8/d10/d11/f59 x:0 0 0 2024-03-22T06:14:56.395 INFO:tasks.workunit.client.0.smithi006.stdout:9/249: fdatasync d8/f9 0 2024-03-22T06:14:56.396 INFO:tasks.workunit.client.0.smithi006.stdout:8/429: dread d9/db/dc/de/d3e/f56 [0,4194304] 0 2024-03-22T06:14:56.396 INFO:tasks.workunit.client.0.smithi006.stdout:8/430: creat d9/db/d12/d66/d6c/d71/fa2 x:0 0 0 2024-03-22T06:14:56.398 INFO:tasks.workunit.client.0.smithi006.stdout:6/313: mknod d6/d1d/d64/c75 0 2024-03-22T06:14:56.398 INFO:tasks.workunit.client.0.smithi006.stdout:7/300: dread f2 [0,4194304] 0 2024-03-22T06:14:56.400 INFO:tasks.workunit.client.0.smithi006.stdout:4/354: dwrite d6/d1b/d20/f33 [0,4194304] 0 2024-03-22T06:14:56.400 INFO:tasks.workunit.client.0.smithi006.stdout:4/355: stat d6/d1b/f54 0 2024-03-22T06:14:56.400 INFO:tasks.workunit.client.0.smithi006.stdout:4/356: readlink d6/l61 0 2024-03-22T06:14:56.406 INFO:tasks.workunit.client.0.smithi006.stdout:1/296: mknod d8/d33/c64 0 2024-03-22T06:14:56.407 INFO:tasks.workunit.client.0.smithi006.stdout:0/230: creat db/d3d/d4e/f54 x:0 0 0 2024-03-22T06:14:56.407 INFO:tasks.workunit.client.0.smithi006.stdout:0/231: fdatasync db/dc/d38/f3a 0 2024-03-22T06:14:56.408 INFO:tasks.workunit.client.0.smithi006.stdout:0/232: write db/d10/d2a/f51 [113129,128793] 0 2024-03-22T06:14:56.416 INFO:tasks.workunit.client.0.smithi006.stdout:2/323: dwrite d13/f6f [0,4194304] 0 2024-03-22T06:14:56.416 INFO:tasks.workunit.client.0.smithi006.stdout:2/324: rename d13 to d13/d69/d7a 22 2024-03-22T06:14:56.416 INFO:tasks.workunit.client.0.smithi006.stdout:2/325: creat d13/d20/d35/d65/f7b x:0 0 0 2024-03-22T06:14:56.417 INFO:tasks.workunit.client.0.smithi006.stdout:6/314: unlink d6/d1d/d32/d4f/c60 0 2024-03-22T06:14:56.418 INFO:tasks.workunit.client.0.smithi006.stdout:6/315: fsync d6/d1b/f48 0 2024-03-22T06:14:56.418 INFO:tasks.workunit.client.0.smithi006.stdout:6/316: chown d6/d24/l25 9413066 1 2024-03-22T06:14:56.418 INFO:tasks.workunit.client.0.smithi006.stdout:6/317: stat d6/d24/c57 0 2024-03-22T06:14:56.424 INFO:tasks.workunit.client.0.smithi006.stdout:5/350: truncate d0/db/dc/d14/f3f 2250908 0 2024-03-22T06:14:56.425 INFO:tasks.workunit.client.0.smithi006.stdout:4/357: truncate d6/d13/d3b/f51 94911 0 2024-03-22T06:14:56.425 INFO:tasks.workunit.client.0.smithi006.stdout:4/358: stat d6/d13/d3b/l40 0 2024-03-22T06:14:56.428 INFO:tasks.workunit.client.0.smithi006.stdout:8/431: getdents d9/d5e 0 2024-03-22T06:14:56.428 INFO:tasks.workunit.client.0.smithi006.stdout:8/432: fdatasync f7 0 2024-03-22T06:14:56.430 INFO:tasks.workunit.client.0.smithi006.stdout:9/250: dwrite d8/d10/f56 [0,4194304] 0 2024-03-22T06:14:56.430 INFO:tasks.workunit.client.0.smithi006.stdout:9/251: read f7 [7727,49263] 0 2024-03-22T06:14:56.431 INFO:tasks.workunit.client.0.smithi006.stdout:6/318: dread d6/d24/f31 [0,4194304] 0 2024-03-22T06:14:56.431 INFO:tasks.workunit.client.0.smithi006.stdout:6/319: write d6/d1d/d32/f41 [1712500,43063] 0 2024-03-22T06:14:56.431 INFO:tasks.workunit.client.0.smithi006.stdout:6/320: chown d6/d1d/d32/c70 3043683 1 2024-03-22T06:14:56.431 INFO:tasks.workunit.client.0.smithi006.stdout:7/301: dwrite d5/fe [0,4194304] 0 2024-03-22T06:14:56.439 INFO:tasks.workunit.client.0.smithi006.stdout:2/326: rename d13/d20/d26/l1d to d13/d20/d35/l7c 0 2024-03-22T06:14:56.439 INFO:tasks.workunit.client.0.smithi006.stdout:0/233: dwrite db/dc/d38/f4b [0,4194304] 0 2024-03-22T06:14:56.439 INFO:tasks.workunit.client.0.smithi006.stdout:1/297: dwrite d8/d9/f4b [0,4194304] 0 2024-03-22T06:14:56.441 INFO:tasks.workunit.client.0.smithi006.stdout:4/359: rmdir d6/d1b/d20/d21/d77 39 2024-03-22T06:14:56.444 INFO:tasks.workunit.client.0.smithi006.stdout:8/433: mkdir d9/d14/d47/d94/da3 0 2024-03-22T06:14:56.449 INFO:tasks.workunit.client.0.smithi006.stdout:6/321: dwrite d6/d24/d44/f46 [0,4194304] 0 2024-03-22T06:14:56.449 INFO:tasks.workunit.client.0.smithi006.stdout:6/322: chown d6/d24/f54 173952 1 2024-03-22T06:14:56.461 INFO:tasks.workunit.client.0.smithi006.stdout:9/252: rename d8/d10/d11/f13 to d8/d10/d23/d3e/f5a 0 2024-03-22T06:14:56.464 INFO:tasks.workunit.client.0.smithi006.stdout:4/360: dwrite d6/d1b/d20/d6b/f75 [0,4194304] 0 2024-03-22T06:14:56.466 INFO:tasks.workunit.client.0.smithi006.stdout:6/323: dwrite d6/f11 [0,4194304] 0 2024-03-22T06:14:56.468 INFO:tasks.workunit.client.0.smithi006.stdout:2/327: creat d13/d20/f7d x:0 0 0 2024-03-22T06:14:56.468 INFO:tasks.workunit.client.0.smithi006.stdout:2/328: dread d13/d34/d4d/f5c [0,4194304] 0 2024-03-22T06:14:56.479 INFO:tasks.workunit.client.0.smithi006.stdout:1/298: mknod d8/d9/d3a/c65 0 2024-03-22T06:14:56.479 INFO:tasks.workunit.client.0.smithi006.stdout:1/299: chown d8/d9/d13/d24/l4c 29 1 2024-03-22T06:14:56.481 INFO:tasks.workunit.client.0.smithi006.stdout:2/329: dwrite d13/d34/d4d/d4e/d75/d62/f66 [0,4194304] 0 2024-03-22T06:14:56.488 INFO:tasks.workunit.client.0.smithi006.stdout:5/351: write d0/db/dc/d14/f3f [2393330,29569] 0 2024-03-22T06:14:56.489 INFO:tasks.workunit.client.0.smithi006.stdout:2/330: dread d13/d20/f4b [0,4194304] 0 2024-03-22T06:14:56.490 INFO:tasks.workunit.client.0.smithi006.stdout:2/331: creat d13/d53/d6b/f7e x:0 0 0 2024-03-22T06:14:56.490 INFO:tasks.workunit.client.0.smithi006.stdout:2/332: readlink d13/l44 0 2024-03-22T06:14:56.491 INFO:tasks.workunit.client.0.smithi006.stdout:8/434: creat d9/db/d12/d66/d6c/d80/fa4 x:0 0 0 2024-03-22T06:14:56.498 INFO:tasks.workunit.client.0.smithi006.stdout:7/302: rmdir d5/d1e/d3f 39 2024-03-22T06:14:56.503 INFO:tasks.workunit.client.0.smithi006.stdout:9/253: rename d8/l3f to d8/d10/d20/d45/d49/l5b 0 2024-03-22T06:14:56.509 INFO:tasks.workunit.client.0.smithi006.stdout:4/361: creat d6/d1b/d20/d21/f7e x:0 0 0 2024-03-22T06:14:56.509 INFO:tasks.workunit.client.0.smithi006.stdout:4/362: chown d6/d1b/d20/d21/d77 7 1 2024-03-22T06:14:56.516 INFO:tasks.workunit.client.0.smithi006.stdout:7/303: dwrite d5/d1e/d29/d1f/f3e [0,4194304] 0 2024-03-22T06:14:56.517 INFO:tasks.workunit.client.0.smithi006.stdout:7/304: write d5/d1e/d29/f5d [379096,66732] 0 2024-03-22T06:14:56.517 INFO:tasks.workunit.client.0.smithi006.stdout:7/305: chown d5/d1e/d29/d10/l1b 132533527 1 2024-03-22T06:14:56.517 INFO:tasks.workunit.client.0.smithi006.stdout:7/306: readlink d5/d1e/l4d 0 2024-03-22T06:14:56.519 INFO:tasks.workunit.client.0.smithi006.stdout:2/333: symlink d13/d69/l7f 0 2024-03-22T06:14:56.520 INFO:tasks.workunit.client.0.smithi006.stdout:8/435: link d9/d14/d47/l4e d9/db/d12/d66/d6c/d71/d79/la5 0 2024-03-22T06:14:56.524 INFO:tasks.workunit.client.0.smithi006.stdout:6/324: rename d6/d1b/d2c/c50 to d6/c76 0 2024-03-22T06:14:56.524 INFO:tasks.workunit.client.0.smithi006.stdout:6/325: readlink d6/d24/l25 0 2024-03-22T06:14:56.524 INFO:tasks.workunit.client.0.smithi006.stdout:6/326: stat d6/d1b/l22 0 2024-03-22T06:14:56.526 INFO:tasks.workunit.client.0.smithi006.stdout:4/363: link d6/d1b/f2d d6/d1b/d20/d21/d2a/f7f 0 2024-03-22T06:14:56.528 INFO:tasks.workunit.client.0.smithi006.stdout:5/352: getdents d0/db/d16/d43 0 2024-03-22T06:14:56.528 INFO:tasks.workunit.client.0.smithi006.stdout:5/353: chown d0/db/l62 100 1 2024-03-22T06:14:56.528 INFO:tasks.workunit.client.0.smithi006.stdout:7/307: dread d5/d1e/d3f/f4b [0,4194304] 0 2024-03-22T06:14:56.534 INFO:tasks.workunit.client.0.smithi006.stdout:2/334: creat d13/d34/d6e/d72/f80 x:0 0 0 2024-03-22T06:14:56.536 INFO:tasks.workunit.client.0.smithi006.stdout:1/300: rename d8/d9/f1f to d8/d9/d3a/d40/d44/f66 0 2024-03-22T06:14:56.541 INFO:tasks.workunit.client.0.smithi006.stdout:5/354: mknod d0/db/d16/d1c/d34/d3c/d54/c68 0 2024-03-22T06:14:56.543 INFO:tasks.workunit.client.0.smithi006.stdout:7/308: truncate d5/d1e/d29/d10/f3d 1190980 0 2024-03-22T06:14:56.553 INFO:tasks.workunit.client.0.smithi006.stdout:7/309: dread d5/fe [0,4194304] 0 2024-03-22T06:14:56.611 INFO:tasks.workunit.client.0.smithi006.stdout:5/355: mkdir d0/db/d11/d52/d69 0 2024-03-22T06:14:56.611 INFO:tasks.workunit.client.0.smithi006.stdout:5/356: chown d0/db/d50/d58 3 1 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:5/357: chown d0/db/d16/d1c/d34 3565149 1 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:5/358: fsync d0/db/f29 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:8/436: rename d9/db/dc/de/l15 to d9/db/la6 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:8/437: getdents d9/db/d12/d66/d6c/d80/d85/d89 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:8/438: chown d9/d5e/f5f 137835715 1 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:8/439: readlink d9/db/dc/l38 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:7/310: write d5/fd [591879,913] 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:1/301: dwrite d8/d33/f4d [0,4194304] 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:5/359: creat d0/db/d11/d52/d69/f6a x:0 0 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:7/311: creat d5/f5f x:0 0 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:8/440: dwrite d9/db/dc/fd [0,4194304] 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:1/302: creat d8/d9/f67 x:0 0 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:8/441: dread d9/d14/f95 [0,4194304] 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:6/327: getdents d6/d1b/d2c 0 2024-03-22T06:14:56.612 INFO:tasks.workunit.client.0.smithi006.stdout:7/312: symlink d5/d1e/d29/d59/d5c/l60 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:8/442: unlink d9/db/d35/f55 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:8/443: readlink d9/d14/d44/l86 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:6/328: unlink d6/fd 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:7/313: mkdir d5/d1e/d29/d1f/d2d/d61 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:5/360: dread d0/db/dc/d14/f1d [0,4194304] 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:8/444: mkdir d9/db/d37/d4a/da7 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:8/445: creat d9/db/d12/d1e/fa8 x:0 0 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:8/446: write d9/db/d12/d1e/f39 [4947432,122875] 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:1/303: dread d8/d9/d10/f19 [0,4194304] 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:6/329: mkdir d6/d77 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:7/314: mknod d5/d1e/c62 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:5/361: creat d0/db/d16/d1c/d34/d4d/f6b x:0 0 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:5/362: truncate d0/db/d16/d1c/d34/d4d/f6b 242265 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:8/447: mkdir d9/db/d35/da9 0 2024-03-22T06:14:56.613 INFO:tasks.workunit.client.0.smithi006.stdout:6/330: mknod d6/d1d/d5c/c78 0 2024-03-22T06:14:56.614 INFO:tasks.workunit.client.0.smithi006.stdout:7/315: mkdir d5/d1e/d63 0 2024-03-22T06:14:56.614 INFO:tasks.workunit.client.0.smithi006.stdout:5/363: dread d0/db/f29 [0,4194304] 0 2024-03-22T06:14:56.614 INFO:tasks.workunit.client.0.smithi006.stdout:8/448: creat d9/db/d35/da9/faa x:0 0 0 2024-03-22T06:14:56.614 INFO:tasks.workunit.client.0.smithi006.stdout:8/449: write d9/db/f7b [1899664,22580] 0 2024-03-22T06:14:56.614 INFO:tasks.workunit.client.0.smithi006.stdout:6/331: dread d6/d1b/f2d [0,4194304] 0 2024-03-22T06:14:56.614 INFO:tasks.workunit.client.0.smithi006.stdout:6/332: fdatasync d6/d1d/f20 0 2024-03-22T06:14:56.614 INFO:tasks.workunit.client.0.smithi006.stdout:5/364: getdents d0/db/dc/d14 0 2024-03-22T06:14:56.614 INFO:tasks.workunit.client.0.smithi006.stdout:5/365: write d0/db/d16/f49 [681102,121752] 0 2024-03-22T06:14:56.614 INFO:tasks.workunit.client.0.smithi006.stdout:5/366: write d0/db/d11/f63 [811506,24898] 0 2024-03-22T06:14:56.615 INFO:tasks.workunit.client.0.smithi006.stdout:5/367: fsync d0/db/d16/f49 0 2024-03-22T06:14:56.615 INFO:tasks.workunit.client.0.smithi006.stdout:5/368: write d0/db/d50/d58/f5f [180572,28100] 0 2024-03-22T06:14:56.615 INFO:tasks.workunit.client.0.smithi006.stdout:6/333: mknod d6/d1d/d5c/c79 0 2024-03-22T06:14:56.615 INFO:tasks.workunit.client.0.smithi006.stdout:6/334: write d6/f40 [696903,94642] 0 2024-03-22T06:14:56.615 INFO:tasks.workunit.client.0.smithi006.stdout:5/369: dread d0/db/d50/d58/f5f [0,4194304] 0 2024-03-22T06:14:56.616 INFO:tasks.workunit.client.0.smithi006.stdout:6/335: symlink d6/d1d/l7a 0 2024-03-22T06:14:56.622 INFO:tasks.workunit.client.0.smithi006.stdout:5/370: getdents d0/db/d11/d52/d69 0 2024-03-22T06:14:56.622 INFO:tasks.workunit.client.0.smithi006.stdout:5/371: dread - d0/db/d11/f3d zero size 2024-03-22T06:14:56.626 INFO:tasks.workunit.client.0.smithi006.stdout:5/372: getdents d0/db/d11/d52 0 2024-03-22T06:14:56.629 INFO:tasks.workunit.client.0.smithi006.stdout:7/316: dwrite d5/d1e/f22 [0,4194304] 0 2024-03-22T06:14:56.629 INFO:tasks.workunit.client.0.smithi006.stdout:8/450: dwrite d9/db/f7b [0,4194304] 0 2024-03-22T06:14:56.629 INFO:tasks.workunit.client.0.smithi006.stdout:8/451: chown d9/db/f2e 69750 1 2024-03-22T06:14:56.629 INFO:tasks.workunit.client.0.smithi006.stdout:8/452: fdatasync d9/db/d35/f3c 0 2024-03-22T06:14:56.645 INFO:tasks.workunit.client.0.smithi006.stdout:1/304: dwrite d8/d9/d10/f19 [0,4194304] 0 2024-03-22T06:14:56.645 INFO:tasks.workunit.client.0.smithi006.stdout:1/305: rename d8/d9 to d8/d9/d3a/d18/d2a/d68 22 2024-03-22T06:14:56.649 INFO:tasks.workunit.client.0.smithi006.stdout:7/317: dwrite d5/d1e/d3f/f48 [0,4194304] 0 2024-03-22T06:14:56.649 INFO:tasks.workunit.client.0.smithi006.stdout:7/318: read - d5/d1e/d29/d10/f11 zero size 2024-03-22T06:14:56.649 INFO:tasks.workunit.client.0.smithi006.stdout:7/319: getdents d5/d1e/d29/d1f/d2d/d55 0 2024-03-22T06:14:56.660 INFO:tasks.workunit.client.0.smithi006.stdout:7/320: unlink d5/d1e/d29/f4a 0 2024-03-22T06:14:56.660 INFO:tasks.workunit.client.0.smithi006.stdout:7/321: fdatasync d5/f9 0 2024-03-22T06:14:56.660 INFO:tasks.workunit.client.0.smithi006.stdout:6/336: dwrite d6/d1d/f26 [0,4194304] 0 2024-03-22T06:14:56.668 INFO:tasks.workunit.client.0.smithi006.stdout:8/453: dwrite d9/db/d12/d66/d6c/d80/fa4 [0,4194304] 0 2024-03-22T06:14:56.668 INFO:tasks.workunit.client.0.smithi006.stdout:8/454: creat d9/db/d12/d1e/fab x:0 0 0 2024-03-22T06:14:56.670 INFO:tasks.workunit.client.0.smithi006.stdout:1/306: dwrite d8/d9/d13/d24/d32/d34/f36 [0,4194304] 0 2024-03-22T06:14:56.670 INFO:tasks.workunit.client.0.smithi006.stdout:1/307: stat d8/d9/d3a/f20 0 2024-03-22T06:14:56.670 INFO:tasks.workunit.client.0.smithi006.stdout:1/308: write d8/d9/f54 [282627,25185] 0 2024-03-22T06:14:56.671 INFO:tasks.workunit.client.0.smithi006.stdout:1/309: chown d8/d9/l17 3314033 1 2024-03-22T06:14:56.672 INFO:tasks.workunit.client.0.smithi006.stdout:8/455: dread d9/db/d37/f41 [0,4194304] 0 2024-03-22T06:14:56.673 INFO:tasks.workunit.client.0.smithi006.stdout:6/337: chown d6/d1b/d2c/f43 152 1 2024-03-22T06:14:56.685 INFO:tasks.workunit.client.0.smithi006.stdout:8/456: mknod d9/db/d12/d66/cac 0 2024-03-22T06:14:56.685 INFO:tasks.workunit.client.0.smithi006.stdout:8/457: write d9/db/d12/d1e/f2d [133209,24840] 0 2024-03-22T06:14:56.686 INFO:tasks.workunit.client.0.smithi006.stdout:6/338: write d6/d1d/d5c/f5e [86525,22464] 0 2024-03-22T06:14:56.688 INFO:tasks.workunit.client.0.smithi006.stdout:1/310: getdents d8/d9/d13 0 2024-03-22T06:14:56.689 INFO:tasks.workunit.client.0.smithi006.stdout:8/458: mkdir d9/db/d37/dad 0 2024-03-22T06:14:56.694 INFO:tasks.workunit.client.0.smithi006.stdout:8/459: symlink d9/db/dc/lae 0 2024-03-22T06:14:56.698 INFO:tasks.workunit.client.0.smithi006.stdout:8/460: symlink d9/db/d37/dad/laf 0 2024-03-22T06:14:56.703 INFO:tasks.workunit.client.0.smithi006.stdout:8/461: creat d9/db/d12/d66/d6c/d80/d85/d89/fb0 x:0 0 0 2024-03-22T06:14:56.706 INFO:tasks.workunit.client.0.smithi006.stdout:8/462: creat d9/db/d12/d66/d6c/d92/fb1 x:0 0 0 2024-03-22T06:14:56.706 INFO:tasks.workunit.client.0.smithi006.stdout:8/463: chown d9/db/dc/de 1 1 2024-03-22T06:14:56.706 INFO:tasks.workunit.client.0.smithi006.stdout:8/464: truncate d9/db/d37/d4a/f5b 358223 0 2024-03-22T06:14:56.707 INFO:tasks.workunit.client.0.smithi006.stdout:8/465: truncate d9/db/d37/d4a/f96 77837 0 2024-03-22T06:14:56.709 INFO:tasks.workunit.client.0.smithi006.stdout:8/466: unlink d9/d14/d47/l58 0 2024-03-22T06:14:56.709 INFO:tasks.workunit.client.0.smithi006.stdout:8/467: fdatasync d9/db/d35/f3c 0 2024-03-22T06:14:56.715 INFO:tasks.workunit.client.0.smithi006.stdout:1/311: dwrite d8/d9/f54 [0,4194304] 0 2024-03-22T06:14:56.715 INFO:tasks.workunit.client.0.smithi006.stdout:1/312: chown d8/d9/d3a/l5c 4582080 1 2024-03-22T06:14:56.719 INFO:tasks.workunit.client.0.smithi006.stdout:1/313: mknod d8/d9/d13/d39/d60/c69 0 2024-03-22T06:14:56.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.724+0000 7ff75efdc700 1 -- 172.21.15.6:0/4169638570 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff75810f6e0 con 0x7ff74806bbd0 2024-03-22T06:14:56.725 INFO:tasks.workunit.client.0.smithi006.stdout:8/468: dwrite d9/d5e/f69 [0,4194304] 0 2024-03-22T06:14:56.726 INFO:tasks.workunit.client.0.smithi006.stdout:6/339: rename d6/c63 to d6/d1b/c7b 0 2024-03-22T06:14:56.731 INFO:tasks.workunit.client.0.smithi006.stdout:3/346: sync 2024-03-22T06:14:56.731 INFO:tasks.workunit.client.0.smithi006.stdout:3/347: truncate d0/fa 4664280 0 2024-03-22T06:14:56.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.739+0000 7ff7557fa700 1 -- 172.21.15.6:0/4169638570 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7ff75810f6e0 con 0x7ff74806bbd0 2024-03-22T06:14:56.739 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:14:56.739 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (6m) 2m ago 7m 23.6M - 0.25.0 c8568f914cd2 bf3cfbba5c2f 2024-03-22T06:14:56.739 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (8m) 2m ago 8m 6656k - 17.2.7-726-g9c4e0c34 837124da804f a49ab95d42e6 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (7m) 9s ago 7m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0a36af1f8ed0 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 running (6m) 2m ago 7m 84.9M - 9.4.7 954c08fa6188 8b365bc3fd8b 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (2m) 2m ago 2m 14.6M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (2m) 2m ago 2m 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (2m) 9s ago 2m 13.0M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (2m) 9s ago 2m 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:9283 running (9m) 2m ago 9m 479M - 17.2.7-726-g9c4e0c34 837124da804f 2cea0b1a5c60 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283 running (37s) 9s ago 7m 489M - 19.0.0-2299-gea01217f aa461e7c5e7f ff1047688325 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (9m) 2m ago 9m 47.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f b2fe36c6b005 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (7m) 9s ago 7m 39.9M 2048M 17.2.7-726-g9c4e0c34 837124da804f 0559848da9ab 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (8m) 2m ago 8m 16.2M - 1.5.0 0da6a335fe13 0ef65efb379b 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (7m) 9s ago 7m 14.9M - 1.5.0 0da6a335fe13 58a93ac942d3 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (6m) 2m ago 6m 48.5M 4289M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:14:56.740 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (5m) 2m ago 5m 45.1M 4289M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:14:56.741 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (5m) 2m ago 5m 47.2M 4289M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:14:56.741 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (4m) 9s ago 4m 125M 2581M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:14:56.741 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (4m) 9s ago 4m 108M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:14:56.741 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (4m) 9s ago 4m 120M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:14:56.741 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (6m) 2m ago 7m 56.6M - 2.43.0 a07b618ecd1d 59d1a24d5af8 2024-03-22T06:14:56.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.741+0000 7ff75efdc700 1 -- 172.21.15.6:0/4169638570 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7ff74806bbd0 msgr2=0x7ff74806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:56.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.741+0000 7ff75efdc700 1 --2- 172.21.15.6:0/4169638570 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7ff74806bbd0 0x7ff74806e080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff744009fd0 tx=0x7ff744009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:56.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.741+0000 7ff75efdc700 1 -- 172.21.15.6:0/4169638570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7580fbcd0 msgr2=0x7ff75810d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:56.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.741+0000 7ff75efdc700 1 --2- 172.21.15.6:0/4169638570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7580fbcd0 0x7ff75810d980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff74c002a80 tx=0x7ff74c0046b0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.741+0000 7ff75efdc700 1 -- 172.21.15.6:0/4169638570 shutdown_connections 2024-03-22T06:14:56.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.741+0000 7ff75efdc700 1 --2- 172.21.15.6:0/4169638570 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7ff74806bbd0 0x7ff74806e080 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.741+0000 7ff75efdc700 1 --2- 172.21.15.6:0/4169638570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7580fbcd0 0x7ff75810d980 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.741+0000 7ff75efdc700 1 --2- 172.21.15.6:0/4169638570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7580fcf80 0x7ff75810dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.742+0000 7ff75efdc700 1 -- 172.21.15.6:0/4169638570 >> 172.21.15.6:0/4169638570 conn(0x7ff7580f7430 msgr2=0x7ff7580f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:56.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.742+0000 7ff75efdc700 1 -- 172.21.15.6:0/4169638570 shutdown_connections 2024-03-22T06:14:56.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.742+0000 7ff75efdc700 1 -- 172.21.15.6:0/4169638570 wait complete. 2024-03-22T06:14:56.743 INFO:tasks.workunit.client.0.smithi006.stdout:1/314: dwrite d8/d9/d3a/d18/f56 [0,4194304] 0 2024-03-22T06:14:56.746 INFO:tasks.workunit.client.0.smithi006.stdout:1/315: dread d8/d33/f5b [0,4194304] 0 2024-03-22T06:14:56.749 INFO:tasks.workunit.client.0.smithi006.stdout:3/348: dwrite d0/d23/d33/f4d [0,4194304] 0 2024-03-22T06:14:56.749 INFO:tasks.workunit.client.0.smithi006.stdout:3/349: write d0/f56 [1038041,79163] 0 2024-03-22T06:14:56.749 INFO:tasks.workunit.client.0.smithi006.stdout:3/350: chown d0/d8/d9/l11 129284920 1 2024-03-22T06:14:56.751 INFO:tasks.workunit.client.0.smithi006.stdout:8/469: dwrite d9/db/d37/d4a/f96 [0,4194304] 0 2024-03-22T06:14:56.760 INFO:tasks.workunit.client.0.smithi006.stdout:1/316: mknod d8/d9/d13/d39/c6a 0 2024-03-22T06:14:56.765 INFO:tasks.workunit.client.0.smithi006.stdout:8/470: dwrite d9/db/d12/f3d [0,4194304] 0 2024-03-22T06:14:56.769 INFO:tasks.workunit.client.0.smithi006.stdout:1/317: rmdir d8/d9/d13/d24 39 2024-03-22T06:14:56.772 INFO:tasks.workunit.client.0.smithi006.stdout:8/471: creat d9/d14/d47/d94/da3/fb2 x:0 0 0 2024-03-22T06:14:56.772 INFO:tasks.workunit.client.0.smithi006.stdout:8/472: dread - d9/f21 zero size 2024-03-22T06:14:56.773 INFO:tasks.workunit.client.0.smithi006.stdout:1/318: creat d8/d9/d3a/f6b x:0 0 0 2024-03-22T06:14:56.775 INFO:tasks.workunit.client.0.smithi006.stdout:6/340: mkdir d6/d7c 0 2024-03-22T06:14:56.776 INFO:tasks.workunit.client.0.smithi006.stdout:8/473: mkdir d9/db/d37/d76/db3 0 2024-03-22T06:14:56.776 INFO:tasks.workunit.client.0.smithi006.stdout:8/474: write d9/d14/d47/d94/da3/fb2 [686449,119892] 0 2024-03-22T06:14:56.778 INFO:tasks.workunit.client.0.smithi006.stdout:1/319: rename d8/d9/d13/d39/l42 to d8/d33/l6c 0 2024-03-22T06:14:56.783 INFO:tasks.workunit.client.0.smithi006.stdout:8/475: rename d9/db/f20 to d9/fb4 0 2024-03-22T06:14:56.783 INFO:tasks.workunit.client.0.smithi006.stdout:8/476: chown d9 77 1 2024-03-22T06:14:56.784 INFO:tasks.workunit.client.0.smithi006.stdout:8/477: dread - d9/d14/d44/f97 zero size 2024-03-22T06:14:56.784 INFO:tasks.workunit.client.0.smithi006.stdout:8/478: chown d9/db/d12/d66/cac 0 1 2024-03-22T06:14:56.787 INFO:tasks.workunit.client.0.smithi006.stdout:6/341: link d6/d24/d44/f5f d6/d1b/d2c/f7d 0 2024-03-22T06:14:56.789 INFO:tasks.workunit.client.0.smithi006.stdout:8/479: creat d9/fb5 x:0 0 0 2024-03-22T06:14:56.789 INFO:tasks.workunit.client.0.smithi006.stdout:8/480: write d9/db/d12/d1e/f2d [413625,37697] 0 2024-03-22T06:14:56.795 INFO:tasks.workunit.client.0.smithi006.stdout:1/320: dwrite d8/d9/d13/d24/d32/f41 [0,4194304] 0 2024-03-22T06:14:56.795 INFO:tasks.workunit.client.0.smithi006.stdout:1/321: chown d8/d9/d13/d24 6188508 1 2024-03-22T06:14:56.801 INFO:tasks.workunit.client.0.smithi006.stdout:1/322: creat d8/d9/d13/f6d x:0 0 0 2024-03-22T06:14:56.802 INFO:tasks.workunit.client.0.smithi006.stdout:1/323: chown d8 425596162 1 2024-03-22T06:14:56.806 INFO:tasks.workunit.client.0.smithi006.stdout:8/481: dwrite d9/db/d12/d66/d6c/d80/d85/d89/fb0 [0,4194304] 0 2024-03-22T06:14:56.806 INFO:tasks.workunit.client.0.smithi006.stdout:8/482: creat d9/db/d12/fb6 x:0 0 0 2024-03-22T06:14:56.814 INFO:tasks.workunit.client.0.smithi006.stdout:8/483: dread d9/db/d35/f6f [0,4194304] 0 2024-03-22T06:14:56.821 INFO:tasks.workunit.client.0.smithi006.stdout:8/484: write d9/db/dc/de/d3e/f73 [984404,40419] 0 2024-03-22T06:14:56.821 INFO:tasks.workunit.client.0.smithi006.stdout:8/485: fsync d9/db/d12/f33 0 2024-03-22T06:14:56.821 INFO:tasks.workunit.client.0.smithi006.stdout:8/486: chown d9/db/d12/d66/f9b 36 1 2024-03-22T06:14:56.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.865+0000 7fb13a710700 1 -- 172.21.15.6:0/3175514527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1340fe040 msgr2=0x7fb1340fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:56.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.865+0000 7fb13a710700 1 --2- 172.21.15.6:0/3175514527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1340fe040 0x7fb1340fe4b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb128004940 tx=0x7fb128009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:56.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.865+0000 7fb13a710700 1 -- 172.21.15.6:0/3175514527 shutdown_connections 2024-03-22T06:14:56.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.865+0000 7fb13a710700 1 --2- 172.21.15.6:0/3175514527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1340fe040 0x7fb1340fe4b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.865+0000 7fb13a710700 1 --2- 172.21.15.6:0/3175514527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1340fce40 0x7fb1340fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.865+0000 7fb13a710700 1 -- 172.21.15.6:0/3175514527 >> 172.21.15.6:0/3175514527 conn(0x7fb1340f8570 msgr2=0x7fb1340fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:56.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.865+0000 7fb13a710700 1 -- 172.21.15.6:0/3175514527 shutdown_connections 2024-03-22T06:14:56.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.865+0000 7fb13a710700 1 -- 172.21.15.6:0/3175514527 wait complete. 2024-03-22T06:14:56.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.866+0000 7fb13a710700 1 Processor -- start 2024-03-22T06:14:56.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.866+0000 7fb13a710700 1 -- start start 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.866+0000 7fb13a710700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1340fce40 0x7fb13410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.866+0000 7fb13a710700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1340fe040 0x7fb13410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.866+0000 7fb13a710700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13410e580 con 0x7fb1340fe040 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.866+0000 7fb13a710700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13410edd0 con 0x7fb1340fce40 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.866+0000 7fb1337fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1340fe040 0x7fb13410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.866+0000 7fb1337fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1340fe040 0x7fb13410df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33150/0 (socket says 172.21.15.6:33150) 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.866+0000 7fb1337fe700 1 -- 172.21.15.6:0/42104182 learned_addr learned my addr 172.21.15.6:0/42104182 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.867+0000 7fb133fff700 1 --2- 172.21.15.6:0/42104182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1340fce40 0x7fb13410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.867+0000 7fb1337fe700 1 -- 172.21.15.6:0/42104182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1340fce40 msgr2=0x7fb13410da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.867+0000 7fb1337fe700 1 --2- 172.21.15.6:0/42104182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1340fce40 0x7fb13410da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.867+0000 7fb1337fe700 1 -- 172.21.15.6:0/42104182 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb128005040 con 0x7fb1340fe040 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.867+0000 7fb1337fe700 1 --2- 172.21.15.6:0/42104182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1340fe040 0x7fb13410df80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb12800b3b0 tx=0x7fb128009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:56.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.867+0000 7fb130ff9700 1 -- 172.21.15.6:0/42104182 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb12800e040 con 0x7fb1340fe040 2024-03-22T06:14:56.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.867+0000 7fb130ff9700 1 -- 172.21.15.6:0/42104182 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb12800bcc0 con 0x7fb1340fe040 2024-03-22T06:14:56.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.867+0000 7fb13a710700 1 -- 172.21.15.6:0/42104182 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13410f050 con 0x7fb1340fe040 2024-03-22T06:14:56.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.867+0000 7fb130ff9700 1 -- 172.21.15.6:0/42104182 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb12801c630 con 0x7fb1340fe040 2024-03-22T06:14:56.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.867+0000 7fb13a710700 1 -- 172.21.15.6:0/42104182 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13410f590 con 0x7fb1340fe040 2024-03-22T06:14:56.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.868+0000 7fb13a710700 1 -- 172.21.15.6:0/42104182 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13405f5f0 con 0x7fb1340fe040 2024-03-22T06:14:56.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.868+0000 7fb130ff9700 1 -- 172.21.15.6:0/42104182 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7fb128013980 con 0x7fb1340fe040 2024-03-22T06:14:56.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.869+0000 7fb130ff9700 1 --2- 172.21.15.6:0/42104182 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb11c06bb10 0x7fb11c06dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:56.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.869+0000 7fb130ff9700 1 -- 172.21.15.6:0/42104182 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fb12808b520 con 0x7fb1340fe040 2024-03-22T06:14:56.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.869+0000 7fb133fff700 1 --2- 172.21.15.6:0/42104182 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb11c06bb10 0x7fb11c06dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:56.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.870+0000 7fb133fff700 1 --2- 172.21.15.6:0/42104182 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb11c06bb10 0x7fb11c06dfc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb124007fd0 tx=0x7fb124009040 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:56.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:56.874+0000 7fb130ff9700 1 -- 172.21.15.6:0/42104182 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7fb1280588f0 con 0x7fb1340fe040 2024-03-22T06:14:56.912 INFO:tasks.workunit.client.0.smithi006.stdout:8/487: creat d9/db/d37/dad/fb7 x:0 0 0 2024-03-22T06:14:56.915 INFO:tasks.workunit.client.0.smithi006.stdout:8/488: mkdir d9/db/d12/d66/d6c/d80/d85/d89/db8 0 2024-03-22T06:14:56.915 INFO:tasks.workunit.client.0.smithi006.stdout:8/489: creat d9/db/d12/d66/d6c/d92/fb9 x:0 0 0 2024-03-22T06:14:56.915 INFO:tasks.workunit.client.0.smithi006.stdout:8/490: creat d9/db/d37/d4a/d4d/fba x:0 0 0 2024-03-22T06:14:56.915 INFO:tasks.workunit.client.0.smithi006.stdout:8/491: write d9/db/d12/fb6 [351156,128950] 0 2024-03-22T06:14:56.919 INFO:tasks.workunit.client.0.smithi006.stdout:8/492: truncate d9/db/dc/de/d3e/f56 435423 0 2024-03-22T06:14:56.922 INFO:tasks.workunit.client.0.smithi006.stdout:8/493: truncate d9/db/f7b 1249638 0 2024-03-22T06:14:56.935 INFO:tasks.workunit.client.0.smithi006.stdout:8/494: dwrite d9/db/d12/d1e/f39 [0,4194304] 0 2024-03-22T06:14:56.939 INFO:tasks.workunit.client.0.smithi006.stdout:8/495: creat d9/db/d35/fbb x:0 0 0 2024-03-22T06:14:56.947 INFO:tasks.workunit.client.0.smithi006.stdout:8/496: dread d9/db/dc/fd [0,4194304] 0 2024-03-22T06:14:56.951 INFO:tasks.workunit.client.0.smithi006.stdout:0/234: sync 2024-03-22T06:14:56.951 INFO:tasks.workunit.client.0.smithi006.stdout:0/235: fsync db/d10/f23 0 2024-03-22T06:14:56.972 INFO:tasks.workunit.client.0.smithi006.stdout:0/236: dwrite db/dc/f21 [0,4194304] 0 2024-03-22T06:14:56.972 INFO:tasks.workunit.client.0.smithi006.stdout:8/497: dwrite d9/db/f2e [4194304,4194304] 0 2024-03-22T06:14:56.972 INFO:tasks.workunit.client.0.smithi006.stdout:8/498: read - d9/d14/d47/f64 zero size 2024-03-22T06:14:56.973 INFO:tasks.workunit.client.0.smithi006.stdout:8/499: chown d9/db/dc/de/l6a 342345 1 2024-03-22T06:14:56.976 INFO:tasks.workunit.client.0.smithi006.stdout:0/237: getdents db/d10/d33 0 2024-03-22T06:14:56.978 INFO:tasks.workunit.client.0.smithi006.stdout:0/238: dread db/d10/d2a/f51 [0,4194304] 0 2024-03-22T06:14:56.978 INFO:tasks.workunit.client.0.smithi006.stdout:0/239: chown db/d10/d35/f43 149102 1 2024-03-22T06:14:56.984 INFO:tasks.workunit.client.0.smithi006.stdout:0/240: rename db/c2d to db/dc/d38/c55 0 2024-03-22T06:14:56.989 INFO:tasks.workunit.client.0.smithi006.stdout:0/241: rename db/l12 to db/dc/d38/l56 0 2024-03-22T06:14:56.989 INFO:tasks.workunit.client.0.smithi006.stdout:0/242: stat db/f45 0 2024-03-22T06:14:57.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:56 smithi117 ceph-mon[36514]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:57.021 INFO:tasks.workunit.client.0.smithi006.stdout:4/364: sync 2024-03-22T06:14:57.021 INFO:tasks.workunit.client.0.smithi006.stdout:9/254: sync 2024-03-22T06:14:57.028 INFO:tasks.workunit.client.0.smithi006.stdout:9/255: dread d8/d10/f17 [0,4194304] 0 2024-03-22T06:14:57.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:56 smithi006 ceph-mon[29301]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:57.036 INFO:tasks.workunit.client.0.smithi006.stdout:4/365: dwrite f1 [4194304,4194304] 0 2024-03-22T06:14:57.122 INFO:tasks.workunit.client.0.smithi006.stdout:9/256: mknod d8/d10/d47/c5c 0 2024-03-22T06:14:57.122 INFO:tasks.workunit.client.0.smithi006.stdout:9/257: readlink d8/d10/d23/d3e/l4b 0 2024-03-22T06:14:57.122 INFO:tasks.workunit.client.0.smithi006.stdout:9/258: fdatasync d8/d10/d23/d3e/f5a 0 2024-03-22T06:14:57.122 INFO:tasks.workunit.client.0.smithi006.stdout:9/259: read f4 [3397702,101210] 0 2024-03-22T06:14:57.126 INFO:tasks.workunit.client.0.smithi006.stdout:4/366: symlink d6/d1b/d20/d21/d2a/d43/d4b/l80 0 2024-03-22T06:14:57.126 INFO:tasks.workunit.client.0.smithi006.stdout:4/367: truncate d6/d1b/f54 2288626 0 2024-03-22T06:14:57.139 INFO:tasks.workunit.client.0.smithi006.stdout:4/368: unlink d6/l18 0 2024-03-22T06:14:57.143 INFO:tasks.workunit.client.0.smithi006.stdout:9/260: rmdir d8/d10/d28/d48/d50 39 2024-03-22T06:14:57.150 INFO:tasks.workunit.client.0.smithi006.stdout:9/261: dread d8/d10/d28/f1d [0,4194304] 0 2024-03-22T06:14:57.153 INFO:tasks.workunit.client.0.smithi006.stdout:9/262: link d8/d10/d23/d3e/f5a d8/d1f/f5d 0 2024-03-22T06:14:57.156 INFO:tasks.workunit.client.0.smithi006.stdout:9/263: creat d8/d10/d28/d48/d50/f5e x:0 0 0 2024-03-22T06:14:57.164 INFO:tasks.workunit.client.0.smithi006.stdout:9/264: dread d8/d10/d28/f3d [0,4194304] 0 2024-03-22T06:14:57.165 INFO:tasks.workunit.client.0.smithi006.stdout:5/373: sync 2024-03-22T06:14:57.165 INFO:tasks.workunit.client.0.smithi006.stdout:2/335: sync 2024-03-22T06:14:57.165 INFO:tasks.workunit.client.0.smithi006.stdout:5/374: creat d0/db/f6c x:0 0 0 2024-03-22T06:14:57.169 INFO:tasks.workunit.client.0.smithi006.stdout:2/336: symlink d13/d69/l81 0 2024-03-22T06:14:57.169 INFO:tasks.workunit.client.0.smithi006.stdout:2/337: truncate d13/d20/d35/d65/f77 481314 0 2024-03-22T06:14:57.169 INFO:tasks.workunit.client.0.smithi006.stdout:2/338: stat l10 0 2024-03-22T06:14:57.171 INFO:tasks.workunit.client.0.smithi006.stdout:5/375: symlink d0/db/d11/l6d 0 2024-03-22T06:14:57.171 INFO:tasks.workunit.client.0.smithi006.stdout:5/376: readlink d0/db/d16/d1c/d34/l3a 0 2024-03-22T06:14:57.171 INFO:tasks.workunit.client.0.smithi006.stdout:5/377: stat d0/db/d16/l17 0 2024-03-22T06:14:57.173 INFO:tasks.workunit.client.0.smithi006.stdout:5/378: creat d0/db/d16/d1c/d34/d35/f6e x:0 0 0 2024-03-22T06:14:57.174 INFO:tasks.workunit.client.0.smithi006.stdout:2/339: mkdir d13/d53/d29/d82 0 2024-03-22T06:14:57.176 INFO:tasks.workunit.client.0.smithi006.stdout:5/379: truncate d0/db/d16/d1c/d34/d4d/f6b 122324 0 2024-03-22T06:14:57.178 INFO:tasks.workunit.client.0.smithi006.stdout:2/340: creat d13/d34/d4d/d4e/f83 x:0 0 0 2024-03-22T06:14:57.180 INFO:tasks.workunit.client.0.smithi006.stdout:5/380: rename d0/db/dc/d14/f5a to d0/db/d50/d58/f6f 0 2024-03-22T06:14:57.181 INFO:tasks.workunit.client.0.smithi006.stdout:5/381: readlink d0/db/d11/l6d 0 2024-03-22T06:14:57.181 INFO:tasks.workunit.client.0.smithi006.stdout:5/382: write d0/db/d16/f30 [9131861,84223] 0 2024-03-22T06:14:57.181 INFO:tasks.workunit.client.0.smithi006.stdout:5/383: write d0/db/dc/d14/f45 [1867679,47175] 0 2024-03-22T06:14:57.182 INFO:tasks.workunit.client.0.smithi006.stdout:9/265: dwrite f4 [0,4194304] 0 2024-03-22T06:14:57.182 INFO:tasks.workunit.client.0.smithi006.stdout:2/341: link d13/d34/d6e/d72/f80 d13/d34/d6e/f84 0 2024-03-22T06:14:57.184 INFO:tasks.workunit.client.0.smithi006.stdout:2/342: dread d13/d34/f60 [0,4194304] 0 2024-03-22T06:14:57.187 INFO:tasks.workunit.client.0.smithi006.stdout:5/384: getdents d0/db/d16 0 2024-03-22T06:14:57.190 INFO:tasks.workunit.client.0.smithi006.stdout:2/343: mknod d13/d34/c85 0 2024-03-22T06:14:57.192 INFO:tasks.workunit.client.0.smithi006.stdout:2/344: dread d13/d20/d26/d19/f3e [0,4194304] 0 2024-03-22T06:14:57.192 INFO:tasks.workunit.client.0.smithi006.stdout:2/345: readlink d13/d34/d4d/l54 0 2024-03-22T06:14:57.197 INFO:tasks.workunit.client.0.smithi006.stdout:5/385: mkdir d0/db/d16/d70 0 2024-03-22T06:14:57.199 INFO:tasks.workunit.client.0.smithi006.stdout:9/266: dwrite f6 [0,4194304] 0 2024-03-22T06:14:57.200 INFO:tasks.workunit.client.0.smithi006.stdout:5/386: chown d0/db/d16/d1c/d34/d3c/d5e/d66 610942821 1 2024-03-22T06:14:57.200 INFO:tasks.workunit.client.0.smithi006.stdout:5/387: stat d0/db/d16/d1c/d34/c51 0 2024-03-22T06:14:57.201 INFO:tasks.workunit.client.0.smithi006.stdout:2/346: link d13/d53/d29/f47 d13/d20/d35/f86 0 2024-03-22T06:14:57.203 INFO:tasks.workunit.client.0.smithi006.stdout:9/267: symlink d8/d10/d20/d45/d49/l5f 0 2024-03-22T06:14:57.210 INFO:tasks.workunit.client.0.smithi006.stdout:2/347: mknod d13/d34/d6e/c87 0 2024-03-22T06:14:57.211 INFO:tasks.workunit.client.0.smithi006.stdout:5/388: mknod d0/db/d16/d43/c71 0 2024-03-22T06:14:57.214 INFO:tasks.workunit.client.0.smithi006.stdout:2/348: creat d13/d34/d4d/f88 x:0 0 0 2024-03-22T06:14:57.214 INFO:tasks.workunit.client.0.smithi006.stdout:2/349: truncate d13/d20/d26/d19/f48 759527 0 2024-03-22T06:14:57.215 INFO:tasks.workunit.client.0.smithi006.stdout:5/389: mknod d0/db/d16/d1c/d34/d35/d5d/c72 0 2024-03-22T06:14:57.217 INFO:tasks.workunit.client.0.smithi006.stdout:9/268: dwrite d8/d10/d23/f40 [0,4194304] 0 2024-03-22T06:14:57.217 INFO:tasks.workunit.client.0.smithi006.stdout:9/269: creat d8/d10/d11/f60 x:0 0 0 2024-03-22T06:14:57.217 INFO:tasks.workunit.client.0.smithi006.stdout:2/350: mknod d13/d34/d4d/d4e/d75/d62/c89 0 2024-03-22T06:14:57.224 INFO:tasks.workunit.client.0.smithi006.stdout:9/270: read d8/d10/d28/d48/f4f [2666473,87598] 0 2024-03-22T06:14:57.224 INFO:tasks.workunit.client.0.smithi006.stdout:9/271: fsync f3 0 2024-03-22T06:14:57.224 INFO:tasks.workunit.client.0.smithi006.stdout:9/272: creat d8/d10/d28/d48/d50/f61 x:0 0 0 2024-03-22T06:14:57.224 INFO:tasks.workunit.client.0.smithi006.stdout:9/273: readlink d8/d10/d20/d45/d49/l5f 0 2024-03-22T06:14:57.226 INFO:tasks.workunit.client.0.smithi006.stdout:5/390: dread d0/db/d16/d1c/d34/d3c/f47 [0,4194304] 0 2024-03-22T06:14:57.226 INFO:tasks.workunit.client.0.smithi006.stdout:2/351: creat d13/d53/f8a x:0 0 0 2024-03-22T06:14:57.226 INFO:tasks.workunit.client.0.smithi006.stdout:2/352: write d13/d20/f5a [808137,108163] 0 2024-03-22T06:14:57.226 INFO:tasks.workunit.client.0.smithi006.stdout:2/353: creat d13/d20/f8b x:0 0 0 2024-03-22T06:14:57.238 INFO:tasks.workunit.client.0.smithi006.stdout:5/391: dread d0/f48 [0,4194304] 0 2024-03-22T06:14:57.238 INFO:tasks.workunit.client.0.smithi006.stdout:2/354: rename d13/d20/d35/d65/f77 to d13/d34/f8c 0 2024-03-22T06:14:57.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.246+0000 7fb13a710700 1 -- 172.21.15.6:0/42104182 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb1341058d0 con 0x7fb1340fe040 2024-03-22T06:14:57.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.247+0000 7fb130ff9700 1 -- 172.21.15.6:0/42104182 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fb128058480 con 0x7fb1340fe040 2024-03-22T06:14:57.247 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:14:57.247 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 1 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:14:57.248 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-22T06:14:57.249 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 1 2024-03-22T06:14:57.249 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:14:57.249 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:14:57.250 INFO:tasks.workunit.client.0.smithi006.stdout:9/274: rmdir d8/d10/d20 39 2024-03-22T06:14:57.250 INFO:tasks.workunit.client.0.smithi006.stdout:9/275: fsync d8/ff 0 2024-03-22T06:14:57.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.250+0000 7fb13a710700 1 -- 172.21.15.6:0/42104182 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb11c06bb10 msgr2=0x7fb11c06dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:57.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.250+0000 7fb13a710700 1 --2- 172.21.15.6:0/42104182 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb11c06bb10 0x7fb11c06dfc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb124007fd0 tx=0x7fb124009040 comp rx=0 tx=0).stop 2024-03-22T06:14:57.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.250+0000 7fb13a710700 1 -- 172.21.15.6:0/42104182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1340fe040 msgr2=0x7fb13410df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:57.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.250+0000 7fb13a710700 1 --2- 172.21.15.6:0/42104182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1340fe040 0x7fb13410df80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb12800b3b0 tx=0x7fb128009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:57.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.250+0000 7fb13a710700 1 -- 172.21.15.6:0/42104182 shutdown_connections 2024-03-22T06:14:57.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.250+0000 7fb13a710700 1 --2- 172.21.15.6:0/42104182 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb11c06bb10 0x7fb11c06dfc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.250+0000 7fb13a710700 1 --2- 172.21.15.6:0/42104182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1340fce40 0x7fb13410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.250+0000 7fb13a710700 1 --2- 172.21.15.6:0/42104182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1340fe040 0x7fb13410df80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.250+0000 7fb13a710700 1 -- 172.21.15.6:0/42104182 >> 172.21.15.6:0/42104182 conn(0x7fb1340f8570 msgr2=0x7fb1340f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:57.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.250+0000 7fb13a710700 1 -- 172.21.15.6:0/42104182 shutdown_connections 2024-03-22T06:14:57.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.250+0000 7fb13a710700 1 -- 172.21.15.6:0/42104182 wait complete. 2024-03-22T06:14:57.254 INFO:tasks.workunit.client.0.smithi006.stdout:5/392: creat d0/db/d16/d1c/d34/f73 x:0 0 0 2024-03-22T06:14:57.259 INFO:tasks.workunit.client.0.smithi006.stdout:9/276: stat d8/d10/d20/d45/d49/l5b 0 2024-03-22T06:14:57.259 INFO:tasks.workunit.client.0.smithi006.stdout:9/277: fdatasync d8/d10/d28/f35 0 2024-03-22T06:14:57.260 INFO:tasks.workunit.client.0.smithi006.stdout:5/393: dread d0/f1a [0,4194304] 0 2024-03-22T06:14:57.260 INFO:tasks.workunit.client.0.smithi006.stdout:5/394: chown d0/db/dc/d14/f3f 10965072 1 2024-03-22T06:14:57.262 INFO:tasks.workunit.client.0.smithi006.stdout:2/355: rmdir d13/d53/d29 39 2024-03-22T06:14:57.262 INFO:tasks.workunit.client.0.smithi006.stdout:2/356: write d13/d20/f8b [239209,49416] 0 2024-03-22T06:14:57.262 INFO:tasks.workunit.client.0.smithi006.stdout:2/357: creat d13/d34/f8d x:0 0 0 2024-03-22T06:14:57.263 INFO:tasks.workunit.client.0.smithi006.stdout:9/278: getdents d8/d10/d20 0 2024-03-22T06:14:57.263 INFO:tasks.workunit.client.0.smithi006.stdout:9/279: write d8/d1f/f5d [757306,55034] 0 2024-03-22T06:14:57.263 INFO:tasks.workunit.client.0.smithi006.stdout:9/280: dread - d8/d10/d11/f60 zero size 2024-03-22T06:14:57.272 INFO:tasks.workunit.client.0.smithi006.stdout:9/281: rename d8/d10/d28/c2b to d8/c62 0 2024-03-22T06:14:57.276 INFO:tasks.workunit.client.0.smithi006.stdout:5/395: dread d0/db/d16/f49 [0,4194304] 0 2024-03-22T06:14:57.278 INFO:tasks.workunit.client.0.smithi006.stdout:2/358: dwrite d13/d20/d35/f45 [0,4194304] 0 2024-03-22T06:14:57.280 INFO:tasks.workunit.client.0.smithi006.stdout:5/396: creat d0/db/dc/d14/f74 x:0 0 0 2024-03-22T06:14:57.280 INFO:tasks.workunit.client.0.smithi006.stdout:5/397: write d0/db/d11/d52/d69/f6a [24731,108887] 0 2024-03-22T06:14:57.280 INFO:tasks.workunit.client.0.smithi006.stdout:2/359: dread f7 [0,4194304] 0 2024-03-22T06:14:57.281 INFO:tasks.workunit.client.0.smithi006.stdout:2/360: dread f7 [0,4194304] 0 2024-03-22T06:14:57.282 INFO:tasks.workunit.client.0.smithi006.stdout:9/282: dread d8/d10/d23/d3e/f5a [0,4194304] 0 2024-03-22T06:14:57.289 INFO:tasks.workunit.client.0.smithi006.stdout:2/361: write d13/d20/f4b [2550889,95428] 0 2024-03-22T06:14:57.289 INFO:tasks.workunit.client.0.smithi006.stdout:2/362: write d13/d34/d6e/d72/f80 [457417,111438] 0 2024-03-22T06:14:57.292 INFO:tasks.workunit.client.0.smithi006.stdout:9/283: write d8/d10/d23/f27 [3747981,114901] 0 2024-03-22T06:14:57.295 INFO:tasks.workunit.client.0.smithi006.stdout:5/398: dwrite d0/db/dc/d14/f45 [0,4194304] 0 2024-03-22T06:14:57.297 INFO:tasks.workunit.client.0.smithi006.stdout:9/284: mknod d8/d10/d28/c63 0 2024-03-22T06:14:57.302 INFO:tasks.workunit.client.0.smithi006.stdout:2/363: dread d13/d20/f4b [0,4194304] 0 2024-03-22T06:14:57.302 INFO:tasks.workunit.client.0.smithi006.stdout:5/399: rename d0/db/f25 to d0/db/d16/d70/f75 0 2024-03-22T06:14:57.309 INFO:tasks.workunit.client.0.smithi006.stdout:2/364: dread d13/d53/d29/f5f [0,4194304] 0 2024-03-22T06:14:57.310 INFO:tasks.workunit.client.0.smithi006.stdout:7/322: sync 2024-03-22T06:14:57.310 INFO:tasks.workunit.client.0.smithi006.stdout:3/351: sync 2024-03-22T06:14:57.310 INFO:tasks.workunit.client.0.smithi006.stdout:3/352: creat d0/d8/d1c/d3e/d68/f6d x:0 0 0 2024-03-22T06:14:57.310 INFO:tasks.workunit.client.0.smithi006.stdout:3/353: creat d0/d23/d33/d3d/f6e x:0 0 0 2024-03-22T06:14:57.311 INFO:tasks.workunit.client.0.smithi006.stdout:7/323: read d5/d1e/d29/f5d [14473,61760] 0 2024-03-22T06:14:57.311 INFO:tasks.workunit.client.0.smithi006.stdout:6/342: sync 2024-03-22T06:14:57.311 INFO:tasks.workunit.client.0.smithi006.stdout:6/343: readlink d6/d1b/d2c/l36 0 2024-03-22T06:14:57.311 INFO:tasks.workunit.client.0.smithi006.stdout:6/344: fsync d6/d24/f58 0 2024-03-22T06:14:57.313 INFO:tasks.workunit.client.0.smithi006.stdout:9/285: creat d8/f64 x:0 0 0 2024-03-22T06:14:57.313 INFO:tasks.workunit.client.0.smithi006.stdout:9/286: stat d8/f9 0 2024-03-22T06:14:57.316 INFO:tasks.workunit.client.0.smithi006.stdout:5/400: mkdir d0/db/d76 0 2024-03-22T06:14:57.316 INFO:tasks.workunit.client.0.smithi006.stdout:5/401: chown d0/db/f57 852779028 1 2024-03-22T06:14:57.316 INFO:tasks.workunit.client.0.smithi006.stdout:5/402: truncate d0/db/d16/d1c/d34/f53 468469 0 2024-03-22T06:14:57.316 INFO:tasks.workunit.client.0.smithi006.stdout:5/403: read - d0/db/f60 zero size 2024-03-22T06:14:57.322 INFO:tasks.workunit.client.0.smithi006.stdout:3/354: dread d0/d4/d31/f3b [0,4194304] 0 2024-03-22T06:14:57.322 INFO:tasks.workunit.client.0.smithi006.stdout:9/287: creat d8/d10/d23/d3e/f65 x:0 0 0 2024-03-22T06:14:57.322 INFO:tasks.workunit.client.0.smithi006.stdout:9/288: stat d8/d10/d20/d45/d49/l5f 0 2024-03-22T06:14:57.324 INFO:tasks.workunit.client.0.smithi006.stdout:6/345: dread d6/d1b/d2c/f59 [0,4194304] 0 2024-03-22T06:14:57.324 INFO:tasks.workunit.client.0.smithi006.stdout:6/346: dread - d6/d1d/d64/f6a zero size 2024-03-22T06:14:57.325 INFO:tasks.workunit.client.0.smithi006.stdout:2/365: dread d13/d34/d6e/f84 [0,4194304] 0 2024-03-22T06:14:57.327 INFO:tasks.workunit.client.0.smithi006.stdout:3/355: rename d0/d4/d3a/c3c to d0/d23/d27/c6f 0 2024-03-22T06:14:57.327 INFO:tasks.workunit.client.0.smithi006.stdout:3/356: chown d0/d8/f1a 186224 1 2024-03-22T06:14:57.329 INFO:tasks.workunit.client.0.smithi006.stdout:9/289: unlink d8/d10/d28/f1d 0 2024-03-22T06:14:57.329 INFO:tasks.workunit.client.0.smithi006.stdout:9/290: stat f1 0 2024-03-22T06:14:57.332 INFO:tasks.workunit.client.0.smithi006.stdout:6/347: truncate d6/d1b/d2c/f43 2173580 0 2024-03-22T06:14:57.332 INFO:tasks.workunit.client.0.smithi006.stdout:6/348: creat d6/d1d/d32/d4f/f7e x:0 0 0 2024-03-22T06:14:57.334 INFO:tasks.workunit.client.0.smithi006.stdout:7/324: dwrite d5/d1e/d29/d10/f11 [0,4194304] 0 2024-03-22T06:14:57.336 INFO:tasks.workunit.client.0.smithi006.stdout:2/366: rename d13/d20/d35/l3c to d13/d20/d26/l8e 0 2024-03-22T06:14:57.336 INFO:tasks.workunit.client.0.smithi006.stdout:2/367: truncate d13/d53/d29/f47 1524685 0 2024-03-22T06:14:57.346 INFO:tasks.workunit.client.0.smithi006.stdout:2/368: mkdir d13/d8f 0 2024-03-22T06:14:57.346 INFO:tasks.workunit.client.0.smithi006.stdout:2/369: chown d13/d69 90 1 2024-03-22T06:14:57.349 INFO:tasks.workunit.client.0.smithi006.stdout:7/325: dread d5/d1e/d3f/f45 [0,4194304] 0 2024-03-22T06:14:57.349 INFO:tasks.workunit.client.0.smithi006.stdout:5/404: dwrite d0/f5 [0,4194304] 0 2024-03-22T06:14:57.349 INFO:tasks.workunit.client.0.smithi006.stdout:7/326: truncate d5/fd 1711513 0 2024-03-22T06:14:57.351 INFO:tasks.workunit.client.0.smithi006.stdout:9/291: dwrite d8/d10/d23/f40 [0,4194304] 0 2024-03-22T06:14:57.351 INFO:tasks.workunit.client.0.smithi006.stdout:6/349: rename d6/c76 to d6/d24/c7f 0 2024-03-22T06:14:57.351 INFO:tasks.workunit.client.0.smithi006.stdout:3/357: dwrite d0/d23/d27/f59 [0,4194304] 0 2024-03-22T06:14:57.351 INFO:tasks.workunit.client.0.smithi006.stdout:3/358: write d0/d23/d27/f47 [378637,42557] 0 2024-03-22T06:14:57.353 INFO:tasks.workunit.client.0.smithi006.stdout:2/370: link d13/f18 d13/d20/d35/f90 0 2024-03-22T06:14:57.360 INFO:tasks.workunit.client.0.smithi006.stdout:7/327: symlink d5/l64 0 2024-03-22T06:14:57.360 INFO:tasks.workunit.client.0.smithi006.stdout:7/328: write d5/d1e/d3f/f5a [872201,69941] 0 2024-03-22T06:14:57.361 INFO:tasks.workunit.client.0.smithi006.stdout:6/350: dread d6/d1b/f2a [0,4194304] 0 2024-03-22T06:14:57.361 INFO:tasks.workunit.client.0.smithi006.stdout:6/351: rename d6/d1d to d6/d1d/d5c/d80 22 2024-03-22T06:14:57.362 INFO:tasks.workunit.client.0.smithi006.stdout:5/405: dwrite d0/db/d16/d1c/d34/d35/f6e [0,4194304] 0 2024-03-22T06:14:57.363 INFO:tasks.workunit.client.0.smithi006.stdout:9/292: creat d8/d10/d20/d45/f66 x:0 0 0 2024-03-22T06:14:57.363 INFO:tasks.workunit.client.0.smithi006.stdout:9/293: dread - d8/d10/d28/d48/d50/f5e zero size 2024-03-22T06:14:57.366 INFO:tasks.workunit.client.0.smithi006.stdout:3/359: link d0/c1f d0/d8/d6a/c70 0 2024-03-22T06:14:57.366 INFO:tasks.workunit.client.0.smithi006.stdout:3/360: write d0/d8/d1c/d3e/d68/f6b [475647,21213] 0 2024-03-22T06:14:57.370 INFO:tasks.workunit.client.0.smithi006.stdout:6/352: dread d6/d24/d44/f4d [0,4194304] 0 2024-03-22T06:14:57.371 INFO:tasks.workunit.client.0.smithi006.stdout:7/329: mknod d5/d1e/d63/c65 0 2024-03-22T06:14:57.371 INFO:tasks.workunit.client.0.smithi006.stdout:7/330: stat d5/d1e/d29/d1f 0 2024-03-22T06:14:57.371 INFO:tasks.workunit.client.0.smithi006.stdout:7/331: write d5/d1e/d29/d1f/d2d/f51 [883820,27901] 0 2024-03-22T06:14:57.371 INFO:tasks.workunit.client.0.smithi006.stdout:7/332: write d5/d1e/d3f/f4b [4317607,104278] 0 2024-03-22T06:14:57.371 INFO:tasks.workunit.client.0.smithi006.stdout:7/333: fsync d5/d1e/d29/f54 0 2024-03-22T06:14:57.372 INFO:tasks.workunit.client.0.smithi006.stdout:2/371: getdents d13/d20/d26 0 2024-03-22T06:14:57.377 INFO:tasks.workunit.client.0.smithi006.stdout:5/406: mkdir d0/db/d16/d1c/d34/d3c/d54/d65/d77 0 2024-03-22T06:14:57.380 INFO:tasks.workunit.client.0.smithi006.stdout:9/294: mknod d8/d1f/c67 0 2024-03-22T06:14:57.382 INFO:tasks.workunit.client.0.smithi006.stdout:3/361: rename d0/c1f to d0/d8/d6a/c71 0 2024-03-22T06:14:57.386 INFO:tasks.workunit.client.0.smithi006.stdout:7/334: unlink d5/d1e/d29/c46 0 2024-03-22T06:14:57.388 INFO:tasks.workunit.client.0.smithi006.stdout:2/372: stat d13/d53/d29/c2e 0 2024-03-22T06:14:57.391 INFO:tasks.workunit.client.0.smithi006.stdout:5/407: dwrite d0/db/d16/d1c/d34/f53 [0,4194304] 0 2024-03-22T06:14:57.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.395+0000 7f1069e48700 1 -- 172.21.15.6:0/766391603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10640fff00 msgr2=0x7f1064102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:57.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.395+0000 7f1069e48700 1 --2- 172.21.15.6:0/766391603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10640fff00 0x7f1064102320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1058004940 tx=0x7f1058009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:57.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.395+0000 7f1069e48700 1 -- 172.21.15.6:0/766391603 shutdown_connections 2024-03-22T06:14:57.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.395+0000 7f1069e48700 1 --2- 172.21.15.6:0/766391603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10640fff00 0x7f1064102320 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.395+0000 7f1069e48700 1 --2- 172.21.15.6:0/766391603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1064066ef0 0x7f10640ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.395+0000 7f1069e48700 1 -- 172.21.15.6:0/766391603 >> 172.21.15.6:0/766391603 conn(0x7f10640f7430 msgr2=0x7f10640f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:57.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.395+0000 7f1069e48700 1 -- 172.21.15.6:0/766391603 shutdown_connections 2024-03-22T06:14:57.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.395+0000 7f1069e48700 1 -- 172.21.15.6:0/766391603 wait complete. 2024-03-22T06:14:57.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.395+0000 7f1069e48700 1 Processor -- start 2024-03-22T06:14:57.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f1069e48700 1 -- start start 2024-03-22T06:14:57.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f1069e48700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1064066ef0 0x7f106410d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:57.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f1069e48700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10640fff00 0x7f106410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:57.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f1069e48700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106410e490 con 0x7f10640fff00 2024-03-22T06:14:57.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f1069e48700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106410e600 con 0x7f1064066ef0 2024-03-22T06:14:57.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f1062ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10640fff00 0x7f106410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:57.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f1062ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10640fff00 0x7f106410de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33178/0 (socket says 172.21.15.6:33178) 2024-03-22T06:14:57.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f1062ffd700 1 -- 172.21.15.6:0/669875527 learned_addr learned my addr 172.21.15.6:0/669875527 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:57.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f10637fe700 1 --2- 172.21.15.6:0/669875527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1064066ef0 0x7f106410d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:57.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f1062ffd700 1 -- 172.21.15.6:0/669875527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1064066ef0 msgr2=0x7f106410d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:57.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f1062ffd700 1 --2- 172.21.15.6:0/669875527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1064066ef0 0x7f106410d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.396+0000 7f1062ffd700 1 -- 172.21.15.6:0/669875527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1058005040 con 0x7f10640fff00 2024-03-22T06:14:57.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.397+0000 7f1062ffd700 1 --2- 172.21.15.6:0/669875527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10640fff00 0x7f106410de90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1058009150 tx=0x7f1058009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:57.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.397+0000 7f104bfff700 1 -- 172.21.15.6:0/669875527 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f105800e040 con 0x7f10640fff00 2024-03-22T06:14:57.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.397+0000 7f104bfff700 1 -- 172.21.15.6:0/669875527 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1058004290 con 0x7f10640fff00 2024-03-22T06:14:57.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.397+0000 7f1069e48700 1 -- 172.21.15.6:0/669875527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1064113000 con 0x7f10640fff00 2024-03-22T06:14:57.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.397+0000 7f104bfff700 1 -- 172.21.15.6:0/669875527 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f105801c630 con 0x7f10640fff00 2024-03-22T06:14:57.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.397+0000 7f1069e48700 1 -- 172.21.15.6:0/669875527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10641135c0 con 0x7f10640fff00 2024-03-22T06:14:57.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.398+0000 7f1069e48700 1 -- 172.21.15.6:0/669875527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106404c710 con 0x7f10640fff00 2024-03-22T06:14:57.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.399+0000 7f104bfff700 1 -- 172.21.15.6:0/669875527 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f105800bbf0 con 0x7f10640fff00 2024-03-22T06:14:57.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.399+0000 7f104bfff700 1 --2- 172.21.15.6:0/669875527 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f104c06bb20 0x7f104c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:57.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.400+0000 7f104bfff700 1 -- 172.21.15.6:0/669875527 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f105808b3d0 con 0x7f10640fff00 2024-03-22T06:14:57.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.400+0000 7f10637fe700 1 --2- 172.21.15.6:0/669875527 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f104c06bb20 0x7f104c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:57.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.400+0000 7f10637fe700 1 --2- 172.21.15.6:0/669875527 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f104c06bb20 0x7f104c06dfd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1054007fd0 tx=0x7f1054009040 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:57.403 INFO:tasks.workunit.client.0.smithi006.stdout:9/295: creat d8/d1f/f68 x:0 0 0 2024-03-22T06:14:57.406 INFO:tasks.workunit.client.0.smithi006.stdout:3/362: mkdir d0/d8/d1c/d72 0 2024-03-22T06:14:57.407 INFO:tasks.workunit.client.0.smithi006.stdout:3/363: write d0/d8/f17 [531559,123264] 0 2024-03-22T06:14:57.407 INFO:tasks.workunit.client.0.smithi006.stdout:3/364: chown d0/d8/d1c/d3e/d5f/l4a 16094449 1 2024-03-22T06:14:57.407 INFO:tasks.workunit.client.0.smithi006.stdout:7/335: mkdir d5/d1e/d29/d66 0 2024-03-22T06:14:57.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.408+0000 7f104bfff700 1 -- 172.21.15.6:0/669875527 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f1058058820 con 0x7f10640fff00 2024-03-22T06:14:57.410 INFO:tasks.workunit.client.0.smithi006.stdout:2/373: mknod d13/d53/d6b/c91 0 2024-03-22T06:14:57.410 INFO:tasks.workunit.client.0.smithi006.stdout:2/374: creat d13/f92 x:0 0 0 2024-03-22T06:14:57.416 INFO:tasks.workunit.client.0.smithi006.stdout:5/408: creat d0/db/d16/d1c/f78 x:0 0 0 2024-03-22T06:14:57.416 INFO:tasks.workunit.client.0.smithi006.stdout:5/409: chown d0/db/d11/d52 2 1 2024-03-22T06:14:57.418 INFO:tasks.workunit.client.0.smithi006.stdout:9/296: truncate f3 829081 0 2024-03-22T06:14:57.418 INFO:tasks.workunit.client.0.smithi006.stdout:9/297: creat d8/d10/d28/d48/d50/f69 x:0 0 0 2024-03-22T06:14:57.419 INFO:tasks.workunit.client.0.smithi006.stdout:9/298: fsync d8/d10/d28/d48/d50/f61 0 2024-03-22T06:14:57.426 INFO:tasks.workunit.client.0.smithi006.stdout:3/365: symlink d0/d8/d9/d24/l73 0 2024-03-22T06:14:57.429 INFO:tasks.workunit.client.0.smithi006.stdout:5/410: creat d0/db/d11/d61/f79 x:0 0 0 2024-03-22T06:14:57.429 INFO:tasks.workunit.client.0.smithi006.stdout:5/411: dread - d0/db/dc/d14/f74 zero size 2024-03-22T06:14:57.433 INFO:tasks.workunit.client.0.smithi006.stdout:6/353: rename d6/d1d/d32/d4f to d6/d1d/d5c/d81 0 2024-03-22T06:14:57.435 INFO:tasks.workunit.client.0.smithi006.stdout:7/336: rename d5/d1e/d29/d10/f11 to d5/d1e/d29/d1f/d2d/f67 0 2024-03-22T06:14:57.436 INFO:tasks.workunit.client.0.smithi006.stdout:7/337: fdatasync d5/d1e/d29/d1f/d2d/f4c 0 2024-03-22T06:14:57.443 INFO:tasks.workunit.client.0.smithi006.stdout:3/366: rename d0/d23/d4c/f61 to d0/d8/d1c/f74 0 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:3/367: creat d0/d8/d1c/d3e/d68/f75 x:0 0 0 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:3/368: dread - d0/d8/d1c/d3e/d68/f75 zero size 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:1/324: sync 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:1/325: readlink d8/d9/d3a/d18/l5e 0 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:4/369: sync 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:0/243: sync 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:8/500: sync 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:2/375: dwrite d13/d20/d26/d19/f59 [0,4194304] 0 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:2/376: write d13/d20/d35/f45 [4823157,99807] 0 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:9/299: dwrite d8/d10/d20/f2c [0,4194304] 0 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:9/300: chown d8/d1f 4666 1 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:9/301: truncate d8/d10/d28/f35 11772 0 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:1/326: dread d8/d33/f5b [0,4194304] 0 2024-03-22T06:14:57.444 INFO:tasks.workunit.client.0.smithi006.stdout:7/338: mknod d5/d1e/d29/d59/d5c/c68 0 2024-03-22T06:14:57.446 INFO:tasks.workunit.client.0.smithi006.stdout:5/412: rename d0/db/d50/d58/f5f to d0/db/d11/f7a 0 2024-03-22T06:14:57.447 INFO:tasks.workunit.client.0.smithi006.stdout:6/354: dread d6/f42 [0,4194304] 0 2024-03-22T06:14:57.454 INFO:tasks.workunit.client.0.smithi006.stdout:1/327: write d8/d9/d13/d24/d32/f41 [1994528,119989] 0 2024-03-22T06:14:57.454 INFO:tasks.workunit.client.0.smithi006.stdout:1/328: truncate d8/d9/d3a/d18/f37 598550 0 2024-03-22T06:14:57.457 INFO:tasks.workunit.client.0.smithi006.stdout:3/369: creat d0/d8/d1c/f76 x:0 0 0 2024-03-22T06:14:57.460 INFO:tasks.workunit.client.0.smithi006.stdout:0/244: rmdir db/d4a 0 2024-03-22T06:14:57.465 INFO:tasks.workunit.client.0.smithi006.stdout:1/329: write d8/d9/d3a/f20 [4799399,128541] 0 2024-03-22T06:14:57.467 INFO:tasks.workunit.client.0.smithi006.stdout:2/377: mknod d13/d34/c93 0 2024-03-22T06:14:57.467 INFO:tasks.workunit.client.0.smithi006.stdout:2/378: chown d13/d53/d29 14198689 1 2024-03-22T06:14:57.467 INFO:tasks.workunit.client.0.smithi006.stdout:2/379: chown d13/d69/l81 4230770 1 2024-03-22T06:14:57.471 INFO:tasks.workunit.client.0.smithi006.stdout:4/370: rename d6/d13/l79 to d6/d13/d3b/l81 0 2024-03-22T06:14:57.472 INFO:tasks.workunit.client.0.smithi006.stdout:9/302: dwrite d8/d10/d28/f55 [0,4194304] 0 2024-03-22T06:14:57.473 INFO:tasks.workunit.client.0.smithi006.stdout:8/501: dwrite d9/db/dc/fd [0,4194304] 0 2024-03-22T06:14:57.473 INFO:tasks.workunit.client.0.smithi006.stdout:8/502: write d9/db/d37/d4a/f82 [665654,130449] 0 2024-03-22T06:14:57.473 INFO:tasks.workunit.client.0.smithi006.stdout:5/413: link d0/db/d16/d1c/d34/f4b d0/db/d16/d1c/d34/d3c/d5e/d66/f7b 0 2024-03-22T06:14:57.474 INFO:tasks.workunit.client.0.smithi006.stdout:7/339: dwrite d5/d1e/d29/f13 [0,4194304] 0 2024-03-22T06:14:57.474 INFO:tasks.workunit.client.0.smithi006.stdout:7/340: readlink d5/l19 0 2024-03-22T06:14:57.475 INFO:tasks.workunit.client.0.smithi006.stdout:6/355: creat d6/d1b/f82 x:0 0 0 2024-03-22T06:14:57.479 INFO:tasks.workunit.client.0.smithi006.stdout:3/370: mknod d0/d8/d1c/d72/c77 0 2024-03-22T06:14:57.479 INFO:tasks.workunit.client.0.smithi006.stdout:3/371: fsync d0/d23/f25 0 2024-03-22T06:14:57.479 INFO:tasks.workunit.client.0.smithi006.stdout:3/372: readlink d0/d8/d9/d24/l54 0 2024-03-22T06:14:57.481 INFO:tasks.workunit.client.0.smithi006.stdout:0/245: creat db/d10/d33/f57 x:0 0 0 2024-03-22T06:14:57.481 INFO:tasks.workunit.client.0.smithi006.stdout:0/246: chown db/c29 0 1 2024-03-22T06:14:57.485 INFO:tasks.workunit.client.0.smithi006.stdout:2/380: creat d13/d53/d29/f94 x:0 0 0 2024-03-22T06:14:57.485 INFO:tasks.workunit.client.0.smithi006.stdout:2/381: fsync d13/d34/d4d/f5d 0 2024-03-22T06:14:57.487 INFO:tasks.workunit.client.0.smithi006.stdout:0/247: dread db/d10/f2f [0,4194304] 0 2024-03-22T06:14:57.488 INFO:tasks.workunit.client.0.smithi006.stdout:4/371: read d6/d1b/d20/f22 [110943,5431] 0 2024-03-22T06:14:57.497 INFO:tasks.workunit.client.0.smithi006.stdout:9/303: symlink d8/d10/d28/d39/l6a 0 2024-03-22T06:14:57.497 INFO:tasks.workunit.client.0.smithi006.stdout:8/503: rmdir d9/db/dc 39 2024-03-22T06:14:57.497 INFO:tasks.workunit.client.0.smithi006.stdout:8/504: dread - d9/db/d12/d66/f9b zero size 2024-03-22T06:14:57.505 INFO:tasks.workunit.client.0.smithi006.stdout:7/341: creat d5/d1e/d29/d1f/d2d/d41/f69 x:0 0 0 2024-03-22T06:14:57.505 INFO:tasks.workunit.client.0.smithi006.stdout:7/342: chown d5/fd 46374485 1 2024-03-22T06:14:57.506 INFO:tasks.workunit.client.0.smithi006.stdout:8/505: write d9/db/d37/d4a/f96 [3435972,50687] 0 2024-03-22T06:14:57.507 INFO:tasks.workunit.client.0.smithi006.stdout:6/356: mkdir d6/d7c/d83 0 2024-03-22T06:14:57.514 INFO:tasks.workunit.client.0.smithi006.stdout:2/382: creat d13/d53/d29/d82/f95 x:0 0 0 2024-03-22T06:14:57.514 INFO:tasks.workunit.client.0.smithi006.stdout:2/383: chown d13/d20/d35/d65/f68 112 1 2024-03-22T06:14:57.523 INFO:tasks.workunit.client.0.smithi006.stdout:9/304: mknod d8/d10/d20/d45/d49/c6b 0 2024-03-22T06:14:57.523 INFO:tasks.workunit.client.0.smithi006.stdout:9/305: chown d8/d1f 1071072003 1 2024-03-22T06:14:57.524 INFO:tasks.workunit.client.0.smithi006.stdout:8/506: dwrite d9/db/d12/d66/d6c/d92/fb1 [0,4194304] 0 2024-03-22T06:14:57.526 INFO:tasks.workunit.client.0.smithi006.stdout:5/414: rmdir d0/db/d16/d1c/d34/d3c 39 2024-03-22T06:14:57.526 INFO:tasks.workunit.client.0.smithi006.stdout:5/415: truncate d0/db/d11/f41 4419372 0 2024-03-22T06:14:57.528 INFO:tasks.workunit.client.0.smithi006.stdout:3/373: getdents d0/d8/d1c/d49 0 2024-03-22T06:14:57.531 INFO:tasks.workunit.client.0.smithi006.stdout:7/343: dwrite d5/d1e/f22 [0,4194304] 0 2024-03-22T06:14:57.532 INFO:tasks.workunit.client.0.smithi006.stdout:2/384: unlink d13/d34/d4d/d4e/d75/d62/f6a 0 2024-03-22T06:14:57.532 INFO:tasks.workunit.client.0.smithi006.stdout:2/385: rename d13/d34/d4d/d4e/d75 to d13/d34/d4d/d4e/d75/d96 22 2024-03-22T06:14:57.532 INFO:tasks.workunit.client.0.smithi006.stdout:2/386: chown fc 819404889 1 2024-03-22T06:14:57.537 INFO:tasks.workunit.client.0.smithi006.stdout:4/372: truncate d6/d1b/d20/d6b/f75 3894426 0 2024-03-22T06:14:57.538 INFO:tasks.workunit.client.0.smithi006.stdout:4/373: chown d6/d13/d3b/l3e 730 1 2024-03-22T06:14:57.544 INFO:tasks.workunit.client.0.smithi006.stdout:5/416: fdatasync d0/db/d16/f30 0 2024-03-22T06:14:57.545 INFO:tasks.workunit.client.0.smithi006.stdout:5/417: stat d0/db 0 2024-03-22T06:14:57.545 INFO:tasks.workunit.client.0.smithi006.stdout:7/344: write d5/d1e/f3a [1350816,17380] 0 2024-03-22T06:14:57.549 INFO:tasks.workunit.client.0.smithi006.stdout:5/418: stat d0/db/d16/f46 0 2024-03-22T06:14:57.549 INFO:tasks.workunit.client.0.smithi006.stdout:5/419: write d0/db/d11/f3d [538997,83151] 0 2024-03-22T06:14:57.552 INFO:tasks.workunit.client.0.smithi006.stdout:4/374: dread d6/fb [0,4194304] 0 2024-03-22T06:14:57.552 INFO:tasks.workunit.client.0.smithi006.stdout:4/375: getdents d6/d1b/d20/d6b/d7b 0 2024-03-22T06:14:57.552 INFO:tasks.workunit.client.0.smithi006.stdout:4/376: write d6/d1b/d20/f22 [3162688,107912] 0 2024-03-22T06:14:57.553 INFO:tasks.workunit.client.0.smithi006.stdout:5/420: creat d0/db/d76/f7c x:0 0 0 2024-03-22T06:14:57.553 INFO:tasks.workunit.client.0.smithi006.stdout:5/421: creat d0/db/d11/d61/f7d x:0 0 0 2024-03-22T06:14:57.556 INFO:tasks.workunit.client.0.smithi006.stdout:4/377: creat d6/d13/d71/f82 x:0 0 0 2024-03-22T06:14:57.561 INFO:tasks.workunit.client.0.smithi006.stdout:5/422: symlink d0/db/d50/l7e 0 2024-03-22T06:14:57.563 INFO:tasks.workunit.client.0.smithi006.stdout:9/306: dwrite f4 [0,4194304] 0 2024-03-22T06:14:57.563 INFO:tasks.workunit.client.0.smithi006.stdout:9/307: creat d8/d10/f6c x:0 0 0 2024-03-22T06:14:57.563 INFO:tasks.workunit.client.0.smithi006.stdout:9/308: chown d8/fc 3136 1 2024-03-22T06:14:57.565 INFO:tasks.workunit.client.0.smithi006.stdout:5/423: mknod d0/db/d50/c7f 0 2024-03-22T06:14:57.565 INFO:tasks.workunit.client.0.smithi006.stdout:5/424: creat d0/db/d50/f80 x:0 0 0 2024-03-22T06:14:57.566 INFO:tasks.workunit.client.0.smithi006.stdout:4/378: link d6/d1b/d20/d21/l41 d6/d1b/d20/d6b/d60/l83 0 2024-03-22T06:14:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:8/507: dwrite d9/db/f7b [0,4194304] 0 2024-03-22T06:14:57.568 INFO:tasks.workunit.client.0.smithi006.stdout:3/374: dwrite d0/d23/d27/f48 [0,4194304] 0 2024-03-22T06:14:57.568 INFO:tasks.workunit.client.0.smithi006.stdout:3/375: fsync d0/d8/d1c/d3e/d68/f75 0 2024-03-22T06:14:57.570 INFO:tasks.workunit.client.0.smithi006.stdout:2/387: dwrite d13/d20/d35/f45 [0,4194304] 0 2024-03-22T06:14:57.570 INFO:tasks.workunit.client.0.smithi006.stdout:2/388: read - d13/d53/d6b/f76 zero size 2024-03-22T06:14:57.571 INFO:tasks.workunit.client.0.smithi006.stdout:9/309: truncate d8/ff 7344655 0 2024-03-22T06:14:57.577 INFO:tasks.workunit.client.0.smithi006.stdout:2/389: dread d13/d20/f57 [0,4194304] 0 2024-03-22T06:14:57.579 INFO:tasks.workunit.client.0.smithi006.stdout:5/425: truncate d0/db/d16/f46 2675286 0 2024-03-22T06:14:57.579 INFO:tasks.workunit.client.0.smithi006.stdout:7/345: dwrite d5/d1e/d29/f5d [0,4194304] 0 2024-03-22T06:14:57.583 INFO:tasks.workunit.client.0.smithi006.stdout:3/376: write d0/d23/d27/f48 [910430,47877] 0 2024-03-22T06:14:57.583 INFO:tasks.workunit.client.0.smithi006.stdout:5/426: link d0/db/f60 d0/db/d76/f81 0 2024-03-22T06:14:57.589 INFO:tasks.workunit.client.0.smithi006.stdout:2/390: dread fc [0,4194304] 0 2024-03-22T06:14:57.590 INFO:tasks.workunit.client.0.smithi006.stdout:2/391: read d13/d34/d4d/f5c [143856,108317] 0 2024-03-22T06:14:57.592 INFO:tasks.workunit.client.0.smithi006.stdout:7/346: rename d5/cc to d5/d1e/d29/d59/d5c/c6a 0 2024-03-22T06:14:57.593 INFO:tasks.workunit.client.0.smithi006.stdout:4/379: write d6/d1b/d20/f33 [1169679,88781] 0 2024-03-22T06:14:57.594 INFO:tasks.workunit.client.0.smithi006.stdout:3/377: mknod d0/d8/d1c/d3e/d5f/c78 0 2024-03-22T06:14:57.597 INFO:tasks.workunit.client.0.smithi006.stdout:5/427: mkdir d0/db/d50/d82 0 2024-03-22T06:14:57.597 INFO:tasks.workunit.client.0.smithi006.stdout:5/428: creat d0/db/d50/f83 x:0 0 0 2024-03-22T06:14:57.599 INFO:tasks.workunit.client.0.smithi006.stdout:9/310: dwrite d8/d1f/f5d [0,4194304] 0 2024-03-22T06:14:57.601 INFO:tasks.workunit.client.0.smithi006.stdout:8/508: dwrite d9/db/dc/de/d3e/f73 [0,4194304] 0 2024-03-22T06:14:57.603 INFO:tasks.workunit.client.0.smithi006.stdout:4/380: dread d6/f8 [4194304,4194304] 0 2024-03-22T06:14:57.604 INFO:tasks.workunit.client.0.smithi006.stdout:7/347: rmdir d5/d1e/d29/d59/d5c 39 2024-03-22T06:14:57.615 INFO:tasks.workunit.client.0.smithi006.stdout:9/311: symlink d8/d10/d20/d45/l6d 0 2024-03-22T06:14:57.621 INFO:tasks.workunit.client.0.smithi006.stdout:2/392: rmdir d13/d53 39 2024-03-22T06:14:57.624 INFO:tasks.workunit.client.0.smithi006.stdout:8/509: symlink d9/d14/d47/lbc 0 2024-03-22T06:14:57.624 INFO:tasks.workunit.client.0.smithi006.stdout:8/510: chown d9/db/d12/d66/d6c/f7c 230965895 1 2024-03-22T06:14:57.624 INFO:tasks.workunit.client.0.smithi006.stdout:4/381: mknod d6/d1b/d46/c84 0 2024-03-22T06:14:57.625 INFO:tasks.workunit.client.0.smithi006.stdout:4/382: write d6/d1b/d20/d21/f7e [967365,85872] 0 2024-03-22T06:14:57.627 INFO:tasks.workunit.client.0.smithi006.stdout:7/348: truncate d5/d1e/d29/d1f/d2d/d55/f58 549091 0 2024-03-22T06:14:57.631 INFO:tasks.workunit.client.0.smithi006.stdout:5/429: symlink d0/db/d16/l84 0 2024-03-22T06:14:57.631 INFO:tasks.workunit.client.0.smithi006.stdout:8/511: mknod d9/db/dc/de/d3e/cbd 0 2024-03-22T06:14:57.633 INFO:tasks.workunit.client.0.smithi006.stdout:4/383: symlink d6/d1b/d20/d21/d2a/l85 0 2024-03-22T06:14:57.635 INFO:tasks.workunit.client.0.smithi006.stdout:5/430: rename d0/db/d16/d1c/d34/d3c to d0/db/d16/d1c/d34/d35/d85 0 2024-03-22T06:14:57.635 INFO:tasks.workunit.client.0.smithi006.stdout:5/431: chown d0/db/d16 25 1 2024-03-22T06:14:57.637 INFO:tasks.workunit.client.0.smithi006.stdout:8/512: symlink d9/db/d35/da9/lbe 0 2024-03-22T06:14:57.640 INFO:tasks.workunit.client.0.smithi006.stdout:5/432: creat d0/db/d16/d1c/d34/d35/d85/d54/d65/d77/f86 x:0 0 0 2024-03-22T06:14:57.640 INFO:tasks.workunit.client.0.smithi006.stdout:5/433: fdatasync d0/db/d16/f46 0 2024-03-22T06:14:57.641 INFO:tasks.workunit.client.0.smithi006.stdout:4/384: dread f1 [0,4194304] 0 2024-03-22T06:14:57.642 INFO:tasks.workunit.client.0.smithi006.stdout:8/513: mknod d9/d14/d47/d94/da3/cbf 0 2024-03-22T06:14:57.642 INFO:tasks.workunit.client.0.smithi006.stdout:8/514: creat d9/d14/d47/d94/da3/fc0 x:0 0 0 2024-03-22T06:14:57.647 INFO:tasks.workunit.client.0.smithi006.stdout:5/434: truncate d0/db/d16/d1c/f40 2094114 0 2024-03-22T06:14:57.647 INFO:tasks.workunit.client.0.smithi006.stdout:4/385: symlink d6/d1b/d20/d21/d77/l86 0 2024-03-22T06:14:57.647 INFO:tasks.workunit.client.0.smithi006.stdout:4/386: dread - d6/d1b/f2d zero size 2024-03-22T06:14:57.653 INFO:tasks.workunit.client.0.smithi006.stdout:5/435: write d0/f48 [1581032,123464] 0 2024-03-22T06:14:57.653 INFO:tasks.workunit.client.0.smithi006.stdout:5/436: chown d0/db/d16/d1c/d34/d35/d85/d54/d65/d77/f86 0 1 2024-03-22T06:14:57.653 INFO:tasks.workunit.client.0.smithi006.stdout:5/437: write d0/db/d50/f83 [469621,77577] 0 2024-03-22T06:14:57.653 INFO:tasks.workunit.client.0.smithi006.stdout:5/438: dread - d0/db/d16/d1c/d34/d35/d85/f55 zero size 2024-03-22T06:14:57.653 INFO:tasks.workunit.client.0.smithi006.stdout:5/439: chown d0/db/d16/d1c/f78 7722522 1 2024-03-22T06:14:57.653 INFO:tasks.workunit.client.0.smithi006.stdout:5/440: truncate d0/db/f6c 433933 0 2024-03-22T06:14:57.654 INFO:tasks.workunit.client.0.smithi006.stdout:8/515: rename d9/db/d12/d66/d6c/l78 to d9/db/d12/d1e/lc1 0 2024-03-22T06:14:57.654 INFO:tasks.workunit.client.0.smithi006.stdout:8/516: chown d9/db/d12/d7e/f8b 493062 1 2024-03-22T06:14:57.654 INFO:tasks.workunit.client.0.smithi006.stdout:8/517: write d9/db/d12/f43 [464924,53393] 0 2024-03-22T06:14:57.656 INFO:tasks.workunit.client.0.smithi006.stdout:4/387: write d6/f30 [884557,112397] 0 2024-03-22T06:14:57.658 INFO:tasks.workunit.client.0.smithi006.stdout:4/388: write d6/d1b/d46/f5a [64955,68228] 0 2024-03-22T06:14:57.658 INFO:tasks.workunit.client.0.smithi006.stdout:4/389: write d6/d1b/d46/f58 [833289,35599] 0 2024-03-22T06:14:57.658 INFO:tasks.workunit.client.0.smithi006.stdout:4/390: dread - d6/d13/d71/f82 zero size 2024-03-22T06:14:57.689 INFO:tasks.workunit.client.0.smithi006.stdout:5/441: getdents d0/db/d16/d1c/d34/d35/d85 0 2024-03-22T06:14:57.689 INFO:tasks.workunit.client.0.smithi006.stdout:5/442: fdatasync d0/db/d16/d1c/f40 0 2024-03-22T06:14:57.690 INFO:tasks.workunit.client.0.smithi006.stdout:8/518: creat d9/d5e/fc2 x:0 0 0 2024-03-22T06:14:57.695 INFO:tasks.workunit.client.0.smithi006.stdout:5/443: readlink d0/db/d16/l2a 0 2024-03-22T06:14:57.702 INFO:tasks.workunit.client.0.smithi006.stdout:5/444: dread d0/db/d11/f2d [4194304,4194304] 0 2024-03-22T06:14:57.716 INFO:tasks.workunit.client.0.smithi006.stdout:5/445: rename d0/f3e to d0/db/d16/d1c/d34/d35/d85/d54/f87 0 2024-03-22T06:14:57.718 INFO:tasks.workunit.client.0.smithi006.stdout:8/519: rmdir d9/db/dc/de/d3e 39 2024-03-22T06:14:57.718 INFO:tasks.workunit.client.0.smithi006.stdout:8/520: readlink d9/l45 0 2024-03-22T06:14:57.726 INFO:tasks.workunit.client.0.smithi006.stdout:8/521: rename d9/db/d12/d66/d6c/d80/d85 to d9/db/d37/d76/db3/dc3 0 2024-03-22T06:14:57.729 INFO:tasks.workunit.client.0.smithi006.stdout:8/522: mkdir d9/db/d37/dc4 0 2024-03-22T06:14:57.734 INFO:tasks.workunit.client.0.smithi006.stdout:7/349: dwrite d5/d1e/d29/d10/f3d [0,4194304] 0 2024-03-22T06:14:57.734 INFO:tasks.workunit.client.0.smithi006.stdout:5/446: dwrite d0/db/d50/f83 [0,4194304] 0 2024-03-22T06:14:57.735 INFO:tasks.workunit.client.0.smithi006.stdout:4/391: rmdir d6 39 2024-03-22T06:14:57.743 INFO:tasks.workunit.client.0.smithi006.stdout:5/447: truncate d0/db/d16/f46 1574330 0 2024-03-22T06:14:57.743 INFO:tasks.workunit.client.0.smithi006.stdout:5/448: creat d0/db/d16/d1c/d34/f88 x:0 0 0 2024-03-22T06:14:57.745 INFO:tasks.workunit.client.0.smithi006.stdout:7/350: dread d5/d1e/d29/d1f/d2d/f47 [0,4194304] 0 2024-03-22T06:14:57.746 INFO:tasks.workunit.client.0.smithi006.stdout:4/392: mknod d6/d1b/d20/d21/d2a/d43/c87 0 2024-03-22T06:14:57.748 INFO:tasks.workunit.client.0.smithi006.stdout:5/449: rmdir d0/db/d16/d1c/d34/d35/d85/d5e 39 2024-03-22T06:14:57.750 INFO:tasks.workunit.client.0.smithi006.stdout:4/393: rename d6/d13/d73 to d6/d1b/d20/d6b/d88 0 2024-03-22T06:14:57.750 INFO:tasks.workunit.client.0.smithi006.stdout:4/394: truncate d6/d13/d3b/f4f 322255 0 2024-03-22T06:14:57.754 INFO:tasks.workunit.client.0.smithi006.stdout:8/523: dwrite d9/db/d35/fbb [0,4194304] 0 2024-03-22T06:14:57.754 INFO:tasks.workunit.client.0.smithi006.stdout:7/351: dread d5/d1e/d29/d1f/d2d/f67 [0,4194304] 0 2024-03-22T06:14:57.754 INFO:tasks.workunit.client.0.smithi006.stdout:7/352: stat d5/d1e/d3f/f48 0 2024-03-22T06:14:57.754 INFO:tasks.workunit.client.0.smithi006.stdout:4/395: getdents d6/d1b/d20 0 2024-03-22T06:14:57.754 INFO:tasks.workunit.client.0.smithi006.stdout:4/396: write d6/d1b/d20/f24 [3227262,95479] 0 2024-03-22T06:14:57.754 INFO:tasks.workunit.client.0.smithi006.stdout:4/397: write f3 [5011999,59492] 0 2024-03-22T06:14:57.758 INFO:tasks.workunit.client.0.smithi006.stdout:5/450: getdents d0/db/d16/d1c/d34/d35 0 2024-03-22T06:14:57.760 INFO:tasks.workunit.client.0.smithi006.stdout:7/353: write d5/d1e/d3f/f45 [3711280,92720] 0 2024-03-22T06:14:57.761 INFO:tasks.workunit.client.0.smithi006.stdout:7/354: fdatasync d5/f9 0 2024-03-22T06:14:57.761 INFO:tasks.workunit.client.0.smithi006.stdout:7/355: chown d5/d1e/d29/d59/d5c/c68 786911973 1 2024-03-22T06:14:57.761 INFO:tasks.workunit.client.0.smithi006.stdout:7/356: getdents d5/d1e/d29/d1f/d2d/d55 0 2024-03-22T06:14:57.761 INFO:tasks.workunit.client.0.smithi006.stdout:7/357: chown d5/d1e/d29/d1f/d2d/d41/f42 614200466 1 2024-03-22T06:14:57.769 INFO:tasks.workunit.client.0.smithi006.stdout:5/451: creat d0/db/d16/d1c/d34/d35/d85/f89 x:0 0 0 2024-03-22T06:14:57.772 INFO:tasks.workunit.client.0.smithi006.stdout:4/398: dwrite d6/d1b/d20/d6b/f75 [0,4194304] 0 2024-03-22T06:14:57.772 INFO:tasks.workunit.client.0.smithi006.stdout:4/399: fdatasync d6/d1b/d20/f24 0 2024-03-22T06:14:57.775 INFO:tasks.workunit.client.0.smithi006.stdout:7/358: link d5/d1e/d29/d1f/d2d/c35 d5/d1e/d29/d1f/c6b 0 2024-03-22T06:14:57.775 INFO:tasks.workunit.client.0.smithi006.stdout:7/359: chown d5/d1e/d29/l2b 57660 1 2024-03-22T06:14:57.784 INFO:tasks.workunit.client.0.smithi006.stdout:4/400: rename d6/d13/c7c to d6/d1b/d20/c89 0 2024-03-22T06:14:57.784 INFO:tasks.workunit.client.0.smithi006.stdout:4/401: write d6/d1b/d20/d21/d2a/d5c/f6c [295923,20035] 0 2024-03-22T06:14:57.784 INFO:tasks.workunit.client.0.smithi006.stdout:4/402: truncate d6/d1b/d20/d21/d2a/d5c/f6c 1239188 0 2024-03-22T06:14:57.787 INFO:tasks.workunit.client.0.smithi006.stdout:5/452: getdents d0/db/d11/d61 0 2024-03-22T06:14:57.789 INFO:tasks.workunit.client.0.smithi006.stdout:4/403: mkdir d6/d1b/d20/d21/d2a/d8a 0 2024-03-22T06:14:57.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.791+0000 7f1069e48700 1 -- 172.21.15.6:0/669875527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f10641138c0 con 0x7f10640fff00 2024-03-22T06:14:57.793 INFO:tasks.workunit.client.0.smithi006.stdout:7/360: dwrite d5/f9 [0,4194304] 0 2024-03-22T06:14:57.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.792+0000 7f104bfff700 1 -- 172.21.15.6:0/669875527 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1805 (secure 0 0 0) 0x7f10580583b0 con 0x7f10640fff00 2024-03-22T06:14:57.793 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:14:57.793 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:14:57.794 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:14:57.795 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:14:57.796 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:14:57.796 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:14:57.796 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:14:57.796 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:14:57.796 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:14:57.796 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:14:57.796 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:14:57.796 INFO:tasks.workunit.client.0.smithi006.stdout:4/404: unlink d6/ff 0 2024-03-22T06:14:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.795+0000 7f1069e48700 1 -- 172.21.15.6:0/669875527 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f104c06bb20 msgr2=0x7f104c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.795+0000 7f1069e48700 1 --2- 172.21.15.6:0/669875527 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f104c06bb20 0x7f104c06dfd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1054007fd0 tx=0x7f1054009040 comp rx=0 tx=0).stop 2024-03-22T06:14:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.795+0000 7f1069e48700 1 -- 172.21.15.6:0/669875527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10640fff00 msgr2=0x7f106410de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.795+0000 7f1069e48700 1 --2- 172.21.15.6:0/669875527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10640fff00 0x7f106410de90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1058009150 tx=0x7f1058009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.795+0000 7f1069e48700 1 -- 172.21.15.6:0/669875527 shutdown_connections 2024-03-22T06:14:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.795+0000 7f1069e48700 1 --2- 172.21.15.6:0/669875527 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f104c06bb20 0x7f104c06dfd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.795+0000 7f1069e48700 1 --2- 172.21.15.6:0/669875527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1064066ef0 0x7f106410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.795+0000 7f1069e48700 1 --2- 172.21.15.6:0/669875527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10640fff00 0x7f106410de90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.795+0000 7f1069e48700 1 -- 172.21.15.6:0/669875527 >> 172.21.15.6:0/669875527 conn(0x7f10640f7430 msgr2=0x7f10640f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:57.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.795+0000 7f1069e48700 1 -- 172.21.15.6:0/669875527 shutdown_connections 2024-03-22T06:14:57.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.795+0000 7f1069e48700 1 -- 172.21.15.6:0/669875527 wait complete. 2024-03-22T06:14:57.798 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:14:57.798 INFO:tasks.workunit.client.0.smithi006.stdout:7/361: truncate d5/d1e/d3f/f48 1492170 0 2024-03-22T06:14:57.799 INFO:tasks.workunit.client.0.smithi006.stdout:7/362: write d5/fd [1909454,80760] 0 2024-03-22T06:14:57.799 INFO:tasks.workunit.client.0.smithi006.stdout:4/405: creat d6/d1b/d20/d21/d77/f8b x:0 0 0 2024-03-22T06:14:57.799 INFO:tasks.workunit.client.0.smithi006.stdout:4/406: readlink d6/d13/d3b/l72 0 2024-03-22T06:14:57.799 INFO:tasks.workunit.client.0.smithi006.stdout:4/407: chown d6/d1b/f5b 43879500 1 2024-03-22T06:14:57.801 INFO:tasks.workunit.client.0.smithi006.stdout:7/363: creat d5/d1e/d3f/f6c x:0 0 0 2024-03-22T06:14:57.803 INFO:tasks.workunit.client.0.smithi006.stdout:4/408: unlink d6/d1b/d20/f24 0 2024-03-22T06:14:57.809 INFO:tasks.workunit.client.0.smithi006.stdout:5/453: dwrite d0/db/d11/d61/f79 [0,4194304] 0 2024-03-22T06:14:57.809 INFO:tasks.workunit.client.0.smithi006.stdout:5/454: truncate d0/db/d50/f80 23263 0 2024-03-22T06:14:57.817 INFO:tasks.workunit.client.0.smithi006.stdout:7/364: dread d5/d1e/d29/d10/f3d [0,4194304] 0 2024-03-22T06:14:57.817 INFO:tasks.workunit.client.0.smithi006.stdout:7/365: chown d5/d1e/d29/d1f/l27 263200 1 2024-03-22T06:14:57.818 INFO:tasks.workunit.client.0.smithi006.stdout:5/455: mknod d0/db/d50/d82/c8a 0 2024-03-22T06:14:57.818 INFO:tasks.workunit.client.0.smithi006.stdout:5/456: write d0/db/d16/f46 [1730994,107553] 0 2024-03-22T06:14:57.822 INFO:tasks.workunit.client.0.smithi006.stdout:7/366: creat d5/d1e/d29/d1f/d2d/d61/f6d x:0 0 0 2024-03-22T06:14:57.824 INFO:tasks.workunit.client.0.smithi006.stdout:4/409: dwrite d6/f12 [8388608,4194304] 0 2024-03-22T06:14:57.824 INFO:tasks.workunit.client.0.smithi006.stdout:4/410: chown d6/d1b/d20/d21/d2a/d43/d4b/l55 8 1 2024-03-22T06:14:57.824 INFO:tasks.workunit.client.0.smithi006.stdout:4/411: write d6/d13/f6a [2205592,12946] 0 2024-03-22T06:14:57.827 INFO:tasks.workunit.client.0.smithi006.stdout:5/457: mknod d0/db/d16/d1c/d34/d35/d85/d54/d65/d77/c8b 0 2024-03-22T06:14:57.833 INFO:tasks.workunit.client.0.smithi006.stdout:4/412: dread d6/f7 [0,4194304] 0 2024-03-22T06:14:57.834 INFO:tasks.workunit.client.0.smithi006.stdout:4/413: chown d6/d1b/f2d 679 1 2024-03-22T06:14:57.883 INFO:tasks.workunit.client.0.smithi006.stdout:7/367: dwrite d5/d1e/f22 [0,4194304] 0 2024-03-22T06:14:57.884 INFO:tasks.workunit.client.0.smithi006.stdout:5/458: dwrite d0/db/f1e [0,4194304] 0 2024-03-22T06:14:57.890 INFO:tasks.workunit.client.0.smithi006.stdout:7/368: symlink d5/d1e/d63/l6e 0 2024-03-22T06:14:57.892 INFO:tasks.workunit.client.0.smithi006.stdout:5/459: write d0/db/dc/d14/f45 [1840231,27841] 0 2024-03-22T06:14:57.892 INFO:tasks.workunit.client.0.smithi006.stdout:5/460: write d0/db/d16/d1c/f78 [537486,36341] 0 2024-03-22T06:14:57.895 INFO:tasks.workunit.client.0.smithi006.stdout:7/369: mkdir d5/d1e/d29/d10/d6f 0 2024-03-22T06:14:57.898 INFO:tasks.workunit.client.0.smithi006.stdout:4/414: rmdir d6/d1b/d20 39 2024-03-22T06:14:57.916 INFO:tasks.workunit.client.0.smithi006.stdout:5/461: dwrite d0/db/d16/d1c/f40 [0,4194304] 0 2024-03-22T06:14:57.923 INFO:tasks.workunit.client.0.smithi006.stdout:7/370: dwrite d5/d1e/d29/f5d [0,4194304] 0 2024-03-22T06:14:57.923 INFO:tasks.workunit.client.0.smithi006.stdout:5/462: rename d0/db/d16/d1c/d34/d35/f6e to d0/db/d50/d58/f8c 0 2024-03-22T06:14:57.926 INFO:tasks.workunit.client.0.smithi006.stdout:7/371: write d5/d1e/f31 [485076,6651] 0 2024-03-22T06:14:57.927 INFO:tasks.workunit.client.0.smithi006.stdout:5/463: mknod d0/db/dc/d14/c8d 0 2024-03-22T06:14:57.927 INFO:tasks.workunit.client.0.smithi006.stdout:5/464: chown d0/db/d16/d43/f67 918 1 2024-03-22T06:14:57.927 INFO:tasks.workunit.client.0.smithi006.stdout:5/465: fsync d0/db/d50/f5b 0 2024-03-22T06:14:57.927 INFO:tasks.workunit.client.0.smithi006.stdout:5/466: fdatasync d0/db/dc/f3b 0 2024-03-22T06:14:57.929 INFO:tasks.workunit.client.0.smithi006.stdout:7/372: mkdir d5/d1e/d63/d70 0 2024-03-22T06:14:57.930 INFO:tasks.workunit.client.0.smithi006.stdout:7/373: truncate d5/d1e/d3f/f48 1505945 0 2024-03-22T06:14:57.932 INFO:tasks.workunit.client.0.smithi006.stdout:5/467: rename d0/db/dc/f32 to d0/db/d50/f8e 0 2024-03-22T06:14:57.942 INFO:tasks.workunit.client.0.smithi006.stdout:7/374: dread d5/d1e/d29/f5d [0,4194304] 0 2024-03-22T06:14:57.948 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:57 smithi006 ceph-mon[29301]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:57.948 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:57 smithi006 ceph-mon[29301]: pgmap v12: 65 pgs: 65 active+clean; 728 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 73 MiB/s rd, 107 MiB/s wr, 465 op/s 2024-03-22T06:14:57.948 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:57 smithi006 ceph-mon[29301]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:57.948 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:57 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/42104182' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:14:57.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.946+0000 7fb7d92c7700 1 -- 172.21.15.6:0/3860408418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7d40fbce0 msgr2=0x7fb7d40fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:57.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.946+0000 7fb7d92c7700 1 --2- 172.21.15.6:0/3860408418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7d40fbce0 0x7fb7d40fc0f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4004880 tx=0x7fb7c4009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:57.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.947+0000 7fb7d92c7700 1 -- 172.21.15.6:0/3860408418 shutdown_connections 2024-03-22T06:14:57.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.947+0000 7fb7d92c7700 1 --2- 172.21.15.6:0/3860408418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d40fcf90 0x7fb7d40fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.947+0000 7fb7d92c7700 1 --2- 172.21.15.6:0/3860408418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7d40fbce0 0x7fb7d40fc0f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.947+0000 7fb7d92c7700 1 -- 172.21.15.6:0/3860408418 >> 172.21.15.6:0/3860408418 conn(0x7fb7d40f7480 msgr2=0x7fb7d40f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:57.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.947+0000 7fb7d92c7700 1 -- 172.21.15.6:0/3860408418 shutdown_connections 2024-03-22T06:14:57.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.947+0000 7fb7d92c7700 1 -- 172.21.15.6:0/3860408418 wait complete. 2024-03-22T06:14:57.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.948+0000 7fb7d92c7700 1 Processor -- start 2024-03-22T06:14:57.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.948+0000 7fb7d92c7700 1 -- start start 2024-03-22T06:14:57.950 INFO:tasks.workunit.client.0.smithi006.stdout:7/375: dread d5/fe [0,4194304] 0 2024-03-22T06:14:57.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.948+0000 7fb7d92c7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7d40fbce0 0x7fb7d410c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:57.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.948+0000 7fb7d92c7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d40fcf90 0x7fb7d410ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:57.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.948+0000 7fb7d92c7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d410d450 con 0x7fb7d40fcf90 2024-03-22T06:14:57.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.948+0000 7fb7d92c7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d410dc90 con 0x7fb7d40fbce0 2024-03-22T06:14:57.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.948+0000 7fb7d2ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7d40fbce0 0x7fb7d410c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:57.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.948+0000 7fb7d2ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7d40fbce0 0x7fb7d410c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:49288/0 (socket says 172.21.15.6:49288) 2024-03-22T06:14:57.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.948+0000 7fb7d2ffd700 1 -- 172.21.15.6:0/2572838437 learned_addr learned my addr 172.21.15.6:0/2572838437 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:57.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.949+0000 7fb7d27fc700 1 --2- 172.21.15.6:0/2572838437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d40fcf90 0x7fb7d410ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:57.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.949+0000 7fb7d2ffd700 1 -- 172.21.15.6:0/2572838437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d40fcf90 msgr2=0x7fb7d410ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:57.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.949+0000 7fb7d2ffd700 1 --2- 172.21.15.6:0/2572838437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d40fcf90 0x7fb7d410ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:57.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.949+0000 7fb7d2ffd700 1 -- 172.21.15.6:0/2572838437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4005040 con 0x7fb7d40fbce0 2024-03-22T06:14:57.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.949+0000 7fb7d2ffd700 1 --2- 172.21.15.6:0/2572838437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7d40fbce0 0x7fb7d410c8c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4008cd0 tx=0x7fb7c4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:57.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.949+0000 7fb7bb7fe700 1 -- 172.21.15.6:0/2572838437 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7c400e030 con 0x7fb7d40fbce0 2024-03-22T06:14:57.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.949+0000 7fb7bb7fe700 1 -- 172.21.15.6:0/2572838437 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7c4013e60 con 0x7fb7d40fbce0 2024-03-22T06:14:57.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.949+0000 7fb7d92c7700 1 -- 172.21.15.6:0/2572838437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d410df10 con 0x7fb7d40fbce0 2024-03-22T06:14:57.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.949+0000 7fb7bb7fe700 1 -- 172.21.15.6:0/2572838437 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7c40123c0 con 0x7fb7d40fbce0 2024-03-22T06:14:57.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.949+0000 7fb7d92c7700 1 -- 172.21.15.6:0/2572838437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d410e4d0 con 0x7fb7d40fbce0 2024-03-22T06:14:57.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.950+0000 7fb7d92c7700 1 -- 172.21.15.6:0/2572838437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d405f5f0 con 0x7fb7d40fbce0 2024-03-22T06:14:57.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.951+0000 7fb7bb7fe700 1 -- 172.21.15.6:0/2572838437 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7fb7c4013990 con 0x7fb7d40fbce0 2024-03-22T06:14:57.954 INFO:tasks.workunit.client.0.smithi006.stdout:5/468: dwrite d0/db/dc/d14/f1d [4194304,4194304] 0 2024-03-22T06:14:57.954 INFO:tasks.workunit.client.0.smithi006.stdout:5/469: truncate d0/db/d16/f49 1051260 0 2024-03-22T06:14:57.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.951+0000 7fb7bb7fe700 1 --2- 172.21.15.6:0/2572838437 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb7bc06bb10 0x7fb7bc06dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:57.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.951+0000 7fb7bb7fe700 1 -- 172.21.15.6:0/2572838437 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fb7c408c030 con 0x7fb7d40fbce0 2024-03-22T06:14:57.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.952+0000 7fb7d27fc700 1 --2- 172.21.15.6:0/2572838437 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb7bc06bb10 0x7fb7bc06dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:57.956 INFO:tasks.workunit.client.0.smithi006.stdout:5/470: dread d0/db/f6c [0,4194304] 0 2024-03-22T06:14:57.956 INFO:tasks.workunit.client.0.smithi006.stdout:5/471: chown d0/l7 22 1 2024-03-22T06:14:57.956 INFO:tasks.workunit.client.0.smithi006.stdout:5/472: creat d0/db/dc/f8f x:0 0 0 2024-03-22T06:14:57.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.952+0000 7fb7d27fc700 1 --2- 172.21.15.6:0/2572838437 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb7bc06bb10 0x7fb7bc06dfc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80046c0 tx=0x7fb7c800b040 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:57.957 INFO:tasks.workunit.client.0.smithi006.stdout:5/473: dread d0/db/d11/f41 [4194304,4194304] 0 2024-03-22T06:14:57.957 INFO:tasks.workunit.client.0.smithi006.stdout:5/474: dread - d0/db/d16/d1c/d34/d35/d85/f89 zero size 2024-03-22T06:14:57.957 INFO:tasks.workunit.client.0.smithi006.stdout:5/475: truncate d0/db/f60 923732 0 2024-03-22T06:14:57.957 INFO:tasks.workunit.client.0.smithi006.stdout:5/476: write d0/db/d50/d58/f8c [5173471,25154] 0 2024-03-22T06:14:57.957 INFO:tasks.workunit.client.0.smithi006.stdout:5/477: chown d0/db/l62 212356 1 2024-03-22T06:14:57.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:57.957+0000 7fb7bb7fe700 1 -- 172.21.15.6:0/2572838437 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7fb7c40559f0 con 0x7fb7d40fbce0 2024-03-22T06:14:58.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:57 smithi117 ceph-mon[36514]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:58.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:57 smithi117 ceph-mon[36514]: pgmap v12: 65 pgs: 65 active+clean; 728 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 73 MiB/s rd, 107 MiB/s wr, 465 op/s 2024-03-22T06:14:58.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:57 smithi117 ceph-mon[36514]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:14:58.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:57 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/42104182' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:14:58.019 INFO:tasks.workunit.client.0.smithi006.stdout:7/376: symlink d5/d1e/d29/d59/l71 0 2024-03-22T06:14:58.019 INFO:tasks.workunit.client.0.smithi006.stdout:7/377: chown d5/d1e/d29/d1f/d2d/d41/f42 541365 1 2024-03-22T06:14:58.019 INFO:tasks.workunit.client.0.smithi006.stdout:7/378: creat d5/d1e/d29/f72 x:0 0 0 2024-03-22T06:14:58.019 INFO:tasks.workunit.client.0.smithi006.stdout:7/379: chown d5/c16 4 1 2024-03-22T06:14:58.026 INFO:tasks.workunit.client.0.smithi006.stdout:5/478: symlink d0/db/d16/d1c/d34/d35/d85/d5e/d66/l90 0 2024-03-22T06:14:58.026 INFO:tasks.workunit.client.0.smithi006.stdout:5/479: write d0/db/d11/f7a [349833,51406] 0 2024-03-22T06:14:58.044 INFO:tasks.workunit.client.0.smithi006.stdout:5/480: dwrite d0/db/d11/d61/f7d [0,4194304] 0 2024-03-22T06:14:58.044 INFO:tasks.workunit.client.0.smithi006.stdout:5/481: fsync d0/db/dc/f3b 0 2024-03-22T06:14:58.044 INFO:tasks.workunit.client.0.smithi006.stdout:5/482: chown d0/db/d50/f83 972271156 1 2024-03-22T06:14:58.044 INFO:tasks.workunit.client.0.smithi006.stdout:5/483: write d0/db/d50/d58/f6f [3471029,43006] 0 2024-03-22T06:14:58.044 INFO:tasks.workunit.client.0.smithi006.stdout:5/484: creat d0/db/d50/d58/f91 x:0 0 0 2024-03-22T06:14:58.047 INFO:tasks.workunit.client.0.smithi006.stdout:7/380: symlink d5/d1e/d63/d70/l73 0 2024-03-22T06:14:58.058 INFO:tasks.workunit.client.0.smithi006.stdout:7/381: link d5/d1e/d29/d1f/l57 d5/d1e/l74 0 2024-03-22T06:14:58.059 INFO:tasks.workunit.client.0.smithi006.stdout:7/382: unlink d5/c52 0 2024-03-22T06:14:58.063 INFO:tasks.workunit.client.0.smithi006.stdout:7/383: link d5/c34 d5/d1e/d63/c75 0 2024-03-22T06:14:58.065 INFO:tasks.workunit.client.0.smithi006.stdout:7/384: unlink d5/c16 0 2024-03-22T06:14:58.066 INFO:tasks.workunit.client.0.smithi006.stdout:7/385: chown d5/d1e/d29/d1f/d2d/c43 705492 1 2024-03-22T06:14:58.066 INFO:tasks.workunit.client.0.smithi006.stdout:7/386: write d5/d1e/d29/d10/f25 [2149227,65683] 0 2024-03-22T06:14:58.066 INFO:tasks.workunit.client.0.smithi006.stdout:7/387: chown d5/d1e/d29/d1f/d2d/d41/f44 710 1 2024-03-22T06:14:58.066 INFO:tasks.workunit.client.0.smithi006.stdout:7/388: creat d5/d1e/d3f/f76 x:0 0 0 2024-03-22T06:14:58.070 INFO:tasks.workunit.client.0.smithi006.stdout:7/389: rename d5/d1e/f31 to d5/d1e/d29/d59/f77 0 2024-03-22T06:14:58.072 INFO:tasks.workunit.client.0.smithi006.stdout:5/485: getdents d0/db/dc 0 2024-03-22T06:14:58.073 INFO:tasks.workunit.client.0.smithi006.stdout:5/486: write d0/db/d50/f80 [455537,37389] 0 2024-03-22T06:14:58.076 INFO:tasks.workunit.client.0.smithi006.stdout:7/390: getdents d5/d1e/d29/d1f/d2d/d41 0 2024-03-22T06:14:58.081 INFO:tasks.workunit.client.0.smithi006.stdout:7/391: rename d5/d1e/d29/l30 to d5/d1e/d29/d59/l78 0 2024-03-22T06:14:58.081 INFO:tasks.workunit.client.0.smithi006.stdout:7/392: truncate d5/f49 614632 0 2024-03-22T06:14:58.084 INFO:tasks.workunit.client.0.smithi006.stdout:7/393: link d5/d1e/d29/d59/d5c/c6a d5/d1e/d29/d59/d5c/c79 0 2024-03-22T06:14:58.084 INFO:tasks.workunit.client.0.smithi006.stdout:7/394: fsync d5/d1e/d29/d1f/d2d/d41/f69 0 2024-03-22T06:14:58.088 INFO:tasks.workunit.client.0.smithi006.stdout:5/487: dwrite d0/db/f29 [4194304,4194304] 0 2024-03-22T06:14:58.093 INFO:tasks.workunit.client.0.smithi006.stdout:5/488: dread d0/db/d11/f7a [0,4194304] 0 2024-03-22T06:14:58.095 INFO:tasks.workunit.client.0.smithi006.stdout:7/395: dread d5/d1e/d29/f2a [0,4194304] 0 2024-03-22T06:14:58.095 INFO:tasks.workunit.client.0.smithi006.stdout:7/396: dread - d5/d1e/d29/d1f/d2d/d41/f69 zero size 2024-03-22T06:14:58.096 INFO:tasks.workunit.client.0.smithi006.stdout:5/489: mkdir d0/db/d11/d52/d69/d92 0 2024-03-22T06:14:58.101 INFO:tasks.workunit.client.0.smithi006.stdout:7/397: rename d5/d1e/d29/d1f/d2d/d41/f42 to d5/d1e/d29/d59/f7a 0 2024-03-22T06:14:58.101 INFO:tasks.workunit.client.0.smithi006.stdout:7/398: dread - d5/d1e/d29/f72 zero size 2024-03-22T06:14:58.101 INFO:tasks.workunit.client.0.smithi006.stdout:7/399: write d5/d1e/d29/d59/f77 [1674961,44794] 0 2024-03-22T06:14:58.101 INFO:tasks.workunit.client.0.smithi006.stdout:7/400: stat d5/d1e/d29/d1f/d2d/d41/f69 0 2024-03-22T06:14:58.103 INFO:tasks.workunit.client.0.smithi006.stdout:5/490: symlink d0/db/d50/d82/l93 0 2024-03-22T06:14:58.109 INFO:tasks.workunit.client.0.smithi006.stdout:7/401: rename d5/fd to d5/d1e/d29/d1f/d2d/d41/f7b 0 2024-03-22T06:14:58.111 INFO:tasks.workunit.client.0.smithi006.stdout:5/491: symlink d0/db/d11/d52/d69/d92/l94 0 2024-03-22T06:14:58.113 INFO:tasks.workunit.client.0.smithi006.stdout:7/402: dread d5/d1e/d29/f5d [0,4194304] 0 2024-03-22T06:14:58.114 INFO:tasks.workunit.client.0.smithi006.stdout:7/403: chown d5/d1e/d29/d1f/d2d/f51 931 1 2024-03-22T06:14:58.115 INFO:tasks.workunit.client.0.smithi006.stdout:5/492: symlink d0/db/d50/d82/l95 0 2024-03-22T06:14:58.120 INFO:tasks.workunit.client.0.smithi006.stdout:5/493: rmdir d0/db/d16/d1c/d34/d35/d85/d5c 0 2024-03-22T06:14:58.120 INFO:tasks.workunit.client.0.smithi006.stdout:5/494: write d0/db/f6c [1104209,113496] 0 2024-03-22T06:14:58.120 INFO:tasks.workunit.client.0.smithi006.stdout:5/495: stat d0/db/d76/f81 0 2024-03-22T06:14:58.125 INFO:tasks.workunit.client.0.smithi006.stdout:1/330: sync 2024-03-22T06:14:58.125 INFO:tasks.workunit.client.0.smithi006.stdout:1/331: write d8/d9/d13/d24/d32/f41 [4934626,74970] 0 2024-03-22T06:14:58.125 INFO:tasks.workunit.client.0.smithi006.stdout:1/332: chown d8/d9/f50 1700239137 1 2024-03-22T06:14:58.132 INFO:tasks.workunit.client.0.smithi006.stdout:1/333: creat d8/d9/d13/d39/d60/f6e x:0 0 0 2024-03-22T06:14:58.151 INFO:tasks.workunit.client.0.smithi006.stdout:7/404: truncate d5/d1e/d29/d59/f77 1141 0 2024-03-22T06:14:58.152 INFO:tasks.workunit.client.0.smithi006.stdout:7/405: chown d5/d1e/d29/f72 21 1 2024-03-22T06:14:58.152 INFO:tasks.workunit.client.0.smithi006.stdout:7/406: stat d5/d1e/d29/d1f/d2d/f37 0 2024-03-22T06:14:58.154 INFO:tasks.workunit.client.0.smithi006.stdout:7/407: mknod d5/d1e/c7c 0 2024-03-22T06:14:58.154 INFO:tasks.workunit.client.0.smithi006.stdout:7/408: creat d5/d1e/d3f/f7d x:0 0 0 2024-03-22T06:14:58.161 INFO:tasks.workunit.client.0.smithi006.stdout:5/496: rename d0/db/d11 to d0/db/d50/d96 0 2024-03-22T06:14:58.163 INFO:tasks.workunit.client.0.smithi006.stdout:7/409: read d5/f9 [578747,17761] 0 2024-03-22T06:14:58.163 INFO:tasks.workunit.client.0.smithi006.stdout:7/410: stat d5/d1e/d3f 0 2024-03-22T06:14:58.163 INFO:tasks.workunit.client.0.smithi006.stdout:7/411: dread d5/f49 [0,4194304] 0 2024-03-22T06:14:58.167 INFO:tasks.workunit.client.0.smithi006.stdout:5/497: mkdir d0/db/d16/d1c/d97 0 2024-03-22T06:14:58.167 INFO:tasks.workunit.client.0.smithi006.stdout:5/498: chown d0/db/dc/c1f 168 1 2024-03-22T06:14:58.167 INFO:tasks.workunit.client.0.smithi006.stdout:5/499: chown d0/db/f1e 3613879 1 2024-03-22T06:14:58.170 INFO:tasks.workunit.client.0.smithi006.stdout:5/500: mkdir d0/d98 0 2024-03-22T06:14:58.174 INFO:tasks.workunit.client.0.smithi006.stdout:5/501: unlink d0/db/d16/d1c/d34/f88 0 2024-03-22T06:14:58.180 INFO:tasks.workunit.client.0.smithi006.stdout:7/412: dwrite d5/d1e/d3f/f5a [0,4194304] 0 2024-03-22T06:14:58.182 INFO:tasks.workunit.client.0.smithi006.stdout:5/502: rename d0/db/dc/d14 to d0/db/d99 0 2024-03-22T06:14:58.182 INFO:tasks.workunit.client.0.smithi006.stdout:5/503: creat d0/db/d76/f9a x:0 0 0 2024-03-22T06:14:58.186 INFO:tasks.workunit.client.0.smithi006.stdout:7/413: getdents d5/d1e/d29/d59/d5c 0 2024-03-22T06:14:58.189 INFO:tasks.workunit.client.0.smithi006.stdout:5/504: creat d0/db/d16/d1c/d34/d35/d85/d5e/d66/f9b x:0 0 0 2024-03-22T06:14:58.191 INFO:tasks.workunit.client.0.smithi006.stdout:7/414: link d5/d1e/d3f/f76 d5/d1e/d29/d10/f7e 0 2024-03-22T06:14:58.193 INFO:tasks.workunit.client.0.smithi006.stdout:5/505: getdents d0/db/d16/d1c/d34/d35/d85/d54/d65/d77 0 2024-03-22T06:14:58.193 INFO:tasks.workunit.client.0.smithi006.stdout:5/506: write d0/db/d16/d1c/d34/d4d/f6b [835367,1820] 0 2024-03-22T06:14:58.193 INFO:tasks.workunit.client.0.smithi006.stdout:5/507: write d0/db/d99/f45 [2831413,117450] 0 2024-03-22T06:14:58.193 INFO:tasks.workunit.client.0.smithi006.stdout:5/508: write d0/db/dc/f8f [973734,107996] 0 2024-03-22T06:14:58.195 INFO:tasks.workunit.client.0.smithi006.stdout:5/509: mknod d0/db/d16/d70/c9c 0 2024-03-22T06:14:58.195 INFO:tasks.workunit.client.0.smithi006.stdout:5/510: write d0/db/d16/d43/f67 [489611,45139] 0 2024-03-22T06:14:58.195 INFO:tasks.workunit.client.0.smithi006.stdout:5/511: stat d0/db/d50/c7f 0 2024-03-22T06:14:58.196 INFO:tasks.workunit.client.0.smithi006.stdout:5/512: mkdir d0/db/d16/d1c/d34/d35/d5d/d9d 0 2024-03-22T06:14:58.197 INFO:tasks.workunit.client.0.smithi006.stdout:5/513: readlink d0/db/l2f 0 2024-03-22T06:14:58.197 INFO:tasks.workunit.client.0.smithi006.stdout:5/514: dread - d0/db/d16/d1c/d34/d35/d85/f89 zero size 2024-03-22T06:14:58.198 INFO:tasks.workunit.client.0.smithi006.stdout:5/515: getdents d0/db/d16/d43 0 2024-03-22T06:14:58.201 INFO:tasks.workunit.client.0.smithi006.stdout:5/516: truncate d0/db/d16/d1c/f40 1405992 0 2024-03-22T06:14:58.201 INFO:tasks.workunit.client.0.smithi006.stdout:5/517: fdatasync d0/db/f29 0 2024-03-22T06:14:58.201 INFO:tasks.workunit.client.0.smithi006.stdout:5/518: truncate d0/db/dc/f8f 1368480 0 2024-03-22T06:14:58.203 INFO:tasks.workunit.client.0.smithi006.stdout:7/415: write d5/f56 [3091633,72917] 0 2024-03-22T06:14:58.203 INFO:tasks.workunit.client.0.smithi006.stdout:5/519: write d0/db/f6c [249005,1276] 0 2024-03-22T06:14:58.203 INFO:tasks.workunit.client.0.smithi006.stdout:5/520: write d0/db/d50/d96/d52/d69/f6a [112521,2562] 0 2024-03-22T06:14:58.205 INFO:tasks.workunit.client.0.smithi006.stdout:7/416: link d5/d1e/d29/d1f/l57 d5/l7f 0 2024-03-22T06:14:58.206 INFO:tasks.workunit.client.0.smithi006.stdout:5/521: symlink d0/db/d16/d1c/d34/d35/d5d/d9d/l9e 0 2024-03-22T06:14:58.206 INFO:tasks.workunit.client.0.smithi006.stdout:5/522: chown d0/db/d16/d1c/d34/d35/d85/d54/d65/d77/f86 7 1 2024-03-22T06:14:58.209 INFO:tasks.workunit.client.0.smithi006.stdout:7/417: mkdir d5/d1e/d63/d70/d80 0 2024-03-22T06:14:58.210 INFO:tasks.workunit.client.0.smithi006.stdout:7/418: creat d5/d1e/d3f/f81 x:0 0 0 2024-03-22T06:14:58.211 INFO:tasks.workunit.client.0.smithi006.stdout:5/523: mknod d0/db/d50/d96/d52/d69/c9f 0 2024-03-22T06:14:58.222 INFO:tasks.workunit.client.0.smithi006.stdout:7/419: mknod d5/d1e/d29/d1f/d2d/c82 0 2024-03-22T06:14:58.222 INFO:tasks.workunit.client.0.smithi006.stdout:7/420: write f0 [239263,60618] 0 2024-03-22T06:14:58.225 INFO:tasks.workunit.client.0.smithi006.stdout:5/524: mknod d0/db/d16/d1c/d34/d35/d85/d54/d65/ca0 0 2024-03-22T06:14:58.228 INFO:tasks.workunit.client.0.smithi006.stdout:7/421: creat d5/d1e/d29/d59/f83 x:0 0 0 2024-03-22T06:14:58.230 INFO:tasks.workunit.client.0.smithi006.stdout:5/525: dread d0/db/d99/f3f [0,4194304] 0 2024-03-22T06:14:58.230 INFO:tasks.workunit.client.0.smithi006.stdout:5/526: fdatasync d0/db/d16/d1c/d34/d35/d85/f47 0 2024-03-22T06:14:58.235 INFO:tasks.workunit.client.0.smithi006.stdout:5/527: rmdir d0/db/d16/d1c/d34/d35/d85/d54/d65 39 2024-03-22T06:14:58.248 INFO:tasks.workunit.client.0.smithi006.stdout:7/422: dread d5/d1e/d3f/f4b [0,4194304] 0 2024-03-22T06:14:58.248 INFO:tasks.workunit.client.0.smithi006.stdout:7/423: fdatasync d5/d1e/d29/d1f/f3e 0 2024-03-22T06:14:58.251 INFO:tasks.workunit.client.0.smithi006.stdout:7/424: mkdir d5/d1e/d29/d1f/d2d/d55/d84 0 2024-03-22T06:14:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.252+0000 7fb7d92c7700 1 -- 172.21.15.6:0/2572838437 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7d410e7b0 con 0x7fb7bc06bb10 2024-03-22T06:14:58.254 INFO:tasks.workunit.client.0.smithi006.stdout:7/425: rename d5/d1e/d63/d70 to d5/d1e/d63/d85 0 2024-03-22T06:14:58.254 INFO:tasks.workunit.client.0.smithi006.stdout:7/426: truncate d5/d1e/d29/d1f/d2d/f37 393119 0 2024-03-22T06:14:58.254 INFO:tasks.workunit.client.0.smithi006.stdout:7/427: fsync d5/d1e/d3f/f4b 0 2024-03-22T06:14:58.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.254+0000 7fb7bb7fe700 1 -- 172.21.15.6:0/2572838437 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fb7d410e7b0 con 0x7fb7bc06bb10 2024-03-22T06:14:58.255 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:14:58.255 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:14:58.255 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:14:58.255 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:14:58.255 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [], 2024-03-22T06:14:58.255 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "1/21 daemons upgraded", 2024-03-22T06:14:58.255 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "", 2024-03-22T06:14:58.255 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:14:58.255 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:14:58.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.256+0000 7fb7d92c7700 1 -- 172.21.15.6:0/2572838437 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb7bc06bb10 msgr2=0x7fb7bc06dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:58.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.256+0000 7fb7d92c7700 1 --2- 172.21.15.6:0/2572838437 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb7bc06bb10 0x7fb7bc06dfc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80046c0 tx=0x7fb7c800b040 comp rx=0 tx=0).stop 2024-03-22T06:14:58.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.256+0000 7fb7d92c7700 1 -- 172.21.15.6:0/2572838437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7d40fbce0 msgr2=0x7fb7d410c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:58.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.256+0000 7fb7d92c7700 1 --2- 172.21.15.6:0/2572838437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7d40fbce0 0x7fb7d410c8c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4008cd0 tx=0x7fb7c4009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:58.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.256+0000 7fb7d92c7700 1 -- 172.21.15.6:0/2572838437 shutdown_connections 2024-03-22T06:14:58.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.256+0000 7fb7d92c7700 1 --2- 172.21.15.6:0/2572838437 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb7bc06bb10 0x7fb7bc06dfc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:58.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.256+0000 7fb7d92c7700 1 --2- 172.21.15.6:0/2572838437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7d40fbce0 0x7fb7d410c8c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:58.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.256+0000 7fb7d92c7700 1 --2- 172.21.15.6:0/2572838437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7d40fcf90 0x7fb7d410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:58.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.256+0000 7fb7d92c7700 1 -- 172.21.15.6:0/2572838437 >> 172.21.15.6:0/2572838437 conn(0x7fb7d40f7480 msgr2=0x7fb7d40f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:58.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.256+0000 7fb7d92c7700 1 -- 172.21.15.6:0/2572838437 shutdown_connections 2024-03-22T06:14:58.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.256+0000 7fb7d92c7700 1 -- 172.21.15.6:0/2572838437 wait complete. 2024-03-22T06:14:58.263 INFO:tasks.workunit.client.0.smithi006.stdout:7/428: mknod d5/d1e/d29/d1f/d2d/d61/c86 0 2024-03-22T06:14:58.264 INFO:tasks.workunit.client.0.smithi006.stdout:7/429: fsync d5/f56 0 2024-03-22T06:14:58.264 INFO:tasks.workunit.client.0.smithi006.stdout:7/430: write d5/d1e/d29/d1f/f38 [1075594,73670] 0 2024-03-22T06:14:58.264 INFO:tasks.workunit.client.0.smithi006.stdout:7/431: write d5/d1e/d3f/f7d [38484,90856] 0 2024-03-22T06:14:58.264 INFO:tasks.workunit.client.0.smithi006.stdout:7/432: rename d5/d1e/d29/d59/d5c/l60 to d5/d1e/d3f/l87 0 2024-03-22T06:14:58.264 INFO:tasks.workunit.client.0.smithi006.stdout:7/433: write d5/d1e/d29/d10/f25 [4791669,75909] 0 2024-03-22T06:14:58.264 INFO:tasks.workunit.client.0.smithi006.stdout:7/434: truncate d5/d1e/d29/d10/f3d 4338310 0 2024-03-22T06:14:58.264 INFO:tasks.workunit.client.0.smithi006.stdout:7/435: chown d5/d1e/d29/d1f/d2d/d55/d84 110637605 1 2024-03-22T06:14:58.264 INFO:tasks.workunit.client.0.smithi006.stdout:7/436: chown d5/f1d 2481224 1 2024-03-22T06:14:58.265 INFO:tasks.workunit.client.0.smithi006.stdout:7/437: mknod d5/d1e/d29/d59/c88 0 2024-03-22T06:14:58.267 INFO:tasks.workunit.client.0.smithi006.stdout:7/438: rename d5/d1e/c62 to d5/d1e/d29/d10/d6f/c89 0 2024-03-22T06:14:58.267 INFO:tasks.workunit.client.0.smithi006.stdout:7/439: write f2 [1340535,94647] 0 2024-03-22T06:14:58.268 INFO:tasks.workunit.client.0.smithi006.stdout:5/528: dwrite d0/db/d16/d1c/d34/f73 [0,4194304] 0 2024-03-22T06:14:58.270 INFO:tasks.workunit.client.0.smithi006.stdout:7/440: symlink d5/d1e/d29/d66/l8a 0 2024-03-22T06:14:58.270 INFO:tasks.workunit.client.0.smithi006.stdout:7/441: fdatasync d5/f28 0 2024-03-22T06:14:58.276 INFO:tasks.workunit.client.0.smithi006.stdout:7/442: symlink d5/d1e/d29/d66/l8b 0 2024-03-22T06:14:58.279 INFO:tasks.workunit.client.0.smithi006.stdout:7/443: truncate d5/d1e/d3f/f4b 2867412 0 2024-03-22T06:14:58.281 INFO:tasks.workunit.client.0.smithi006.stdout:7/444: read d5/d1e/d29/d1f/d2d/f51 [56554,36946] 0 2024-03-22T06:14:58.282 INFO:tasks.workunit.client.0.smithi006.stdout:5/529: dwrite d0/db/d50/d96/d52/d69/f6a [0,4194304] 0 2024-03-22T06:14:58.283 INFO:tasks.workunit.client.0.smithi006.stdout:7/445: creat d5/d1e/d29/d1f/d2d/d55/f8c x:0 0 0 2024-03-22T06:14:58.289 INFO:tasks.workunit.client.0.smithi006.stdout:5/530: unlink d0/db/d50/d96/f7a 0 2024-03-22T06:14:58.292 INFO:tasks.workunit.client.0.smithi006.stdout:7/446: mknod d5/d1e/d29/d59/c8d 0 2024-03-22T06:14:58.292 INFO:tasks.workunit.client.0.smithi006.stdout:7/447: creat d5/d1e/d29/d59/f8e x:0 0 0 2024-03-22T06:14:58.292 INFO:tasks.workunit.client.0.smithi006.stdout:7/448: write d5/d1e/d29/d1f/f38 [1057864,105348] 0 2024-03-22T06:14:58.304 INFO:tasks.workunit.client.0.smithi006.stdout:5/531: truncate d0/db/d99/f45 2983897 0 2024-03-22T06:14:58.308 INFO:tasks.workunit.client.0.smithi006.stdout:7/449: rename d5/d1e/d3f/l87 to d5/d1e/d63/d85/d80/l8f 0 2024-03-22T06:14:58.320 INFO:tasks.workunit.client.0.smithi006.stdout:5/532: mknod d0/db/d50/ca1 0 2024-03-22T06:14:58.321 INFO:tasks.workunit.client.0.smithi006.stdout:5/533: stat d0/db/d16/f46 0 2024-03-22T06:14:58.321 INFO:tasks.workunit.client.0.smithi006.stdout:5/534: write d0/db/d50/d58/f6f [2759012,30181] 0 2024-03-22T06:14:58.335 INFO:tasks.workunit.client.0.smithi006.stdout:7/450: getdents d5/d1e/d29/d59 0 2024-03-22T06:14:58.336 INFO:tasks.workunit.client.0.smithi006.stdout:7/451: chown d5/d1e/d3f/f4b 9424064 1 2024-03-22T06:14:58.336 INFO:tasks.workunit.client.0.smithi006.stdout:7/452: creat d5/d1e/d3f/f90 x:0 0 0 2024-03-22T06:14:58.336 INFO:tasks.workunit.client.0.smithi006.stdout:7/453: fdatasync d5/fe 0 2024-03-22T06:14:58.336 INFO:tasks.workunit.client.0.smithi006.stdout:7/454: write f2 [1989694,88762] 0 2024-03-22T06:14:58.351 INFO:tasks.workunit.client.0.smithi006.stdout:7/455: dwrite d5/d1e/d29/d10/f3d [0,4194304] 0 2024-03-22T06:14:58.352 INFO:tasks.workunit.client.0.smithi006.stdout:7/456: chown d5/d1e/d3f/f5a 3248113 1 2024-03-22T06:14:58.352 INFO:tasks.workunit.client.0.smithi006.stdout:7/457: getdents d5/d1e/d29/d66 0 2024-03-22T06:14:58.354 INFO:tasks.workunit.client.0.smithi006.stdout:7/458: mkdir d5/d1e/d29/d66/d91 0 2024-03-22T06:14:58.361 INFO:tasks.workunit.client.0.smithi006.stdout:5/535: truncate d0/db/d99/f45 2201144 0 2024-03-22T06:14:58.363 INFO:tasks.workunit.client.0.smithi006.stdout:5/536: rename d0/f39 to d0/db/d16/d1c/d97/fa2 0 2024-03-22T06:14:58.384 INFO:tasks.workunit.client.0.smithi006.stdout:7/459: dwrite d5/d1e/d29/f54 [0,4194304] 0 2024-03-22T06:14:58.384 INFO:tasks.workunit.client.0.smithi006.stdout:5/537: dwrite d0/db/d16/f46 [0,4194304] 0 2024-03-22T06:14:58.393 INFO:tasks.workunit.client.0.smithi006.stdout:7/460: dread d5/d1e/d29/d10/f25 [4194304,4194304] 0 2024-03-22T06:14:58.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.395+0000 7fa2e6661700 1 -- 172.21.15.6:0/1748376680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e00fbcd0 msgr2=0x7fa2e00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:58.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.395+0000 7fa2e6661700 1 --2- 172.21.15.6:0/1748376680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e00fbcd0 0x7fa2e00fc0e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0004940 tx=0x7fa2d0009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:58.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.395+0000 7fa2e6661700 1 -- 172.21.15.6:0/1748376680 shutdown_connections 2024-03-22T06:14:58.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.395+0000 7fa2e6661700 1 --2- 172.21.15.6:0/1748376680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e00fcf80 0x7fa2e00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:58.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.395+0000 7fa2e6661700 1 --2- 172.21.15.6:0/1748376680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e00fbcd0 0x7fa2e00fc0e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:58.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.395+0000 7fa2e6661700 1 -- 172.21.15.6:0/1748376680 >> 172.21.15.6:0/1748376680 conn(0x7fa2e00f7430 msgr2=0x7fa2e00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:58.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.395+0000 7fa2e6661700 1 -- 172.21.15.6:0/1748376680 shutdown_connections 2024-03-22T06:14:58.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.395+0000 7fa2e6661700 1 -- 172.21.15.6:0/1748376680 wait complete. 2024-03-22T06:14:58.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.396+0000 7fa2e6661700 1 Processor -- start 2024-03-22T06:14:58.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.396+0000 7fa2e6661700 1 -- start start 2024-03-22T06:14:58.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2e6661700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e00fbcd0 0x7fa2e010da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:58.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2e6661700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e00fcf80 0x7fa2e010dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:58.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2e6661700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e010e660 con 0x7fa2e00fcf80 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2e6661700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e010ee00 con 0x7fa2e00fbcd0 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e00fcf80 0x7fa2e010dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e00fcf80 0x7fa2e010dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33204/0 (socket says 172.21.15.6:33204) 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2df7fe700 1 -- 172.21.15.6:0/2835634599 learned_addr learned my addr 172.21.15.6:0/2835634599 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2dffff700 1 --2- 172.21.15.6:0/2835634599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e00fbcd0 0x7fa2e010da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2df7fe700 1 -- 172.21.15.6:0/2835634599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e00fbcd0 msgr2=0x7fa2e010da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2df7fe700 1 --2- 172.21.15.6:0/2835634599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e00fbcd0 0x7fa2e010da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2df7fe700 1 -- 172.21.15.6:0/2835634599 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0005040 con 0x7fa2e00fcf80 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2df7fe700 1 --2- 172.21.15.6:0/2835634599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e00fcf80 0x7fa2e010dfd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400b790 tx=0x7fa2d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.397+0000 7fa2dcff9700 1 -- 172.21.15.6:0/2835634599 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d400f770 con 0x7fa2e00fcf80 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.398+0000 7fa2dcff9700 1 -- 172.21.15.6:0/2835634599 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2d400fda0 con 0x7fa2e00fcf80 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.398+0000 7fa2dcff9700 1 -- 172.21.15.6:0/2835634599 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d400d310 con 0x7fa2e00fcf80 2024-03-22T06:14:58.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.398+0000 7fa2e6661700 1 -- 172.21.15.6:0/2835634599 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e010f0e0 con 0x7fa2e00fcf80 2024-03-22T06:14:58.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.398+0000 7fa2e6661700 1 -- 172.21.15.6:0/2835634599 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e010f6a0 con 0x7fa2e00fcf80 2024-03-22T06:14:58.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.399+0000 7fa2e6661700 1 -- 172.21.15.6:0/2835634599 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e0101b10 con 0x7fa2e00fcf80 2024-03-22T06:14:58.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.399+0000 7fa2dcff9700 1 -- 172.21.15.6:0/2835634599 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7fa2d400f8d0 con 0x7fa2e00fcf80 2024-03-22T06:14:58.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.400+0000 7fa2dcff9700 1 --2- 172.21.15.6:0/2835634599 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fa2c806bb10 0x7fa2c806dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:14:58.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.400+0000 7fa2dcff9700 1 -- 172.21.15.6:0/2835634599 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fa2d408a710 con 0x7fa2e00fcf80 2024-03-22T06:14:58.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.400+0000 7fa2dffff700 1 --2- 172.21.15.6:0/2835634599 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fa2c806bb10 0x7fa2c806dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:14:58.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.401+0000 7fa2dffff700 1 --2- 172.21.15.6:0/2835634599 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fa2c806bb10 0x7fa2c806dfc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000b3b0 tx=0x7fa2d0009e30 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:14:58.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.404+0000 7fa2dcff9700 1 -- 172.21.15.6:0/2835634599 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7fa2d4057b70 con 0x7fa2e00fcf80 2024-03-22T06:14:58.424 INFO:tasks.workunit.client.0.smithi006.stdout:0/248: sync 2024-03-22T06:14:58.424 INFO:tasks.workunit.client.0.smithi006.stdout:1/334: sync 2024-03-22T06:14:58.424 INFO:tasks.workunit.client.0.smithi006.stdout:2/393: sync 2024-03-22T06:14:58.424 INFO:tasks.workunit.client.0.smithi006.stdout:2/394: fsync d13/d20/d26/d19/f3e 0 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:4/415: sync 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:4/416: chown d6/d13 44935 1 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:3/378: sync 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:9/312: sync 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:3/379: dread - d0/d8/d1c/d3e/d68/f6d zero size 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:3/380: truncate d0/d23/d27/f47 740363 0 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:9/313: write d8/d10/d28/f44 [5175484,45613] 0 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:6/357: sync 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:8/524: sync 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:8/525: write d9/db/d12/d7e/f8b [996482,82420] 0 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:8/526: fsync d9/db/f2e 0 2024-03-22T06:14:58.425 INFO:tasks.workunit.client.0.smithi006.stdout:8/527: read - d9/d14/d44/f8e zero size 2024-03-22T06:14:58.429 INFO:tasks.workunit.client.0.smithi006.stdout:7/461: dwrite d5/d1e/d3f/f5a [4194304,4194304] 0 2024-03-22T06:14:58.429 INFO:tasks.workunit.client.0.smithi006.stdout:7/462: stat d5/d1e/d29/d1f/d2d/d55/f8c 0 2024-03-22T06:14:58.429 INFO:tasks.workunit.client.0.smithi006.stdout:7/463: read - d5/d1e/d3f/f76 zero size 2024-03-22T06:14:58.431 INFO:tasks.workunit.client.0.smithi006.stdout:0/249: write db/d10/f2f [3432772,47816] 0 2024-03-22T06:14:58.431 INFO:tasks.workunit.client.0.smithi006.stdout:0/250: write db/d10/d2a/f51 [188274,64143] 0 2024-03-22T06:14:58.431 INFO:tasks.workunit.client.0.smithi006.stdout:0/251: fsync db/d10/d33/f4d 0 2024-03-22T06:14:58.432 INFO:tasks.workunit.client.0.smithi006.stdout:2/395: rename d13/d34/d4d/d4e/f83 to d13/d53/d6b/f97 0 2024-03-22T06:14:58.435 INFO:tasks.workunit.client.0.smithi006.stdout:3/381: mkdir d0/d8/d1c/d49/d79 0 2024-03-22T06:14:58.437 INFO:tasks.workunit.client.0.smithi006.stdout:9/314: mknod d8/d10/d47/c6e 0 2024-03-22T06:14:58.438 INFO:tasks.workunit.client.0.smithi006.stdout:9/315: chown d8/d10/d20/d45/d49/l5b 48 1 2024-03-22T06:14:58.438 INFO:tasks.workunit.client.0.smithi006.stdout:9/316: read d8/d10/d28/f3d [1247721,127649] 0 2024-03-22T06:14:58.438 INFO:tasks.workunit.client.0.smithi006.stdout:9/317: chown f4 883 1 2024-03-22T06:14:58.439 INFO:tasks.workunit.client.0.smithi006.stdout:6/358: creat d6/d1b/d2c/d4a/f84 x:0 0 0 2024-03-22T06:14:58.441 INFO:tasks.workunit.client.0.smithi006.stdout:4/417: dwrite d6/f12 [4194304,4194304] 0 2024-03-22T06:14:58.442 INFO:tasks.workunit.client.0.smithi006.stdout:4/418: truncate d6/d1b/d20/d21/d77/f8b 703636 0 2024-03-22T06:14:58.447 INFO:tasks.workunit.client.0.smithi006.stdout:0/252: mknod db/dc/d38/c58 0 2024-03-22T06:14:58.450 INFO:tasks.workunit.client.0.smithi006.stdout:8/528: rename d9/db/d12/d66/d6c/d71/l7d to d9/db/d37/lc5 0 2024-03-22T06:14:58.453 INFO:tasks.workunit.client.0.smithi006.stdout:2/396: symlink d13/d53/d29/d6c/l98 0 2024-03-22T06:14:58.456 INFO:tasks.workunit.client.0.smithi006.stdout:3/382: link d0/d8/d6a/c71 d0/d8/c7a 0 2024-03-22T06:14:58.457 INFO:tasks.workunit.client.0.smithi006.stdout:3/383: creat d0/d23/d4c/f7b x:0 0 0 2024-03-22T06:14:58.458 INFO:tasks.workunit.client.0.smithi006.stdout:4/419: dwrite d6/d1b/f47 [0,4194304] 0 2024-03-22T06:14:58.458 INFO:tasks.workunit.client.0.smithi006.stdout:3/384: dread d0/d8/d1c/f2c [0,4194304] 0 2024-03-22T06:14:58.459 INFO:tasks.workunit.client.0.smithi006.stdout:9/318: truncate d8/fa 97401 0 2024-03-22T06:14:58.459 INFO:tasks.workunit.client.0.smithi006.stdout:9/319: fdatasync f1 0 2024-03-22T06:14:58.459 INFO:tasks.workunit.client.0.smithi006.stdout:9/320: chown d8/d10/d11/f59 69231 1 2024-03-22T06:14:58.459 INFO:tasks.workunit.client.0.smithi006.stdout:9/321: fdatasync d8/d10/d20/f2c 0 2024-03-22T06:14:58.460 INFO:tasks.workunit.client.0.smithi006.stdout:9/322: creat d8/d10/d28/d48/d50/f6f x:0 0 0 2024-03-22T06:14:58.461 INFO:tasks.workunit.client.0.smithi006.stdout:6/359: mknod d6/d1d/c85 0 2024-03-22T06:14:58.466 INFO:tasks.workunit.client.0.smithi006.stdout:0/253: mknod db/d3d/c59 0 2024-03-22T06:14:58.467 INFO:tasks.workunit.client.0.smithi006.stdout:7/464: rename d5/d1e/d29/d10/l2e to d5/d1e/d63/d85/l92 0 2024-03-22T06:14:58.467 INFO:tasks.workunit.client.0.smithi006.stdout:8/529: mknod d9/d14/d47/d94/da3/cc6 0 2024-03-22T06:14:58.477 INFO:tasks.workunit.client.0.smithi006.stdout:6/360: dwrite d6/d1b/d2c/f59 [0,4194304] 0 2024-03-22T06:14:58.479 INFO:tasks.workunit.client.0.smithi006.stdout:4/420: creat d6/d1b/d20/d6b/f8c x:0 0 0 2024-03-22T06:14:58.479 INFO:tasks.workunit.client.0.smithi006.stdout:4/421: write d6/fa [1336103,100641] 0 2024-03-22T06:14:58.479 INFO:tasks.workunit.client.0.smithi006.stdout:4/422: truncate d6/d1b/d46/f58 1550774 0 2024-03-22T06:14:58.485 INFO:tasks.workunit.client.0.smithi006.stdout:3/385: creat d0/d4/d3a/f7c x:0 0 0 2024-03-22T06:14:58.485 INFO:tasks.workunit.client.0.smithi006.stdout:3/386: truncate d0/f56 2128780 0 2024-03-22T06:14:58.488 INFO:tasks.workunit.client.0.smithi006.stdout:4/423: dread f1 [4194304,4194304] 0 2024-03-22T06:14:58.489 INFO:tasks.workunit.client.0.smithi006.stdout:4/424: readlink d6/l61 0 2024-03-22T06:14:58.489 INFO:tasks.workunit.client.0.smithi006.stdout:4/425: chown d6 0 1 2024-03-22T06:14:58.489 INFO:tasks.workunit.client.0.smithi006.stdout:4/426: write d6/f59 [523104,28346] 0 2024-03-22T06:14:58.489 INFO:tasks.workunit.client.0.smithi006.stdout:0/254: mknod db/d3d/c5a 0 2024-03-22T06:14:58.492 INFO:tasks.workunit.client.0.smithi006.stdout:6/361: dwrite d6/d1b/f48 [4194304,4194304] 0 2024-03-22T06:14:58.492 INFO:tasks.workunit.client.0.smithi006.stdout:6/362: dread - d6/d1d/d5c/d81/f5d zero size 2024-03-22T06:14:58.493 INFO:tasks.workunit.client.0.smithi006.stdout:6/363: readlink d6/d1b/d2c/d4a/l5b 0 2024-03-22T06:14:58.493 INFO:tasks.workunit.client.0.smithi006.stdout:6/364: chown d6/d1b 92 1 2024-03-22T06:14:58.493 INFO:tasks.workunit.client.0.smithi006.stdout:9/323: rename d8/d10/d23/d3e/f65 to d8/d1f/d34/f70 0 2024-03-22T06:14:58.493 INFO:tasks.workunit.client.0.smithi006.stdout:7/465: mknod d5/c93 0 2024-03-22T06:14:58.502 INFO:tasks.workunit.client.0.smithi006.stdout:3/387: creat d0/d8/d1c/d3e/d5f/f7d x:0 0 0 2024-03-22T06:14:58.503 INFO:tasks.workunit.client.0.smithi006.stdout:4/427: mkdir d6/d1b/d20/d21/d2a/d43/d8d 0 2024-03-22T06:14:58.510 INFO:tasks.workunit.client.0.smithi006.stdout:0/255: rename db/dc/d37/c3e to db/d10/d2a/d41/d4f/c5b 0 2024-03-22T06:14:58.511 INFO:tasks.workunit.client.0.smithi006.stdout:4/428: dread f2 [0,4194304] 0 2024-03-22T06:14:58.512 INFO:tasks.workunit.client.0.smithi006.stdout:9/324: unlink d8/d10/d28/c29 0 2024-03-22T06:14:58.512 INFO:tasks.workunit.client.0.smithi006.stdout:9/325: fsync d8/d10/d11/f60 0 2024-03-22T06:14:58.512 INFO:tasks.workunit.client.0.smithi006.stdout:7/466: mkdir d5/d1e/d3f/d94 0 2024-03-22T06:14:58.514 INFO:tasks.workunit.client.0.smithi006.stdout:9/326: write d8/fc [848036,8668] 0 2024-03-22T06:14:58.514 INFO:tasks.workunit.client.0.smithi006.stdout:9/327: chown d8/d10/d28/f57 216991822 1 2024-03-22T06:14:58.514 INFO:tasks.workunit.client.0.smithi006.stdout:9/328: write d8/d1f/f68 [747012,97610] 0 2024-03-22T06:14:58.514 INFO:tasks.workunit.client.0.smithi006.stdout:8/530: getdents d9/db/d12 0 2024-03-22T06:14:58.515 INFO:tasks.workunit.client.0.smithi006.stdout:8/531: creat d9/fc7 x:0 0 0 2024-03-22T06:14:58.515 INFO:tasks.workunit.client.0.smithi006.stdout:8/532: creat d9/db/d35/da9/fc8 x:0 0 0 2024-03-22T06:14:58.517 INFO:tasks.workunit.client.0.smithi006.stdout:6/365: symlink d6/d77/l86 0 2024-03-22T06:14:58.517 INFO:tasks.workunit.client.0.smithi006.stdout:6/366: readlink d6/d1d/d5c/d81/l68 0 2024-03-22T06:14:58.521 INFO:tasks.workunit.client.0.smithi006.stdout:0/256: symlink db/d10/d2a/d41/d4f/l5c 0 2024-03-22T06:14:58.521 INFO:tasks.workunit.client.0.smithi006.stdout:0/257: chown db/d10/d33 3 1 2024-03-22T06:14:58.524 INFO:tasks.workunit.client.0.smithi006.stdout:4/429: creat d6/d1b/d20/d21/d2a/d8a/f8e x:0 0 0 2024-03-22T06:14:58.525 INFO:tasks.workunit.client.0.smithi006.stdout:7/467: truncate d5/d1e/d29/f54 1844603 0 2024-03-22T06:14:58.526 INFO:tasks.workunit.client.0.smithi006.stdout:4/430: write d6/f4a [764066,43746] 0 2024-03-22T06:14:58.531 INFO:tasks.workunit.client.0.smithi006.stdout:7/468: write d5/d1e/d29/d1f/d2d/f67 [1369564,72659] 0 2024-03-22T06:14:58.531 INFO:tasks.workunit.client.0.smithi006.stdout:7/469: chown d5/d1e/d29/d59/f8e 64732 1 2024-03-22T06:14:58.531 INFO:tasks.workunit.client.0.smithi006.stdout:7/470: dread - d5/d1e/d29/d59/f8e zero size 2024-03-22T06:14:58.535 INFO:tasks.workunit.client.0.smithi006.stdout:0/258: unlink db/f30 0 2024-03-22T06:14:58.536 INFO:tasks.workunit.client.0.smithi006.stdout:6/367: dwrite d6/d1b/d2c/f7d [0,4194304] 0 2024-03-22T06:14:58.536 INFO:tasks.workunit.client.0.smithi006.stdout:7/471: write d5/d1e/d29/f2a [1428342,96941] 0 2024-03-22T06:14:58.536 INFO:tasks.workunit.client.0.smithi006.stdout:7/472: fdatasync d5/d1e/d29/d1f/d2d/f51 0 2024-03-22T06:14:58.540 INFO:tasks.workunit.client.0.smithi006.stdout:4/431: truncate d6/fb 3245889 0 2024-03-22T06:14:58.543 INFO:tasks.workunit.client.0.smithi006.stdout:8/533: rmdir d9/db/d12/d1e 39 2024-03-22T06:14:58.551 INFO:tasks.workunit.client.0.smithi006.stdout:7/473: dwrite d5/d1e/d3f/f6c [0,4194304] 0 2024-03-22T06:14:58.552 INFO:tasks.workunit.client.0.smithi006.stdout:8/534: dread d9/db/d35/f6f [0,4194304] 0 2024-03-22T06:14:58.552 INFO:tasks.workunit.client.0.smithi006.stdout:8/535: chown d9/db/dc/de/d3e/f3f 3263879 1 2024-03-22T06:14:58.556 INFO:tasks.workunit.client.0.smithi006.stdout:4/432: mkdir d6/d1b/d20/d21/d2a/d43/d8d/d8f 0 2024-03-22T06:14:58.561 INFO:tasks.workunit.client.0.smithi006.stdout:8/536: symlink d9/db/dc/lc9 0 2024-03-22T06:14:58.562 INFO:tasks.workunit.client.0.smithi006.stdout:6/368: truncate d6/d6f/f6e 3032366 0 2024-03-22T06:14:58.569 INFO:tasks.workunit.client.0.smithi006.stdout:3/388: rename d0/d8/d9/d24 to d0/d7e 0 2024-03-22T06:14:58.573 INFO:tasks.workunit.client.0.smithi006.stdout:4/433: creat d6/d1b/d20/d21/d77/f90 x:0 0 0 2024-03-22T06:14:58.574 INFO:tasks.workunit.client.0.smithi006.stdout:8/537: mkdir d9/db/d37/d76/dca 0 2024-03-22T06:14:58.577 INFO:tasks.workunit.client.0.smithi006.stdout:9/329: rename d8/d10/d28/d39/l6a to d8/d10/d23/d3e/l71 0 2024-03-22T06:14:58.579 INFO:tasks.workunit.client.0.smithi006.stdout:3/389: link d0/d4/d31/f3b d0/d23/f7f 0 2024-03-22T06:14:58.580 INFO:tasks.workunit.client.0.smithi006.stdout:9/330: dread d8/d1f/f68 [0,4194304] 0 2024-03-22T06:14:58.580 INFO:tasks.workunit.client.0.smithi006.stdout:9/331: dread d8/d10/d28/f35 [0,4194304] 0 2024-03-22T06:14:58.580 INFO:tasks.workunit.client.0.smithi006.stdout:9/332: creat d8/d10/f72 x:0 0 0 2024-03-22T06:14:58.581 INFO:tasks.workunit.client.0.smithi006.stdout:4/434: truncate d6/f1f 391917 0 2024-03-22T06:14:58.581 INFO:tasks.workunit.client.0.smithi006.stdout:4/435: write d6/fa [12690042,51585] 0 2024-03-22T06:14:58.581 INFO:tasks.workunit.client.0.smithi006.stdout:4/436: stat d6/d13/d3b 0 2024-03-22T06:14:58.583 INFO:tasks.workunit.client.0.smithi006.stdout:0/259: rename db/l28 to db/d3d/d4e/l5d 0 2024-03-22T06:14:58.585 INFO:tasks.workunit.client.0.smithi006.stdout:3/390: creat d0/d7e/f80 x:0 0 0 2024-03-22T06:14:58.590 INFO:tasks.workunit.client.0.smithi006.stdout:9/333: symlink d8/d10/d23/d3e/l73 0 2024-03-22T06:14:58.591 INFO:tasks.workunit.client.0.smithi006.stdout:9/334: dread d8/d10/f2e [0,4194304] 0 2024-03-22T06:14:58.591 INFO:tasks.workunit.client.0.smithi006.stdout:4/437: symlink d6/d1b/l91 0 2024-03-22T06:14:58.595 INFO:tasks.workunit.client.0.smithi006.stdout:8/538: rename d9/db/dc/l1b to d9/d14/d47/d94/da3/lcb 0 2024-03-22T06:14:58.598 INFO:tasks.workunit.client.0.smithi006.stdout:0/260: getdents db/d10/d2a/d41/d4f 0 2024-03-22T06:14:58.598 INFO:tasks.workunit.client.0.smithi006.stdout:0/261: creat db/d10/d2a/f5e x:0 0 0 2024-03-22T06:14:58.599 INFO:tasks.workunit.client.0.smithi006.stdout:3/391: dwrite d0/d8/d1c/d3e/f60 [0,4194304] 0 2024-03-22T06:14:58.599 INFO:tasks.workunit.client.0.smithi006.stdout:3/392: creat d0/d8/d1c/d3e/d5f/f81 x:0 0 0 2024-03-22T06:14:58.602 INFO:tasks.workunit.client.0.smithi006.stdout:9/335: mknod d8/d10/d23/c74 0 2024-03-22T06:14:58.607 INFO:tasks.workunit.client.0.smithi006.stdout:4/438: rename d6/f11 to d6/d1b/d20/d21/f92 0 2024-03-22T06:14:58.607 INFO:tasks.workunit.client.0.smithi006.stdout:3/393: dread d0/d23/d33/f4d [0,4194304] 0 2024-03-22T06:14:58.608 INFO:tasks.workunit.client.0.smithi006.stdout:8/539: mkdir d9/d14/dcc 0 2024-03-22T06:14:58.613 INFO:tasks.workunit.client.0.smithi006.stdout:0/262: symlink db/d10/d2a/l5f 0 2024-03-22T06:14:58.624 INFO:tasks.workunit.client.0.smithi006.stdout:4/439: mkdir d6/d1b/d20/d63/d93 0 2024-03-22T06:14:58.624 INFO:tasks.workunit.client.0.smithi006.stdout:4/440: write d6/d13/f6a [1864794,86026] 0 2024-03-22T06:14:58.624 INFO:tasks.workunit.client.0.smithi006.stdout:4/441: readlink d6/d1b/d20/d6b/d60/l83 0 2024-03-22T06:14:58.626 INFO:tasks.workunit.client.0.smithi006.stdout:3/394: mkdir d0/d23/d27/d82 0 2024-03-22T06:14:58.627 INFO:tasks.workunit.client.0.smithi006.stdout:0/263: dwrite db/f45 [0,4194304] 0 2024-03-22T06:14:58.632 INFO:tasks.workunit.client.0.smithi006.stdout:8/540: creat d9/db/d37/d76/db3/dc3/d89/db8/fcd x:0 0 0 2024-03-22T06:14:58.632 INFO:tasks.workunit.client.0.smithi006.stdout:4/442: dread d6/fb [0,4194304] 0 2024-03-22T06:14:58.632 INFO:tasks.workunit.client.0.smithi006.stdout:4/443: getdents d6/d1b/d20/d6b/d7b 0 2024-03-22T06:14:58.633 INFO:tasks.workunit.client.0.smithi006.stdout:9/336: rename d8/d10/d28/d48 to d8/d10/d28/d39/d75 0 2024-03-22T06:14:58.634 INFO:tasks.workunit.client.0.smithi006.stdout:9/337: readlink d8/d10/d20/d45/l6d 0 2024-03-22T06:14:58.634 INFO:tasks.workunit.client.0.smithi006.stdout:9/338: truncate d8/d10/d28/f44 5568697 0 2024-03-22T06:14:58.640 INFO:tasks.workunit.client.0.smithi006.stdout:8/541: mknod d9/d14/dcc/cce 0 2024-03-22T06:14:58.640 INFO:tasks.workunit.client.0.smithi006.stdout:8/542: stat d9/db/d37/l9c 0 2024-03-22T06:14:58.642 INFO:tasks.workunit.client.0.smithi006.stdout:4/444: unlink d6/d1b/d20/d21/d77/f8b 0 2024-03-22T06:14:58.645 INFO:tasks.workunit.client.0.smithi006.stdout:9/339: creat d8/d10/d28/d39/d75/d50/f76 x:0 0 0 2024-03-22T06:14:58.648 INFO:tasks.workunit.client.0.smithi006.stdout:8/543: symlink d9/db/dc/lcf 0 2024-03-22T06:14:58.648 INFO:tasks.workunit.client.0.smithi006.stdout:0/264: dwrite db/dc/f21 [0,4194304] 0 2024-03-22T06:14:58.659 INFO:tasks.workunit.client.0.smithi006.stdout:4/445: dwrite d6/d1b/d20/d6b/d60/f67 [0,4194304] 0 2024-03-22T06:14:58.659 INFO:tasks.workunit.client.0.smithi006.stdout:4/446: creat d6/d1b/d20/d21/d77/f94 x:0 0 0 2024-03-22T06:14:58.660 INFO:tasks.workunit.client.0.smithi006.stdout:8/544: dread d9/db/d12/d66/d6c/d80/fa4 [0,4194304] 0 2024-03-22T06:14:58.661 INFO:tasks.workunit.client.0.smithi006.stdout:8/545: truncate d9/db/d35/da9/fc8 694709 0 2024-03-22T06:14:58.661 INFO:tasks.workunit.client.0.smithi006.stdout:8/546: dread - d9/db/d12/d66/f7f zero size 2024-03-22T06:14:58.669 INFO:tasks.workunit.client.0.smithi006.stdout:4/447: mkdir d6/d1b/d20/d6b/d95 0 2024-03-22T06:14:58.670 INFO:tasks.workunit.client.0.smithi006.stdout:8/547: mknod d9/db/d37/d4a/cd0 0 2024-03-22T06:14:58.670 INFO:tasks.workunit.client.0.smithi006.stdout:4/448: dread d6/d1b/d46/f5a [0,4194304] 0 2024-03-22T06:14:58.672 INFO:tasks.workunit.client.0.smithi006.stdout:4/449: mknod d6/d1b/d20/d6b/d88/c96 0 2024-03-22T06:14:58.673 INFO:tasks.workunit.client.0.smithi006.stdout:4/450: dread - d6/d1b/d20/d21/d2a/f3f zero size 2024-03-22T06:14:58.674 INFO:tasks.workunit.client.0.smithi006.stdout:0/265: dwrite db/d10/d35/f53 [0,4194304] 0 2024-03-22T06:14:58.674 INFO:tasks.workunit.client.0.smithi006.stdout:0/266: chown db/d10/c31 4936 1 2024-03-22T06:14:58.674 INFO:tasks.workunit.client.0.smithi006.stdout:0/267: read db/d10/d33/f4d [588286,47390] 0 2024-03-22T06:14:58.674 INFO:tasks.workunit.client.0.smithi006.stdout:9/340: dwrite d8/d1f/f68 [0,4194304] 0 2024-03-22T06:14:58.675 INFO:tasks.workunit.client.0.smithi006.stdout:0/268: fdatasync db/d10/d35/f43 0 2024-03-22T06:14:58.675 INFO:tasks.workunit.client.0.smithi006.stdout:0/269: write db/dc/d38/f3a [4987882,100781] 0 2024-03-22T06:14:58.675 INFO:tasks.workunit.client.0.smithi006.stdout:8/548: mknod d9/db/d12/d7e/cd1 0 2024-03-22T06:14:58.675 INFO:tasks.workunit.client.0.smithi006.stdout:4/451: mknod d6/d1b/d20/d21/d77/c97 0 2024-03-22T06:14:58.683 INFO:tasks.workunit.client.0.smithi006.stdout:8/549: dread d9/db/dc/de/f22 [0,4194304] 0 2024-03-22T06:14:58.683 INFO:tasks.workunit.client.0.smithi006.stdout:8/550: stat d9/d14/d47/lbc 0 2024-03-22T06:14:58.685 INFO:tasks.workunit.client.0.smithi006.stdout:4/452: dread d6/d1b/f5b [0,4194304] 0 2024-03-22T06:14:58.686 INFO:tasks.workunit.client.0.smithi006.stdout:0/270: write db/f45 [334304,100061] 0 2024-03-22T06:14:58.692 INFO:tasks.workunit.client.0.smithi006.stdout:4/453: unlink d6/d1b/d20/d21/d2a/f3f 0 2024-03-22T06:14:58.692 INFO:tasks.workunit.client.0.smithi006.stdout:4/454: chown d6/d1b/d20/d6b/d60/l83 377 1 2024-03-22T06:14:58.697 INFO:tasks.workunit.client.0.smithi006.stdout:0/271: symlink db/d3d/d4e/l60 0 2024-03-22T06:14:58.706 INFO:tasks.workunit.client.0.smithi006.stdout:4/455: creat d6/d1b/d20/d6b/d88/f98 x:0 0 0 2024-03-22T06:14:58.718 INFO:tasks.workunit.client.0.smithi006.stdout:4/456: unlink d6/d1b/d20/d21/d2a/l57 0 2024-03-22T06:14:58.720 INFO:tasks.workunit.client.0.smithi006.stdout:8/551: rename d9/db/d37 to d9/db/d35/dd2 0 2024-03-22T06:14:58.720 INFO:tasks.workunit.client.0.smithi006.stdout:8/552: chown d9/db/d12/d66/d6c/d80/fa4 572856166 1 2024-03-22T06:14:58.721 INFO:tasks.workunit.client.0.smithi006.stdout:4/457: truncate d6/f7 2845894 0 2024-03-22T06:14:58.724 INFO:tasks.workunit.client.0.smithi006.stdout:0/272: rename db/d10/c31 to db/d10/d2a/d41/d4f/c61 0 2024-03-22T06:14:58.724 INFO:tasks.workunit.client.0.smithi006.stdout:0/273: write db/d10/d33/f57 [570104,48697] 0 2024-03-22T06:14:58.725 INFO:tasks.workunit.client.0.smithi006.stdout:4/458: symlink d6/d1b/d20/d6b/d60/l99 0 2024-03-22T06:14:58.725 INFO:tasks.workunit.client.0.smithi006.stdout:4/459: stat d6/d1b/d20/d21/d2a/l85 0 2024-03-22T06:14:58.726 INFO:tasks.workunit.client.0.smithi006.stdout:0/274: stat db/d10/c11 0 2024-03-22T06:14:58.729 INFO:tasks.workunit.client.0.smithi006.stdout:8/553: dread f2 [0,4194304] 0 2024-03-22T06:14:58.729 INFO:tasks.workunit.client.0.smithi006.stdout:8/554: write d9/db/d35/dd2/f4b [224415,95215] 0 2024-03-22T06:14:58.730 INFO:tasks.workunit.client.0.smithi006.stdout:8/555: stat d9/db/d35/dd2/d76/db3/dc3/d89 0 2024-03-22T06:14:58.730 INFO:tasks.workunit.client.0.smithi006.stdout:4/460: rename d6/d1b/d20/d21/d2a/d43/d8d to d6/d1b/d20/d63/d93/d9a 0 2024-03-22T06:14:58.730 INFO:tasks.workunit.client.0.smithi006.stdout:4/461: readlink d6/d13/l56 0 2024-03-22T06:14:58.732 INFO:tasks.workunit.client.0.smithi006.stdout:8/556: mknod d9/db/d35/dd2/d4a/d4d/cd3 0 2024-03-22T06:14:58.732 INFO:tasks.workunit.client.0.smithi006.stdout:8/557: chown d9/db/d12/d66/f7f 198 1 2024-03-22T06:14:58.733 INFO:tasks.workunit.client.0.smithi006.stdout:8/558: truncate d9/db/dc/de/d3e/f56 904057 0 2024-03-22T06:14:58.734 INFO:tasks.workunit.client.0.smithi006.stdout:4/462: rename d6/f59 to d6/d1b/d20/d21/d2a/d8a/f9b 0 2024-03-22T06:14:58.735 INFO:tasks.workunit.client.0.smithi006.stdout:4/463: stat d6/l19 0 2024-03-22T06:14:58.737 INFO:tasks.workunit.client.0.smithi006.stdout:4/464: creat d6/d13/f9c x:0 0 0 2024-03-22T06:14:58.743 INFO:tasks.workunit.client.0.smithi006.stdout:4/465: mknod d6/d1b/d20/d21/d2a/d43/c9d 0 2024-03-22T06:14:58.745 INFO:tasks.workunit.client.0.smithi006.stdout:0/275: dwrite db/d10/d2a/f34 [0,4194304] 0 2024-03-22T06:14:58.745 INFO:tasks.workunit.client.0.smithi006.stdout:4/466: creat d6/d1b/d20/d21/d2a/f9e x:0 0 0 2024-03-22T06:14:58.745 INFO:tasks.workunit.client.0.smithi006.stdout:4/467: stat d6/d1b/d20/d21/l6f 0 2024-03-22T06:14:58.748 INFO:tasks.workunit.client.0.smithi006.stdout:0/276: link db/dc/c2b db/d3d/d4e/c62 0 2024-03-22T06:14:58.750 INFO:tasks.workunit.client.0.smithi006.stdout:4/468: mkdir d6/d1b/d20/d21/d9f 0 2024-03-22T06:14:58.753 INFO:tasks.workunit.client.0.smithi006.stdout:0/277: creat db/dc/d37/f63 x:0 0 0 2024-03-22T06:14:58.753 INFO:tasks.workunit.client.0.smithi006.stdout:0/278: write db/d10/d33/f4d [3483197,128097] 0 2024-03-22T06:14:58.755 INFO:tasks.workunit.client.0.smithi006.stdout:0/279: creat db/f64 x:0 0 0 2024-03-22T06:14:58.755 INFO:tasks.workunit.client.0.smithi006.stdout:0/280: creat db/d3d/d4e/f65 x:0 0 0 2024-03-22T06:14:58.755 INFO:tasks.workunit.client.0.smithi006.stdout:0/281: chown db/d10/f14 1429741 1 2024-03-22T06:14:58.760 INFO:tasks.workunit.client.0.smithi006.stdout:4/469: dread d6/d1b/d20/d21/f92 [0,4194304] 0 2024-03-22T06:14:58.762 INFO:tasks.workunit.client.0.smithi006.stdout:4/470: dread d6/d1b/d20/d21/f7e [0,4194304] 0 2024-03-22T06:14:58.764 INFO:tasks.workunit.client.0.smithi006.stdout:4/471: mkdir d6/d1b/d20/d21/d77/da0 0 2024-03-22T06:14:58.779 INFO:tasks.workunit.client.0.smithi006.stdout:4/472: dwrite d6/f4a [0,4194304] 0 2024-03-22T06:14:58.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:58 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/669875527' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:14:58.788 INFO:tasks.workunit.client.0.smithi006.stdout:4/473: dread d6/d1b/d20/d6b/f75 [0,4194304] 0 2024-03-22T06:14:58.794 INFO:tasks.workunit.client.0.smithi006.stdout:4/474: mkdir d6/d1b/d20/d63/d93/d9a/d8f/da1 0 2024-03-22T06:14:58.794 INFO:tasks.workunit.client.0.smithi006.stdout:4/475: stat d6/d1b/c34 0 2024-03-22T06:14:58.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.825+0000 7fa2e6661700 1 -- 172.21.15.6:0/2835634599 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa2e005f5f0 con 0x7fa2e00fcf80 2024-03-22T06:14:58.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.825+0000 7fa2dcff9700 1 -- 172.21.15.6:0/2835634599 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa2d4057700 con 0x7fa2e00fcf80 2024-03-22T06:14:58.826 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_OK 2024-03-22T06:14:58.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.828+0000 7fa2e6661700 1 -- 172.21.15.6:0/2835634599 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fa2c806bb10 msgr2=0x7fa2c806dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:58.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.828+0000 7fa2e6661700 1 --2- 172.21.15.6:0/2835634599 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fa2c806bb10 0x7fa2c806dfc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000b3b0 tx=0x7fa2d0009e30 comp rx=0 tx=0).stop 2024-03-22T06:14:58.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.828+0000 7fa2e6661700 1 -- 172.21.15.6:0/2835634599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e00fcf80 msgr2=0x7fa2e010dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:14:58.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.828+0000 7fa2e6661700 1 --2- 172.21.15.6:0/2835634599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e00fcf80 0x7fa2e010dfd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400b790 tx=0x7fa2d4009250 comp rx=0 tx=0).stop 2024-03-22T06:14:58.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.828+0000 7fa2e6661700 1 -- 172.21.15.6:0/2835634599 shutdown_connections 2024-03-22T06:14:58.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.828+0000 7fa2e6661700 1 --2- 172.21.15.6:0/2835634599 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fa2c806bb10 0x7fa2c806dfc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:58.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.828+0000 7fa2e6661700 1 --2- 172.21.15.6:0/2835634599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e00fbcd0 0x7fa2e010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:58.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.828+0000 7fa2e6661700 1 --2- 172.21.15.6:0/2835634599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2e00fcf80 0x7fa2e010dfd0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:14:58.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.828+0000 7fa2e6661700 1 -- 172.21.15.6:0/2835634599 >> 172.21.15.6:0/2835634599 conn(0x7fa2e00f7430 msgr2=0x7fa2e0065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:14:58.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.828+0000 7fa2e6661700 1 -- 172.21.15.6:0/2835634599 shutdown_connections 2024-03-22T06:14:58.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:14:58.828+0000 7fa2e6661700 1 -- 172.21.15.6:0/2835634599 wait complete. 2024-03-22T06:14:58.893 INFO:tasks.workunit.client.0.smithi006.stdout:4/476: mknod d6/d1b/d20/d21/ca2 0 2024-03-22T06:14:58.895 INFO:tasks.workunit.client.0.smithi006.stdout:4/477: symlink d6/d1b/d20/la3 0 2024-03-22T06:14:58.903 INFO:tasks.workunit.client.0.smithi006.stdout:4/478: write d6/fa [10883828,113764] 0 2024-03-22T06:14:58.905 INFO:tasks.workunit.client.0.smithi006.stdout:4/479: symlink d6/d1b/d20/d63/d93/d9a/la4 0 2024-03-22T06:14:58.908 INFO:tasks.workunit.client.0.smithi006.stdout:4/480: rename d6/d1b/d20/d21/d2a/d43/c87 to d6/d1b/d20/d63/ca5 0 2024-03-22T06:14:58.913 INFO:tasks.workunit.client.0.smithi006.stdout:4/481: dread d6/f8 [4194304,4194304] 0 2024-03-22T06:14:58.913 INFO:tasks.workunit.client.0.smithi006.stdout:4/482: chown d6/d1b/d20/d21/d2a/d43/d4b 4722196 1 2024-03-22T06:14:58.913 INFO:tasks.workunit.client.0.smithi006.stdout:4/483: readlink d6/d1b/d20/d63/d93/d9a/la4 0 2024-03-22T06:14:58.913 INFO:tasks.workunit.client.0.smithi006.stdout:4/484: fdatasync d6/d13/d3b/f4e 0 2024-03-22T06:14:58.914 INFO:tasks.workunit.client.0.smithi006.stdout:4/485: creat d6/d1b/d20/d21/d77/fa6 x:0 0 0 2024-03-22T06:14:58.916 INFO:tasks.workunit.client.0.smithi006.stdout:4/486: creat d6/d1b/d20/d21/d2a/d43/d4b/fa7 x:0 0 0 2024-03-22T06:14:58.920 INFO:tasks.workunit.client.0.smithi006.stdout:4/487: mkdir d6/d1b/d20/d21/d2a/d43/da8 0 2024-03-22T06:14:58.923 INFO:tasks.workunit.client.0.smithi006.stdout:4/488: rename d6/d13/d3b/l72 to d6/d1b/d20/d21/d77/da0/la9 0 2024-03-22T06:14:58.923 INFO:tasks.workunit.client.0.smithi006.stdout:4/489: dread - d6/f62 zero size 2024-03-22T06:14:58.926 INFO:tasks.workunit.client.0.smithi006.stdout:4/490: rename d6/d1b/d20/d21/d2a/f7f to d6/d1b/d20/d63/d93/faa 0 2024-03-22T06:14:58.928 INFO:tasks.workunit.client.0.smithi006.stdout:4/491: creat d6/d1b/d20/d21/fab x:0 0 0 2024-03-22T06:14:58.931 INFO:tasks.workunit.client.0.smithi006.stdout:4/492: creat d6/d1b/d20/d63/d93/d9a/fac x:0 0 0 2024-03-22T06:14:58.931 INFO:tasks.workunit.client.0.smithi006.stdout:4/493: readlink d6/d1b/l1d 0 2024-03-22T06:14:58.933 INFO:tasks.workunit.client.0.smithi006.stdout:4/494: creat d6/d1b/d20/d21/d9f/fad x:0 0 0 2024-03-22T06:14:58.933 INFO:tasks.workunit.client.0.smithi006.stdout:4/495: read f3 [2815172,6996] 0 2024-03-22T06:14:59.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:58 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/669875527' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:14:59.038 INFO:tasks.workunit.client.0.smithi006.stdout:5/538: sync 2024-03-22T06:14:59.038 INFO:tasks.workunit.client.0.smithi006.stdout:5/539: stat d0/db/d16/d43/f67 0 2024-03-22T06:14:59.044 INFO:tasks.workunit.client.0.smithi006.stdout:5/540: write d0/db/f29 [7537089,45692] 0 2024-03-22T06:14:59.059 INFO:tasks.workunit.client.0.smithi006.stdout:5/541: dwrite d0/db/d16/d1c/d34/d35/d85/d5e/d66/f7b [0,4194304] 0 2024-03-22T06:14:59.074 INFO:tasks.workunit.client.0.smithi006.stdout:5/542: getdents d0/db/d50/d96 0 2024-03-22T06:14:59.090 INFO:tasks.workunit.client.0.smithi006.stdout:5/543: dwrite d0/db/d50/f5b [0,4194304] 0 2024-03-22T06:14:59.098 INFO:tasks.workunit.client.0.smithi006.stdout:5/544: dread d0/db/d99/f1d [4194304,4194304] 0 2024-03-22T06:14:59.098 INFO:tasks.workunit.client.0.smithi006.stdout:5/545: chown d0/db/d16/d1c/d34/c38 146 1 2024-03-22T06:14:59.105 INFO:tasks.workunit.client.0.smithi006.stdout:5/546: write d0/db/d50/d96/d61/f79 [309755,51633] 0 2024-03-22T06:14:59.108 INFO:tasks.workunit.client.0.smithi006.stdout:5/547: link d0/db/d16/f49 d0/db/d50/d96/d52/d69/fa3 0 2024-03-22T06:14:59.119 INFO:tasks.workunit.client.0.smithi006.stdout:1/335: sync 2024-03-22T06:14:59.119 INFO:tasks.workunit.client.0.smithi006.stdout:2/397: sync 2024-03-22T06:14:59.119 INFO:tasks.workunit.client.0.smithi006.stdout:1/336: chown d8/d9/d13/d24/d32/d34 404474 1 2024-03-22T06:14:59.121 INFO:tasks.workunit.client.0.smithi006.stdout:2/398: mkdir d13/d20/d26/d19/d99 0 2024-03-22T06:14:59.123 INFO:tasks.workunit.client.0.smithi006.stdout:2/399: write d13/d20/d26/f15 [1457200,77094] 0 2024-03-22T06:14:59.123 INFO:tasks.workunit.client.0.smithi006.stdout:5/548: dwrite d0/db/d50/d58/f91 [0,4194304] 0 2024-03-22T06:14:59.123 INFO:tasks.workunit.client.0.smithi006.stdout:5/549: readlink d0/db/d16/d1c/d34/d35/d5d/d9d/l9e 0 2024-03-22T06:14:59.123 INFO:tasks.workunit.client.0.smithi006.stdout:5/550: chown d0/db/d99/f2b 160989388 1 2024-03-22T06:14:59.126 INFO:tasks.workunit.client.0.smithi006.stdout:5/551: dread d0/db/dc/f3b [0,4194304] 0 2024-03-22T06:14:59.135 INFO:tasks.workunit.client.0.smithi006.stdout:1/337: dwrite d8/d9/d13/f6d [0,4194304] 0 2024-03-22T06:14:59.135 INFO:tasks.workunit.client.0.smithi006.stdout:1/338: write d8/d9/d3a/f6b [320471,76901] 0 2024-03-22T06:14:59.237 INFO:tasks.workunit.client.0.smithi006.stdout:2/400: creat d13/d34/d4d/d4e/d75/f9a x:0 0 0 2024-03-22T06:14:59.241 INFO:tasks.workunit.client.0.smithi006.stdout:5/552: creat d0/fa4 x:0 0 0 2024-03-22T06:14:59.250 INFO:tasks.workunit.client.0.smithi006.stdout:2/401: creat d13/d34/d6e/f9b x:0 0 0 2024-03-22T06:14:59.251 INFO:tasks.workunit.client.0.smithi006.stdout:2/402: chown d13/d34/c93 1072700979 1 2024-03-22T06:14:59.251 INFO:tasks.workunit.client.0.smithi006.stdout:2/403: chown d13/d53/d29/d82 27 1 2024-03-22T06:14:59.251 INFO:tasks.workunit.client.0.smithi006.stdout:2/404: readlink d13/l44 0 2024-03-22T06:14:59.266 INFO:tasks.workunit.client.0.smithi006.stdout:2/405: dwrite d13/d20/d26/f2f [0,4194304] 0 2024-03-22T06:14:59.269 INFO:tasks.workunit.client.0.smithi006.stdout:2/406: link d13/d20/d26/d19/f48 d13/d34/d4d/d4e/d75/d62/f9c 0 2024-03-22T06:14:59.270 INFO:tasks.workunit.client.0.smithi006.stdout:2/407: dread d13/d20/d26/d19/f73 [0,4194304] 0 2024-03-22T06:14:59.271 INFO:tasks.workunit.client.0.smithi006.stdout:2/408: fsync d13/d20/f4b 0 2024-03-22T06:14:59.272 INFO:tasks.workunit.client.0.smithi006.stdout:2/409: truncate d13/f6f 2104527 0 2024-03-22T06:14:59.304 INFO:tasks.workunit.client.0.smithi006.stdout:2/410: getdents d13/d34/d4d/d4e/d75 0 2024-03-22T06:14:59.304 INFO:tasks.workunit.client.0.smithi006.stdout:2/411: write d13/d53/f8a [595458,21445] 0 2024-03-22T06:14:59.305 INFO:tasks.workunit.client.0.smithi006.stdout:2/412: creat d13/d53/d6b/f9d x:0 0 0 2024-03-22T06:14:59.306 INFO:tasks.workunit.client.0.smithi006.stdout:2/413: dread - d13/d53/d29/f67 zero size 2024-03-22T06:14:59.308 INFO:tasks.workunit.client.0.smithi006.stdout:2/414: symlink d13/d53/l9e 0 2024-03-22T06:14:59.310 INFO:tasks.workunit.client.0.smithi006.stdout:2/415: getdents d13/d53/d29 0 2024-03-22T06:14:59.313 INFO:tasks.workunit.client.0.smithi006.stdout:2/416: symlink d13/d53/d29/l9f 0 2024-03-22T06:14:59.313 INFO:tasks.workunit.client.0.smithi006.stdout:2/417: truncate d13/d20/d35/d65/f68 528773 0 2024-03-22T06:14:59.313 INFO:tasks.workunit.client.0.smithi006.stdout:7/474: sync 2024-03-22T06:14:59.313 INFO:tasks.workunit.client.0.smithi006.stdout:6/369: sync 2024-03-22T06:14:59.315 INFO:tasks.workunit.client.0.smithi006.stdout:2/418: creat d13/d53/fa0 x:0 0 0 2024-03-22T06:14:59.316 INFO:tasks.workunit.client.0.smithi006.stdout:7/475: truncate d5/d1e/d29/d1f/d2d/d55/f58 3412074 0 2024-03-22T06:14:59.319 INFO:tasks.workunit.client.0.smithi006.stdout:2/419: mknod d13/d69/ca1 0 2024-03-22T06:14:59.319 INFO:tasks.workunit.client.0.smithi006.stdout:2/420: creat d13/d20/d35/fa2 x:0 0 0 2024-03-22T06:14:59.320 INFO:tasks.workunit.client.0.smithi006.stdout:2/421: dread d13/d20/d26/d19/f3e [0,4194304] 0 2024-03-22T06:14:59.320 INFO:tasks.workunit.client.0.smithi006.stdout:7/476: truncate d5/f56 3050814 0 2024-03-22T06:14:59.326 INFO:tasks.workunit.client.0.smithi006.stdout:7/477: stat d5/d1e/d29/d1f/l57 0 2024-03-22T06:14:59.326 INFO:tasks.workunit.client.0.smithi006.stdout:7/478: write d5/d1e/d3f/f4b [1177877,15987] 0 2024-03-22T06:14:59.327 INFO:tasks.workunit.client.0.smithi006.stdout:7/479: fsync d5/d1e/d29/d1f/d2d/d41/f7b 0 2024-03-22T06:14:59.330 INFO:tasks.workunit.client.0.smithi006.stdout:2/422: write d13/d20/f52 [4037635,28162] 0 2024-03-22T06:14:59.337 INFO:tasks.workunit.client.0.smithi006.stdout:7/480: write d5/d1e/d29/d1f/d2d/f4c [28246,27572] 0 2024-03-22T06:14:59.337 INFO:tasks.workunit.client.0.smithi006.stdout:7/481: write d5/d1e/d29/d59/f7a [1199304,74409] 0 2024-03-22T06:14:59.339 INFO:tasks.workunit.client.0.smithi006.stdout:7/482: creat d5/d1e/f95 x:0 0 0 2024-03-22T06:14:59.341 INFO:tasks.workunit.client.0.smithi006.stdout:7/483: mknod d5/c96 0 2024-03-22T06:14:59.342 INFO:tasks.workunit.client.0.smithi006.stdout:7/484: chown d5/d1e/d3f/d94 644526713 1 2024-03-22T06:14:59.344 INFO:tasks.workunit.client.0.smithi006.stdout:7/485: rmdir d5/d1e/d29/d59 39 2024-03-22T06:14:59.344 INFO:tasks.workunit.client.0.smithi006.stdout:7/486: stat d5/d1e/d29/d1f 0 2024-03-22T06:14:59.344 INFO:tasks.workunit.client.0.smithi006.stdout:7/487: readlink d5/l7f 0 2024-03-22T06:14:59.345 INFO:tasks.workunit.client.0.smithi006.stdout:7/488: symlink d5/d1e/l97 0 2024-03-22T06:14:59.348 INFO:tasks.workunit.client.0.smithi006.stdout:7/489: creat d5/d1e/d63/d85/f98 x:0 0 0 2024-03-22T06:14:59.348 INFO:tasks.workunit.client.0.smithi006.stdout:7/490: fdatasync d5/d1e/d29/d59/f8e 0 2024-03-22T06:14:59.351 INFO:tasks.workunit.client.0.smithi006.stdout:7/491: getdents d5/d1e/d29/d1f/d2d/d55 0 2024-03-22T06:14:59.374 INFO:tasks.workunit.client.0.smithi006.stdout:7/492: dwrite d5/d1e/d3f/f4b [0,4194304] 0 2024-03-22T06:14:59.385 INFO:tasks.workunit.client.0.smithi006.stdout:2/423: write d13/d34/d4d/d4e/d75/d62/f9c [778619,45038] 0 2024-03-22T06:14:59.385 INFO:tasks.workunit.client.0.smithi006.stdout:2/424: creat d13/d34/d6e/fa3 x:0 0 0 2024-03-22T06:14:59.386 INFO:tasks.workunit.client.0.smithi006.stdout:7/493: dwrite d5/d1e/d3f/f81 [0,4194304] 0 2024-03-22T06:14:59.386 INFO:tasks.workunit.client.0.smithi006.stdout:7/494: dread - d5/d1e/d29/d59/f8e zero size 2024-03-22T06:14:59.386 INFO:tasks.workunit.client.0.smithi006.stdout:7/495: chown d5/d1e/d29/d1f/l27 44821 1 2024-03-22T06:14:59.387 INFO:tasks.workunit.client.0.smithi006.stdout:2/425: rmdir d13/d69 39 2024-03-22T06:14:59.392 INFO:tasks.workunit.client.0.smithi006.stdout:2/426: symlink d13/d20/d26/la4 0 2024-03-22T06:14:59.395 INFO:tasks.workunit.client.0.smithi006.stdout:7/496: dread d5/d1e/d3f/f5a [4194304,4194304] 0 2024-03-22T06:14:59.396 INFO:tasks.workunit.client.0.smithi006.stdout:2/427: rename d13/d53/d29 to d13/d34/d4d/d4e/d75/da5 0 2024-03-22T06:14:59.397 INFO:tasks.workunit.client.0.smithi006.stdout:7/497: mknod d5/d1e/d29/c99 0 2024-03-22T06:14:59.398 INFO:tasks.workunit.client.0.smithi006.stdout:7/498: chown d5/d1e/d29/d1f 9799138 1 2024-03-22T06:14:59.398 INFO:tasks.workunit.client.0.smithi006.stdout:7/499: readlink d5/d1e/d29/l2b 0 2024-03-22T06:14:59.399 INFO:tasks.workunit.client.0.smithi006.stdout:2/428: mkdir d13/d20/da6 0 2024-03-22T06:14:59.401 INFO:tasks.workunit.client.0.smithi006.stdout:7/500: rename d5/d1e/d29/d1f/c6b to d5/d1e/d29/d1f/d2d/d55/d84/c9a 0 2024-03-22T06:14:59.405 INFO:tasks.workunit.client.0.smithi006.stdout:7/501: rmdir d5/d1e/d29/d1f/d2d/d55/d84 39 2024-03-22T06:14:59.406 INFO:tasks.workunit.client.0.smithi006.stdout:7/502: read d5/d1e/d29/d59/f77 [85,27988] 0 2024-03-22T06:14:59.408 INFO:tasks.workunit.client.0.smithi006.stdout:2/429: creat d13/d34/d4d/d4e/d75/da5/d82/fa7 x:0 0 0 2024-03-22T06:14:59.409 INFO:tasks.workunit.client.0.smithi006.stdout:7/503: rename d5/d1e/d29/d1f/c33 to d5/d1e/d29/d1f/c9b 0 2024-03-22T06:14:59.412 INFO:tasks.workunit.client.0.smithi006.stdout:7/504: symlink d5/d1e/d63/d85/l9c 0 2024-03-22T06:14:59.412 INFO:tasks.workunit.client.0.smithi006.stdout:3/395: sync 2024-03-22T06:14:59.412 INFO:tasks.workunit.client.0.smithi006.stdout:3/396: read d0/d23/d27/f48 [4111850,30546] 0 2024-03-22T06:14:59.414 INFO:tasks.workunit.client.0.smithi006.stdout:7/505: creat d5/d1e/d29/d59/f9d x:0 0 0 2024-03-22T06:14:59.420 INFO:tasks.workunit.client.0.smithi006.stdout:3/397: dread d0/d8/d9/f32 [0,4194304] 0 2024-03-22T06:14:59.430 INFO:tasks.workunit.client.0.smithi006.stdout:3/398: rmdir d0/d7e 39 2024-03-22T06:14:59.431 INFO:tasks.workunit.client.0.smithi006.stdout:2/430: dwrite f6 [0,4194304] 0 2024-03-22T06:14:59.432 INFO:tasks.workunit.client.0.smithi006.stdout:3/399: mkdir d0/d8/d1c/d3e/d5f/d83 0 2024-03-22T06:14:59.434 INFO:tasks.workunit.client.0.smithi006.stdout:2/431: dread d13/d20/d35/d65/f68 [0,4194304] 0 2024-03-22T06:14:59.435 INFO:tasks.workunit.client.0.smithi006.stdout:3/400: mknod d0/d23/d33/d3d/d58/c84 0 2024-03-22T06:14:59.436 INFO:tasks.workunit.client.0.smithi006.stdout:2/432: mknod d13/d20/da6/ca8 0 2024-03-22T06:14:59.439 INFO:tasks.workunit.client.0.smithi006.stdout:3/401: symlink d0/d4/l85 0 2024-03-22T06:14:59.439 INFO:tasks.workunit.client.0.smithi006.stdout:3/402: write d0/d23/d33/d3d/f6e [506904,85974] 0 2024-03-22T06:14:59.439 INFO:tasks.workunit.client.0.smithi006.stdout:3/403: chown d0/d7e/f80 18540 1 2024-03-22T06:14:59.439 INFO:tasks.workunit.client.0.smithi006.stdout:3/404: chown d0/d8 11009385 1 2024-03-22T06:14:59.439 INFO:tasks.workunit.client.0.smithi006.stdout:3/405: creat d0/d23/d33/d3d/f86 x:0 0 0 2024-03-22T06:14:59.439 INFO:tasks.workunit.client.0.smithi006.stdout:2/433: mknod d13/d34/d4d/d4e/d75/ca9 0 2024-03-22T06:14:59.443 INFO:tasks.workunit.client.0.smithi006.stdout:3/406: rename d0/d8/d1c/f74 to d0/d8/d1c/d72/f87 0 2024-03-22T06:14:59.443 INFO:tasks.workunit.client.0.smithi006.stdout:3/407: fsync d0/d8/d1c/d3e/d68/f6d 0 2024-03-22T06:14:59.445 INFO:tasks.workunit.client.0.smithi006.stdout:2/434: truncate d13/d20/d26/f15 1239155 0 2024-03-22T06:14:59.445 INFO:tasks.workunit.client.0.smithi006.stdout:2/435: truncate d13/d20/f7d 667963 0 2024-03-22T06:14:59.445 INFO:tasks.workunit.client.0.smithi006.stdout:2/436: chown d13/d34/d4d/d4e/d75/da5/d82/fa7 6 1 2024-03-22T06:14:59.445 INFO:tasks.workunit.client.0.smithi006.stdout:2/437: write d13/d20/d26/f79 [1335359,127442] 0 2024-03-22T06:14:59.447 INFO:tasks.workunit.client.0.smithi006.stdout:3/408: mkdir d0/d8/d88 0 2024-03-22T06:14:59.457 INFO:tasks.workunit.client.0.smithi006.stdout:3/409: write d0/d23/d27/f59 [1460831,52610] 0 2024-03-22T06:14:59.460 INFO:tasks.workunit.client.0.smithi006.stdout:8/559: sync 2024-03-22T06:14:59.460 INFO:tasks.workunit.client.0.smithi006.stdout:9/341: sync 2024-03-22T06:14:59.461 INFO:tasks.workunit.client.0.smithi006.stdout:2/438: dwrite d13/d20/f8b [0,4194304] 0 2024-03-22T06:14:59.461 INFO:tasks.workunit.client.0.smithi006.stdout:2/439: fdatasync d13/d34/d6e/f84 0 2024-03-22T06:14:59.462 INFO:tasks.workunit.client.0.smithi006.stdout:9/342: getdents d8/d10/d28/d39 0 2024-03-22T06:14:59.463 INFO:tasks.workunit.client.0.smithi006.stdout:7/506: rmdir d5 39 2024-03-22T06:14:59.463 INFO:tasks.workunit.client.0.smithi006.stdout:7/507: write d5/d1e/d3f/f4b [4827453,95656] 0 2024-03-22T06:14:59.471 INFO:tasks.workunit.client.0.smithi006.stdout:9/343: creat d8/d10/d20/d45/d49/f77 x:0 0 0 2024-03-22T06:14:59.474 INFO:tasks.workunit.client.0.smithi006.stdout:8/560: write d9/d14/f34 [928865,93306] 0 2024-03-22T06:14:59.475 INFO:tasks.workunit.client.0.smithi006.stdout:7/508: link d5/l64 d5/d1e/d63/d85/l9e 0 2024-03-22T06:14:59.480 INFO:tasks.workunit.client.0.smithi006.stdout:8/561: creat d9/db/d35/dd2/d76/db3/dc3/d89/fd4 x:0 0 0 2024-03-22T06:14:59.480 INFO:tasks.workunit.client.0.smithi006.stdout:8/562: fsync d9/d5e/f5f 0 2024-03-22T06:14:59.481 INFO:tasks.workunit.client.0.smithi006.stdout:3/410: dwrite d0/d23/d33/d3d/f86 [0,4194304] 0 2024-03-22T06:14:59.483 INFO:tasks.workunit.client.0.smithi006.stdout:2/440: dwrite f12 [0,4194304] 0 2024-03-22T06:14:59.483 INFO:tasks.workunit.client.0.smithi006.stdout:7/509: rename d5/fe to d5/d1e/d29/d66/d91/f9f 0 2024-03-22T06:14:59.483 INFO:tasks.workunit.client.0.smithi006.stdout:7/510: chown d5/d1e/d63/d85/f98 1979709 1 2024-03-22T06:14:59.483 INFO:tasks.workunit.client.0.smithi006.stdout:7/511: write d5/f1d [4725562,14535] 0 2024-03-22T06:14:59.483 INFO:tasks.workunit.client.0.smithi006.stdout:9/344: dread d8/d10/d28/f44 [0,4194304] 0 2024-03-22T06:14:59.483 INFO:tasks.workunit.client.0.smithi006.stdout:9/345: fsync f6 0 2024-03-22T06:14:59.508 INFO:tasks.workunit.client.0.smithi006.stdout:9/346: dwrite d8/d10/d11/f16 [0,4194304] 0 2024-03-22T06:14:59.508 INFO:tasks.workunit.client.0.smithi006.stdout:9/347: truncate d8/d10/d11/f59 926933 0 2024-03-22T06:14:59.509 INFO:tasks.workunit.client.0.smithi006.stdout:8/563: dwrite d9/db/d12/f33 [0,4194304] 0 2024-03-22T06:14:59.509 INFO:tasks.workunit.client.0.smithi006.stdout:8/564: write d9/db/dc/de/f22 [2750106,19221] 0 2024-03-22T06:14:59.510 INFO:tasks.workunit.client.0.smithi006.stdout:8/565: chown d9/db/dc 540096 1 2024-03-22T06:14:59.527 INFO:tasks.workunit.client.0.smithi006.stdout:3/411: mkdir d0/d8/d1c/d89 0 2024-03-22T06:14:59.531 INFO:tasks.workunit.client.0.smithi006.stdout:2/441: unlink d13/d53/d6b/c91 0 2024-03-22T06:14:59.542 INFO:tasks.workunit.client.0.smithi006.stdout:2/442: symlink d13/d53/laa 0 2024-03-22T06:14:59.544 INFO:tasks.workunit.client.0.smithi006.stdout:4/496: sync 2024-03-22T06:14:59.544 INFO:tasks.workunit.client.0.smithi006.stdout:0/282: sync 2024-03-22T06:14:59.544 INFO:tasks.workunit.client.0.smithi006.stdout:1/339: sync 2024-03-22T06:14:59.544 INFO:tasks.workunit.client.0.smithi006.stdout:6/370: sync 2024-03-22T06:14:59.544 INFO:tasks.workunit.client.0.smithi006.stdout:0/283: creat db/dc/d38/f66 x:0 0 0 2024-03-22T06:14:59.546 INFO:tasks.workunit.client.0.smithi006.stdout:4/497: truncate d6/d13/f6a 1583697 0 2024-03-22T06:14:59.548 INFO:tasks.workunit.client.0.smithi006.stdout:6/371: symlink d6/d1d/l87 0 2024-03-22T06:14:59.549 INFO:tasks.workunit.client.0.smithi006.stdout:5/553: sync 2024-03-22T06:14:59.552 INFO:tasks.workunit.client.0.smithi006.stdout:5/554: dread d0/db/d16/f49 [0,4194304] 0 2024-03-22T06:14:59.552 INFO:tasks.workunit.client.0.smithi006.stdout:0/284: rmdir db/d3d 39 2024-03-22T06:14:59.555 INFO:tasks.workunit.client.0.smithi006.stdout:4/498: getdents d6/d1b/d20/d63/d93 0 2024-03-22T06:14:59.557 INFO:tasks.workunit.client.0.smithi006.stdout:4/499: truncate d6/f42 5141537 0 2024-03-22T06:14:59.559 INFO:tasks.workunit.client.0.smithi006.stdout:5/555: rename d0/db/d50/ca1 to d0/db/d16/ca5 0 2024-03-22T06:14:59.559 INFO:tasks.workunit.client.0.smithi006.stdout:5/556: truncate d0/db/d16/d1c/d34/d35/d85/d5e/d66/f9b 811692 0 2024-03-22T06:14:59.559 INFO:tasks.workunit.client.0.smithi006.stdout:5/557: write d0/db/d16/d1c/f78 [643025,31814] 0 2024-03-22T06:14:59.560 INFO:tasks.workunit.client.0.smithi006.stdout:5/558: readlink d0/db/d16/l84 0 2024-03-22T06:14:59.560 INFO:tasks.workunit.client.0.smithi006.stdout:5/559: chown d0/db/d16/d70 76654 1 2024-03-22T06:14:59.562 INFO:tasks.workunit.client.0.smithi006.stdout:5/560: readlink d0/db/d16/l17 0 2024-03-22T06:14:59.562 INFO:tasks.workunit.client.0.smithi006.stdout:5/561: getdents d0/db/d50/d96 0 2024-03-22T06:14:59.562 INFO:tasks.workunit.client.0.smithi006.stdout:5/562: fsync d0/db/d50/d96/d61/f79 0 2024-03-22T06:14:59.563 INFO:tasks.workunit.client.0.smithi006.stdout:5/563: stat d0/db/d16/c64 0 2024-03-22T06:14:59.563 INFO:tasks.workunit.client.0.smithi006.stdout:5/564: creat d0/db/d16/d1c/d34/fa6 x:0 0 0 2024-03-22T06:14:59.566 INFO:tasks.workunit.client.0.smithi006.stdout:5/565: creat d0/db/d50/d96/d61/fa7 x:0 0 0 2024-03-22T06:14:59.567 INFO:tasks.workunit.client.0.smithi006.stdout:0/285: dread db/d10/d35/f53 [0,4194304] 0 2024-03-22T06:14:59.567 INFO:tasks.workunit.client.0.smithi006.stdout:0/286: creat db/d10/d33/f67 x:0 0 0 2024-03-22T06:14:59.570 INFO:tasks.workunit.client.0.smithi006.stdout:5/566: getdents d0 0 2024-03-22T06:14:59.571 INFO:tasks.workunit.client.0.smithi006.stdout:1/340: dread d8/d9/d13/f28 [0,4194304] 0 2024-03-22T06:14:59.571 INFO:tasks.workunit.client.0.smithi006.stdout:1/341: readlink d8/d9/d3a/d18/l5e 0 2024-03-22T06:14:59.573 INFO:tasks.workunit.client.0.smithi006.stdout:5/567: link d0/db/d99/f74 d0/db/d16/d1c/d34/d35/fa8 0 2024-03-22T06:14:59.575 INFO:tasks.workunit.client.0.smithi006.stdout:2/443: dwrite fc [0,4194304] 0 2024-03-22T06:14:59.575 INFO:tasks.workunit.client.0.smithi006.stdout:4/500: dread d6/d1b/f47 [0,4194304] 0 2024-03-22T06:14:59.575 INFO:tasks.workunit.client.0.smithi006.stdout:4/501: write d6/f12 [9841287,21268] 0 2024-03-22T06:14:59.575 INFO:tasks.workunit.client.0.smithi006.stdout:0/287: write db/dc/f21 [1397298,110570] 0 2024-03-22T06:14:59.575 INFO:tasks.workunit.client.0.smithi006.stdout:6/372: dwrite d6/d1b/d2c/f72 [0,4194304] 0 2024-03-22T06:14:59.575 INFO:tasks.workunit.client.0.smithi006.stdout:6/373: rename d6 to d6/d1d/d5c/d88 22 2024-03-22T06:14:59.575 INFO:tasks.workunit.client.0.smithi006.stdout:1/342: link d8/d9/d13/f26 d8/d9/d13/d24/d32/f6f 0 2024-03-22T06:14:59.577 INFO:tasks.workunit.client.0.smithi006.stdout:2/444: creat d13/d20/d26/fab x:0 0 0 2024-03-22T06:14:59.588 INFO:tasks.workunit.client.0.smithi006.stdout:5/568: read d0/db/d16/d1c/d34/f4b [2354746,35948] 0 2024-03-22T06:14:59.596 INFO:tasks.workunit.client.0.smithi006.stdout:4/502: dwrite d6/d13/d71/f82 [0,4194304] 0 2024-03-22T06:14:59.596 INFO:tasks.workunit.client.0.smithi006.stdout:4/503: creat d6/d1b/d20/d6b/fae x:0 0 0 2024-03-22T06:14:59.617 INFO:tasks.workunit.client.0.smithi006.stdout:4/504: dwrite d6/d1b/d20/d6b/d60/f67 [4194304,4194304] 0 2024-03-22T06:14:59.643 INFO:tasks.workunit.client.0.smithi006.stdout:0/288: creat db/d10/d2a/d41/d4f/f68 x:0 0 0 2024-03-22T06:14:59.645 INFO:tasks.workunit.client.0.smithi006.stdout:6/374: mkdir d6/d1d/d5c/d89 0 2024-03-22T06:14:59.645 INFO:tasks.workunit.client.0.smithi006.stdout:1/343: symlink d8/d9/d13/d24/l70 0 2024-03-22T06:14:59.648 INFO:tasks.workunit.client.0.smithi006.stdout:2/445: stat d13/d34/d4d/d4e/d75/l41 0 2024-03-22T06:14:59.649 INFO:tasks.workunit.client.0.smithi006.stdout:5/569: link d0/db/d50/d96/c56 d0/db/d76/ca9 0 2024-03-22T06:14:59.658 INFO:tasks.workunit.client.0.smithi006.stdout:6/375: dread d6/d24/d44/f5f [0,4194304] 0 2024-03-22T06:14:59.658 INFO:tasks.workunit.client.0.smithi006.stdout:2/446: dread fb [4194304,4194304] 0 2024-03-22T06:14:59.660 INFO:tasks.workunit.client.0.smithi006.stdout:4/505: symlink d6/d1b/d20/d21/d2a/d5c/laf 0 2024-03-22T06:14:59.661 INFO:tasks.workunit.client.0.smithi006.stdout:4/506: write d6/d13/d3b/f51 [38154,111639] 0 2024-03-22T06:14:59.663 INFO:tasks.workunit.client.0.smithi006.stdout:0/289: link db/d10/f1a db/d10/d35/f69 0 2024-03-22T06:14:59.663 INFO:tasks.workunit.client.0.smithi006.stdout:1/344: dwrite d8/d33/f5b [0,4194304] 0 2024-03-22T06:14:59.665 INFO:tasks.workunit.client.0.smithi006.stdout:5/570: creat d0/db/d50/d96/faa x:0 0 0 2024-03-22T06:14:59.665 INFO:tasks.workunit.client.0.smithi006.stdout:5/571: write d0/db/d50/f80 [1410869,65315] 0 2024-03-22T06:14:59.665 INFO:tasks.workunit.client.0.smithi006.stdout:5/572: chown d0/db/d16/d1c/d34/d35/d85/d5e/d66/f9b 83684 1 2024-03-22T06:14:59.665 INFO:tasks.workunit.client.0.smithi006.stdout:5/573: dread - d0/db/d76/f9a zero size 2024-03-22T06:14:59.665 INFO:tasks.workunit.client.0.smithi006.stdout:5/574: stat d0/db/l62 0 2024-03-22T06:14:59.674 INFO:tasks.workunit.client.0.smithi006.stdout:5/575: dread d0/db/d50/d96/f41 [0,4194304] 0 2024-03-22T06:14:59.678 INFO:tasks.workunit.client.0.smithi006.stdout:0/290: dwrite db/dc/f44 [0,4194304] 0 2024-03-22T06:14:59.707 INFO:tasks.workunit.client.0.smithi006.stdout:2/447: truncate f7 198459 0 2024-03-22T06:14:59.707 INFO:tasks.workunit.client.0.smithi006.stdout:2/448: dread - d13/d20/d35/fa2 zero size 2024-03-22T06:14:59.709 INFO:tasks.workunit.client.0.smithi006.stdout:4/507: creat d6/d1b/d20/d63/d93/fb0 x:0 0 0 2024-03-22T06:14:59.720 INFO:tasks.workunit.client.0.smithi006.stdout:6/376: dwrite d6/f66 [0,4194304] 0 2024-03-22T06:14:59.724 INFO:tasks.workunit.client.0.smithi006.stdout:0/291: mknod db/d10/d2a/d41/d4f/c6a 0 2024-03-22T06:14:59.725 INFO:tasks.workunit.client.0.smithi006.stdout:0/292: creat db/f6b x:0 0 0 2024-03-22T06:14:59.725 INFO:tasks.workunit.client.0.smithi006.stdout:0/293: write db/d10/d2a/f51 [516310,128197] 0 2024-03-22T06:14:59.774 INFO:tasks.workunit.client.0.smithi006.stdout:5/576: link d0/db/f60 d0/db/dc/fab 0 2024-03-22T06:14:59.819 INFO:tasks.workunit.client.0.smithi006.stdout:2/449: symlink d13/d34/d4d/d4e/d75/d62/lac 0 2024-03-22T06:14:59.820 INFO:tasks.workunit.client.0.smithi006.stdout:2/450: truncate d13/d20/d26/d19/f73 557489 0 2024-03-22T06:14:59.823 INFO:tasks.workunit.client.0.smithi006.stdout:6/377: mkdir d6/d1d/d5c/d81/d8a 0 2024-03-22T06:14:59.823 INFO:tasks.workunit.client.0.smithi006.stdout:6/378: dread - d6/d1d/d64/f6a zero size 2024-03-22T06:14:59.827 INFO:tasks.workunit.client.0.smithi006.stdout:0/294: truncate db/d10/f14 2390581 0 2024-03-22T06:14:59.841 INFO:tasks.workunit.client.0.smithi006.stdout:5/577: symlink d0/db/d50/lac 0 2024-03-22T06:14:59.841 INFO:tasks.workunit.client.0.smithi006.stdout:5/578: chown d0/db/d99/f1d 23711721 1 2024-03-22T06:14:59.841 INFO:tasks.workunit.client.0.smithi006.stdout:5/579: chown d0/db/f1e 3 1 2024-03-22T06:14:59.842 INFO:tasks.workunit.client.0.smithi006.stdout:6/379: link d6/d1b/f82 d6/d1d/d5c/d89/f8b 0 2024-03-22T06:14:59.844 INFO:tasks.workunit.client.0.smithi006.stdout:2/451: dwrite d13/d20/f52 [0,4194304] 0 2024-03-22T06:14:59.844 INFO:tasks.workunit.client.0.smithi006.stdout:2/452: fdatasync d13/f6f 0 2024-03-22T06:14:59.844 INFO:tasks.workunit.client.0.smithi006.stdout:0/295: dwrite db/dc/d38/f4b [0,4194304] 0 2024-03-22T06:14:59.845 INFO:tasks.workunit.client.0.smithi006.stdout:0/296: stat db/dc/d37 0 2024-03-22T06:14:59.847 INFO:tasks.workunit.client.0.smithi006.stdout:5/580: mkdir d0/db/d50/d58/dad 0 2024-03-22T06:14:59.847 INFO:tasks.workunit.client.0.smithi006.stdout:5/581: fdatasync d0/db/f1e 0 2024-03-22T06:14:59.847 INFO:tasks.workunit.client.0.smithi006.stdout:5/582: creat d0/db/d50/d96/fae x:0 0 0 2024-03-22T06:14:59.855 INFO:tasks.workunit.client.0.smithi006.stdout:2/453: unlink d13/d34/d4d/d4e/d75/l40 0 2024-03-22T06:14:59.859 INFO:tasks.workunit.client.0.smithi006.stdout:0/297: mkdir db/d10/d35/d6c 0 2024-03-22T06:14:59.868 INFO:tasks.workunit.client.0.smithi006.stdout:6/380: dwrite d6/d1d/d5c/f5e [0,4194304] 0 2024-03-22T06:14:59.868 INFO:tasks.workunit.client.0.smithi006.stdout:5/583: dwrite d0/db/d50/f5b [0,4194304] 0 2024-03-22T06:14:59.870 INFO:tasks.workunit.client.0.smithi006.stdout:0/298: rmdir db/d3d/d4e 39 2024-03-22T06:14:59.870 INFO:tasks.workunit.client.0.smithi006.stdout:0/299: fsync db/dc/f21 0 2024-03-22T06:14:59.880 INFO:tasks.workunit.client.0.smithi006.stdout:2/454: dwrite d13/d34/d4d/d4e/d75/da5/f94 [0,4194304] 0 2024-03-22T06:14:59.881 INFO:tasks.workunit.client.0.smithi006.stdout:2/455: dread fe [0,4194304] 0 2024-03-22T06:14:59.881 INFO:tasks.workunit.client.0.smithi006.stdout:2/456: creat d13/d20/fad x:0 0 0 2024-03-22T06:14:59.882 INFO:tasks.workunit.client.0.smithi006.stdout:5/584: symlink d0/db/d16/d43/laf 0 2024-03-22T06:14:59.883 INFO:tasks.workunit.client.0.smithi006.stdout:0/300: symlink db/d10/d2a/d41/d4f/l6d 0 2024-03-22T06:14:59.884 INFO:tasks.workunit.client.0.smithi006.stdout:6/381: dwrite d6/d24/f38 [0,4194304] 0 2024-03-22T06:14:59.884 INFO:tasks.workunit.client.0.smithi006.stdout:6/382: chown d6/d1b/d2c/f7d 37033334 1 2024-03-22T06:14:59.889 INFO:tasks.workunit.client.0.smithi006.stdout:2/457: creat d13/d34/d4d/d4e/d75/da5/d82/fae x:0 0 0 2024-03-22T06:14:59.897 INFO:tasks.workunit.client.0.smithi006.stdout:6/383: unlink d6/d1b/d2c/d4a/f5a 0 2024-03-22T06:14:59.897 INFO:tasks.workunit.client.0.smithi006.stdout:2/458: creat d13/d34/d4d/d4e/d75/faf x:0 0 0 2024-03-22T06:14:59.898 INFO:tasks.workunit.client.0.smithi006.stdout:2/459: symlink d13/d34/d6e/d72/lb0 0 2024-03-22T06:14:59.905 INFO:tasks.workunit.client.0.smithi006.stdout:6/384: read d6/d1d/f20 [974666,42165] 0 2024-03-22T06:14:59.909 INFO:tasks.workunit.client.0.smithi006.stdout:6/385: link d6/d6f/f6e d6/d1b/d2c/f8c 0 2024-03-22T06:14:59.909 INFO:tasks.workunit.client.0.smithi006.stdout:6/386: write d6/d1d/d5c/d81/f5d [96990,7952] 0 2024-03-22T06:14:59.909 INFO:tasks.workunit.client.0.smithi006.stdout:6/387: write d6/d24/f38 [4517969,2136] 0 2024-03-22T06:14:59.911 INFO:tasks.workunit.client.0.smithi006.stdout:1/345: mkdir d8/d71 0 2024-03-22T06:14:59.928 INFO:tasks.workunit.client.0.smithi006.stdout:1/346: dwrite d8/d9/f35 [0,4194304] 0 2024-03-22T06:14:59.943 INFO:tasks.workunit.client.0.smithi006.stdout:1/347: dwrite d8/d9/f35 [4194304,4194304] 0 2024-03-22T06:14:59.943 INFO:tasks.workunit.client.0.smithi006.stdout:1/348: chown d8/d9/l47 279 1 2024-03-22T06:14:59.945 INFO:tasks.workunit.client.0.smithi006.stdout:1/349: mknod d8/d9/d13/d39/d60/c72 0 2024-03-22T06:14:59.952 INFO:tasks.workunit.client.0.smithi006.stdout:1/350: dread d8/d9/d3a/d18/f56 [0,4194304] 0 2024-03-22T06:14:59.953 INFO:tasks.workunit.client.0.smithi006.stdout:1/351: getdents d8/d9/d13/d24/d32/d34 0 2024-03-22T06:14:59.956 INFO:tasks.workunit.client.0.smithi006.stdout:1/352: mknod d8/d9/d13/d39/c73 0 2024-03-22T06:14:59.958 INFO:tasks.workunit.client.0.smithi006.stdout:1/353: rmdir d8/d9/d13/d24/d32/d34 39 2024-03-22T06:14:59.982 INFO:tasks.workunit.client.0.smithi006.stdout:6/388: fsync d6/d1d/d5c/d89/f8b 0 2024-03-22T06:14:59.984 INFO:tasks.workunit.client.0.smithi006.stdout:6/389: truncate d6/d1d/f26 3214282 0 2024-03-22T06:14:59.984 INFO:tasks.workunit.client.0.smithi006.stdout:6/390: write d6/f66 [557323,83312] 0 2024-03-22T06:14:59.989 INFO:tasks.workunit.client.0.smithi006.stdout:0/301: rmdir db/d10/d33 39 2024-03-22T06:14:59.995 INFO:tasks.workunit.client.0.smithi006.stdout:5/585: symlink d0/db/d16/d1c/lb0 0 2024-03-22T06:14:59.996 INFO:tasks.workunit.client.0.smithi006.stdout:5/586: dread - d0/db/d16/d1c/d34/fa6 zero size 2024-03-22T06:15:00.000 INFO:tasks.workunit.client.0.smithi006.stdout:5/587: mkdir d0/db/d16/d43/db1 0 2024-03-22T06:15:00.003 INFO:tasks.workunit.client.0.smithi006.stdout:5/588: creat d0/db/d16/d1c/d34/d35/d85/d54/d65/fb2 x:0 0 0 2024-03-22T06:15:00.004 INFO:tasks.workunit.client.0.smithi006.stdout:6/391: dwrite d6/d1b/d2c/f59 [0,4194304] 0 2024-03-22T06:15:00.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:59 smithi117 ceph-mon[36514]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:15:00.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:59 smithi117 ceph-mon[36514]: pgmap v13: 65 pgs: 65 active+clean; 728 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 66 MiB/s rd, 92 MiB/s wr, 141 op/s 2024-03-22T06:15:00.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:14:59 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2835634599' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:15:00.006 INFO:tasks.workunit.client.0.smithi006.stdout:5/589: symlink d0/db/lb3 0 2024-03-22T06:15:00.006 INFO:tasks.workunit.client.0.smithi006.stdout:5/590: stat d0/db/d50/d96/f63 0 2024-03-22T06:15:00.006 INFO:tasks.workunit.client.0.smithi006.stdout:5/591: write d0/db/d16/d70/f75 [3900625,97681] 0 2024-03-22T06:15:00.010 INFO:tasks.workunit.client.0.smithi006.stdout:6/392: unlink d6/d1b/f48 0 2024-03-22T06:15:00.013 INFO:tasks.workunit.client.0.smithi006.stdout:5/592: mkdir d0/db/d76/db4 0 2024-03-22T06:15:00.027 INFO:tasks.workunit.client.0.smithi006.stdout:6/393: dwrite d6/f51 [0,4194304] 0 2024-03-22T06:15:00.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:59 smithi006 ceph-mon[29301]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:15:00.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:59 smithi006 ceph-mon[29301]: pgmap v13: 65 pgs: 65 active+clean; 728 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 66 MiB/s rd, 92 MiB/s wr, 141 op/s 2024-03-22T06:15:00.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:14:59 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2835634599' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:15:00.035 INFO:tasks.workunit.client.0.smithi006.stdout:4/508: rename d6/d13/d71 to d6/d1b/d20/d6b/d60/db1 0 2024-03-22T06:15:00.040 INFO:tasks.workunit.client.0.smithi006.stdout:2/460: rename ld to d13/d34/d4d/lb1 0 2024-03-22T06:15:00.041 INFO:tasks.workunit.client.0.smithi006.stdout:4/509: symlink d6/d1b/d20/d21/d77/da0/lb2 0 2024-03-22T06:15:00.044 INFO:tasks.workunit.client.0.smithi006.stdout:2/461: symlink d13/d34/d4d/d4e/lb2 0 2024-03-22T06:15:00.046 INFO:tasks.workunit.client.0.smithi006.stdout:4/510: dread d6/d1b/f5b [0,4194304] 0 2024-03-22T06:15:00.047 INFO:tasks.workunit.client.0.smithi006.stdout:2/462: creat d13/d20/d26/d19/fb3 x:0 0 0 2024-03-22T06:15:00.047 INFO:tasks.workunit.client.0.smithi006.stdout:2/463: chown d13/d69/l81 18588 1 2024-03-22T06:15:00.048 INFO:tasks.workunit.client.0.smithi006.stdout:2/464: readlink d13/d34/d4d/d4e/d75/l37 0 2024-03-22T06:15:00.049 INFO:tasks.workunit.client.0.smithi006.stdout:4/511: symlink d6/d1b/d20/d63/d93/lb3 0 2024-03-22T06:15:00.049 INFO:tasks.workunit.client.0.smithi006.stdout:4/512: write d6/f42 [5925565,98099] 0 2024-03-22T06:15:00.053 INFO:tasks.workunit.client.0.smithi006.stdout:4/513: rename d6/d1b/d20/d63/d93/d9a/la4 to d6/d1b/d20/d21/d2a/d43/d4b/lb4 0 2024-03-22T06:15:00.055 INFO:tasks.workunit.client.0.smithi006.stdout:2/465: dread d13/d20/d26/d19/f59 [0,4194304] 0 2024-03-22T06:15:00.055 INFO:tasks.workunit.client.0.smithi006.stdout:2/466: chown d13/d53/f5e 46 1 2024-03-22T06:15:00.058 INFO:tasks.workunit.client.0.smithi006.stdout:2/467: symlink d13/d20/d35/lb4 0 2024-03-22T06:15:00.058 INFO:tasks.workunit.client.0.smithi006.stdout:2/468: readlink d13/l71 0 2024-03-22T06:15:00.058 INFO:tasks.workunit.client.0.smithi006.stdout:2/469: chown d13/d20/d35/fa2 2297050 1 2024-03-22T06:15:00.065 INFO:tasks.workunit.client.0.smithi006.stdout:4/514: rename d6/d1b/d20/d21/d2a/d8a to d6/d1b/d20/d63/d93/d9a/d8f/da1/db5 0 2024-03-22T06:15:00.113 INFO:tasks.workunit.client.0.smithi006.stdout:4/515: getdents d6/d1b/d46 0 2024-03-22T06:15:00.122 INFO:tasks.workunit.client.0.smithi006.stdout:4/516: creat d6/d1b/d20/d63/d93/d9a/d8f/da1/fb6 x:0 0 0 2024-03-22T06:15:00.165 INFO:tasks.workunit.client.0.smithi006.stdout:5/593: rmdir d0/db/d16/d1c/d34/d35/d85/d54/d65 39 2024-03-22T06:15:00.166 INFO:tasks.workunit.client.0.smithi006.stdout:5/594: dread d0/db/f60 [0,4194304] 0 2024-03-22T06:15:00.166 INFO:tasks.workunit.client.0.smithi006.stdout:5/595: write d0/db/d16/d70/f75 [4909123,32010] 0 2024-03-22T06:15:00.167 INFO:tasks.workunit.client.0.smithi006.stdout:5/596: read d0/db/d50/d96/f63 [670920,2092] 0 2024-03-22T06:15:00.172 INFO:tasks.workunit.client.0.smithi006.stdout:5/597: mkdir d0/db/d50/d82/db5 0 2024-03-22T06:15:00.174 INFO:tasks.workunit.client.0.smithi006.stdout:5/598: mknod d0/db/d50/cb6 0 2024-03-22T06:15:00.187 INFO:tasks.workunit.client.0.smithi006.stdout:9/348: sync 2024-03-22T06:15:00.187 INFO:tasks.workunit.client.0.smithi006.stdout:7/512: sync 2024-03-22T06:15:00.187 INFO:tasks.workunit.client.0.smithi006.stdout:3/412: sync 2024-03-22T06:15:00.187 INFO:tasks.workunit.client.0.smithi006.stdout:7/513: fdatasync d5/d1e/d29/d1f/d2d/f37 0 2024-03-22T06:15:00.187 INFO:tasks.workunit.client.0.smithi006.stdout:8/566: sync 2024-03-22T06:15:00.187 INFO:tasks.workunit.client.0.smithi006.stdout:8/567: chown d9/db/d35/dd2/d4a/f82 80 1 2024-03-22T06:15:00.188 INFO:tasks.workunit.client.0.smithi006.stdout:5/599: dwrite d0/db/dc/f3b [0,4194304] 0 2024-03-22T06:15:00.188 INFO:tasks.workunit.client.0.smithi006.stdout:5/600: write d0/db/d16/d1c/d34/d35/d85/f89 [675895,84585] 0 2024-03-22T06:15:00.190 INFO:tasks.workunit.client.0.smithi006.stdout:9/349: rename d8/d10/d28/l1e to d8/d10/d20/d45/d49/l78 0 2024-03-22T06:15:00.193 INFO:tasks.workunit.client.0.smithi006.stdout:3/413: symlink d0/d4/d3a/l8a 0 2024-03-22T06:15:00.193 INFO:tasks.workunit.client.0.smithi006.stdout:3/414: write d0/d8/d1c/d3e/d5f/f81 [775477,38812] 0 2024-03-22T06:15:00.195 INFO:tasks.workunit.client.0.smithi006.stdout:7/514: dread d5/d1e/d3f/f45 [0,4194304] 0 2024-03-22T06:15:00.195 INFO:tasks.workunit.client.0.smithi006.stdout:7/515: read - d5/f5f zero size 2024-03-22T06:15:00.195 INFO:tasks.workunit.client.0.smithi006.stdout:2/470: rmdir d13/d34/d4d 39 2024-03-22T06:15:00.195 INFO:tasks.workunit.client.0.smithi006.stdout:2/471: write d13/d20/f5a [846117,69913] 0 2024-03-22T06:15:00.195 INFO:tasks.workunit.client.0.smithi006.stdout:2/472: creat d13/d53/d6b/fb5 x:0 0 0 2024-03-22T06:15:00.195 INFO:tasks.workunit.client.0.smithi006.stdout:8/568: rmdir d9/db/d12/d66/d6c 39 2024-03-22T06:15:00.195 INFO:tasks.workunit.client.0.smithi006.stdout:8/569: chown d9/db/d12/d66/d6c/l6e 5 1 2024-03-22T06:15:00.201 INFO:tasks.workunit.client.0.smithi006.stdout:5/601: unlink d0/db/d50/d96/d61/fa7 0 2024-03-22T06:15:00.201 INFO:tasks.workunit.client.0.smithi006.stdout:5/602: write d0/db/d99/f45 [326338,123957] 0 2024-03-22T06:15:00.203 INFO:tasks.workunit.client.0.smithi006.stdout:9/350: creat d8/d1f/d34/f79 x:0 0 0 2024-03-22T06:15:00.205 INFO:tasks.workunit.client.0.smithi006.stdout:3/415: symlink d0/d8/d1c/d3e/l8b 0 2024-03-22T06:15:00.206 INFO:tasks.workunit.client.0.smithi006.stdout:7/516: truncate d5/d1e/d29/d1f/d2d/f4c 774524 0 2024-03-22T06:15:00.211 INFO:tasks.workunit.client.0.smithi006.stdout:5/603: write d0/db/d50/f5b [3728805,16908] 0 2024-03-22T06:15:00.211 INFO:tasks.workunit.client.0.smithi006.stdout:5/604: read - d0/db/d50/d96/fae zero size 2024-03-22T06:15:00.214 INFO:tasks.workunit.client.0.smithi006.stdout:2/473: creat d13/d34/d4d/d4e/d75/d62/fb6 x:0 0 0 2024-03-22T06:15:00.214 INFO:tasks.workunit.client.0.smithi006.stdout:2/474: fsync d13/f18 0 2024-03-22T06:15:00.215 INFO:tasks.workunit.client.0.smithi006.stdout:2/475: write d13/d53/d6b/f97 [570649,9670] 0 2024-03-22T06:15:00.217 INFO:tasks.workunit.client.0.smithi006.stdout:8/570: stat d9/db/d12/d7e/c9e 0 2024-03-22T06:15:00.219 INFO:tasks.workunit.client.0.smithi006.stdout:9/351: creat d8/d10/d20/d45/d4d/f7a x:0 0 0 2024-03-22T06:15:00.219 INFO:tasks.workunit.client.0.smithi006.stdout:9/352: dread - d8/d10/d28/d39/d75/d50/f69 zero size 2024-03-22T06:15:00.219 INFO:tasks.workunit.client.0.smithi006.stdout:9/353: creat d8/d10/d20/d45/f7b x:0 0 0 2024-03-22T06:15:00.220 INFO:tasks.workunit.client.0.smithi006.stdout:2/476: dread d13/f18 [0,4194304] 0 2024-03-22T06:15:00.220 INFO:tasks.workunit.client.0.smithi006.stdout:9/354: chown d8/d10/d28/d39/f46 10409478 1 2024-03-22T06:15:00.220 INFO:tasks.workunit.client.0.smithi006.stdout:9/355: creat d8/f7c x:0 0 0 2024-03-22T06:15:00.223 INFO:tasks.workunit.client.0.smithi006.stdout:3/416: mkdir d0/d8/d1c/d3e/d5f/d83/d8c 0 2024-03-22T06:15:00.226 INFO:tasks.workunit.client.0.smithi006.stdout:7/517: creat d5/d1e/d29/d1f/d2d/d55/d84/fa0 x:0 0 0 2024-03-22T06:15:00.226 INFO:tasks.workunit.client.0.smithi006.stdout:7/518: readlink d5/d1e/d29/d10/l40 0 2024-03-22T06:15:00.229 INFO:tasks.workunit.client.0.smithi006.stdout:4/517: rmdir d6/d1b/d20/d63/d93/d9a 39 2024-03-22T06:15:00.233 INFO:tasks.workunit.client.0.smithi006.stdout:5/605: dwrite d0/db/d16/d1c/d97/fa2 [0,4194304] 0 2024-03-22T06:15:00.234 INFO:tasks.workunit.client.0.smithi006.stdout:5/606: fsync d0/db/d50/d96/d61/f7d 0 2024-03-22T06:15:00.234 INFO:tasks.workunit.client.0.smithi006.stdout:8/571: dwrite d9/d14/d47/f70 [0,4194304] 0 2024-03-22T06:15:00.234 INFO:tasks.workunit.client.0.smithi006.stdout:8/572: dread - d9/f6b zero size 2024-03-22T06:15:00.237 INFO:tasks.workunit.client.0.smithi006.stdout:2/477: creat d13/d34/d4d/d4e/d75/fb7 x:0 0 0 2024-03-22T06:15:00.239 INFO:tasks.workunit.client.0.smithi006.stdout:9/356: unlink d8/d10/d20/d45/d49/l5f 0 2024-03-22T06:15:00.241 INFO:tasks.workunit.client.0.smithi006.stdout:3/417: truncate d0/d8/f1a 841088 0 2024-03-22T06:15:00.241 INFO:tasks.workunit.client.0.smithi006.stdout:3/418: creat d0/d8/d1c/d3e/f8d x:0 0 0 2024-03-22T06:15:00.243 INFO:tasks.workunit.client.0.smithi006.stdout:7/519: mkdir d5/d1e/d63/d85/da1 0 2024-03-22T06:15:00.243 INFO:tasks.workunit.client.0.smithi006.stdout:7/520: truncate d5/d1e/d29/d1f/d2d/d55/d84/fa0 512499 0 2024-03-22T06:15:00.254 INFO:tasks.workunit.client.0.smithi006.stdout:2/478: dwrite d13/f23 [4194304,4194304] 0 2024-03-22T06:15:00.259 INFO:tasks.workunit.client.0.smithi006.stdout:5/607: dread d0/db/d16/f21 [0,4194304] 0 2024-03-22T06:15:00.265 INFO:tasks.workunit.client.0.smithi006.stdout:9/357: unlink d8/d10/d20/f2d 0 2024-03-22T06:15:00.265 INFO:tasks.workunit.client.0.smithi006.stdout:9/358: stat d8/d10/d20 0 2024-03-22T06:15:00.268 INFO:tasks.workunit.client.0.smithi006.stdout:3/419: mkdir d0/d8/d88/d8e 0 2024-03-22T06:15:00.269 INFO:tasks.workunit.client.0.smithi006.stdout:7/521: symlink d5/d1e/d29/la2 0 2024-03-22T06:15:00.273 INFO:tasks.workunit.client.0.smithi006.stdout:2/479: rename l10 to d13/d34/d4d/d4e/lb8 0 2024-03-22T06:15:00.275 INFO:tasks.workunit.client.0.smithi006.stdout:5/608: mkdir d0/db/d50/d96/d52/d69/d92/db7 0 2024-03-22T06:15:00.277 INFO:tasks.workunit.client.0.smithi006.stdout:8/573: mknod d9/db/d12/cd5 0 2024-03-22T06:15:00.281 INFO:tasks.workunit.client.0.smithi006.stdout:3/420: creat d0/d23/d33/d3d/d58/f8f x:0 0 0 2024-03-22T06:15:00.283 INFO:tasks.workunit.client.0.smithi006.stdout:7/522: unlink d5/d1e/l74 0 2024-03-22T06:15:00.290 INFO:tasks.workunit.client.0.smithi006.stdout:5/609: link d0/db/d16/l17 d0/db/d50/d96/d61/lb8 0 2024-03-22T06:15:00.290 INFO:tasks.workunit.client.0.smithi006.stdout:5/610: chown d0/db/d50/d96/f63 31631 1 2024-03-22T06:15:00.292 INFO:tasks.workunit.client.0.smithi006.stdout:8/574: fdatasync d9/db/d12/d1e/f2d 0 2024-03-22T06:15:00.296 INFO:tasks.workunit.client.0.smithi006.stdout:3/421: mkdir d0/d8/d1c/d3e/d5f/d90 0 2024-03-22T06:15:00.296 INFO:tasks.workunit.client.0.smithi006.stdout:8/575: dread d9/db/dc/de/f22 [0,4194304] 0 2024-03-22T06:15:00.296 INFO:tasks.workunit.client.0.smithi006.stdout:8/576: chown d9/db/d35/dd2/lc5 2083490964 1 2024-03-22T06:15:00.296 INFO:tasks.workunit.client.0.smithi006.stdout:8/577: write d9/fa [1161784,10100] 0 2024-03-22T06:15:00.298 INFO:tasks.workunit.client.0.smithi006.stdout:7/523: creat d5/d1e/d29/d66/fa3 x:0 0 0 2024-03-22T06:15:00.300 INFO:tasks.workunit.client.0.smithi006.stdout:5/611: mkdir d0/db/d50/db9 0 2024-03-22T06:15:00.302 INFO:tasks.workunit.client.0.smithi006.stdout:2/480: dwrite d13/d34/d4d/d4e/d75/da5/f67 [0,4194304] 0 2024-03-22T06:15:00.305 INFO:tasks.workunit.client.0.smithi006.stdout:8/578: mknod d9/db/d35/dd2/d76/dca/cd6 0 2024-03-22T06:15:00.307 INFO:tasks.workunit.client.0.smithi006.stdout:7/524: creat d5/d1e/d63/d85/da1/fa4 x:0 0 0 2024-03-22T06:15:00.307 INFO:tasks.workunit.client.0.smithi006.stdout:7/525: truncate d5/d1e/d63/d85/da1/fa4 43874 0 2024-03-22T06:15:00.311 INFO:tasks.workunit.client.0.smithi006.stdout:3/422: rename d0/d7e/c40 to d0/c91 0 2024-03-22T06:15:00.311 INFO:tasks.workunit.client.0.smithi006.stdout:3/423: stat d0/d7e/f80 0 2024-03-22T06:15:00.319 INFO:tasks.workunit.client.0.smithi006.stdout:2/481: mkdir d13/d20/d26/db9 0 2024-03-22T06:15:00.319 INFO:tasks.workunit.client.0.smithi006.stdout:2/482: chown d13/d53 8573 1 2024-03-22T06:15:00.321 INFO:tasks.workunit.client.0.smithi006.stdout:8/579: symlink d9/db/dc/de/ld7 0 2024-03-22T06:15:00.323 INFO:tasks.workunit.client.0.smithi006.stdout:2/483: dread d13/d53/f8a [0,4194304] 0 2024-03-22T06:15:00.323 INFO:tasks.workunit.client.0.smithi006.stdout:2/484: readlink d13/l17 0 2024-03-22T06:15:00.324 INFO:tasks.workunit.client.0.smithi006.stdout:7/526: mknod d5/d1e/d29/d1f/d2d/d55/ca5 0 2024-03-22T06:15:00.327 INFO:tasks.workunit.client.0.smithi006.stdout:5/612: rmdir d0/db/d16/d1c/d34/d4d 39 2024-03-22T06:15:00.331 INFO:tasks.workunit.client.0.smithi006.stdout:3/424: dwrite d0/d23/d27/f59 [4194304,4194304] 0 2024-03-22T06:15:00.331 INFO:tasks.workunit.client.0.smithi006.stdout:3/425: readlink d0/d8/d1c/d3e/l4b 0 2024-03-22T06:15:00.332 INFO:tasks.workunit.client.0.smithi006.stdout:2/485: mkdir d13/d20/da6/dba 0 2024-03-22T06:15:00.336 INFO:tasks.workunit.client.0.smithi006.stdout:5/613: rename d0/db/f57 to d0/db/d16/d1c/d34/d35/d85/d54/d65/d77/fba 0 2024-03-22T06:15:00.336 INFO:tasks.workunit.client.0.smithi006.stdout:5/614: chown d0/db/d99/c23 2288062 1 2024-03-22T06:15:00.336 INFO:tasks.workunit.client.0.smithi006.stdout:5/615: creat d0/db/d50/d96/d61/fbb x:0 0 0 2024-03-22T06:15:00.336 INFO:tasks.workunit.client.0.smithi006.stdout:5/616: write d0/db/d50/f5b [4047651,30184] 0 2024-03-22T06:15:00.337 INFO:tasks.workunit.client.0.smithi006.stdout:5/617: fdatasync d0/db/d16/d1c/d34/d35/d85/f47 0 2024-03-22T06:15:00.337 INFO:tasks.workunit.client.0.smithi006.stdout:5/618: chown d0/db/l62 3503 1 2024-03-22T06:15:00.342 INFO:tasks.workunit.client.0.smithi006.stdout:3/426: rename d0/d23/d27/f59 to d0/d8/d1c/d3e/d5f/f92 0 2024-03-22T06:15:00.342 INFO:tasks.workunit.client.0.smithi006.stdout:3/427: chown d0/d8/d1c/d49/f67 18597212 1 2024-03-22T06:15:00.342 INFO:tasks.workunit.client.0.smithi006.stdout:3/428: chown d0/d8 32 1 2024-03-22T06:15:00.343 INFO:tasks.workunit.client.0.smithi006.stdout:2/486: write d13/d20/f3d [23210,69669] 0 2024-03-22T06:15:00.348 INFO:tasks.workunit.client.0.smithi006.stdout:5/619: unlink d0/db/d50/d96/c56 0 2024-03-22T06:15:00.348 INFO:tasks.workunit.client.0.smithi006.stdout:5/620: write d0/db/d16/d1c/d34/d35/d85/d54/d65/d77/f86 [731326,38682] 0 2024-03-22T06:15:00.348 INFO:tasks.workunit.client.0.smithi006.stdout:5/621: truncate d0/db/d16/d1c/d34/fa6 915103 0 2024-03-22T06:15:00.349 INFO:tasks.workunit.client.0.smithi006.stdout:7/527: dwrite d5/d1e/d29/d1f/d2d/d41/f69 [0,4194304] 0 2024-03-22T06:15:00.353 INFO:tasks.workunit.client.0.smithi006.stdout:8/580: dwrite f7 [0,4194304] 0 2024-03-22T06:15:00.355 INFO:tasks.workunit.client.0.smithi006.stdout:2/487: rename d13/f21 to d13/d20/d35/d65/fbb 0 2024-03-22T06:15:00.355 INFO:tasks.workunit.client.0.smithi006.stdout:2/488: chown d13/d34/d4d/lb1 3253946 1 2024-03-22T06:15:00.355 INFO:tasks.workunit.client.0.smithi006.stdout:2/489: fsync d13/d34/f8c 0 2024-03-22T06:15:00.358 INFO:tasks.workunit.client.0.smithi006.stdout:5/622: unlink d0/db/d99/f45 0 2024-03-22T06:15:00.366 INFO:tasks.workunit.client.0.smithi006.stdout:3/429: dwrite d0/d23/f63 [0,4194304] 0 2024-03-22T06:15:00.367 INFO:tasks.workunit.client.0.smithi006.stdout:7/528: dwrite d5/d1e/d29/d1f/d2d/d55/d84/fa0 [0,4194304] 0 2024-03-22T06:15:00.367 INFO:tasks.workunit.client.0.smithi006.stdout:7/529: truncate d5/f9 4599816 0 2024-03-22T06:15:00.367 INFO:tasks.workunit.client.0.smithi006.stdout:7/530: chown d5/d1e/d29/d10 57331270 1 2024-03-22T06:15:00.367 INFO:tasks.workunit.client.0.smithi006.stdout:7/531: write d5/d1e/d63/d85/f98 [525204,89911] 0 2024-03-22T06:15:00.369 INFO:tasks.workunit.client.0.smithi006.stdout:2/490: creat d13/d34/d4d/d4e/d75/da5/fbc x:0 0 0 2024-03-22T06:15:00.369 INFO:tasks.workunit.client.0.smithi006.stdout:2/491: creat d13/d34/d4d/d4e/d75/d62/fbd x:0 0 0 2024-03-22T06:15:00.372 INFO:tasks.workunit.client.0.smithi006.stdout:5/623: mkdir d0/db/d16/d1c/dbc 0 2024-03-22T06:15:00.372 INFO:tasks.workunit.client.0.smithi006.stdout:1/354: sync 2024-03-22T06:15:00.372 INFO:tasks.workunit.client.0.smithi006.stdout:0/302: sync 2024-03-22T06:15:00.372 INFO:tasks.workunit.client.0.smithi006.stdout:6/394: sync 2024-03-22T06:15:00.372 INFO:tasks.workunit.client.0.smithi006.stdout:6/395: chown d6/d1d/f71 3473 1 2024-03-22T06:15:00.372 INFO:tasks.workunit.client.0.smithi006.stdout:6/396: truncate d6/d1b/f82 767904 0 2024-03-22T06:15:00.373 INFO:tasks.workunit.client.0.smithi006.stdout:3/430: creat d0/d7e/f93 x:0 0 0 2024-03-22T06:15:00.373 INFO:tasks.workunit.client.0.smithi006.stdout:3/431: truncate d0/d8/d1c/d3e/d68/f6b 876203 0 2024-03-22T06:15:00.380 INFO:tasks.workunit.client.0.smithi006.stdout:7/532: truncate d5/d1e/d29/d1f/d2d/d55/f58 181645 0 2024-03-22T06:15:00.380 INFO:tasks.workunit.client.0.smithi006.stdout:8/581: dwrite d9/db/d12/f90 [0,4194304] 0 2024-03-22T06:15:00.381 INFO:tasks.workunit.client.0.smithi006.stdout:8/582: write d9/db/d35/dd2/d4a/d4d/f8d [4567694,97533] 0 2024-03-22T06:15:00.384 INFO:tasks.workunit.client.0.smithi006.stdout:2/492: rename d13/d34/f8d to d13/fbe 0 2024-03-22T06:15:00.386 INFO:tasks.workunit.client.0.smithi006.stdout:5/624: truncate d0/db/d16/d1c/d34/f73 2609907 0 2024-03-22T06:15:00.388 INFO:tasks.workunit.client.0.smithi006.stdout:3/432: dwrite d0/d8/d1c/d3e/f60 [0,4194304] 0 2024-03-22T06:15:00.388 INFO:tasks.workunit.client.0.smithi006.stdout:3/433: truncate d0/d23/d4c/f7b 213056 0 2024-03-22T06:15:00.389 INFO:tasks.workunit.client.0.smithi006.stdout:0/303: unlink db/ld 0 2024-03-22T06:15:00.391 INFO:tasks.workunit.client.0.smithi006.stdout:1/355: creat d8/d9/d3a/d18/d2a/f74 x:0 0 0 2024-03-22T06:15:00.391 INFO:tasks.workunit.client.0.smithi006.stdout:1/356: fdatasync d8/d33/f4d 0 2024-03-22T06:15:00.391 INFO:tasks.workunit.client.0.smithi006.stdout:1/357: chown d8/d9/d3a/f6b 1763836790 1 2024-03-22T06:15:00.395 INFO:tasks.workunit.client.0.smithi006.stdout:6/397: creat d6/d1d/d32/f8d x:0 0 0 2024-03-22T06:15:00.397 INFO:tasks.workunit.client.0.smithi006.stdout:7/533: creat d5/d1e/d63/d85/fa6 x:0 0 0 2024-03-22T06:15:00.398 INFO:tasks.workunit.client.0.smithi006.stdout:3/434: write d0/d8/d1c/f4f [4192595,87651] 0 2024-03-22T06:15:00.405 INFO:tasks.workunit.client.0.smithi006.stdout:0/304: dwrite db/f6b [0,4194304] 0 2024-03-22T06:15:00.410 INFO:tasks.workunit.client.0.smithi006.stdout:8/583: symlink d9/db/d35/dd2/d4a/da7/ld8 0 2024-03-22T06:15:00.410 INFO:tasks.workunit.client.0.smithi006.stdout:8/584: truncate d9/db/d35/da9/faa 238202 0 2024-03-22T06:15:00.412 INFO:tasks.workunit.client.0.smithi006.stdout:6/398: dwrite d6/d1d/d32/f8d [0,4194304] 0 2024-03-22T06:15:00.413 INFO:tasks.workunit.client.0.smithi006.stdout:6/399: stat d6/d7c/d83 0 2024-03-22T06:15:00.413 INFO:tasks.workunit.client.0.smithi006.stdout:6/400: dread - d6/d1d/d32/f67 zero size 2024-03-22T06:15:00.413 INFO:tasks.workunit.client.0.smithi006.stdout:6/401: chown d6/d24/d44/f4d 110586 1 2024-03-22T06:15:00.413 INFO:tasks.workunit.client.0.smithi006.stdout:6/402: stat d6/d1d/d32/c49 0 2024-03-22T06:15:00.413 INFO:tasks.workunit.client.0.smithi006.stdout:6/403: write d6/d1d/f20 [324569,18587] 0 2024-03-22T06:15:00.413 INFO:tasks.workunit.client.0.smithi006.stdout:2/493: creat d13/d34/d4d/fbf x:0 0 0 2024-03-22T06:15:00.416 INFO:tasks.workunit.client.0.smithi006.stdout:5/625: mknod d0/db/d50/d82/db5/cbd 0 2024-03-22T06:15:00.417 INFO:tasks.workunit.client.0.smithi006.stdout:6/404: dread d6/d1b/f23 [0,4194304] 0 2024-03-22T06:15:00.420 INFO:tasks.workunit.client.0.smithi006.stdout:1/358: rename d8/d9/d3a/d18/c3c to d8/d9/d3a/d18/c75 0 2024-03-22T06:15:00.420 INFO:tasks.workunit.client.0.smithi006.stdout:1/359: creat d8/d9/d3a/d40/d44/f76 x:0 0 0 2024-03-22T06:15:00.422 INFO:tasks.workunit.client.0.smithi006.stdout:7/534: mknod d5/d1e/ca7 0 2024-03-22T06:15:00.422 INFO:tasks.workunit.client.0.smithi006.stdout:7/535: write d5/f28 [909986,120490] 0 2024-03-22T06:15:00.424 INFO:tasks.workunit.client.0.smithi006.stdout:3/435: creat d0/d8/d1c/d72/f94 x:0 0 0 2024-03-22T06:15:00.427 INFO:tasks.workunit.client.0.smithi006.stdout:0/305: dwrite db/dc/f21 [0,4194304] 0 2024-03-22T06:15:00.427 INFO:tasks.workunit.client.0.smithi006.stdout:0/306: fdatasync db/dc/f21 0 2024-03-22T06:15:00.427 INFO:tasks.workunit.client.0.smithi006.stdout:0/307: truncate db/d10/d33/f57 767579 0 2024-03-22T06:15:00.427 INFO:tasks.workunit.client.0.smithi006.stdout:0/308: chown db/d10/d33/f57 367072049 1 2024-03-22T06:15:00.431 INFO:tasks.workunit.client.0.smithi006.stdout:8/585: symlink d9/db/d12/d66/d6c/ld9 0 2024-03-22T06:15:00.431 INFO:tasks.workunit.client.0.smithi006.stdout:8/586: creat d9/d14/d47/d94/da3/fda x:0 0 0 2024-03-22T06:15:00.449 INFO:tasks.workunit.client.0.smithi006.stdout:3/436: dwrite d0/d8/f1a [0,4194304] 0 2024-03-22T06:15:00.449 INFO:tasks.workunit.client.0.smithi006.stdout:3/437: chown d0/d23/d33 1 1 2024-03-22T06:15:00.458 INFO:tasks.workunit.client.0.smithi006.stdout:2/494: rename d13/d53/fa0 to d13/d53/d6b/fc0 0 2024-03-22T06:15:00.459 INFO:tasks.workunit.client.0.smithi006.stdout:5/626: creat d0/db/d50/db9/fbe x:0 0 0 2024-03-22T06:15:00.459 INFO:tasks.workunit.client.0.smithi006.stdout:5/627: readlink d0/db/d16/d1c/lb0 0 2024-03-22T06:15:00.461 INFO:tasks.workunit.client.0.smithi006.stdout:6/405: getdents d6/d1b/d2c 0 2024-03-22T06:15:00.461 INFO:tasks.workunit.client.0.smithi006.stdout:6/406: write d6/d1d/f71 [592568,42659] 0 2024-03-22T06:15:00.467 INFO:tasks.workunit.client.0.smithi006.stdout:1/360: creat d8/d9/d13/d24/d32/d34/f77 x:0 0 0 2024-03-22T06:15:00.468 INFO:tasks.workunit.client.0.smithi006.stdout:1/361: dread - d8/d9/f67 zero size 2024-03-22T06:15:00.469 INFO:tasks.workunit.client.0.smithi006.stdout:7/536: creat d5/d1e/d29/d1f/d2d/d55/fa8 x:0 0 0 2024-03-22T06:15:00.479 INFO:tasks.workunit.client.0.smithi006.stdout:0/309: link db/dc/d38/c55 db/d10/d35/c6e 0 2024-03-22T06:15:00.480 INFO:tasks.workunit.client.0.smithi006.stdout:0/310: write db/dc/d38/f4b [4763831,60132] 0 2024-03-22T06:15:00.484 INFO:tasks.workunit.client.0.smithi006.stdout:8/587: truncate d9/db/d12/f43 950000 0 2024-03-22T06:15:00.488 INFO:tasks.workunit.client.0.smithi006.stdout:3/438: mkdir d0/d8/d1c/d95 0 2024-03-22T06:15:00.488 INFO:tasks.workunit.client.0.smithi006.stdout:3/439: creat d0/d8/d1c/d3e/f96 x:0 0 0 2024-03-22T06:15:00.489 INFO:tasks.workunit.client.0.smithi006.stdout:3/440: dread d0/d23/d33/f65 [0,4194304] 0 2024-03-22T06:15:00.490 INFO:tasks.workunit.client.0.smithi006.stdout:3/441: dread d0/d23/d27/f47 [0,4194304] 0 2024-03-22T06:15:00.491 INFO:tasks.workunit.client.0.smithi006.stdout:2/495: symlink d13/d34/d4d/d4e/d75/d62/lc1 0 2024-03-22T06:15:00.491 INFO:tasks.workunit.client.0.smithi006.stdout:2/496: write d13/d34/d4d/d4e/d75/da5/d82/f95 [267368,94259] 0 2024-03-22T06:15:00.493 INFO:tasks.workunit.client.0.smithi006.stdout:5/628: unlink d0/db/d16/d1c/lb0 0 2024-03-22T06:15:00.496 INFO:tasks.workunit.client.0.smithi006.stdout:6/407: creat d6/d7c/f8e x:0 0 0 2024-03-22T06:15:00.496 INFO:tasks.workunit.client.0.smithi006.stdout:0/311: dwrite db/d10/f1a [0,4194304] 0 2024-03-22T06:15:00.496 INFO:tasks.workunit.client.0.smithi006.stdout:0/312: chown db/dc/f44 351846832 1 2024-03-22T06:15:00.496 INFO:tasks.workunit.client.0.smithi006.stdout:0/313: write db/d10/d2a/d41/d4f/f68 [886058,60584] 0 2024-03-22T06:15:00.509 INFO:tasks.workunit.client.0.smithi006.stdout:1/362: rename d8/d9/d13/d24/d32/f41 to d8/d9/d3a/d18/f78 0 2024-03-22T06:15:00.509 INFO:tasks.workunit.client.0.smithi006.stdout:1/363: chown d8/d9/d13/l25 20454622 1 2024-03-22T06:15:00.509 INFO:tasks.workunit.client.0.smithi006.stdout:1/364: write d8/d9/d13/d24/d32/d34/f45 [4648732,21413] 0 2024-03-22T06:15:00.510 INFO:tasks.workunit.client.0.smithi006.stdout:7/537: truncate d5/d1e/d29/f5d 3354053 0 2024-03-22T06:15:00.515 INFO:tasks.workunit.client.0.smithi006.stdout:8/588: unlink d9/d14/d47/l4e 0 2024-03-22T06:15:00.515 INFO:tasks.workunit.client.0.smithi006.stdout:8/589: fsync d9/db/f7b 0 2024-03-22T06:15:00.518 INFO:tasks.workunit.client.0.smithi006.stdout:6/408: dwrite d6/d1b/d2c/f43 [0,4194304] 0 2024-03-22T06:15:00.519 INFO:tasks.workunit.client.0.smithi006.stdout:6/409: readlink d6/d77/l86 0 2024-03-22T06:15:00.519 INFO:tasks.workunit.client.0.smithi006.stdout:6/410: write d6/d1d/f20 [3828231,121922] 0 2024-03-22T06:15:00.519 INFO:tasks.workunit.client.0.smithi006.stdout:0/314: dwrite db/f45 [0,4194304] 0 2024-03-22T06:15:00.519 INFO:tasks.workunit.client.0.smithi006.stdout:3/442: getdents d0/d4/d53 0 2024-03-22T06:15:00.519 INFO:tasks.workunit.client.0.smithi006.stdout:3/443: chown d0/d8/d1c/d49 244 1 2024-03-22T06:15:00.519 INFO:tasks.workunit.client.0.smithi006.stdout:3/444: readlink d0/d7e/l73 0 2024-03-22T06:15:00.524 INFO:tasks.workunit.client.0.smithi006.stdout:2/497: creat d13/d20/da6/fc2 x:0 0 0 2024-03-22T06:15:00.528 INFO:tasks.workunit.client.0.smithi006.stdout:5/629: unlink d0/db/d16/d1c/f78 0 2024-03-22T06:15:00.528 INFO:tasks.workunit.client.0.smithi006.stdout:5/630: readlink d0/db/l26 0 2024-03-22T06:15:00.529 INFO:tasks.workunit.client.0.smithi006.stdout:0/315: write db/f45 [1821986,102861] 0 2024-03-22T06:15:00.530 INFO:tasks.workunit.client.0.smithi006.stdout:6/411: write d6/d24/f38 [283385,59518] 0 2024-03-22T06:15:00.530 INFO:tasks.workunit.client.0.smithi006.stdout:0/316: write db/d10/f1b [2922600,76789] 0 2024-03-22T06:15:00.530 INFO:tasks.workunit.client.0.smithi006.stdout:0/317: chown db/d10/d2a 63062848 1 2024-03-22T06:15:00.530 INFO:tasks.workunit.client.0.smithi006.stdout:6/412: read d6/d1d/f71 [75293,1462] 0 2024-03-22T06:15:00.532 INFO:tasks.workunit.client.0.smithi006.stdout:1/365: link d8/d9/d13/d24/l4c d8/d9/d13/d39/l79 0 2024-03-22T06:15:00.532 INFO:tasks.workunit.client.0.smithi006.stdout:1/366: rename d8/d9/d13/d39 to d8/d9/d13/d39/d60/d7a 22 2024-03-22T06:15:00.532 INFO:tasks.workunit.client.0.smithi006.stdout:1/367: fdatasync d8/d9/d3a/f20 0 2024-03-22T06:15:00.532 INFO:tasks.workunit.client.0.smithi006.stdout:3/445: dwrite d0/d23/d27/f47 [0,4194304] 0 2024-03-22T06:15:00.534 INFO:tasks.workunit.client.0.smithi006.stdout:3/446: dread d0/d23/d33/f44 [0,4194304] 0 2024-03-22T06:15:00.535 INFO:tasks.workunit.client.0.smithi006.stdout:6/413: dread d6/d1d/f71 [0,4194304] 0 2024-03-22T06:15:00.535 INFO:tasks.workunit.client.0.smithi006.stdout:6/414: fsync d6/d1d/d32/f41 0 2024-03-22T06:15:00.535 INFO:tasks.workunit.client.0.smithi006.stdout:6/415: fdatasync d6/d1b/f21 0 2024-03-22T06:15:00.535 INFO:tasks.workunit.client.0.smithi006.stdout:7/538: unlink d5/c96 0 2024-03-22T06:15:00.538 INFO:tasks.workunit.client.0.smithi006.stdout:8/590: mknod d9/d5e/cdb 0 2024-03-22T06:15:00.544 INFO:tasks.workunit.client.0.smithi006.stdout:2/498: mkdir d13/d34/d4d/d4e/d75/da5/d82/dc3 0 2024-03-22T06:15:00.548 INFO:tasks.workunit.client.0.smithi006.stdout:5/631: unlink d0/db/d16/l2a 0 2024-03-22T06:15:00.552 INFO:tasks.workunit.client.0.smithi006.stdout:1/368: mknod d8/d71/c7b 0 2024-03-22T06:15:00.558 INFO:tasks.workunit.client.0.smithi006.stdout:3/447: mkdir d0/d23/d27/d82/d97 0 2024-03-22T06:15:00.558 INFO:tasks.workunit.client.0.smithi006.stdout:3/448: fdatasync d0/d8/f17 0 2024-03-22T06:15:00.559 INFO:tasks.workunit.client.0.smithi006.stdout:6/416: symlink d6/d6f/l8f 0 2024-03-22T06:15:00.560 INFO:tasks.workunit.client.0.smithi006.stdout:6/417: dread d6/d1d/f71 [0,4194304] 0 2024-03-22T06:15:00.561 INFO:tasks.workunit.client.0.smithi006.stdout:6/418: chown d6/d24/d44/f5f 7402 1 2024-03-22T06:15:00.562 INFO:tasks.workunit.client.0.smithi006.stdout:7/539: mknod d5/d1e/d29/d10/ca9 0 2024-03-22T06:15:00.562 INFO:tasks.workunit.client.0.smithi006.stdout:7/540: chown d5/d1e/d29/d1f/d2d/c82 64911658 1 2024-03-22T06:15:00.562 INFO:tasks.workunit.client.0.smithi006.stdout:7/541: chown d5/d1e/f95 15311 1 2024-03-22T06:15:00.564 INFO:tasks.workunit.client.0.smithi006.stdout:8/591: link d9/d14/f95 d9/fdc 0 2024-03-22T06:15:00.568 INFO:tasks.workunit.client.0.smithi006.stdout:6/419: dread d6/f51 [0,4194304] 0 2024-03-22T06:15:00.569 INFO:tasks.workunit.client.0.smithi006.stdout:8/592: dread d9/db/d35/dd2/d4a/f82 [0,4194304] 0 2024-03-22T06:15:00.569 INFO:tasks.workunit.client.0.smithi006.stdout:8/593: write d9/db/d12/d1e/f6d [515754,73175] 0 2024-03-22T06:15:00.569 INFO:tasks.workunit.client.0.smithi006.stdout:8/594: truncate d9/db/dc/f3a 985115 0 2024-03-22T06:15:00.569 INFO:tasks.workunit.client.0.smithi006.stdout:8/595: stat d9/db/d35/dd2 0 2024-03-22T06:15:00.569 INFO:tasks.workunit.client.0.smithi006.stdout:8/596: dread - d9/d5e/fc2 zero size 2024-03-22T06:15:00.569 INFO:tasks.workunit.client.0.smithi006.stdout:8/597: stat d9/db/d35/dd2/c46 0 2024-03-22T06:15:00.570 INFO:tasks.workunit.client.0.smithi006.stdout:5/632: symlink d0/db/d50/d96/d52/lbf 0 2024-03-22T06:15:00.571 INFO:tasks.workunit.client.0.smithi006.stdout:0/318: getdents db/dc/d38 0 2024-03-22T06:15:00.571 INFO:tasks.workunit.client.0.smithi006.stdout:8/598: dread d9/db/dc/de/f22 [0,4194304] 0 2024-03-22T06:15:00.573 INFO:tasks.workunit.client.0.smithi006.stdout:4/518: sync 2024-03-22T06:15:00.575 INFO:tasks.workunit.client.0.smithi006.stdout:1/369: mkdir d8/d9/d10/d7c 0 2024-03-22T06:15:00.575 INFO:tasks.workunit.client.0.smithi006.stdout:1/370: truncate d8/d9/d3a/f20 8483121 0 2024-03-22T06:15:00.578 INFO:tasks.workunit.client.0.smithi006.stdout:3/449: mkdir d0/d4/d31/d98 0 2024-03-22T06:15:00.579 INFO:tasks.workunit.client.0.smithi006.stdout:8/599: dread d9/d14/d47/f70 [0,4194304] 0 2024-03-22T06:15:00.580 INFO:tasks.workunit.client.0.smithi006.stdout:1/371: fsync d8/d9/d13/d24/d32/d34/f36 0 2024-03-22T06:15:00.581 INFO:tasks.workunit.client.0.smithi006.stdout:3/450: fdatasync d0/d8/d1c/d49/f5e 0 2024-03-22T06:15:00.581 INFO:tasks.workunit.client.0.smithi006.stdout:7/542: mkdir d5/d1e/d29/d1f/d2d/d61/daa 0 2024-03-22T06:15:00.588 INFO:tasks.workunit.client.0.smithi006.stdout:2/499: mknod d13/d34/d4d/d4e/d75/da5/d6c/cc4 0 2024-03-22T06:15:00.591 INFO:tasks.workunit.client.0.smithi006.stdout:6/420: unlink d6/d1b/d2c/d4a/f84 0 2024-03-22T06:15:00.592 INFO:tasks.workunit.client.0.smithi006.stdout:0/319: truncate db/d10/d35/f43 597495 0 2024-03-22T06:15:00.594 INFO:tasks.workunit.client.0.smithi006.stdout:4/519: symlink d6/d1b/d20/d21/d2a/d43/d4b/lb7 0 2024-03-22T06:15:00.596 INFO:tasks.workunit.client.0.smithi006.stdout:9/359: sync 2024-03-22T06:15:00.598 INFO:tasks.workunit.client.0.smithi006.stdout:1/372: creat d8/d9/d3a/d40/f7d x:0 0 0 2024-03-22T06:15:00.600 INFO:tasks.workunit.client.0.smithi006.stdout:3/451: rmdir d0/d8/d88/d8e 0 2024-03-22T06:15:00.601 INFO:tasks.workunit.client.0.smithi006.stdout:0/320: dread db/f6b [0,4194304] 0 2024-03-22T06:15:00.603 INFO:tasks.workunit.client.0.smithi006.stdout:2/500: mkdir d13/d53/dc5 0 2024-03-22T06:15:00.605 INFO:tasks.workunit.client.0.smithi006.stdout:5/633: dwrite d0/db/d16/d1c/d34/d4d/f6b [0,4194304] 0 2024-03-22T06:15:00.605 INFO:tasks.workunit.client.0.smithi006.stdout:6/421: chown d6/l1a 820326119 1 2024-03-22T06:15:00.605 INFO:tasks.workunit.client.0.smithi006.stdout:8/600: dwrite f7 [4194304,4194304] 0 2024-03-22T06:15:00.608 INFO:tasks.workunit.client.0.smithi006.stdout:4/520: mkdir d6/d1b/d20/d63/d93/db8 0 2024-03-22T06:15:00.610 INFO:tasks.workunit.client.0.smithi006.stdout:2/501: write d13/d34/d4d/f5d [3261779,47111] 0 2024-03-22T06:15:00.610 INFO:tasks.workunit.client.0.smithi006.stdout:1/373: dread d8/d9/f35 [0,4194304] 0 2024-03-22T06:15:00.610 INFO:tasks.workunit.client.0.smithi006.stdout:1/374: chown d8/d9/d3a/d18/f62 37326448 1 2024-03-22T06:15:00.610 INFO:tasks.workunit.client.0.smithi006.stdout:1/375: write d8/d9/f50 [791679,2835] 0 2024-03-22T06:15:00.613 INFO:tasks.workunit.client.0.smithi006.stdout:8/601: write d9/db/d12/d66/d6c/d92/fb1 [342344,34059] 0 2024-03-22T06:15:00.614 INFO:tasks.workunit.client.0.smithi006.stdout:9/360: rename d8/fc to d8/d10/d23/f7d 0 2024-03-22T06:15:00.616 INFO:tasks.workunit.client.0.smithi006.stdout:3/452: creat d0/d8/d1c/d49/d79/f99 x:0 0 0 2024-03-22T06:15:00.616 INFO:tasks.workunit.client.0.smithi006.stdout:3/453: chown d0/d23/d33/f44 1 1 2024-03-22T06:15:00.618 INFO:tasks.workunit.client.0.smithi006.stdout:1/376: dread d8/d9/d13/f26 [0,4194304] 0 2024-03-22T06:15:00.625 INFO:tasks.workunit.client.0.smithi006.stdout:0/321: read db/d10/f14 [403683,39062] 0 2024-03-22T06:15:00.637 INFO:tasks.workunit.client.0.smithi006.stdout:6/422: truncate d6/d1d/f26 129629 0 2024-03-22T06:15:00.638 INFO:tasks.workunit.client.0.smithi006.stdout:4/521: stat d6/d1b/l5f 0 2024-03-22T06:15:00.638 INFO:tasks.workunit.client.0.smithi006.stdout:4/522: chown d6/d1b/d20/d21/d2a 957220126 1 2024-03-22T06:15:00.643 INFO:tasks.workunit.client.0.smithi006.stdout:0/322: dwrite db/d10/f1a [0,4194304] 0 2024-03-22T06:15:00.643 INFO:tasks.workunit.client.0.smithi006.stdout:0/323: fdatasync db/dc/f44 0 2024-03-22T06:15:00.646 INFO:tasks.workunit.client.0.smithi006.stdout:2/502: mknod d13/d20/d26/db9/cc6 0 2024-03-22T06:15:00.646 INFO:tasks.workunit.client.0.smithi006.stdout:2/503: readlink d13/l71 0 2024-03-22T06:15:00.652 INFO:tasks.workunit.client.0.smithi006.stdout:8/602: mkdir d9/db/d35/dd2/dc4/ddd 0 2024-03-22T06:15:00.654 INFO:tasks.workunit.client.0.smithi006.stdout:5/634: rename d0/db/d16/d1c/d97 to d0/db/d16/d1c/d34/d35/d85/d5e/d66/dc0 0 2024-03-22T06:15:00.654 INFO:tasks.workunit.client.0.smithi006.stdout:5/635: creat d0/db/d16/d1c/d34/d35/d85/fc1 x:0 0 0 2024-03-22T06:15:00.659 INFO:tasks.workunit.client.0.smithi006.stdout:9/361: getdents d8/d10/d28/d39/d75/d50/d38 0 2024-03-22T06:15:00.659 INFO:tasks.workunit.client.0.smithi006.stdout:9/362: chown d8/d10/d47/c5c 84624 1 2024-03-22T06:15:00.659 INFO:tasks.workunit.client.0.smithi006.stdout:9/363: write d8/d10/d20/d45/d49/f77 [296114,34420] 0 2024-03-22T06:15:00.660 INFO:tasks.workunit.client.0.smithi006.stdout:1/377: getdents d8/d9/d13/d24/d32 0 2024-03-22T06:15:00.669 INFO:tasks.workunit.client.0.smithi006.stdout:1/378: dread d8/d9/d3a/f20 [0,4194304] 0 2024-03-22T06:15:00.669 INFO:tasks.workunit.client.0.smithi006.stdout:1/379: truncate d8/d9/d3a/f6b 1075313 0 2024-03-22T06:15:00.695 INFO:tasks.workunit.client.0.smithi006.stdout:6/423: unlink d6/d1d/d5c/l69 0 2024-03-22T06:15:00.696 INFO:tasks.workunit.client.0.smithi006.stdout:6/424: getdents d6/d1d/d5c/d81/d8a 0 2024-03-22T06:15:00.696 INFO:tasks.workunit.client.0.smithi006.stdout:6/425: stat d6/d1b/d2c/f4c 0 2024-03-22T06:15:00.698 INFO:tasks.workunit.client.0.smithi006.stdout:4/523: creat d6/fb9 x:0 0 0 2024-03-22T06:15:00.701 INFO:tasks.workunit.client.0.smithi006.stdout:0/324: creat db/d3d/d4e/f6f x:0 0 0 2024-03-22T06:15:00.701 INFO:tasks.workunit.client.0.smithi006.stdout:0/325: truncate db/d10/d2a/d41/d4f/f68 1982943 0 2024-03-22T06:15:00.704 INFO:tasks.workunit.client.0.smithi006.stdout:2/504: rmdir d13/d8f 0 2024-03-22T06:15:00.704 INFO:tasks.workunit.client.0.smithi006.stdout:2/505: creat d13/d20/da6/fc7 x:0 0 0 2024-03-22T06:15:00.704 INFO:tasks.workunit.client.0.smithi006.stdout:2/506: creat d13/d53/fc8 x:0 0 0 2024-03-22T06:15:00.704 INFO:tasks.workunit.client.0.smithi006.stdout:2/507: truncate d13/d20/f4b 4410489 0 2024-03-22T06:15:00.709 INFO:tasks.workunit.client.0.smithi006.stdout:0/326: dread db/d10/d2a/f34 [0,4194304] 0 2024-03-22T06:15:00.709 INFO:tasks.workunit.client.0.smithi006.stdout:0/327: chown db/dc 7834000 1 2024-03-22T06:15:00.710 INFO:tasks.workunit.client.0.smithi006.stdout:5/636: chown d0/db/d76/db4 496094 1 2024-03-22T06:15:00.716 INFO:tasks.workunit.client.0.smithi006.stdout:1/380: unlink d8/d9/d3a/d18/d2a/l3d 0 2024-03-22T06:15:00.716 INFO:tasks.workunit.client.0.smithi006.stdout:1/381: write d8/d9/d3a/d18/f62 [40943,75050] 0 2024-03-22T06:15:00.728 INFO:tasks.workunit.client.0.smithi006.stdout:6/426: creat d6/d24/d44/f90 x:0 0 0 2024-03-22T06:15:00.729 INFO:tasks.workunit.client.0.smithi006.stdout:4/524: creat d6/d1b/d20/d63/fba x:0 0 0 2024-03-22T06:15:00.729 INFO:tasks.workunit.client.0.smithi006.stdout:4/525: readlink d6/d1b/d20/d21/d2a/d43/d4b/l5d 0 2024-03-22T06:15:00.737 INFO:tasks.workunit.client.0.smithi006.stdout:2/508: rename d13/d20/d26/d19/f48 to d13/d34/fc9 0 2024-03-22T06:15:00.738 INFO:tasks.workunit.client.0.smithi006.stdout:8/603: rmdir d9/db/d12/d66/d6c/d80 39 2024-03-22T06:15:00.742 INFO:tasks.workunit.client.0.smithi006.stdout:1/382: mknod d8/d9/c7e 0 2024-03-22T06:15:00.742 INFO:tasks.workunit.client.0.smithi006.stdout:1/383: readlink d8/d9/d13/d24/l70 0 2024-03-22T06:15:00.742 INFO:tasks.workunit.client.0.smithi006.stdout:1/384: write d8/d9/d13/d24/d32/d34/f77 [793478,4008] 0 2024-03-22T06:15:00.744 INFO:tasks.workunit.client.0.smithi006.stdout:6/427: mkdir d6/d1b/d2c/d4a/d91 0 2024-03-22T06:15:00.744 INFO:tasks.workunit.client.0.smithi006.stdout:6/428: dread - d6/d24/d44/f90 zero size 2024-03-22T06:15:00.744 INFO:tasks.workunit.client.0.smithi006.stdout:6/429: getdents d6/d1b/d2c/d4a/d91 0 2024-03-22T06:15:00.746 INFO:tasks.workunit.client.0.smithi006.stdout:0/328: rename c2 to db/dc/d38/c70 0 2024-03-22T06:15:00.746 INFO:tasks.workunit.client.0.smithi006.stdout:0/329: dread - db/d3d/d4e/f54 zero size 2024-03-22T06:15:00.749 INFO:tasks.workunit.client.0.smithi006.stdout:2/509: mkdir d13/d20/d26/dca 0 2024-03-22T06:15:00.749 INFO:tasks.workunit.client.0.smithi006.stdout:2/510: creat d13/d20/fcb x:0 0 0 2024-03-22T06:15:00.751 INFO:tasks.workunit.client.0.smithi006.stdout:4/526: dwrite d6/d1b/d46/f5a [0,4194304] 0 2024-03-22T06:15:00.752 INFO:tasks.workunit.client.0.smithi006.stdout:5/637: unlink d0/db/d50/f8e 0 2024-03-22T06:15:00.752 INFO:tasks.workunit.client.0.smithi006.stdout:5/638: truncate d0/db/d50/f5b 4940824 0 2024-03-22T06:15:00.754 INFO:tasks.workunit.client.0.smithi006.stdout:1/385: creat d8/d9/d13/d24/f7f x:0 0 0 2024-03-22T06:15:00.755 INFO:tasks.workunit.client.0.smithi006.stdout:1/386: readlink d8/d9/d13/d39/l52 0 2024-03-22T06:15:00.755 INFO:tasks.workunit.client.0.smithi006.stdout:1/387: write d8/d9/f1e [2842203,56274] 0 2024-03-22T06:15:00.755 INFO:tasks.workunit.client.0.smithi006.stdout:1/388: chown d8/d9/d13/d24/d32/d34 782925331 1 2024-03-22T06:15:00.758 INFO:tasks.workunit.client.0.smithi006.stdout:8/604: rename d9/db/dc/lcf to d9/db/d35/dd2/dc4/ddd/lde 0 2024-03-22T06:15:00.760 INFO:tasks.workunit.client.0.smithi006.stdout:0/330: write db/d10/d35/f69 [2322243,117073] 0 2024-03-22T06:15:00.760 INFO:tasks.workunit.client.0.smithi006.stdout:0/331: write db/f45 [2142470,122770] 0 2024-03-22T06:15:00.761 INFO:tasks.workunit.client.0.smithi006.stdout:2/511: mkdir d13/d34/d4d/d4e/d75/dcc 0 2024-03-22T06:15:00.767 INFO:tasks.workunit.client.0.smithi006.stdout:5/639: write d0/db/d16/d1c/d34/d35/d85/f55 [46698,8128] 0 2024-03-22T06:15:00.775 INFO:tasks.workunit.client.0.smithi006.stdout:4/527: rename d6/d1b/d20/d6b/d88/f98 to d6/d1b/d20/d6b/d95/fbb 0 2024-03-22T06:15:00.779 INFO:tasks.workunit.client.0.smithi006.stdout:0/332: dwrite db/d3d/d4e/f54 [0,4194304] 0 2024-03-22T06:15:00.780 INFO:tasks.workunit.client.0.smithi006.stdout:2/512: dwrite d13/d34/d4d/d4e/d75/faf [0,4194304] 0 2024-03-22T06:15:00.781 INFO:tasks.workunit.client.0.smithi006.stdout:8/605: truncate d9/db/d35/f3c 3096518 0 2024-03-22T06:15:00.788 INFO:tasks.workunit.client.0.smithi006.stdout:6/430: rmdir d6/d1b/d2c/d4a/d91 0 2024-03-22T06:15:00.792 INFO:tasks.workunit.client.0.smithi006.stdout:4/528: mkdir d6/d1b/d20/d63/d93/db8/dbc 0 2024-03-22T06:15:00.801 INFO:tasks.workunit.client.0.smithi006.stdout:2/513: rename d13/d20/d26/c1f to d13/d34/d6e/ccd 0 2024-03-22T06:15:00.802 INFO:tasks.workunit.client.0.smithi006.stdout:2/514: write d13/f92 [765826,84012] 0 2024-03-22T06:15:00.803 INFO:tasks.workunit.client.0.smithi006.stdout:0/333: symlink db/d10/d2a/d41/l71 0 2024-03-22T06:15:00.804 INFO:tasks.workunit.client.0.smithi006.stdout:0/334: fdatasync db/f64 0 2024-03-22T06:15:00.805 INFO:tasks.workunit.client.0.smithi006.stdout:8/606: creat d9/db/d12/d66/d6c/d80/fdf x:0 0 0 2024-03-22T06:15:00.808 INFO:tasks.workunit.client.0.smithi006.stdout:4/529: dread d6/d13/f17 [4194304,4194304] 0 2024-03-22T06:15:00.808 INFO:tasks.workunit.client.0.smithi006.stdout:4/530: creat d6/fbd x:0 0 0 2024-03-22T06:15:00.808 INFO:tasks.workunit.client.0.smithi006.stdout:4/531: chown d6/d1b/d20/d6b/d88/c50 27950892 1 2024-03-22T06:15:00.808 INFO:tasks.workunit.client.0.smithi006.stdout:5/640: truncate d0/db/d76/f81 1861755 0 2024-03-22T06:15:00.818 INFO:tasks.workunit.client.0.smithi006.stdout:6/431: mkdir d6/d7c/d92 0 2024-03-22T06:15:00.820 INFO:tasks.workunit.client.0.smithi006.stdout:2/515: rmdir d13/d20/d35 39 2024-03-22T06:15:00.823 INFO:tasks.workunit.client.0.smithi006.stdout:8/607: dwrite d9/db/d12/f43 [0,4194304] 0 2024-03-22T06:15:00.823 INFO:tasks.workunit.client.0.smithi006.stdout:0/335: mknod db/c72 0 2024-03-22T06:15:00.829 INFO:tasks.workunit.client.0.smithi006.stdout:5/641: mkdir d0/db/d16/d1c/d34/d35/d5d/dc2 0 2024-03-22T06:15:00.830 INFO:tasks.workunit.client.0.smithi006.stdout:4/532: dwrite d6/d1b/d20/d21/d2a/d5c/f6c [0,4194304] 0 2024-03-22T06:15:00.834 INFO:tasks.workunit.client.0.smithi006.stdout:2/516: mknod d13/d53/cce 0 2024-03-22T06:15:00.837 INFO:tasks.workunit.client.0.smithi006.stdout:2/517: dread d13/d20/d26/d19/f73 [0,4194304] 0 2024-03-22T06:15:00.841 INFO:tasks.workunit.client.0.smithi006.stdout:8/608: mknod d9/db/d35/dd2/d4a/da7/ce0 0 2024-03-22T06:15:00.843 INFO:tasks.workunit.client.0.smithi006.stdout:0/336: truncate db/d10/f14 1470324 0 2024-03-22T06:15:00.843 INFO:tasks.workunit.client.0.smithi006.stdout:0/337: chown db/dc/l16 835991 1 2024-03-22T06:15:00.850 INFO:tasks.workunit.client.0.smithi006.stdout:5/642: dwrite d0/f1a [4194304,4194304] 0 2024-03-22T06:15:00.856 INFO:tasks.workunit.client.0.smithi006.stdout:0/338: read db/dc/d38/f4b [1718611,67465] 0 2024-03-22T06:15:00.856 INFO:tasks.workunit.client.0.smithi006.stdout:0/339: stat c7 0 2024-03-22T06:15:00.858 INFO:tasks.workunit.client.0.smithi006.stdout:1/389: truncate d8/d9/f1e 4438984 0 2024-03-22T06:15:00.869 INFO:tasks.workunit.client.0.smithi006.stdout:0/340: dwrite db/d10/d2a/f5e [0,4194304] 0 2024-03-22T06:15:00.873 INFO:tasks.workunit.client.0.smithi006.stdout:2/518: unlink d13/d20/d26/d19/f1c 0 2024-03-22T06:15:00.873 INFO:tasks.workunit.client.0.smithi006.stdout:2/519: truncate d13/d34/d4d/d4e/d75/da5/d82/fa7 257571 0 2024-03-22T06:15:00.875 INFO:tasks.workunit.client.0.smithi006.stdout:8/609: creat d9/db/dc/de/d3e/fe1 x:0 0 0 2024-03-22T06:15:00.875 INFO:tasks.workunit.client.0.smithi006.stdout:8/610: stat d9/db/d35/dd2/d76 0 2024-03-22T06:15:00.882 INFO:tasks.workunit.client.0.smithi006.stdout:5/643: creat d0/db/d76/db4/fc3 x:0 0 0 2024-03-22T06:15:00.882 INFO:tasks.workunit.client.0.smithi006.stdout:5/644: chown d0/db/d16/d70 812 1 2024-03-22T06:15:00.882 INFO:tasks.workunit.client.0.smithi006.stdout:5/645: read d0/db/d16/d1c/d34/d35/d85/d5e/d66/f9b [389297,3567] 0 2024-03-22T06:15:00.884 INFO:tasks.workunit.client.0.smithi006.stdout:1/390: mkdir d8/d9/d3a/d40/d44/d80 0 2024-03-22T06:15:00.886 INFO:tasks.workunit.client.0.smithi006.stdout:6/432: rename d6/d1b/f23 to d6/d1d/d64/f93 0 2024-03-22T06:15:00.889 INFO:tasks.workunit.client.0.smithi006.stdout:8/611: dwrite d9/db/d35/dd2/d4a/d4d/f75 [0,4194304] 0 2024-03-22T06:15:00.894 INFO:tasks.workunit.client.0.smithi006.stdout:0/341: mknod db/d10/d2a/d41/c73 0 2024-03-22T06:15:00.896 INFO:tasks.workunit.client.0.smithi006.stdout:2/520: creat d13/d20/d35/fcf x:0 0 0 2024-03-22T06:15:00.896 INFO:tasks.workunit.client.0.smithi006.stdout:2/521: creat d13/d20/fd0 x:0 0 0 2024-03-22T06:15:00.900 INFO:tasks.workunit.client.0.smithi006.stdout:4/533: rename d6/d1b/d20/d21/d2a/d43/d4b/l55 to d6/d1b/d20/d21/d77/lbe 0 2024-03-22T06:15:00.900 INFO:tasks.workunit.client.0.smithi006.stdout:4/534: chown d6/l14 19226 1 2024-03-22T06:15:00.900 INFO:tasks.workunit.client.0.smithi006.stdout:4/535: stat d6/d13/d3b/c74 0 2024-03-22T06:15:00.901 INFO:tasks.workunit.client.0.smithi006.stdout:6/433: mkdir d6/d6f/d94 0 2024-03-22T06:15:00.903 INFO:tasks.workunit.client.0.smithi006.stdout:8/612: mknod d9/db/d35/dd2/d4a/d4d/ce2 0 2024-03-22T06:15:00.904 INFO:tasks.workunit.client.0.smithi006.stdout:0/342: mknod db/dc/c74 0 2024-03-22T06:15:00.909 INFO:tasks.workunit.client.0.smithi006.stdout:2/522: symlink d13/d20/ld1 0 2024-03-22T06:15:00.909 INFO:tasks.workunit.client.0.smithi006.stdout:2/523: creat d13/d20/d35/fd2 x:0 0 0 2024-03-22T06:15:00.909 INFO:tasks.workunit.client.0.smithi006.stdout:2/524: creat d13/d20/d35/d65/fd3 x:0 0 0 2024-03-22T06:15:00.912 INFO:tasks.workunit.client.0.smithi006.stdout:1/391: rename d8/d9/d13/d24/d32/c57 to d8/d9/d13/d24/c81 0 2024-03-22T06:15:00.915 INFO:tasks.workunit.client.0.smithi006.stdout:4/536: rmdir d6/d1b/d20/d21/d2a/d43 39 2024-03-22T06:15:00.915 INFO:tasks.workunit.client.0.smithi006.stdout:4/537: truncate d6/d1b/d20/d21/d77/f90 546037 0 2024-03-22T06:15:00.915 INFO:tasks.workunit.client.0.smithi006.stdout:4/538: dread - d6/d1b/d20/d63/d93/faa zero size 2024-03-22T06:15:00.915 INFO:tasks.workunit.client.0.smithi006.stdout:4/539: write d6/d13/d3b/f4e [1577091,24709] 0 2024-03-22T06:15:00.915 INFO:tasks.workunit.client.0.smithi006.stdout:4/540: creat d6/d1b/d20/d6b/d88/fbf x:0 0 0 2024-03-22T06:15:00.915 INFO:tasks.workunit.client.0.smithi006.stdout:4/541: stat d6/d1b/d20/d21/d2a/d43/f7d 0 2024-03-22T06:15:00.915 INFO:tasks.workunit.client.0.smithi006.stdout:4/542: chown d6/d1b/d20/d21/fab 132 1 2024-03-22T06:15:00.916 INFO:tasks.workunit.client.0.smithi006.stdout:1/392: dread d8/d9/f50 [0,4194304] 0 2024-03-22T06:15:00.921 INFO:tasks.workunit.client.0.smithi006.stdout:8/613: mkdir d9/db/dc/de3 0 2024-03-22T06:15:00.922 INFO:tasks.workunit.client.0.smithi006.stdout:0/343: truncate db/d10/f1a 3862911 0 2024-03-22T06:15:00.923 INFO:tasks.workunit.client.0.smithi006.stdout:1/393: dread d8/d9/f4b [0,4194304] 0 2024-03-22T06:15:00.929 INFO:tasks.workunit.client.0.smithi006.stdout:2/525: symlink d13/d34/d4d/d4e/ld4 0 2024-03-22T06:15:00.931 INFO:tasks.workunit.client.0.smithi006.stdout:4/543: dwrite d6/d1b/d20/d63/fba [0,4194304] 0 2024-03-22T06:15:00.933 INFO:tasks.workunit.client.0.smithi006.stdout:5/646: rename d0/db/d16/f49 to d0/db/d99/fc4 0 2024-03-22T06:15:00.933 INFO:tasks.workunit.client.0.smithi006.stdout:5/647: chown d0/db/d76 64252808 1 2024-03-22T06:15:00.934 INFO:tasks.workunit.client.0.smithi006.stdout:5/648: write d0/db/d50/d96/d52/d69/fa3 [938967,28276] 0 2024-03-22T06:15:00.939 INFO:tasks.workunit.client.0.smithi006.stdout:9/364: sync 2024-03-22T06:15:00.939 INFO:tasks.workunit.client.0.smithi006.stdout:7/543: sync 2024-03-22T06:15:00.939 INFO:tasks.workunit.client.0.smithi006.stdout:3/454: sync 2024-03-22T06:15:00.944 INFO:tasks.workunit.client.0.smithi006.stdout:6/434: creat d6/d1b/d2c/f95 x:0 0 0 2024-03-22T06:15:00.948 INFO:tasks.workunit.client.0.smithi006.stdout:8/614: mkdir d9/db/d35/dd2/d4a/d4d/de4 0 2024-03-22T06:15:00.955 INFO:tasks.workunit.client.0.smithi006.stdout:0/344: dwrite db/d3d/d4e/f54 [0,4194304] 0 2024-03-22T06:15:00.957 INFO:tasks.workunit.client.0.smithi006.stdout:5/649: dread - d0/db/d76/f7c zero size 2024-03-22T06:15:00.964 INFO:tasks.workunit.client.0.smithi006.stdout:3/455: symlink d0/d23/d4c/l9a 0 2024-03-22T06:15:00.967 INFO:tasks.workunit.client.0.smithi006.stdout:5/650: write d0/db/d16/d1c/d34/f73 [1140171,123430] 0 2024-03-22T06:15:00.967 INFO:tasks.workunit.client.0.smithi006.stdout:5/651: chown d0/db/d16/d1c/c24 53980 1 2024-03-22T06:15:00.967 INFO:tasks.workunit.client.0.smithi006.stdout:8/615: mknod d9/db/d12/d66/d6c/d71/ce5 0 2024-03-22T06:15:00.967 INFO:tasks.workunit.client.0.smithi006.stdout:8/616: fdatasync d9/d14/d47/f70 0 2024-03-22T06:15:00.969 INFO:tasks.workunit.client.0.smithi006.stdout:7/544: dwrite d5/d1e/d29/d66/d91/f9f [0,4194304] 0 2024-03-22T06:15:00.969 INFO:tasks.workunit.client.0.smithi006.stdout:4/544: dwrite d6/d1b/d20/d21/d77/f94 [0,4194304] 0 2024-03-22T06:15:00.969 INFO:tasks.workunit.client.0.smithi006.stdout:7/545: creat d5/d1e/d63/d85/fab x:0 0 0 2024-03-22T06:15:00.969 INFO:tasks.workunit.client.0.smithi006.stdout:7/546: chown d5/d1e/d29/d1f/d2d/f51 3 1 2024-03-22T06:15:00.970 INFO:tasks.workunit.client.0.smithi006.stdout:1/394: dwrite d8/d9/d3a/d18/f62 [0,4194304] 0 2024-03-22T06:15:00.970 INFO:tasks.workunit.client.0.smithi006.stdout:7/547: dread d5/d1e/d29/d1f/d2d/d55/f58 [0,4194304] 0 2024-03-22T06:15:00.975 INFO:tasks.workunit.client.0.smithi006.stdout:6/435: dwrite d6/d1b/d2c/f72 [0,4194304] 0 2024-03-22T06:15:00.977 INFO:tasks.workunit.client.0.smithi006.stdout:0/345: symlink db/d10/d35/l75 0 2024-03-22T06:15:00.977 INFO:tasks.workunit.client.0.smithi006.stdout:0/346: stat db/d3d/d4e/c52 0 2024-03-22T06:15:00.980 INFO:tasks.workunit.client.0.smithi006.stdout:5/652: dread d0/db/d16/d1c/d34/d35/d85/d5e/d66/dc0/fa2 [0,4194304] 0 2024-03-22T06:15:00.981 INFO:tasks.workunit.client.0.smithi006.stdout:9/365: getdents d8 0 2024-03-22T06:15:00.984 INFO:tasks.workunit.client.0.smithi006.stdout:3/456: creat d0/d8/d1c/d95/f9b x:0 0 0 2024-03-22T06:15:00.990 INFO:tasks.workunit.client.0.smithi006.stdout:4/545: dwrite d6/fb9 [0,4194304] 0 2024-03-22T06:15:00.990 INFO:tasks.workunit.client.0.smithi006.stdout:6/436: dwrite d6/d1b/d2c/f95 [0,4194304] 0 2024-03-22T06:15:00.994 INFO:tasks.workunit.client.0.smithi006.stdout:4/546: dread d6/f42 [4194304,4194304] 0 2024-03-22T06:15:00.994 INFO:tasks.workunit.client.0.smithi006.stdout:4/547: dread d6/d13/d3b/f4f [0,4194304] 0 2024-03-22T06:15:00.997 INFO:tasks.workunit.client.0.smithi006.stdout:1/395: creat d8/d9/d10/d7c/f82 x:0 0 0 2024-03-22T06:15:00.997 INFO:tasks.workunit.client.0.smithi006.stdout:1/396: dread - d8/d9/d10/d7c/f82 zero size 2024-03-22T06:15:01.000 INFO:tasks.workunit.client.0.smithi006.stdout:7/548: mkdir d5/d1e/d63/d85/dac 0 2024-03-22T06:15:01.003 INFO:tasks.workunit.client.0.smithi006.stdout:0/347: mkdir db/d3d/d4e/d76 0 2024-03-22T06:15:01.003 INFO:tasks.workunit.client.0.smithi006.stdout:0/348: chown db/dc/l20 85560180 1 2024-03-22T06:15:01.003 INFO:tasks.workunit.client.0.smithi006.stdout:0/349: creat db/dc/d37/f77 x:0 0 0 2024-03-22T06:15:01.006 INFO:tasks.workunit.client.0.smithi006.stdout:5/653: rename d0/db/d76/f7c to d0/fc5 0 2024-03-22T06:15:01.006 INFO:tasks.workunit.client.0.smithi006.stdout:5/654: chown d0/db/d16/f21 27240956 1 2024-03-22T06:15:01.008 INFO:tasks.workunit.client.0.smithi006.stdout:9/366: symlink d8/d10/d28/d39/d75/d50/d38/l7e 0 2024-03-22T06:15:01.008 INFO:tasks.workunit.client.0.smithi006.stdout:9/367: readlink d8/d10/d28/l26 0 2024-03-22T06:15:01.008 INFO:tasks.workunit.client.0.smithi006.stdout:9/368: dread - d8/d10/d28/d39/d75/d50/f76 zero size 2024-03-22T06:15:01.008 INFO:tasks.workunit.client.0.smithi006.stdout:9/369: creat d8/d10/d23/d3e/f7f x:0 0 0 2024-03-22T06:15:01.009 INFO:tasks.workunit.client.0.smithi006.stdout:1/397: dread d8/d9/d3a/d18/f1b [0,4194304] 0 2024-03-22T06:15:01.009 INFO:tasks.workunit.client.0.smithi006.stdout:1/398: dread - d8/d9/d13/d24/f7f zero size 2024-03-22T06:15:01.009 INFO:tasks.workunit.client.0.smithi006.stdout:1/399: getdents d8/d9/d3a/d40/d59 0 2024-03-22T06:15:01.010 INFO:tasks.workunit.client.0.smithi006.stdout:3/457: symlink d0/d8/d88/l9c 0 2024-03-22T06:15:01.018 INFO:tasks.workunit.client.0.smithi006.stdout:6/437: symlink d6/d24/l96 0 2024-03-22T06:15:01.020 INFO:tasks.workunit.client.0.smithi006.stdout:4/548: unlink d6/d1b/d20/d21/f7e 0 2024-03-22T06:15:01.023 INFO:tasks.workunit.client.0.smithi006.stdout:7/549: getdents d5/d1e 0 2024-03-22T06:15:01.023 INFO:tasks.workunit.client.0.smithi006.stdout:7/550: stat d5/d1e/d29/d1f/d2d/d55/d84/fa0 0 2024-03-22T06:15:01.023 INFO:tasks.workunit.client.0.smithi006.stdout:7/551: chown d5/d1e/d29/d59/c8d 211376736 1 2024-03-22T06:15:01.023 INFO:tasks.workunit.client.0.smithi006.stdout:7/552: fsync d5/f1d 0 2024-03-22T06:15:01.023 INFO:tasks.workunit.client.0.smithi006.stdout:7/553: truncate d5/d1e/d29/d59/f83 157439 0 2024-03-22T06:15:01.024 INFO:tasks.workunit.client.0.smithi006.stdout:9/370: dwrite d8/d10/d28/d39/d75/d50/f69 [0,4194304] 0 2024-03-22T06:15:01.027 INFO:tasks.workunit.client.0.smithi006.stdout:0/350: link c7 db/d10/d35/c78 0 2024-03-22T06:15:01.029 INFO:tasks.workunit.client.0.smithi006.stdout:5/655: mknod d0/db/d16/d70/cc6 0 2024-03-22T06:15:01.030 INFO:tasks.workunit.client.0.smithi006.stdout:5/656: creat d0/db/d50/d58/fc7 x:0 0 0 2024-03-22T06:15:01.036 INFO:tasks.workunit.client.0.smithi006.stdout:1/400: rename d8/d9/d3a/d18/l5e to d8/l83 0 2024-03-22T06:15:01.039 INFO:tasks.workunit.client.0.smithi006.stdout:4/549: link d6/d13/d3b/l40 d6/d1b/d20/d63/d93/d9a/lc0 0 2024-03-22T06:15:01.039 INFO:tasks.workunit.client.0.smithi006.stdout:4/550: write f3 [2532679,96649] 0 2024-03-22T06:15:01.045 INFO:tasks.workunit.client.0.smithi006.stdout:0/351: rmdir db/d3d 39 2024-03-22T06:15:01.056 INFO:tasks.workunit.client.0.smithi006.stdout:5/657: symlink d0/db/d50/d96/d52/lc8 0 2024-03-22T06:15:01.058 INFO:tasks.workunit.client.0.smithi006.stdout:3/458: rename d0/d8/d9/l15 to d0/d8/d1c/d3e/d5f/d90/l9d 0 2024-03-22T06:15:01.058 INFO:tasks.workunit.client.0.smithi006.stdout:3/459: truncate d0/d8/d1c/d3e/f8d 824900 0 2024-03-22T06:15:01.058 INFO:tasks.workunit.client.0.smithi006.stdout:3/460: fdatasync d0/d4/d3a/f7c 0 2024-03-22T06:15:01.059 INFO:tasks.workunit.client.0.smithi006.stdout:1/401: mknod d8/d9/c84 0 2024-03-22T06:15:01.060 INFO:tasks.workunit.client.0.smithi006.stdout:1/402: write d8/d9/d13/d39/d60/f6e [546634,87637] 0 2024-03-22T06:15:01.060 INFO:tasks.workunit.client.0.smithi006.stdout:1/403: fsync d8/d9/d10/f19 0 2024-03-22T06:15:01.060 INFO:tasks.workunit.client.0.smithi006.stdout:1/404: read - d8/d9/d3a/d18/d2a/f74 zero size 2024-03-22T06:15:01.062 INFO:tasks.workunit.client.0.smithi006.stdout:4/551: rmdir d6/d1b/d20/d6b/d95 39 2024-03-22T06:15:01.066 INFO:tasks.workunit.client.0.smithi006.stdout:0/352: rmdir db/d10/d2a 39 2024-03-22T06:15:01.071 INFO:tasks.workunit.client.0.smithi006.stdout:0/353: creat db/dc/d38/d50/f79 x:0 0 0 2024-03-22T06:15:01.071 INFO:tasks.workunit.client.0.smithi006.stdout:0/354: chown db/d10/d35/c78 338788090 1 2024-03-22T06:15:01.075 INFO:tasks.workunit.client.0.smithi006.stdout:6/438: rename d6/f6c to d6/d1b/f97 0 2024-03-22T06:15:01.078 INFO:tasks.workunit.client.0.smithi006.stdout:5/658: write d0/db/f1e [1673231,9492] 0 2024-03-22T06:15:01.078 INFO:tasks.workunit.client.0.smithi006.stdout:5/659: fdatasync d0/db/d50/db9/fbe 0 2024-03-22T06:15:01.078 INFO:tasks.workunit.client.0.smithi006.stdout:5/660: fsync d0/db/d50/d96/f41 0 2024-03-22T06:15:01.081 INFO:tasks.workunit.client.0.smithi006.stdout:7/554: rename f0 to d5/d1e/d63/d85/fad 0 2024-03-22T06:15:01.085 INFO:tasks.workunit.client.0.smithi006.stdout:6/439: read d6/d1b/d2c/f72 [573707,34905] 0 2024-03-22T06:15:01.085 INFO:tasks.workunit.client.0.smithi006.stdout:3/461: dwrite d0/d8/d1c/d3e/d68/f6d [0,4194304] 0 2024-03-22T06:15:01.085 INFO:tasks.workunit.client.0.smithi006.stdout:3/462: chown d0/d23/d4c/l2d 1 1 2024-03-22T06:15:01.085 INFO:tasks.workunit.client.0.smithi006.stdout:3/463: write d0/d8/d1c/f4f [8043649,14080] 0 2024-03-22T06:15:01.087 INFO:tasks.workunit.client.0.smithi006.stdout:4/552: dwrite d6/d13/d3b/f51 [0,4194304] 0 2024-03-22T06:15:01.090 INFO:tasks.workunit.client.0.smithi006.stdout:9/371: rename d8/d1f/c67 to d8/d10/d11/c80 0 2024-03-22T06:15:01.090 INFO:tasks.workunit.client.0.smithi006.stdout:9/372: write f1 [3314730,1206] 0 2024-03-22T06:15:01.092 INFO:tasks.workunit.client.0.smithi006.stdout:6/440: mkdir d6/d1d/d5c/d81/d98 0 2024-03-22T06:15:01.096 INFO:tasks.workunit.client.0.smithi006.stdout:3/464: symlink d0/d4/d31/l9e 0 2024-03-22T06:15:01.101 INFO:tasks.workunit.client.0.smithi006.stdout:7/555: dwrite d5/f49 [0,4194304] 0 2024-03-22T06:15:01.101 INFO:tasks.workunit.client.0.smithi006.stdout:7/556: chown d5/d1e/d3f/f5a 920 1 2024-03-22T06:15:01.101 INFO:tasks.workunit.client.0.smithi006.stdout:7/557: fsync d5/d1e/d63/d85/f98 0 2024-03-22T06:15:01.102 INFO:tasks.workunit.client.0.smithi006.stdout:4/553: write d6/d1b/d20/d21/d2a/d5c/f66 [3946675,3865] 0 2024-03-22T06:15:01.102 INFO:tasks.workunit.client.0.smithi006.stdout:5/661: dwrite d0/db/d16/d1c/d34/f4b [0,4194304] 0 2024-03-22T06:15:01.103 INFO:tasks.workunit.client.0.smithi006.stdout:1/405: rename d8/c15 to d8/d9/c85 0 2024-03-22T06:15:01.103 INFO:tasks.workunit.client.0.smithi006.stdout:1/406: read d8/d9/d3a/d18/f78 [2050243,97521] 0 2024-03-22T06:15:01.106 INFO:tasks.workunit.client.0.smithi006.stdout:9/373: mknod d8/d10/d11/c81 0 2024-03-22T06:15:01.107 INFO:tasks.workunit.client.0.smithi006.stdout:6/441: mkdir d6/d1d/d99 0 2024-03-22T06:15:01.110 INFO:tasks.workunit.client.0.smithi006.stdout:3/465: truncate d0/d23/d33/d3d/f5b 335539 0 2024-03-22T06:15:01.110 INFO:tasks.workunit.client.0.smithi006.stdout:3/466: creat d0/d8/d1c/d3e/d5f/f9f x:0 0 0 2024-03-22T06:15:01.110 INFO:tasks.workunit.client.0.smithi006.stdout:3/467: creat d0/d7e/fa0 x:0 0 0 2024-03-22T06:15:01.110 INFO:tasks.workunit.client.0.smithi006.stdout:3/468: creat d0/d8/d1c/d3e/d5f/fa1 x:0 0 0 2024-03-22T06:15:01.110 INFO:tasks.workunit.client.0.smithi006.stdout:3/469: stat d0/d8/d1c/f4f 0 2024-03-22T06:15:01.110 INFO:tasks.workunit.client.0.smithi006.stdout:3/470: truncate d0/d8/d1c/d72/f94 512372 0 2024-03-22T06:15:01.110 INFO:tasks.workunit.client.0.smithi006.stdout:3/471: creat d0/d23/d4c/fa2 x:0 0 0 2024-03-22T06:15:01.110 INFO:tasks.workunit.client.0.smithi006.stdout:3/472: fdatasync d0/d8/d1c/d72/f94 0 2024-03-22T06:15:01.114 INFO:tasks.workunit.client.0.smithi006.stdout:7/558: symlink d5/d1e/d29/d1f/d2d/d61/daa/lae 0 2024-03-22T06:15:01.117 INFO:tasks.workunit.client.0.smithi006.stdout:4/554: symlink d6/d1b/d20/d63/d93/d9a/d8f/da1/lc1 0 2024-03-22T06:15:01.117 INFO:tasks.workunit.client.0.smithi006.stdout:4/555: dread - d6/d1b/d20/d63/d93/d9a/d8f/da1/fb6 zero size 2024-03-22T06:15:01.117 INFO:tasks.workunit.client.0.smithi006.stdout:4/556: fsync d6/fbd 0 2024-03-22T06:15:01.117 INFO:tasks.workunit.client.0.smithi006.stdout:3/473: dread d0/d8/f17 [0,4194304] 0 2024-03-22T06:15:01.118 INFO:tasks.workunit.client.0.smithi006.stdout:4/557: dread d6/d1b/d20/d21/d77/f90 [0,4194304] 0 2024-03-22T06:15:01.122 INFO:tasks.workunit.client.0.smithi006.stdout:1/407: truncate d8/d9/f54 3189140 0 2024-03-22T06:15:01.122 INFO:tasks.workunit.client.0.smithi006.stdout:7/559: dread d5/d1e/d3f/f81 [0,4194304] 0 2024-03-22T06:15:01.124 INFO:tasks.workunit.client.0.smithi006.stdout:0/355: rename db/d10/c17 to db/dc/c7a 0 2024-03-22T06:15:01.134 INFO:tasks.workunit.client.0.smithi006.stdout:9/374: mkdir d8/d10/d82 0 2024-03-22T06:15:01.134 INFO:tasks.workunit.client.0.smithi006.stdout:9/375: write d8/d10/d28/f25 [5026272,99673] 0 2024-03-22T06:15:01.134 INFO:tasks.workunit.client.0.smithi006.stdout:9/376: chown d8/d1f/d34/f70 4129241 1 2024-03-22T06:15:01.135 INFO:tasks.workunit.client.0.smithi006.stdout:6/442: truncate d6/d24/d44/f5f 3811245 0 2024-03-22T06:15:01.145 INFO:tasks.workunit.client.0.smithi006.stdout:7/560: dwrite d5/f56 [0,4194304] 0 2024-03-22T06:15:01.150 INFO:tasks.workunit.client.0.smithi006.stdout:1/408: mknod d8/d9/d3a/c86 0 2024-03-22T06:15:01.152 INFO:tasks.workunit.client.0.smithi006.stdout:3/474: rename d0/d23/d27 to d0/d8/d1c/d3e/d5f/da3 0 2024-03-22T06:15:01.152 INFO:tasks.workunit.client.0.smithi006.stdout:3/475: fsync d0/d23/d33/d3d/f86 0 2024-03-22T06:15:01.152 INFO:tasks.workunit.client.0.smithi006.stdout:3/476: creat d0/d23/d4c/fa4 x:0 0 0 2024-03-22T06:15:01.152 INFO:tasks.workunit.client.0.smithi006.stdout:3/477: read d0/d8/d1c/d3e/d68/f6b [334729,107429] 0 2024-03-22T06:15:01.154 INFO:tasks.workunit.client.0.smithi006.stdout:0/356: mkdir db/d10/d35/d7b 0 2024-03-22T06:15:01.154 INFO:tasks.workunit.client.0.smithi006.stdout:0/357: write db/d10/f2f [2407823,2894] 0 2024-03-22T06:15:01.154 INFO:tasks.workunit.client.0.smithi006.stdout:0/358: dread - db/f64 zero size 2024-03-22T06:15:01.158 INFO:tasks.workunit.client.0.smithi006.stdout:9/377: mknod d8/d1f/c83 0 2024-03-22T06:15:01.158 INFO:tasks.workunit.client.0.smithi006.stdout:9/378: chown d8/d10/d20/d45/d49/l78 4054 1 2024-03-22T06:15:01.162 INFO:tasks.workunit.client.0.smithi006.stdout:0/359: write db/d10/d35/f53 [3784060,32917] 0 2024-03-22T06:15:01.162 INFO:tasks.workunit.client.0.smithi006.stdout:6/443: symlink d6/d1d/d5c/d89/l9a 0 2024-03-22T06:15:01.164 INFO:tasks.workunit.client.0.smithi006.stdout:7/561: symlink d5/d1e/d3f/laf 0 2024-03-22T06:15:01.164 INFO:tasks.workunit.client.0.smithi006.stdout:7/562: read d5/d1e/f3a [2121351,93052] 0 2024-03-22T06:15:01.168 INFO:tasks.workunit.client.0.smithi006.stdout:3/478: dwrite d0/d8/d1c/d3e/d5f/f7d [0,4194304] 0 2024-03-22T06:15:01.170 INFO:tasks.workunit.client.0.smithi006.stdout:3/479: dread d0/d23/d33/f44 [0,4194304] 0 2024-03-22T06:15:01.173 INFO:tasks.workunit.client.0.smithi006.stdout:4/558: rmdir d6/d1b/d20/d21 39 2024-03-22T06:15:01.174 INFO:tasks.workunit.client.0.smithi006.stdout:4/559: write d6/d1b/d20/d6b/d60/f68 [389404,121142] 0 2024-03-22T06:15:01.175 INFO:tasks.workunit.client.0.smithi006.stdout:1/409: rename d8/d9/d3a/d40/d44/l2f to d8/d9/d3a/d18/l87 0 2024-03-22T06:15:01.175 INFO:tasks.workunit.client.0.smithi006.stdout:1/410: chown d8/d9/l2e 240637 1 2024-03-22T06:15:01.181 INFO:tasks.workunit.client.0.smithi006.stdout:9/379: unlink d8/d1f/d34/f79 0 2024-03-22T06:15:01.181 INFO:tasks.workunit.client.0.smithi006.stdout:9/380: chown d8/d10/d20/c4c 1610 1 2024-03-22T06:15:01.188 INFO:tasks.workunit.client.0.smithi006.stdout:7/563: symlink d5/d1e/d29/d1f/d2d/lb0 0 2024-03-22T06:15:01.195 INFO:tasks.workunit.client.0.smithi006.stdout:3/480: creat d0/d8/d1c/d3e/d5f/da3/d82/d97/fa5 x:0 0 0 2024-03-22T06:15:01.196 INFO:tasks.workunit.client.0.smithi006.stdout:4/560: mknod d6/d1b/d20/d6b/d60/db1/cc2 0 2024-03-22T06:15:01.199 INFO:tasks.workunit.client.0.smithi006.stdout:0/360: rename db/l39 to db/d10/d2a/d41/d4f/l7c 0 2024-03-22T06:15:01.199 INFO:tasks.workunit.client.0.smithi006.stdout:0/361: creat db/d3d/d4e/f7d x:0 0 0 2024-03-22T06:15:01.200 INFO:tasks.workunit.client.0.smithi006.stdout:1/411: creat d8/d9/d3a/d40/d44/d80/f88 x:0 0 0 2024-03-22T06:15:01.203 INFO:tasks.workunit.client.0.smithi006.stdout:9/381: creat d8/d10/d28/d39/d75/d50/f84 x:0 0 0 2024-03-22T06:15:01.203 INFO:tasks.workunit.client.0.smithi006.stdout:9/382: chown d8/d10/d28/d39/d75/d50/d38/c3b 10 1 2024-03-22T06:15:01.205 INFO:tasks.workunit.client.0.smithi006.stdout:4/561: dread d6/d1b/d20/d6b/f75 [0,4194304] 0 2024-03-22T06:15:01.210 INFO:tasks.workunit.client.0.smithi006.stdout:7/564: creat d5/d1e/d29/d59/d5c/fb1 x:0 0 0 2024-03-22T06:15:01.211 INFO:tasks.workunit.client.0.smithi006.stdout:4/562: dread f2 [4194304,4194304] 0 2024-03-22T06:15:01.211 INFO:tasks.workunit.client.0.smithi006.stdout:4/563: chown d6/d1b/d20/d6b/d88/c96 74747 1 2024-03-22T06:15:01.211 INFO:tasks.workunit.client.0.smithi006.stdout:4/564: chown f3 3930956 1 2024-03-22T06:15:01.213 INFO:tasks.workunit.client.0.smithi006.stdout:3/481: mkdir d0/d23/d33/da6 0 2024-03-22T06:15:01.213 INFO:tasks.workunit.client.0.smithi006.stdout:3/482: chown d0/d8/d1c/d3e/d5f/f92 0 1 2024-03-22T06:15:01.220 INFO:tasks.workunit.client.0.smithi006.stdout:3/483: write d0/d23/d33/f4d [2375031,94296] 0 2024-03-22T06:15:01.224 INFO:tasks.workunit.client.0.smithi006.stdout:0/362: creat db/dc/f7e x:0 0 0 2024-03-22T06:15:01.225 INFO:tasks.workunit.client.0.smithi006.stdout:9/383: dwrite d8/d10/d28/d39/f46 [0,4194304] 0 2024-03-22T06:15:01.225 INFO:tasks.workunit.client.0.smithi006.stdout:1/412: dwrite d8/d9/d3a/d40/d44/d80/f88 [0,4194304] 0 2024-03-22T06:15:01.227 INFO:tasks.workunit.client.0.smithi006.stdout:7/565: mknod d5/d1e/d63/d85/d80/cb2 0 2024-03-22T06:15:01.227 INFO:tasks.workunit.client.0.smithi006.stdout:7/566: fsync d5/f56 0 2024-03-22T06:15:01.233 INFO:tasks.workunit.client.0.smithi006.stdout:0/363: creat db/d10/d2a/d41/f7f x:0 0 0 2024-03-22T06:15:01.233 INFO:tasks.workunit.client.0.smithi006.stdout:0/364: dread - db/dc/d37/f77 zero size 2024-03-22T06:15:01.233 INFO:tasks.workunit.client.0.smithi006.stdout:0/365: fsync db/d3d/d4e/f54 0 2024-03-22T06:15:01.238 INFO:tasks.workunit.client.0.smithi006.stdout:9/384: unlink d8/d10/f72 0 2024-03-22T06:15:01.239 INFO:tasks.workunit.client.0.smithi006.stdout:9/385: write f4 [4751944,16396] 0 2024-03-22T06:15:01.240 INFO:tasks.workunit.client.0.smithi006.stdout:3/484: rename d0/d8/d1c/d3e/d5f/da3/d82 to d0/d8/d1c/d3e/d5f/d90/da7 0 2024-03-22T06:15:01.240 INFO:tasks.workunit.client.0.smithi006.stdout:3/485: chown d0/d8/d1c/d3e/d5f/d90/da7 213513 1 2024-03-22T06:15:01.240 INFO:tasks.workunit.client.0.smithi006.stdout:3/486: fdatasync d0/d8/f13 0 2024-03-22T06:15:01.241 INFO:tasks.workunit.client.0.smithi006.stdout:9/386: dread f3 [0,4194304] 0 2024-03-22T06:15:01.246 INFO:tasks.workunit.client.0.smithi006.stdout:7/567: mkdir d5/d1e/d63/d85/dac/db3 0 2024-03-22T06:15:01.251 INFO:tasks.workunit.client.0.smithi006.stdout:0/366: unlink db/d10/d2a/l40 0 2024-03-22T06:15:01.251 INFO:tasks.workunit.client.0.smithi006.stdout:0/367: chown db/d3d/l3f 49769 1 2024-03-22T06:15:01.251 INFO:tasks.workunit.client.0.smithi006.stdout:0/368: chown db/l32 12 1 2024-03-22T06:15:01.257 INFO:tasks.workunit.client.0.smithi006.stdout:3/487: getdents d0/d8/d9 0 2024-03-22T06:15:01.260 INFO:tasks.workunit.client.0.smithi006.stdout:6/444: write d6/d1b/d2c/f8c [844534,100700] 0 2024-03-22T06:15:01.261 INFO:tasks.workunit.client.0.smithi006.stdout:0/369: unlink db/d10/c11 0 2024-03-22T06:15:01.264 INFO:tasks.workunit.client.0.smithi006.stdout:3/488: dread d0/d8/f13 [0,4194304] 0 2024-03-22T06:15:01.264 INFO:tasks.workunit.client.0.smithi006.stdout:6/445: chown d6/f11 97 1 2024-03-22T06:15:01.272 INFO:tasks.workunit.client.0.smithi006.stdout:3/489: dread d0/d23/d33/f4d [0,4194304] 0 2024-03-22T06:15:01.272 INFO:tasks.workunit.client.0.smithi006.stdout:3/490: stat d0/d8/d1c/d3e/d5f/da3/f47 0 2024-03-22T06:15:01.274 INFO:tasks.workunit.client.0.smithi006.stdout:3/491: truncate d0/d23/f7f 3718717 0 2024-03-22T06:15:01.274 INFO:tasks.workunit.client.0.smithi006.stdout:3/492: write d0/d8/d1c/d3e/d5f/f9f [79890,43553] 0 2024-03-22T06:15:01.275 INFO:tasks.workunit.client.0.smithi006.stdout:6/446: dread d6/d1b/d2c/f43 [0,4194304] 0 2024-03-22T06:15:01.276 INFO:tasks.workunit.client.0.smithi006.stdout:9/387: dwrite d8/d10/d28/f25 [0,4194304] 0 2024-03-22T06:15:01.277 INFO:tasks.workunit.client.0.smithi006.stdout:7/568: dwrite d5/d1e/d29/d59/d5c/fb1 [0,4194304] 0 2024-03-22T06:15:01.277 INFO:tasks.workunit.client.0.smithi006.stdout:0/370: dwrite db/d10/f23 [4194304,4194304] 0 2024-03-22T06:15:01.277 INFO:tasks.workunit.client.0.smithi006.stdout:0/371: creat db/f80 x:0 0 0 2024-03-22T06:15:01.278 INFO:tasks.workunit.client.0.smithi006.stdout:3/493: mknod d0/d23/d33/d3d/d58/ca8 0 2024-03-22T06:15:01.279 INFO:tasks.workunit.client.0.smithi006.stdout:9/388: write d8/d10/d28/f25 [4663537,10638] 0 2024-03-22T06:15:01.279 INFO:tasks.workunit.client.0.smithi006.stdout:9/389: read f6 [4356056,19311] 0 2024-03-22T06:15:01.279 INFO:tasks.workunit.client.0.smithi006.stdout:9/390: creat d8/d10/d20/d45/d4d/f85 x:0 0 0 2024-03-22T06:15:01.280 INFO:tasks.workunit.client.0.smithi006.stdout:3/494: dread d0/d23/d4c/f7b [0,4194304] 0 2024-03-22T06:15:01.281 INFO:tasks.workunit.client.0.smithi006.stdout:3/495: write d0/d8/d1c/d3e/d68/f6b [706112,54848] 0 2024-03-22T06:15:01.281 INFO:tasks.workunit.client.0.smithi006.stdout:3/496: write d0/d8/d1c/d3e/f8d [1455587,63719] 0 2024-03-22T06:15:01.282 INFO:tasks.workunit.client.0.smithi006.stdout:3/497: chown d0/d8/d1c/d3e/d5f/l4a 16702 1 2024-03-22T06:15:01.282 INFO:tasks.workunit.client.0.smithi006.stdout:3/498: stat d0/d23/d33/d3d/f6e 0 2024-03-22T06:15:01.282 INFO:tasks.workunit.client.0.smithi006.stdout:3/499: write d0/d8/f1a [4258377,124832] 0 2024-03-22T06:15:01.290 INFO:tasks.workunit.client.0.smithi006.stdout:6/447: dread d6/f13 [0,4194304] 0 2024-03-22T06:15:01.290 INFO:tasks.workunit.client.0.smithi006.stdout:6/448: fdatasync d6/d1b/d2c/f7d 0 2024-03-22T06:15:01.290 INFO:tasks.workunit.client.0.smithi006.stdout:6/449: fdatasync d6/d24/d44/f46 0 2024-03-22T06:15:01.290 INFO:tasks.workunit.client.0.smithi006.stdout:7/569: symlink d5/d1e/d63/lb4 0 2024-03-22T06:15:01.298 INFO:tasks.workunit.client.0.smithi006.stdout:3/500: unlink d0/d8/d1c/d3e/d5f/c78 0 2024-03-22T06:15:01.300 INFO:tasks.workunit.client.0.smithi006.stdout:6/450: chown d6/d1b/f82 11 1 2024-03-22T06:15:01.300 INFO:tasks.workunit.client.0.smithi006.stdout:3/501: fdatasync d0/d8/d1c/d3e/d5f/f9f 0 2024-03-22T06:15:01.302 INFO:tasks.workunit.client.0.smithi006.stdout:7/570: symlink d5/d1e/d3f/d94/lb5 0 2024-03-22T06:15:01.303 INFO:tasks.workunit.client.0.smithi006.stdout:0/372: dwrite db/d10/d2a/d41/f7f [0,4194304] 0 2024-03-22T06:15:01.303 INFO:tasks.workunit.client.0.smithi006.stdout:9/391: dwrite d8/d10/d11/f59 [0,4194304] 0 2024-03-22T06:15:01.305 INFO:tasks.workunit.client.0.smithi006.stdout:6/451: write d6/f13 [334473,6522] 0 2024-03-22T06:15:01.307 INFO:tasks.workunit.client.0.smithi006.stdout:7/571: creat d5/d1e/d3f/fb6 x:0 0 0 2024-03-22T06:15:01.307 INFO:tasks.workunit.client.0.smithi006.stdout:7/572: chown d5/d1e/d63/d85/d80/cb2 37561025 1 2024-03-22T06:15:01.313 INFO:tasks.workunit.client.0.smithi006.stdout:6/452: dread d6/d1b/f97 [0,4194304] 0 2024-03-22T06:15:01.313 INFO:tasks.workunit.client.0.smithi006.stdout:6/453: truncate d6/f51 4670435 0 2024-03-22T06:15:01.313 INFO:tasks.workunit.client.0.smithi006.stdout:0/373: mkdir db/d3d/d4e/d76/d81 0 2024-03-22T06:15:01.320 INFO:tasks.workunit.client.0.smithi006.stdout:9/392: symlink d8/d10/d23/l86 0 2024-03-22T06:15:01.322 INFO:tasks.workunit.client.0.smithi006.stdout:7/573: truncate d5/f9 1770709 0 2024-03-22T06:15:01.327 INFO:tasks.workunit.client.0.smithi006.stdout:6/454: getdents d6/d24 0 2024-03-22T06:15:01.331 INFO:tasks.workunit.client.0.smithi006.stdout:0/374: unlink db/d10/d33/c3c 0 2024-03-22T06:15:01.331 INFO:tasks.workunit.client.0.smithi006.stdout:0/375: chown db/l46 14 1 2024-03-22T06:15:01.331 INFO:tasks.workunit.client.0.smithi006.stdout:0/376: creat db/d10/d2a/d41/f82 x:0 0 0 2024-03-22T06:15:01.331 INFO:tasks.workunit.client.0.smithi006.stdout:0/377: write db/d3d/d4e/f7d [669908,1666] 0 2024-03-22T06:15:01.331 INFO:tasks.workunit.client.0.smithi006.stdout:3/502: dwrite d0/d23/f7f [0,4194304] 0 2024-03-22T06:15:01.332 INFO:tasks.workunit.client.0.smithi006.stdout:9/393: creat d8/d10/d47/f87 x:0 0 0 2024-03-22T06:15:01.332 INFO:tasks.workunit.client.0.smithi006.stdout:9/394: readlink d8/d10/l30 0 2024-03-22T06:15:01.332 INFO:tasks.workunit.client.0.smithi006.stdout:9/395: write d8/d1f/d34/f70 [760505,11292] 0 2024-03-22T06:15:01.333 INFO:tasks.workunit.client.0.smithi006.stdout:9/396: creat d8/d1f/d34/f88 x:0 0 0 2024-03-22T06:15:01.333 INFO:tasks.workunit.client.0.smithi006.stdout:9/397: chown d8/d10/d28/f57 0 1 2024-03-22T06:15:01.333 INFO:tasks.workunit.client.0.smithi006.stdout:9/398: getdents d8/d10/d82 0 2024-03-22T06:15:01.334 INFO:tasks.workunit.client.0.smithi006.stdout:6/455: creat d6/d6f/f9b x:0 0 0 2024-03-22T06:15:01.339 INFO:tasks.workunit.client.0.smithi006.stdout:3/503: creat d0/d4/fa9 x:0 0 0 2024-03-22T06:15:01.341 INFO:tasks.workunit.client.0.smithi006.stdout:7/574: dwrite d5/d1e/d63/d85/da1/fa4 [0,4194304] 0 2024-03-22T06:15:01.346 INFO:tasks.workunit.client.0.smithi006.stdout:6/456: fsync d6/d1b/f3c 0 2024-03-22T06:15:01.346 INFO:tasks.workunit.client.0.smithi006.stdout:6/457: fdatasync d6/d24/d44/f90 0 2024-03-22T06:15:01.346 INFO:tasks.workunit.client.0.smithi006.stdout:6/458: chown d6/d1d/d64/f6a 1375005 1 2024-03-22T06:15:01.348 INFO:tasks.workunit.client.0.smithi006.stdout:0/378: symlink db/dc/d38/l83 0 2024-03-22T06:15:01.350 INFO:tasks.workunit.client.0.smithi006.stdout:3/504: truncate d0/fa 918773 0 2024-03-22T06:15:01.350 INFO:tasks.workunit.client.0.smithi006.stdout:3/505: truncate d0/d8/d1c/d3e/d5f/fa1 361703 0 2024-03-22T06:15:01.351 INFO:tasks.workunit.client.0.smithi006.stdout:7/575: write d5/d1e/d3f/f4b [327082,120318] 0 2024-03-22T06:15:01.354 INFO:tasks.workunit.client.0.smithi006.stdout:7/576: dread d5/d1e/d29/d1f/d2d/f51 [0,4194304] 0 2024-03-22T06:15:01.377 INFO:tasks.workunit.client.0.smithi006.stdout:7/577: dwrite d5/d1e/d3f/f76 [0,4194304] 0 2024-03-22T06:15:01.389 INFO:tasks.workunit.client.0.smithi006.stdout:9/399: rmdir d8/d10 39 2024-03-22T06:15:01.393 INFO:tasks.workunit.client.0.smithi006.stdout:6/459: creat d6/f9c x:0 0 0 2024-03-22T06:15:01.393 INFO:tasks.workunit.client.0.smithi006.stdout:6/460: chown d6/d1d/d5c/d81/c55 5668182 1 2024-03-22T06:15:01.401 INFO:tasks.workunit.client.0.smithi006.stdout:3/506: mknod d0/d8/d1c/d95/caa 0 2024-03-22T06:15:01.407 INFO:tasks.workunit.client.0.smithi006.stdout:9/400: dwrite f4 [0,4194304] 0 2024-03-22T06:15:01.407 INFO:tasks.workunit.client.0.smithi006.stdout:9/401: fdatasync d8/d10/d11/f59 0 2024-03-22T06:15:01.407 INFO:tasks.workunit.client.0.smithi006.stdout:9/402: read d8/d10/d28/f3d [2433417,4348] 0 2024-03-22T06:15:01.407 INFO:tasks.workunit.client.0.smithi006.stdout:9/403: chown d8/d10/d28/f3d 1 1 2024-03-22T06:15:01.407 INFO:tasks.workunit.client.0.smithi006.stdout:7/578: mkdir d5/d1e/d29/d1f/d2d/d41/db7 0 2024-03-22T06:15:01.408 INFO:tasks.workunit.client.0.smithi006.stdout:7/579: dread - d5/d1e/d63/d85/fa6 zero size 2024-03-22T06:15:01.408 INFO:tasks.workunit.client.0.smithi006.stdout:7/580: stat d5/d1e/d29/f54 0 2024-03-22T06:15:01.409 INFO:tasks.workunit.client.0.smithi006.stdout:6/461: symlink d6/d7c/d83/l9d 0 2024-03-22T06:15:01.410 INFO:tasks.workunit.client.0.smithi006.stdout:0/379: mknod db/d10/c84 0 2024-03-22T06:15:01.413 INFO:tasks.workunit.client.0.smithi006.stdout:3/507: rmdir d0/d8 39 2024-03-22T06:15:01.413 INFO:tasks.workunit.client.0.smithi006.stdout:3/508: truncate d0/d4/d3a/f7c 434395 0 2024-03-22T06:15:01.421 INFO:tasks.workunit.client.0.smithi006.stdout:9/404: mkdir d8/d10/d23/d89 0 2024-03-22T06:15:01.421 INFO:tasks.workunit.client.0.smithi006.stdout:9/405: write d8/f64 [968350,51021] 0 2024-03-22T06:15:01.421 INFO:tasks.workunit.client.0.smithi006.stdout:9/406: chown d8/d10/d23/d3e/f7f 0 1 2024-03-22T06:15:01.424 INFO:tasks.workunit.client.0.smithi006.stdout:3/509: rmdir d0/d23/d33/da6 0 2024-03-22T06:15:01.426 INFO:tasks.workunit.client.0.smithi006.stdout:3/510: write d0/d23/d33/d3d/f6e [7770,109074] 0 2024-03-22T06:15:01.427 INFO:tasks.workunit.client.0.smithi006.stdout:9/407: mknod d8/d1f/c8a 0 2024-03-22T06:15:01.430 INFO:tasks.workunit.client.0.smithi006.stdout:6/462: getdents d6/d1b/d2c/d4a 0 2024-03-22T06:15:01.432 INFO:tasks.workunit.client.0.smithi006.stdout:7/581: dwrite d5/d1e/d29/d1f/f3e [0,4194304] 0 2024-03-22T06:15:01.432 INFO:tasks.workunit.client.0.smithi006.stdout:7/582: creat d5/d1e/d29/d1f/d2d/d55/fb8 x:0 0 0 2024-03-22T06:15:01.432 INFO:tasks.workunit.client.0.smithi006.stdout:7/583: chown d5/d1e/d29/d1f/d2d/d61/f6d 308247 1 2024-03-22T06:15:01.433 INFO:tasks.workunit.client.0.smithi006.stdout:3/511: mkdir d0/d4/d31/d98/dab 0 2024-03-22T06:15:01.435 INFO:tasks.workunit.client.0.smithi006.stdout:0/380: dwrite db/d10/f23 [8388608,4194304] 0 2024-03-22T06:15:01.436 INFO:tasks.workunit.client.0.smithi006.stdout:0/381: fsync db/d10/d2a/d41/f82 0 2024-03-22T06:15:01.440 INFO:tasks.workunit.client.0.smithi006.stdout:3/512: mknod d0/d8/d1c/d3e/d5f/d83/d8c/cac 0 2024-03-22T06:15:01.443 INFO:tasks.workunit.client.0.smithi006.stdout:9/408: dwrite d8/d10/f2e [0,4194304] 0 2024-03-22T06:15:01.444 INFO:tasks.workunit.client.0.smithi006.stdout:6/463: getdents d6/d1d 0 2024-03-22T06:15:01.444 INFO:tasks.workunit.client.0.smithi006.stdout:6/464: truncate d6/d1b/f3c 1240080 0 2024-03-22T06:15:01.444 INFO:tasks.workunit.client.0.smithi006.stdout:6/465: stat d6/d1b/d2c/d4a 0 2024-03-22T06:15:01.446 INFO:tasks.workunit.client.0.smithi006.stdout:2/526: sync 2024-03-22T06:15:01.446 INFO:tasks.workunit.client.0.smithi006.stdout:5/662: sync 2024-03-22T06:15:01.446 INFO:tasks.workunit.client.0.smithi006.stdout:8/617: sync 2024-03-22T06:15:01.453 INFO:tasks.workunit.client.0.smithi006.stdout:0/382: getdents db/dc/d38 0 2024-03-22T06:15:01.453 INFO:tasks.workunit.client.0.smithi006.stdout:0/383: fdatasync db/d3d/d4e/f65 0 2024-03-22T06:15:01.453 INFO:tasks.workunit.client.0.smithi006.stdout:0/384: write db/d3d/d4e/f65 [187488,33821] 0 2024-03-22T06:15:01.454 INFO:tasks.workunit.client.0.smithi006.stdout:0/385: dread - db/d10/d33/f67 zero size 2024-03-22T06:15:01.456 INFO:tasks.workunit.client.0.smithi006.stdout:3/513: dwrite d0/d8/d1c/d3e/f96 [0,4194304] 0 2024-03-22T06:15:01.458 INFO:tasks.workunit.client.0.smithi006.stdout:9/409: rename d8/d10/d11/f60 to d8/d10/d82/f8b 0 2024-03-22T06:15:01.458 INFO:tasks.workunit.client.0.smithi006.stdout:9/410: chown d8/d10/d11/c32 0 1 2024-03-22T06:15:01.462 INFO:tasks.workunit.client.0.smithi006.stdout:2/527: chown d13/d20/f74 1052082 1 2024-03-22T06:15:01.462 INFO:tasks.workunit.client.0.smithi006.stdout:2/528: truncate d13/d34/f8c 750821 0 2024-03-22T06:15:01.462 INFO:tasks.workunit.client.0.smithi006.stdout:2/529: fsync d13/d20/d35/fa2 0 2024-03-22T06:15:01.462 INFO:tasks.workunit.client.0.smithi006.stdout:2/530: readlink d13/d53/l9e 0 2024-03-22T06:15:01.463 INFO:tasks.workunit.client.0.smithi006.stdout:2/531: fdatasync d13/d34/d4d/d4e/d75/d62/fbd 0 2024-03-22T06:15:01.463 INFO:tasks.workunit.client.0.smithi006.stdout:2/532: write d13/d20/fd0 [180225,13398] 0 2024-03-22T06:15:01.463 INFO:tasks.workunit.client.0.smithi006.stdout:2/533: write d13/d34/d4d/d4e/d75/da5/d6c/f78 [236090,72450] 0 2024-03-22T06:15:01.465 INFO:tasks.workunit.client.0.smithi006.stdout:5/663: creat d0/db/d50/d96/d52/fc9 x:0 0 0 2024-03-22T06:15:01.467 INFO:tasks.workunit.client.0.smithi006.stdout:8/618: symlink d9/db/d35/dd2/le6 0 2024-03-22T06:15:01.469 INFO:tasks.workunit.client.0.smithi006.stdout:0/386: readlink db/dc/le 0 2024-03-22T06:15:01.469 INFO:tasks.workunit.client.0.smithi006.stdout:4/565: sync 2024-03-22T06:15:01.469 INFO:tasks.workunit.client.0.smithi006.stdout:1/413: sync 2024-03-22T06:15:01.474 INFO:tasks.workunit.client.0.smithi006.stdout:3/514: symlink d0/d8/d1c/d49/lad 0 2024-03-22T06:15:01.475 INFO:tasks.workunit.client.0.smithi006.stdout:9/411: dwrite d8/d10/d20/d45/d4d/f7a [0,4194304] 0 2024-03-22T06:15:01.476 INFO:tasks.workunit.client.0.smithi006.stdout:6/466: getdents d6/d1d/d5c 0 2024-03-22T06:15:01.485 INFO:tasks.workunit.client.0.smithi006.stdout:5/664: creat d0/db/d16/fca x:0 0 0 2024-03-22T06:15:01.490 INFO:tasks.workunit.client.0.smithi006.stdout:8/619: creat d9/d14/d47/d94/fe7 x:0 0 0 2024-03-22T06:15:01.491 INFO:tasks.workunit.client.0.smithi006.stdout:0/387: symlink db/d10/d2a/l85 0 2024-03-22T06:15:01.491 INFO:tasks.workunit.client.0.smithi006.stdout:0/388: readlink db/d3d/l3f 0 2024-03-22T06:15:01.494 INFO:tasks.workunit.client.0.smithi006.stdout:4/566: mknod d6/d1b/cc3 0 2024-03-22T06:15:01.494 INFO:tasks.workunit.client.0.smithi006.stdout:4/567: read - d6/d1b/d20/d21/d2a/d43/f4c zero size 2024-03-22T06:15:01.496 INFO:tasks.workunit.client.0.smithi006.stdout:5/665: dread d0/db/d50/f5b [0,4194304] 0 2024-03-22T06:15:01.496 INFO:tasks.workunit.client.0.smithi006.stdout:5/666: chown d0/db/d16/d1c/d34/d35/d85/d5e/d66/f7b 0 1 2024-03-22T06:15:01.498 INFO:tasks.workunit.client.0.smithi006.stdout:1/414: mknod d8/d9/d13/c89 0 2024-03-22T06:15:01.499 INFO:tasks.workunit.client.0.smithi006.stdout:1/415: fsync d8/d9/d13/d39/d60/f6e 0 2024-03-22T06:15:01.499 INFO:tasks.workunit.client.0.smithi006.stdout:3/515: dwrite d0/d23/d4c/f35 [4194304,4194304] 0 2024-03-22T06:15:01.499 INFO:tasks.workunit.client.0.smithi006.stdout:3/516: creat d0/d8/d1c/d3e/d5f/fae x:0 0 0 2024-03-22T06:15:01.499 INFO:tasks.workunit.client.0.smithi006.stdout:3/517: chown d0/d23/d33/d3d/d58/ca8 593685140 1 2024-03-22T06:15:01.502 INFO:tasks.workunit.client.0.smithi006.stdout:9/412: creat d8/d10/d20/d45/d49/f8c x:0 0 0 2024-03-22T06:15:01.506 INFO:tasks.workunit.client.0.smithi006.stdout:3/518: dread d0/d23/f63 [4194304,4194304] 0 2024-03-22T06:15:01.508 INFO:tasks.workunit.client.0.smithi006.stdout:5/667: write d0/db/d50/d58/f91 [428406,5605] 0 2024-03-22T06:15:01.508 INFO:tasks.workunit.client.0.smithi006.stdout:5/668: write d0/db/d99/f74 [609465,46532] 0 2024-03-22T06:15:01.508 INFO:tasks.workunit.client.0.smithi006.stdout:6/467: dwrite d6/d1d/d32/f41 [0,4194304] 0 2024-03-22T06:15:01.508 INFO:tasks.workunit.client.0.smithi006.stdout:6/468: creat d6/d6f/f9e x:0 0 0 2024-03-22T06:15:01.511 INFO:tasks.workunit.client.0.smithi006.stdout:8/620: mkdir d9/db/d35/dd2/dc4/ddd/de8 0 2024-03-22T06:15:01.513 INFO:tasks.workunit.client.0.smithi006.stdout:0/389: creat db/d10/d35/d6c/f86 x:0 0 0 2024-03-22T06:15:01.513 INFO:tasks.workunit.client.0.smithi006.stdout:0/390: read - db/dc/d38/d50/f79 zero size 2024-03-22T06:15:01.516 INFO:tasks.workunit.client.0.smithi006.stdout:4/568: symlink d6/d1b/d20/d6b/lc4 0 2024-03-22T06:15:01.517 INFO:tasks.workunit.client.0.smithi006.stdout:4/569: read d6/d1b/d20/d21/d77/f90 [528276,15347] 0 2024-03-22T06:15:01.518 INFO:tasks.workunit.client.0.smithi006.stdout:1/416: symlink d8/d9/d13/l8a 0 2024-03-22T06:15:01.522 INFO:tasks.workunit.client.0.smithi006.stdout:9/413: mkdir d8/d10/d20/d8d 0 2024-03-22T06:15:01.526 INFO:tasks.workunit.client.0.smithi006.stdout:3/519: symlink d0/d8/d9/laf 0 2024-03-22T06:15:01.526 INFO:tasks.workunit.client.0.smithi006.stdout:3/520: chown d0/d4/d31/l9e 59620 1 2024-03-22T06:15:01.531 INFO:tasks.workunit.client.0.smithi006.stdout:9/414: write d8/d10/d11/f59 [701809,101179] 0 2024-03-22T06:15:01.531 INFO:tasks.workunit.client.0.smithi006.stdout:9/415: fdatasync d8/f64 0 2024-03-22T06:15:01.532 INFO:tasks.workunit.client.0.smithi006.stdout:5/669: mkdir d0/db/d50/d96/d52/d69/d92/db7/dcb 0 2024-03-22T06:15:01.532 INFO:tasks.workunit.client.0.smithi006.stdout:5/670: write d0/db/d16/d1c/d34/d35/d85/d5e/d66/f9b [294009,47295] 0 2024-03-22T06:15:01.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:01 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:01.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:01 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:01.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:01 smithi006 ceph-mon[29301]: pgmap v14: 65 pgs: 65 active+clean; 1.0 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 113 MiB/s rd, 145 MiB/s wr, 234 op/s 2024-03-22T06:15:01.538 INFO:tasks.workunit.client.0.smithi006.stdout:1/417: dwrite d8/d9/f35 [0,4194304] 0 2024-03-22T06:15:01.539 INFO:tasks.workunit.client.0.smithi006.stdout:8/621: mknod d9/db/dc/de/ce9 0 2024-03-22T06:15:01.544 INFO:tasks.workunit.client.0.smithi006.stdout:9/416: write f4 [3602918,33718] 0 2024-03-22T06:15:01.545 INFO:tasks.workunit.client.0.smithi006.stdout:4/570: unlink d6/d1b/d20/l39 0 2024-03-22T06:15:01.546 INFO:tasks.workunit.client.0.smithi006.stdout:1/418: write d8/d9/d3a/d40/d44/f66 [1060983,36879] 0 2024-03-22T06:15:01.547 INFO:tasks.workunit.client.0.smithi006.stdout:3/521: symlink d0/d8/d1c/d3e/d5f/lb0 0 2024-03-22T06:15:01.552 INFO:tasks.workunit.client.0.smithi006.stdout:5/671: rename d0/db/d16/d1c/d34/c38 to d0/db/d16/d1c/d34/ccc 0 2024-03-22T06:15:01.555 INFO:tasks.workunit.client.0.smithi006.stdout:6/469: getdents d6/d24/d44 0 2024-03-22T06:15:01.556 INFO:tasks.workunit.client.0.smithi006.stdout:8/622: fsync d9/db/d12/f5d 0 2024-03-22T06:15:01.562 INFO:tasks.workunit.client.0.smithi006.stdout:9/417: creat d8/d10/d28/d39/d75/d50/f8e x:0 0 0 2024-03-22T06:15:01.562 INFO:tasks.workunit.client.0.smithi006.stdout:9/418: fdatasync d8/d1f/d34/f70 0 2024-03-22T06:15:01.562 INFO:tasks.workunit.client.0.smithi006.stdout:9/419: readlink d8/d10/d28/l51 0 2024-03-22T06:15:01.562 INFO:tasks.workunit.client.0.smithi006.stdout:9/420: read - d8/f7c zero size 2024-03-22T06:15:01.563 INFO:tasks.workunit.client.0.smithi006.stdout:6/470: write d6/d1b/d2c/f7d [199905,25045] 0 2024-03-22T06:15:01.564 INFO:tasks.workunit.client.0.smithi006.stdout:6/471: chown d6/d6f/d94 18897155 1 2024-03-22T06:15:01.564 INFO:tasks.workunit.client.0.smithi006.stdout:4/571: link d6/d1b/l91 d6/d1b/d20/d63/d93/lc5 0 2024-03-22T06:15:01.564 INFO:tasks.workunit.client.0.smithi006.stdout:4/572: creat d6/d1b/d20/d63/d93/fc6 x:0 0 0 2024-03-22T06:15:01.568 INFO:tasks.workunit.client.0.smithi006.stdout:3/522: unlink d0/d8/f13 0 2024-03-22T06:15:01.568 INFO:tasks.workunit.client.0.smithi006.stdout:3/523: write d0/d23/d33/d3d/d58/f8f [1006329,48098] 0 2024-03-22T06:15:01.569 INFO:tasks.workunit.client.0.smithi006.stdout:5/672: truncate d0/db/d50/d96/d61/f7d 424229 0 2024-03-22T06:15:01.570 INFO:tasks.workunit.client.0.smithi006.stdout:9/421: dread d8/d10/d23/d3e/f5a [0,4194304] 0 2024-03-22T06:15:01.570 INFO:tasks.workunit.client.0.smithi006.stdout:9/422: chown d8/d10/d11/f42 56857 1 2024-03-22T06:15:01.572 INFO:tasks.workunit.client.0.smithi006.stdout:0/391: rename db/d10/d2a/d41/d4f to db/d87 0 2024-03-22T06:15:01.576 INFO:tasks.workunit.client.0.smithi006.stdout:6/472: symlink d6/d7c/l9f 0 2024-03-22T06:15:01.576 INFO:tasks.workunit.client.0.smithi006.stdout:6/473: write d6/d1d/d5c/d81/f5d [568508,35368] 0 2024-03-22T06:15:01.582 INFO:tasks.workunit.client.0.smithi006.stdout:3/524: rmdir d0/d4/d31/d98 39 2024-03-22T06:15:01.583 INFO:tasks.workunit.client.0.smithi006.stdout:3/525: write d0/d8/d1c/d72/f87 [141700,66211] 0 2024-03-22T06:15:01.583 INFO:tasks.workunit.client.0.smithi006.stdout:3/526: stat d0/d8/f1a 0 2024-03-22T06:15:01.584 INFO:tasks.workunit.client.0.smithi006.stdout:5/673: stat d0/db/l26 0 2024-03-22T06:15:01.586 INFO:tasks.workunit.client.0.smithi006.stdout:9/423: mknod d8/c8f 0 2024-03-22T06:15:01.591 INFO:tasks.workunit.client.0.smithi006.stdout:8/623: rename d9/db/dc/de/d3e/f3f to d9/db/d12/d66/d6c/d92/fea 0 2024-03-22T06:15:01.599 INFO:tasks.workunit.client.0.smithi006.stdout:8/624: write d9/db/d12/d1e/f30 [1141257,92011] 0 2024-03-22T06:15:01.603 INFO:tasks.workunit.client.0.smithi006.stdout:3/527: getdents d0/d4/d31/d98 0 2024-03-22T06:15:01.606 INFO:tasks.workunit.client.0.smithi006.stdout:1/419: truncate d8/d9/d13/d39/f63 2871183 0 2024-03-22T06:15:01.608 INFO:tasks.workunit.client.0.smithi006.stdout:5/674: link d0/db/d16/d1c/d34/fa6 d0/db/d16/d1c/d34/d35/d5d/d9d/fcd 0 2024-03-22T06:15:01.609 INFO:tasks.workunit.client.0.smithi006.stdout:5/675: fsync d0/db/d16/d1c/d34/d35/d85/d5e/d66/dc0/fa2 0 2024-03-22T06:15:01.610 INFO:tasks.workunit.client.0.smithi006.stdout:9/424: creat d8/d10/d23/d89/f90 x:0 0 0 2024-03-22T06:15:01.613 INFO:tasks.workunit.client.0.smithi006.stdout:0/392: dwrite db/dc/d37/f63 [0,4194304] 0 2024-03-22T06:15:01.615 INFO:tasks.workunit.client.0.smithi006.stdout:4/573: rename d6/d1b/d20/d6b/d60/db1/cc2 to d6/d1b/d20/d63/d93/d9a/d8f/cc7 0 2024-03-22T06:15:01.619 INFO:tasks.workunit.client.0.smithi006.stdout:6/474: getdents d6/d1d/d32 0 2024-03-22T06:15:01.620 INFO:tasks.workunit.client.0.smithi006.stdout:6/475: readlink d6/d1b/d2c/d4a/l5b 0 2024-03-22T06:15:01.620 INFO:tasks.workunit.client.0.smithi006.stdout:6/476: truncate d6/d7c/f8e 701326 0 2024-03-22T06:15:01.620 INFO:tasks.workunit.client.0.smithi006.stdout:6/477: write d6/d1b/d2c/f4b [938658,98190] 0 2024-03-22T06:15:01.628 INFO:tasks.workunit.client.0.smithi006.stdout:8/625: symlink d9/db/dc/leb 0 2024-03-22T06:15:01.632 INFO:tasks.workunit.client.0.smithi006.stdout:1/420: mknod d8/d9/d3a/c8b 0 2024-03-22T06:15:01.637 INFO:tasks.workunit.client.0.smithi006.stdout:5/676: link d0/l6 d0/db/d16/d1c/d34/d35/d85/d54/lce 0 2024-03-22T06:15:01.639 INFO:tasks.workunit.client.0.smithi006.stdout:9/425: mkdir d8/d10/d28/d39/d75/d50/d91 0 2024-03-22T06:15:01.639 INFO:tasks.workunit.client.0.smithi006.stdout:9/426: creat d8/d1f/d34/f92 x:0 0 0 2024-03-22T06:15:01.646 INFO:tasks.workunit.client.0.smithi006.stdout:4/574: link f3 d6/d1b/fc8 0 2024-03-22T06:15:01.647 INFO:tasks.workunit.client.0.smithi006.stdout:6/478: creat d6/d24/d44/fa0 x:0 0 0 2024-03-22T06:15:01.649 INFO:tasks.workunit.client.0.smithi006.stdout:6/479: dread d6/d1d/f26 [0,4194304] 0 2024-03-22T06:15:01.649 INFO:tasks.workunit.client.0.smithi006.stdout:6/480: write d6/d24/f54 [360019,88238] 0 2024-03-22T06:15:01.649 INFO:tasks.workunit.client.0.smithi006.stdout:3/528: mkdir d0/d8/d9/db1 0 2024-03-22T06:15:01.649 INFO:tasks.workunit.client.0.smithi006.stdout:1/421: mknod d8/d9/d13/d24/d32/d34/c8c 0 2024-03-22T06:15:01.651 INFO:tasks.workunit.client.0.smithi006.stdout:5/677: creat d0/db/d50/d58/dad/fcf x:0 0 0 2024-03-22T06:15:01.651 INFO:tasks.workunit.client.0.smithi006.stdout:5/678: chown d0/db/d16/d1c/d34/d4d/f6b 195 1 2024-03-22T06:15:01.651 INFO:tasks.workunit.client.0.smithi006.stdout:5/679: fsync d0/db/d16/f46 0 2024-03-22T06:15:01.651 INFO:tasks.workunit.client.0.smithi006.stdout:5/680: chown d0/l7 41653 1 2024-03-22T06:15:01.651 INFO:tasks.workunit.client.0.smithi006.stdout:5/681: fdatasync d0/db/d16/d1c/d34/d35/d85/fc1 0 2024-03-22T06:15:01.652 INFO:tasks.workunit.client.0.smithi006.stdout:5/682: dread - d0/db/d50/d96/fae zero size 2024-03-22T06:15:01.652 INFO:tasks.workunit.client.0.smithi006.stdout:5/683: chown d0/db/d16/d1c/d34/d35/d85/d5e/d66/l90 4 1 2024-03-22T06:15:01.653 INFO:tasks.workunit.client.0.smithi006.stdout:0/393: rename db/d87/f68 to db/dc/f88 0 2024-03-22T06:15:01.654 INFO:tasks.workunit.client.0.smithi006.stdout:4/575: mkdir d6/d1b/d20/d21/d2a/dc9 0 2024-03-22T06:15:01.657 INFO:tasks.workunit.client.0.smithi006.stdout:1/422: creat d8/d9/f8d x:0 0 0 2024-03-22T06:15:01.657 INFO:tasks.workunit.client.0.smithi006.stdout:5/684: symlink d0/db/d16/d43/ld0 0 2024-03-22T06:15:01.660 INFO:tasks.workunit.client.0.smithi006.stdout:8/626: rename d9/l7a to d9/db/d35/dd2/d76/dca/lec 0 2024-03-22T06:15:01.660 INFO:tasks.workunit.client.0.smithi006.stdout:8/627: write d9/db/d35/da9/fc8 [1029608,99320] 0 2024-03-22T06:15:01.661 INFO:tasks.workunit.client.0.smithi006.stdout:0/394: symlink db/dc/d38/d50/l89 0 2024-03-22T06:15:01.661 INFO:tasks.workunit.client.0.smithi006.stdout:0/395: stat db/dc/d37 0 2024-03-22T06:15:01.663 INFO:tasks.workunit.client.0.smithi006.stdout:6/481: dread d6/d1b/f2d [0,4194304] 0 2024-03-22T06:15:01.664 INFO:tasks.workunit.client.0.smithi006.stdout:3/529: link d0/l19 d0/d8/d1c/d3e/d5f/lb2 0 2024-03-22T06:15:01.665 INFO:tasks.workunit.client.0.smithi006.stdout:1/423: creat d8/d71/f8e x:0 0 0 2024-03-22T06:15:01.666 INFO:tasks.workunit.client.0.smithi006.stdout:5/685: dread - d0/db/d76/f9a zero size 2024-03-22T06:15:01.667 INFO:tasks.workunit.client.0.smithi006.stdout:9/427: rename d8/d10/d23/d89/f90 to d8/d10/d28/d39/f93 0 2024-03-22T06:15:01.672 INFO:tasks.workunit.client.0.smithi006.stdout:6/482: dread d6/d1b/d2c/f59 [0,4194304] 0 2024-03-22T06:15:01.672 INFO:tasks.workunit.client.0.smithi006.stdout:6/483: read d6/d1b/d2c/f7d [659171,106530] 0 2024-03-22T06:15:01.675 INFO:tasks.workunit.client.0.smithi006.stdout:4/576: symlink d6/d1b/d20/d21/d77/da0/lca 0 2024-03-22T06:15:01.680 INFO:tasks.workunit.client.0.smithi006.stdout:9/428: write d8/d10/d20/f2c [3812050,18581] 0 2024-03-22T06:15:01.680 INFO:tasks.workunit.client.0.smithi006.stdout:9/429: dread - d8/d10/d23/d3e/f7f zero size 2024-03-22T06:15:01.682 INFO:tasks.workunit.client.0.smithi006.stdout:6/484: read d6/d24/d44/f4d [592595,30204] 0 2024-03-22T06:15:01.693 INFO:tasks.workunit.client.0.smithi006.stdout:1/424: dwrite d8/d9/d13/d24/f7f [0,4194304] 0 2024-03-22T06:15:01.693 INFO:tasks.workunit.client.0.smithi006.stdout:8/628: dwrite d9/db/d12/f26 [0,4194304] 0 2024-03-22T06:15:01.699 INFO:tasks.workunit.client.0.smithi006.stdout:5/686: getdents d0/db/d16/d1c/d34/d35/d85/d54 0 2024-03-22T06:15:01.702 INFO:tasks.workunit.client.0.smithi006.stdout:0/396: getdents db/d10/d35/d6c 0 2024-03-22T06:15:01.702 INFO:tasks.workunit.client.0.smithi006.stdout:0/397: creat db/dc/d37/f8a x:0 0 0 2024-03-22T06:15:01.702 INFO:tasks.workunit.client.0.smithi006.stdout:0/398: chown db/d87/l7c 0 1 2024-03-22T06:15:01.706 INFO:tasks.workunit.client.0.smithi006.stdout:4/577: rename d6/f8 to d6/d1b/d20/d21/d2a/dc9/fcb 0 2024-03-22T06:15:01.707 INFO:tasks.workunit.client.0.smithi006.stdout:3/530: mkdir d0/d8/db3 0 2024-03-22T06:15:01.710 INFO:tasks.workunit.client.0.smithi006.stdout:9/430: symlink d8/d1f/l94 0 2024-03-22T06:15:01.710 INFO:tasks.workunit.client.0.smithi006.stdout:9/431: truncate d8/d10/d23/f40 4608759 0 2024-03-22T06:15:01.711 INFO:tasks.workunit.client.0.smithi006.stdout:1/425: dwrite d8/d71/f8e [0,4194304] 0 2024-03-22T06:15:01.711 INFO:tasks.workunit.client.0.smithi006.stdout:1/426: fsync d8/d9/d13/d24/d32/d34/f77 0 2024-03-22T06:15:01.712 INFO:tasks.workunit.client.0.smithi006.stdout:6/485: mknod d6/d1b/d2c/d4a/ca1 0 2024-03-22T06:15:01.718 INFO:tasks.workunit.client.0.smithi006.stdout:4/578: dread d6/d1b/d20/f33 [0,4194304] 0 2024-03-22T06:15:01.720 INFO:tasks.workunit.client.0.smithi006.stdout:0/399: creat db/d3d/d4e/d76/d81/f8b x:0 0 0 2024-03-22T06:15:01.721 INFO:tasks.workunit.client.0.smithi006.stdout:1/427: write d8/d9/d13/d24/d32/d34/f36 [2277310,22709] 0 2024-03-22T06:15:01.727 INFO:tasks.workunit.client.0.smithi006.stdout:5/687: rename d0/db/d50/d96/d61/fbb to d0/db/d16/d70/fd1 0 2024-03-22T06:15:01.728 INFO:tasks.workunit.client.0.smithi006.stdout:4/579: dread d6/f42 [0,4194304] 0 2024-03-22T06:15:01.728 INFO:tasks.workunit.client.0.smithi006.stdout:4/580: truncate d6/d13/d3b/f4e 1706471 0 2024-03-22T06:15:01.728 INFO:tasks.workunit.client.0.smithi006.stdout:3/531: mknod d0/d4/d31/cb4 0 2024-03-22T06:15:01.731 INFO:tasks.workunit.client.0.smithi006.stdout:9/432: unlink l5 0 2024-03-22T06:15:01.731 INFO:tasks.workunit.client.0.smithi006.stdout:9/433: write d8/d10/d20/d45/d49/f8c [1341,1766] 0 2024-03-22T06:15:01.731 INFO:tasks.workunit.client.0.smithi006.stdout:9/434: readlink d8/d10/d11/l18 0 2024-03-22T06:15:01.735 INFO:tasks.workunit.client.0.smithi006.stdout:6/486: unlink d6/d24/l96 0 2024-03-22T06:15:01.736 INFO:tasks.workunit.client.0.smithi006.stdout:8/629: getdents d9/d14/d47/d94 0 2024-03-22T06:15:01.738 INFO:tasks.workunit.client.0.smithi006.stdout:7/584: sync 2024-03-22T06:15:01.738 INFO:tasks.workunit.client.0.smithi006.stdout:2/534: sync 2024-03-22T06:15:01.741 INFO:tasks.workunit.client.0.smithi006.stdout:1/428: mkdir d8/d9/d13/d39/d60/d8f 0 2024-03-22T06:15:01.745 INFO:tasks.workunit.client.0.smithi006.stdout:5/688: symlink d0/db/d16/d1c/d34/d35/d5d/d9d/ld2 0 2024-03-22T06:15:01.745 INFO:tasks.workunit.client.0.smithi006.stdout:7/585: dread d5/d1e/d29/d1f/d2d/d55/d84/fa0 [0,4194304] 0 2024-03-22T06:15:01.746 INFO:tasks.workunit.client.0.smithi006.stdout:4/581: rmdir d6/d1b/d20/d63/d93/d9a/d8f/da1 39 2024-03-22T06:15:01.748 INFO:tasks.workunit.client.0.smithi006.stdout:3/532: mkdir d0/d8/d1c/d49/db5 0 2024-03-22T06:15:01.750 INFO:tasks.workunit.client.0.smithi006.stdout:9/435: symlink d8/d10/d23/d3e/l95 0 2024-03-22T06:15:01.752 INFO:tasks.workunit.client.0.smithi006.stdout:6/487: creat d6/d7c/d83/fa2 x:0 0 0 2024-03-22T06:15:01.752 INFO:tasks.workunit.client.0.smithi006.stdout:8/630: rmdir d9/db 39 2024-03-22T06:15:01.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:01 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:01.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:01 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:01.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:01 smithi117 ceph-mon[36514]: pgmap v14: 65 pgs: 65 active+clean; 1.0 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 113 MiB/s rd, 145 MiB/s wr, 234 op/s 2024-03-22T06:15:01.756 INFO:tasks.workunit.client.0.smithi006.stdout:2/535: dwrite d13/d53/f64 [0,4194304] 0 2024-03-22T06:15:01.756 INFO:tasks.workunit.client.0.smithi006.stdout:1/429: link d8/d9/d3a/d18/d2a/c30 d8/d9/d13/d24/c90 0 2024-03-22T06:15:01.756 INFO:tasks.workunit.client.0.smithi006.stdout:1/430: write d8/d9/d3a/d18/f1b [4588797,111702] 0 2024-03-22T06:15:01.759 INFO:tasks.workunit.client.0.smithi006.stdout:5/689: truncate d0/db/d50/d58/f6f 211721 0 2024-03-22T06:15:01.761 INFO:tasks.workunit.client.0.smithi006.stdout:7/586: mkdir d5/d1e/d29/db9 0 2024-03-22T06:15:01.761 INFO:tasks.workunit.client.0.smithi006.stdout:7/587: readlink d5/d1e/d29/d59/l78 0 2024-03-22T06:15:01.763 INFO:tasks.workunit.client.0.smithi006.stdout:4/582: mknod d6/d1b/d46/ccc 0 2024-03-22T06:15:01.766 INFO:tasks.workunit.client.0.smithi006.stdout:3/533: mkdir d0/d4/d31/d98/dab/db6 0 2024-03-22T06:15:01.766 INFO:tasks.workunit.client.0.smithi006.stdout:3/534: readlink d0/d4/d3a/l8a 0 2024-03-22T06:15:01.766 INFO:tasks.workunit.client.0.smithi006.stdout:3/535: truncate d0/d23/d4c/fa2 62081 0 2024-03-22T06:15:01.767 INFO:tasks.workunit.client.0.smithi006.stdout:5/690: read d0/db/d16/d1c/d34/f4b [121279,119423] 0 2024-03-22T06:15:01.769 INFO:tasks.workunit.client.0.smithi006.stdout:9/436: mknod d8/d10/d82/c96 0 2024-03-22T06:15:01.773 INFO:tasks.workunit.client.0.smithi006.stdout:6/488: mknod d6/d1d/d99/ca3 0 2024-03-22T06:15:01.773 INFO:tasks.workunit.client.0.smithi006.stdout:6/489: dread - d6/d24/d44/f90 zero size 2024-03-22T06:15:01.774 INFO:tasks.workunit.client.0.smithi006.stdout:8/631: stat d9/db/d12/d66/d6c/d71/d79/la5 0 2024-03-22T06:15:01.780 INFO:tasks.workunit.client.0.smithi006.stdout:2/536: creat d13/d53/dc5/fd5 x:0 0 0 2024-03-22T06:15:01.782 INFO:tasks.workunit.client.0.smithi006.stdout:1/431: symlink d8/l91 0 2024-03-22T06:15:01.784 INFO:tasks.workunit.client.0.smithi006.stdout:7/588: creat d5/d1e/fba x:0 0 0 2024-03-22T06:15:01.785 INFO:tasks.workunit.client.0.smithi006.stdout:4/583: unlink d6/d13/c1e 0 2024-03-22T06:15:01.787 INFO:tasks.workunit.client.0.smithi006.stdout:5/691: creat d0/db/d16/d1c/d34/d35/d85/d54/fd3 x:0 0 0 2024-03-22T06:15:01.788 INFO:tasks.workunit.client.0.smithi006.stdout:5/692: readlink d0/db/d50/d82/l93 0 2024-03-22T06:15:01.788 INFO:tasks.workunit.client.0.smithi006.stdout:5/693: truncate d0/db/d50/d96/d52/fc9 784946 0 2024-03-22T06:15:01.788 INFO:tasks.workunit.client.0.smithi006.stdout:9/437: dwrite d8/d10/f56 [0,4194304] 0 2024-03-22T06:15:01.793 INFO:tasks.workunit.client.0.smithi006.stdout:7/589: read d5/d1e/d29/d1f/d2d/d41/f69 [575822,67957] 0 2024-03-22T06:15:01.796 INFO:tasks.workunit.client.0.smithi006.stdout:2/537: link d13/d53/d6b/f97 d13/d34/d4d/d4e/d75/d62/fd6 0 2024-03-22T06:15:01.796 INFO:tasks.workunit.client.0.smithi006.stdout:2/538: write d13/f61 [1041185,95690] 0 2024-03-22T06:15:01.798 INFO:tasks.workunit.client.0.smithi006.stdout:2/539: dread d13/d34/d4d/d4e/d75/d62/f9c [0,4194304] 0 2024-03-22T06:15:01.800 INFO:tasks.workunit.client.0.smithi006.stdout:4/584: mknod d6/d1b/d20/d21/ccd 0 2024-03-22T06:15:01.803 INFO:tasks.workunit.client.0.smithi006.stdout:5/694: mkdir d0/db/d16/d1c/d34/d35/d85/d54/d65/dd4 0 2024-03-22T06:15:01.804 INFO:tasks.workunit.client.0.smithi006.stdout:9/438: creat d8/d10/d47/f97 x:0 0 0 2024-03-22T06:15:01.804 INFO:tasks.workunit.client.0.smithi006.stdout:9/439: fsync d8/d10/d28/f25 0 2024-03-22T06:15:01.804 INFO:tasks.workunit.client.0.smithi006.stdout:9/440: write d8/d10/d28/d39/f93 [168554,125310] 0 2024-03-22T06:15:01.805 INFO:tasks.workunit.client.0.smithi006.stdout:9/441: dread - d8/d10/d28/d39/d75/d50/f6f zero size 2024-03-22T06:15:01.806 INFO:tasks.workunit.client.0.smithi006.stdout:6/490: getdents d6/d7c 0 2024-03-22T06:15:01.808 INFO:tasks.workunit.client.0.smithi006.stdout:7/590: getdents d5/d1e/d29/d1f 0 2024-03-22T06:15:01.810 INFO:tasks.workunit.client.0.smithi006.stdout:8/632: getdents d9/d5e 0 2024-03-22T06:15:01.813 INFO:tasks.workunit.client.0.smithi006.stdout:1/432: rmdir d8/d9/d10/d7c 39 2024-03-22T06:15:01.818 INFO:tasks.workunit.client.0.smithi006.stdout:5/695: readlink d0/db/lb3 0 2024-03-22T06:15:01.820 INFO:tasks.workunit.client.0.smithi006.stdout:9/442: rename d8/d10/d28/c63 to d8/d1f/d34/c98 0 2024-03-22T06:15:01.823 INFO:tasks.workunit.client.0.smithi006.stdout:6/491: symlink d6/d6f/la4 0 2024-03-22T06:15:01.826 INFO:tasks.workunit.client.0.smithi006.stdout:8/633: rmdir d9/db/d35/dd2/d76/db3 39 2024-03-22T06:15:01.826 INFO:tasks.workunit.client.0.smithi006.stdout:8/634: chown d9/d14/d47/d94/da3 4324 1 2024-03-22T06:15:01.828 INFO:tasks.workunit.client.0.smithi006.stdout:2/540: getdents d13/d34/d6e/d72 0 2024-03-22T06:15:01.830 INFO:tasks.workunit.client.0.smithi006.stdout:4/585: dwrite d6/d1b/d20/d21/d77/f94 [0,4194304] 0 2024-03-22T06:15:01.831 INFO:tasks.workunit.client.0.smithi006.stdout:5/696: getdents d0/db/d16/d1c/d34/d35/d5d 0 2024-03-22T06:15:01.835 INFO:tasks.workunit.client.0.smithi006.stdout:7/591: rename d5/d1e/ca7 to d5/d1e/d29/cbb 0 2024-03-22T06:15:01.835 INFO:tasks.workunit.client.0.smithi006.stdout:7/592: stat d5/cf 0 2024-03-22T06:15:01.839 INFO:tasks.workunit.client.0.smithi006.stdout:4/586: write d6/d1b/d20/d63/fba [2345344,20295] 0 2024-03-22T06:15:01.839 INFO:tasks.workunit.client.0.smithi006.stdout:4/587: readlink d6/d1b/l5f 0 2024-03-22T06:15:01.846 INFO:tasks.workunit.client.0.smithi006.stdout:9/443: dwrite d8/d10/d28/d39/d75/d50/f69 [0,4194304] 0 2024-03-22T06:15:01.846 INFO:tasks.workunit.client.0.smithi006.stdout:9/444: fsync d8/d10/d28/d39/d75/d50/f61 0 2024-03-22T06:15:01.847 INFO:tasks.workunit.client.0.smithi006.stdout:1/433: rename d8/d9/f50 to d8/d9/d3a/f92 0 2024-03-22T06:15:01.847 INFO:tasks.workunit.client.0.smithi006.stdout:6/492: dwrite d6/d24/d44/f46 [0,4194304] 0 2024-03-22T06:15:01.847 INFO:tasks.workunit.client.0.smithi006.stdout:6/493: stat d6/d24/d44/f46 0 2024-03-22T06:15:01.851 INFO:tasks.workunit.client.0.smithi006.stdout:2/541: read f12 [2282069,82471] 0 2024-03-22T06:15:01.851 INFO:tasks.workunit.client.0.smithi006.stdout:7/593: mkdir d5/d1e/dbc 0 2024-03-22T06:15:01.851 INFO:tasks.workunit.client.0.smithi006.stdout:7/594: chown d5/d1e/d29/d1f/c9b 515192 1 2024-03-22T06:15:01.851 INFO:tasks.workunit.client.0.smithi006.stdout:7/595: readlink d5/d1e/d29/la2 0 2024-03-22T06:15:01.857 INFO:tasks.workunit.client.0.smithi006.stdout:4/588: link d6/c23 d6/d1b/d20/d6b/d88/cce 0 2024-03-22T06:15:01.862 INFO:tasks.workunit.client.0.smithi006.stdout:9/445: mknod d8/d10/c99 0 2024-03-22T06:15:01.864 INFO:tasks.workunit.client.0.smithi006.stdout:1/434: rename d8/d9/d3a/d18/l51 to d8/d9/d3a/d18/l93 0 2024-03-22T06:15:01.869 INFO:tasks.workunit.client.0.smithi006.stdout:8/635: truncate d9/db/d12/f26 1072935 0 2024-03-22T06:15:01.871 INFO:tasks.workunit.client.0.smithi006.stdout:5/697: dwrite d0/db/f60 [0,4194304] 0 2024-03-22T06:15:01.871 INFO:tasks.workunit.client.0.smithi006.stdout:6/494: dwrite d6/d1d/f71 [0,4194304] 0 2024-03-22T06:15:01.873 INFO:tasks.workunit.client.0.smithi006.stdout:2/542: rename d13/d20/f8b to d13/d53/dc5/fd7 0 2024-03-22T06:15:01.873 INFO:tasks.workunit.client.0.smithi006.stdout:2/543: dread - d13/d20/fad zero size 2024-03-22T06:15:01.876 INFO:tasks.workunit.client.0.smithi006.stdout:7/596: symlink d5/lbd 0 2024-03-22T06:15:01.879 INFO:tasks.workunit.client.0.smithi006.stdout:1/435: read d8/d9/d10/f19 [4097430,119160] 0 2024-03-22T06:15:01.881 INFO:tasks.workunit.client.0.smithi006.stdout:1/436: dread d8/d9/d3a/f92 [0,4194304] 0 2024-03-22T06:15:01.883 INFO:tasks.workunit.client.0.smithi006.stdout:8/636: dwrite d9/db/d12/d1e/f39 [0,4194304] 0 2024-03-22T06:15:01.883 INFO:tasks.workunit.client.0.smithi006.stdout:8/637: read - d9/db/d12/d1e/f51 zero size 2024-03-22T06:15:01.888 INFO:tasks.workunit.client.0.smithi006.stdout:4/589: mknod d6/d1b/d20/d6b/d88/ccf 0 2024-03-22T06:15:01.893 INFO:tasks.workunit.client.0.smithi006.stdout:9/446: creat d8/d10/d23/d3e/f9a x:0 0 0 2024-03-22T06:15:01.893 INFO:tasks.workunit.client.0.smithi006.stdout:9/447: stat d8/d10/d20/d45/d49/f77 0 2024-03-22T06:15:01.899 INFO:tasks.workunit.client.0.smithi006.stdout:5/698: link d0/db/d76/db4/fc3 d0/db/d50/fd5 0 2024-03-22T06:15:01.899 INFO:tasks.workunit.client.0.smithi006.stdout:5/699: chown d0/db/d76/db4 124410 1 2024-03-22T06:15:01.901 INFO:tasks.workunit.client.0.smithi006.stdout:8/638: dwrite d9/db/d35/dd2/dad/fb7 [0,4194304] 0 2024-03-22T06:15:01.901 INFO:tasks.workunit.client.0.smithi006.stdout:8/639: readlink d9/db/d35/dd2/d4a/da7/ld8 0 2024-03-22T06:15:01.901 INFO:tasks.workunit.client.0.smithi006.stdout:7/597: dwrite d5/d1e/d29/d10/f3d [0,4194304] 0 2024-03-22T06:15:01.904 INFO:tasks.workunit.client.0.smithi006.stdout:2/544: mkdir d13/d53/d6b/dd8 0 2024-03-22T06:15:01.906 INFO:tasks.workunit.client.0.smithi006.stdout:2/545: dread d13/d20/fd0 [0,4194304] 0 2024-03-22T06:15:01.909 INFO:tasks.workunit.client.0.smithi006.stdout:1/437: mkdir d8/d9/d13/d39/d94 0 2024-03-22T06:15:01.909 INFO:tasks.workunit.client.0.smithi006.stdout:1/438: chown d8/d9/l3b 932768344 1 2024-03-22T06:15:01.913 INFO:tasks.workunit.client.0.smithi006.stdout:4/590: mknod d6/d1b/d20/d63/d93/d9a/cd0 0 2024-03-22T06:15:01.916 INFO:tasks.workunit.client.0.smithi006.stdout:9/448: link d8/d10/d28/f25 d8/d10/d20/d45/f9b 0 2024-03-22T06:15:01.919 INFO:tasks.workunit.client.0.smithi006.stdout:5/700: mkdir d0/db/d16/d1c/dbc/dd6 0 2024-03-22T06:15:01.924 INFO:tasks.workunit.client.0.smithi006.stdout:8/640: creat d9/db/fed x:0 0 0 2024-03-22T06:15:01.926 INFO:tasks.workunit.client.0.smithi006.stdout:2/546: mkdir d13/d34/d4d/d4e/dd9 0 2024-03-22T06:15:01.926 INFO:tasks.workunit.client.0.smithi006.stdout:2/547: chown d13/d34/f60 48541 1 2024-03-22T06:15:01.929 INFO:tasks.workunit.client.0.smithi006.stdout:6/495: rename d6/c53 to d6/d24/ca5 0 2024-03-22T06:15:01.929 INFO:tasks.workunit.client.0.smithi006.stdout:6/496: read d6/d1d/f20 [69098,93139] 0 2024-03-22T06:15:01.931 INFO:tasks.workunit.client.0.smithi006.stdout:1/439: symlink d8/d9/d13/d24/d32/d34/l95 0 2024-03-22T06:15:01.933 INFO:tasks.workunit.client.0.smithi006.stdout:9/449: getdents d8/d10/d20 0 2024-03-22T06:15:01.933 INFO:tasks.workunit.client.0.smithi006.stdout:9/450: write d8/d10/d28/f44 [5766074,56196] 0 2024-03-22T06:15:01.937 INFO:tasks.workunit.client.0.smithi006.stdout:5/701: creat d0/db/d16/d1c/d34/d4d/fd7 x:0 0 0 2024-03-22T06:15:01.940 INFO:tasks.workunit.client.0.smithi006.stdout:8/641: mkdir d9/db/d35/dd2/d76/dca/dee 0 2024-03-22T06:15:01.940 INFO:tasks.workunit.client.0.smithi006.stdout:8/642: truncate d9/d14/d47/d94/da3/fc0 872612 0 2024-03-22T06:15:01.944 INFO:tasks.workunit.client.0.smithi006.stdout:7/598: rename d5/c93 to d5/d1e/d29/d1f/d2d/d61/cbe 0 2024-03-22T06:15:01.948 INFO:tasks.workunit.client.0.smithi006.stdout:7/599: dread d5/d1e/d29/d1f/d2d/d55/d84/fa0 [0,4194304] 0 2024-03-22T06:15:01.948 INFO:tasks.workunit.client.0.smithi006.stdout:1/440: mkdir d8/d9/d10/d7c/d96 0 2024-03-22T06:15:01.949 INFO:tasks.workunit.client.0.smithi006.stdout:7/600: getdents d5/d1e/d29/db9 0 2024-03-22T06:15:01.953 INFO:tasks.workunit.client.0.smithi006.stdout:7/601: creat d5/d1e/d29/d1f/d2d/d61/fbf x:0 0 0 2024-03-22T06:15:01.955 INFO:tasks.workunit.client.0.smithi006.stdout:8/643: getdents d9/db/d12/d66/d6c/d80 0 2024-03-22T06:15:01.957 INFO:tasks.workunit.client.0.smithi006.stdout:7/602: rename d5/f1d to d5/d1e/d29/d1f/d2d/d61/daa/fc0 0 2024-03-22T06:15:01.959 INFO:tasks.workunit.client.0.smithi006.stdout:6/497: dwrite d6/d6f/f6e [0,4194304] 0 2024-03-22T06:15:01.960 INFO:tasks.workunit.client.0.smithi006.stdout:8/644: creat d9/db/d35/dd2/d4a/da7/fef x:0 0 0 2024-03-22T06:15:01.960 INFO:tasks.workunit.client.0.smithi006.stdout:8/645: stat d9/d5e/fc2 0 2024-03-22T06:15:01.960 INFO:tasks.workunit.client.0.smithi006.stdout:8/646: fsync d9/db/dc/f3a 0 2024-03-22T06:15:01.960 INFO:tasks.workunit.client.0.smithi006.stdout:7/603: creat d5/d1e/d63/d85/d80/fc1 x:0 0 0 2024-03-22T06:15:01.962 INFO:tasks.workunit.client.0.smithi006.stdout:2/548: dwrite d13/d20/d35/d65/f7b [0,4194304] 0 2024-03-22T06:15:01.964 INFO:tasks.workunit.client.0.smithi006.stdout:7/604: rmdir d5/d1e/d29/d10/d6f 39 2024-03-22T06:15:01.965 INFO:tasks.workunit.client.0.smithi006.stdout:6/498: unlink d6/d1b/c7b 0 2024-03-22T06:15:01.968 INFO:tasks.workunit.client.0.smithi006.stdout:5/702: dwrite d0/db/d16/d1c/d34/f53 [0,4194304] 0 2024-03-22T06:15:01.969 INFO:tasks.workunit.client.0.smithi006.stdout:2/549: unlink d13/f6f 0 2024-03-22T06:15:01.969 INFO:tasks.workunit.client.0.smithi006.stdout:1/441: dwrite d8/d9/d13/d24/d32/d34/f77 [0,4194304] 0 2024-03-22T06:15:01.971 INFO:tasks.workunit.client.0.smithi006.stdout:7/605: unlink d5/d1e/d63/d85/da1/fa4 0 2024-03-22T06:15:01.971 INFO:tasks.workunit.client.0.smithi006.stdout:7/606: readlink d5/d1e/d29/d10/l1b 0 2024-03-22T06:15:01.971 INFO:tasks.workunit.client.0.smithi006.stdout:7/607: chown d5/d1e/d29/f2a 6345324 1 2024-03-22T06:15:01.972 INFO:tasks.workunit.client.0.smithi006.stdout:8/647: write d9/db/d35/dd2/dad/fb7 [1998510,37735] 0 2024-03-22T06:15:01.972 INFO:tasks.workunit.client.0.smithi006.stdout:9/451: dwrite d8/d10/d28/d39/d75/d50/f76 [0,4194304] 0 2024-03-22T06:15:01.974 INFO:tasks.workunit.client.0.smithi006.stdout:5/703: creat d0/db/d16/d1c/d34/d35/d85/d54/d65/dd4/fd8 x:0 0 0 2024-03-22T06:15:01.977 INFO:tasks.workunit.client.0.smithi006.stdout:2/550: mkdir d13/d34/d4d/d4e/d75/da5/d82/dda 0 2024-03-22T06:15:01.977 INFO:tasks.workunit.client.0.smithi006.stdout:2/551: write d13/d20/d26/f79 [944579,22129] 0 2024-03-22T06:15:01.977 INFO:tasks.workunit.client.0.smithi006.stdout:2/552: write d13/d20/fad [623572,82650] 0 2024-03-22T06:15:01.981 INFO:tasks.workunit.client.0.smithi006.stdout:9/452: write d8/d10/d28/f55 [3745294,105100] 0 2024-03-22T06:15:01.983 INFO:tasks.workunit.client.0.smithi006.stdout:1/442: stat d8/d9/l16 0 2024-03-22T06:15:01.986 INFO:tasks.workunit.client.0.smithi006.stdout:6/499: unlink d6/d1b/c27 0 2024-03-22T06:15:01.986 INFO:tasks.workunit.client.0.smithi006.stdout:6/500: write d6/d7c/d83/fa2 [136602,74341] 0 2024-03-22T06:15:01.988 INFO:tasks.workunit.client.0.smithi006.stdout:2/553: write d13/d34/d4d/d4e/d75/da5/f67 [742170,117680] 0 2024-03-22T06:15:01.995 INFO:tasks.workunit.client.0.smithi006.stdout:8/648: mkdir d9/db/d12/d66/d6c/d80/df0 0 2024-03-22T06:15:01.996 INFO:tasks.workunit.client.0.smithi006.stdout:8/649: write d9/d14/d47/d94/da3/fb2 [587681,35095] 0 2024-03-22T06:15:01.999 INFO:tasks.workunit.client.0.smithi006.stdout:5/704: creat d0/fd9 x:0 0 0 2024-03-22T06:15:02.003 INFO:tasks.workunit.client.0.smithi006.stdout:1/443: dwrite d8/d9/d3a/d40/d44/f53 [0,4194304] 0 2024-03-22T06:15:02.003 INFO:tasks.workunit.client.0.smithi006.stdout:1/444: chown d8/d9/d3a/d40/d44/f53 5828 1 2024-03-22T06:15:02.015 INFO:tasks.workunit.client.0.smithi006.stdout:6/501: mknod d6/d1d/d32/ca6 0 2024-03-22T06:15:02.021 INFO:tasks.workunit.client.0.smithi006.stdout:1/445: dwrite d8/d9/d13/f5a [0,4194304] 0 2024-03-22T06:15:02.022 INFO:tasks.workunit.client.0.smithi006.stdout:5/705: dwrite d0/db/d76/f81 [0,4194304] 0 2024-03-22T06:15:02.025 INFO:tasks.workunit.client.0.smithi006.stdout:6/502: symlink d6/d1d/d5c/la7 0 2024-03-22T06:15:02.028 INFO:tasks.workunit.client.0.smithi006.stdout:2/554: rename f6 to d13/d34/d4d/fdb 0 2024-03-22T06:15:02.028 INFO:tasks.workunit.client.0.smithi006.stdout:2/555: fdatasync d13/d20/d35/fcf 0 2024-03-22T06:15:02.029 INFO:tasks.workunit.client.0.smithi006.stdout:2/556: stat d13/d69/ca1 0 2024-03-22T06:15:02.033 INFO:tasks.workunit.client.0.smithi006.stdout:1/446: mkdir d8/d9/d3a/d40/d44/d97 0 2024-03-22T06:15:02.034 INFO:tasks.workunit.client.0.smithi006.stdout:1/447: fsync d8/d9/d13/f5a 0 2024-03-22T06:15:02.036 INFO:tasks.workunit.client.0.smithi006.stdout:5/706: truncate d0/db/d50/d96/f41 2710423 0 2024-03-22T06:15:02.040 INFO:tasks.workunit.client.0.smithi006.stdout:8/650: rename d9/db/d12/d66/d6c/f7c to d9/db/d35/dd2/d4a/da7/ff1 0 2024-03-22T06:15:02.043 INFO:tasks.workunit.client.0.smithi006.stdout:2/557: unlink d13/d20/fd0 0 2024-03-22T06:15:02.045 INFO:tasks.workunit.client.0.smithi006.stdout:1/448: symlink d8/d9/d10/l98 0 2024-03-22T06:15:02.045 INFO:tasks.workunit.client.0.smithi006.stdout:1/449: chown d8/le 5158 1 2024-03-22T06:15:02.045 INFO:tasks.workunit.client.0.smithi006.stdout:1/450: fsync d8/d9/d13/d39/f63 0 2024-03-22T06:15:02.049 INFO:tasks.workunit.client.0.smithi006.stdout:1/451: getdents d8 0 2024-03-22T06:15:02.052 INFO:tasks.workunit.client.0.smithi006.stdout:5/707: read d0/db/d50/d96/f2d [3389305,90207] 0 2024-03-22T06:15:02.052 INFO:tasks.workunit.client.0.smithi006.stdout:5/708: stat d0/db/d16/d1c/d34/f53 0 2024-03-22T06:15:02.052 INFO:tasks.workunit.client.0.smithi006.stdout:5/709: chown d0/db/d99/fc4 103853939 1 2024-03-22T06:15:02.052 INFO:tasks.workunit.client.0.smithi006.stdout:6/503: dwrite d6/f13 [0,4194304] 0 2024-03-22T06:15:02.061 INFO:tasks.workunit.client.0.smithi006.stdout:5/710: dread d0/db/fd [8388608,4194304] 0 2024-03-22T06:15:02.061 INFO:tasks.workunit.client.0.smithi006.stdout:5/711: fdatasync d0/db/d50/d96/fae 0 2024-03-22T06:15:02.064 INFO:tasks.workunit.client.0.smithi006.stdout:6/504: read d6/d1b/d2c/f8c [1819791,114552] 0 2024-03-22T06:15:02.064 INFO:tasks.workunit.client.0.smithi006.stdout:2/558: dwrite d13/d34/d4d/f5c [0,4194304] 0 2024-03-22T06:15:02.067 INFO:tasks.workunit.client.0.smithi006.stdout:2/559: dread d13/d34/d4d/d4e/d75/da5/d6c/f78 [0,4194304] 0 2024-03-22T06:15:02.075 INFO:tasks.workunit.client.0.smithi006.stdout:2/560: symlink d13/d53/ldc 0 2024-03-22T06:15:02.075 INFO:tasks.workunit.client.0.smithi006.stdout:1/452: getdents d8/d9/d3a/d40/d44 0 2024-03-22T06:15:02.075 INFO:tasks.workunit.client.0.smithi006.stdout:1/453: creat d8/d9/d13/d24/d32/d34/f99 x:0 0 0 2024-03-22T06:15:02.075 INFO:tasks.workunit.client.0.smithi006.stdout:1/454: write d8/d9/d13/f6d [4782145,39835] 0 2024-03-22T06:15:02.078 INFO:tasks.workunit.client.0.smithi006.stdout:2/561: truncate d13/d20/d35/f86 1771940 0 2024-03-22T06:15:02.078 INFO:tasks.workunit.client.0.smithi006.stdout:2/562: fdatasync d13/d20/d26/d19/f73 0 2024-03-22T06:15:02.087 INFO:tasks.workunit.client.0.smithi006.stdout:2/563: dread d13/d34/d4d/f5d [0,4194304] 0 2024-03-22T06:15:02.087 INFO:tasks.workunit.client.0.smithi006.stdout:1/455: read d8/d9/d13/f26 [3891268,93675] 0 2024-03-22T06:15:02.089 INFO:tasks.workunit.client.0.smithi006.stdout:5/712: dwrite d0/db/d16/d1c/d34/d4d/f6b [0,4194304] 0 2024-03-22T06:15:02.089 INFO:tasks.workunit.client.0.smithi006.stdout:5/713: fsync d0/db/d16/d1c/d34/d35/d85/f89 0 2024-03-22T06:15:02.089 INFO:tasks.workunit.client.0.smithi006.stdout:5/714: fsync d0/f1a 0 2024-03-22T06:15:02.090 INFO:tasks.workunit.client.0.smithi006.stdout:6/505: dwrite d6/d1b/d2c/f43 [0,4194304] 0 2024-03-22T06:15:02.090 INFO:tasks.workunit.client.0.smithi006.stdout:6/506: dread - d6/d1d/d64/f6a zero size 2024-03-22T06:15:02.090 INFO:tasks.workunit.client.0.smithi006.stdout:6/507: write d6/d24/d44/f5f [3232082,106219] 0 2024-03-22T06:15:02.091 INFO:tasks.workunit.client.0.smithi006.stdout:2/564: truncate d13/d34/d4d/d4e/d75/da5/f94 292376 0 2024-03-22T06:15:02.091 INFO:tasks.workunit.client.0.smithi006.stdout:2/565: dread - d13/d34/d4d/f88 zero size 2024-03-22T06:15:02.091 INFO:tasks.workunit.client.0.smithi006.stdout:2/566: chown d13/d20/d35/d65/f7b 7 1 2024-03-22T06:15:02.091 INFO:tasks.workunit.client.0.smithi006.stdout:2/567: dread - d13/d53/d6b/fc0 zero size 2024-03-22T06:15:02.093 INFO:tasks.workunit.client.0.smithi006.stdout:1/456: rename d8/d9/d3a/c1a to d8/d9/d10/c9a 0 2024-03-22T06:15:02.101 INFO:tasks.workunit.client.0.smithi006.stdout:2/568: stat d13/d20/d35/l7c 0 2024-03-22T06:15:02.102 INFO:tasks.workunit.client.0.smithi006.stdout:5/715: dread d0/db/d16/f21 [4194304,4194304] 0 2024-03-22T06:15:02.104 INFO:tasks.workunit.client.0.smithi006.stdout:6/508: rename d6/d1d/f26 to d6/d24/fa8 0 2024-03-22T06:15:02.106 INFO:tasks.workunit.client.0.smithi006.stdout:1/457: dwrite d8/d9/d3a/f6b [0,4194304] 0 2024-03-22T06:15:02.109 INFO:tasks.workunit.client.0.smithi006.stdout:2/569: rename d13/f92 to d13/d53/d6b/fdd 0 2024-03-22T06:15:02.109 INFO:tasks.workunit.client.0.smithi006.stdout:2/570: creat d13/d53/dc5/fde x:0 0 0 2024-03-22T06:15:02.109 INFO:tasks.workunit.client.0.smithi006.stdout:2/571: creat d13/d34/d6e/fdf x:0 0 0 2024-03-22T06:15:02.124 INFO:tasks.workunit.client.0.smithi006.stdout:6/509: getdents d6/d7c/d83 0 2024-03-22T06:15:02.127 INFO:tasks.workunit.client.0.smithi006.stdout:1/458: dwrite d8/d9/d3a/d18/f37 [0,4194304] 0 2024-03-22T06:15:02.127 INFO:tasks.workunit.client.0.smithi006.stdout:1/459: creat d8/d9/d13/d24/f9b x:0 0 0 2024-03-22T06:15:02.127 INFO:tasks.workunit.client.0.smithi006.stdout:1/460: chown d8/d33/l6c 108259447 1 2024-03-22T06:15:02.127 INFO:tasks.workunit.client.0.smithi006.stdout:1/461: creat d8/d33/f9c x:0 0 0 2024-03-22T06:15:02.128 INFO:tasks.workunit.client.0.smithi006.stdout:2/572: getdents d13/d20/d35/d65 0 2024-03-22T06:15:02.130 INFO:tasks.workunit.client.0.smithi006.stdout:6/510: dread d6/f51 [0,4194304] 0 2024-03-22T06:15:02.130 INFO:tasks.workunit.client.0.smithi006.stdout:6/511: chown d6/d1b/d2c/f59 9043039 1 2024-03-22T06:15:02.140 INFO:tasks.workunit.client.0.smithi006.stdout:2/573: rmdir d13/d34/d4d/d4e/d75/d62 39 2024-03-22T06:15:02.140 INFO:tasks.workunit.client.0.smithi006.stdout:2/574: fdatasync d13/f23 0 2024-03-22T06:15:02.140 INFO:tasks.workunit.client.0.smithi006.stdout:2/575: chown d13/d20/d26/db9/cc6 1 1 2024-03-22T06:15:02.141 INFO:tasks.workunit.client.0.smithi006.stdout:2/576: dread d13/d20/d35/d65/f68 [0,4194304] 0 2024-03-22T06:15:02.145 INFO:tasks.workunit.client.0.smithi006.stdout:2/577: rename d13/d34/d4d/d4e/d75/da5/d82/fa7 to d13/d34/d4d/d4e/fe0 0 2024-03-22T06:15:02.146 INFO:tasks.workunit.client.0.smithi006.stdout:2/578: write d13/d34/d4d/d4e/d75/da5/d6c/f78 [319547,5406] 0 2024-03-22T06:15:02.146 INFO:tasks.workunit.client.0.smithi006.stdout:2/579: stat d13/d34/d4d/d4e/d75/d62/fb6 0 2024-03-22T06:15:02.148 INFO:tasks.workunit.client.0.smithi006.stdout:2/580: rmdir d13/d20/d26/db9 39 2024-03-22T06:15:02.152 INFO:tasks.workunit.client.0.smithi006.stdout:2/581: unlink d13/d34/c93 0 2024-03-22T06:15:02.153 INFO:tasks.workunit.client.0.smithi006.stdout:2/582: truncate d13/d34/d4d/fbf 149753 0 2024-03-22T06:15:02.153 INFO:tasks.workunit.client.0.smithi006.stdout:6/512: dwrite d6/d24/d44/f5f [0,4194304] 0 2024-03-22T06:15:02.153 INFO:tasks.workunit.client.0.smithi006.stdout:6/513: chown d6/d1d/d5c/d89 1 1 2024-03-22T06:15:02.153 INFO:tasks.workunit.client.0.smithi006.stdout:6/514: write d6/d1d/d5c/d89/f8b [1524172,100392] 0 2024-03-22T06:15:02.156 INFO:tasks.workunit.client.0.smithi006.stdout:6/515: rename d6/d1d/d64/f6a to d6/d6f/d94/fa9 0 2024-03-22T06:15:02.164 INFO:tasks.workunit.client.0.smithi006.stdout:2/583: dread d13/d34/d4d/fdb [0,4194304] 0 2024-03-22T06:15:02.171 INFO:tasks.workunit.client.0.smithi006.stdout:2/584: dread fe [0,4194304] 0 2024-03-22T06:15:02.171 INFO:tasks.workunit.client.0.smithi006.stdout:2/585: write d13/d20/d35/fd2 [838853,8551] 0 2024-03-22T06:15:02.171 INFO:tasks.workunit.client.0.smithi006.stdout:2/586: chown d13/d20/f42 1623705 1 2024-03-22T06:15:02.173 INFO:tasks.workunit.client.0.smithi006.stdout:2/587: rename d13/d34/d6e/f84 to d13/d20/d26/fe1 0 2024-03-22T06:15:02.174 INFO:tasks.workunit.client.0.smithi006.stdout:6/516: dwrite d6/d1d/d5c/d81/f7e [0,4194304] 0 2024-03-22T06:15:02.177 INFO:tasks.workunit.client.0.smithi006.stdout:2/588: getdents d13 0 2024-03-22T06:15:02.177 INFO:tasks.workunit.client.0.smithi006.stdout:2/589: truncate d13/d53/d6b/f97 619245 0 2024-03-22T06:15:02.179 INFO:tasks.workunit.client.0.smithi006.stdout:2/590: mknod d13/d20/ce2 0 2024-03-22T06:15:02.179 INFO:tasks.workunit.client.0.smithi006.stdout:2/591: truncate d13/d34/d4d/d4e/d75/da5/d82/fae 114113 0 2024-03-22T06:15:02.180 INFO:tasks.workunit.client.0.smithi006.stdout:2/592: write d13/d34/d4d/d4e/d75/da5/f67 [4183701,51623] 0 2024-03-22T06:15:02.182 INFO:tasks.workunit.client.0.smithi006.stdout:6/517: read d6/d1d/d32/f8d [3309645,103469] 0 2024-03-22T06:15:02.183 INFO:tasks.workunit.client.0.smithi006.stdout:2/593: mkdir d13/d34/d4d/d4e/de3 0 2024-03-22T06:15:02.183 INFO:tasks.workunit.client.0.smithi006.stdout:2/594: chown d13/d53/c2d 192 1 2024-03-22T06:15:02.183 INFO:tasks.workunit.client.0.smithi006.stdout:2/595: chown d13/d20/d26/d19/fb3 3464582 1 2024-03-22T06:15:02.185 INFO:tasks.workunit.client.0.smithi006.stdout:2/596: read d13/d53/d6b/fdd [126133,108938] 0 2024-03-22T06:15:02.185 INFO:tasks.workunit.client.0.smithi006.stdout:2/597: chown d13/d34/d4d/d4e/d75/d62/lac 813950 1 2024-03-22T06:15:02.187 INFO:tasks.workunit.client.0.smithi006.stdout:2/598: mknod d13/d53/d6b/dd8/ce4 0 2024-03-22T06:15:02.187 INFO:tasks.workunit.client.0.smithi006.stdout:2/599: fsync d13/d20/d26/f79 0 2024-03-22T06:15:02.206 INFO:tasks.workunit.client.0.smithi006.stdout:6/518: dwrite d6/d1b/d2c/f59 [4194304,4194304] 0 2024-03-22T06:15:02.207 INFO:tasks.workunit.client.0.smithi006.stdout:6/519: chown d6/d24/f58 1 1 2024-03-22T06:15:02.207 INFO:tasks.workunit.client.0.smithi006.stdout:2/600: dwrite d13/d34/d4d/d4e/d75/d62/fbd [0,4194304] 0 2024-03-22T06:15:02.207 INFO:tasks.workunit.client.0.smithi006.stdout:2/601: dread - d13/d20/f74 zero size 2024-03-22T06:15:02.207 INFO:tasks.workunit.client.0.smithi006.stdout:2/602: dread - d13/d53/fc8 zero size 2024-03-22T06:15:02.220 INFO:tasks.workunit.client.0.smithi006.stdout:6/520: dwrite d6/d1b/d2c/f74 [0,4194304] 0 2024-03-22T06:15:02.227 INFO:tasks.workunit.client.0.smithi006.stdout:2/603: mknod d13/d53/dc5/ce5 0 2024-03-22T06:15:02.232 INFO:tasks.workunit.client.0.smithi006.stdout:6/521: rmdir d6/d1d/d5c/d81/d98 0 2024-03-22T06:15:02.232 INFO:tasks.workunit.client.0.smithi006.stdout:6/522: readlink d6/d1d/d32/l3f 0 2024-03-22T06:15:02.235 INFO:tasks.workunit.client.0.smithi006.stdout:2/604: creat d13/d53/d6b/dd8/fe6 x:0 0 0 2024-03-22T06:15:02.235 INFO:tasks.workunit.client.0.smithi006.stdout:2/605: read - d13/d34/d4d/d4e/d75/f9a zero size 2024-03-22T06:15:02.235 INFO:tasks.workunit.client.0.smithi006.stdout:2/606: chown d13/d34/d4d/d4e/d75/da5/d82 173626 1 2024-03-22T06:15:02.235 INFO:tasks.workunit.client.0.smithi006.stdout:2/607: stat d13/d34/d4d/d4e/ld4 0 2024-03-22T06:15:02.235 INFO:tasks.workunit.client.0.smithi006.stdout:2/608: chown d13/d34/d6e/d72/f80 6231 1 2024-03-22T06:15:02.237 INFO:tasks.workunit.client.0.smithi006.stdout:2/609: mknod d13/d34/d4d/d4e/d75/dcc/ce7 0 2024-03-22T06:15:02.240 INFO:tasks.workunit.client.0.smithi006.stdout:6/523: getdents d6/d24 0 2024-03-22T06:15:02.240 INFO:tasks.workunit.client.0.smithi006.stdout:6/524: stat d6/d1b/d2c/f4b 0 2024-03-22T06:15:02.242 INFO:tasks.workunit.client.0.smithi006.stdout:6/525: creat d6/d7c/d92/faa x:0 0 0 2024-03-22T06:15:02.244 INFO:tasks.workunit.client.0.smithi006.stdout:6/526: write d6/f40 [1733845,54538] 0 2024-03-22T06:15:02.247 INFO:tasks.workunit.client.0.smithi006.stdout:6/527: mkdir d6/d1d/d5c/d81/dab 0 2024-03-22T06:15:02.247 INFO:tasks.workunit.client.0.smithi006.stdout:6/528: readlink d6/d1d/d5c/la7 0 2024-03-22T06:15:02.248 INFO:tasks.workunit.client.0.smithi006.stdout:6/529: chown d6/d1d/l7a 128 1 2024-03-22T06:15:02.264 INFO:tasks.workunit.client.0.smithi006.stdout:3/536: sync 2024-03-22T06:15:02.264 INFO:tasks.workunit.client.0.smithi006.stdout:0/400: sync 2024-03-22T06:15:02.265 INFO:tasks.workunit.client.0.smithi006.stdout:4/591: sync 2024-03-22T06:15:02.265 INFO:tasks.workunit.client.0.smithi006.stdout:6/530: dwrite d6/d1d/d32/f41 [0,4194304] 0 2024-03-22T06:15:02.268 INFO:tasks.workunit.client.0.smithi006.stdout:3/537: creat d0/d8/d1c/d3e/d5f/d90/da7/d97/fb7 x:0 0 0 2024-03-22T06:15:02.268 INFO:tasks.workunit.client.0.smithi006.stdout:3/538: readlink d0/d8/d1c/d3e/d5f/d90/l9d 0 2024-03-22T06:15:02.270 INFO:tasks.workunit.client.0.smithi006.stdout:4/592: stat d6/d1b/d20/d63/d93/d9a/lc0 0 2024-03-22T06:15:02.272 INFO:tasks.workunit.client.0.smithi006.stdout:6/531: fdatasync d6/f16 0 2024-03-22T06:15:02.274 INFO:tasks.workunit.client.0.smithi006.stdout:0/401: dread db/dc/d37/f63 [0,4194304] 0 2024-03-22T06:15:02.274 INFO:tasks.workunit.client.0.smithi006.stdout:0/402: readlink db/d10/l1e 0 2024-03-22T06:15:02.274 INFO:tasks.workunit.client.0.smithi006.stdout:0/403: dread - db/f64 zero size 2024-03-22T06:15:02.279 INFO:tasks.workunit.client.0.smithi006.stdout:4/593: rename d6/c23 to d6/d1b/d20/d21/d2a/d43/da8/cd1 0 2024-03-22T06:15:02.279 INFO:tasks.workunit.client.0.smithi006.stdout:4/594: stat d6/d1b/d20/d63/d93/d9a/d8f 0 2024-03-22T06:15:02.279 INFO:tasks.workunit.client.0.smithi006.stdout:4/595: readlink d6/d13/d3b/l40 0 2024-03-22T06:15:02.280 INFO:tasks.workunit.client.0.smithi006.stdout:6/532: mkdir d6/d1d/d5c/d81/d8a/dac 0 2024-03-22T06:15:02.281 INFO:tasks.workunit.client.0.smithi006.stdout:6/533: creat d6/d7c/d92/fad x:0 0 0 2024-03-22T06:15:02.282 INFO:tasks.workunit.client.0.smithi006.stdout:3/539: getdents d0/d8/d6a 0 2024-03-22T06:15:02.282 INFO:tasks.workunit.client.0.smithi006.stdout:3/540: fdatasync d0/d8/d9/f32 0 2024-03-22T06:15:02.282 INFO:tasks.workunit.client.0.smithi006.stdout:3/541: chown d0/d8/d9/l5a 263153 1 2024-03-22T06:15:02.283 INFO:tasks.workunit.client.0.smithi006.stdout:0/404: symlink db/d10/d33/l8c 0 2024-03-22T06:15:02.287 INFO:tasks.workunit.client.0.smithi006.stdout:4/596: rename d6/d13/d3b/l3e to d6/d1b/d20/d21/d2a/dc9/ld2 0 2024-03-22T06:15:02.287 INFO:tasks.workunit.client.0.smithi006.stdout:4/597: read f2 [1219289,17187] 0 2024-03-22T06:15:02.287 INFO:tasks.workunit.client.0.smithi006.stdout:4/598: fdatasync d6/fb 0 2024-03-22T06:15:02.291 INFO:tasks.workunit.client.0.smithi006.stdout:6/534: truncate d6/d1b/d2c/f43 2862961 0 2024-03-22T06:15:02.292 INFO:tasks.workunit.client.0.smithi006.stdout:6/535: chown d6/d24/c3a 1 1 2024-03-22T06:15:02.295 INFO:tasks.workunit.client.0.smithi006.stdout:3/542: unlink d0/l45 0 2024-03-22T06:15:02.299 INFO:tasks.workunit.client.0.smithi006.stdout:0/405: symlink db/d3d/d4e/d76/l8d 0 2024-03-22T06:15:02.303 INFO:tasks.workunit.client.0.smithi006.stdout:3/543: unlink d0/d4/d31/cb4 0 2024-03-22T06:15:02.305 INFO:tasks.workunit.client.0.smithi006.stdout:0/406: creat db/dc/d38/f8e x:0 0 0 2024-03-22T06:15:02.305 INFO:tasks.workunit.client.0.smithi006.stdout:0/407: fdatasync db/d10/d35/f43 0 2024-03-22T06:15:02.306 INFO:tasks.workunit.client.0.smithi006.stdout:3/544: truncate d0/d8/d1c/d3e/d5f/da3/f48 23202 0 2024-03-22T06:15:02.307 INFO:tasks.workunit.client.0.smithi006.stdout:3/545: dread d0/d8/d1c/d49/f67 [0,4194304] 0 2024-03-22T06:15:02.308 INFO:tasks.workunit.client.0.smithi006.stdout:0/408: rename l3 to db/l8f 0 2024-03-22T06:15:02.309 INFO:tasks.workunit.client.0.smithi006.stdout:4/599: dwrite d6/d1b/d20/d63/d93/faa [0,4194304] 0 2024-03-22T06:15:02.309 INFO:tasks.workunit.client.0.smithi006.stdout:4/600: write f3 [3311867,57435] 0 2024-03-22T06:15:02.309 INFO:tasks.workunit.client.0.smithi006.stdout:4/601: fsync d6/d1b/d20/d63/d93/fc6 0 2024-03-22T06:15:02.309 INFO:tasks.workunit.client.0.smithi006.stdout:4/602: dread - d6/d1b/d20/d21/d2a/d43/d4b/fa7 zero size 2024-03-22T06:15:02.309 INFO:tasks.workunit.client.0.smithi006.stdout:6/536: dwrite d6/f11 [0,4194304] 0 2024-03-22T06:15:02.310 INFO:tasks.workunit.client.0.smithi006.stdout:6/537: write d6/d7c/d92/faa [480853,4993] 0 2024-03-22T06:15:02.310 INFO:tasks.workunit.client.0.smithi006.stdout:3/546: mknod d0/d23/d33/d3d/cb8 0 2024-03-22T06:15:02.314 INFO:tasks.workunit.client.0.smithi006.stdout:0/409: rename db/d10/d35/d6c/f86 to db/d3d/d4e/f90 0 2024-03-22T06:15:02.314 INFO:tasks.workunit.client.0.smithi006.stdout:0/410: fdatasync db/d10/d33/f4d 0 2024-03-22T06:15:02.322 INFO:tasks.workunit.client.0.smithi006.stdout:4/603: truncate d6/f12 8342740 0 2024-03-22T06:15:02.322 INFO:tasks.workunit.client.0.smithi006.stdout:4/604: chown d6/d1b/d20/d6b/d95 2269618 1 2024-03-22T06:15:02.328 INFO:tasks.workunit.client.0.smithi006.stdout:3/547: unlink d0/l64 0 2024-03-22T06:15:02.331 INFO:tasks.workunit.client.0.smithi006.stdout:0/411: creat db/d10/d33/f91 x:0 0 0 2024-03-22T06:15:02.333 INFO:tasks.workunit.client.0.smithi006.stdout:4/605: creat d6/d1b/d20/d21/d2a/d43/da8/fd3 x:0 0 0 2024-03-22T06:15:02.333 INFO:tasks.workunit.client.0.smithi006.stdout:4/606: chown d6/fbd 3 1 2024-03-22T06:15:02.333 INFO:tasks.workunit.client.0.smithi006.stdout:4/607: stat d6/d1b/d20/d6b/d60/db1 0 2024-03-22T06:15:02.336 INFO:tasks.workunit.client.0.smithi006.stdout:3/548: dread d0/f3 [0,4194304] 0 2024-03-22T06:15:02.336 INFO:tasks.workunit.client.0.smithi006.stdout:6/538: mkdir d6/d1b/dae 0 2024-03-22T06:15:02.336 INFO:tasks.workunit.client.0.smithi006.stdout:6/539: dread - d6/d6f/f6b zero size 2024-03-22T06:15:02.337 INFO:tasks.workunit.client.0.smithi006.stdout:0/412: creat db/d3d/f92 x:0 0 0 2024-03-22T06:15:02.339 INFO:tasks.workunit.client.0.smithi006.stdout:4/608: link d6/d1b/d20/d6b/d60/l83 d6/d1b/d20/d6b/d60/ld4 0 2024-03-22T06:15:02.341 INFO:tasks.workunit.client.0.smithi006.stdout:3/549: write d0/d23/f7f [1094875,22311] 0 2024-03-22T06:15:02.341 INFO:tasks.workunit.client.0.smithi006.stdout:3/550: readlink d0/d4/l85 0 2024-03-22T06:15:02.341 INFO:tasks.workunit.client.0.smithi006.stdout:6/540: mknod d6/d1d/caf 0 2024-03-22T06:15:02.347 INFO:tasks.workunit.client.0.smithi006.stdout:0/413: rename db/f64 to db/d87/f93 0 2024-03-22T06:15:02.352 INFO:tasks.workunit.client.0.smithi006.stdout:4/609: symlink d6/d1b/d20/d6b/d60/ld5 0 2024-03-22T06:15:02.357 INFO:tasks.workunit.client.0.smithi006.stdout:4/610: symlink d6/d13/ld6 0 2024-03-22T06:15:02.388 INFO:tasks.workunit.client.0.smithi006.stdout:0/414: dwrite db/dc/d37/f77 [0,4194304] 0 2024-03-22T06:15:02.389 INFO:tasks.workunit.client.0.smithi006.stdout:0/415: write db/f45 [3485785,125821] 0 2024-03-22T06:15:02.389 INFO:tasks.workunit.client.0.smithi006.stdout:0/416: creat db/dc/d38/f94 x:0 0 0 2024-03-22T06:15:02.389 INFO:tasks.workunit.client.0.smithi006.stdout:0/417: chown db/d3d/d4e/d76/d81 629 1 2024-03-22T06:15:02.389 INFO:tasks.workunit.client.0.smithi006.stdout:0/418: write db/d3d/d4e/f90 [232181,74789] 0 2024-03-22T06:15:02.389 INFO:tasks.workunit.client.0.smithi006.stdout:6/541: dwrite d6/d7c/d83/fa2 [0,4194304] 0 2024-03-22T06:15:02.389 INFO:tasks.workunit.client.0.smithi006.stdout:4/611: dwrite d6/d1b/d20/d63/fba [0,4194304] 0 2024-03-22T06:15:02.389 INFO:tasks.workunit.client.0.smithi006.stdout:3/551: dwrite d0/d23/d33/f44 [0,4194304] 0 2024-03-22T06:15:02.389 INFO:tasks.workunit.client.0.smithi006.stdout:3/552: stat d0/d8/d1c/d3e/d5f/da3/f47 0 2024-03-22T06:15:02.390 INFO:tasks.workunit.client.0.smithi006.stdout:3/553: stat d0/d8/d1c/d95/f9b 0 2024-03-22T06:15:02.390 INFO:tasks.workunit.client.0.smithi006.stdout:3/554: truncate d0/d23/d33/f65 977913 0 2024-03-22T06:15:02.390 INFO:tasks.workunit.client.0.smithi006.stdout:3/555: creat d0/d23/d33/d3d/fb9 x:0 0 0 2024-03-22T06:15:02.399 INFO:tasks.workunit.client.0.smithi006.stdout:6/542: truncate d6/d1d/d32/f45 503235 0 2024-03-22T06:15:02.402 INFO:tasks.workunit.client.0.smithi006.stdout:4/612: truncate d6/f7 66508 0 2024-03-22T06:15:02.404 INFO:tasks.workunit.client.0.smithi006.stdout:3/556: dwrite d0/d8/d1c/d72/f87 [0,4194304] 0 2024-03-22T06:15:02.420 INFO:tasks.workunit.client.0.smithi006.stdout:6/543: unlink d6/d1d/d64/c75 0 2024-03-22T06:15:02.423 INFO:tasks.workunit.client.0.smithi006.stdout:4/613: rename d6/d1b/d20/d21/l76 to d6/d1b/d20/ld7 0 2024-03-22T06:15:02.424 INFO:tasks.workunit.client.0.smithi006.stdout:4/614: write d6/d1b/d20/d63/d93/fc6 [631355,7431] 0 2024-03-22T06:15:02.428 INFO:tasks.workunit.client.0.smithi006.stdout:6/544: write d6/d1d/f71 [103705,120244] 0 2024-03-22T06:15:02.436 INFO:tasks.workunit.client.0.smithi006.stdout:0/419: truncate db/f45 3326713 0 2024-03-22T06:15:02.436 INFO:tasks.workunit.client.0.smithi006.stdout:0/420: write db/d10/f1a [4901179,34915] 0 2024-03-22T06:15:02.438 INFO:tasks.workunit.client.0.smithi006.stdout:3/557: link d0/d8/d1c/d3e/d5f/lb0 d0/d23/lba 0 2024-03-22T06:15:02.441 INFO:tasks.workunit.client.0.smithi006.stdout:4/615: mknod d6/d1b/d20/d63/d93/d9a/d8f/da1/db5/cd8 0 2024-03-22T06:15:02.449 INFO:tasks.workunit.client.0.smithi006.stdout:0/421: stat db/d10/c2e 0 2024-03-22T06:15:02.451 INFO:tasks.workunit.client.0.smithi006.stdout:3/558: mkdir d0/d4/dbb 0 2024-03-22T06:15:02.452 INFO:tasks.workunit.client.0.smithi006.stdout:3/559: fsync d0/d8/d1c/d3e/d68/f6b 0 2024-03-22T06:15:02.453 INFO:tasks.workunit.client.0.smithi006.stdout:6/545: dwrite d6/d1d/d5c/f5e [0,4194304] 0 2024-03-22T06:15:02.453 INFO:tasks.workunit.client.0.smithi006.stdout:6/546: write d6/d1b/d2c/f72 [5148445,67955] 0 2024-03-22T06:15:02.454 INFO:tasks.workunit.client.0.smithi006.stdout:6/547: truncate d6/d7c/d92/faa 730846 0 2024-03-22T06:15:02.454 INFO:tasks.workunit.client.0.smithi006.stdout:6/548: fsync d6/d1d/d32/f8d 0 2024-03-22T06:15:02.455 INFO:tasks.workunit.client.0.smithi006.stdout:0/422: rename db/d87/l5c to db/d10/d35/d7b/l95 0 2024-03-22T06:15:02.456 INFO:tasks.workunit.client.0.smithi006.stdout:0/423: dread - db/d10/d2a/d41/f82 zero size 2024-03-22T06:15:02.456 INFO:tasks.workunit.client.0.smithi006.stdout:0/424: fsync db/dc/d38/f8e 0 2024-03-22T06:15:02.461 INFO:tasks.workunit.client.0.smithi006.stdout:6/549: creat d6/d1d/d5c/d89/fb0 x:0 0 0 2024-03-22T06:15:02.462 INFO:tasks.workunit.client.0.smithi006.stdout:6/550: write d6/f40 [1102534,31786] 0 2024-03-22T06:15:02.464 INFO:tasks.workunit.client.0.smithi006.stdout:3/560: rename d0/d8/d1c/f2c to d0/d8/d9/db1/fbc 0 2024-03-22T06:15:02.465 INFO:tasks.workunit.client.0.smithi006.stdout:4/616: dwrite d6/d1b/d20/d6b/d60/f68 [0,4194304] 0 2024-03-22T06:15:02.467 INFO:tasks.workunit.client.0.smithi006.stdout:0/425: rmdir db/d10/d35/d7b 39 2024-03-22T06:15:02.467 INFO:tasks.workunit.client.0.smithi006.stdout:0/426: read db/d10/f1a [3599057,21614] 0 2024-03-22T06:15:02.467 INFO:tasks.workunit.client.0.smithi006.stdout:0/427: creat db/f96 x:0 0 0 2024-03-22T06:15:02.471 INFO:tasks.workunit.client.0.smithi006.stdout:6/551: link d6/d24/d44/f46 d6/d1d/d99/fb1 0 2024-03-22T06:15:02.473 INFO:tasks.workunit.client.0.smithi006.stdout:0/428: read db/d10/d35/f69 [4067416,59373] 0 2024-03-22T06:15:02.473 INFO:tasks.workunit.client.0.smithi006.stdout:8/651: sync 2024-03-22T06:15:02.473 INFO:tasks.workunit.client.0.smithi006.stdout:7/608: sync 2024-03-22T06:15:02.473 INFO:tasks.workunit.client.0.smithi006.stdout:9/453: sync 2024-03-22T06:15:02.474 INFO:tasks.workunit.client.0.smithi006.stdout:5/716: sync 2024-03-22T06:15:02.474 INFO:tasks.workunit.client.0.smithi006.stdout:2/610: sync 2024-03-22T06:15:02.474 INFO:tasks.workunit.client.0.smithi006.stdout:1/462: sync 2024-03-22T06:15:02.474 INFO:tasks.workunit.client.0.smithi006.stdout:1/463: fsync d8/d9/d13/d24/d32/f6f 0 2024-03-22T06:15:02.474 INFO:tasks.workunit.client.0.smithi006.stdout:1/464: dread - d8/d9/d13/d24/f9b zero size 2024-03-22T06:15:02.484 INFO:tasks.workunit.client.0.smithi006.stdout:7/609: mknod d5/d1e/dbc/cc2 0 2024-03-22T06:15:02.487 INFO:tasks.workunit.client.0.smithi006.stdout:8/652: dread d9/db/d12/d1e/f39 [0,4194304] 0 2024-03-22T06:15:02.487 INFO:tasks.workunit.client.0.smithi006.stdout:2/611: creat d13/d34/fe8 x:0 0 0 2024-03-22T06:15:02.489 INFO:tasks.workunit.client.0.smithi006.stdout:2/612: mknod d13/d34/d4d/d4e/d75/da5/d6c/ce9 0 2024-03-22T06:15:02.489 INFO:tasks.workunit.client.0.smithi006.stdout:2/613: truncate d13/d53/d6b/dd8/fe6 325376 0 2024-03-22T06:15:02.492 INFO:tasks.workunit.client.0.smithi006.stdout:5/717: rename d0/db/d16/d1c/d34/d35/d85/d54 to d0/db/d16/d1c/d34/d35/d85/dda 0 2024-03-22T06:15:02.494 INFO:tasks.workunit.client.0.smithi006.stdout:0/429: read db/d3d/d4e/f54 [3438007,39309] 0 2024-03-22T06:15:02.495 INFO:tasks.workunit.client.0.smithi006.stdout:2/614: mkdir d13/d20/d26/dca/dea 0 2024-03-22T06:15:02.498 INFO:tasks.workunit.client.0.smithi006.stdout:5/718: unlink d0/db/fd 0 2024-03-22T06:15:02.499 INFO:tasks.workunit.client.0.smithi006.stdout:5/719: fdatasync d0/db/d16/d1c/d34/f53 0 2024-03-22T06:15:02.506 INFO:tasks.workunit.client.0.smithi006.stdout:7/610: dread d5/d1e/d3f/f45 [0,4194304] 0 2024-03-22T06:15:02.509 INFO:tasks.workunit.client.0.smithi006.stdout:7/611: rename d5/d1e/c7c to d5/cc3 0 2024-03-22T06:15:02.509 INFO:tasks.workunit.client.0.smithi006.stdout:7/612: creat d5/d1e/d63/d85/d80/fc4 x:0 0 0 2024-03-22T06:15:02.516 INFO:tasks.workunit.client.0.smithi006.stdout:4/617: dwrite d6/d1b/d20/d6b/fae [0,4194304] 0 2024-03-22T06:15:02.516 INFO:tasks.workunit.client.0.smithi006.stdout:4/618: chown d6/d1b/d20/d63/d93/d9a/lc0 1028 1 2024-03-22T06:15:02.518 INFO:tasks.workunit.client.0.smithi006.stdout:4/619: write d6/d1b/f54 [1852759,35107] 0 2024-03-22T06:15:02.521 INFO:tasks.workunit.client.0.smithi006.stdout:4/620: mknod d6/d1b/d20/d21/d2a/d43/da8/cd9 0 2024-03-22T06:15:02.532 INFO:tasks.workunit.client.0.smithi006.stdout:4/621: read d6/d1b/d20/d63/fba [2345680,76830] 0 2024-03-22T06:15:02.532 INFO:tasks.workunit.client.0.smithi006.stdout:4/622: write d6/d13/f17 [8766521,29695] 0 2024-03-22T06:15:02.533 INFO:tasks.workunit.client.0.smithi006.stdout:4/623: mknod d6/cda 0 2024-03-22T06:15:02.534 INFO:tasks.workunit.client.0.smithi006.stdout:9/454: dwrite d8/d10/d47/f97 [0,4194304] 0 2024-03-22T06:15:02.535 INFO:tasks.workunit.client.0.smithi006.stdout:6/552: dwrite d6/d24/f31 [0,4194304] 0 2024-03-22T06:15:02.535 INFO:tasks.workunit.client.0.smithi006.stdout:6/553: chown d6/d24/c7f 1 1 2024-03-22T06:15:02.535 INFO:tasks.workunit.client.0.smithi006.stdout:6/554: stat d6/d24/f58 0 2024-03-22T06:15:02.536 INFO:tasks.workunit.client.0.smithi006.stdout:1/465: dwrite d8/d71/f8e [4194304,4194304] 0 2024-03-22T06:15:02.536 INFO:tasks.workunit.client.0.smithi006.stdout:1/466: truncate d8/d9/d3a/d40/f7d 497424 0 2024-03-22T06:15:02.536 INFO:tasks.workunit.client.0.smithi006.stdout:8/653: dwrite d9/d14/d47/f64 [0,4194304] 0 2024-03-22T06:15:02.536 INFO:tasks.workunit.client.0.smithi006.stdout:8/654: dread - d9/d14/d47/d94/da3/fda zero size 2024-03-22T06:15:02.536 INFO:tasks.workunit.client.0.smithi006.stdout:8/655: readlink d9/db/d12/d66/d6c/d71/d79/la5 0 2024-03-22T06:15:02.537 INFO:tasks.workunit.client.0.smithi006.stdout:4/624: rmdir d6/d1b/d20/d21/d9f 39 2024-03-22T06:15:02.537 INFO:tasks.workunit.client.0.smithi006.stdout:3/561: dwrite d0/f56 [0,4194304] 0 2024-03-22T06:15:02.537 INFO:tasks.workunit.client.0.smithi006.stdout:0/430: dwrite db/d3d/f92 [0,4194304] 0 2024-03-22T06:15:02.537 INFO:tasks.workunit.client.0.smithi006.stdout:2/615: dwrite d13/d34/d4d/f88 [0,4194304] 0 2024-03-22T06:15:02.537 INFO:tasks.workunit.client.0.smithi006.stdout:0/431: read db/dc/d38/f4b [454196,93622] 0 2024-03-22T06:15:02.537 INFO:tasks.workunit.client.0.smithi006.stdout:2/616: truncate d13/d34/d4d/d4e/d75/fb7 111529 0 2024-03-22T06:15:02.538 INFO:tasks.workunit.client.0.smithi006.stdout:5/720: dwrite d0/db/d99/f74 [0,4194304] 0 2024-03-22T06:15:02.541 INFO:tasks.workunit.client.0.smithi006.stdout:1/467: dread d8/d9/f2d [0,4194304] 0 2024-03-22T06:15:02.543 INFO:tasks.workunit.client.0.smithi006.stdout:9/455: mknod d8/d10/d20/c9c 0 2024-03-22T06:15:02.545 INFO:tasks.workunit.client.0.smithi006.stdout:4/625: dread d6/d1b/d20/f33 [0,4194304] 0 2024-03-22T06:15:02.545 INFO:tasks.workunit.client.0.smithi006.stdout:4/626: fsync d6/d1b/f47 0 2024-03-22T06:15:02.551 INFO:tasks.workunit.client.0.smithi006.stdout:8/656: creat d9/db/d35/dd2/d76/db3/dc3/ff2 x:0 0 0 2024-03-22T06:15:02.551 INFO:tasks.workunit.client.0.smithi006.stdout:8/657: readlink d9/db/d35/dd2/le6 0 2024-03-22T06:15:02.552 INFO:tasks.workunit.client.0.smithi006.stdout:6/555: dwrite d6/d1b/d2c/f43 [0,4194304] 0 2024-03-22T06:15:02.553 INFO:tasks.workunit.client.0.smithi006.stdout:3/562: getdents d0/d8/d1c/d3e/d5f/da3 0 2024-03-22T06:15:02.553 INFO:tasks.workunit.client.0.smithi006.stdout:3/563: chown d0/d8/d1c/d3e/d5f/lb2 34926 1 2024-03-22T06:15:02.562 INFO:tasks.workunit.client.0.smithi006.stdout:0/432: link db/dc/d38/c58 db/d10/d35/d6c/c97 0 2024-03-22T06:15:02.565 INFO:tasks.workunit.client.0.smithi006.stdout:6/556: read d6/d1d/d32/f41 [1777436,90563] 0 2024-03-22T06:15:02.566 INFO:tasks.workunit.client.0.smithi006.stdout:2/617: mkdir d13/d34/d6e/d72/deb 0 2024-03-22T06:15:02.566 INFO:tasks.workunit.client.0.smithi006.stdout:2/618: chown d13/l3a 10281 1 2024-03-22T06:15:02.566 INFO:tasks.workunit.client.0.smithi006.stdout:2/619: creat d13/d20/da6/fec x:0 0 0 2024-03-22T06:15:02.568 INFO:tasks.workunit.client.0.smithi006.stdout:5/721: mknod d0/db/d16/d43/cdb 0 2024-03-22T06:15:02.570 INFO:tasks.workunit.client.0.smithi006.stdout:1/468: unlink d8/l83 0 2024-03-22T06:15:02.572 INFO:tasks.workunit.client.0.smithi006.stdout:9/456: rename d8/d10/d82/f8b to d8/d10/d28/f9d 0 2024-03-22T06:15:02.574 INFO:tasks.workunit.client.0.smithi006.stdout:4/627: mknod d6/d1b/d20/d63/d93/d9a/cdb 0 2024-03-22T06:15:02.578 INFO:tasks.workunit.client.0.smithi006.stdout:1/469: read d8/d9/d3a/f6b [2723181,122220] 0 2024-03-22T06:15:02.578 INFO:tasks.workunit.client.0.smithi006.stdout:1/470: dread - d8/d33/f9c zero size 2024-03-22T06:15:02.582 INFO:tasks.workunit.client.0.smithi006.stdout:3/564: mkdir d0/d23/d33/d3d/dbd 0 2024-03-22T06:15:02.586 INFO:tasks.workunit.client.0.smithi006.stdout:7/613: getdents d5/d1e/d63/d85/d80 0 2024-03-22T06:15:02.586 INFO:tasks.workunit.client.0.smithi006.stdout:7/614: truncate d5/d1e/d29/d59/f9d 476468 0 2024-03-22T06:15:02.592 INFO:tasks.workunit.client.0.smithi006.stdout:5/722: mkdir d0/db/d16/d43/ddc 0 2024-03-22T06:15:02.597 INFO:tasks.workunit.client.0.smithi006.stdout:9/457: creat d8/f9e x:0 0 0 2024-03-22T06:15:02.598 INFO:tasks.workunit.client.0.smithi006.stdout:3/565: dwrite d0/d8/d1c/d3e/d5f/d90/da7/d97/fb7 [0,4194304] 0 2024-03-22T06:15:02.598 INFO:tasks.workunit.client.0.smithi006.stdout:3/566: dread - d0/d7e/f93 zero size 2024-03-22T06:15:02.615 INFO:tasks.workunit.client.0.smithi006.stdout:0/433: truncate db/dc/f44 1667184 0 2024-03-22T06:15:02.617 INFO:tasks.workunit.client.0.smithi006.stdout:5/723: dwrite d0/db/d50/f80 [0,4194304] 0 2024-03-22T06:15:02.617 INFO:tasks.workunit.client.0.smithi006.stdout:9/458: dwrite d8/d10/d20/f2c [0,4194304] 0 2024-03-22T06:15:02.618 INFO:tasks.workunit.client.0.smithi006.stdout:7/615: truncate d5/d1e/d63/d85/fad 466030 0 2024-03-22T06:15:02.621 INFO:tasks.workunit.client.0.smithi006.stdout:2/620: rmdir d13/d34/d4d/d4e/d75 39 2024-03-22T06:15:02.622 INFO:tasks.workunit.client.0.smithi006.stdout:8/658: rename d9/db/d35/dd2/d4a/d4d/fba to d9/db/d12/d66/d6c/d71/d79/ff3 0 2024-03-22T06:15:02.629 INFO:tasks.workunit.client.0.smithi006.stdout:3/567: symlink d0/d8/d9/db1/lbe 0 2024-03-22T06:15:02.631 INFO:tasks.workunit.client.0.smithi006.stdout:0/434: dread db/d10/d35/f69 [0,4194304] 0 2024-03-22T06:15:02.631 INFO:tasks.workunit.client.0.smithi006.stdout:0/435: fsync db/dc/d38/f66 0 2024-03-22T06:15:02.631 INFO:tasks.workunit.client.0.smithi006.stdout:0/436: chown db/d3d/d4e/c52 662173643 1 2024-03-22T06:15:02.634 INFO:tasks.workunit.client.0.smithi006.stdout:4/628: getdents d6/d1b/d20/d63/d93 0 2024-03-22T06:15:02.634 INFO:tasks.workunit.client.0.smithi006.stdout:7/616: read d5/d1e/d29/f54 [3177763,112010] 0 2024-03-22T06:15:02.644 INFO:tasks.workunit.client.0.smithi006.stdout:5/724: dread d0/db/d16/d1c/d34/fa6 [0,4194304] 0 2024-03-22T06:15:02.644 INFO:tasks.workunit.client.0.smithi006.stdout:5/725: truncate d0/db/d50/d58/dad/fcf 843869 0 2024-03-22T06:15:02.644 INFO:tasks.workunit.client.0.smithi006.stdout:2/621: mknod d13/ced 0 2024-03-22T06:15:02.646 INFO:tasks.workunit.client.0.smithi006.stdout:8/659: mknod d9/d14/d47/d94/cf4 0 2024-03-22T06:15:02.652 INFO:tasks.workunit.client.0.smithi006.stdout:2/622: symlink d13/d20/lee 0 2024-03-22T06:15:02.656 INFO:tasks.workunit.client.0.smithi006.stdout:6/557: rename d6/d7c/f8e to d6/d77/fb2 0 2024-03-22T06:15:02.657 INFO:tasks.workunit.client.0.smithi006.stdout:2/623: readlink d13/d34/d4d/d4e/d75/da5/l55 0 2024-03-22T06:15:02.660 INFO:tasks.workunit.client.0.smithi006.stdout:7/617: rename d5/d1e/d29/d59/l71 to d5/d1e/d63/d85/d80/lc5 0 2024-03-22T06:15:02.660 INFO:tasks.workunit.client.0.smithi006.stdout:7/618: dread - d5/d1e/d29/d1f/d2d/d55/fb8 zero size 2024-03-22T06:15:02.661 INFO:tasks.workunit.client.0.smithi006.stdout:6/558: dread d6/d7c/d92/faa [0,4194304] 0 2024-03-22T06:15:02.662 INFO:tasks.workunit.client.0.smithi006.stdout:2/624: mkdir d13/d20/d26/dca/def 0 2024-03-22T06:15:02.662 INFO:tasks.workunit.client.0.smithi006.stdout:2/625: write d13/d34/d4d/d4e/d75/fb7 [556655,7789] 0 2024-03-22T06:15:02.664 INFO:tasks.workunit.client.0.smithi006.stdout:3/568: dwrite d0/d7e/f93 [0,4194304] 0 2024-03-22T06:15:02.665 INFO:tasks.workunit.client.0.smithi006.stdout:7/619: truncate d5/d1e/d29/d1f/d2d/f37 449142 0 2024-03-22T06:15:02.669 INFO:tasks.workunit.client.0.smithi006.stdout:0/437: dwrite db/d87/f93 [0,4194304] 0 2024-03-22T06:15:02.672 INFO:tasks.workunit.client.0.smithi006.stdout:2/626: creat d13/ff0 x:0 0 0 2024-03-22T06:15:02.676 INFO:tasks.workunit.client.0.smithi006.stdout:3/569: symlink d0/d8/d1c/d3e/d5f/d90/lbf 0 2024-03-22T06:15:02.680 INFO:tasks.workunit.client.0.smithi006.stdout:4/629: dwrite f3 [4194304,4194304] 0 2024-03-22T06:15:02.680 INFO:tasks.workunit.client.0.smithi006.stdout:8/660: dwrite d9/db/d12/d1e/f30 [0,4194304] 0 2024-03-22T06:15:02.680 INFO:tasks.workunit.client.0.smithi006.stdout:8/661: readlink d9/db/dc/de/ld7 0 2024-03-22T06:15:02.682 INFO:tasks.workunit.client.0.smithi006.stdout:6/559: dread d6/f18 [0,4194304] 0 2024-03-22T06:15:02.682 INFO:tasks.workunit.client.0.smithi006.stdout:6/560: chown d6/d6f/d94 291 1 2024-03-22T06:15:02.683 INFO:tasks.workunit.client.0.smithi006.stdout:5/726: dwrite d0/db/d16/d70/fd1 [0,4194304] 0 2024-03-22T06:15:02.685 INFO:tasks.workunit.client.0.smithi006.stdout:2/627: symlink d13/d20/d26/dca/lf1 0 2024-03-22T06:15:02.688 INFO:tasks.workunit.client.0.smithi006.stdout:3/570: creat d0/d8/d1c/d89/fc0 x:0 0 0 2024-03-22T06:15:02.688 INFO:tasks.workunit.client.0.smithi006.stdout:3/571: write d0/f56 [4764929,66771] 0 2024-03-22T06:15:02.690 INFO:tasks.workunit.client.0.smithi006.stdout:4/630: symlink d6/d1b/d20/d6b/d60/ldc 0 2024-03-22T06:15:02.692 INFO:tasks.workunit.client.0.smithi006.stdout:0/438: dwrite db/d10/f2f [4194304,4194304] 0 2024-03-22T06:15:02.694 INFO:tasks.workunit.client.0.smithi006.stdout:2/628: dread d13/d34/d4d/f88 [0,4194304] 0 2024-03-22T06:15:02.694 INFO:tasks.workunit.client.0.smithi006.stdout:8/662: creat d9/db/d35/dd2/d76/db3/dc3/d89/db8/ff5 x:0 0 0 2024-03-22T06:15:02.708 INFO:tasks.workunit.client.0.smithi006.stdout:5/727: dwrite d0/db/d99/fc4 [0,4194304] 0 2024-03-22T06:15:02.710 INFO:tasks.workunit.client.0.smithi006.stdout:3/572: dwrite d0/d23/d4c/f7b [0,4194304] 0 2024-03-22T06:15:02.716 INFO:tasks.workunit.client.0.smithi006.stdout:6/561: rmdir d6/d1d/d32 39 2024-03-22T06:15:02.720 INFO:tasks.workunit.client.0.smithi006.stdout:6/562: dread d6/d24/f54 [0,4194304] 0 2024-03-22T06:15:02.720 INFO:tasks.workunit.client.0.smithi006.stdout:6/563: chown d6/d1d/d5c/d81/l68 1189841 1 2024-03-22T06:15:02.722 INFO:tasks.workunit.client.0.smithi006.stdout:4/631: symlink d6/d1b/d20/d63/d93/db8/dbc/ldd 0 2024-03-22T06:15:02.724 INFO:tasks.workunit.client.0.smithi006.stdout:2/629: dwrite d13/d20/da6/fc2 [0,4194304] 0 2024-03-22T06:15:02.724 INFO:tasks.workunit.client.0.smithi006.stdout:7/620: dwrite d5/d1e/d63/d85/fad [0,4194304] 0 2024-03-22T06:15:02.726 INFO:tasks.workunit.client.0.smithi006.stdout:0/439: unlink db/c72 0 2024-03-22T06:15:02.727 INFO:tasks.workunit.client.0.smithi006.stdout:8/663: truncate d9/db/d35/dd2/d4a/f5b 57114 0 2024-03-22T06:15:02.741 INFO:tasks.workunit.client.0.smithi006.stdout:5/728: creat d0/db/d76/fdd x:0 0 0 2024-03-22T06:15:02.747 INFO:tasks.workunit.client.0.smithi006.stdout:5/729: read d0/db/d50/d96/d61/f7d [367824,58999] 0 2024-03-22T06:15:02.748 INFO:tasks.workunit.client.0.smithi006.stdout:3/573: symlink d0/d8/d1c/d3e/d5f/d83/d8c/lc1 0 2024-03-22T06:15:02.749 INFO:tasks.workunit.client.0.smithi006.stdout:6/564: mknod d6/d1d/d5c/d89/cb3 0 2024-03-22T06:15:02.752 INFO:tasks.workunit.client.0.smithi006.stdout:4/632: rename d6/d1b/l5f to d6/lde 0 2024-03-22T06:15:02.752 INFO:tasks.workunit.client.0.smithi006.stdout:4/633: truncate d6/f7 358062 0 2024-03-22T06:15:02.754 INFO:tasks.workunit.client.0.smithi006.stdout:2/630: dwrite d13/d20/d26/f2f [4194304,4194304] 0 2024-03-22T06:15:02.755 INFO:tasks.workunit.client.0.smithi006.stdout:8/664: dwrite d9/db/d12/d66/f7f [0,4194304] 0 2024-03-22T06:15:02.756 INFO:tasks.workunit.client.0.smithi006.stdout:2/631: dread d13/d53/f5e [0,4194304] 0 2024-03-22T06:15:02.756 INFO:tasks.workunit.client.0.smithi006.stdout:2/632: creat d13/d34/ff2 x:0 0 0 2024-03-22T06:15:02.756 INFO:tasks.workunit.client.0.smithi006.stdout:2/633: read - d13/d34/ff2 zero size 2024-03-22T06:15:02.756 INFO:tasks.workunit.client.0.smithi006.stdout:2/634: fsync d13/d34/d4d/f5d 0 2024-03-22T06:15:02.756 INFO:tasks.workunit.client.0.smithi006.stdout:2/635: readlink d13/d34/l36 0 2024-03-22T06:15:02.757 INFO:tasks.workunit.client.0.smithi006.stdout:2/636: write d13/ff0 [393762,69898] 0 2024-03-22T06:15:02.757 INFO:tasks.workunit.client.0.smithi006.stdout:2/637: getdents d13/d34/d4d/d4e/d75/da5/d82/dda 0 2024-03-22T06:15:02.757 INFO:tasks.workunit.client.0.smithi006.stdout:7/621: dwrite d5/d1e/d29/d1f/d2d/d55/fb8 [0,4194304] 0 2024-03-22T06:15:02.757 INFO:tasks.workunit.client.0.smithi006.stdout:5/730: write d0/db/f60 [3292603,47785] 0 2024-03-22T06:15:02.758 INFO:tasks.workunit.client.0.smithi006.stdout:0/440: link db/d10/c84 db/dc/d37/c98 0 2024-03-22T06:15:02.763 INFO:tasks.workunit.client.0.smithi006.stdout:6/565: creat d6/d1b/dae/fb4 x:0 0 0 2024-03-22T06:15:02.763 INFO:tasks.workunit.client.0.smithi006.stdout:6/566: creat d6/d1b/d2c/fb5 x:0 0 0 2024-03-22T06:15:02.763 INFO:tasks.workunit.client.0.smithi006.stdout:4/634: dread d6/d13/d3b/f51 [0,4194304] 0 2024-03-22T06:15:02.763 INFO:tasks.workunit.client.0.smithi006.stdout:4/635: write d6/fbd [857646,91492] 0 2024-03-22T06:15:02.767 INFO:tasks.workunit.client.0.smithi006.stdout:3/574: dread d0/f56 [0,4194304] 0 2024-03-22T06:15:02.767 INFO:tasks.workunit.client.0.smithi006.stdout:3/575: write d0/d8/d1c/d3e/d5f/d90/da7/d97/fa5 [762786,3863] 0 2024-03-22T06:15:02.770 INFO:tasks.workunit.client.0.smithi006.stdout:8/665: mkdir d9/db/d35/da9/df6 0 2024-03-22T06:15:02.774 INFO:tasks.workunit.client.0.smithi006.stdout:2/638: symlink d13/d20/d35/d65/lf3 0 2024-03-22T06:15:02.784 INFO:tasks.workunit.client.0.smithi006.stdout:7/622: creat d5/d1e/d29/d1f/fc6 x:0 0 0 2024-03-22T06:15:02.784 INFO:tasks.workunit.client.0.smithi006.stdout:7/623: dread - d5/d1e/d63/d85/d80/fc4 zero size 2024-03-22T06:15:02.784 INFO:tasks.workunit.client.0.smithi006.stdout:7/624: creat d5/d1e/d29/d59/d5c/fc7 x:0 0 0 2024-03-22T06:15:02.784 INFO:tasks.workunit.client.0.smithi006.stdout:7/625: truncate d5/d1e/d29/d1f/f38 1434118 0 2024-03-22T06:15:02.784 INFO:tasks.workunit.client.0.smithi006.stdout:7/626: chown d5/d1e/d29/d1f/d2d/c43 5 1 2024-03-22T06:15:02.786 INFO:tasks.workunit.client.0.smithi006.stdout:0/441: symlink db/d10/d35/d7b/l99 0 2024-03-22T06:15:02.786 INFO:tasks.workunit.client.0.smithi006.stdout:0/442: chown db/d87/l6d 782513 1 2024-03-22T06:15:02.787 INFO:tasks.workunit.client.0.smithi006.stdout:0/443: write db/d10/f2f [8841558,56790] 0 2024-03-22T06:15:02.790 INFO:tasks.workunit.client.0.smithi006.stdout:6/567: link d6/d1b/d2c/f4c d6/d1b/dae/fb6 0 2024-03-22T06:15:02.791 INFO:tasks.workunit.client.0.smithi006.stdout:2/639: dwrite d13/d20/f52 [0,4194304] 0 2024-03-22T06:15:02.791 INFO:tasks.workunit.client.0.smithi006.stdout:2/640: dread - d13/d20/fcb zero size 2024-03-22T06:15:02.808 INFO:tasks.workunit.client.0.smithi006.stdout:3/576: symlink d0/d8/d1c/d3e/d5f/lc2 0 2024-03-22T06:15:02.808 INFO:tasks.workunit.client.0.smithi006.stdout:8/666: creat d9/db/d35/da9/df6/ff7 x:0 0 0 2024-03-22T06:15:02.811 INFO:tasks.workunit.client.0.smithi006.stdout:7/627: rmdir d5/d1e/d63 39 2024-03-22T06:15:02.811 INFO:tasks.workunit.client.0.smithi006.stdout:7/628: chown d5/d1e/d29/d1f/d2d/d55/d84/c9a 1 1 2024-03-22T06:15:02.811 INFO:tasks.workunit.client.0.smithi006.stdout:7/629: chown d5/d1e/d29/d1f/d2d/d41/c5e 144382 1 2024-03-22T06:15:02.811 INFO:tasks.workunit.client.0.smithi006.stdout:6/568: dwrite d6/d6f/f6e [0,4194304] 0 2024-03-22T06:15:02.812 INFO:tasks.workunit.client.0.smithi006.stdout:4/636: dwrite d6/d1b/d20/d21/d77/fa6 [0,4194304] 0 2024-03-22T06:15:02.813 INFO:tasks.workunit.client.0.smithi006.stdout:5/731: mknod d0/db/d16/d43/ddc/cde 0 2024-03-22T06:15:02.818 INFO:tasks.workunit.client.0.smithi006.stdout:2/641: creat d13/d34/d6e/d72/deb/ff4 x:0 0 0 2024-03-22T06:15:02.818 INFO:tasks.workunit.client.0.smithi006.stdout:2/642: write d13/d34/d4d/fbf [395061,116919] 0 2024-03-22T06:15:02.822 INFO:tasks.workunit.client.0.smithi006.stdout:3/577: mknod d0/d4/d31/cc3 0 2024-03-22T06:15:02.824 INFO:tasks.workunit.client.0.smithi006.stdout:5/732: dread d0/db/d16/d70/fd1 [0,4194304] 0 2024-03-22T06:15:02.826 INFO:tasks.workunit.client.0.smithi006.stdout:8/667: mkdir d9/d14/d47/d94/da3/df8 0 2024-03-22T06:15:02.826 INFO:tasks.workunit.client.0.smithi006.stdout:2/643: read d13/d20/d35/f45 [4167420,128317] 0 2024-03-22T06:15:02.827 INFO:tasks.workunit.client.0.smithi006.stdout:7/630: truncate d5/d1e/d3f/f4b 4840319 0 2024-03-22T06:15:02.827 INFO:tasks.workunit.client.0.smithi006.stdout:7/631: creat d5/d1e/d3f/fc8 x:0 0 0 2024-03-22T06:15:02.829 INFO:tasks.workunit.client.0.smithi006.stdout:6/569: truncate d6/d24/d44/f46 1120149 0 2024-03-22T06:15:02.831 INFO:tasks.workunit.client.0.smithi006.stdout:3/578: dread d0/d23/d33/d3d/f86 [0,4194304] 0 2024-03-22T06:15:02.834 INFO:tasks.workunit.client.0.smithi006.stdout:5/733: dread d0/db/d50/f80 [0,4194304] 0 2024-03-22T06:15:02.834 INFO:tasks.workunit.client.0.smithi006.stdout:5/734: chown d0/l7 5 1 2024-03-22T06:15:02.843 INFO:tasks.workunit.client.0.smithi006.stdout:4/637: creat d6/d1b/d20/d63/d93/fdf x:0 0 0 2024-03-22T06:15:02.843 INFO:tasks.workunit.client.0.smithi006.stdout:4/638: fsync d6/d1b/d20/d21/d2a/d5c/f66 0 2024-03-22T06:15:02.843 INFO:tasks.workunit.client.0.smithi006.stdout:4/639: read - d6/d1b/d20/d63/d93/d9a/fac zero size 2024-03-22T06:15:02.845 INFO:tasks.workunit.client.0.smithi006.stdout:0/444: symlink db/d10/l9a 0 2024-03-22T06:15:02.845 INFO:tasks.workunit.client.0.smithi006.stdout:0/445: write db/d3d/d4e/f7d [750572,2823] 0 2024-03-22T06:15:02.845 INFO:tasks.workunit.client.0.smithi006.stdout:0/446: write db/d87/f93 [4489726,126753] 0 2024-03-22T06:15:02.845 INFO:tasks.workunit.client.0.smithi006.stdout:0/447: creat db/d3d/d4e/f9b x:0 0 0 2024-03-22T06:15:02.851 INFO:tasks.workunit.client.0.smithi006.stdout:2/644: truncate d13/d20/d26/fe1 163314 0 2024-03-22T06:15:02.852 INFO:tasks.workunit.client.0.smithi006.stdout:2/645: chown d13/d34/ff2 70 1 2024-03-22T06:15:02.855 INFO:tasks.workunit.client.0.smithi006.stdout:7/632: mkdir d5/d1e/d29/db9/dc9 0 2024-03-22T06:15:02.855 INFO:tasks.workunit.client.0.smithi006.stdout:7/633: write d5/d1e/d29/d1f/fc6 [398026,71578] 0 2024-03-22T06:15:02.856 INFO:tasks.workunit.client.0.smithi006.stdout:6/570: stat d6/cf 0 2024-03-22T06:15:02.858 INFO:tasks.workunit.client.0.smithi006.stdout:3/579: mknod d0/d23/d4c/cc4 0 2024-03-22T06:15:02.860 INFO:tasks.workunit.client.0.smithi006.stdout:5/735: creat d0/db/d16/d43/ddc/fdf x:0 0 0 2024-03-22T06:15:02.866 INFO:tasks.workunit.client.0.smithi006.stdout:0/448: link db/dc/d38/f8e db/d10/d35/d6c/f9c 0 2024-03-22T06:15:02.866 INFO:tasks.workunit.client.0.smithi006.stdout:0/449: chown db/f6b 995731 1 2024-03-22T06:15:02.868 INFO:tasks.workunit.client.0.smithi006.stdout:8/668: mknod d9/db/d12/cf9 0 2024-03-22T06:15:02.871 INFO:tasks.workunit.client.0.smithi006.stdout:2/646: mknod d13/d53/d6b/cf5 0 2024-03-22T06:15:02.872 INFO:tasks.workunit.client.0.smithi006.stdout:2/647: write d13/d53/dc5/fd5 [1025072,46602] 0 2024-03-22T06:15:02.872 INFO:tasks.workunit.client.0.smithi006.stdout:2/648: fdatasync d13/d34/d4d/d4e/d75/d62/fb6 0 2024-03-22T06:15:02.872 INFO:tasks.workunit.client.0.smithi006.stdout:4/640: dwrite d6/d1b/d20/d6b/f8c [0,4194304] 0 2024-03-22T06:15:02.872 INFO:tasks.workunit.client.0.smithi006.stdout:4/641: stat d6/d1b/d20/d63/d93/fb0 0 2024-03-22T06:15:02.872 INFO:tasks.workunit.client.0.smithi006.stdout:4/642: dread - d6/d13/d3b/f7a zero size 2024-03-22T06:15:02.872 INFO:tasks.workunit.client.0.smithi006.stdout:7/634: dwrite d5/d1e/d3f/f7d [0,4194304] 0 2024-03-22T06:15:02.873 INFO:tasks.workunit.client.0.smithi006.stdout:3/580: mknod d0/d8/d9/cc5 0 2024-03-22T06:15:02.874 INFO:tasks.workunit.client.0.smithi006.stdout:3/581: truncate d0/d8/d1c/d3e/d5f/fa1 1044087 0 2024-03-22T06:15:02.874 INFO:tasks.workunit.client.0.smithi006.stdout:5/736: creat d0/d98/fe0 x:0 0 0 2024-03-22T06:15:02.876 INFO:tasks.workunit.client.0.smithi006.stdout:0/450: unlink db/dc/d37/f8a 0 2024-03-22T06:15:02.880 INFO:tasks.workunit.client.0.smithi006.stdout:8/669: rename d9/db/d35/dd2/d4a/c57 to d9/d5e/cfa 0 2024-03-22T06:15:02.880 INFO:tasks.workunit.client.0.smithi006.stdout:8/670: write d9/db/d35/dd2/d76/db3/dc3/d89/fb0 [4567360,109751] 0 2024-03-22T06:15:02.882 INFO:tasks.workunit.client.0.smithi006.stdout:2/649: creat d13/d34/d4d/d4e/d75/da5/d82/dc3/ff6 x:0 0 0 2024-03-22T06:15:02.882 INFO:tasks.workunit.client.0.smithi006.stdout:2/650: chown d13/d34/d4d/d4e/d75/da5/l9f 4456764 1 2024-03-22T06:15:02.882 INFO:tasks.workunit.client.0.smithi006.stdout:2/651: chown d13/d20/da6/fc2 814 1 2024-03-22T06:15:02.882 INFO:tasks.workunit.client.0.smithi006.stdout:2/652: fsync d13/d34/d4d/d4e/d75/da5/d82/fae 0 2024-03-22T06:15:02.882 INFO:tasks.workunit.client.0.smithi006.stdout:2/653: chown d13/d34/d4d/d4e/d75/da5/d6c/l98 173 1 2024-03-22T06:15:02.889 INFO:tasks.workunit.client.0.smithi006.stdout:6/571: mkdir d6/d77/db7 0 2024-03-22T06:15:02.889 INFO:tasks.workunit.client.0.smithi006.stdout:6/572: write d6/d1d/f20 [1015657,71935] 0 2024-03-22T06:15:02.889 INFO:tasks.workunit.client.0.smithi006.stdout:6/573: stat d6/d6f/la4 0 2024-03-22T06:15:02.892 INFO:tasks.workunit.client.0.smithi006.stdout:3/582: dwrite d0/d7e/f80 [0,4194304] 0 2024-03-22T06:15:02.895 INFO:tasks.workunit.client.0.smithi006.stdout:7/635: mkdir d5/d1e/d63/d85/dca 0 2024-03-22T06:15:02.895 INFO:tasks.workunit.client.0.smithi006.stdout:7/636: chown d5/d1e/d29/d1f/d2d/d61/f6d 37 1 2024-03-22T06:15:02.895 INFO:tasks.workunit.client.0.smithi006.stdout:1/471: sync 2024-03-22T06:15:02.895 INFO:tasks.workunit.client.0.smithi006.stdout:9/459: sync 2024-03-22T06:15:02.895 INFO:tasks.workunit.client.0.smithi006.stdout:9/460: fdatasync d8/d10/d28/f3d 0 2024-03-22T06:15:02.895 INFO:tasks.workunit.client.0.smithi006.stdout:9/461: fsync d8/fb 0 2024-03-22T06:15:02.897 INFO:tasks.workunit.client.0.smithi006.stdout:5/737: rmdir d0/db/d50/d96/d52/d69/d92/db7 39 2024-03-22T06:15:02.897 INFO:tasks.workunit.client.0.smithi006.stdout:5/738: chown d0/l6 2 1 2024-03-22T06:15:02.898 INFO:tasks.workunit.client.0.smithi006.stdout:0/451: mkdir db/d10/d33/d9d 0 2024-03-22T06:15:02.901 INFO:tasks.workunit.client.0.smithi006.stdout:4/643: rename d6/d1b/d20/f33 to d6/d1b/d20/d21/d2a/dc9/fe0 0 2024-03-22T06:15:02.905 INFO:tasks.workunit.client.0.smithi006.stdout:8/671: getdents d9/db/d12/d66/d6c/d71/d79 0 2024-03-22T06:15:02.906 INFO:tasks.workunit.client.0.smithi006.stdout:0/452: write db/d10/f23 [6532025,122144] 0 2024-03-22T06:15:02.907 INFO:tasks.workunit.client.0.smithi006.stdout:6/574: rmdir d6/d1d/d5c/d81/d8a 39 2024-03-22T06:15:02.913 INFO:tasks.workunit.client.0.smithi006.stdout:2/654: dwrite d13/d20/d35/fcf [0,4194304] 0 2024-03-22T06:15:02.913 INFO:tasks.workunit.client.0.smithi006.stdout:9/462: dread d8/d10/d20/d45/f9b [0,4194304] 0 2024-03-22T06:15:02.913 INFO:tasks.workunit.client.0.smithi006.stdout:9/463: write d8/d10/d28/d39/d75/d50/f8e [233999,93033] 0 2024-03-22T06:15:02.914 INFO:tasks.workunit.client.0.smithi006.stdout:9/464: chown d8/d10/l24 174820 1 2024-03-22T06:15:02.918 INFO:tasks.workunit.client.0.smithi006.stdout:9/465: read d8/fa [15459,114146] 0 2024-03-22T06:15:02.918 INFO:tasks.workunit.client.0.smithi006.stdout:9/466: chown d8/d10/d11/c36 934368759 1 2024-03-22T06:15:02.921 INFO:tasks.workunit.client.0.smithi006.stdout:4/644: unlink d6/d1b/d20/d63/d93/d9a/cdb 0 2024-03-22T06:15:02.921 INFO:tasks.workunit.client.0.smithi006.stdout:4/645: dread - d6/d1b/d20/d6b/d95/fbb zero size 2024-03-22T06:15:02.921 INFO:tasks.workunit.client.0.smithi006.stdout:4/646: truncate f3 8542835 0 2024-03-22T06:15:02.927 INFO:tasks.workunit.client.0.smithi006.stdout:7/637: getdents d5/d1e/d63/d85/da1 0 2024-03-22T06:15:02.927 INFO:tasks.workunit.client.0.smithi006.stdout:7/638: write d5/d1e/d3f/f90 [395621,86371] 0 2024-03-22T06:15:02.932 INFO:tasks.workunit.client.0.smithi006.stdout:4/647: getdents d6/d1b/d20/d21/d9f 0 2024-03-22T06:15:02.932 INFO:tasks.workunit.client.0.smithi006.stdout:4/648: chown d6/d1b/d20/d6b/d60/ld5 0 1 2024-03-22T06:15:02.933 INFO:tasks.workunit.client.0.smithi006.stdout:6/575: mknod d6/d77/cb8 0 2024-03-22T06:15:02.934 INFO:tasks.workunit.client.0.smithi006.stdout:7/639: mkdir d5/d1e/d29/d1f/d2d/d41/db7/dcb 0 2024-03-22T06:15:02.934 INFO:tasks.workunit.client.0.smithi006.stdout:7/640: truncate d5/d1e/d29/d1f/d2d/f37 2340886 0 2024-03-22T06:15:02.938 INFO:tasks.workunit.client.0.smithi006.stdout:5/739: rename d0/db/d50/d96/d61 to d0/db/d50/db9/de1 0 2024-03-22T06:15:02.940 INFO:tasks.workunit.client.0.smithi006.stdout:4/649: mknod d6/d1b/d20/d21/d2a/d43/d4b/ce1 0 2024-03-22T06:15:02.944 INFO:tasks.workunit.client.0.smithi006.stdout:6/576: unlink d6/d77/fb2 0 2024-03-22T06:15:02.944 INFO:tasks.workunit.client.0.smithi006.stdout:6/577: truncate d6/d6f/f6b 697108 0 2024-03-22T06:15:02.944 INFO:tasks.workunit.client.0.smithi006.stdout:6/578: creat d6/d7c/d92/fb9 x:0 0 0 2024-03-22T06:15:02.945 INFO:tasks.workunit.client.0.smithi006.stdout:5/740: creat d0/db/d50/d96/d52/d69/fe2 x:0 0 0 2024-03-22T06:15:02.947 INFO:tasks.workunit.client.0.smithi006.stdout:4/650: getdents d6/d1b/d20/d63/d93/db8/dbc 0 2024-03-22T06:15:02.947 INFO:tasks.workunit.client.0.smithi006.stdout:4/651: dread - d6/d1b/d20/d63/d93/fb0 zero size 2024-03-22T06:15:02.950 INFO:tasks.workunit.client.0.smithi006.stdout:1/472: dwrite d8/d9/d3a/d40/d44/f76 [0,4194304] 0 2024-03-22T06:15:02.950 INFO:tasks.workunit.client.0.smithi006.stdout:1/473: fsync d8/d9/f8d 0 2024-03-22T06:15:02.950 INFO:tasks.workunit.client.0.smithi006.stdout:0/453: dwrite db/dc/d37/f77 [0,4194304] 0 2024-03-22T06:15:02.950 INFO:tasks.workunit.client.0.smithi006.stdout:0/454: chown db/dc/d38/d50/l89 0 1 2024-03-22T06:15:02.950 INFO:tasks.workunit.client.0.smithi006.stdout:8/672: dwrite d9/d14/f34 [0,4194304] 0 2024-03-22T06:15:02.951 INFO:tasks.workunit.client.0.smithi006.stdout:8/673: truncate d9/d14/d47/d94/da3/fda 397317 0 2024-03-22T06:15:02.951 INFO:tasks.workunit.client.0.smithi006.stdout:9/467: dwrite d8/fd [4194304,4194304] 0 2024-03-22T06:15:02.951 INFO:tasks.workunit.client.0.smithi006.stdout:9/468: chown d8/d10/d11/f42 1427 1 2024-03-22T06:15:02.952 INFO:tasks.workunit.client.0.smithi006.stdout:9/469: write d8/d10/d28/d39/d75/d50/f5e [751650,75973] 0 2024-03-22T06:15:02.952 INFO:tasks.workunit.client.0.smithi006.stdout:2/655: dwrite d13/d34/d4d/f5d [0,4194304] 0 2024-03-22T06:15:02.953 INFO:tasks.workunit.client.0.smithi006.stdout:2/656: dread d13/d20/f4b [4194304,4194304] 0 2024-03-22T06:15:02.953 INFO:tasks.workunit.client.0.smithi006.stdout:3/583: dwrite d0/d4/fa9 [0,4194304] 0 2024-03-22T06:15:02.953 INFO:tasks.workunit.client.0.smithi006.stdout:3/584: chown d0/d8/d1c/d3e/l8b 682 1 2024-03-22T06:15:02.953 INFO:tasks.workunit.client.0.smithi006.stdout:6/579: unlink d6/d1b/d2c/f4c 0 2024-03-22T06:15:02.956 INFO:tasks.workunit.client.0.smithi006.stdout:5/741: read d0/db/d50/d96/d52/d69/fa3 [2532666,128458] 0 2024-03-22T06:15:02.960 INFO:tasks.workunit.client.0.smithi006.stdout:0/455: mknod db/d3d/d4e/c9e 0 2024-03-22T06:15:02.963 INFO:tasks.workunit.client.0.smithi006.stdout:5/742: dread d0/db/d16/d1c/d34/d35/d85/dda/d65/d77/f86 [0,4194304] 0 2024-03-22T06:15:02.966 INFO:tasks.workunit.client.0.smithi006.stdout:8/674: creat d9/db/d12/d66/d6c/d80/df0/ffb x:0 0 0 2024-03-22T06:15:02.966 INFO:tasks.workunit.client.0.smithi006.stdout:8/675: fsync d9/db/dc/de/d3e/f56 0 2024-03-22T06:15:02.966 INFO:tasks.workunit.client.0.smithi006.stdout:8/676: readlink d9/d14/d47/d94/da3/lcb 0 2024-03-22T06:15:02.966 INFO:tasks.workunit.client.0.smithi006.stdout:8/677: chown d9/db/d12/d66/d6c/f63 3602 1 2024-03-22T06:15:02.966 INFO:tasks.workunit.client.0.smithi006.stdout:8/678: fsync d9/d14/d44/f8e 0 2024-03-22T06:15:02.972 INFO:tasks.workunit.client.0.smithi006.stdout:1/474: dread d8/d9/d3a/d40/d44/f66 [4194304,4194304] 0 2024-03-22T06:15:02.972 INFO:tasks.workunit.client.0.smithi006.stdout:1/475: readlink d8/d9/l2e 0 2024-03-22T06:15:02.980 INFO:tasks.workunit.client.0.smithi006.stdout:9/470: mknod d8/d10/d28/d39/c9f 0 2024-03-22T06:15:02.987 INFO:tasks.workunit.client.0.smithi006.stdout:4/652: dwrite d6/d1b/d20/d6b/f8c [0,4194304] 0 2024-03-22T06:15:02.987 INFO:tasks.workunit.client.0.smithi006.stdout:2/657: dwrite d13/d34/ff2 [0,4194304] 0 2024-03-22T06:15:02.987 INFO:tasks.workunit.client.0.smithi006.stdout:0/456: dwrite db/dc/d37/f77 [0,4194304] 0 2024-03-22T06:15:02.987 INFO:tasks.workunit.client.0.smithi006.stdout:0/457: write db/d10/d33/f42 [2048342,66542] 0 2024-03-22T06:15:02.987 INFO:tasks.workunit.client.0.smithi006.stdout:0/458: write db/dc/d37/f77 [4860048,57752] 0 2024-03-22T06:15:02.988 INFO:tasks.workunit.client.0.smithi006.stdout:3/585: rename d0/c91 to d0/d23/d33/d3d/dbd/cc6 0 2024-03-22T06:15:02.990 INFO:tasks.workunit.client.0.smithi006.stdout:6/580: mknod d6/d1d/d5c/d81/d8a/dac/cba 0 2024-03-22T06:15:02.995 INFO:tasks.workunit.client.0.smithi006.stdout:8/679: link d9/db/d35/dd2/d4a/d4d/cd3 d9/db/d35/dd2/cfc 0 2024-03-22T06:15:02.995 INFO:tasks.workunit.client.0.smithi006.stdout:8/680: fsync d9/db/f4f 0 2024-03-22T06:15:02.996 INFO:tasks.workunit.client.0.smithi006.stdout:5/743: dread d0/db/dc/f8f [0,4194304] 0 2024-03-22T06:15:02.999 INFO:tasks.workunit.client.0.smithi006.stdout:1/476: symlink d8/d9/d10/d7c/d96/l9d 0 2024-03-22T06:15:03.000 INFO:tasks.workunit.client.0.smithi006.stdout:2/658: dwrite d13/d34/d6e/f9b [0,4194304] 0 2024-03-22T06:15:03.002 INFO:tasks.workunit.client.0.smithi006.stdout:9/471: link f7 d8/d10/d20/fa0 0 2024-03-22T06:15:03.007 INFO:tasks.workunit.client.0.smithi006.stdout:5/744: write d0/db/dc/f8f [1016946,77433] 0 2024-03-22T06:15:03.008 INFO:tasks.workunit.client.0.smithi006.stdout:4/653: creat d6/d13/d3b/fe2 x:0 0 0 2024-03-22T06:15:03.016 INFO:tasks.workunit.client.0.smithi006.stdout:3/586: link d0/d8/d1c/d3e/d5f/d90/lbf d0/d8/d9/db1/lc7 0 2024-03-22T06:15:03.020 INFO:tasks.workunit.client.0.smithi006.stdout:1/477: rename d8/d9/d13/d24/l70 to d8/d9/d3a/d40/d44/d80/l9e 0 2024-03-22T06:15:03.022 INFO:tasks.workunit.client.0.smithi006.stdout:4/654: write d6/d1b/d20/d6b/d60/f67 [8231273,20145] 0 2024-03-22T06:15:03.028 INFO:tasks.workunit.client.0.smithi006.stdout:8/681: dwrite d9/db/f2e [0,4194304] 0 2024-03-22T06:15:03.028 INFO:tasks.workunit.client.0.smithi006.stdout:8/682: truncate d9/db/d12/d1e/f51 232861 0 2024-03-22T06:15:03.030 INFO:tasks.workunit.client.0.smithi006.stdout:9/472: dwrite d8/d10/d20/d45/d49/f77 [0,4194304] 0 2024-03-22T06:15:03.034 INFO:tasks.workunit.client.0.smithi006.stdout:1/478: dread d8/d9/d3a/d40/d44/f53 [0,4194304] 0 2024-03-22T06:15:03.034 INFO:tasks.workunit.client.0.smithi006.stdout:1/479: creat d8/d33/f9f x:0 0 0 2024-03-22T06:15:03.036 INFO:tasks.workunit.client.0.smithi006.stdout:2/659: mknod d13/d34/d4d/cf7 0 2024-03-22T06:15:03.039 INFO:tasks.workunit.client.0.smithi006.stdout:5/745: creat d0/db/d16/d1c/dbc/dd6/fe3 x:0 0 0 2024-03-22T06:15:03.039 INFO:tasks.workunit.client.0.smithi006.stdout:0/459: rmdir db/d3d/d4e/d76/d81 39 2024-03-22T06:15:03.041 INFO:tasks.workunit.client.0.smithi006.stdout:4/655: dwrite d6/d1b/d20/d21/d2a/d43/f7d [4194304,4194304] 0 2024-03-22T06:15:03.045 INFO:tasks.workunit.client.0.smithi006.stdout:3/587: rmdir d0/d4/d31/d98/dab/db6 0 2024-03-22T06:15:03.045 INFO:tasks.workunit.client.0.smithi006.stdout:3/588: fdatasync d0/d8/d9/db1/fbc 0 2024-03-22T06:15:03.045 INFO:tasks.workunit.client.0.smithi006.stdout:3/589: chown d0/d8/d1c/d3e/d5f/lc2 89781 1 2024-03-22T06:15:03.045 INFO:tasks.workunit.client.0.smithi006.stdout:3/590: dread - d0/d8/d1c/f76 zero size 2024-03-22T06:15:03.047 INFO:tasks.workunit.client.0.smithi006.stdout:6/581: rmdir d6/d77 39 2024-03-22T06:15:03.047 INFO:tasks.workunit.client.0.smithi006.stdout:6/582: dread - d6/d1d/d32/f67 zero size 2024-03-22T06:15:03.048 INFO:tasks.workunit.client.0.smithi006.stdout:4/656: write d6/d1b/d46/f5a [1590935,59404] 0 2024-03-22T06:15:03.049 INFO:tasks.workunit.client.0.smithi006.stdout:4/657: fsync d6/d1b/d20/d6b/d60/f68 0 2024-03-22T06:15:03.049 INFO:tasks.workunit.client.0.smithi006.stdout:4/658: chown d6/d1b/d20/d63/d93/lc5 27 1 2024-03-22T06:15:03.054 INFO:tasks.workunit.client.0.smithi006.stdout:9/473: dwrite d8/f21 [0,4194304] 0 2024-03-22T06:15:03.055 INFO:tasks.workunit.client.0.smithi006.stdout:8/683: dwrite d9/db/d12/d66/d6c/d80/fdf [0,4194304] 0 2024-03-22T06:15:03.070 INFO:tasks.workunit.client.0.smithi006.stdout:8/684: dwrite d9/db/d12/d66/f7f [0,4194304] 0 2024-03-22T06:15:03.079 INFO:tasks.workunit.client.0.smithi006.stdout:8/685: write f1 [167920,124694] 0 2024-03-22T06:15:03.079 INFO:tasks.workunit.client.0.smithi006.stdout:1/480: rmdir d8/d9/d13 39 2024-03-22T06:15:03.084 INFO:tasks.workunit.client.0.smithi006.stdout:2/660: mkdir d13/d34/d4d/d4e/d75/da5/d82/dc3/df8 0 2024-03-22T06:15:03.084 INFO:tasks.workunit.client.0.smithi006.stdout:1/481: dread d8/d9/d3a/d40/d44/f66 [4194304,4194304] 0 2024-03-22T06:15:03.087 INFO:tasks.workunit.client.0.smithi006.stdout:8/686: write f7 [2001838,93859] 0 2024-03-22T06:15:03.088 INFO:tasks.workunit.client.0.smithi006.stdout:8/687: fdatasync d9/db/d35/dd2/d4a/da7/ff1 0 2024-03-22T06:15:03.091 INFO:tasks.workunit.client.0.smithi006.stdout:8/688: write d9/db/dc/f3a [551648,62659] 0 2024-03-22T06:15:03.097 INFO:tasks.workunit.client.0.smithi006.stdout:6/583: link d6/d24/f31 d6/d1d/d5c/d81/d8a/dac/fbb 0 2024-03-22T06:15:03.099 INFO:tasks.workunit.client.0.smithi006.stdout:5/746: dwrite d0/db/d16/fca [0,4194304] 0 2024-03-22T06:15:03.100 INFO:tasks.workunit.client.0.smithi006.stdout:4/659: mknod d6/d1b/d20/d63/ce3 0 2024-03-22T06:15:03.104 INFO:tasks.workunit.client.0.smithi006.stdout:9/474: symlink d8/d10/d28/d39/d75/la1 0 2024-03-22T06:15:03.110 INFO:tasks.workunit.client.0.smithi006.stdout:4/660: dread d6/d1b/d20/d6b/d60/db1/f82 [0,4194304] 0 2024-03-22T06:15:03.111 INFO:tasks.workunit.client.0.smithi006.stdout:3/591: dwrite d0/d8/f17 [0,4194304] 0 2024-03-22T06:15:03.117 INFO:tasks.workunit.client.0.smithi006.stdout:2/661: mknod d13/d34/d6e/d72/cf9 0 2024-03-22T06:15:03.118 INFO:tasks.workunit.client.0.smithi006.stdout:1/482: symlink d8/d9/d10/d7c/d96/la0 0 2024-03-22T06:15:03.122 INFO:tasks.workunit.client.0.smithi006.stdout:8/689: mknod d9/db/d12/d7e/cfd 0 2024-03-22T06:15:03.126 INFO:tasks.workunit.client.0.smithi006.stdout:2/662: read d13/d53/dc5/fd7 [2718128,39066] 0 2024-03-22T06:15:03.126 INFO:tasks.workunit.client.0.smithi006.stdout:2/663: chown d13/d20/d26/f15 42035 1 2024-03-22T06:15:03.128 INFO:tasks.workunit.client.0.smithi006.stdout:5/747: rmdir d0/db/d16/d1c/d34 39 2024-03-22T06:15:03.128 INFO:tasks.workunit.client.0.smithi006.stdout:5/748: readlink d0/db/d50/d96/d52/lc8 0 2024-03-22T06:15:03.129 INFO:tasks.workunit.client.0.smithi006.stdout:4/661: rmdir d6/d1b/d20/d63/d93/d9a/d8f 39 2024-03-22T06:15:03.130 INFO:tasks.workunit.client.0.smithi006.stdout:4/662: write f2 [1103460,29684] 0 2024-03-22T06:15:03.132 INFO:tasks.workunit.client.0.smithi006.stdout:3/592: creat d0/d8/d1c/d3e/d5f/da3/fc8 x:0 0 0 2024-03-22T06:15:03.141 INFO:tasks.workunit.client.0.smithi006.stdout:2/664: dwrite d13/d20/d26/d19/f59 [0,4194304] 0 2024-03-22T06:15:03.141 INFO:tasks.workunit.client.0.smithi006.stdout:2/665: creat d13/d53/dc5/ffa x:0 0 0 2024-03-22T06:15:03.141 INFO:tasks.workunit.client.0.smithi006.stdout:1/483: unlink d8/d9/f4b 0 2024-03-22T06:15:03.144 INFO:tasks.workunit.client.0.smithi006.stdout:8/690: link d9/d5e/cdb d9/db/dc/de/d3e/cfe 0 2024-03-22T06:15:03.144 INFO:tasks.workunit.client.0.smithi006.stdout:8/691: write d9/db/d12/d66/d6c/d71/fa2 [25320,21381] 0 2024-03-22T06:15:03.147 INFO:tasks.workunit.client.0.smithi006.stdout:5/749: creat d0/db/d16/d1c/d34/d35/d85/dda/d65/dd4/fe4 x:0 0 0 2024-03-22T06:15:03.149 INFO:tasks.workunit.client.0.smithi006.stdout:4/663: creat d6/d1b/d20/d21/d2a/d43/fe4 x:0 0 0 2024-03-22T06:15:03.149 INFO:tasks.workunit.client.0.smithi006.stdout:4/664: creat d6/d1b/d20/d6b/d88/fe5 x:0 0 0 2024-03-22T06:15:03.150 INFO:tasks.workunit.client.0.smithi006.stdout:3/593: dwrite d0/d23/d4c/f35 [4194304,4194304] 0 2024-03-22T06:15:03.150 INFO:tasks.workunit.client.0.smithi006.stdout:3/594: fsync d0/d8/d1c/d72/f94 0 2024-03-22T06:15:03.151 INFO:tasks.workunit.client.0.smithi006.stdout:3/595: creat d0/d4/fc9 x:0 0 0 2024-03-22T06:15:03.151 INFO:tasks.workunit.client.0.smithi006.stdout:6/584: mkdir d6/d1b/dae/dbc 0 2024-03-22T06:15:03.167 INFO:tasks.workunit.client.0.smithi006.stdout:1/484: mkdir d8/d9/d3a/d40/da1 0 2024-03-22T06:15:03.169 INFO:tasks.workunit.client.0.smithi006.stdout:8/692: creat d9/db/d12/d66/fff x:0 0 0 2024-03-22T06:15:03.169 INFO:tasks.workunit.client.0.smithi006.stdout:8/693: truncate d9/db/d35/dd2/d76/db3/dc3/d89/fd4 946905 0 2024-03-22T06:15:03.178 INFO:tasks.workunit.client.0.smithi006.stdout:4/665: dwrite d6/fa [0,4194304] 0 2024-03-22T06:15:03.179 INFO:tasks.workunit.client.0.smithi006.stdout:1/485: creat d8/fa2 x:0 0 0 2024-03-22T06:15:03.179 INFO:tasks.workunit.client.0.smithi006.stdout:1/486: creat d8/d9/d13/d24/fa3 x:0 0 0 2024-03-22T06:15:03.181 INFO:tasks.workunit.client.0.smithi006.stdout:8/694: mknod d9/db/d12/d66/d6c/d92/c100 0 2024-03-22T06:15:03.181 INFO:tasks.workunit.client.0.smithi006.stdout:8/695: creat d9/db/d12/d66/d6c/d71/f101 x:0 0 0 2024-03-22T06:15:03.181 INFO:tasks.workunit.client.0.smithi006.stdout:8/696: write d9/db/d35/dd2/d4a/f96 [3168788,99182] 0 2024-03-22T06:15:03.182 INFO:tasks.workunit.client.0.smithi006.stdout:6/585: dwrite d6/d1d/d5c/d81/d8a/dac/fbb [0,4194304] 0 2024-03-22T06:15:03.182 INFO:tasks.workunit.client.0.smithi006.stdout:5/750: dwrite d0/db/d50/d96/d52/d69/fe2 [0,4194304] 0 2024-03-22T06:15:03.182 INFO:tasks.workunit.client.0.smithi006.stdout:5/751: chown d0/db/d50 5 1 2024-03-22T06:15:03.182 INFO:tasks.workunit.client.0.smithi006.stdout:5/752: write d0/db/d76/f9a [980610,66692] 0 2024-03-22T06:15:03.182 INFO:tasks.workunit.client.0.smithi006.stdout:2/666: dwrite f7 [0,4194304] 0 2024-03-22T06:15:03.182 INFO:tasks.workunit.client.0.smithi006.stdout:2/667: creat d13/d20/d26/d19/ffb x:0 0 0 2024-03-22T06:15:03.184 INFO:tasks.workunit.client.0.smithi006.stdout:3/596: link d0/d8/d1c/d3e/d5f/lc2 d0/d8/d88/lca 0 2024-03-22T06:15:03.189 INFO:tasks.workunit.client.0.smithi006.stdout:4/666: creat d6/d1b/d20/fe6 x:0 0 0 2024-03-22T06:15:03.192 INFO:tasks.workunit.client.0.smithi006.stdout:1/487: mknod d8/ca4 0 2024-03-22T06:15:03.196 INFO:tasks.workunit.client.0.smithi006.stdout:8/697: mknod d9/db/d35/dd2/d76/c102 0 2024-03-22T06:15:03.198 INFO:tasks.workunit.client.0.smithi006.stdout:6/586: symlink d6/d1d/d5c/d89/lbd 0 2024-03-22T06:15:03.210 INFO:tasks.workunit.client.0.smithi006.stdout:2/668: creat d13/d69/ffc x:0 0 0 2024-03-22T06:15:03.210 INFO:tasks.workunit.client.0.smithi006.stdout:3/597: mkdir d0/d23/d33/d3d/dcb 0 2024-03-22T06:15:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: Adjusting osd_memory_target on smithi006 to 1559M 2024-03-22T06:15:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:15:03.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:15:03.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-22T06:15:03.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:02 smithi117 ceph-mon[36514]: Updating smithi117:/etc/ceph/ceph.conf 2024-03-22T06:15:03.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:03.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: Adjusting osd_memory_target on smithi006 to 1559M 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-22T06:15:03.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:02 smithi006 ceph-mon[29301]: Updating smithi117:/etc/ceph/ceph.conf 2024-03-22T06:15:03.289 INFO:tasks.workunit.client.0.smithi006.stdout:4/667: creat d6/d13/fe7 x:0 0 0 2024-03-22T06:15:03.291 INFO:tasks.workunit.client.0.smithi006.stdout:1/488: symlink d8/d9/d10/d7c/d96/la5 0 2024-03-22T06:15:03.292 INFO:tasks.workunit.client.0.smithi006.stdout:8/698: truncate d9/db/d35/f3c 2758732 0 2024-03-22T06:15:03.292 INFO:tasks.workunit.client.0.smithi006.stdout:8/699: truncate d9/db/d12/d66/d6c/d80/df0/ffb 876086 0 2024-03-22T06:15:03.293 INFO:tasks.workunit.client.0.smithi006.stdout:6/587: chown d6/c8 0 1 2024-03-22T06:15:03.295 INFO:tasks.workunit.client.0.smithi006.stdout:5/753: mknod d0/db/ce5 0 2024-03-22T06:15:03.297 INFO:tasks.workunit.client.0.smithi006.stdout:2/669: unlink d13/d20/d26/db9/cc6 0 2024-03-22T06:15:03.297 INFO:tasks.workunit.client.0.smithi006.stdout:2/670: creat d13/d34/d4d/d4e/d75/da5/d82/dc3/ffd x:0 0 0 2024-03-22T06:15:03.297 INFO:tasks.workunit.client.0.smithi006.stdout:2/671: fdatasync d13/d34/f60 0 2024-03-22T06:15:03.299 INFO:tasks.workunit.client.0.smithi006.stdout:2/672: dread d13/d34/d4d/d4e/d75/da5/f94 [0,4194304] 0 2024-03-22T06:15:03.308 INFO:tasks.workunit.client.0.smithi006.stdout:3/598: write d0/d8/f1a [2339917,96676] 0 2024-03-22T06:15:03.315 INFO:tasks.workunit.client.0.smithi006.stdout:4/668: mknod d6/d1b/d20/d6b/d60/ce8 0 2024-03-22T06:15:03.316 INFO:tasks.workunit.client.0.smithi006.stdout:3/599: dread d0/d7e/f93 [0,4194304] 0 2024-03-22T06:15:03.321 INFO:tasks.workunit.client.0.smithi006.stdout:8/700: rename d9/db/d35/dd2/dc4/ddd/lde to d9/db/dc/de3/l103 0 2024-03-22T06:15:03.322 INFO:tasks.workunit.client.0.smithi006.stdout:6/588: creat d6/d1d/d99/fbe x:0 0 0 2024-03-22T06:15:03.329 INFO:tasks.workunit.client.0.smithi006.stdout:8/701: dread d9/db/dc/de/f17 [0,4194304] 0 2024-03-22T06:15:03.329 INFO:tasks.workunit.client.0.smithi006.stdout:2/673: creat d13/d20/d26/dca/dea/ffe x:0 0 0 2024-03-22T06:15:03.332 INFO:tasks.workunit.client.0.smithi006.stdout:4/669: unlink d6/d1b/d20/d63/d93/faa 0 2024-03-22T06:15:03.334 INFO:tasks.workunit.client.0.smithi006.stdout:3/600: creat d0/d4/d31/fcc x:0 0 0 2024-03-22T06:15:03.334 INFO:tasks.workunit.client.0.smithi006.stdout:3/601: write d0/d7e/fa0 [158172,8740] 0 2024-03-22T06:15:03.335 INFO:tasks.workunit.client.0.smithi006.stdout:3/602: chown d0/d8/d9 0 1 2024-03-22T06:15:03.335 INFO:tasks.workunit.client.0.smithi006.stdout:3/603: readlink d0/d23/d33/l50 0 2024-03-22T06:15:03.335 INFO:tasks.workunit.client.0.smithi006.stdout:3/604: chown d0/d23/d33/d3d/d58/ca8 317 1 2024-03-22T06:15:03.335 INFO:tasks.workunit.client.0.smithi006.stdout:3/605: stat d0/d8/d1c/d3e/f96 0 2024-03-22T06:15:03.335 INFO:tasks.workunit.client.0.smithi006.stdout:8/702: dread d9/db/d12/d66/d6c/d80/fa4 [0,4194304] 0 2024-03-22T06:15:03.336 INFO:tasks.workunit.client.0.smithi006.stdout:1/489: rmdir d8/d9/d13/d39/d60 39 2024-03-22T06:15:03.337 INFO:tasks.workunit.client.0.smithi006.stdout:1/490: chown d8/d9/c7e 221536086 1 2024-03-22T06:15:03.341 INFO:tasks.workunit.client.0.smithi006.stdout:6/589: mknod d6/d1d/d5c/d89/cbf 0 2024-03-22T06:15:03.343 INFO:tasks.workunit.client.0.smithi006.stdout:5/754: rmdir d0/db/d16/d1c/dbc/dd6 39 2024-03-22T06:15:03.346 INFO:tasks.workunit.client.0.smithi006.stdout:2/674: rename d13/d20/d26/l5b to d13/d34/d6e/lff 0 2024-03-22T06:15:03.350 INFO:tasks.workunit.client.0.smithi006.stdout:6/590: dread d6/d24/f31 [0,4194304] 0 2024-03-22T06:15:03.350 INFO:tasks.workunit.client.0.smithi006.stdout:4/670: creat d6/d1b/fe9 x:0 0 0 2024-03-22T06:15:03.351 INFO:tasks.workunit.client.0.smithi006.stdout:3/606: symlink d0/d8/d1c/d95/lcd 0 2024-03-22T06:15:03.354 INFO:tasks.workunit.client.0.smithi006.stdout:1/491: creat d8/d9/d10/d7c/fa6 x:0 0 0 2024-03-22T06:15:03.354 INFO:tasks.workunit.client.0.smithi006.stdout:1/492: write d8/d33/f9f [929936,110392] 0 2024-03-22T06:15:03.359 INFO:tasks.workunit.client.0.smithi006.stdout:6/591: dread d6/d1b/d2c/f7d [0,4194304] 0 2024-03-22T06:15:03.363 INFO:tasks.workunit.client.0.smithi006.stdout:5/755: symlink d0/db/d16/d1c/le6 0 2024-03-22T06:15:03.364 INFO:tasks.workunit.client.0.smithi006.stdout:5/756: write d0/db/d16/d1c/d34/f4b [3566971,3685] 0 2024-03-22T06:15:03.364 INFO:tasks.workunit.client.0.smithi006.stdout:5/757: stat d0/db/d16/d1c/d34/d35/d85/d5e 0 2024-03-22T06:15:03.367 INFO:tasks.workunit.client.0.smithi006.stdout:2/675: link d13/d20/d35/d65/fbb d13/d34/d4d/d4e/d75/d62/f100 0 2024-03-22T06:15:03.368 INFO:tasks.workunit.client.0.smithi006.stdout:7/641: sync 2024-03-22T06:15:03.368 INFO:tasks.workunit.client.0.smithi006.stdout:7/642: creat d5/d1e/fcc x:0 0 0 2024-03-22T06:15:03.368 INFO:tasks.workunit.client.0.smithi006.stdout:7/643: write d5/d1e/d29/d1f/fc6 [1313957,47662] 0 2024-03-22T06:15:03.370 INFO:tasks.workunit.client.0.smithi006.stdout:4/671: readlink d6/d1b/d20/d21/d77/lbe 0 2024-03-22T06:15:03.372 INFO:tasks.workunit.client.0.smithi006.stdout:3/607: creat d0/d8/d1c/d3e/d68/fce x:0 0 0 2024-03-22T06:15:03.372 INFO:tasks.workunit.client.0.smithi006.stdout:3/608: stat d0/d8/d1c/d3e/d5f/f92 0 2024-03-22T06:15:03.372 INFO:tasks.workunit.client.0.smithi006.stdout:3/609: readlink d0/d8/d88/l9c 0 2024-03-22T06:15:03.373 INFO:tasks.workunit.client.0.smithi006.stdout:6/592: dwrite d6/d7c/d92/fad [0,4194304] 0 2024-03-22T06:15:03.375 INFO:tasks.workunit.client.0.smithi006.stdout:8/703: getdents d9/db/d12/d7e 0 2024-03-22T06:15:03.376 INFO:tasks.workunit.client.0.smithi006.stdout:1/493: mknod d8/d9/d3a/d18/ca7 0 2024-03-22T06:15:03.377 INFO:tasks.workunit.client.0.smithi006.stdout:1/494: write d8/d9/d13/d24/d32/f6f [257746,35241] 0 2024-03-22T06:15:03.379 INFO:tasks.workunit.client.0.smithi006.stdout:5/758: getdents d0/db/d16/d1c/d34/d35/d85/dda/d65 0 2024-03-22T06:15:03.384 INFO:tasks.workunit.client.0.smithi006.stdout:7/644: mkdir d5/d1e/d29/d59/dcd 0 2024-03-22T06:15:03.385 INFO:tasks.workunit.client.0.smithi006.stdout:7/645: chown d5/d1e/d29/d59/l78 323 1 2024-03-22T06:15:03.385 INFO:tasks.workunit.client.0.smithi006.stdout:4/672: write d6/d1b/d20/d21/d2a/d43/f7d [6476088,36571] 0 2024-03-22T06:15:03.385 INFO:tasks.workunit.client.0.smithi006.stdout:2/676: dwrite d13/ff0 [0,4194304] 0 2024-03-22T06:15:03.386 INFO:tasks.workunit.client.0.smithi006.stdout:3/610: creat d0/fcf x:0 0 0 2024-03-22T06:15:03.390 INFO:tasks.workunit.client.0.smithi006.stdout:7/646: write d5/d1e/d3f/f81 [3078844,88252] 0 2024-03-22T06:15:03.398 INFO:tasks.workunit.client.0.smithi006.stdout:7/647: dread d5/d1e/d29/d10/f25 [0,4194304] 0 2024-03-22T06:15:03.400 INFO:tasks.workunit.client.0.smithi006.stdout:8/704: mkdir d9/db/d35/dd2/dc4/d104 0 2024-03-22T06:15:03.400 INFO:tasks.workunit.client.0.smithi006.stdout:8/705: stat d9/db/d35/dd2/d76/db3 0 2024-03-22T06:15:03.403 INFO:tasks.workunit.client.0.smithi006.stdout:1/495: truncate d8/d9/d3a/d18/f56 4017260 0 2024-03-22T06:15:03.403 INFO:tasks.workunit.client.0.smithi006.stdout:1/496: truncate d8/d9/d10/f4e 739845 0 2024-03-22T06:15:03.403 INFO:tasks.workunit.client.0.smithi006.stdout:1/497: write d8/d9/d3a/f6b [161870,70030] 0 2024-03-22T06:15:03.403 INFO:tasks.workunit.client.0.smithi006.stdout:1/498: chown d8/d9/f1e 449540323 1 2024-03-22T06:15:03.404 INFO:tasks.workunit.client.0.smithi006.stdout:1/499: getdents d8/d9/d13/d39/d94 0 2024-03-22T06:15:03.404 INFO:tasks.workunit.client.0.smithi006.stdout:1/500: write d8/d9/d13/d24/fa3 [1000934,3169] 0 2024-03-22T06:15:03.404 INFO:tasks.workunit.client.0.smithi006.stdout:1/501: stat d8/d9/d10/d7c 0 2024-03-22T06:15:03.404 INFO:tasks.workunit.client.0.smithi006.stdout:1/502: creat d8/d9/d10/d7c/fa8 x:0 0 0 2024-03-22T06:15:03.405 INFO:tasks.workunit.client.0.smithi006.stdout:5/759: mknod d0/db/d16/d1c/d34/d35/d5d/ce7 0 2024-03-22T06:15:03.407 INFO:tasks.workunit.client.0.smithi006.stdout:4/673: mknod d6/d1b/d20/d63/d93/d9a/cea 0 2024-03-22T06:15:03.413 INFO:tasks.workunit.client.0.smithi006.stdout:2/677: mknod d13/d20/da6/dba/c101 0 2024-03-22T06:15:03.413 INFO:tasks.workunit.client.0.smithi006.stdout:2/678: dread - d13/d53/dc5/fde zero size 2024-03-22T06:15:03.414 INFO:tasks.workunit.client.0.smithi006.stdout:2/679: truncate d13/d20/d35/fd2 1827864 0 2024-03-22T06:15:03.414 INFO:tasks.workunit.client.0.smithi006.stdout:4/674: dread d6/f12 [0,4194304] 0 2024-03-22T06:15:03.414 INFO:tasks.workunit.client.0.smithi006.stdout:4/675: chown d6/d1b/d20/d6b/d95 59 1 2024-03-22T06:15:03.414 INFO:tasks.workunit.client.0.smithi006.stdout:2/680: write d13/d34/d4d/d4e/d75/da5/f94 [415345,130756] 0 2024-03-22T06:15:03.415 INFO:tasks.workunit.client.0.smithi006.stdout:3/611: symlink d0/d23/d33/d3d/d58/ld0 0 2024-03-22T06:15:03.419 INFO:tasks.workunit.client.0.smithi006.stdout:7/648: mknod d5/d1e/d29/cce 0 2024-03-22T06:15:03.421 INFO:tasks.workunit.client.0.smithi006.stdout:8/706: rename d9/db/d12/d7e/f8b to d9/db/d35/da9/df6/f105 0 2024-03-22T06:15:03.421 INFO:tasks.workunit.client.0.smithi006.stdout:0/460: sync 2024-03-22T06:15:03.422 INFO:tasks.workunit.client.0.smithi006.stdout:8/707: write d9/f21 [63170,110162] 0 2024-03-22T06:15:03.422 INFO:tasks.workunit.client.0.smithi006.stdout:2/681: dread d13/d20/d35/d65/f7b [0,4194304] 0 2024-03-22T06:15:03.423 INFO:tasks.workunit.client.0.smithi006.stdout:6/593: write d6/d1d/d32/f45 [1375907,129265] 0 2024-03-22T06:15:03.423 INFO:tasks.workunit.client.0.smithi006.stdout:6/594: readlink d6/d7c/d83/l9d 0 2024-03-22T06:15:03.431 INFO:tasks.workunit.client.0.smithi006.stdout:8/708: dread d9/db/d35/dd2/d4a/d4d/f75 [0,4194304] 0 2024-03-22T06:15:03.431 INFO:tasks.workunit.client.0.smithi006.stdout:1/503: link d8/d9/d13/c55 d8/d9/d13/d24/d32/ca9 0 2024-03-22T06:15:03.432 INFO:tasks.workunit.client.0.smithi006.stdout:5/760: mkdir d0/de8 0 2024-03-22T06:15:03.437 INFO:tasks.workunit.client.0.smithi006.stdout:4/676: creat d6/d1b/d46/feb x:0 0 0 2024-03-22T06:15:03.453 INFO:tasks.workunit.client.0.smithi006.stdout:3/612: rename d0/d8/d1c/d3e/d5f/fae to d0/d8/d1c/fd1 0 2024-03-22T06:15:03.453 INFO:tasks.workunit.client.0.smithi006.stdout:3/613: getdents d0/d4/dbb 0 2024-03-22T06:15:03.453 INFO:tasks.workunit.client.0.smithi006.stdout:3/614: stat d0/d8/d1c/d49/f5e 0 2024-03-22T06:15:03.454 INFO:tasks.workunit.client.0.smithi006.stdout:1/504: dwrite d8/d9/d3a/f92 [0,4194304] 0 2024-03-22T06:15:03.455 INFO:tasks.workunit.client.0.smithi006.stdout:0/461: creat db/d87/f9f x:0 0 0 2024-03-22T06:15:03.459 INFO:tasks.workunit.client.0.smithi006.stdout:2/682: link d13/d53/cce d13/d34/d4d/d4e/d75/dcc/c102 0 2024-03-22T06:15:03.461 INFO:tasks.workunit.client.0.smithi006.stdout:6/595: mknod d6/d77/db7/cc0 0 2024-03-22T06:15:03.468 INFO:tasks.workunit.client.0.smithi006.stdout:7/649: symlink d5/d1e/d29/d66/lcf 0 2024-03-22T06:15:03.468 INFO:tasks.workunit.client.0.smithi006.stdout:6/596: write d6/f11 [2718875,96672] 0 2024-03-22T06:15:03.471 INFO:tasks.workunit.client.0.smithi006.stdout:4/677: rename d6/d1b/d20/d21/fab to d6/d1b/d20/d63/fec 0 2024-03-22T06:15:03.475 INFO:tasks.workunit.client.0.smithi006.stdout:1/505: mkdir d8/d9/d13/d24/d32/d34/daa 0 2024-03-22T06:15:03.476 INFO:tasks.workunit.client.0.smithi006.stdout:2/683: fsync d13/d34/d4d/d4e/d75/da5/f47 0 2024-03-22T06:15:03.477 INFO:tasks.workunit.client.0.smithi006.stdout:8/709: getdents d9/db/dc/de/d3e 0 2024-03-22T06:15:03.477 INFO:tasks.workunit.client.0.smithi006.stdout:8/710: write d9/db/d35/dd2/d76/db3/dc3/d89/db8/ff5 [236488,68539] 0 2024-03-22T06:15:03.480 INFO:tasks.workunit.client.0.smithi006.stdout:7/650: unlink d5/d1e/d29/d1f/d2d/lb0 0 2024-03-22T06:15:03.484 INFO:tasks.workunit.client.0.smithi006.stdout:3/615: rename d0/d8/d1c/d89/fc0 to d0/d8/d1c/d3e/d5f/d83/fd2 0 2024-03-22T06:15:03.484 INFO:tasks.workunit.client.0.smithi006.stdout:3/616: truncate d0/d23/d4c/fa2 190360 0 2024-03-22T06:15:03.484 INFO:tasks.workunit.client.0.smithi006.stdout:3/617: fsync d0/f56 0 2024-03-22T06:15:03.485 INFO:tasks.workunit.client.0.smithi006.stdout:1/506: link d8/d33/f9c d8/d9/d3a/d40/d44/d80/fab 0 2024-03-22T06:15:03.488 INFO:tasks.workunit.client.0.smithi006.stdout:9/475: sync 2024-03-22T06:15:03.490 INFO:tasks.workunit.client.0.smithi006.stdout:2/684: dwrite d13/d20/d26/d19/f3e [0,4194304] 0 2024-03-22T06:15:03.490 INFO:tasks.workunit.client.0.smithi006.stdout:2/685: truncate d13/d53/f5e 756556 0 2024-03-22T06:15:03.498 INFO:tasks.workunit.client.0.smithi006.stdout:8/711: symlink d9/d14/d47/d94/l106 0 2024-03-22T06:15:03.502 INFO:tasks.workunit.client.0.smithi006.stdout:7/651: unlink d5/d1e/d29/d10/l1c 0 2024-03-22T06:15:03.503 INFO:tasks.workunit.client.0.smithi006.stdout:0/462: rename db/dc/d38/c70 to db/d3d/d4e/ca0 0 2024-03-22T06:15:03.506 INFO:tasks.workunit.client.0.smithi006.stdout:7/652: read d5/d1e/d29/d1f/d2d/d55/d84/fa0 [1754112,130254] 0 2024-03-22T06:15:03.507 INFO:tasks.workunit.client.0.smithi006.stdout:9/476: dwrite d8/d10/d11/f3c [0,4194304] 0 2024-03-22T06:15:03.507 INFO:tasks.workunit.client.0.smithi006.stdout:9/477: write d8/d10/d28/d39/d75/d50/d38/f58 [832818,81532] 0 2024-03-22T06:15:03.507 INFO:tasks.workunit.client.0.smithi006.stdout:9/478: readlink d8/d10/d23/d3e/l4b 0 2024-03-22T06:15:03.508 INFO:tasks.workunit.client.0.smithi006.stdout:1/507: link d8/d9/d10/d7c/f82 d8/fac 0 2024-03-22T06:15:03.508 INFO:tasks.workunit.client.0.smithi006.stdout:1/508: dread - d8/fa2 zero size 2024-03-22T06:15:03.515 INFO:tasks.workunit.client.0.smithi006.stdout:3/618: rename d0/d8/d1c/d3e/d5f/da3/c6f to d0/d8/d1c/d3e/d68/cd3 0 2024-03-22T06:15:03.516 INFO:tasks.workunit.client.0.smithi006.stdout:8/712: dwrite d9/db/d35/dd2/f4b [0,4194304] 0 2024-03-22T06:15:03.528 INFO:tasks.workunit.client.0.smithi006.stdout:7/653: link d5/d1e/dbc/cc2 d5/cd0 0 2024-03-22T06:15:03.533 INFO:tasks.workunit.client.0.smithi006.stdout:9/479: dwrite f6 [0,4194304] 0 2024-03-22T06:15:03.534 INFO:tasks.workunit.client.0.smithi006.stdout:9/480: chown d8/d10/d23/d3e/l95 7265701 1 2024-03-22T06:15:03.534 INFO:tasks.workunit.client.0.smithi006.stdout:8/713: dwrite d9/db/d35/da9/fc8 [0,4194304] 0 2024-03-22T06:15:03.534 INFO:tasks.workunit.client.0.smithi006.stdout:1/509: unlink d8/d9/d13/d24/f9b 0 2024-03-22T06:15:03.534 INFO:tasks.workunit.client.0.smithi006.stdout:1/510: read - d8/d9/d3a/d18/f61 zero size 2024-03-22T06:15:03.538 INFO:tasks.workunit.client.0.smithi006.stdout:2/686: mknod d13/d34/d4d/d4e/d75/da5/c103 0 2024-03-22T06:15:03.539 INFO:tasks.workunit.client.0.smithi006.stdout:7/654: dread d5/d1e/f3a [0,4194304] 0 2024-03-22T06:15:03.539 INFO:tasks.workunit.client.0.smithi006.stdout:7/655: write d5/d1e/d3f/f90 [1529985,109251] 0 2024-03-22T06:15:03.540 INFO:tasks.workunit.client.0.smithi006.stdout:0/463: rename db/d3d/d4e/d76/d81 to db/d10/d35/d7b/da1 0 2024-03-22T06:15:03.543 INFO:tasks.workunit.client.0.smithi006.stdout:9/481: write d8/d10/d23/d3e/f5a [3632721,60118] 0 2024-03-22T06:15:03.543 INFO:tasks.workunit.client.0.smithi006.stdout:9/482: truncate d8/fa 604667 0 2024-03-22T06:15:03.546 INFO:tasks.workunit.client.0.smithi006.stdout:1/511: dread d8/d9/f54 [0,4194304] 0 2024-03-22T06:15:03.546 INFO:tasks.workunit.client.0.smithi006.stdout:1/512: fdatasync d8/d9/f67 0 2024-03-22T06:15:03.555 INFO:tasks.workunit.client.0.smithi006.stdout:9/483: dread d8/d10/d28/d39/d75/f4f [0,4194304] 0 2024-03-22T06:15:03.555 INFO:tasks.workunit.client.0.smithi006.stdout:9/484: dread - d8/d10/d28/d39/d75/d50/f61 zero size 2024-03-22T06:15:03.559 INFO:tasks.workunit.client.0.smithi006.stdout:7/656: dwrite d5/d1e/d29/d59/f9d [0,4194304] 0 2024-03-22T06:15:03.559 INFO:tasks.workunit.client.0.smithi006.stdout:7/657: write d5/d1e/d63/d85/d80/fc4 [176294,6090] 0 2024-03-22T06:15:03.559 INFO:tasks.workunit.client.0.smithi006.stdout:8/714: symlink d9/d14/d47/d94/l107 0 2024-03-22T06:15:03.560 INFO:tasks.workunit.client.0.smithi006.stdout:2/687: truncate d13/d20/f52 1382387 0 2024-03-22T06:15:03.562 INFO:tasks.workunit.client.0.smithi006.stdout:0/464: unlink db/dc/f88 0 2024-03-22T06:15:03.563 INFO:tasks.workunit.client.0.smithi006.stdout:0/465: read db/dc/d38/f3a [4932070,128408] 0 2024-03-22T06:15:03.564 INFO:tasks.workunit.client.0.smithi006.stdout:3/619: getdents d0/d8/d1c/d3e/d68 0 2024-03-22T06:15:03.567 INFO:tasks.workunit.client.0.smithi006.stdout:1/513: mknod d8/d9/d13/d24/d32/d34/cad 0 2024-03-22T06:15:03.572 INFO:tasks.workunit.client.0.smithi006.stdout:7/658: mknod d5/d1e/d29/d1f/d2d/cd1 0 2024-03-22T06:15:03.572 INFO:tasks.workunit.client.0.smithi006.stdout:7/659: chown d5/d1e/d29/d1f/d2d/d55/fa8 3 1 2024-03-22T06:15:03.573 INFO:tasks.workunit.client.0.smithi006.stdout:9/485: write d8/d10/d11/f3c [4397540,43223] 0 2024-03-22T06:15:03.573 INFO:tasks.workunit.client.0.smithi006.stdout:9/486: creat d8/d10/d28/d39/d75/d50/fa2 x:0 0 0 2024-03-22T06:15:03.574 INFO:tasks.workunit.client.0.smithi006.stdout:8/715: unlink d9/db/d12/d1e/fab 0 2024-03-22T06:15:03.576 INFO:tasks.workunit.client.0.smithi006.stdout:2/688: mkdir d13/d20/d35/d104 0 2024-03-22T06:15:03.577 INFO:tasks.workunit.client.0.smithi006.stdout:2/689: readlink d13/d34/d6e/lff 0 2024-03-22T06:15:03.579 INFO:tasks.workunit.client.0.smithi006.stdout:0/466: fdatasync db/dc/f7e 0 2024-03-22T06:15:03.580 INFO:tasks.workunit.client.0.smithi006.stdout:3/620: unlink d0/d8/d6a/c70 0 2024-03-22T06:15:03.581 INFO:tasks.workunit.client.0.smithi006.stdout:3/621: creat d0/d8/d1c/d3e/d5f/fd4 x:0 0 0 2024-03-22T06:15:03.587 INFO:tasks.workunit.client.0.smithi006.stdout:1/514: rename d8/d9/d13/d24/d32/d34/c8c to d8/cae 0 2024-03-22T06:15:03.587 INFO:tasks.workunit.client.0.smithi006.stdout:1/515: chown d8/d9/f35 6 1 2024-03-22T06:15:03.590 INFO:tasks.workunit.client.0.smithi006.stdout:7/660: stat d5/d1e/d29/d59/d5c/c79 0 2024-03-22T06:15:03.590 INFO:tasks.workunit.client.0.smithi006.stdout:7/661: truncate d5/d1e/d63/d85/fab 134274 0 2024-03-22T06:15:03.590 INFO:tasks.workunit.client.0.smithi006.stdout:7/662: truncate d5/d1e/d29/d1f/d2d/d61/fbf 910890 0 2024-03-22T06:15:03.599 INFO:tasks.workunit.client.0.smithi006.stdout:9/487: symlink d8/d10/d20/la3 0 2024-03-22T06:15:03.599 INFO:tasks.workunit.client.0.smithi006.stdout:9/488: read - d8/d10/d28/d39/d75/d50/fa2 zero size 2024-03-22T06:15:03.599 INFO:tasks.workunit.client.0.smithi006.stdout:1/516: write d8/d9/d3a/d18/f62 [205809,118735] 0 2024-03-22T06:15:03.602 INFO:tasks.workunit.client.0.smithi006.stdout:8/716: symlink d9/db/dc/de3/l108 0 2024-03-22T06:15:03.608 INFO:tasks.workunit.client.0.smithi006.stdout:0/467: unlink db/d3d/f92 0 2024-03-22T06:15:03.615 INFO:tasks.workunit.client.0.smithi006.stdout:3/622: creat d0/d4/d31/d98/dab/fd5 x:0 0 0 2024-03-22T06:15:03.615 INFO:tasks.workunit.client.0.smithi006.stdout:3/623: dread - d0/d8/d1c/d95/f9b zero size 2024-03-22T06:15:03.615 INFO:tasks.workunit.client.0.smithi006.stdout:3/624: chown d0/l42 1 1 2024-03-22T06:15:03.618 INFO:tasks.workunit.client.0.smithi006.stdout:7/663: rmdir d5/d1e/d29/db9/dc9 0 2024-03-22T06:15:03.620 INFO:tasks.workunit.client.0.smithi006.stdout:3/625: write d0/f56 [4044729,55424] 0 2024-03-22T06:15:03.620 INFO:tasks.workunit.client.0.smithi006.stdout:3/626: chown d0/d23/d33/d3d/f86 527 1 2024-03-22T06:15:03.621 INFO:tasks.workunit.client.0.smithi006.stdout:1/517: mkdir d8/d9/d13/d24/daf 0 2024-03-22T06:15:03.622 INFO:tasks.workunit.client.0.smithi006.stdout:3/627: dread d0/d8/d1c/d3e/d5f/d90/da7/d97/fa5 [0,4194304] 0 2024-03-22T06:15:03.626 INFO:tasks.workunit.client.0.smithi006.stdout:8/717: rename d9/fc7 to d9/db/d35/dd2/d76/db3/dc3/f109 0 2024-03-22T06:15:03.626 INFO:tasks.workunit.client.0.smithi006.stdout:3/628: rename d0/d8/d9 to d0/d8/d9/db1/dd6 22 2024-03-22T06:15:03.628 INFO:tasks.workunit.client.0.smithi006.stdout:0/468: creat db/d10/d33/d9d/fa2 x:0 0 0 2024-03-22T06:15:03.631 INFO:tasks.workunit.client.0.smithi006.stdout:7/664: creat d5/d1e/d29/d1f/d2d/d41/fd2 x:0 0 0 2024-03-22T06:15:03.636 INFO:tasks.workunit.client.0.smithi006.stdout:8/718: truncate d9/db/dc/de/f17 937172 0 2024-03-22T06:15:03.637 INFO:tasks.workunit.client.0.smithi006.stdout:7/665: symlink d5/d1e/d63/d85/dca/ld3 0 2024-03-22T06:15:03.638 INFO:tasks.workunit.client.0.smithi006.stdout:7/666: fdatasync d5/d1e/d3f/f5a 0 2024-03-22T06:15:03.642 INFO:tasks.workunit.client.0.smithi006.stdout:8/719: rename d9/db/d12/d66/d6c/ld9 to d9/db/d35/dd2/l10a 0 2024-03-22T06:15:03.642 INFO:tasks.workunit.client.0.smithi006.stdout:8/720: creat d9/d14/d47/d94/da3/f10b x:0 0 0 2024-03-22T06:15:03.643 INFO:tasks.workunit.client.0.smithi006.stdout:1/518: dwrite d8/d9/d3a/d18/f37 [0,4194304] 0 2024-03-22T06:15:03.643 INFO:tasks.workunit.client.0.smithi006.stdout:1/519: creat d8/d9/d3a/d40/fb0 x:0 0 0 2024-03-22T06:15:03.644 INFO:tasks.workunit.client.0.smithi006.stdout:7/667: creat d5/d1e/d29/d1f/d2d/d41/db7/dcb/fd4 x:0 0 0 2024-03-22T06:15:03.645 INFO:tasks.workunit.client.0.smithi006.stdout:8/721: dread d9/db/dc/de/f22 [0,4194304] 0 2024-03-22T06:15:03.651 INFO:tasks.workunit.client.0.smithi006.stdout:1/520: mkdir d8/d9/d3a/d40/d59/db1 0 2024-03-22T06:15:03.657 INFO:tasks.workunit.client.0.smithi006.stdout:7/668: dread d5/d1e/d29/f54 [0,4194304] 0 2024-03-22T06:15:03.660 INFO:tasks.workunit.client.0.smithi006.stdout:8/722: dread d9/fa [4194304,4194304] 0 2024-03-22T06:15:03.731 INFO:tasks.workunit.client.0.smithi006.stdout:1/521: truncate d8/d9/d3a/d40/d44/f66 574945 0 2024-03-22T06:15:03.732 INFO:tasks.workunit.client.0.smithi006.stdout:7/669: mknod d5/d1e/d63/d85/dac/db3/cd5 0 2024-03-22T06:15:03.733 INFO:tasks.workunit.client.0.smithi006.stdout:7/670: stat d5/d1e/f95 0 2024-03-22T06:15:03.733 INFO:tasks.workunit.client.0.smithi006.stdout:7/671: write d5/d1e/d63/d85/d80/fc4 [40926,112518] 0 2024-03-22T06:15:03.734 INFO:tasks.workunit.client.0.smithi006.stdout:8/723: unlink d9/db/d35/dd2/lc5 0 2024-03-22T06:15:03.740 INFO:tasks.workunit.client.0.smithi006.stdout:1/522: dread d8/d33/f4d [0,4194304] 0 2024-03-22T06:15:03.743 INFO:tasks.workunit.client.0.smithi006.stdout:7/672: link d5/l23 d5/d1e/d63/d85/dca/ld6 0 2024-03-22T06:15:03.744 INFO:tasks.workunit.client.0.smithi006.stdout:9/489: rmdir d8/d10/d28/d39/d75/d50 39 2024-03-22T06:15:03.745 INFO:tasks.workunit.client.0.smithi006.stdout:8/724: rmdir d9/db/d35/dd2/d4a/d4d 39 2024-03-22T06:15:03.748 INFO:tasks.workunit.client.0.smithi006.stdout:1/523: link d8/d9/d3a/d40/fb0 d8/d9/d13/d39/d60/fb2 0 2024-03-22T06:15:03.748 INFO:tasks.workunit.client.0.smithi006.stdout:1/524: creat d8/d9/d13/d24/d32/d34/fb3 x:0 0 0 2024-03-22T06:15:03.752 INFO:tasks.workunit.client.0.smithi006.stdout:8/725: mkdir d9/db/d12/d66/d10c 0 2024-03-22T06:15:03.753 INFO:tasks.workunit.client.0.smithi006.stdout:8/726: dread d9/d14/d47/d94/da3/fda [0,4194304] 0 2024-03-22T06:15:03.753 INFO:tasks.workunit.client.0.smithi006.stdout:8/727: truncate d9/db/d12/d66/d6c/d71/f101 933577 0 2024-03-22T06:15:03.759 INFO:tasks.workunit.client.0.smithi006.stdout:9/490: dwrite d8/f9e [0,4194304] 0 2024-03-22T06:15:03.760 INFO:tasks.workunit.client.0.smithi006.stdout:7/673: getdents d5/d1e/d29/d1f/d2d/d55 0 2024-03-22T06:15:03.763 INFO:tasks.workunit.client.0.smithi006.stdout:8/728: rename d9/db/d12/d66/fff to d9/db/d35/dd2/dc4/ddd/f10d 0 2024-03-22T06:15:03.775 INFO:tasks.workunit.client.0.smithi006.stdout:7/674: creat d5/d1e/d29/d59/fd7 x:0 0 0 2024-03-22T06:15:03.775 INFO:tasks.workunit.client.0.smithi006.stdout:7/675: fdatasync d5/d1e/d29/d59/d5c/fb1 0 2024-03-22T06:15:03.775 INFO:tasks.workunit.client.0.smithi006.stdout:7/676: write d5/d1e/d63/d85/f98 [555929,62517] 0 2024-03-22T06:15:03.775 INFO:tasks.workunit.client.0.smithi006.stdout:8/729: creat d9/db/f10e x:0 0 0 2024-03-22T06:15:03.783 INFO:tasks.workunit.client.0.smithi006.stdout:7/677: readlink d5/d1e/d29/d66/l8b 0 2024-03-22T06:15:03.784 INFO:tasks.workunit.client.0.smithi006.stdout:8/730: creat d9/db/dc/de/d3e/f10f x:0 0 0 2024-03-22T06:15:03.789 INFO:tasks.workunit.client.0.smithi006.stdout:7/678: mknod d5/d1e/d63/d85/cd8 0 2024-03-22T06:15:03.789 INFO:tasks.workunit.client.0.smithi006.stdout:7/679: fdatasync d5/d1e/d29/d1f/f38 0 2024-03-22T06:15:03.791 INFO:tasks.workunit.client.0.smithi006.stdout:8/731: creat d9/db/d12/f110 x:0 0 0 2024-03-22T06:15:03.793 INFO:tasks.workunit.client.0.smithi006.stdout:9/491: rmdir d8/d10/d23/d89 0 2024-03-22T06:15:03.797 INFO:tasks.workunit.client.0.smithi006.stdout:7/680: symlink d5/d1e/d63/ld9 0 2024-03-22T06:15:03.798 INFO:tasks.workunit.client.0.smithi006.stdout:7/681: chown d5/d1e/d29/db9 18448630 1 2024-03-22T06:15:03.799 INFO:tasks.workunit.client.0.smithi006.stdout:8/732: link d9/db/d12/d66/cac d9/db/d12/d7e/c111 0 2024-03-22T06:15:03.801 INFO:tasks.workunit.client.0.smithi006.stdout:7/682: creat d5/d1e/d29/d1f/d2d/fda x:0 0 0 2024-03-22T06:15:03.801 INFO:tasks.workunit.client.0.smithi006.stdout:8/733: dread d9/db/f4f [0,4194304] 0 2024-03-22T06:15:03.804 INFO:tasks.workunit.client.0.smithi006.stdout:8/734: mknod d9/db/d35/dd2/dc4/ddd/c112 0 2024-03-22T06:15:03.808 INFO:tasks.workunit.client.0.smithi006.stdout:8/735: getdents d9/db/d35 0 2024-03-22T06:15:03.812 INFO:tasks.workunit.client.0.smithi006.stdout:8/736: symlink d9/db/d35/dd2/d4a/d4d/l113 0 2024-03-22T06:15:03.812 INFO:tasks.workunit.client.0.smithi006.stdout:8/737: write d9/db/d12/d66/d6c/d71/d79/ff3 [142856,24475] 0 2024-03-22T06:15:03.812 INFO:tasks.workunit.client.0.smithi006.stdout:8/738: write d9/db/d35/dd2/d4a/da7/fef [727129,111584] 0 2024-03-22T06:15:03.813 INFO:tasks.workunit.client.0.smithi006.stdout:9/492: dwrite d8/d10/d23/f7d [4194304,4194304] 0 2024-03-22T06:15:03.818 INFO:tasks.workunit.client.0.smithi006.stdout:4/678: sync 2024-03-22T06:15:03.818 INFO:tasks.workunit.client.0.smithi006.stdout:5/761: sync 2024-03-22T06:15:03.818 INFO:tasks.workunit.client.0.smithi006.stdout:6/597: sync 2024-03-22T06:15:03.818 INFO:tasks.workunit.client.0.smithi006.stdout:5/762: chown d0/db/d16/d1c/d34/d35/d5d 16195 1 2024-03-22T06:15:03.818 INFO:tasks.workunit.client.0.smithi006.stdout:4/679: chown d6/d1b/d20/d6b/d7b 151 1 2024-03-22T06:15:03.818 INFO:tasks.workunit.client.0.smithi006.stdout:5/763: chown d0/db/d16/d1c/d34/d35/d85/d5e/d66 262116361 1 2024-03-22T06:15:03.827 INFO:tasks.workunit.client.0.smithi006.stdout:9/493: mknod d8/d10/d28/d39/ca4 0 2024-03-22T06:15:03.829 INFO:tasks.workunit.client.0.smithi006.stdout:7/683: dwrite d5/d1e/d29/d1f/d2d/d41/f7b [0,4194304] 0 2024-03-22T06:15:03.832 INFO:tasks.workunit.client.0.smithi006.stdout:9/494: symlink d8/d10/d47/la5 0 2024-03-22T06:15:03.832 INFO:tasks.workunit.client.0.smithi006.stdout:4/680: dread d6/d1b/d20/d6b/f75 [0,4194304] 0 2024-03-22T06:15:03.835 INFO:tasks.workunit.client.0.smithi006.stdout:1/525: rmdir d8/d9/d13/d24 39 2024-03-22T06:15:03.837 INFO:tasks.workunit.client.0.smithi006.stdout:8/739: dwrite d9/db/d12/d1e/f39 [0,4194304] 0 2024-03-22T06:15:03.838 INFO:tasks.workunit.client.0.smithi006.stdout:6/598: dwrite d6/d1b/d2c/f7d [4194304,4194304] 0 2024-03-22T06:15:03.839 INFO:tasks.workunit.client.0.smithi006.stdout:4/681: creat d6/d1b/d20/d21/d77/da0/fed x:0 0 0 2024-03-22T06:15:03.842 INFO:tasks.workunit.client.0.smithi006.stdout:1/526: stat d8/d9/l49 0 2024-03-22T06:15:03.842 INFO:tasks.workunit.client.0.smithi006.stdout:1/527: chown d8/d9/d13/d39/d60/fb2 1 1 2024-03-22T06:15:03.842 INFO:tasks.workunit.client.0.smithi006.stdout:1/528: readlink d8/le 0 2024-03-22T06:15:03.842 INFO:tasks.workunit.client.0.smithi006.stdout:7/684: dread d5/d1e/d3f/f6c [0,4194304] 0 2024-03-22T06:15:03.846 INFO:tasks.workunit.client.0.smithi006.stdout:5/764: dread d0/db/d16/f21 [0,4194304] 0 2024-03-22T06:15:03.850 INFO:tasks.workunit.client.0.smithi006.stdout:9/495: dwrite d8/d10/d23/f40 [0,4194304] 0 2024-03-22T06:15:03.850 INFO:tasks.workunit.client.0.smithi006.stdout:9/496: write d8/d10/d28/d39/d75/d50/fa2 [44675,108566] 0 2024-03-22T06:15:03.856 INFO:tasks.workunit.client.0.smithi006.stdout:8/740: creat d9/db/d35/dd2/d76/f114 x:0 0 0 2024-03-22T06:15:03.856 INFO:tasks.workunit.client.0.smithi006.stdout:8/741: truncate d9/fb5 7688 0 2024-03-22T06:15:03.859 INFO:tasks.workunit.client.0.smithi006.stdout:6/599: getdents d6/d1d/d32 0 2024-03-22T06:15:03.861 INFO:tasks.workunit.client.0.smithi006.stdout:1/529: mknod d8/d9/d10/d7c/d96/cb4 0 2024-03-22T06:15:03.861 INFO:tasks.workunit.client.0.smithi006.stdout:1/530: fdatasync d8/d9/d3a/f6b 0 2024-03-22T06:15:03.861 INFO:tasks.workunit.client.0.smithi006.stdout:1/531: fdatasync d8/d9/d10/d7c/fa6 0 2024-03-22T06:15:03.863 INFO:tasks.workunit.client.0.smithi006.stdout:5/765: creat d0/db/d16/d43/fe9 x:0 0 0 2024-03-22T06:15:03.864 INFO:tasks.workunit.client.0.smithi006.stdout:4/682: dwrite d6/d1b/d20/d21/d2a/d43/fe4 [0,4194304] 0 2024-03-22T06:15:03.864 INFO:tasks.workunit.client.0.smithi006.stdout:4/683: getdents d6/d1b/d20/d63/d93/db8/dbc 0 2024-03-22T06:15:03.867 INFO:tasks.workunit.client.0.smithi006.stdout:9/497: mkdir d8/d10/da6 0 2024-03-22T06:15:03.867 INFO:tasks.workunit.client.0.smithi006.stdout:9/498: fsync d8/ff 0 2024-03-22T06:15:03.867 INFO:tasks.workunit.client.0.smithi006.stdout:9/499: readlink d8/d10/l24 0 2024-03-22T06:15:03.869 INFO:tasks.workunit.client.0.smithi006.stdout:5/766: dread d0/db/d50/d96/d52/d69/fa3 [0,4194304] 0 2024-03-22T06:15:03.870 INFO:tasks.workunit.client.0.smithi006.stdout:8/742: symlink d9/db/dc/de/d3e/l115 0 2024-03-22T06:15:03.872 INFO:tasks.workunit.client.0.smithi006.stdout:1/532: mkdir d8/d9/d13/d39/d94/db5 0 2024-03-22T06:15:03.872 INFO:tasks.workunit.client.0.smithi006.stdout:7/685: dwrite d5/f5f [0,4194304] 0 2024-03-22T06:15:03.872 INFO:tasks.workunit.client.0.smithi006.stdout:6/600: dread d6/d24/f38 [0,4194304] 0 2024-03-22T06:15:03.878 INFO:tasks.workunit.client.0.smithi006.stdout:9/500: truncate f6 3510951 0 2024-03-22T06:15:03.888 INFO:tasks.workunit.client.0.smithi006.stdout:1/533: symlink d8/d9/d3a/d40/d59/lb6 0 2024-03-22T06:15:03.888 INFO:tasks.workunit.client.0.smithi006.stdout:2/690: sync 2024-03-22T06:15:03.888 INFO:tasks.workunit.client.0.smithi006.stdout:2/691: chown d13/d20/d26/l8e 12 1 2024-03-22T06:15:03.896 INFO:tasks.workunit.client.0.smithi006.stdout:2/692: dread d13/d20/da6/fc2 [0,4194304] 0 2024-03-22T06:15:03.897 INFO:tasks.workunit.client.0.smithi006.stdout:7/686: creat d5/d1e/d29/d66/fdb x:0 0 0 2024-03-22T06:15:03.901 INFO:tasks.workunit.client.0.smithi006.stdout:6/601: creat d6/fc1 x:0 0 0 2024-03-22T06:15:03.901 INFO:tasks.workunit.client.0.smithi006.stdout:6/602: truncate d6/d1d/d5c/f5e 4951698 0 2024-03-22T06:15:03.904 INFO:tasks.workunit.client.0.smithi006.stdout:5/767: getdents d0/db/d50/d58/dad 0 2024-03-22T06:15:03.904 INFO:tasks.workunit.client.0.smithi006.stdout:5/768: chown d0/db/d50/d96/d52 49067653 1 2024-03-22T06:15:03.906 INFO:tasks.workunit.client.0.smithi006.stdout:1/534: symlink d8/d71/lb7 0 2024-03-22T06:15:03.907 INFO:tasks.workunit.client.0.smithi006.stdout:9/501: dwrite d8/d10/d47/f87 [0,4194304] 0 2024-03-22T06:15:03.908 INFO:tasks.workunit.client.0.smithi006.stdout:2/693: mkdir d13/d34/d4d/d4e/d105 0 2024-03-22T06:15:03.910 INFO:tasks.workunit.client.0.smithi006.stdout:5/769: creat d0/db/d99/fea x:0 0 0 2024-03-22T06:15:03.911 INFO:tasks.workunit.client.0.smithi006.stdout:5/770: dread d0/db/d50/d96/d52/fc9 [0,4194304] 0 2024-03-22T06:15:03.911 INFO:tasks.workunit.client.0.smithi006.stdout:5/771: creat d0/db/d16/d43/feb x:0 0 0 2024-03-22T06:15:03.912 INFO:tasks.workunit.client.0.smithi006.stdout:8/743: dwrite d9/db/d12/fb6 [0,4194304] 0 2024-03-22T06:15:03.912 INFO:tasks.workunit.client.0.smithi006.stdout:3/629: sync 2024-03-22T06:15:03.912 INFO:tasks.workunit.client.0.smithi006.stdout:0/469: sync 2024-03-22T06:15:03.913 INFO:tasks.workunit.client.0.smithi006.stdout:0/470: write db/dc/d38/f8e [800902,60224] 0 2024-03-22T06:15:03.913 INFO:tasks.workunit.client.0.smithi006.stdout:7/687: dwrite d5/d1e/d29/f2a [4194304,4194304] 0 2024-03-22T06:15:03.913 INFO:tasks.workunit.client.0.smithi006.stdout:7/688: fdatasync d5/d1e/d29/d1f/d2d/d41/db7/dcb/fd4 0 2024-03-22T06:15:03.913 INFO:tasks.workunit.client.0.smithi006.stdout:1/535: dread d8/d9/d3a/f6b [0,4194304] 0 2024-03-22T06:15:03.913 INFO:tasks.workunit.client.0.smithi006.stdout:1/536: chown d8/d33/f5b 5939 1 2024-03-22T06:15:03.919 INFO:tasks.workunit.client.0.smithi006.stdout:2/694: rename d13/d20/lee to d13/d34/d4d/d4e/d75/da5/d6c/l106 0 2024-03-22T06:15:03.919 INFO:tasks.workunit.client.0.smithi006.stdout:2/695: dread - d13/d53/dc5/fde zero size 2024-03-22T06:15:03.924 INFO:tasks.workunit.client.0.smithi006.stdout:5/772: link d0/db/d16/d1c/le6 d0/db/d16/d1c/d34/d35/d85/dda/d65/lec 0 2024-03-22T06:15:03.924 INFO:tasks.workunit.client.0.smithi006.stdout:5/773: fsync d0/db/d16/f21 0 2024-03-22T06:15:03.927 INFO:tasks.workunit.client.0.smithi006.stdout:9/502: write d8/d10/d20/d45/d49/f77 [2481832,29201] 0 2024-03-22T06:15:03.927 INFO:tasks.workunit.client.0.smithi006.stdout:6/603: dwrite d6/d24/d44/f46 [0,4194304] 0 2024-03-22T06:15:03.932 INFO:tasks.workunit.client.0.smithi006.stdout:8/744: creat d9/db/d35/f116 x:0 0 0 2024-03-22T06:15:03.932 INFO:tasks.workunit.client.0.smithi006.stdout:8/745: write d9/db/d35/dd2/dad/fb7 [3182854,61581] 0 2024-03-22T06:15:03.932 INFO:tasks.workunit.client.0.smithi006.stdout:8/746: dread - d9/db/d12/d1e/fa8 zero size 2024-03-22T06:15:03.934 INFO:tasks.workunit.client.0.smithi006.stdout:0/471: mknod db/d10/d33/ca3 0 2024-03-22T06:15:03.936 INFO:tasks.workunit.client.0.smithi006.stdout:6/604: dread d6/f16 [0,4194304] 0 2024-03-22T06:15:03.940 INFO:tasks.workunit.client.0.smithi006.stdout:7/689: mkdir d5/d1e/d29/d66/d91/ddc 0 2024-03-22T06:15:03.940 INFO:tasks.workunit.client.0.smithi006.stdout:7/690: chown d5/d1e/d63/d85/fab 2672 1 2024-03-22T06:15:03.943 INFO:tasks.workunit.client.0.smithi006.stdout:1/537: rename d8/d9/l3b to d8/d9/d13/lb8 0 2024-03-22T06:15:03.947 INFO:tasks.workunit.client.0.smithi006.stdout:2/696: mkdir d13/d34/d4d/d4e/d75/da5/d6c/d107 0 2024-03-22T06:15:03.947 INFO:tasks.workunit.client.0.smithi006.stdout:2/697: dread - d13/d53/d6b/fc0 zero size 2024-03-22T06:15:03.959 INFO:tasks.workunit.client.0.smithi006.stdout:7/691: write d5/d1e/d3f/f7d [3570615,97012] 0 2024-03-22T06:15:03.963 INFO:tasks.workunit.client.0.smithi006.stdout:3/630: mknod d0/d8/cd7 0 2024-03-22T06:15:03.964 INFO:tasks.workunit.client.0.smithi006.stdout:3/631: dread d0/d23/d4c/fa2 [0,4194304] 0 2024-03-22T06:15:03.964 INFO:tasks.workunit.client.0.smithi006.stdout:3/632: fdatasync d0/f56 0 2024-03-22T06:15:03.964 INFO:tasks.workunit.client.0.smithi006.stdout:3/633: dread - d0/d8/d1c/d3e/d68/fce zero size 2024-03-22T06:15:03.964 INFO:tasks.workunit.client.0.smithi006.stdout:3/634: write d0/fcf [751297,72926] 0 2024-03-22T06:15:03.968 INFO:tasks.workunit.client.0.smithi006.stdout:8/747: rename d9/db/d35/dd2/l9c to d9/db/d35/dd2/l117 0 2024-03-22T06:15:03.970 INFO:tasks.workunit.client.0.smithi006.stdout:0/472: symlink db/d10/d2a/la4 0 2024-03-22T06:15:03.970 INFO:tasks.workunit.client.0.smithi006.stdout:0/473: fsync db/d10/f23 0 2024-03-22T06:15:03.971 INFO:tasks.workunit.client.0.smithi006.stdout:0/474: read db/d10/d33/f42 [439491,21649] 0 2024-03-22T06:15:03.972 INFO:tasks.workunit.client.0.smithi006.stdout:0/475: write db/d3d/d4e/f90 [1200705,115599] 0 2024-03-22T06:15:03.972 INFO:tasks.workunit.client.0.smithi006.stdout:0/476: chown db/dc/f44 1298 1 2024-03-22T06:15:03.972 INFO:tasks.workunit.client.0.smithi006.stdout:0/477: fdatasync db/d10/d35/d6c/f9c 0 2024-03-22T06:15:03.975 INFO:tasks.workunit.client.0.smithi006.stdout:1/538: rename d8/le to d8/d9/d13/d24/d32/d34/lb9 0 2024-03-22T06:15:03.977 INFO:tasks.workunit.client.0.smithi006.stdout:5/774: rmdir d0/db/d16/d1c/d34/d4d 39 2024-03-22T06:15:03.982 INFO:tasks.workunit.client.0.smithi006.stdout:7/692: mkdir d5/d1e/d29/d59/dcd/ddd 0 2024-03-22T06:15:03.983 INFO:tasks.workunit.client.0.smithi006.stdout:9/503: getdents d8/d10/d47 0 2024-03-22T06:15:03.985 INFO:tasks.workunit.client.0.smithi006.stdout:0/478: mknod db/dc/d38/d50/ca5 0 2024-03-22T06:15:03.985 INFO:tasks.workunit.client.0.smithi006.stdout:1/539: dread d8/d9/d13/f5a [0,4194304] 0 2024-03-22T06:15:03.988 INFO:tasks.workunit.client.0.smithi006.stdout:2/698: rename d13/d34/ff2 to d13/d20/d35/d65/f108 0 2024-03-22T06:15:03.988 INFO:tasks.workunit.client.0.smithi006.stdout:8/748: write d9/db/d12/f33 [546123,88742] 0 2024-03-22T06:15:03.989 INFO:tasks.workunit.client.0.smithi006.stdout:2/699: dread d13/d34/d4d/d4e/fe0 [0,4194304] 0 2024-03-22T06:15:03.990 INFO:tasks.workunit.client.0.smithi006.stdout:3/635: dwrite d0/d23/f25 [4194304,4194304] 0 2024-03-22T06:15:03.990 INFO:tasks.workunit.client.0.smithi006.stdout:6/605: dread d6/f1e [0,4194304] 0 2024-03-22T06:15:03.991 INFO:tasks.workunit.client.0.smithi006.stdout:8/749: dread d9/db/d35/dd2/d4a/da7/fef [0,4194304] 0 2024-03-22T06:15:03.992 INFO:tasks.workunit.client.0.smithi006.stdout:5/775: link d0/db/d16/d1c/d34/l3a d0/db/d50/d96/d52/d69/d92/db7/led 0 2024-03-22T06:15:03.992 INFO:tasks.workunit.client.0.smithi006.stdout:5/776: truncate d0/db/d16/d1c/d34/d35/d85/dda/fd3 542144 0 2024-03-22T06:15:03.992 INFO:tasks.workunit.client.0.smithi006.stdout:5/777: chown d0/db/d99/f1d 3 1 2024-03-22T06:15:03.996 INFO:tasks.workunit.client.0.smithi006.stdout:6/606: write d6/d1d/d5c/d81/d8a/dac/fbb [2210980,28137] 0 2024-03-22T06:15:04.001 INFO:tasks.workunit.client.0.smithi006.stdout:0/479: mkdir db/d10/d35/d7b/da1/da6 0 2024-03-22T06:15:04.001 INFO:tasks.workunit.client.0.smithi006.stdout:5/778: dread d0/db/d50/f80 [0,4194304] 0 2024-03-22T06:15:04.001 INFO:tasks.workunit.client.0.smithi006.stdout:5/779: readlink d0/db/d50/d82/l93 0 2024-03-22T06:15:04.001 INFO:tasks.workunit.client.0.smithi006.stdout:5/780: chown d0/db/d50/d96/d52/d69 5 1 2024-03-22T06:15:04.002 INFO:tasks.workunit.client.0.smithi006.stdout:5/781: stat d0/db/d76/f81 0 2024-03-22T06:15:04.002 INFO:tasks.workunit.client.0.smithi006.stdout:5/782: write d0/db/d50/d58/f6f [256803,12080] 0 2024-03-22T06:15:04.006 INFO:tasks.workunit.client.0.smithi006.stdout:1/540: creat d8/d9/d10/d7c/d96/fba x:0 0 0 2024-03-22T06:15:04.006 INFO:tasks.workunit.client.0.smithi006.stdout:1/541: fsync d8/d9/d13/f28 0 2024-03-22T06:15:04.010 INFO:tasks.workunit.client.0.smithi006.stdout:2/700: unlink d13/d53/d6b/cf5 0 2024-03-22T06:15:04.013 INFO:tasks.workunit.client.0.smithi006.stdout:9/504: dwrite d8/d1f/f5d [0,4194304] 0 2024-03-22T06:15:04.013 INFO:tasks.workunit.client.0.smithi006.stdout:8/750: mknod d9/db/d35/dd2/dc4/d104/c118 0 2024-03-22T06:15:04.013 INFO:tasks.workunit.client.0.smithi006.stdout:8/751: dread - d9/db/d12/d66/f9b zero size 2024-03-22T06:15:04.018 INFO:tasks.workunit.client.0.smithi006.stdout:6/607: rename d6/f40 to d6/d1d/d99/fc2 0 2024-03-22T06:15:04.019 INFO:tasks.workunit.client.0.smithi006.stdout:7/693: symlink d5/d1e/d29/d66/lde 0 2024-03-22T06:15:04.025 INFO:tasks.workunit.client.0.smithi006.stdout:8/752: write d9/d14/d47/f64 [2126389,30820] 0 2024-03-22T06:15:04.025 INFO:tasks.workunit.client.0.smithi006.stdout:1/542: getdents d8/d9/d13/d39/d60 0 2024-03-22T06:15:04.025 INFO:tasks.workunit.client.0.smithi006.stdout:3/636: dwrite d0/d7e/f80 [0,4194304] 0 2024-03-22T06:15:04.025 INFO:tasks.workunit.client.0.smithi006.stdout:6/608: write d6/d24/d44/f5f [2780392,77679] 0 2024-03-22T06:15:04.026 INFO:tasks.workunit.client.0.smithi006.stdout:2/701: symlink d13/d20/d26/d19/d99/l109 0 2024-03-22T06:15:04.032 INFO:tasks.workunit.client.0.smithi006.stdout:9/505: creat d8/d10/d28/fa7 x:0 0 0 2024-03-22T06:15:04.036 INFO:tasks.workunit.client.0.smithi006.stdout:7/694: symlink d5/d1e/d63/ldf 0 2024-03-22T06:15:04.036 INFO:tasks.workunit.client.0.smithi006.stdout:7/695: readlink d5/d1e/d63/d85/d80/l8f 0 2024-03-22T06:15:04.036 INFO:tasks.workunit.client.0.smithi006.stdout:7/696: chown d5/d1e/d29/d1f/d2d/d61/c86 0 1 2024-03-22T06:15:04.036 INFO:tasks.workunit.client.0.smithi006.stdout:7/697: write d5/d1e/d29/d1f/f38 [2691605,45284] 0 2024-03-22T06:15:04.041 INFO:tasks.workunit.client.0.smithi006.stdout:8/753: creat d9/db/d35/dd2/dc4/f119 x:0 0 0 2024-03-22T06:15:04.041 INFO:tasks.workunit.client.0.smithi006.stdout:8/754: truncate d9/f6b 617576 0 2024-03-22T06:15:04.043 INFO:tasks.workunit.client.0.smithi006.stdout:1/543: mkdir d8/d71/dbb 0 2024-03-22T06:15:04.043 INFO:tasks.workunit.client.0.smithi006.stdout:1/544: write d8/d9/d3a/d18/f78 [3670391,80489] 0 2024-03-22T06:15:04.043 INFO:tasks.workunit.client.0.smithi006.stdout:1/545: write d8/d9/d3a/d18/f62 [701352,4680] 0 2024-03-22T06:15:04.043 INFO:tasks.workunit.client.0.smithi006.stdout:3/637: dwrite d0/d23/d4c/f30 [0,4194304] 0 2024-03-22T06:15:04.048 INFO:tasks.workunit.client.0.smithi006.stdout:6/609: link d6/d1d/d32/c4e d6/d6f/cc3 0 2024-03-22T06:15:04.052 INFO:tasks.workunit.client.0.smithi006.stdout:2/702: truncate d13/d20/d35/d65/f108 4149928 0 2024-03-22T06:15:04.054 INFO:tasks.workunit.client.0.smithi006.stdout:9/506: mknod d8/d1f/d34/ca8 0 2024-03-22T06:15:04.057 INFO:tasks.workunit.client.0.smithi006.stdout:7/698: mknod d5/d1e/d29/d59/d5c/ce0 0 2024-03-22T06:15:04.057 INFO:tasks.workunit.client.0.smithi006.stdout:7/699: stat d5/d1e/d29/d1f/d2d/c43 0 2024-03-22T06:15:04.057 INFO:tasks.workunit.client.0.smithi006.stdout:7/700: read - d5/d1e/d29/d1f/d2d/d55/f8c zero size 2024-03-22T06:15:04.059 INFO:tasks.workunit.client.0.smithi006.stdout:8/755: creat d9/db/d35/dd2/d4a/f11a x:0 0 0 2024-03-22T06:15:04.060 INFO:tasks.workunit.client.0.smithi006.stdout:8/756: truncate d9/d5e/fc2 465911 0 2024-03-22T06:15:04.060 INFO:tasks.workunit.client.0.smithi006.stdout:8/757: fsync f7 0 2024-03-22T06:15:04.067 INFO:tasks.workunit.client.0.smithi006.stdout:1/546: creat d8/d9/d13/d39/d94/fbc x:0 0 0 2024-03-22T06:15:04.067 INFO:tasks.workunit.client.0.smithi006.stdout:1/547: fdatasync d8/d9/d3a/f20 0 2024-03-22T06:15:04.068 INFO:tasks.workunit.client.0.smithi006.stdout:3/638: getdents d0/d8/d1c/d3e/d5f/d83/d8c 0 2024-03-22T06:15:04.068 INFO:tasks.workunit.client.0.smithi006.stdout:3/639: dread - d0/d8/d1c/d95/f9b zero size 2024-03-22T06:15:04.076 INFO:tasks.workunit.client.0.smithi006.stdout:9/507: creat d8/d10/d28/d39/d75/d50/d38/fa9 x:0 0 0 2024-03-22T06:15:04.077 INFO:tasks.workunit.client.0.smithi006.stdout:7/701: getdents d5/d1e/d63/d85/dac 0 2024-03-22T06:15:04.080 INFO:tasks.workunit.client.0.smithi006.stdout:8/758: mknod d9/db/d35/dd2/d76/db3/dc3/c11b 0 2024-03-22T06:15:04.083 INFO:tasks.workunit.client.0.smithi006.stdout:1/548: mknod d8/d9/cbd 0 2024-03-22T06:15:04.086 INFO:tasks.workunit.client.0.smithi006.stdout:9/508: write d8/d1f/f68 [3186672,34819] 0 2024-03-22T06:15:04.086 INFO:tasks.workunit.client.0.smithi006.stdout:9/509: stat d8/d10/d28/d39/d75/d50/f84 0 2024-03-22T06:15:04.096 INFO:tasks.workunit.client.0.smithi006.stdout:7/702: dread d5/d1e/f22 [0,4194304] 0 2024-03-22T06:15:04.098 INFO:tasks.workunit.client.0.smithi006.stdout:2/703: mkdir d13/d34/d4d/d4e/d75/da5/d10a 0 2024-03-22T06:15:04.099 INFO:tasks.workunit.client.0.smithi006.stdout:8/759: creat d9/d14/dcc/f11c x:0 0 0 2024-03-22T06:15:04.101 INFO:tasks.workunit.client.0.smithi006.stdout:1/549: symlink d8/d9/d3a/d40/d44/d80/lbe 0 2024-03-22T06:15:04.101 INFO:tasks.workunit.client.0.smithi006.stdout:1/550: dread - d8/d9/d13/d24/d32/d34/f99 zero size 2024-03-22T06:15:04.102 INFO:tasks.workunit.client.0.smithi006.stdout:8/760: dread d9/db/d35/da9/df6/f105 [0,4194304] 0 2024-03-22T06:15:04.104 INFO:tasks.workunit.client.0.smithi006.stdout:3/640: getdents d0/d23/d4c 0 2024-03-22T06:15:04.105 INFO:tasks.workunit.client.0.smithi006.stdout:3/641: dread d0/d7e/fa0 [0,4194304] 0 2024-03-22T06:15:04.105 INFO:tasks.workunit.client.0.smithi006.stdout:2/704: dread d13/d20/d35/d65/f7b [0,4194304] 0 2024-03-22T06:15:04.109 INFO:tasks.workunit.client.0.smithi006.stdout:9/510: truncate d8/d10/d28/f55 2650689 0 2024-03-22T06:15:04.116 INFO:tasks.workunit.client.0.smithi006.stdout:2/705: write d13/d34/d4d/f5c [3340348,55939] 0 2024-03-22T06:15:04.116 INFO:tasks.workunit.client.0.smithi006.stdout:2/706: write d13/d20/f74 [6050,63528] 0 2024-03-22T06:15:04.119 INFO:tasks.workunit.client.0.smithi006.stdout:7/703: link d5/d1e/d29/d1f/d2d/d61/daa/fc0 d5/d1e/fe1 0 2024-03-22T06:15:04.119 INFO:tasks.workunit.client.0.smithi006.stdout:7/704: write d5/d1e/d63/d85/fab [544204,88199] 0 2024-03-22T06:15:04.120 INFO:tasks.workunit.client.0.smithi006.stdout:1/551: truncate d8/d9/d3a/f6b 3910681 0 2024-03-22T06:15:04.121 INFO:tasks.workunit.client.0.smithi006.stdout:1/552: fsync d8/d9/d13/d39/d60/f6e 0 2024-03-22T06:15:04.121 INFO:tasks.workunit.client.0.smithi006.stdout:1/553: dread - d8/d9/f67 zero size 2024-03-22T06:15:04.121 INFO:tasks.workunit.client.0.smithi006.stdout:1/554: write d8/d9/d3a/d18/f62 [4744567,68164] 0 2024-03-22T06:15:04.126 INFO:tasks.workunit.client.0.smithi006.stdout:3/642: symlink d0/d8/d1c/d89/ld8 0 2024-03-22T06:15:04.126 INFO:tasks.workunit.client.0.smithi006.stdout:9/511: dwrite d8/d10/f17 [4194304,4194304] 0 2024-03-22T06:15:04.126 INFO:tasks.workunit.client.0.smithi006.stdout:9/512: dread - d8/d10/d11/f42 zero size 2024-03-22T06:15:04.133 INFO:tasks.workunit.client.0.smithi006.stdout:1/555: creat d8/d9/d3a/d40/da1/fbf x:0 0 0 2024-03-22T06:15:04.134 INFO:tasks.workunit.client.0.smithi006.stdout:1/556: write d8/d33/f9c [475070,124066] 0 2024-03-22T06:15:04.135 INFO:tasks.workunit.client.0.smithi006.stdout:8/761: getdents d9/db/d35/dd2/dc4 0 2024-03-22T06:15:04.138 INFO:tasks.workunit.client.0.smithi006.stdout:3/643: creat d0/d8/d9/fd9 x:0 0 0 2024-03-22T06:15:04.138 INFO:tasks.workunit.client.0.smithi006.stdout:3/644: readlink d0/d8/d1c/d95/lcd 0 2024-03-22T06:15:04.138 INFO:tasks.workunit.client.0.smithi006.stdout:3/645: write d0/d8/d1c/d49/f67 [1431103,109522] 0 2024-03-22T06:15:04.138 INFO:tasks.workunit.client.0.smithi006.stdout:3/646: truncate d0/d23/d4c/fa4 632402 0 2024-03-22T06:15:04.142 INFO:tasks.workunit.client.0.smithi006.stdout:9/513: mknod d8/d10/d20/d45/caa 0 2024-03-22T06:15:04.142 INFO:tasks.workunit.client.0.smithi006.stdout:4/684: sync 2024-03-22T06:15:04.143 INFO:tasks.workunit.client.0.smithi006.stdout:2/707: dwrite d13/d20/da6/fec [0,4194304] 0 2024-03-22T06:15:04.143 INFO:tasks.workunit.client.0.smithi006.stdout:7/705: dwrite d5/f9 [0,4194304] 0 2024-03-22T06:15:04.147 INFO:tasks.workunit.client.0.smithi006.stdout:8/762: mknod d9/db/d12/d7e/c11d 0 2024-03-22T06:15:04.150 INFO:tasks.workunit.client.0.smithi006.stdout:3/647: symlink d0/d8/d1c/d49/lda 0 2024-03-22T06:15:04.151 INFO:tasks.workunit.client.0.smithi006.stdout:3/648: truncate d0/d8/d1c/d3e/d5f/da3/fc8 851037 0 2024-03-22T06:15:04.152 INFO:tasks.workunit.client.0.smithi006.stdout:9/514: truncate d8/d10/d20/fa0 56644 0 2024-03-22T06:15:04.152 INFO:tasks.workunit.client.0.smithi006.stdout:4/685: write d6/d1b/d20/d6b/f8c [1616883,121346] 0 2024-03-22T06:15:04.161 INFO:tasks.workunit.client.0.smithi006.stdout:2/708: mkdir d13/d53/d6b/d10b 0 2024-03-22T06:15:04.162 INFO:tasks.workunit.client.0.smithi006.stdout:7/706: symlink d5/d1e/d63/d85/le2 0 2024-03-22T06:15:04.167 INFO:tasks.workunit.client.0.smithi006.stdout:8/763: mkdir d9/db/d12/d1e/d11e 0 2024-03-22T06:15:04.167 INFO:tasks.workunit.client.0.smithi006.stdout:8/764: truncate d9/db/d12/d66/d6c/d80/fa0 463910 0 2024-03-22T06:15:04.170 INFO:tasks.workunit.client.0.smithi006.stdout:9/515: dwrite d8/d10/d28/d39/d75/d50/f8e [0,4194304] 0 2024-03-22T06:15:04.170 INFO:tasks.workunit.client.0.smithi006.stdout:9/516: write d8/d10/d20/d45/d4d/f7a [4680913,122540] 0 2024-03-22T06:15:04.180 INFO:tasks.workunit.client.0.smithi006.stdout:3/649: symlink d0/d8/d88/ldb 0 2024-03-22T06:15:04.180 INFO:tasks.workunit.client.0.smithi006.stdout:3/650: write d0/d8/d1c/d3e/d5f/f7d [4213718,7899] 0 2024-03-22T06:15:04.180 INFO:tasks.workunit.client.0.smithi006.stdout:3/651: truncate d0/d8/d1c/d72/f94 886304 0 2024-03-22T06:15:04.180 INFO:tasks.workunit.client.0.smithi006.stdout:3/652: fdatasync d0/d8/d1c/d49/f67 0 2024-03-22T06:15:04.181 INFO:tasks.workunit.client.0.smithi006.stdout:1/557: write d8/d9/d10/f4e [763512,100075] 0 2024-03-22T06:15:04.186 INFO:tasks.workunit.client.0.smithi006.stdout:4/686: symlink d6/d1b/d20/d21/d9f/lee 0 2024-03-22T06:15:04.186 INFO:tasks.workunit.client.0.smithi006.stdout:4/687: dread - d6/d1b/d46/feb zero size 2024-03-22T06:15:04.188 INFO:tasks.workunit.client.0.smithi006.stdout:3/653: write d0/d8/d1c/d3e/f96 [3325354,91005] 0 2024-03-22T06:15:04.189 INFO:tasks.workunit.client.0.smithi006.stdout:2/709: link d13/d34/d4d/d4e/d75/d62/fb6 d13/d34/d6e/d72/deb/f10c 0 2024-03-22T06:15:04.189 INFO:tasks.workunit.client.0.smithi006.stdout:3/654: write d0/d8/d9/db1/fbc [26435,49303] 0 2024-03-22T06:15:04.192 INFO:tasks.workunit.client.0.smithi006.stdout:7/707: symlink d5/d1e/d63/d85/da1/le3 0 2024-03-22T06:15:04.192 INFO:tasks.workunit.client.0.smithi006.stdout:7/708: write d5/d1e/d63/d85/d80/fc4 [293653,51575] 0 2024-03-22T06:15:04.203 INFO:tasks.workunit.client.0.smithi006.stdout:9/517: rename d8/d10/d28/d39/d75/d50/f5e to d8/d10/da6/fab 0 2024-03-22T06:15:04.203 INFO:tasks.workunit.client.0.smithi006.stdout:2/710: dwrite d13/d34/f60 [4194304,4194304] 0 2024-03-22T06:15:04.204 INFO:tasks.workunit.client.0.smithi006.stdout:2/711: fdatasync d13/d34/d4d/d4e/d75/da5/d6c/f78 0 2024-03-22T06:15:04.208 INFO:tasks.workunit.client.0.smithi006.stdout:1/558: mkdir d8/d9/d13/d24/d32/dc0 0 2024-03-22T06:15:04.208 INFO:tasks.workunit.client.0.smithi006.stdout:1/559: stat d8/d9/d10/d7c/f82 0 2024-03-22T06:15:04.210 INFO:tasks.workunit.client.0.smithi006.stdout:4/688: mknod d6/d1b/d20/d21/d2a/cef 0 2024-03-22T06:15:04.210 INFO:tasks.workunit.client.0.smithi006.stdout:4/689: creat d6/d1b/d20/ff0 x:0 0 0 2024-03-22T06:15:04.215 INFO:tasks.workunit.client.0.smithi006.stdout:3/655: symlink d0/d23/d4c/ldc 0 2024-03-22T06:15:04.224 INFO:tasks.workunit.client.0.smithi006.stdout:8/765: rmdir d9/db/d35/dd2/d76/db3/dc3/d89 39 2024-03-22T06:15:04.228 INFO:tasks.workunit.client.0.smithi006.stdout:9/518: rmdir d8/d10/d20/d8d 0 2024-03-22T06:15:04.231 INFO:tasks.workunit.client.0.smithi006.stdout:3/656: dwrite d0/d8/d9/f32 [0,4194304] 0 2024-03-22T06:15:04.233 INFO:tasks.workunit.client.0.smithi006.stdout:2/712: mknod d13/d20/d35/c10d 0 2024-03-22T06:15:04.233 INFO:tasks.workunit.client.0.smithi006.stdout:2/713: dread - d13/d34/fe8 zero size 2024-03-22T06:15:04.233 INFO:tasks.workunit.client.0.smithi006.stdout:2/714: readlink d13/d20/d26/l8e 0 2024-03-22T06:15:04.233 INFO:tasks.workunit.client.0.smithi006.stdout:2/715: dread - d13/d20/d26/fab zero size 2024-03-22T06:15:04.233 INFO:tasks.workunit.client.0.smithi006.stdout:8/766: dread d9/db/d35/fbb [0,4194304] 0 2024-03-22T06:15:04.236 INFO:tasks.workunit.client.0.smithi006.stdout:1/560: symlink d8/d33/lc1 0 2024-03-22T06:15:04.236 INFO:tasks.workunit.client.0.smithi006.stdout:8/767: dread d9/d14/d47/d94/da3/fb2 [0,4194304] 0 2024-03-22T06:15:04.240 INFO:tasks.workunit.client.0.smithi006.stdout:4/690: stat d6/d1b/d20/ld7 0 2024-03-22T06:15:04.240 INFO:tasks.workunit.client.0.smithi006.stdout:4/691: dread - d6/f62 zero size 2024-03-22T06:15:04.241 INFO:tasks.workunit.client.0.smithi006.stdout:7/709: getdents d5/d1e/d29/d1f/d2d/d61 0 2024-03-22T06:15:04.248 INFO:tasks.workunit.client.0.smithi006.stdout:9/519: dwrite d8/d10/f12 [0,4194304] 0 2024-03-22T06:15:04.248 INFO:tasks.workunit.client.0.smithi006.stdout:3/657: unlink d0/d8/d1c/d3e/d68/f6d 0 2024-03-22T06:15:04.250 INFO:tasks.workunit.client.0.smithi006.stdout:8/768: dread d9/d14/f34 [0,4194304] 0 2024-03-22T06:15:04.250 INFO:tasks.workunit.client.0.smithi006.stdout:8/769: dread - d9/db/dc/de/d3e/fe1 zero size 2024-03-22T06:15:04.253 INFO:tasks.workunit.client.0.smithi006.stdout:1/561: symlink d8/d9/d3a/d40/d44/d80/lc2 0 2024-03-22T06:15:04.253 INFO:tasks.workunit.client.0.smithi006.stdout:1/562: write d8/d9/d13/d24/d32/d34/f36 [4053816,119551] 0 2024-03-22T06:15:04.253 INFO:tasks.workunit.client.0.smithi006.stdout:1/563: chown d8/d9/d13/d39/c46 39991413 1 2024-03-22T06:15:04.253 INFO:tasks.workunit.client.0.smithi006.stdout:1/564: chown d8/d9/d13/d24/d32/d34/f77 41136 1 2024-03-22T06:15:04.253 INFO:tasks.workunit.client.0.smithi006.stdout:1/565: write d8/d9/d13/d24/fa3 [1391396,36677] 0 2024-03-22T06:15:04.257 INFO:tasks.workunit.client.0.smithi006.stdout:7/710: mkdir d5/d1e/d29/d1f/de4 0 2024-03-22T06:15:04.263 INFO:tasks.workunit.client.0.smithi006.stdout:9/520: rename d8/d1f/c8a to d8/d10/da6/cac 0 2024-03-22T06:15:04.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:15:04.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:15:04.264 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:15:04.265 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: pgmap v15: 65 pgs: 65 active+clean; 1.0 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 97 MiB/s rd, 119 MiB/s wr, 189 op/s 2024-03-22T06:15:04.265 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:15:04.265 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:15:04.265 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:15:04.265 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.265 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.265 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.266 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.266 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.266 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.266 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.266 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.266 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:04 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.278 INFO:tasks.workunit.client.0.smithi006.stdout:8/770: mknod d9/db/d35/dd2/dc4/d104/c11f 0 2024-03-22T06:15:04.278 INFO:tasks.workunit.client.0.smithi006.stdout:8/771: chown d9/db/d12/d66/d6c/d80/fdf 2036534976 1 2024-03-22T06:15:04.279 INFO:tasks.workunit.client.0.smithi006.stdout:8/772: readlink d9/db/dc/leb 0 2024-03-22T06:15:04.279 INFO:tasks.workunit.client.0.smithi006.stdout:8/773: write d9/db/d35/da9/df6/f105 [1627463,128114] 0 2024-03-22T06:15:04.279 INFO:tasks.workunit.client.0.smithi006.stdout:8/774: chown d9/d14/d47/d94/da3/cbf 2313231 1 2024-03-22T06:15:04.280 INFO:tasks.workunit.client.0.smithi006.stdout:2/716: rmdir d13/d69 39 2024-03-22T06:15:04.286 INFO:tasks.workunit.client.0.smithi006.stdout:1/566: creat d8/d9/d3a/d18/fc3 x:0 0 0 2024-03-22T06:15:04.287 INFO:tasks.workunit.client.0.smithi006.stdout:1/567: chown d8/d9/d3a/d40/d44/f53 73 1 2024-03-22T06:15:04.289 INFO:tasks.workunit.client.0.smithi006.stdout:4/692: mkdir d6/d1b/d20/d21/d77/df1 0 2024-03-22T06:15:04.289 INFO:tasks.workunit.client.0.smithi006.stdout:4/693: write d6/d1b/d20/d6b/d88/fbf [658416,115789] 0 2024-03-22T06:15:04.289 INFO:tasks.workunit.client.0.smithi006.stdout:4/694: write d6/d1b/d20/d6b/f64 [4742485,40518] 0 2024-03-22T06:15:04.292 INFO:tasks.workunit.client.0.smithi006.stdout:7/711: rename d5/d1e/d29/d59/c8d to d5/d1e/d29/d1f/d2d/d55/d84/ce5 0 2024-03-22T06:15:04.292 INFO:tasks.workunit.client.0.smithi006.stdout:7/712: creat d5/d1e/d63/d85/fe6 x:0 0 0 2024-03-22T06:15:04.292 INFO:tasks.workunit.client.0.smithi006.stdout:1/568: write d8/d9/d13/d39/f63 [437307,73613] 0 2024-03-22T06:15:04.294 INFO:tasks.workunit.client.0.smithi006.stdout:9/521: mknod d8/d1f/cad 0 2024-03-22T06:15:04.294 INFO:tasks.workunit.client.0.smithi006.stdout:9/522: fsync d8/d1f/f68 0 2024-03-22T06:15:04.297 INFO:tasks.workunit.client.0.smithi006.stdout:3/658: getdents d0/d23/d33/d3d 0 2024-03-22T06:15:04.301 INFO:tasks.workunit.client.0.smithi006.stdout:1/569: dread d8/d9/d3a/d40/d44/f76 [0,4194304] 0 2024-03-22T06:15:04.304 INFO:tasks.workunit.client.0.smithi006.stdout:9/523: write d8/d10/d28/d39/f46 [8172794,124204] 0 2024-03-22T06:15:04.305 INFO:tasks.workunit.client.0.smithi006.stdout:3/659: dread d0/d8/d1c/f4f [4194304,4194304] 0 2024-03-22T06:15:04.307 INFO:tasks.workunit.client.0.smithi006.stdout:8/775: creat d9/db/d35/dd2/d76/db3/dc3/d89/db8/f120 x:0 0 0 2024-03-22T06:15:04.307 INFO:tasks.workunit.client.0.smithi006.stdout:8/776: fdatasync d9/db/dc/de/d3e/f73 0 2024-03-22T06:15:04.307 INFO:tasks.workunit.client.0.smithi006.stdout:8/777: fdatasync d9/db/d12/d66/f9b 0 2024-03-22T06:15:04.308 INFO:tasks.workunit.client.0.smithi006.stdout:2/717: mkdir d13/d53/dc5/d10e 0 2024-03-22T06:15:04.316 INFO:tasks.workunit.client.0.smithi006.stdout:7/713: truncate d5/d1e/d29/f5d 2580460 0 2024-03-22T06:15:04.316 INFO:tasks.workunit.client.0.smithi006.stdout:7/714: fdatasync d5/d1e/d29/d1f/d2d/d55/f8c 0 2024-03-22T06:15:04.317 INFO:tasks.workunit.client.0.smithi006.stdout:1/570: symlink d8/d33/lc4 0 2024-03-22T06:15:04.324 INFO:tasks.workunit.client.0.smithi006.stdout:9/524: unlink d8/d10/d28/d39/d75/d50/f76 0 2024-03-22T06:15:04.326 INFO:tasks.workunit.client.0.smithi006.stdout:9/525: fsync d8/d10/d23/f7d 0 2024-03-22T06:15:04.327 INFO:tasks.workunit.client.0.smithi006.stdout:9/526: creat d8/d10/d23/d3e/fae x:0 0 0 2024-03-22T06:15:04.328 INFO:tasks.workunit.client.0.smithi006.stdout:6/610: sync 2024-03-22T06:15:04.328 INFO:tasks.workunit.client.0.smithi006.stdout:5/783: sync 2024-03-22T06:15:04.328 INFO:tasks.workunit.client.0.smithi006.stdout:0/480: sync 2024-03-22T06:15:04.328 INFO:tasks.workunit.client.0.smithi006.stdout:0/481: getdents db/d10/d35/d7b/da1/da6 0 2024-03-22T06:15:04.329 INFO:tasks.workunit.client.0.smithi006.stdout:3/660: mkdir d0/d8/d88/ddd 0 2024-03-22T06:15:04.329 INFO:tasks.workunit.client.0.smithi006.stdout:3/661: creat d0/d4/d3a/fde x:0 0 0 2024-03-22T06:15:04.329 INFO:tasks.workunit.client.0.smithi006.stdout:3/662: readlink d0/d8/d1c/d3e/d5f/l4a 0 2024-03-22T06:15:04.330 INFO:tasks.workunit.client.0.smithi006.stdout:8/778: creat d9/d14/f121 x:0 0 0 2024-03-22T06:15:04.331 INFO:tasks.workunit.client.0.smithi006.stdout:3/663: write d0/d8/d1c/d72/f94 [334635,24203] 0 2024-03-22T06:15:04.333 INFO:tasks.workunit.client.0.smithi006.stdout:2/718: rmdir d13/d34/d4d/d4e/d75/d62 39 2024-03-22T06:15:04.333 INFO:tasks.workunit.client.0.smithi006.stdout:2/719: chown d13/d34/c85 15319200 1 2024-03-22T06:15:04.333 INFO:tasks.workunit.client.0.smithi006.stdout:2/720: write d13/d34/d4d/d4e/d75/da5/d82/fae [869685,74124] 0 2024-03-22T06:15:04.333 INFO:tasks.workunit.client.0.smithi006.stdout:2/721: chown d13/d34/d4d/d4e/d75 115 1 2024-03-22T06:15:04.336 INFO:tasks.workunit.client.0.smithi006.stdout:4/695: mkdir d6/d1b/d20/d21/d77/df2 0 2024-03-22T06:15:04.336 INFO:tasks.workunit.client.0.smithi006.stdout:4/696: fdatasync d6/d1b/d20/d21/d2a/d43/d4b/fa7 0 2024-03-22T06:15:04.339 INFO:tasks.workunit.client.0.smithi006.stdout:7/715: mknod d5/d1e/d63/d85/d80/ce7 0 2024-03-22T06:15:04.341 INFO:tasks.workunit.client.0.smithi006.stdout:1/571: rename d8/d9/d13/d39/c6a to d8/d9/d3a/d40/d59/db1/cc5 0 2024-03-22T06:15:04.341 INFO:tasks.workunit.client.0.smithi006.stdout:1/572: stat d8/d9/d13/d24/d32/d34/f99 0 2024-03-22T06:15:04.341 INFO:tasks.workunit.client.0.smithi006.stdout:1/573: creat d8/d9/d3a/d18/d2a/fc6 x:0 0 0 2024-03-22T06:15:04.345 INFO:tasks.workunit.client.0.smithi006.stdout:9/527: dwrite d8/f7c [0,4194304] 0 2024-03-22T06:15:04.347 INFO:tasks.workunit.client.0.smithi006.stdout:8/779: mkdir d9/db/d12/d66/d10c/d122 0 2024-03-22T06:15:04.348 INFO:tasks.workunit.client.0.smithi006.stdout:8/780: creat d9/db/d35/dd2/d76/f123 x:0 0 0 2024-03-22T06:15:04.348 INFO:tasks.workunit.client.0.smithi006.stdout:8/781: dread d9/d14/d47/d94/da3/fda [0,4194304] 0 2024-03-22T06:15:04.351 INFO:tasks.workunit.client.0.smithi006.stdout:3/664: mkdir d0/d8/d88/ddf 0 2024-03-22T06:15:04.355 INFO:tasks.workunit.client.0.smithi006.stdout:2/722: rmdir d13/d34/d4d/d4e/dd9 0 2024-03-22T06:15:04.361 INFO:tasks.workunit.client.0.smithi006.stdout:4/697: creat d6/d1b/d20/d6b/d95/ff3 x:0 0 0 2024-03-22T06:15:04.362 INFO:tasks.workunit.client.0.smithi006.stdout:7/716: creat d5/d1e/d29/d1f/d2d/d55/fe8 x:0 0 0 2024-03-22T06:15:04.364 INFO:tasks.workunit.client.0.smithi006.stdout:3/665: write d0/d8/d1c/d3e/d5f/da3/f47 [1396621,51289] 0 2024-03-22T06:15:04.367 INFO:tasks.workunit.client.0.smithi006.stdout:1/574: dwrite d8/d9/d3a/d40/d44/d80/fab [0,4194304] 0 2024-03-22T06:15:04.367 INFO:tasks.workunit.client.0.smithi006.stdout:6/611: dwrite d6/d7c/d92/faa [0,4194304] 0 2024-03-22T06:15:04.367 INFO:tasks.workunit.client.0.smithi006.stdout:5/784: rename d0/f48 to d0/db/d16/d1c/d34/d35/d85/dda/fee 0 2024-03-22T06:15:04.368 INFO:tasks.workunit.client.0.smithi006.stdout:5/785: dread d0/db/d16/d1c/d34/d35/d85/f55 [0,4194304] 0 2024-03-22T06:15:04.368 INFO:tasks.workunit.client.0.smithi006.stdout:5/786: creat d0/db/d50/d96/d52/d69/fef x:0 0 0 2024-03-22T06:15:04.375 INFO:tasks.workunit.client.0.smithi006.stdout:8/782: truncate d9/db/d35/dd2/dad/fb7 1504866 0 2024-03-22T06:15:04.378 INFO:tasks.workunit.client.0.smithi006.stdout:4/698: truncate d6/d1b/d20/d21/d2a/dc9/fcb 6107886 0 2024-03-22T06:15:04.378 INFO:tasks.workunit.client.0.smithi006.stdout:4/699: fdatasync d6/d1b/d20/f22 0 2024-03-22T06:15:04.378 INFO:tasks.workunit.client.0.smithi006.stdout:4/700: truncate d6/d1b/d20/d63/d93/d9a/d8f/da1/fb6 160649 0 2024-03-22T06:15:04.379 INFO:tasks.workunit.client.0.smithi006.stdout:4/701: read d6/d1b/d20/d63/d93/d9a/d8f/da1/db5/f9b [544155,67624] 0 2024-03-22T06:15:04.379 INFO:tasks.workunit.client.0.smithi006.stdout:4/702: chown d6/d1b/d20/d6b/d88/fbf 3546946 1 2024-03-22T06:15:04.379 INFO:tasks.workunit.client.0.smithi006.stdout:4/703: creat d6/ff4 x:0 0 0 2024-03-22T06:15:04.383 INFO:tasks.workunit.client.0.smithi006.stdout:3/666: rmdir d0/d8/d1c/d49/db5 0 2024-03-22T06:15:04.386 INFO:tasks.workunit.client.0.smithi006.stdout:5/787: dwrite d0/db/d99/f3f [0,4194304] 0 2024-03-22T06:15:04.396 INFO:tasks.workunit.client.0.smithi006.stdout:6/612: mkdir d6/d1d/d5c/dc4 0 2024-03-22T06:15:04.396 INFO:tasks.workunit.client.0.smithi006.stdout:6/613: stat d6/d1d/d99/ca3 0 2024-03-22T06:15:04.398 INFO:tasks.workunit.client.0.smithi006.stdout:9/528: rename d8/f9 to d8/d10/d28/d39/d75/d50/d38/faf 0 2024-03-22T06:15:04.402 INFO:tasks.workunit.client.0.smithi006.stdout:5/788: dwrite d0/db/f1e [4194304,4194304] 0 2024-03-22T06:15:04.402 INFO:tasks.workunit.client.0.smithi006.stdout:8/783: creat d9/db/d35/dd2/dc4/d104/f124 x:0 0 0 2024-03-22T06:15:04.405 INFO:tasks.workunit.client.0.smithi006.stdout:5/789: write d0/db/d50/f80 [233535,56507] 0 2024-03-22T06:15:04.407 INFO:tasks.workunit.client.0.smithi006.stdout:4/704: link d6/d1b/d20/d6b/f75 d6/d1b/d20/d63/d93/ff5 0 2024-03-22T06:15:04.409 INFO:tasks.workunit.client.0.smithi006.stdout:7/717: getdents d5/d1e/d29/d1f/d2d/d41 0 2024-03-22T06:15:04.413 INFO:tasks.workunit.client.0.smithi006.stdout:3/667: mkdir d0/d4/d53/de0 0 2024-03-22T06:15:04.416 INFO:tasks.workunit.client.0.smithi006.stdout:4/705: dread d6/f30 [0,4194304] 0 2024-03-22T06:15:04.423 INFO:tasks.workunit.client.0.smithi006.stdout:6/614: mknod d6/d1d/d64/cc5 0 2024-03-22T06:15:04.423 INFO:tasks.workunit.client.0.smithi006.stdout:6/615: dread - d6/d6f/f9e zero size 2024-03-22T06:15:04.427 INFO:tasks.workunit.client.0.smithi006.stdout:5/790: dwrite d0/db/d50/db9/de1/f79 [0,4194304] 0 2024-03-22T06:15:04.434 INFO:tasks.workunit.client.0.smithi006.stdout:7/718: truncate d5/d1e/d3f/f76 1235952 0 2024-03-22T06:15:04.437 INFO:tasks.workunit.client.0.smithi006.stdout:3/668: mknod d0/d8/d1c/ce1 0 2024-03-22T06:15:04.441 INFO:tasks.workunit.client.0.smithi006.stdout:2/723: rename d13/d34/d6e/d72 to d13/d69/d10f 0 2024-03-22T06:15:04.446 INFO:tasks.workunit.client.0.smithi006.stdout:7/719: symlink d5/d1e/d29/d1f/d2d/d61/le9 0 2024-03-22T06:15:04.450 INFO:tasks.workunit.client.0.smithi006.stdout:3/669: read d0/d8/f17 [2892337,101904] 0 2024-03-22T06:15:04.452 INFO:tasks.workunit.client.0.smithi006.stdout:2/724: mknod d13/c110 0 2024-03-22T06:15:04.455 INFO:tasks.workunit.client.0.smithi006.stdout:7/720: symlink d5/d1e/d3f/lea 0 2024-03-22T06:15:04.455 INFO:tasks.workunit.client.0.smithi006.stdout:7/721: dread - d5/d1e/d3f/fc8 zero size 2024-03-22T06:15:04.458 INFO:tasks.workunit.client.0.smithi006.stdout:6/616: dread d6/d1b/f2a [0,4194304] 0 2024-03-22T06:15:04.459 INFO:tasks.workunit.client.0.smithi006.stdout:5/791: dwrite d0/db/d99/f3f [0,4194304] 0 2024-03-22T06:15:04.459 INFO:tasks.workunit.client.0.smithi006.stdout:5/792: chown d0/db/dc/f8f 795 1 2024-03-22T06:15:04.459 INFO:tasks.workunit.client.0.smithi006.stdout:5/793: write d0/db/d16/d43/f67 [523839,99592] 0 2024-03-22T06:15:04.460 INFO:tasks.workunit.client.0.smithi006.stdout:3/670: mkdir d0/d23/d33/d3d/de2 0 2024-03-22T06:15:04.464 INFO:tasks.workunit.client.0.smithi006.stdout:6/617: truncate d6/d7c/d83/fa2 2823771 0 2024-03-22T06:15:04.464 INFO:tasks.workunit.client.0.smithi006.stdout:6/618: read d6/d1b/f2d [2178270,52134] 0 2024-03-22T06:15:04.467 INFO:tasks.workunit.client.0.smithi006.stdout:1/575: rename d8/d9/d3a/d18/d2a to d8/d9/d13/d24/d32/dc7 0 2024-03-22T06:15:04.471 INFO:tasks.workunit.client.0.smithi006.stdout:3/671: truncate d0/d8/d1c/d3e/d68/f6b 415294 0 2024-03-22T06:15:04.471 INFO:tasks.workunit.client.0.smithi006.stdout:3/672: read d0/d23/f25 [684123,21199] 0 2024-03-22T06:15:04.471 INFO:tasks.workunit.client.0.smithi006.stdout:3/673: dread - d0/d8/d1c/fd1 zero size 2024-03-22T06:15:04.471 INFO:tasks.workunit.client.0.smithi006.stdout:1/576: stat d8/d9/c4f 0 2024-03-22T06:15:04.475 INFO:tasks.workunit.client.0.smithi006.stdout:9/529: rename d8/d10/d23/d3e/l95 to d8/d1f/d34/lb0 0 2024-03-22T06:15:04.477 INFO:tasks.workunit.client.0.smithi006.stdout:3/674: rmdir d0/d8/d1c/d3e/d5f/d90 39 2024-03-22T06:15:04.477 INFO:tasks.workunit.client.0.smithi006.stdout:3/675: chown d0/d8/d9/l5a 636504 1 2024-03-22T06:15:04.480 INFO:tasks.workunit.client.0.smithi006.stdout:1/577: dread d8/d9/d3a/d40/d44/d80/fab [0,4194304] 0 2024-03-22T06:15:04.481 INFO:tasks.workunit.client.0.smithi006.stdout:1/578: truncate d8/d9/d3a/d40/da1/fbf 904972 0 2024-03-22T06:15:04.482 INFO:tasks.workunit.client.0.smithi006.stdout:9/530: dread f4 [0,4194304] 0 2024-03-22T06:15:04.482 INFO:tasks.workunit.client.0.smithi006.stdout:3/676: symlink d0/d23/d33/d3d/dbd/le3 0 2024-03-22T06:15:04.482 INFO:tasks.workunit.client.0.smithi006.stdout:3/677: dread - d0/d23/d33/d3d/fb9 zero size 2024-03-22T06:15:04.483 INFO:tasks.workunit.client.0.smithi006.stdout:8/784: rename d9/d14/d47/d94 to d9/db/d12/d66/d10c/d125 0 2024-03-22T06:15:04.483 INFO:tasks.workunit.client.0.smithi006.stdout:8/785: truncate d9/db/d35/dd2/d4a/f96 4603933 0 2024-03-22T06:15:04.484 INFO:tasks.workunit.client.0.smithi006.stdout:8/786: fsync d9/db/d12/f5d 0 2024-03-22T06:15:04.486 INFO:tasks.workunit.client.0.smithi006.stdout:1/579: truncate d8/d9/d13/f6d 4301974 0 2024-03-22T06:15:04.487 INFO:tasks.workunit.client.0.smithi006.stdout:8/787: write d9/db/d35/dd2/d76/db3/dc3/d89/fd4 [10960,4457] 0 2024-03-22T06:15:04.488 INFO:tasks.workunit.client.0.smithi006.stdout:9/531: symlink d8/d10/d28/d39/d75/lb1 0 2024-03-22T06:15:04.488 INFO:tasks.workunit.client.0.smithi006.stdout:9/532: fsync d8/d10/d28/f3d 0 2024-03-22T06:15:04.491 INFO:tasks.workunit.client.0.smithi006.stdout:4/706: rename d6/d1b/d20/d21/d77/da0/lb2 to d6/d1b/d20/d21/d77/df2/lf6 0 2024-03-22T06:15:04.491 INFO:tasks.workunit.client.0.smithi006.stdout:4/707: chown d6/lde 230318 1 2024-03-22T06:15:04.494 INFO:tasks.workunit.client.0.smithi006.stdout:1/580: creat d8/d9/d13/d24/fc8 x:0 0 0 2024-03-22T06:15:04.499 INFO:tasks.workunit.client.0.smithi006.stdout:3/678: dwrite d0/d23/f7f [0,4194304] 0 2024-03-22T06:15:04.499 INFO:tasks.workunit.client.0.smithi006.stdout:7/722: rename d5/d1e/d29/d1f/d2d/f37 to d5/d1e/d63/d85/d80/feb 0 2024-03-22T06:15:04.503 INFO:tasks.workunit.client.0.smithi006.stdout:4/708: chown d6/d1b/d20/d21/d77/l86 12268 1 2024-03-22T06:15:04.503 INFO:tasks.workunit.client.0.smithi006.stdout:4/709: dread - d6/d1b/d20/d6b/d88/fe5 zero size 2024-03-22T06:15:04.503 INFO:tasks.workunit.client.0.smithi006.stdout:1/581: truncate d8/d9/f1e 208651 0 2024-03-22T06:15:04.505 INFO:tasks.workunit.client.0.smithi006.stdout:8/788: getdents d9/db/d12 0 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: pgmap v15: 65 pgs: 65 active+clean; 1.0 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 97 MiB/s rd, 119 MiB/s wr, 189 op/s 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:04 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:04.508 INFO:tasks.workunit.client.0.smithi006.stdout:3/679: unlink d0/d8/d1c/c5c 0 2024-03-22T06:15:04.508 INFO:tasks.workunit.client.0.smithi006.stdout:3/680: readlink d0/d8/d1c/d89/ld8 0 2024-03-22T06:15:04.511 INFO:tasks.workunit.client.0.smithi006.stdout:9/533: rename d8/d10/d20/d45/f7b to d8/d10/d47/fb2 0 2024-03-22T06:15:04.514 INFO:tasks.workunit.client.0.smithi006.stdout:4/710: getdents d6/d1b/d20/d6b 0 2024-03-22T06:15:04.516 INFO:tasks.workunit.client.0.smithi006.stdout:1/582: creat d8/d9/d13/d39/fc9 x:0 0 0 2024-03-22T06:15:04.517 INFO:tasks.workunit.client.0.smithi006.stdout:1/583: readlink d8/d9/d13/l8a 0 2024-03-22T06:15:04.519 INFO:tasks.workunit.client.0.smithi006.stdout:8/789: symlink d9/db/d35/dd2/d76/db3/dc3/l126 0 2024-03-22T06:15:04.519 INFO:tasks.workunit.client.0.smithi006.stdout:8/790: dread - d9/db/d35/dd2/dc4/f119 zero size 2024-03-22T06:15:04.520 INFO:tasks.workunit.client.0.smithi006.stdout:9/534: getdents d8/d10 0 2024-03-22T06:15:04.520 INFO:tasks.workunit.client.0.smithi006.stdout:9/535: fdatasync d8/d10/d28/d39/d75/d50/f8e 0 2024-03-22T06:15:04.520 INFO:tasks.workunit.client.0.smithi006.stdout:9/536: stat d8/d10/d11/c32 0 2024-03-22T06:15:04.520 INFO:tasks.workunit.client.0.smithi006.stdout:9/537: dread - d8/d10/d20/d45/d4d/f85 zero size 2024-03-22T06:15:04.521 INFO:tasks.workunit.client.0.smithi006.stdout:7/723: dwrite d5/d1e/d63/d85/d80/feb [0,4194304] 0 2024-03-22T06:15:04.521 INFO:tasks.workunit.client.0.smithi006.stdout:7/724: chown d5/d1e/d63/d85/d80/l8f 45 1 2024-03-22T06:15:04.521 INFO:tasks.workunit.client.0.smithi006.stdout:7/725: stat d5/d1e/d29/d59/d5c/c68 0 2024-03-22T06:15:04.521 INFO:tasks.workunit.client.0.smithi006.stdout:7/726: dread - d5/d1e/d3f/fc8 zero size 2024-03-22T06:15:04.524 INFO:tasks.workunit.client.0.smithi006.stdout:4/711: symlink d6/lf7 0 2024-03-22T06:15:04.524 INFO:tasks.workunit.client.0.smithi006.stdout:4/712: dread - d6/d1b/d20/d21/d77/da0/fed zero size 2024-03-22T06:15:04.526 INFO:tasks.workunit.client.0.smithi006.stdout:1/584: truncate d8/d9/d13/d24/d32/d34/f36 4564462 0 2024-03-22T06:15:04.527 INFO:tasks.workunit.client.0.smithi006.stdout:3/681: dwrite d0/d8/d1c/d3e/d5f/d90/da7/d97/fa5 [0,4194304] 0 2024-03-22T06:15:04.527 INFO:tasks.workunit.client.0.smithi006.stdout:3/682: chown d0/d8/d1c/d95/lcd 4673337 1 2024-03-22T06:15:04.530 INFO:tasks.workunit.client.0.smithi006.stdout:8/791: mknod d9/db/d35/dd2/dc4/ddd/de8/c127 0 2024-03-22T06:15:04.530 INFO:tasks.workunit.client.0.smithi006.stdout:8/792: write d9/d5e/fc2 [1245803,32659] 0 2024-03-22T06:15:04.530 INFO:tasks.workunit.client.0.smithi006.stdout:8/793: chown d9/db 8429 1 2024-03-22T06:15:04.533 INFO:tasks.workunit.client.0.smithi006.stdout:4/713: mknod d6/d1b/d20/d21/d2a/d43/da8/cf8 0 2024-03-22T06:15:04.533 INFO:tasks.workunit.client.0.smithi006.stdout:4/714: stat d6/d1b/d20 0 2024-03-22T06:15:04.536 INFO:tasks.workunit.client.0.smithi006.stdout:1/585: creat d8/d9/d13/d39/d94/db5/fca x:0 0 0 2024-03-22T06:15:04.536 INFO:tasks.workunit.client.0.smithi006.stdout:1/586: write d8/d9/d13/f26 [1059226,9917] 0 2024-03-22T06:15:04.542 INFO:tasks.workunit.client.0.smithi006.stdout:8/794: symlink d9/db/d35/l128 0 2024-03-22T06:15:04.543 INFO:tasks.workunit.client.0.smithi006.stdout:8/795: write d9/d14/dcc/f11c [903003,100914] 0 2024-03-22T06:15:04.543 INFO:tasks.workunit.client.0.smithi006.stdout:8/796: fdatasync d9/db/d12/d1e/f6d 0 2024-03-22T06:15:04.543 INFO:tasks.workunit.client.0.smithi006.stdout:8/797: chown d9/d14/f121 9 1 2024-03-22T06:15:04.543 INFO:tasks.workunit.client.0.smithi006.stdout:7/727: dwrite d5/d1e/d63/d85/d80/fc1 [0,4194304] 0 2024-03-22T06:15:04.543 INFO:tasks.workunit.client.0.smithi006.stdout:9/538: dwrite d8/d10/d28/d39/d75/d50/f84 [0,4194304] 0 2024-03-22T06:15:04.544 INFO:tasks.workunit.client.0.smithi006.stdout:9/539: dread - d8/d10/d23/d3e/fae zero size 2024-03-22T06:15:04.546 INFO:tasks.workunit.client.0.smithi006.stdout:4/715: rename d6/d1b/l91 to d6/d1b/d20/d21/d2a/d43/da8/lf9 0 2024-03-22T06:15:04.546 INFO:tasks.workunit.client.0.smithi006.stdout:4/716: getdents d6/d1b/d20/d6b/d7b 0 2024-03-22T06:15:04.552 INFO:tasks.workunit.client.0.smithi006.stdout:3/683: dwrite d0/d8/d1c/d3e/d5f/d90/da7/d97/fa5 [0,4194304] 0 2024-03-22T06:15:04.555 INFO:tasks.workunit.client.0.smithi006.stdout:1/587: unlink d8/d9/d13/d24/d32/dc7/c30 0 2024-03-22T06:15:04.557 INFO:tasks.workunit.client.0.smithi006.stdout:8/798: symlink d9/db/d12/d66/d10c/l129 0 2024-03-22T06:15:04.558 INFO:tasks.workunit.client.0.smithi006.stdout:8/799: readlink d9/db/d12/d66/d10c/d125/l107 0 2024-03-22T06:15:04.558 INFO:tasks.workunit.client.0.smithi006.stdout:8/800: chown f7 777264847 1 2024-03-22T06:15:04.568 INFO:tasks.workunit.client.0.smithi006.stdout:9/540: dwrite f1 [0,4194304] 0 2024-03-22T06:15:04.568 INFO:tasks.workunit.client.0.smithi006.stdout:4/717: creat d6/d1b/d20/d63/d93/d9a/d8f/da1/ffa x:0 0 0 2024-03-22T06:15:04.570 INFO:tasks.workunit.client.0.smithi006.stdout:1/588: mknod d8/d9/d13/d24/ccb 0 2024-03-22T06:15:04.576 INFO:tasks.workunit.client.0.smithi006.stdout:7/728: dwrite d5/d1e/d3f/f81 [0,4194304] 0 2024-03-22T06:15:04.579 INFO:tasks.workunit.client.0.smithi006.stdout:4/718: creat d6/d1b/d20/d21/d77/df1/ffb x:0 0 0 2024-03-22T06:15:04.588 INFO:tasks.workunit.client.0.smithi006.stdout:8/801: dwrite d9/d14/f34 [0,4194304] 0 2024-03-22T06:15:04.588 INFO:tasks.workunit.client.0.smithi006.stdout:8/802: dread - d9/db/d12/d66/d6c/d92/fb9 zero size 2024-03-22T06:15:04.591 INFO:tasks.workunit.client.0.smithi006.stdout:3/684: dwrite d0/d23/f25 [0,4194304] 0 2024-03-22T06:15:04.591 INFO:tasks.workunit.client.0.smithi006.stdout:7/729: dread d5/d1e/d29/d1f/d2d/f47 [0,4194304] 0 2024-03-22T06:15:04.591 INFO:tasks.workunit.client.0.smithi006.stdout:7/730: creat d5/d1e/d63/d85/fec x:0 0 0 2024-03-22T06:15:04.596 INFO:tasks.workunit.client.0.smithi006.stdout:1/589: symlink d8/d9/d13/d39/d60/lcc 0 2024-03-22T06:15:04.601 INFO:tasks.workunit.client.0.smithi006.stdout:9/541: rename d8/d10/d28/d39/d75 to d8/db3 0 2024-03-22T06:15:04.601 INFO:tasks.workunit.client.0.smithi006.stdout:9/542: fdatasync d8/d1f/d34/f3a 0 2024-03-22T06:15:04.604 INFO:tasks.workunit.client.0.smithi006.stdout:1/590: dread d8/d9/d13/f28 [0,4194304] 0 2024-03-22T06:15:04.604 INFO:tasks.workunit.client.0.smithi006.stdout:1/591: write d8/fac [850289,57749] 0 2024-03-22T06:15:04.604 INFO:tasks.workunit.client.0.smithi006.stdout:1/592: chown d8/d9/d3a/f92 22 1 2024-03-22T06:15:04.605 INFO:tasks.workunit.client.0.smithi006.stdout:8/803: creat d9/db/d12/f12a x:0 0 0 2024-03-22T06:15:04.605 INFO:tasks.workunit.client.0.smithi006.stdout:8/804: write d9/db/dc/de/d3e/f56 [1390443,59253] 0 2024-03-22T06:15:04.627 INFO:tasks.workunit.client.0.smithi006.stdout:4/719: rename d6/d1b/d20/d21/d2a/d43/da8/lf9 to d6/d1b/d20/d21/d2a/d5c/lfc 0 2024-03-22T06:15:04.635 INFO:tasks.workunit.client.0.smithi006.stdout:9/543: creat d8/d10/da6/fb4 x:0 0 0 2024-03-22T06:15:04.636 INFO:tasks.workunit.client.0.smithi006.stdout:9/544: getdents d8/d10 0 2024-03-22T06:15:04.639 INFO:tasks.workunit.client.0.smithi006.stdout:1/593: truncate d8/d9/d3a/d40/d44/f66 38671 0 2024-03-22T06:15:04.643 INFO:tasks.workunit.client.0.smithi006.stdout:7/731: getdents d5/d1e/d29/d59/d5c 0 2024-03-22T06:15:04.643 INFO:tasks.workunit.client.0.smithi006.stdout:7/732: stat d5/d1e/d29/d10/f25 0 2024-03-22T06:15:04.643 INFO:tasks.workunit.client.0.smithi006.stdout:7/733: readlink d5/l64 0 2024-03-22T06:15:04.650 INFO:tasks.workunit.client.0.smithi006.stdout:4/720: truncate d6/f4a 49867 0 2024-03-22T06:15:04.653 INFO:tasks.workunit.client.0.smithi006.stdout:3/685: rename d0/d23/d4c/f7b to d0/d23/fe4 0 2024-03-22T06:15:04.653 INFO:tasks.workunit.client.0.smithi006.stdout:3/686: chown d0/d8/d1c/d3e/d5f/da3 0 1 2024-03-22T06:15:04.658 INFO:tasks.workunit.client.0.smithi006.stdout:9/545: symlink d8/lb5 0 2024-03-22T06:15:04.662 INFO:tasks.workunit.client.0.smithi006.stdout:7/734: symlink d5/d1e/d63/d85/d80/led 0 2024-03-22T06:15:04.662 INFO:tasks.workunit.client.0.smithi006.stdout:7/735: dread - d5/d1e/d3f/fc8 zero size 2024-03-22T06:15:04.713 INFO:tasks.workunit.client.0.smithi006.stdout:8/805: dwrite d9/db/d35/dd2/d4a/d4d/f75 [0,4194304] 0 2024-03-22T06:15:04.713 INFO:tasks.workunit.client.0.smithi006.stdout:8/806: creat d9/db/d12/d66/d6c/d80/df0/f12b x:0 0 0 2024-03-22T06:15:04.735 INFO:tasks.workunit.client.0.smithi006.stdout:4/721: unlink d6/d1b/d20/d21/d2a/d43/d4b/l80 0 2024-03-22T06:15:04.735 INFO:tasks.workunit.client.0.smithi006.stdout:4/722: truncate d6/d1b/d20/d21/d77/da0/fed 1013032 0 2024-03-22T06:15:04.741 INFO:tasks.workunit.client.0.smithi006.stdout:3/687: mkdir d0/d8/d1c/d49/de5 0 2024-03-22T06:15:04.743 INFO:tasks.workunit.client.0.smithi006.stdout:7/736: mknod d5/d1e/d29/db9/cee 0 2024-03-22T06:15:04.753 INFO:tasks.workunit.client.0.smithi006.stdout:1/594: write d8/d9/f54 [2911405,87911] 0 2024-03-22T06:15:04.855 INFO:tasks.workunit.client.0.smithi006.stdout:4/723: dwrite d6/f42 [4194304,4194304] 0 2024-03-22T06:15:04.863 INFO:tasks.workunit.client.0.smithi006.stdout:0/482: sync 2024-03-22T06:15:04.863 INFO:tasks.workunit.client.0.smithi006.stdout:0/483: truncate db/dc/d38/d50/f79 348601 0 2024-03-22T06:15:04.886 INFO:tasks.workunit.client.0.smithi006.stdout:3/688: unlink d0/d8/d88/l9c 0 2024-03-22T06:15:04.887 INFO:tasks.workunit.client.0.smithi006.stdout:3/689: chown d0/d8/d88/ddd 1905294 1 2024-03-22T06:15:04.894 INFO:tasks.workunit.client.0.smithi006.stdout:1/595: mkdir d8/d71/dcd 0 2024-03-22T06:15:04.908 INFO:tasks.workunit.client.0.smithi006.stdout:3/690: creat d0/d23/d33/d3d/dcb/fe6 x:0 0 0 2024-03-22T06:15:04.937 INFO:tasks.workunit.client.0.smithi006.stdout:9/546: dwrite d8/d10/d47/f97 [0,4194304] 0 2024-03-22T06:15:05.099 INFO:tasks.workunit.client.0.smithi006.stdout:8/807: dwrite d9/d14/d44/f8e [0,4194304] 0 2024-03-22T06:15:05.108 INFO:tasks.workunit.client.0.smithi006.stdout:1/596: dwrite d8/d9/d13/d24/d32/dc7/f74 [0,4194304] 0 2024-03-22T06:15:05.122 INFO:tasks.workunit.client.0.smithi006.stdout:1/597: dread d8/d9/f35 [0,4194304] 0 2024-03-22T06:15:05.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:05 smithi117 ceph-mon[36514]: Reconfiguring node-exporter.smithi006 (dependencies changed)... 2024-03-22T06:15:05.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:05 smithi117 ceph-mon[36514]: Reconfiguring daemon node-exporter.smithi006 on smithi006 2024-03-22T06:15:05.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:05 smithi006 ceph-mon[29301]: Reconfiguring node-exporter.smithi006 (dependencies changed)... 2024-03-22T06:15:05.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:05 smithi006 ceph-mon[29301]: Reconfiguring daemon node-exporter.smithi006 on smithi006 2024-03-22T06:15:05.292 INFO:tasks.workunit.client.0.smithi006.stdout:1/598: mknod d8/d9/d13/d24/d32/d34/daa/cce 0 2024-03-22T06:15:05.331 INFO:tasks.workunit.client.0.smithi006.stdout:1/599: creat d8/d9/d13/d24/d32/fcf x:0 0 0 2024-03-22T06:15:05.331 INFO:tasks.workunit.client.0.smithi006.stdout:1/600: chown d8/d9/c4f 269855 1 2024-03-22T06:15:05.754 INFO:tasks.workunit.client.0.smithi006.stdout:8/808: dwrite d9/db/f7b [0,4194304] 0 2024-03-22T06:15:05.810 INFO:tasks.workunit.client.0.smithi006.stdout:8/809: dwrite d9/db/d35/dd2/d4a/da7/fef [0,4194304] 0 2024-03-22T06:15:05.810 INFO:tasks.workunit.client.0.smithi006.stdout:8/810: chown d9/db/fed 1 1 2024-03-22T06:15:05.904 INFO:tasks.workunit.client.0.smithi006.stdout:8/811: dwrite f2 [0,4194304] 0 2024-03-22T06:15:05.910 INFO:tasks.workunit.client.0.smithi006.stdout:8/812: unlink d9/db/dc/de/l6a 0 2024-03-22T06:15:05.910 INFO:tasks.workunit.client.0.smithi006.stdout:8/813: chown d9/db/d12/d1e 89288157 1 2024-03-22T06:15:05.913 INFO:tasks.workunit.client.0.smithi006.stdout:8/814: mknod d9/db/d35/dd2/d76/dca/dee/c12c 0 2024-03-22T06:15:05.981 INFO:tasks.workunit.client.0.smithi006.stdout:8/815: dwrite d9/db/d35/dd2/d4a/d4d/f8d [0,4194304] 0 2024-03-22T06:15:05.981 INFO:tasks.workunit.client.0.smithi006.stdout:8/816: read - d9/db/d12/d66/d6c/d71/f93 zero size 2024-03-22T06:15:05.981 INFO:tasks.workunit.client.0.smithi006.stdout:8/817: truncate d9/db/d12/d1e/f51 724401 0 2024-03-22T06:15:05.981 INFO:tasks.workunit.client.0.smithi006.stdout:8/818: chown d9/db/d12/f3d 7 1 2024-03-22T06:15:06.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:06 smithi117 ceph-mon[36514]: pgmap v16: 65 pgs: 65 active+clean; 1.0 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 97 MiB/s rd, 119 MiB/s wr, 189 op/s 2024-03-22T06:15:06.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:06 smithi006 ceph-mon[29301]: pgmap v16: 65 pgs: 65 active+clean; 1.0 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 97 MiB/s rd, 119 MiB/s wr, 189 op/s 2024-03-22T06:15:06.409 INFO:tasks.workunit.client.0.smithi006.stdout:8/819: mkdir d9/db/d35/dd2/d76/db3/dc3/d12d 0 2024-03-22T06:15:06.409 INFO:tasks.workunit.client.0.smithi006.stdout:8/820: stat d9/db/d35/dd2/d4a/f11a 0 2024-03-22T06:15:06.427 INFO:tasks.workunit.client.0.smithi006.stdout:4/724: mkdir d6/d1b/d20/d63/d93/d9a/dfd 0 2024-03-22T06:15:06.429 INFO:tasks.workunit.client.0.smithi006.stdout:4/725: truncate d6/d1b/d20/d21/d77/fa6 3450443 0 2024-03-22T06:15:06.447 INFO:tasks.workunit.client.0.smithi006.stdout:4/726: dwrite d6/d1b/d20/d21/f92 [4194304,4194304] 0 2024-03-22T06:15:06.453 INFO:tasks.workunit.client.0.smithi006.stdout:4/727: creat d6/d1b/d20/d6b/d60/db1/ffe x:0 0 0 2024-03-22T06:15:07.221 INFO:tasks.workunit.client.0.smithi006.stdout:7/737: rename d5/d1e/d63/d85/l9e to d5/d1e/d29/d1f/d2d/d55/d84/lef 0 2024-03-22T06:15:07.224 INFO:tasks.workunit.client.0.smithi006.stdout:8/821: rename d9/db/d12/d66/d10c/d125/da3/cc6 to d9/d14/d44/c12e 0 2024-03-22T06:15:07.225 INFO:tasks.workunit.client.0.smithi006.stdout:8/822: write d9/db/d12/d66/d6c/d92/fb1 [4127183,21171] 0 2024-03-22T06:15:07.227 INFO:tasks.workunit.client.0.smithi006.stdout:7/738: creat d5/d1e/d29/d1f/de4/ff0 x:0 0 0 2024-03-22T06:15:07.227 INFO:tasks.workunit.client.0.smithi006.stdout:7/739: creat d5/d1e/d29/d1f/de4/ff1 x:0 0 0 2024-03-22T06:15:07.232 INFO:tasks.workunit.client.0.smithi006.stdout:7/740: unlink d5/d1e/d63/d85/d80/led 0 2024-03-22T06:15:07.241 INFO:tasks.workunit.client.0.smithi006.stdout:8/823: dwrite d9/d14/f34 [0,4194304] 0 2024-03-22T06:15:07.259 INFO:tasks.workunit.client.0.smithi006.stdout:8/824: dwrite d9/db/fed [0,4194304] 0 2024-03-22T06:15:07.265 INFO:tasks.workunit.client.0.smithi006.stdout:8/825: rename d9/db/d35/dd2/d76/dca/dee/c12c to d9/db/d12/d66/c12f 0 2024-03-22T06:15:07.265 INFO:tasks.workunit.client.0.smithi006.stdout:8/826: truncate d9/db/d12/d66/d10c/d125/fe7 319305 0 2024-03-22T06:15:07.281 INFO:tasks.workunit.client.0.smithi006.stdout:8/827: dwrite d9/db/d12/d66/d10c/d125/da3/fb2 [0,4194304] 0 2024-03-22T06:15:07.338 INFO:tasks.workunit.client.0.smithi006.stdout:0/484: mknod db/ca7 0 2024-03-22T06:15:07.342 INFO:tasks.workunit.client.0.smithi006.stdout:0/485: truncate db/d10/d33/f57 766678 0 2024-03-22T06:15:07.342 INFO:tasks.workunit.client.0.smithi006.stdout:0/486: read - db/dc/d38/f66 zero size 2024-03-22T06:15:07.356 INFO:tasks.workunit.client.0.smithi006.stdout:0/487: dwrite db/d10/d2a/f51 [0,4194304] 0 2024-03-22T06:15:07.357 INFO:tasks.workunit.client.0.smithi006.stdout:0/488: read db/d10/d35/f53 [771303,34259] 0 2024-03-22T06:15:07.363 INFO:tasks.workunit.client.0.smithi006.stdout:0/489: rename db/dc/c13 to db/dc/d38/d50/ca8 0 2024-03-22T06:15:07.365 INFO:tasks.workunit.client.0.smithi006.stdout:8/828: rmdir d9/db/d35/dd2/d76/db3 39 2024-03-22T06:15:07.366 INFO:tasks.workunit.client.0.smithi006.stdout:0/490: creat db/dc/fa9 x:0 0 0 2024-03-22T06:15:07.368 INFO:tasks.workunit.client.0.smithi006.stdout:8/829: mkdir d9/db/d35/dd2/d76/db3/dc3/d89/db8/d130 0 2024-03-22T06:15:07.368 INFO:tasks.workunit.client.0.smithi006.stdout:8/830: creat d9/db/d12/d66/f131 x:0 0 0 2024-03-22T06:15:07.369 INFO:tasks.workunit.client.0.smithi006.stdout:0/491: creat db/dc/faa x:0 0 0 2024-03-22T06:15:07.373 INFO:tasks.workunit.client.0.smithi006.stdout:8/831: unlink d9/db/d12/d7e/c9e 0 2024-03-22T06:15:07.373 INFO:tasks.workunit.client.0.smithi006.stdout:8/832: readlink d9/db/d35/dd2/d4a/da7/ld8 0 2024-03-22T06:15:07.376 INFO:tasks.workunit.client.0.smithi006.stdout:0/492: mkdir db/dc/dab 0 2024-03-22T06:15:07.379 INFO:tasks.workunit.client.0.smithi006.stdout:0/493: truncate db/f45 2433939 0 2024-03-22T06:15:07.382 INFO:tasks.workunit.client.0.smithi006.stdout:0/494: dread db/dc/f44 [0,4194304] 0 2024-03-22T06:15:07.383 INFO:tasks.workunit.client.0.smithi006.stdout:0/495: readlink db/l46 0 2024-03-22T06:15:07.395 INFO:tasks.workunit.client.0.smithi006.stdout:0/496: dwrite db/dc/d37/f77 [4194304,4194304] 0 2024-03-22T06:15:07.395 INFO:tasks.workunit.client.0.smithi006.stdout:0/497: truncate db/d10/d33/d9d/fa2 637871 0 2024-03-22T06:15:07.399 INFO:tasks.workunit.client.0.smithi006.stdout:0/498: rename db/d10/d35/f43 to db/dc/d37/fac 0 2024-03-22T06:15:07.403 INFO:tasks.workunit.client.0.smithi006.stdout:0/499: truncate db/dc/d38/f4b 4477765 0 2024-03-22T06:15:07.403 INFO:tasks.workunit.client.0.smithi006.stdout:0/500: chown db/d10/d2a/d41/l4c 2033252599 1 2024-03-22T06:15:07.403 INFO:tasks.workunit.client.0.smithi006.stdout:0/501: read - db/d3d/d4e/f6f zero size 2024-03-22T06:15:07.407 INFO:tasks.workunit.client.0.smithi006.stdout:0/502: truncate db/d10/f1a 43079 0 2024-03-22T06:15:07.410 INFO:tasks.workunit.client.0.smithi006.stdout:0/503: mkdir db/d10/d35/d6c/dad 0 2024-03-22T06:15:07.410 INFO:tasks.workunit.client.0.smithi006.stdout:0/504: write db/dc/d38/f3a [5532090,59836] 0 2024-03-22T06:15:07.410 INFO:tasks.workunit.client.0.smithi006.stdout:0/505: write db/d3d/d4e/f6f [594413,116349] 0 2024-03-22T06:15:07.415 INFO:tasks.workunit.client.0.smithi006.stdout:0/506: mknod db/d10/cae 0 2024-03-22T06:15:07.415 INFO:tasks.workunit.client.0.smithi006.stdout:0/507: write db/d10/d33/f4d [513166,23786] 0 2024-03-22T06:15:08.240 INFO:tasks.workunit.client.0.smithi006.stdout:1/601: creat d8/d9/d3a/d40/fd0 x:0 0 0 2024-03-22T06:15:08.240 INFO:tasks.workunit.client.0.smithi006.stdout:1/602: dread - d8/d9/f8d zero size 2024-03-22T06:15:08.242 INFO:tasks.workunit.client.0.smithi006.stdout:1/603: mknod d8/d9/d13/d24/d32/cd1 0 2024-03-22T06:15:08.244 INFO:tasks.workunit.client.0.smithi006.stdout:4/728: symlink d6/lff 0 2024-03-22T06:15:08.250 INFO:tasks.workunit.client.0.smithi006.stdout:1/604: rmdir d8/d9/d13/d24/daf 0 2024-03-22T06:15:08.251 INFO:tasks.workunit.client.0.smithi006.stdout:1/605: fsync d8/d9/d13/d24/d32/dc7/f74 0 2024-03-22T06:15:08.251 INFO:tasks.workunit.client.0.smithi006.stdout:1/606: write d8/d9/f2d [4032119,61657] 0 2024-03-22T06:15:08.253 INFO:tasks.workunit.client.0.smithi006.stdout:1/607: mkdir d8/d9/d3a/d18/dd2 0 2024-03-22T06:15:08.254 INFO:tasks.workunit.client.0.smithi006.stdout:9/547: rmdir d8/d1f 39 2024-03-22T06:15:08.254 INFO:tasks.workunit.client.0.smithi006.stdout:9/548: creat d8/d10/d28/fb6 x:0 0 0 2024-03-22T06:15:08.262 INFO:tasks.workunit.client.0.smithi006.stdout:9/549: rename d8/d10/d11/f59 to d8/d10/d28/d39/fb7 0 2024-03-22T06:15:08.266 INFO:tasks.workunit.client.0.smithi006.stdout:9/550: unlink d8/d10/f6c 0 2024-03-22T06:15:08.268 INFO:tasks.workunit.client.0.smithi006.stdout:9/551: getdents d8/db3 0 2024-03-22T06:15:08.269 INFO:tasks.workunit.client.0.smithi006.stdout:1/608: dwrite d8/d9/d3a/d40/d44/f66 [0,4194304] 0 2024-03-22T06:15:08.275 INFO:tasks.workunit.client.0.smithi006.stdout:1/609: symlink d8/d9/d3a/d40/d59/db1/ld3 0 2024-03-22T06:15:08.275 INFO:tasks.workunit.client.0.smithi006.stdout:1/610: read d8/d9/d3a/d18/f62 [3433374,59244] 0 2024-03-22T06:15:08.275 INFO:tasks.workunit.client.0.smithi006.stdout:1/611: chown d8/d9/d3a/d40/d59 784 1 2024-03-22T06:15:08.278 INFO:tasks.workunit.client.0.smithi006.stdout:1/612: creat d8/d9/d13/d39/d60/fd4 x:0 0 0 2024-03-22T06:15:08.278 INFO:tasks.workunit.client.0.smithi006.stdout:1/613: write d8/d9/f2d [1125317,67709] 0 2024-03-22T06:15:08.278 INFO:tasks.workunit.client.0.smithi006.stdout:1/614: dread - d8/fa2 zero size 2024-03-22T06:15:08.280 INFO:tasks.workunit.client.0.smithi006.stdout:1/615: symlink d8/d9/d10/d7c/d96/ld5 0 2024-03-22T06:15:08.283 INFO:tasks.workunit.client.0.smithi006.stdout:9/552: dwrite d8/db3/d50/f84 [0,4194304] 0 2024-03-22T06:15:08.283 INFO:tasks.workunit.client.0.smithi006.stdout:9/553: readlink d8/d10/d28/l26 0 2024-03-22T06:15:08.283 INFO:tasks.workunit.client.0.smithi006.stdout:9/554: chown d8/d10/d20/d45/f66 17195649 1 2024-03-22T06:15:08.289 INFO:tasks.workunit.client.0.smithi006.stdout:9/555: creat d8/fb8 x:0 0 0 2024-03-22T06:15:08.296 INFO:tasks.workunit.client.0.smithi006.stdout:1/616: dwrite d8/d9/f67 [0,4194304] 0 2024-03-22T06:15:08.302 INFO:tasks.workunit.client.0.smithi006.stdout:1/617: mkdir d8/d9/d10/d7c/dd6 0 2024-03-22T06:15:08.309 INFO:tasks.workunit.client.0.smithi006.stdout:9/556: dwrite d8/d10/d23/d3e/f5a [0,4194304] 0 2024-03-22T06:15:08.318 INFO:tasks.workunit.client.0.smithi006.stdout:1/618: dread d8/d9/d3a/d40/d44/f53 [0,4194304] 0 2024-03-22T06:15:08.318 INFO:tasks.workunit.client.0.smithi006.stdout:1/619: write d8/d9/d13/d39/d94/fbc [761020,23990] 0 2024-03-22T06:15:08.321 INFO:tasks.workunit.client.0.smithi006.stdout:9/557: dread d8/db3/d50/f8e [0,4194304] 0 2024-03-22T06:15:08.321 INFO:tasks.workunit.client.0.smithi006.stdout:1/620: rename d8/d9/d13/d24/d32/ca9 to d8/d9/d3a/d40/d59/cd7 0 2024-03-22T06:15:08.323 INFO:tasks.workunit.client.0.smithi006.stdout:9/558: dread d8/db3/d50/d38/f58 [0,4194304] 0 2024-03-22T06:15:08.325 INFO:tasks.workunit.client.0.smithi006.stdout:1/621: link d8/d9/d3a/d40/d44/d80/fab d8/fd8 0 2024-03-22T06:15:08.327 INFO:tasks.workunit.client.0.smithi006.stdout:9/559: truncate d8/d10/d11/f3c 1650439 0 2024-03-22T06:15:08.330 INFO:tasks.workunit.client.0.smithi006.stdout:1/622: symlink d8/d9/d13/d39/ld9 0 2024-03-22T06:15:08.330 INFO:tasks.workunit.client.0.smithi006.stdout:1/623: write d8/d9/d10/f4e [925337,87416] 0 2024-03-22T06:15:08.331 INFO:tasks.workunit.client.0.smithi006.stdout:1/624: symlink d8/d9/d3a/d18/dd2/lda 0 2024-03-22T06:15:08.332 INFO:tasks.workunit.client.0.smithi006.stdout:1/625: dread - d8/d9/d13/d24/d32/d34/fb3 zero size 2024-03-22T06:15:08.334 INFO:tasks.workunit.client.0.smithi006.stdout:9/560: dread f1 [0,4194304] 0 2024-03-22T06:15:08.334 INFO:tasks.workunit.client.0.smithi006.stdout:1/626: symlink d8/d9/d13/d24/d32/dc0/ldb 0 2024-03-22T06:15:08.334 INFO:tasks.workunit.client.0.smithi006.stdout:1/627: chown d8/d71/dbb 28 1 2024-03-22T06:15:08.334 INFO:tasks.workunit.client.0.smithi006.stdout:1/628: creat d8/d9/d13/d24/d32/fdc x:0 0 0 2024-03-22T06:15:08.337 INFO:tasks.workunit.client.0.smithi006.stdout:9/561: unlink d8/d10/d20/d45/d49/c6b 0 2024-03-22T06:15:08.340 INFO:tasks.workunit.client.0.smithi006.stdout:1/629: symlink d8/d9/d3a/d18/dd2/ldd 0 2024-03-22T06:15:08.340 INFO:tasks.workunit.client.0.smithi006.stdout:1/630: fsync d8/d9/d3a/d40/d44/f76 0 2024-03-22T06:15:08.341 INFO:tasks.workunit.client.0.smithi006.stdout:9/562: truncate d8/f9e 1992454 0 2024-03-22T06:15:08.342 INFO:tasks.workunit.client.0.smithi006.stdout:1/631: truncate d8/d9/d10/d7c/f82 344154 0 2024-03-22T06:15:08.344 INFO:tasks.workunit.client.0.smithi006.stdout:9/563: mkdir d8/d10/d20/d45/db9 0 2024-03-22T06:15:08.344 INFO:tasks.workunit.client.0.smithi006.stdout:9/564: truncate d8/d10/d23/d3e/f9a 997060 0 2024-03-22T06:15:08.347 INFO:tasks.workunit.client.0.smithi006.stdout:9/565: unlink d8/d10/d28/f35 0 2024-03-22T06:15:08.357 INFO:tasks.workunit.client.0.smithi006.stdout:9/566: dread d8/d10/f12 [0,4194304] 0 2024-03-22T06:15:08.357 INFO:tasks.workunit.client.0.smithi006.stdout:9/567: dread - d8/db3/d50/f6f zero size 2024-03-22T06:15:08.359 INFO:tasks.workunit.client.0.smithi006.stdout:9/568: mkdir d8/d10/d28/dba 0 2024-03-22T06:15:08.359 INFO:tasks.workunit.client.0.smithi006.stdout:9/569: fdatasync d8/d1f/d34/f92 0 2024-03-22T06:15:08.361 INFO:tasks.workunit.client.0.smithi006.stdout:1/632: dwrite d8/d9/d10/f4e [0,4194304] 0 2024-03-22T06:15:08.361 INFO:tasks.workunit.client.0.smithi006.stdout:9/570: rename d8/d10/da6/cac to d8/d10/d11/cbb 0 2024-03-22T06:15:08.365 INFO:tasks.workunit.client.0.smithi006.stdout:1/633: unlink d8/d9/d13/l25 0 2024-03-22T06:15:08.367 INFO:tasks.workunit.client.0.smithi006.stdout:9/571: mkdir d8/d10/d47/dbc 0 2024-03-22T06:15:08.367 INFO:tasks.workunit.client.0.smithi006.stdout:9/572: truncate d8/d1f/f5d 5108630 0 2024-03-22T06:15:08.367 INFO:tasks.workunit.client.0.smithi006.stdout:9/573: dread - d8/d10/d47/fb2 zero size 2024-03-22T06:15:08.367 INFO:tasks.workunit.client.0.smithi006.stdout:9/574: write d8/db3/d50/d38/f58 [1263412,32152] 0 2024-03-22T06:15:08.376 INFO:tasks.workunit.client.0.smithi006.stdout:9/575: dread d8/f21 [0,4194304] 0 2024-03-22T06:15:08.377 INFO:tasks.workunit.client.0.smithi006.stdout:9/576: write d8/fb8 [865335,29982] 0 2024-03-22T06:15:08.379 INFO:tasks.workunit.client.0.smithi006.stdout:9/577: creat d8/d10/d28/dba/fbd x:0 0 0 2024-03-22T06:15:08.380 INFO:tasks.workunit.client.0.smithi006.stdout:1/634: dwrite d8/d9/d13/d24/d32/d34/fb3 [0,4194304] 0 2024-03-22T06:15:08.382 INFO:tasks.workunit.client.0.smithi006.stdout:9/578: creat d8/d1f/fbe x:0 0 0 2024-03-22T06:15:08.396 INFO:tasks.workunit.client.0.smithi006.stdout:2/725: sync 2024-03-22T06:15:08.396 INFO:tasks.workunit.client.0.smithi006.stdout:5/794: sync 2024-03-22T06:15:08.396 INFO:tasks.workunit.client.0.smithi006.stdout:5/795: truncate d0/db/d76/db4/fc3 580423 0 2024-03-22T06:15:08.396 INFO:tasks.workunit.client.0.smithi006.stdout:9/579: dread d8/d10/d11/f16 [0,4194304] 0 2024-03-22T06:15:08.401 INFO:tasks.workunit.client.0.smithi006.stdout:5/796: rmdir d0/db/d16/d1c/d34 39 2024-03-22T06:15:08.407 INFO:tasks.workunit.client.0.smithi006.stdout:9/580: rename d8/d10/d20/d45/d4d/f7a to d8/d1f/d34/fbf 0 2024-03-22T06:15:08.411 INFO:tasks.workunit.client.0.smithi006.stdout:9/581: truncate d8/d10/d47/f97 1273052 0 2024-03-22T06:15:08.419 INFO:tasks.workunit.client.0.smithi006.stdout:1/635: dwrite d8/d9/d13/f26 [4194304,4194304] 0 2024-03-22T06:15:08.419 INFO:tasks.workunit.client.0.smithi006.stdout:1/636: fdatasync d8/d71/f8e 0 2024-03-22T06:15:08.420 INFO:tasks.workunit.client.0.smithi006.stdout:2/726: dwrite d13/d34/d4d/d4e/d75/f9a [0,4194304] 0 2024-03-22T06:15:08.421 INFO:tasks.workunit.client.0.smithi006.stdout:5/797: dwrite d0/db/d16/d70/fd1 [4194304,4194304] 0 2024-03-22T06:15:08.422 INFO:tasks.workunit.client.0.smithi006.stdout:2/727: dread d13/d53/f8a [0,4194304] 0 2024-03-22T06:15:08.429 INFO:tasks.workunit.client.0.smithi006.stdout:1/637: dread d8/d9/d13/d24/d32/d34/f45 [0,4194304] 0 2024-03-22T06:15:08.430 INFO:tasks.workunit.client.0.smithi006.stdout:9/582: dwrite d8/d10/d23/d3e/fae [0,4194304] 0 2024-03-22T06:15:08.433 INFO:tasks.workunit.client.0.smithi006.stdout:5/798: symlink d0/db/d16/d43/ddc/lf0 0 2024-03-22T06:15:08.437 INFO:tasks.workunit.client.0.smithi006.stdout:2/728: rename d13/d34/d4d/d4e/d75/ca9 to d13/d69/c111 0 2024-03-22T06:15:08.437 INFO:tasks.workunit.client.0.smithi006.stdout:2/729: read - d13/d20/fcb zero size 2024-03-22T06:15:08.446 INFO:tasks.workunit.client.0.smithi006.stdout:2/730: truncate d13/d34/d4d/d4e/d75/d62/fd6 486248 0 2024-03-22T06:15:08.448 INFO:tasks.workunit.client.0.smithi006.stdout:2/731: write d13/d34/f58 [159562,121998] 0 2024-03-22T06:15:08.448 INFO:tasks.workunit.client.0.smithi006.stdout:2/732: stat d13/d34/d4d/d4e/d75/da5/d82/fae 0 2024-03-22T06:15:08.451 INFO:tasks.workunit.client.0.smithi006.stdout:1/638: dwrite d8/d9/d10/f19 [0,4194304] 0 2024-03-22T06:15:08.451 INFO:tasks.workunit.client.0.smithi006.stdout:5/799: dwrite d0/db/d16/d43/ddc/fdf [0,4194304] 0 2024-03-22T06:15:08.454 INFO:tasks.workunit.client.0.smithi006.stdout:2/733: creat d13/d53/d6b/d10b/f112 x:0 0 0 2024-03-22T06:15:08.454 INFO:tasks.workunit.client.0.smithi006.stdout:2/734: chown d13/d20/d35/d65/f68 23440624 1 2024-03-22T06:15:08.458 INFO:tasks.workunit.client.0.smithi006.stdout:1/639: write d8/d9/d13/d24/d32/dc7/f74 [794978,130528] 0 2024-03-22T06:15:08.460 INFO:tasks.workunit.client.0.smithi006.stdout:5/800: read d0/db/d16/d43/ddc/fdf [431476,63959] 0 2024-03-22T06:15:08.462 INFO:tasks.workunit.client.0.smithi006.stdout:2/735: rmdir d13/d34/d4d/d4e/d75/da5/d82/dc3/df8 0 2024-03-22T06:15:08.468 INFO:tasks.workunit.client.0.smithi006.stdout:1/640: unlink d8/d9/d13/d24/d32/d34/f45 0 2024-03-22T06:15:08.468 INFO:tasks.workunit.client.0.smithi006.stdout:1/641: readlink d8/d9/d13/d39/l52 0 2024-03-22T06:15:08.471 INFO:tasks.workunit.client.0.smithi006.stdout:5/801: creat d0/db/d16/d1c/d34/d35/ff1 x:0 0 0 2024-03-22T06:15:08.471 INFO:tasks.workunit.client.0.smithi006.stdout:5/802: chown d0/db/d50/d96 267330433 1 2024-03-22T06:15:08.473 INFO:tasks.workunit.client.0.smithi006.stdout:0/508: write db/dc/d38/f4b [764970,81077] 0 2024-03-22T06:15:08.475 INFO:tasks.workunit.client.0.smithi006.stdout:0/509: dread db/dc/f44 [0,4194304] 0 2024-03-22T06:15:08.484 INFO:tasks.workunit.client.0.smithi006.stdout:5/803: creat d0/db/d50/d96/d52/d69/d92/db7/ff2 x:0 0 0 2024-03-22T06:15:08.485 INFO:tasks.workunit.client.0.smithi006.stdout:1/642: dwrite d8/d9/d13/f6d [0,4194304] 0 2024-03-22T06:15:08.487 INFO:tasks.workunit.client.0.smithi006.stdout:0/510: rename db/dc/f7e to db/d87/faf 0 2024-03-22T06:15:08.487 INFO:tasks.workunit.client.0.smithi006.stdout:0/511: chown db/d10/d35/d7b/l95 20169446 1 2024-03-22T06:15:08.487 INFO:tasks.workunit.client.0.smithi006.stdout:0/512: write db/dc/f44 [2486562,35325] 0 2024-03-22T06:15:08.487 INFO:tasks.workunit.client.0.smithi006.stdout:0/513: write db/dc/f44 [2765342,100561] 0 2024-03-22T06:15:08.487 INFO:tasks.workunit.client.0.smithi006.stdout:0/514: creat db/d10/d33/fb0 x:0 0 0 2024-03-22T06:15:08.487 INFO:tasks.workunit.client.0.smithi006.stdout:0/515: chown db/l46 1843 1 2024-03-22T06:15:08.487 INFO:tasks.workunit.client.0.smithi006.stdout:0/516: dread - db/d87/faf zero size 2024-03-22T06:15:08.492 INFO:tasks.workunit.client.0.smithi006.stdout:1/643: write d8/d33/f9c [638877,57959] 0 2024-03-22T06:15:08.492 INFO:tasks.workunit.client.0.smithi006.stdout:1/644: readlink d8/d9/d3a/d40/d44/l29 0 2024-03-22T06:15:08.494 INFO:tasks.workunit.client.0.smithi006.stdout:0/517: mknod db/dc/d38/d50/cb1 0 2024-03-22T06:15:08.497 INFO:tasks.workunit.client.0.smithi006.stdout:5/804: rmdir d0/db/d76/db4 39 2024-03-22T06:15:08.500 INFO:tasks.workunit.client.0.smithi006.stdout:1/645: mknod d8/d71/cde 0 2024-03-22T06:15:08.503 INFO:tasks.workunit.client.0.smithi006.stdout:5/805: unlink d0/db/d76/f81 0 2024-03-22T06:15:08.505 INFO:tasks.workunit.client.0.smithi006.stdout:1/646: unlink d8/d9/d13/f5a 0 2024-03-22T06:15:08.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:08 smithi117 ceph-mon[36514]: pgmap v17: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 159 MiB/s rd, 198 MiB/s wr, 321 op/s 2024-03-22T06:15:08.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:08 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:08.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:08 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:08.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:08 smithi117 ceph-mon[36514]: Reconfiguring alertmanager.smithi006 (dependencies changed)... 2024-03-22T06:15:08.509 INFO:tasks.workunit.client.0.smithi006.stdout:5/806: creat d0/db/ff3 x:0 0 0 2024-03-22T06:15:08.512 INFO:tasks.workunit.client.0.smithi006.stdout:5/807: creat d0/db/d50/d96/d52/ff4 x:0 0 0 2024-03-22T06:15:08.512 INFO:tasks.workunit.client.0.smithi006.stdout:5/808: dread - d0/db/d76/fdd zero size 2024-03-22T06:15:08.512 INFO:tasks.workunit.client.0.smithi006.stdout:5/809: fsync d0/db/d16/d1c/d34/d35/d85/dda/f87 0 2024-03-22T06:15:08.512 INFO:tasks.workunit.client.0.smithi006.stdout:5/810: fsync d0/db/d16/d1c/d34/fa6 0 2024-03-22T06:15:08.512 INFO:tasks.workunit.client.0.smithi006.stdout:0/518: dwrite db/d10/d35/f69 [0,4194304] 0 2024-03-22T06:15:08.514 INFO:tasks.workunit.client.0.smithi006.stdout:5/811: mknod d0/db/d16/d1c/cf5 0 2024-03-22T06:15:08.516 INFO:tasks.workunit.client.0.smithi006.stdout:1/647: dread d8/d9/d13/f6d [0,4194304] 0 2024-03-22T06:15:08.517 INFO:tasks.workunit.client.0.smithi006.stdout:0/519: creat db/d10/d35/d6c/fb2 x:0 0 0 2024-03-22T06:15:08.517 INFO:tasks.workunit.client.0.smithi006.stdout:0/520: readlink db/dc/le 0 2024-03-22T06:15:08.517 INFO:tasks.workunit.client.0.smithi006.stdout:0/521: stat db/d3d 0 2024-03-22T06:15:08.518 INFO:tasks.workunit.client.0.smithi006.stdout:5/812: rmdir d0/db/d16 39 2024-03-22T06:15:08.519 INFO:tasks.workunit.client.0.smithi006.stdout:1/648: rmdir d8/d9/d3a/d18/dd2 39 2024-03-22T06:15:08.522 INFO:tasks.workunit.client.0.smithi006.stdout:0/522: symlink db/d10/d2a/lb3 0 2024-03-22T06:15:08.524 INFO:tasks.workunit.client.0.smithi006.stdout:5/813: link d0/db/d50/d96/faa d0/db/d50/d96/d52/d69/ff6 0 2024-03-22T06:15:08.528 INFO:tasks.workunit.client.0.smithi006.stdout:0/523: dread db/dc/f44 [0,4194304] 0 2024-03-22T06:15:08.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:08 smithi006 ceph-mon[29301]: pgmap v17: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 159 MiB/s rd, 198 MiB/s wr, 321 op/s 2024-03-22T06:15:08.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:08 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:08.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:08 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:08.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:08 smithi006 ceph-mon[29301]: Reconfiguring alertmanager.smithi006 (dependencies changed)... 2024-03-22T06:15:08.533 INFO:tasks.workunit.client.0.smithi006.stdout:5/814: write d0/db/f1e [5561210,122392] 0 2024-03-22T06:15:08.533 INFO:tasks.workunit.client.0.smithi006.stdout:5/815: write d0/db/d50/d58/f91 [1434346,31643] 0 2024-03-22T06:15:08.536 INFO:tasks.workunit.client.0.smithi006.stdout:5/816: mkdir d0/db/d50/d58/df7 0 2024-03-22T06:15:08.537 INFO:tasks.workunit.client.0.smithi006.stdout:0/524: dread db/d10/d35/f69 [0,4194304] 0 2024-03-22T06:15:08.541 INFO:tasks.workunit.client.0.smithi006.stdout:0/525: mkdir db/d10/d33/db4 0 2024-03-22T06:15:08.541 INFO:tasks.workunit.client.0.smithi006.stdout:0/526: write db/d3d/d4e/f9b [465475,54295] 0 2024-03-22T06:15:08.543 INFO:tasks.workunit.client.0.smithi006.stdout:5/817: creat d0/db/d16/d70/ff8 x:0 0 0 2024-03-22T06:15:08.545 INFO:tasks.workunit.client.0.smithi006.stdout:0/527: unlink db/d10/d35/l75 0 2024-03-22T06:15:08.545 INFO:tasks.workunit.client.0.smithi006.stdout:0/528: stat db/d10/d33/ca3 0 2024-03-22T06:15:08.547 INFO:tasks.workunit.client.0.smithi006.stdout:5/818: mknod d0/db/d50/d58/df7/cf9 0 2024-03-22T06:15:08.547 INFO:tasks.workunit.client.0.smithi006.stdout:5/819: stat d0/db/l26 0 2024-03-22T06:15:08.549 INFO:tasks.workunit.client.0.smithi006.stdout:0/529: creat db/d10/fb5 x:0 0 0 2024-03-22T06:15:08.549 INFO:tasks.workunit.client.0.smithi006.stdout:0/530: truncate db/d10/d35/d6c/fb2 750539 0 2024-03-22T06:15:08.549 INFO:tasks.workunit.client.0.smithi006.stdout:0/531: write db/d10/d33/fb0 [344170,10882] 0 2024-03-22T06:15:08.551 INFO:tasks.workunit.client.0.smithi006.stdout:5/820: symlink d0/db/d16/d70/lfa 0 2024-03-22T06:15:08.556 INFO:tasks.workunit.client.0.smithi006.stdout:5/821: mkdir d0/db/d16/d1c/d34/d35/d85/d5e/dfb 0 2024-03-22T06:15:08.556 INFO:tasks.workunit.client.0.smithi006.stdout:5/822: chown d0/db/d16/d1c/c24 134 1 2024-03-22T06:15:08.560 INFO:tasks.workunit.client.0.smithi006.stdout:5/823: symlink d0/db/d16/d1c/d34/d35/d85/d5e/d66/dc0/lfc 0 2024-03-22T06:15:08.560 INFO:tasks.workunit.client.0.smithi006.stdout:5/824: fsync d0/db/d16/d43/ddc/fdf 0 2024-03-22T06:15:08.560 INFO:tasks.workunit.client.0.smithi006.stdout:5/825: readlink d0/db/dc/l42 0 2024-03-22T06:15:08.562 INFO:tasks.workunit.client.0.smithi006.stdout:0/532: dwrite db/d10/d2a/d41/f82 [0,4194304] 0 2024-03-22T06:15:08.563 INFO:tasks.workunit.client.0.smithi006.stdout:0/533: chown db/d10/l26 227743174 1 2024-03-22T06:15:08.570 INFO:tasks.workunit.client.0.smithi006.stdout:5/826: write d0/db/d16/d1c/d34/d35/fa8 [1996680,76819] 0 2024-03-22T06:15:08.571 INFO:tasks.workunit.client.0.smithi006.stdout:0/534: symlink db/d3d/lb6 0 2024-03-22T06:15:08.572 INFO:tasks.workunit.client.0.smithi006.stdout:5/827: dread d0/db/d76/f9a [0,4194304] 0 2024-03-22T06:15:08.575 INFO:tasks.workunit.client.0.smithi006.stdout:0/535: mkdir db/d10/d35/d6c/db7 0 2024-03-22T06:15:08.577 INFO:tasks.workunit.client.0.smithi006.stdout:5/828: chown d0/db/d16/ca5 0 1 2024-03-22T06:15:08.578 INFO:tasks.workunit.client.0.smithi006.stdout:5/829: fdatasync d0/fa4 0 2024-03-22T06:15:08.578 INFO:tasks.workunit.client.0.smithi006.stdout:5/830: chown d0/db/d50/f5b 245747 1 2024-03-22T06:15:08.578 INFO:tasks.workunit.client.0.smithi006.stdout:5/831: chown d0/db/d16/d70/lfa 1482900056 1 2024-03-22T06:15:08.578 INFO:tasks.workunit.client.0.smithi006.stdout:5/832: read d0/db/d50/d96/f63 [267521,81789] 0 2024-03-22T06:15:08.580 INFO:tasks.workunit.client.0.smithi006.stdout:0/536: truncate db/d87/f93 2274168 0 2024-03-22T06:15:08.584 INFO:tasks.workunit.client.0.smithi006.stdout:5/833: rename d0/db/d16/d1c/dbc/dd6/fe3 to d0/db/d16/d1c/d34/d35/d85/d5e/dfb/ffd 0 2024-03-22T06:15:09.265 INFO:tasks.workunit.client.0.smithi006.stdout:9/583: getdents d8/d10/d28/d39 0 2024-03-22T06:15:09.265 INFO:tasks.workunit.client.0.smithi006.stdout:9/584: chown d8/d10/d28/dba/fbd 2470252 1 2024-03-22T06:15:09.268 INFO:tasks.workunit.client.0.smithi006.stdout:9/585: mkdir d8/d10/d11/dc0 0 2024-03-22T06:15:09.268 INFO:tasks.workunit.client.0.smithi006.stdout:9/586: readlink d8/d10/d20/la3 0 2024-03-22T06:15:09.269 INFO:tasks.workunit.client.0.smithi006.stdout:9/587: write d8/d10/d28/fa7 [845641,78253] 0 2024-03-22T06:15:09.269 INFO:tasks.workunit.client.0.smithi006.stdout:9/588: write d8/d10/d28/fb6 [1030008,18978] 0 2024-03-22T06:15:09.271 INFO:tasks.workunit.client.0.smithi006.stdout:9/589: truncate d8/d10/d11/f16 1772272 0 2024-03-22T06:15:09.271 INFO:tasks.workunit.client.0.smithi006.stdout:9/590: chown d8/d1f/d34/f70 2169599 1 2024-03-22T06:15:09.271 INFO:tasks.workunit.client.0.smithi006.stdout:9/591: chown d8/d10/d47/c5c 12 1 2024-03-22T06:15:09.271 INFO:tasks.workunit.client.0.smithi006.stdout:4/729: dwrite d6/f4a [0,4194304] 0 2024-03-22T06:15:09.273 INFO:tasks.workunit.client.0.smithi006.stdout:9/592: rmdir d8/d10/d20 39 2024-03-22T06:15:09.281 INFO:tasks.workunit.client.0.smithi006.stdout:4/730: unlink d6/d1b/d20/d63/ce3 0 2024-03-22T06:15:09.284 INFO:tasks.workunit.client.0.smithi006.stdout:4/731: unlink d6/d1b/d20/d63/d93/d9a/d8f/cc7 0 2024-03-22T06:15:09.289 INFO:tasks.workunit.client.0.smithi006.stdout:9/593: dwrite d8/db3/d50/f61 [0,4194304] 0 2024-03-22T06:15:09.289 INFO:tasks.workunit.client.0.smithi006.stdout:9/594: chown d8/d1f/d34/f3a 1790003 1 2024-03-22T06:15:09.289 INFO:tasks.workunit.client.0.smithi006.stdout:9/595: dread - d8/d10/d28/f52 zero size 2024-03-22T06:15:09.302 INFO:tasks.workunit.client.0.smithi006.stdout:4/732: dwrite d6/d1b/d20/d21/d2a/d43/d4b/fa7 [0,4194304] 0 2024-03-22T06:15:09.308 INFO:tasks.workunit.client.0.smithi006.stdout:4/733: link d6/f12 d6/d1b/d20/d21/d77/df1/f100 0 2024-03-22T06:15:09.309 INFO:tasks.workunit.client.0.smithi006.stdout:4/734: chown d6/d1b/d20/d21/d77/df1/ffb 7673 1 2024-03-22T06:15:09.309 INFO:tasks.workunit.client.0.smithi006.stdout:4/735: chown d6/d1b/d20/d63/d93/d9a/d8f/da1/db5/f9b 7 1 2024-03-22T06:15:09.309 INFO:tasks.workunit.client.0.smithi006.stdout:4/736: dread - d6/d1b/d20/d21/d77/df1/ffb zero size 2024-03-22T06:15:09.311 INFO:tasks.workunit.client.0.smithi006.stdout:4/737: mkdir d6/d1b/d46/d101 0 2024-03-22T06:15:09.313 INFO:tasks.workunit.client.0.smithi006.stdout:4/738: fsync d6/d1b/d20/d6b/d95/ff3 0 2024-03-22T06:15:09.330 INFO:tasks.workunit.client.0.smithi006.stdout:1/649: getdents d8/d9/d13/d24/d32/dc0 0 2024-03-22T06:15:09.382 INFO:tasks.workunit.client.0.smithi006.stdout:3/691: sync 2024-03-22T06:15:09.382 INFO:tasks.workunit.client.0.smithi006.stdout:6/619: sync 2024-03-22T06:15:09.382 INFO:tasks.workunit.client.0.smithi006.stdout:6/620: chown d6/d1d/d5c/d89/cbf 1996429 1 2024-03-22T06:15:09.383 INFO:tasks.workunit.client.0.smithi006.stdout:4/739: dwrite d6/d1b/d20/d21/d9f/fad [0,4194304] 0 2024-03-22T06:15:09.388 INFO:tasks.workunit.client.0.smithi006.stdout:6/621: mkdir d6/d1d/dc6 0 2024-03-22T06:15:09.391 INFO:tasks.workunit.client.0.smithi006.stdout:1/650: dwrite d8/d9/d13/d39/d94/db5/fca [0,4194304] 0 2024-03-22T06:15:09.391 INFO:tasks.workunit.client.0.smithi006.stdout:1/651: fsync d8/d33/f9c 0 2024-03-22T06:15:09.394 INFO:tasks.workunit.client.0.smithi006.stdout:4/740: creat d6/d1b/d20/d63/d93/f102 x:0 0 0 2024-03-22T06:15:09.396 INFO:tasks.workunit.client.0.smithi006.stdout:4/741: dread d6/fbd [0,4194304] 0 2024-03-22T06:15:09.397 INFO:tasks.workunit.client.0.smithi006.stdout:4/742: read d6/fbd [760657,52633] 0 2024-03-22T06:15:09.398 INFO:tasks.workunit.client.0.smithi006.stdout:3/692: rename d0/d23 to d0/d8/d1c/d3e/d5f/d90/da7/de7 0 2024-03-22T06:15:09.405 INFO:tasks.workunit.client.0.smithi006.stdout:1/652: symlink d8/d71/dcd/ldf 0 2024-03-22T06:15:09.405 INFO:tasks.workunit.client.0.smithi006.stdout:1/653: chown d8/d9/d3a/d18/f78 97802239 1 2024-03-22T06:15:09.410 INFO:tasks.workunit.client.0.smithi006.stdout:3/693: rename d0/d8/d9/l11 to d0/d8/d1c/d3e/d68/le8 0 2024-03-22T06:15:09.410 INFO:tasks.workunit.client.0.smithi006.stdout:3/694: getdents d0/d8/d88/ddd 0 2024-03-22T06:15:09.412 INFO:tasks.workunit.client.0.smithi006.stdout:4/743: dwrite d6/d13/d3b/f4e [0,4194304] 0 2024-03-22T06:15:09.414 INFO:tasks.workunit.client.0.smithi006.stdout:3/695: mknod d0/d8/d9/db1/ce9 0 2024-03-22T06:15:09.414 INFO:tasks.workunit.client.0.smithi006.stdout:3/696: truncate d0/d8/f17 4241499 0 2024-03-22T06:15:09.419 INFO:tasks.workunit.client.0.smithi006.stdout:1/654: write d8/d9/d3a/d18/f1b [4168028,74972] 0 2024-03-22T06:15:09.428 INFO:tasks.workunit.client.0.smithi006.stdout:3/697: unlink d0/d8/d1c/d3e/d5f/d90/da7/de7/d33/d3d/f6e 0 2024-03-22T06:15:09.431 INFO:tasks.workunit.client.0.smithi006.stdout:4/744: dwrite d6/d1b/d20/d21/d77/df1/ffb [0,4194304] 0 2024-03-22T06:15:09.432 INFO:tasks.workunit.client.0.smithi006.stdout:1/655: symlink d8/d9/d3a/d40/d44/d80/le0 0 2024-03-22T06:15:09.432 INFO:tasks.workunit.client.0.smithi006.stdout:1/656: stat d8/d71/c7b 0 2024-03-22T06:15:09.432 INFO:tasks.workunit.client.0.smithi006.stdout:1/657: creat d8/d9/d13/d39/d94/fe1 x:0 0 0 2024-03-22T06:15:09.432 INFO:tasks.workunit.client.0.smithi006.stdout:1/658: fsync d8/d9/d13/d24/d32/d34/f99 0 2024-03-22T06:15:09.432 INFO:tasks.workunit.client.0.smithi006.stdout:1/659: fdatasync d8/d9/d3a/d40/f7d 0 2024-03-22T06:15:09.434 INFO:tasks.workunit.client.0.smithi006.stdout:3/698: creat d0/d8/d1c/d3e/d5f/d90/da7/de7/d33/d3d/dcb/fea x:0 0 0 2024-03-22T06:15:09.439 INFO:tasks.workunit.client.0.smithi006.stdout:4/745: dread d6/d1b/d20/d6b/fae [0,4194304] 0 2024-03-22T06:15:09.442 INFO:tasks.workunit.client.0.smithi006.stdout:1/660: mknod d8/d9/d13/d39/d60/d8f/ce2 0 2024-03-22T06:15:09.444 INFO:tasks.workunit.client.0.smithi006.stdout:3/699: creat d0/d4/d53/de0/feb x:0 0 0 2024-03-22T06:15:09.447 INFO:tasks.workunit.client.0.smithi006.stdout:4/746: creat d6/d1b/d20/d21/d2a/d43/d4b/f103 x:0 0 0 2024-03-22T06:15:09.447 INFO:tasks.workunit.client.0.smithi006.stdout:4/747: dread - d6/d13/d3b/f7a zero size 2024-03-22T06:15:09.447 INFO:tasks.workunit.client.0.smithi006.stdout:4/748: chown d6/d1b/d20/d63/d93/d9a/cd0 45033 1 2024-03-22T06:15:09.454 INFO:tasks.workunit.client.0.smithi006.stdout:4/749: rmdir d6/d1b/d20/d21/d2a/d43/da8 39 2024-03-22T06:15:09.454 INFO:tasks.workunit.client.0.smithi006.stdout:4/750: readlink d6/d1b/d20/d21/d2a/d43/d4b/lb7 0 2024-03-22T06:15:09.456 INFO:tasks.workunit.client.0.smithi006.stdout:1/661: dwrite d8/d9/f2d [0,4194304] 0 2024-03-22T06:15:09.457 INFO:tasks.workunit.client.0.smithi006.stdout:4/751: truncate d6/d1b/d20/d21/d2a/dc9/fe0 693261 0 2024-03-22T06:15:09.461 INFO:tasks.workunit.client.0.smithi006.stdout:4/752: stat d6/d1b/d20/d6b/d60/l83 0 2024-03-22T06:15:09.479 INFO:tasks.workunit.client.0.smithi006.stdout:4/753: dwrite d6/d1b/d20/d21/d2a/d43/d4b/f103 [0,4194304] 0 2024-03-22T06:15:09.479 INFO:tasks.workunit.client.0.smithi006.stdout:4/754: creat d6/d13/f104 x:0 0 0 2024-03-22T06:15:09.486 INFO:tasks.workunit.client.0.smithi006.stdout:4/755: dread d6/d1b/d20/d6b/f8c [0,4194304] 0 2024-03-22T06:15:09.489 INFO:tasks.workunit.client.0.smithi006.stdout:4/756: mknod d6/d1b/d20/d21/d2a/c105 0 2024-03-22T06:15:09.492 INFO:tasks.workunit.client.0.smithi006.stdout:4/757: symlink d6/d1b/d20/d21/d77/df2/l106 0 2024-03-22T06:15:09.494 INFO:tasks.workunit.client.0.smithi006.stdout:4/758: symlink d6/d1b/d20/d21/d2a/dc9/l107 0 2024-03-22T06:15:09.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:09 smithi117 ceph-mon[36514]: Reconfiguring daemon alertmanager.smithi006 on smithi006 2024-03-22T06:15:09.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:09 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:15:09.512 INFO:tasks.workunit.client.0.smithi006.stdout:4/759: dwrite d6/f4a [0,4194304] 0 2024-03-22T06:15:09.512 INFO:tasks.workunit.client.0.smithi006.stdout:4/760: read - d6/f1c zero size 2024-03-22T06:15:09.516 INFO:tasks.workunit.client.0.smithi006.stdout:4/761: creat d6/d1b/d20/d6b/d60/f108 x:0 0 0 2024-03-22T06:15:09.518 INFO:tasks.workunit.client.0.smithi006.stdout:4/762: chown d6/d1b/d20/d21/d77/c97 956599 1 2024-03-22T06:15:09.526 INFO:tasks.workunit.client.0.smithi006.stdout:4/763: write d6/d13/d3b/f51 [1742446,109685] 0 2024-03-22T06:15:09.528 INFO:tasks.workunit.client.0.smithi006.stdout:4/764: dread d6/f1f [0,4194304] 0 2024-03-22T06:15:09.532 INFO:tasks.workunit.client.0.smithi006.stdout:4/765: creat d6/d1b/d20/d63/d93/db8/dbc/f109 x:0 0 0 2024-03-22T06:15:09.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:09 smithi006 ceph-mon[29301]: Reconfiguring daemon alertmanager.smithi006 on smithi006 2024-03-22T06:15:09.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:09 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:15:09.535 INFO:tasks.workunit.client.0.smithi006.stdout:4/766: link f1 d6/d1b/d20/d63/d93/db8/f10a 0 2024-03-22T06:15:09.543 INFO:tasks.workunit.client.0.smithi006.stdout:4/767: write d6/d1b/d20/d21/d2a/d43/f7d [3846099,47945] 0 2024-03-22T06:15:09.547 INFO:tasks.workunit.client.0.smithi006.stdout:4/768: mknod d6/d1b/d20/d63/d93/d9a/d8f/da1/db5/c10b 0 2024-03-22T06:15:09.547 INFO:tasks.workunit.client.0.smithi006.stdout:4/769: chown d6/d1b/d20/d6b/d60/l99 58179 1 2024-03-22T06:15:09.549 INFO:tasks.workunit.client.0.smithi006.stdout:4/770: mknod d6/d1b/d20/d21/d77/df1/c10c 0 2024-03-22T06:15:09.561 INFO:tasks.workunit.client.0.smithi006.stdout:4/771: read d6/d1b/f2d [4073328,12320] 0 2024-03-22T06:15:09.561 INFO:tasks.workunit.client.0.smithi006.stdout:4/772: chown d6/d1b/d20/d63/ca5 229366931 1 2024-03-22T06:15:09.561 INFO:tasks.workunit.client.0.smithi006.stdout:4/773: creat d6/d1b/d20/d21/d2a/d43/d4b/f10d x:0 0 0 2024-03-22T06:15:09.561 INFO:tasks.workunit.client.0.smithi006.stdout:4/774: stat d6/d13/d3b/f7a 0 2024-03-22T06:15:09.561 INFO:tasks.workunit.client.0.smithi006.stdout:4/775: read d6/f30 [4150356,50539] 0 2024-03-22T06:15:09.561 INFO:tasks.workunit.client.0.smithi006.stdout:4/776: write d6/d13/f17 [298462,61819] 0 2024-03-22T06:15:09.561 INFO:tasks.workunit.client.0.smithi006.stdout:4/777: fsync d6/d1b/d20/d21/d2a/d43/d4b/fa7 0 2024-03-22T06:15:09.561 INFO:tasks.workunit.client.0.smithi006.stdout:4/778: chown d6/d13/d3b 3488 1 2024-03-22T06:15:09.564 INFO:tasks.workunit.client.0.smithi006.stdout:4/779: creat d6/f10e x:0 0 0 2024-03-22T06:15:09.564 INFO:tasks.workunit.client.0.smithi006.stdout:4/780: write d6/d1b/d20/d6b/fae [4973171,28767] 0 2024-03-22T06:15:09.571 INFO:tasks.workunit.client.0.smithi006.stdout:4/781: dread d6/fb9 [0,4194304] 0 2024-03-22T06:15:09.574 INFO:tasks.workunit.client.0.smithi006.stdout:4/782: fdatasync d6/d13/d3b/f51 0 2024-03-22T06:15:09.575 INFO:tasks.workunit.client.0.smithi006.stdout:4/783: mknod d6/d1b/c10f 0 2024-03-22T06:15:09.578 INFO:tasks.workunit.client.0.smithi006.stdout:4/784: truncate d6/fa 3215343 0 2024-03-22T06:15:09.608 INFO:tasks.workunit.client.0.smithi006.stdout:0/537: truncate db/d10/f1a 964907 0 2024-03-22T06:15:09.610 INFO:tasks.workunit.client.0.smithi006.stdout:0/538: dread db/d3d/d4e/f65 [0,4194304] 0 2024-03-22T06:15:09.619 INFO:tasks.workunit.client.0.smithi006.stdout:0/539: read db/d10/f23 [9201313,55395] 0 2024-03-22T06:15:09.628 INFO:tasks.workunit.client.0.smithi006.stdout:0/540: write db/dc/d37/f77 [1887354,60096] 0 2024-03-22T06:15:09.631 INFO:tasks.workunit.client.0.smithi006.stdout:0/541: mknod db/d3d/cb8 0 2024-03-22T06:15:09.631 INFO:tasks.workunit.client.0.smithi006.stdout:0/542: read db/d10/d33/f4d [156961,18133] 0 2024-03-22T06:15:09.634 INFO:tasks.workunit.client.0.smithi006.stdout:0/543: link db/d10/d33/f42 db/dc/d38/fb9 0 2024-03-22T06:15:09.650 INFO:tasks.workunit.client.0.smithi006.stdout:0/544: dwrite db/d3d/d4e/f9b [0,4194304] 0 2024-03-22T06:15:09.655 INFO:tasks.workunit.client.0.smithi006.stdout:0/545: truncate db/d10/d2a/f5e 502654 0 2024-03-22T06:15:09.658 INFO:tasks.workunit.client.0.smithi006.stdout:0/546: getdents db/d10/d33/d9d 0 2024-03-22T06:15:09.659 INFO:tasks.workunit.client.0.smithi006.stdout:0/547: dread - db/f96 zero size 2024-03-22T06:15:09.659 INFO:tasks.workunit.client.0.smithi006.stdout:0/548: chown db/d10/d2a/d41/l4c 0 1 2024-03-22T06:15:09.660 INFO:tasks.workunit.client.0.smithi006.stdout:0/549: link db/d10/d2a/lb3 db/d3d/d4e/d76/lba 0 2024-03-22T06:15:09.660 INFO:tasks.workunit.client.0.smithi006.stdout:0/550: creat db/d10/d35/d6c/fbb x:0 0 0 2024-03-22T06:15:09.661 INFO:tasks.workunit.client.0.smithi006.stdout:0/551: dread db/d10/d35/d6c/fb2 [0,4194304] 0 2024-03-22T06:15:09.666 INFO:tasks.workunit.client.0.smithi006.stdout:0/552: rename db/d10/d2a/l85 to db/d10/d2a/d41/lbc 0 2024-03-22T06:15:09.686 INFO:tasks.workunit.client.0.smithi006.stdout:0/553: dwrite db/dc/d37/f77 [0,4194304] 0 2024-03-22T06:15:09.849 INFO:tasks.workunit.client.0.smithi006.stdout:0/554: link db/dc/d38/d50/f79 db/d10/d35/d7b/da1/da6/fbd 0 2024-03-22T06:15:09.919 INFO:tasks.workunit.client.0.smithi006.stdout:0/555: rmdir db/d10/d35/d6c/db7 0 2024-03-22T06:15:10.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:10 smithi117 ceph-mon[36514]: pgmap v18: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 108 MiB/s rd, 132 MiB/s wr, 225 op/s 2024-03-22T06:15:10.514 INFO:tasks.workunit.client.0.smithi006.stdout:9/596: write d8/d10/d20/fa0 [457017,56916] 0 2024-03-22T06:15:10.517 INFO:tasks.workunit.client.0.smithi006.stdout:9/597: rename d8/db3/d50/d38/l7e to d8/db3/d50/d38/lc1 0 2024-03-22T06:15:10.519 INFO:tasks.workunit.client.0.smithi006.stdout:9/598: getdents d8/d10/d20 0 2024-03-22T06:15:10.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:10 smithi006 ceph-mon[29301]: pgmap v18: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 108 MiB/s rd, 132 MiB/s wr, 225 op/s 2024-03-22T06:15:10.588 INFO:tasks.workunit.client.0.smithi006.stdout:1/662: rmdir d8/d9/d13 39 2024-03-22T06:15:10.588 INFO:tasks.workunit.client.0.smithi006.stdout:1/663: read - d8/d9/d3a/d18/fc3 zero size 2024-03-22T06:15:10.588 INFO:tasks.workunit.client.0.smithi006.stdout:1/664: readlink d8/l91 0 2024-03-22T06:15:10.612 INFO:tasks.workunit.client.0.smithi006.stdout:3/700: dwrite d0/d8/d1c/d3e/d5f/d90/da7/de7/d33/d3d/f5b [0,4194304] 0 2024-03-22T06:15:10.612 INFO:tasks.workunit.client.0.smithi006.stdout:3/701: chown d0/d8/f1a 84 1 2024-03-22T06:15:10.718 INFO:tasks.workunit.client.0.smithi006.stdout:4/785: dread d6/f12 [0,4194304] 0 2024-03-22T06:15:10.721 INFO:tasks.workunit.client.0.smithi006.stdout:4/786: symlink d6/d1b/d46/l110 0 2024-03-22T06:15:10.721 INFO:tasks.workunit.client.0.smithi006.stdout:4/787: chown d6/d1b/d20/d21/d77/da0/lca 54 1 2024-03-22T06:15:10.721 INFO:tasks.workunit.client.0.smithi006.stdout:4/788: truncate d6/d13/d3b/fe2 332529 0 2024-03-22T06:15:10.729 INFO:tasks.workunit.client.0.smithi006.stdout:4/789: read d6/f4a [717118,3191] 0 2024-03-22T06:15:10.730 INFO:tasks.workunit.client.0.smithi006.stdout:4/790: write d6/f62 [374111,41127] 0 2024-03-22T06:15:10.730 INFO:tasks.workunit.client.0.smithi006.stdout:4/791: chown d6/d1b/d20/l44 448726461 1 2024-03-22T06:15:10.745 INFO:tasks.workunit.client.0.smithi006.stdout:4/792: dwrite d6/d13/d3b/f4e [0,4194304] 0 2024-03-22T06:15:10.745 INFO:tasks.workunit.client.0.smithi006.stdout:4/793: write d6/d1b/d20/d63/d93/fc6 [803311,31696] 0 2024-03-22T06:15:10.753 INFO:tasks.workunit.client.0.smithi006.stdout:4/794: write d6/d1b/d20/d6b/d60/f68 [2877800,80983] 0 2024-03-22T06:15:10.762 INFO:tasks.workunit.client.0.smithi006.stdout:4/795: dread d6/d1b/d20/d6b/f75 [0,4194304] 0 2024-03-22T06:15:10.777 INFO:tasks.workunit.client.0.smithi006.stdout:4/796: dwrite d6/f1c [0,4194304] 0 2024-03-22T06:15:10.785 INFO:tasks.workunit.client.0.smithi006.stdout:4/797: getdents d6/d1b/d20/d63/d93/d9a/d8f/da1 0 2024-03-22T06:15:10.790 INFO:tasks.workunit.client.0.smithi006.stdout:4/798: dread d6/f62 [0,4194304] 0 2024-03-22T06:15:10.795 INFO:tasks.workunit.client.0.smithi006.stdout:4/799: dread d6/d1b/f54 [0,4194304] 0 2024-03-22T06:15:10.798 INFO:tasks.workunit.client.0.smithi006.stdout:4/800: creat d6/d1b/d20/d21/d77/f111 x:0 0 0 2024-03-22T06:15:10.849 INFO:tasks.workunit.client.0.smithi006.stdout:4/801: truncate d6/f30 2850384 0 2024-03-22T06:15:10.855 INFO:tasks.workunit.client.0.smithi006.stdout:4/802: mknod d6/d1b/d20/d21/d77/df1/c112 0 2024-03-22T06:15:10.872 INFO:tasks.workunit.client.0.smithi006.stdout:4/803: dwrite d6/d1b/d20/d63/d93/f102 [0,4194304] 0 2024-03-22T06:15:10.872 INFO:tasks.workunit.client.0.smithi006.stdout:4/804: write d6/d1b/f54 [1331239,57189] 0 2024-03-22T06:15:11.628 INFO:tasks.workunit.client.0.smithi006.stdout:4/805: mknod d6/d1b/d20/d63/d93/d9a/d8f/c113 0 2024-03-22T06:15:11.631 INFO:tasks.workunit.client.0.smithi006.stdout:4/806: mknod d6/c114 0 2024-03-22T06:15:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:4/807: dwrite d6/d1b/d20/d63/d93/d9a/fac [0,4194304] 0 2024-03-22T06:15:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:4/808: symlink d6/d1b/d20/d63/d93/d9a/l115 0 2024-03-22T06:15:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:4/809: stat d6/d1b/d20/d21/d2a/f9e 0 2024-03-22T06:15:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:1/665: rmdir d8/d9/d3a/d18/dd2 39 2024-03-22T06:15:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:1/666: chown d8/d9/d13/d39/d94/fbc 477347 1 2024-03-22T06:15:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:1/667: rmdir d8/d9/d13/d24/d32/d34/daa 39 2024-03-22T06:15:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:1/668: dread d8/d9/d13/d39/f63 [0,4194304] 0 2024-03-22T06:15:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:1/669: write d8/d9/d13/d24/d32/dc7/fc6 [637931,53567] 0 2024-03-22T06:15:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:1/670: creat d8/d9/d3a/d18/dd2/fe3 x:0 0 0 2024-03-22T06:15:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:1/671: creat d8/d9/d3a/d40/d59/fe4 x:0 0 0 2024-03-22T06:15:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:1/672: mknod d8/d9/d10/d7c/ce5 0 2024-03-22T06:15:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:1/673: truncate d8/d9/d10/d7c/f82 1106099 0 2024-03-22T06:15:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:1/674: chown d8/d9/d10/f4e 236 1 2024-03-22T06:15:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:1/675: chown d8/d9/f2d 1195824 1 2024-03-22T06:15:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:1/676: fdatasync d8/d9/f54 0 2024-03-22T06:15:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:1/677: symlink d8/d9/d3a/d40/d59/le6 0 2024-03-22T06:15:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:1/678: fsync d8/d9/d3a/f92 0 2024-03-22T06:15:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:1/679: mknod d8/d9/d3a/d40/d44/ce7 0 2024-03-22T06:15:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:1/680: dwrite d8/d9/d13/d39/d94/fbc [0,4194304] 0 2024-03-22T06:15:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:1/681: dread d8/d9/d3a/d40/d44/f66 [0,4194304] 0 2024-03-22T06:15:11.889 INFO:tasks.workunit.client.0.smithi006.stdout:1/682: write d8/d9/d13/f6d [1127902,33058] 0 2024-03-22T06:15:11.894 INFO:tasks.workunit.client.0.smithi006.stdout:1/683: unlink d8/d9/d3a/d18/c75 0 2024-03-22T06:15:12.202 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:12 smithi006 ceph-mon[29301]: pgmap v19: 65 pgs: 65 active+clean; 1.6 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 117 MiB/s rd, 149 MiB/s wr, 855 op/s 2024-03-22T06:15:12.217 INFO:tasks.workunit.client.0.smithi006.stdout:9/599: mkdir d8/d10/d28/dc2 0 2024-03-22T06:15:12.218 INFO:tasks.workunit.client.0.smithi006.stdout:9/600: dread d8/f64 [0,4194304] 0 2024-03-22T06:15:12.222 INFO:tasks.workunit.client.0.smithi006.stdout:9/601: dread d8/d10/d47/f97 [0,4194304] 0 2024-03-22T06:15:12.222 INFO:tasks.workunit.client.0.smithi006.stdout:9/602: getdents d8/d10/d20/d45/db9 0 2024-03-22T06:15:12.225 INFO:tasks.workunit.client.0.smithi006.stdout:9/603: creat d8/db3/d50/d38/fc3 x:0 0 0 2024-03-22T06:15:12.225 INFO:tasks.workunit.client.0.smithi006.stdout:9/604: truncate d8/d10/d47/f97 1417815 0 2024-03-22T06:15:12.225 INFO:tasks.workunit.client.0.smithi006.stdout:9/605: chown d8/d10/l24 8034 1 2024-03-22T06:15:12.225 INFO:tasks.workunit.client.0.smithi006.stdout:9/606: creat d8/d10/d28/dba/fc4 x:0 0 0 2024-03-22T06:15:12.227 INFO:tasks.workunit.client.0.smithi006.stdout:9/607: creat d8/d10/d82/fc5 x:0 0 0 2024-03-22T06:15:12.227 INFO:tasks.workunit.client.0.smithi006.stdout:9/608: write d8/d10/d28/f3d [4983776,18349] 0 2024-03-22T06:15:12.234 INFO:tasks.workunit.client.0.smithi006.stdout:9/609: creat d8/d10/d20/d45/db9/fc6 x:0 0 0 2024-03-22T06:15:12.235 INFO:tasks.workunit.client.0.smithi006.stdout:9/610: unlink d8/d10/d28/f55 0 2024-03-22T06:15:12.239 INFO:tasks.workunit.client.0.smithi006.stdout:9/611: write d8/d10/d28/f25 [2421902,9140] 0 2024-03-22T06:15:12.239 INFO:tasks.workunit.client.0.smithi006.stdout:9/612: fsync d8/d10/d23/f40 0 2024-03-22T06:15:12.239 INFO:tasks.workunit.client.0.smithi006.stdout:9/613: chown d8/d10/l54 17490 1 2024-03-22T06:15:12.239 INFO:tasks.workunit.client.0.smithi006.stdout:9/614: creat d8/d10/d20/d45/d49/fc7 x:0 0 0 2024-03-22T06:15:12.259 INFO:tasks.workunit.client.0.smithi006.stdout:9/615: dwrite d8/d10/d28/f25 [0,4194304] 0 2024-03-22T06:15:12.262 INFO:tasks.workunit.client.0.smithi006.stdout:3/702: rename d0/d8/d1c/d3e/d5f/d90/da7/de7/d33/d3d to d0/d8/d88/ddf/dec 0 2024-03-22T06:15:12.262 INFO:tasks.workunit.client.0.smithi006.stdout:3/703: fdatasync d0/d8/d88/ddf/dec/dcb/fea 0 2024-03-22T06:15:12.262 INFO:tasks.workunit.client.0.smithi006.stdout:3/704: creat d0/d4/d53/de0/fed x:0 0 0 2024-03-22T06:15:12.262 INFO:tasks.workunit.client.0.smithi006.stdout:3/705: write d0/d8/d1c/d3e/d5f/fa1 [423838,114884] 0 2024-03-22T06:15:12.263 INFO:tasks.workunit.client.0.smithi006.stdout:4/810: rename d6/f7 to d6/d1b/d20/d63/f116 0 2024-03-22T06:15:12.263 INFO:tasks.workunit.client.0.smithi006.stdout:4/811: readlink d6/d1b/d20/d21/d2a/d43/d4b/lb7 0 2024-03-22T06:15:12.265 INFO:tasks.workunit.client.0.smithi006.stdout:4/812: symlink d6/d1b/d46/d101/l117 0 2024-03-22T06:15:12.276 INFO:tasks.workunit.client.0.smithi006.stdout:3/706: dread d0/d8/d1c/d3e/d5f/d90/da7/de7/fe4 [0,4194304] 0 2024-03-22T06:15:12.276 INFO:tasks.workunit.client.0.smithi006.stdout:4/813: dread d6/d1b/f2d [0,4194304] 0 2024-03-22T06:15:12.276 INFO:tasks.workunit.client.0.smithi006.stdout:4/814: creat d6/d1b/d20/d21/d2a/d43/d4b/f118 x:0 0 0 2024-03-22T06:15:12.277 INFO:tasks.workunit.client.0.smithi006.stdout:9/616: dwrite d8/d10/d20/d45/f9b [0,4194304] 0 2024-03-22T06:15:12.277 INFO:tasks.workunit.client.0.smithi006.stdout:9/617: creat d8/fc8 x:0 0 0 2024-03-22T06:15:12.278 INFO:tasks.workunit.client.0.smithi006.stdout:9/618: fsync d8/d1f/f5d 0 2024-03-22T06:15:12.278 INFO:tasks.workunit.client.0.smithi006.stdout:9/619: write d8/d1f/d34/f88 [342916,117510] 0 2024-03-22T06:15:12.278 INFO:tasks.workunit.client.0.smithi006.stdout:9/620: fsync d8/d10/d28/d39/f46 0 2024-03-22T06:15:12.280 INFO:tasks.workunit.client.0.smithi006.stdout:9/621: rename d8/d1f/d34/ca8 to d8/d10/d28/dc2/cc9 0 2024-03-22T06:15:12.281 INFO:tasks.workunit.client.0.smithi006.stdout:9/622: mknod d8/d10/d28/dba/cca 0 2024-03-22T06:15:12.281 INFO:tasks.workunit.client.0.smithi006.stdout:9/623: creat d8/d10/d20/d45/db9/fcb x:0 0 0 2024-03-22T06:15:12.284 INFO:tasks.workunit.client.0.smithi006.stdout:9/624: creat d8/d10/d47/dbc/fcc x:0 0 0 2024-03-22T06:15:12.284 INFO:tasks.workunit.client.0.smithi006.stdout:9/625: chown d8/d10/d11/c41 0 1 2024-03-22T06:15:12.285 INFO:tasks.workunit.client.0.smithi006.stdout:3/707: read d0/d8/d88/ddf/dec/f86 [1010479,117785] 0 2024-03-22T06:15:12.288 INFO:tasks.workunit.client.0.smithi006.stdout:3/708: rmdir d0/d8/d1c/d72 39 2024-03-22T06:15:12.288 INFO:tasks.workunit.client.0.smithi006.stdout:3/709: read - d0/d4/d3a/fde zero size 2024-03-22T06:15:12.289 INFO:tasks.workunit.client.0.smithi006.stdout:3/710: mknod d0/d8/cee 0 2024-03-22T06:15:12.292 INFO:tasks.workunit.client.0.smithi006.stdout:3/711: rename d0/d8/d88/ddf/dec/d58/ca8 to d0/d8/d1c/d72/cef 0 2024-03-22T06:15:12.292 INFO:tasks.workunit.client.0.smithi006.stdout:3/712: chown d0/d4/d31/d98/dab/fd5 532084 1 2024-03-22T06:15:12.292 INFO:tasks.workunit.client.0.smithi006.stdout:9/626: dread d8/f7c [0,4194304] 0 2024-03-22T06:15:12.293 INFO:tasks.workunit.client.0.smithi006.stdout:3/713: unlink d0/d7e/l73 0 2024-03-22T06:15:12.294 INFO:tasks.workunit.client.0.smithi006.stdout:9/627: creat d8/d10/d20/d45/fcd x:0 0 0 2024-03-22T06:15:12.295 INFO:tasks.workunit.client.0.smithi006.stdout:3/714: getdents d0/d8/d1c/d3e/d5f/da3 0 2024-03-22T06:15:12.295 INFO:tasks.workunit.client.0.smithi006.stdout:3/715: truncate d0/d4/d3a/f7c 628407 0 2024-03-22T06:15:12.295 INFO:tasks.workunit.client.0.smithi006.stdout:3/716: fdatasync d0/d8/d1c/d72/f94 0 2024-03-22T06:15:12.295 INFO:tasks.workunit.client.0.smithi006.stdout:3/717: write d0/d8/d1c/d3e/d5f/d90/da7/de7/d4c/f35 [8610379,81813] 0 2024-03-22T06:15:12.295 INFO:tasks.workunit.client.0.smithi006.stdout:9/628: rmdir d8/d10/d20 39 2024-03-22T06:15:12.295 INFO:tasks.workunit.client.0.smithi006.stdout:9/629: stat d8/d1f/d34/fbf 0 2024-03-22T06:15:12.297 INFO:tasks.workunit.client.0.smithi006.stdout:3/718: mkdir d0/d8/d88/ddf/df0 0 2024-03-22T06:15:12.298 INFO:tasks.workunit.client.0.smithi006.stdout:9/630: rename d8/d10/d23/f27 to d8/d10/d28/fce 0 2024-03-22T06:15:12.298 INFO:tasks.workunit.client.0.smithi006.stdout:9/631: chown d8/d10/d82 615787229 1 2024-03-22T06:15:12.299 INFO:tasks.workunit.client.0.smithi006.stdout:9/632: rmdir d8/d10/d11 39 2024-03-22T06:15:12.301 INFO:tasks.workunit.client.0.smithi006.stdout:9/633: mknod d8/db3/d50/ccf 0 2024-03-22T06:15:12.302 INFO:tasks.workunit.client.0.smithi006.stdout:9/634: creat d8/d10/d20/d45/d4d/fd0 x:0 0 0 2024-03-22T06:15:12.315 INFO:tasks.workunit.client.0.smithi006.stdout:4/815: creat d6/d1b/d20/d21/d2a/f119 x:0 0 0 2024-03-22T06:15:12.322 INFO:tasks.workunit.client.0.smithi006.stdout:4/816: dread d6/d1b/d20/d63/fba [0,4194304] 0 2024-03-22T06:15:12.322 INFO:tasks.workunit.client.0.smithi006.stdout:9/635: dwrite d8/d10/d20/d45/d4d/f85 [0,4194304] 0 2024-03-22T06:15:12.324 INFO:tasks.workunit.client.0.smithi006.stdout:4/817: rename d6/d1b/d20/d6b/d88/c50 to d6/d1b/d20/d63/d93/d9a/d8f/da1/db5/c11a 0 2024-03-22T06:15:12.343 INFO:tasks.workunit.client.0.smithi006.stdout:4/818: dwrite d6/d1b/d20/f22 [0,4194304] 0 2024-03-22T06:15:12.351 INFO:tasks.workunit.client.0.smithi006.stdout:0/556: symlink db/dc/d38/lbe 0 2024-03-22T06:15:12.351 INFO:tasks.workunit.client.0.smithi006.stdout:0/557: truncate db/dc/d38/fb9 3059060 0 2024-03-22T06:15:12.372 INFO:tasks.workunit.client.0.smithi006.stdout:0/558: dwrite db/d10/d33/f91 [0,4194304] 0 2024-03-22T06:15:12.375 INFO:tasks.workunit.client.0.smithi006.stdout:0/559: getdents db/d3d/d4e/d76 0 2024-03-22T06:15:12.375 INFO:tasks.workunit.client.0.smithi006.stdout:0/560: write db/d10/d33/fb0 [1121823,71062] 0 2024-03-22T06:15:12.375 INFO:tasks.workunit.client.0.smithi006.stdout:0/561: write db/d10/d35/f69 [1851601,56159] 0 2024-03-22T06:15:12.377 INFO:tasks.workunit.client.0.smithi006.stdout:0/562: rename db/d3d/d4e/ca0 to db/d10/d33/d9d/cbf 0 2024-03-22T06:15:12.378 INFO:tasks.workunit.client.0.smithi006.stdout:0/563: getdents db/d10/d33 0 2024-03-22T06:15:12.381 INFO:tasks.workunit.client.0.smithi006.stdout:0/564: creat db/dc/d38/fc0 x:0 0 0 2024-03-22T06:15:12.410 INFO:tasks.workunit.client.0.smithi006.stdout:1/684: rmdir d8/d9/d3a/d18 39 2024-03-22T06:15:12.427 INFO:tasks.workunit.client.0.smithi006.stdout:1/685: dwrite d8/d9/d13/d24/d32/fcf [0,4194304] 0 2024-03-22T06:15:12.427 INFO:tasks.workunit.client.0.smithi006.stdout:1/686: write d8/d9/d13/d39/d94/fbc [4977694,18970] 0 2024-03-22T06:15:12.430 INFO:tasks.workunit.client.0.smithi006.stdout:1/687: getdents d8/d9/d13/d24/d32/d34 0 2024-03-22T06:15:12.445 INFO:tasks.workunit.client.0.smithi006.stdout:1/688: dwrite d8/d9/d3a/f6b [0,4194304] 0 2024-03-22T06:15:12.458 INFO:tasks.workunit.client.0.smithi006.stdout:1/689: dwrite d8/d9/d3a/f6b [0,4194304] 0 2024-03-22T06:15:12.458 INFO:tasks.workunit.client.0.smithi006.stdout:1/690: read - d8/d9/d10/d7c/fa8 zero size 2024-03-22T06:15:12.459 INFO:tasks.workunit.client.0.smithi006.stdout:1/691: creat d8/d9/d13/d24/d32/dc7/fe8 x:0 0 0 2024-03-22T06:15:12.459 INFO:tasks.workunit.client.0.smithi006.stdout:1/692: rmdir d8/d9/d13/d39/d60/d8f 39 2024-03-22T06:15:12.461 INFO:tasks.workunit.client.0.smithi006.stdout:1/693: mknod d8/d71/dcd/ce9 0 2024-03-22T06:15:12.461 INFO:tasks.workunit.client.0.smithi006.stdout:1/694: fdatasync d8/d9/d3a/d40/fd0 0 2024-03-22T06:15:12.484 INFO:tasks.workunit.client.0.smithi006.stdout:1/695: dwrite d8/d9/d13/f26 [0,4194304] 0 2024-03-22T06:15:12.485 INFO:tasks.workunit.client.0.smithi006.stdout:1/696: mkdir d8/d9/d13/d39/d60/d8f/dea 0 2024-03-22T06:15:12.486 INFO:tasks.workunit.client.0.smithi006.stdout:1/697: symlink d8/d9/d10/d7c/d96/leb 0 2024-03-22T06:15:12.486 INFO:tasks.workunit.client.0.smithi006.stdout:1/698: truncate d8/d9/d10/d7c/d96/fba 255180 0 2024-03-22T06:15:12.488 INFO:tasks.workunit.client.0.smithi006.stdout:1/699: creat d8/d9/d13/d39/d60/fec x:0 0 0 2024-03-22T06:15:12.497 INFO:tasks.workunit.client.0.smithi006.stdout:1/700: write d8/d9/d13/d24/d32/d34/f77 [3945849,122242] 0 2024-03-22T06:15:12.497 INFO:tasks.workunit.client.0.smithi006.stdout:1/701: creat d8/d71/fed x:0 0 0 2024-03-22T06:15:12.497 INFO:tasks.workunit.client.0.smithi006.stdout:1/702: read d8/fd8 [2501205,114987] 0 2024-03-22T06:15:12.500 INFO:tasks.workunit.client.0.smithi006.stdout:1/703: symlink d8/d9/d3a/d40/d59/db1/lee 0 2024-03-22T06:15:12.501 INFO:tasks.workunit.client.0.smithi006.stdout:1/704: chown d8/d9/d3a/c86 0 1 2024-03-22T06:15:12.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:12 smithi117 ceph-mon[36514]: pgmap v19: 65 pgs: 65 active+clean; 1.6 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 117 MiB/s rd, 149 MiB/s wr, 855 op/s 2024-03-22T06:15:12.508 INFO:tasks.workunit.client.0.smithi006.stdout:4/819: getdents d6/d1b/d20/d63 0 2024-03-22T06:15:12.508 INFO:tasks.workunit.client.0.smithi006.stdout:4/820: fdatasync f2 0 2024-03-22T06:15:12.508 INFO:tasks.workunit.client.0.smithi006.stdout:4/821: dread - d6/d1b/d20/d63/d93/db8/dbc/f109 zero size 2024-03-22T06:15:12.510 INFO:tasks.workunit.client.0.smithi006.stdout:4/822: creat d6/d1b/d20/d21/d9f/f11b x:0 0 0 2024-03-22T06:15:12.523 INFO:tasks.workunit.client.0.smithi006.stdout:4/823: dwrite d6/d1b/d20/d6b/d95/ff3 [0,4194304] 0 2024-03-22T06:15:12.523 INFO:tasks.workunit.client.0.smithi006.stdout:4/824: creat d6/d13/d3b/f11c x:0 0 0 2024-03-22T06:15:12.527 INFO:tasks.workunit.client.0.smithi006.stdout:3/719: truncate d0/d8/d1c/d3e/d5f/fa1 616453 0 2024-03-22T06:15:12.530 INFO:tasks.workunit.client.0.smithi006.stdout:4/825: mkdir d6/d1b/d20/d21/d11d 0 2024-03-22T06:15:12.535 INFO:tasks.workunit.client.0.smithi006.stdout:4/826: mknod d6/d1b/d20/d21/d11d/c11e 0 2024-03-22T06:15:12.535 INFO:tasks.workunit.client.0.smithi006.stdout:4/827: fsync d6/d1b/f54 0 2024-03-22T06:15:12.535 INFO:tasks.workunit.client.0.smithi006.stdout:3/720: write d0/d8/d1c/d49/f5e [1544436,108808] 0 2024-03-22T06:15:12.537 INFO:tasks.workunit.client.0.smithi006.stdout:9/636: truncate d8/d10/f12 3468273 0 2024-03-22T06:15:12.539 INFO:tasks.workunit.client.0.smithi006.stdout:8/833: sync 2024-03-22T06:15:12.539 INFO:tasks.workunit.client.0.smithi006.stdout:2/736: sync 2024-03-22T06:15:12.539 INFO:tasks.workunit.client.0.smithi006.stdout:6/622: sync 2024-03-22T06:15:12.539 INFO:tasks.workunit.client.0.smithi006.stdout:5/834: sync 2024-03-22T06:15:12.539 INFO:tasks.workunit.client.0.smithi006.stdout:6/623: dread - d6/f9c zero size 2024-03-22T06:15:12.539 INFO:tasks.workunit.client.0.smithi006.stdout:5/835: readlink d0/db/d16/d1c/d34/d35/d5d/d9d/ld2 0 2024-03-22T06:15:12.539 INFO:tasks.workunit.client.0.smithi006.stdout:6/624: fsync d6/d24/d44/f4d 0 2024-03-22T06:15:12.539 INFO:tasks.workunit.client.0.smithi006.stdout:2/737: write d13/d53/d6b/fb5 [271170,49850] 0 2024-03-22T06:15:12.539 INFO:tasks.workunit.client.0.smithi006.stdout:7/741: sync 2024-03-22T06:15:12.541 INFO:tasks.workunit.client.0.smithi006.stdout:4/828: creat d6/d1b/d20/d63/d93/d9a/dfd/f11f x:0 0 0 2024-03-22T06:15:12.547 INFO:tasks.workunit.client.0.smithi006.stdout:3/721: truncate d0/d8/d1c/d3e/d5f/da3/fc8 395534 0 2024-03-22T06:15:12.548 INFO:tasks.workunit.client.0.smithi006.stdout:9/637: truncate d8/d10/d28/fce 782805 0 2024-03-22T06:15:12.552 INFO:tasks.workunit.client.0.smithi006.stdout:6/625: mkdir d6/d7c/dc7 0 2024-03-22T06:15:12.553 INFO:tasks.workunit.client.0.smithi006.stdout:9/638: dread d8/d10/d11/f3c [0,4194304] 0 2024-03-22T06:15:12.554 INFO:tasks.workunit.client.0.smithi006.stdout:5/836: mkdir d0/db/d76/db4/dfe 0 2024-03-22T06:15:12.558 INFO:tasks.workunit.client.0.smithi006.stdout:2/738: link d13/d34/d4d/fbf d13/d20/d26/dca/def/f113 0 2024-03-22T06:15:12.559 INFO:tasks.workunit.client.0.smithi006.stdout:2/739: chown d13/d53 384277219 1 2024-03-22T06:15:12.561 INFO:tasks.workunit.client.0.smithi006.stdout:7/742: mknod d5/d1e/d29/d59/cf2 0 2024-03-22T06:15:12.564 INFO:tasks.workunit.client.0.smithi006.stdout:1/705: getdents d8/d9/d13/d39/d60 0 2024-03-22T06:15:12.566 INFO:tasks.workunit.client.0.smithi006.stdout:6/626: dread d6/d1d/d99/fb1 [0,4194304] 0 2024-03-22T06:15:12.566 INFO:tasks.workunit.client.0.smithi006.stdout:6/627: creat d6/d7c/d92/fc8 x:0 0 0 2024-03-22T06:15:12.566 INFO:tasks.workunit.client.0.smithi006.stdout:6/628: fdatasync d6/f18 0 2024-03-22T06:15:12.567 INFO:tasks.workunit.client.0.smithi006.stdout:2/740: dread d13/d20/f52 [0,4194304] 0 2024-03-22T06:15:12.568 INFO:tasks.workunit.client.0.smithi006.stdout:4/829: dwrite d6/f1f [0,4194304] 0 2024-03-22T06:15:12.568 INFO:tasks.workunit.client.0.smithi006.stdout:4/830: creat d6/d1b/d20/d21/d2a/d43/d4b/f120 x:0 0 0 2024-03-22T06:15:12.568 INFO:tasks.workunit.client.0.smithi006.stdout:9/639: dwrite d8/d10/d28/dba/fc4 [0,4194304] 0 2024-03-22T06:15:12.568 INFO:tasks.workunit.client.0.smithi006.stdout:9/640: stat d8/d10/c99 0 2024-03-22T06:15:12.568 INFO:tasks.workunit.client.0.smithi006.stdout:9/641: chown d8/d10/d47/dbc 2 1 2024-03-22T06:15:12.572 INFO:tasks.workunit.client.0.smithi006.stdout:5/837: fdatasync d0/db/d16/f30 0 2024-03-22T06:15:12.573 INFO:tasks.workunit.client.0.smithi006.stdout:6/629: dread d6/d1b/d2c/f74 [0,4194304] 0 2024-03-22T06:15:12.579 INFO:tasks.workunit.client.0.smithi006.stdout:7/743: unlink d5/d1e/d29/d1f/d2d/d61/fbf 0 2024-03-22T06:15:12.579 INFO:tasks.workunit.client.0.smithi006.stdout:7/744: fdatasync d5/d1e/d29/d1f/d2d/d61/f6d 0 2024-03-22T06:15:12.579 INFO:tasks.workunit.client.0.smithi006.stdout:7/745: write d5/d1e/d29/d1f/d2d/d41/fd2 [665916,62516] 0 2024-03-22T06:15:12.580 INFO:tasks.workunit.client.0.smithi006.stdout:1/706: rmdir d8/d9/d10/d7c/dd6 0 2024-03-22T06:15:12.580 INFO:tasks.workunit.client.0.smithi006.stdout:1/707: fsync d8/d9/d13/d39/d94/fbc 0 2024-03-22T06:15:12.584 INFO:tasks.workunit.client.0.smithi006.stdout:2/741: symlink d13/d20/d26/dca/l114 0 2024-03-22T06:15:12.585 INFO:tasks.workunit.client.0.smithi006.stdout:2/742: dread - d13/d34/d4d/d4e/d75/d62/fb6 zero size 2024-03-22T06:15:12.585 INFO:tasks.workunit.client.0.smithi006.stdout:4/831: dwrite d6/d1b/d20/d21/d2a/f9e [0,4194304] 0 2024-03-22T06:15:12.585 INFO:tasks.workunit.client.0.smithi006.stdout:4/832: read d6/d1b/d20/d63/fba [3603358,43509] 0 2024-03-22T06:15:12.589 INFO:tasks.workunit.client.0.smithi006.stdout:8/834: rename d9/d14/d44 to d9/db/d35/dd2/d76/db3/d132 0 2024-03-22T06:15:12.599 INFO:tasks.workunit.client.0.smithi006.stdout:5/838: mkdir d0/db/d50/d58/dad/dff 0 2024-03-22T06:15:12.599 INFO:tasks.workunit.client.0.smithi006.stdout:5/839: chown d0/db/dc/f8f 118 1 2024-03-22T06:15:12.599 INFO:tasks.workunit.client.0.smithi006.stdout:5/840: chown d0/db/d16/d1c/d34/d35/d85/d5e/d66/dc0/lfc 120 1 2024-03-22T06:15:12.600 INFO:tasks.workunit.client.0.smithi006.stdout:4/833: dwrite d6/fb [0,4194304] 0 2024-03-22T06:15:12.600 INFO:tasks.workunit.client.0.smithi006.stdout:6/630: getdents d6/d1d/d5c/d81/d8a 0 2024-03-22T06:15:12.603 INFO:tasks.workunit.client.0.smithi006.stdout:7/746: link d5/d1e/d29/d1f/l57 d5/d1e/d29/db9/lf3 0 2024-03-22T06:15:12.606 INFO:tasks.workunit.client.0.smithi006.stdout:1/708: mkdir d8/d9/d13/def 0 2024-03-22T06:15:12.611 INFO:tasks.workunit.client.0.smithi006.stdout:2/743: creat d13/d34/d4d/d4e/d105/f115 x:0 0 0 2024-03-22T06:15:12.617 INFO:tasks.workunit.client.0.smithi006.stdout:9/642: rename d8/f64 to d8/d10/d28/fd1 0 2024-03-22T06:15:12.618 INFO:tasks.workunit.client.0.smithi006.stdout:8/835: truncate d9/fb5 629074 0 2024-03-22T06:15:12.618 INFO:tasks.workunit.client.0.smithi006.stdout:8/836: creat d9/d14/dcc/f133 x:0 0 0 2024-03-22T06:15:12.620 INFO:tasks.workunit.client.0.smithi006.stdout:6/631: link d6/d1d/d5c/d89/cbf d6/d7c/dc7/cc9 0 2024-03-22T06:15:12.621 INFO:tasks.workunit.client.0.smithi006.stdout:4/834: dwrite d6/f9 [8388608,4194304] 0 2024-03-22T06:15:12.623 INFO:tasks.workunit.client.0.smithi006.stdout:7/747: dwrite d5/d1e/fcc [0,4194304] 0 2024-03-22T06:15:12.623 INFO:tasks.workunit.client.0.smithi006.stdout:7/748: chown d5/d1e/d29/d66/d91 520052411 1 2024-03-22T06:15:12.627 INFO:tasks.workunit.client.0.smithi006.stdout:2/744: creat d13/f116 x:0 0 0 2024-03-22T06:15:12.627 INFO:tasks.workunit.client.0.smithi006.stdout:2/745: write d13/d53/d6b/dd8/fe6 [147696,125954] 0 2024-03-22T06:15:12.632 INFO:tasks.workunit.client.0.smithi006.stdout:5/841: rename d0/db/d16/d1c/d34/d35/d85/d5e/d66/dc0/fa2 to d0/d98/f100 0 2024-03-22T06:15:12.633 INFO:tasks.workunit.client.0.smithi006.stdout:8/837: dwrite d9/db/d12/d66/f7f [0,4194304] 0 2024-03-22T06:15:12.635 INFO:tasks.workunit.client.0.smithi006.stdout:9/643: mknod d8/d10/d28/d39/cd2 0 2024-03-22T06:15:12.639 INFO:tasks.workunit.client.0.smithi006.stdout:6/632: readlink d6/d1b/l3e 0 2024-03-22T06:15:12.644 INFO:tasks.workunit.client.0.smithi006.stdout:7/749: mknod d5/d1e/d29/d66/d91/ddc/cf4 0 2024-03-22T06:15:12.648 INFO:tasks.workunit.client.0.smithi006.stdout:1/709: rename d8/d9/d3a/d18/l87 to d8/d9/d3a/lf0 0 2024-03-22T06:15:12.650 INFO:tasks.workunit.client.0.smithi006.stdout:5/842: symlink d0/db/d16/d1c/d34/d4d/l101 0 2024-03-22T06:15:12.650 INFO:tasks.workunit.client.0.smithi006.stdout:5/843: readlink d0/db/d16/d43/ld0 0 2024-03-22T06:15:12.651 INFO:tasks.workunit.client.0.smithi006.stdout:5/844: dread - d0/db/d16/d1c/d34/d35/d85/d5e/dfb/ffd zero size 2024-03-22T06:15:12.655 INFO:tasks.workunit.client.0.smithi006.stdout:8/838: mknod d9/db/d35/c134 0 2024-03-22T06:15:12.655 INFO:tasks.workunit.client.0.smithi006.stdout:2/746: read fc [2988510,62294] 0 2024-03-22T06:15:12.655 INFO:tasks.workunit.client.0.smithi006.stdout:9/644: creat d8/db3/d50/fd3 x:0 0 0 2024-03-22T06:15:12.657 INFO:tasks.workunit.client.0.smithi006.stdout:2/747: dread d13/d34/d4d/d4e/d75/da5/d6c/f78 [0,4194304] 0 2024-03-22T06:15:12.658 INFO:tasks.workunit.client.0.smithi006.stdout:6/633: creat d6/d1b/d2c/fca x:0 0 0 2024-03-22T06:15:12.659 INFO:tasks.workunit.client.0.smithi006.stdout:6/634: creat d6/d24/d44/fcb x:0 0 0 2024-03-22T06:15:12.659 INFO:tasks.workunit.client.0.smithi006.stdout:2/748: dread d13/d20/d26/dca/def/f113 [0,4194304] 0 2024-03-22T06:15:12.660 INFO:tasks.workunit.client.0.smithi006.stdout:7/750: creat d5/d1e/d29/d1f/d2d/d41/ff5 x:0 0 0 2024-03-22T06:15:12.660 INFO:tasks.workunit.client.0.smithi006.stdout:7/751: dread - d5/d1e/d29/d1f/d2d/d41/ff5 zero size 2024-03-22T06:15:12.662 INFO:tasks.workunit.client.0.smithi006.stdout:1/710: mkdir d8/d9/d13/d39/d94/db5/df1 0 2024-03-22T06:15:12.664 INFO:tasks.workunit.client.0.smithi006.stdout:5/845: creat d0/db/d50/d58/dad/f102 x:0 0 0 2024-03-22T06:15:12.665 INFO:tasks.workunit.client.0.smithi006.stdout:5/846: dread - d0/db/d50/db9/fbe zero size 2024-03-22T06:15:12.665 INFO:tasks.workunit.client.0.smithi006.stdout:5/847: fsync d0/db/d16/d1c/f4c 0 2024-03-22T06:15:12.666 INFO:tasks.workunit.client.0.smithi006.stdout:5/848: read d0/db/d16/d1c/d34/fa6 [113974,18444] 0 2024-03-22T06:15:12.666 INFO:tasks.workunit.client.0.smithi006.stdout:5/849: chown d0/db/d16/d1c/d34/d35/d85/d5e/d66/dc0 0 1 2024-03-22T06:15:12.667 INFO:tasks.workunit.client.0.smithi006.stdout:8/839: symlink d9/db/d35/dd2/l135 0 2024-03-22T06:15:12.672 INFO:tasks.workunit.client.0.smithi006.stdout:1/711: dread d8/d9/f35 [0,4194304] 0 2024-03-22T06:15:12.672 INFO:tasks.workunit.client.0.smithi006.stdout:1/712: chown l4 3379 1 2024-03-22T06:15:12.672 INFO:tasks.workunit.client.0.smithi006.stdout:1/713: chown d8/d9/d3a/d18/c2c 315 1 2024-03-22T06:15:12.672 INFO:tasks.workunit.client.0.smithi006.stdout:1/714: chown d8/d9/d13/d24/d32/dc7 295400138 1 2024-03-22T06:15:12.672 INFO:tasks.workunit.client.0.smithi006.stdout:2/749: symlink d13/d20/d26/l117 0 2024-03-22T06:15:12.674 INFO:tasks.workunit.client.0.smithi006.stdout:7/752: dread d5/d1e/d29/d59/f7a [0,4194304] 0 2024-03-22T06:15:12.674 INFO:tasks.workunit.client.0.smithi006.stdout:9/645: dread d8/db3/d50/f8e [0,4194304] 0 2024-03-22T06:15:12.676 INFO:tasks.workunit.client.0.smithi006.stdout:8/840: creat d9/db/d35/dd2/d76/db3/d132/f136 x:0 0 0 2024-03-22T06:15:12.678 INFO:tasks.workunit.client.0.smithi006.stdout:1/715: truncate d8/d9/d3a/d40/d44/d80/fab 1770707 0 2024-03-22T06:15:12.679 INFO:tasks.workunit.client.0.smithi006.stdout:1/716: write d8/d9/d13/d39/d60/f6e [396380,83442] 0 2024-03-22T06:15:12.681 INFO:tasks.workunit.client.0.smithi006.stdout:2/750: creat d13/d20/d26/dca/def/f118 x:0 0 0 2024-03-22T06:15:12.683 INFO:tasks.workunit.client.0.smithi006.stdout:6/635: dwrite d6/d24/d44/f46 [0,4194304] 0 2024-03-22T06:15:12.683 INFO:tasks.workunit.client.0.smithi006.stdout:6/636: chown d6/d1d/d5c/d81/d8a/dac/cba 503 1 2024-03-22T06:15:12.683 INFO:tasks.workunit.client.0.smithi006.stdout:7/753: creat d5/d1e/d29/d66/ff6 x:0 0 0 2024-03-22T06:15:12.683 INFO:tasks.workunit.client.0.smithi006.stdout:7/754: stat d5/d1e/d29/d66 0 2024-03-22T06:15:12.685 INFO:tasks.workunit.client.0.smithi006.stdout:5/850: rename d0/db/d50/d96/d52/d69 to d0/db/d50/d82/d103 0 2024-03-22T06:15:12.687 INFO:tasks.workunit.client.0.smithi006.stdout:7/755: dread d5/d1e/d3f/f90 [0,4194304] 0 2024-03-22T06:15:12.687 INFO:tasks.workunit.client.0.smithi006.stdout:7/756: chown d5/d1e/d29/d59/fd7 14773 1 2024-03-22T06:15:12.690 INFO:tasks.workunit.client.0.smithi006.stdout:8/841: mkdir d9/db/d12/d137 0 2024-03-22T06:15:12.693 INFO:tasks.workunit.client.0.smithi006.stdout:1/717: mknod d8/d9/d13/cf2 0 2024-03-22T06:15:12.693 INFO:tasks.workunit.client.0.smithi006.stdout:1/718: fdatasync d8/d9/d3a/f92 0 2024-03-22T06:15:12.694 INFO:tasks.workunit.client.0.smithi006.stdout:5/851: dread d0/db/d16/d1c/d34/d35/d85/f47 [0,4194304] 0 2024-03-22T06:15:12.700 INFO:tasks.workunit.client.0.smithi006.stdout:9/646: getdents d8/d10/d20/d45/d49 0 2024-03-22T06:15:12.701 INFO:tasks.workunit.client.0.smithi006.stdout:6/637: dwrite d6/d1b/f97 [0,4194304] 0 2024-03-22T06:15:12.703 INFO:tasks.workunit.client.0.smithi006.stdout:8/842: link d9/fdc d9/db/d12/d66/d10c/d122/f138 0 2024-03-22T06:15:12.703 INFO:tasks.workunit.client.0.smithi006.stdout:8/843: chown d9/db/d35/dd2/d4a/da7/fef 18517922 1 2024-03-22T06:15:12.706 INFO:tasks.workunit.client.0.smithi006.stdout:1/719: mknod d8/d9/d10/d7c/cf3 0 2024-03-22T06:15:12.708 INFO:tasks.workunit.client.0.smithi006.stdout:5/852: dread d0/db/d16/f21 [0,4194304] 0 2024-03-22T06:15:12.710 INFO:tasks.workunit.client.0.smithi006.stdout:2/751: rename c8 to d13/d34/d4d/d4e/d105/c119 0 2024-03-22T06:15:12.710 INFO:tasks.workunit.client.0.smithi006.stdout:2/752: read d13/d53/dc5/fd7 [700611,97990] 0 2024-03-22T06:15:12.710 INFO:tasks.workunit.client.0.smithi006.stdout:2/753: read d13/d53/d6b/dd8/fe6 [230745,102736] 0 2024-03-22T06:15:12.716 INFO:tasks.workunit.client.0.smithi006.stdout:5/853: symlink d0/db/d16/d1c/d34/d35/d85/d5e/dfb/l104 0 2024-03-22T06:15:12.716 INFO:tasks.workunit.client.0.smithi006.stdout:9/647: write d8/d10/d28/f44 [1369028,54561] 0 2024-03-22T06:15:12.717 INFO:tasks.workunit.client.0.smithi006.stdout:6/638: dread d6/d7c/d92/faa [0,4194304] 0 2024-03-22T06:15:12.717 INFO:tasks.workunit.client.0.smithi006.stdout:6/639: write d6/f51 [5709848,54230] 0 2024-03-22T06:15:12.720 INFO:tasks.workunit.client.0.smithi006.stdout:7/757: rename d5/d1e/d29/d59/dcd to d5/d1e/d29/d1f/d2d/df7 0 2024-03-22T06:15:12.721 INFO:tasks.workunit.client.0.smithi006.stdout:7/758: fdatasync d5/d1e/d3f/f81 0 2024-03-22T06:15:12.721 INFO:tasks.workunit.client.0.smithi006.stdout:7/759: dread - d5/d1e/d29/d66/ff6 zero size 2024-03-22T06:15:12.721 INFO:tasks.workunit.client.0.smithi006.stdout:7/760: write d5/f5f [4948415,4715] 0 2024-03-22T06:15:12.723 INFO:tasks.workunit.client.0.smithi006.stdout:2/754: mkdir d13/d34/d4d/d11a 0 2024-03-22T06:15:12.725 INFO:tasks.workunit.client.0.smithi006.stdout:5/854: mknod d0/db/d16/d1c/c105 0 2024-03-22T06:15:12.726 INFO:tasks.workunit.client.0.smithi006.stdout:8/844: dwrite d9/db/d35/dd2/d4a/da7/fef [4194304,4194304] 0 2024-03-22T06:15:12.726 INFO:tasks.workunit.client.0.smithi006.stdout:8/845: fsync d9/db/d35/da9/df6/ff7 0 2024-03-22T06:15:12.727 INFO:tasks.workunit.client.0.smithi006.stdout:9/648: symlink d8/d10/d11/dc0/ld4 0 2024-03-22T06:15:12.730 INFO:tasks.workunit.client.0.smithi006.stdout:6/640: write d6/d1d/d99/fb1 [2537214,46772] 0 2024-03-22T06:15:12.731 INFO:tasks.workunit.client.0.smithi006.stdout:7/761: unlink d5/d1e/d29/d1f/d2d/d55/fb8 0 2024-03-22T06:15:12.731 INFO:tasks.workunit.client.0.smithi006.stdout:7/762: stat d5/d1e/d29/d10/f3d 0 2024-03-22T06:15:12.733 INFO:tasks.workunit.client.0.smithi006.stdout:2/755: symlink d13/d34/d4d/d4e/d75/da5/d6c/l11b 0 2024-03-22T06:15:12.734 INFO:tasks.workunit.client.0.smithi006.stdout:5/855: truncate d0/db/d16/fca 4124555 0 2024-03-22T06:15:12.740 INFO:tasks.workunit.client.0.smithi006.stdout:8/846: truncate d9/db/d35/fbb 3120709 0 2024-03-22T06:15:12.740 INFO:tasks.workunit.client.0.smithi006.stdout:8/847: dread - d9/db/d35/dd2/d76/db3/dc3/ff2 zero size 2024-03-22T06:15:12.740 INFO:tasks.workunit.client.0.smithi006.stdout:8/848: dread - d9/db/d35/dd2/d76/f123 zero size 2024-03-22T06:15:12.740 INFO:tasks.workunit.client.0.smithi006.stdout:8/849: creat d9/db/d35/dd2/d4a/da7/f139 x:0 0 0 2024-03-22T06:15:12.743 INFO:tasks.workunit.client.0.smithi006.stdout:9/649: rename d8/d10/l24 to d8/d10/d20/d45/d4d/ld5 0 2024-03-22T06:15:12.749 INFO:tasks.workunit.client.0.smithi006.stdout:2/756: mknod d13/d69/d10f/deb/c11c 0 2024-03-22T06:15:12.754 INFO:tasks.workunit.client.0.smithi006.stdout:8/850: dwrite d9/f21 [0,4194304] 0 2024-03-22T06:15:12.758 INFO:tasks.workunit.client.0.smithi006.stdout:2/757: write d13/d34/d4d/f88 [3811172,8961] 0 2024-03-22T06:15:12.759 INFO:tasks.workunit.client.0.smithi006.stdout:7/763: rename d5/d1e/d63/d85/fec to d5/d1e/d63/d85/d80/ff8 0 2024-03-22T06:15:12.766 INFO:tasks.workunit.client.0.smithi006.stdout:9/650: dwrite d8/d10/d20/d45/d49/f77 [0,4194304] 0 2024-03-22T06:15:12.767 INFO:tasks.workunit.client.0.smithi006.stdout:8/851: unlink d9/db/dc/l38 0 2024-03-22T06:15:12.771 INFO:tasks.workunit.client.0.smithi006.stdout:2/758: rmdir d13/d34/d4d/d4e/d75/da5/d6c/d107 0 2024-03-22T06:15:12.774 INFO:tasks.workunit.client.0.smithi006.stdout:7/764: creat d5/d1e/d29/db9/ff9 x:0 0 0 2024-03-22T06:15:12.776 INFO:tasks.workunit.client.0.smithi006.stdout:8/852: write d9/db/d12/f3d [3412671,57466] 0 2024-03-22T06:15:12.778 INFO:tasks.workunit.client.0.smithi006.stdout:9/651: link d8/d10/d20/d45/f66 d8/d10/d20/d45/d49/fd6 0 2024-03-22T06:15:12.780 INFO:tasks.workunit.client.0.smithi006.stdout:2/759: symlink d13/d34/l11d 0 2024-03-22T06:15:12.780 INFO:tasks.workunit.client.0.smithi006.stdout:2/760: readlink d13/d53/laa 0 2024-03-22T06:15:12.781 INFO:tasks.workunit.client.0.smithi006.stdout:7/765: rmdir d5/d1e/d29/d1f/d2d 39 2024-03-22T06:15:12.781 INFO:tasks.workunit.client.0.smithi006.stdout:7/766: stat d5/d1e/d29/d66/l8b 0 2024-03-22T06:15:12.781 INFO:tasks.workunit.client.0.smithi006.stdout:2/761: read d13/d20/d26/f15 [603503,73096] 0 2024-03-22T06:15:12.781 INFO:tasks.workunit.client.0.smithi006.stdout:2/762: dread - d13/d53/dc5/ffa zero size 2024-03-22T06:15:12.783 INFO:tasks.workunit.client.0.smithi006.stdout:8/853: mkdir d9/db/d12/d66/d10c/d122/d13a 0 2024-03-22T06:15:12.789 INFO:tasks.workunit.client.0.smithi006.stdout:7/767: mkdir d5/d1e/d29/d66/d91/ddc/dfa 0 2024-03-22T06:15:12.792 INFO:tasks.workunit.client.0.smithi006.stdout:2/763: mkdir d13/d34/d4d/d4e/d75/da5/d82/d11e 0 2024-03-22T06:15:12.795 INFO:tasks.workunit.client.0.smithi006.stdout:9/652: dread d8/d10/d28/f57 [4194304,4194304] 0 2024-03-22T06:15:12.801 INFO:tasks.workunit.client.0.smithi006.stdout:7/768: rename d5/d1e/d29/d1f/f38 to d5/d1e/d29/d10/ffb 0 2024-03-22T06:15:12.804 INFO:tasks.workunit.client.0.smithi006.stdout:7/769: creat d5/d1e/d29/d66/d91/ffc x:0 0 0 2024-03-22T06:15:12.807 INFO:tasks.workunit.client.0.smithi006.stdout:8/854: dwrite d9/db/d35/dd2/d76/db3/dc3/f109 [0,4194304] 0 2024-03-22T06:15:12.807 INFO:tasks.workunit.client.0.smithi006.stdout:6/641: write d6/f1e [5252399,100982] 0 2024-03-22T06:15:12.813 INFO:tasks.workunit.client.0.smithi006.stdout:8/855: mknod d9/db/d35/dd2/d4a/d4d/c13b 0 2024-03-22T06:15:12.813 INFO:tasks.workunit.client.0.smithi006.stdout:8/856: fsync d9/db/f2e 0 2024-03-22T06:15:12.813 INFO:tasks.workunit.client.0.smithi006.stdout:8/857: write d9/db/d12/d66/d10c/d125/da3/fda [748428,3739] 0 2024-03-22T06:15:12.814 INFO:tasks.workunit.client.0.smithi006.stdout:6/642: rename d6/c8 to d6/d7c/ccc 0 2024-03-22T06:15:12.815 INFO:tasks.workunit.client.0.smithi006.stdout:8/858: rmdir d9/db/d35/dd2/d76 39 2024-03-22T06:15:12.817 INFO:tasks.workunit.client.0.smithi006.stdout:2/764: dwrite d13/d53/dc5/ffa [0,4194304] 0 2024-03-22T06:15:12.817 INFO:tasks.workunit.client.0.smithi006.stdout:2/765: write d13/f61 [2165822,89754] 0 2024-03-22T06:15:12.817 INFO:tasks.workunit.client.0.smithi006.stdout:2/766: truncate d13/d20/d35/d65/fd3 948481 0 2024-03-22T06:15:12.817 INFO:tasks.workunit.client.0.smithi006.stdout:9/653: dwrite d8/ff [4194304,4194304] 0 2024-03-22T06:15:12.817 INFO:tasks.workunit.client.0.smithi006.stdout:9/654: truncate d8/db3/d50/f6f 80910 0 2024-03-22T06:15:12.817 INFO:tasks.workunit.client.0.smithi006.stdout:9/655: dread - d8/d10/d20/d45/f66 zero size 2024-03-22T06:15:12.817 INFO:tasks.workunit.client.0.smithi006.stdout:7/770: write d5/f56 [2735162,102530] 0 2024-03-22T06:15:12.818 INFO:tasks.workunit.client.0.smithi006.stdout:8/859: creat d9/db/f13c x:0 0 0 2024-03-22T06:15:12.825 INFO:tasks.workunit.client.0.smithi006.stdout:2/767: getdents d13/d20/d35/d65 0 2024-03-22T06:15:12.825 INFO:tasks.workunit.client.0.smithi006.stdout:2/768: dread - d13/d20/da6/fc7 zero size 2024-03-22T06:15:12.826 INFO:tasks.workunit.client.0.smithi006.stdout:6/643: dread d6/d1d/d5c/d81/f7e [0,4194304] 0 2024-03-22T06:15:12.826 INFO:tasks.workunit.client.0.smithi006.stdout:9/656: truncate d8/d10/d23/d3e/fae 2230056 0 2024-03-22T06:15:12.830 INFO:tasks.workunit.client.0.smithi006.stdout:8/860: rmdir d9/db/d12/d66/d6c/d71 39 2024-03-22T06:15:12.830 INFO:tasks.workunit.client.0.smithi006.stdout:8/861: dread - d9/db/f13c zero size 2024-03-22T06:15:12.830 INFO:tasks.workunit.client.0.smithi006.stdout:8/862: stat d9/db/d12/d66/d6c/d80/df0/f12b 0 2024-03-22T06:15:12.839 INFO:tasks.workunit.client.0.smithi006.stdout:2/769: mkdir d13/d53/d6b/dd8/d11f 0 2024-03-22T06:15:12.841 INFO:tasks.workunit.client.0.smithi006.stdout:6/644: mkdir d6/d1d/d99/dcd 0 2024-03-22T06:15:12.841 INFO:tasks.workunit.client.0.smithi006.stdout:6/645: write d6/d24/d44/f62 [517699,110665] 0 2024-03-22T06:15:12.842 INFO:tasks.workunit.client.0.smithi006.stdout:9/657: dwrite d8/d1f/f5d [0,4194304] 0 2024-03-22T06:15:12.844 INFO:tasks.workunit.client.0.smithi006.stdout:2/770: mknod d13/d69/c120 0 2024-03-22T06:15:12.844 INFO:tasks.workunit.client.0.smithi006.stdout:2/771: truncate d13/d34/d4d/d4e/d75/fb7 1456980 0 2024-03-22T06:15:12.845 INFO:tasks.workunit.client.0.smithi006.stdout:2/772: stat d13/d34/d4d/d4e/d75/l41 0 2024-03-22T06:15:12.845 INFO:tasks.workunit.client.0.smithi006.stdout:2/773: write d13/d20/d26/dca/def/f113 [489655,40897] 0 2024-03-22T06:15:12.854 INFO:tasks.workunit.client.0.smithi006.stdout:6/646: dread d6/d1d/d5c/f5e [0,4194304] 0 2024-03-22T06:15:12.854 INFO:tasks.workunit.client.0.smithi006.stdout:6/647: chown d6/d6f/f6e 0 1 2024-03-22T06:15:12.855 INFO:tasks.workunit.client.0.smithi006.stdout:9/658: dread d8/d1f/d34/f3a [0,4194304] 0 2024-03-22T06:15:12.855 INFO:tasks.workunit.client.0.smithi006.stdout:9/659: readlink d8/d10/d28/l51 0 2024-03-22T06:15:12.860 INFO:tasks.workunit.client.0.smithi006.stdout:2/774: dwrite d13/d20/f42 [0,4194304] 0 2024-03-22T06:15:12.860 INFO:tasks.workunit.client.0.smithi006.stdout:2/775: stat d13/d53/dc5/ce5 0 2024-03-22T06:15:12.860 INFO:tasks.workunit.client.0.smithi006.stdout:2/776: creat d13/f121 x:0 0 0 2024-03-22T06:15:12.869 INFO:tasks.workunit.client.0.smithi006.stdout:2/777: mkdir d13/d20/d26/d19/d99/d122 0 2024-03-22T06:15:12.869 INFO:tasks.workunit.client.0.smithi006.stdout:2/778: getdents d13/d20/d35/d65 0 2024-03-22T06:15:12.869 INFO:tasks.workunit.client.0.smithi006.stdout:2/779: chown d13/d34/d4d/d4e/d75/dcc/ce7 233 1 2024-03-22T06:15:12.869 INFO:tasks.workunit.client.0.smithi006.stdout:6/648: dwrite d6/fc1 [0,4194304] 0 2024-03-22T06:15:12.875 INFO:tasks.workunit.client.0.smithi006.stdout:6/649: readlink d6/l17 0 2024-03-22T06:15:12.875 INFO:tasks.workunit.client.0.smithi006.stdout:6/650: stat d6/d1d/d5c/d81/d8a 0 2024-03-22T06:15:12.878 INFO:tasks.workunit.client.0.smithi006.stdout:2/780: write d13/d53/dc5/ffa [4087613,79964] 0 2024-03-22T06:15:12.878 INFO:tasks.workunit.client.0.smithi006.stdout:2/781: chown d13/d20/da6/fc2 3 1 2024-03-22T06:15:12.880 INFO:tasks.workunit.client.0.smithi006.stdout:2/782: mknod d13/d34/d4d/d4e/d75/c123 0 2024-03-22T06:15:12.883 INFO:tasks.workunit.client.0.smithi006.stdout:2/783: rename d13/d20/d26/d19/d99/d122 to d13/d20/d26/db9/d124 0 2024-03-22T06:15:12.894 INFO:tasks.workunit.client.0.smithi006.stdout:2/784: write d13/d20/d35/d65/f7b [1508335,103795] 0 2024-03-22T06:15:12.894 INFO:tasks.workunit.client.0.smithi006.stdout:2/785: readlink d13/d34/l11d 0 2024-03-22T06:15:12.894 INFO:tasks.workunit.client.0.smithi006.stdout:2/786: chown d13/d34/d4d/d11a 0 1 2024-03-22T06:15:12.894 INFO:tasks.workunit.client.0.smithi006.stdout:2/787: chown d13/d34/d4d/d4e/d75/da5/d82/f95 2523 1 2024-03-22T06:15:12.896 INFO:tasks.workunit.client.0.smithi006.stdout:6/651: dwrite d6/d7c/d83/fa2 [0,4194304] 0 2024-03-22T06:15:12.896 INFO:tasks.workunit.client.0.smithi006.stdout:6/652: truncate d6/d24/d44/fa0 932666 0 2024-03-22T06:15:12.899 INFO:tasks.workunit.client.0.smithi006.stdout:2/788: fsync fb 0 2024-03-22T06:15:12.901 INFO:tasks.workunit.client.0.smithi006.stdout:6/653: write d6/d1d/d5c/f5e [1102649,73804] 0 2024-03-22T06:15:12.904 INFO:tasks.workunit.client.0.smithi006.stdout:2/789: getdents d13/d34/d4d/d4e/d75/da5/d82/dc3 0 2024-03-22T06:15:12.904 INFO:tasks.workunit.client.0.smithi006.stdout:2/790: fsync d13/f116 0 2024-03-22T06:15:12.907 INFO:tasks.workunit.client.0.smithi006.stdout:2/791: link d13/d20/d35/lb4 d13/l125 0 2024-03-22T06:15:12.908 INFO:tasks.workunit.client.0.smithi006.stdout:6/654: write d6/d7c/d92/fad [2816340,4981] 0 2024-03-22T06:15:12.909 INFO:tasks.workunit.client.0.smithi006.stdout:2/792: fdatasync d13/d34/d4d/d4e/d75/da5/f5f 0 2024-03-22T06:15:12.912 INFO:tasks.workunit.client.0.smithi006.stdout:6/655: creat d6/d1d/fce x:0 0 0 2024-03-22T06:15:12.912 INFO:tasks.workunit.client.0.smithi006.stdout:6/656: creat d6/d24/d44/fcf x:0 0 0 2024-03-22T06:15:12.912 INFO:tasks.workunit.client.0.smithi006.stdout:6/657: write d6/d1b/d2c/f72 [990468,32152] 0 2024-03-22T06:15:12.912 INFO:tasks.workunit.client.0.smithi006.stdout:6/658: write d6/d1d/d5c/d89/f8b [1786926,8856] 0 2024-03-22T06:15:12.914 INFO:tasks.workunit.client.0.smithi006.stdout:2/793: creat d13/d53/f126 x:0 0 0 2024-03-22T06:15:12.921 INFO:tasks.workunit.client.0.smithi006.stdout:2/794: creat d13/d34/d4d/d4e/de3/f127 x:0 0 0 2024-03-22T06:15:12.924 INFO:tasks.workunit.client.0.smithi006.stdout:6/659: rmdir d6/d1d/d99 39 2024-03-22T06:15:12.924 INFO:tasks.workunit.client.0.smithi006.stdout:6/660: chown d6/d77/l86 3227 1 2024-03-22T06:15:12.924 INFO:tasks.workunit.client.0.smithi006.stdout:6/661: chown d6/d1b/d2c/f4b 1295 1 2024-03-22T06:15:12.926 INFO:tasks.workunit.client.0.smithi006.stdout:2/795: creat d13/d69/f128 x:0 0 0 2024-03-22T06:15:12.926 INFO:tasks.workunit.client.0.smithi006.stdout:2/796: readlink d13/d34/l36 0 2024-03-22T06:15:12.928 INFO:tasks.workunit.client.0.smithi006.stdout:6/662: truncate d6/d1b/f3c 1466785 0 2024-03-22T06:15:12.931 INFO:tasks.workunit.client.0.smithi006.stdout:2/797: mkdir d13/d20/d35/d104/d129 0 2024-03-22T06:15:12.937 INFO:tasks.workunit.client.0.smithi006.stdout:6/663: creat d6/d1d/d99/dcd/fd0 x:0 0 0 2024-03-22T06:15:12.937 INFO:tasks.workunit.client.0.smithi006.stdout:6/664: creat d6/d1b/d2c/fd1 x:0 0 0 2024-03-22T06:15:12.939 INFO:tasks.workunit.client.0.smithi006.stdout:2/798: mknod d13/d34/d4d/d4e/d75/da5/d82/c12a 0 2024-03-22T06:15:12.944 INFO:tasks.workunit.client.0.smithi006.stdout:6/665: unlink d6/d1d/d64/f93 0 2024-03-22T06:15:12.946 INFO:tasks.workunit.client.0.smithi006.stdout:6/666: dread d6/d1d/d5c/d81/f5d [0,4194304] 0 2024-03-22T06:15:12.946 INFO:tasks.workunit.client.0.smithi006.stdout:6/667: chown d6/d7c/d92/fb9 322328 1 2024-03-22T06:15:12.946 INFO:tasks.workunit.client.0.smithi006.stdout:2/799: truncate d13/d34/d4d/d4e/d75/d62/f9c 794652 0 2024-03-22T06:15:12.953 INFO:tasks.workunit.client.0.smithi006.stdout:2/800: dread d13/d20/fad [0,4194304] 0 2024-03-22T06:15:12.969 INFO:tasks.workunit.client.0.smithi006.stdout:6/668: dwrite d6/d1b/f2d [0,4194304] 0 2024-03-22T06:15:12.969 INFO:tasks.workunit.client.0.smithi006.stdout:6/669: readlink d6/d1b/d2c/l47 0 2024-03-22T06:15:12.971 INFO:tasks.workunit.client.0.smithi006.stdout:6/670: mknod d6/d6f/cd2 0 2024-03-22T06:15:12.974 INFO:tasks.workunit.client.0.smithi006.stdout:2/801: dwrite d13/d34/f58 [0,4194304] 0 2024-03-22T06:15:12.981 INFO:tasks.workunit.client.0.smithi006.stdout:2/802: truncate d13/d20/f4b 2755006 0 2024-03-22T06:15:12.981 INFO:tasks.workunit.client.0.smithi006.stdout:2/803: write d13/d20/fcb [774067,57827] 0 2024-03-22T06:15:12.985 INFO:tasks.workunit.client.0.smithi006.stdout:2/804: symlink d13/d34/d4d/d4e/d75/l12b 0 2024-03-22T06:15:12.987 INFO:tasks.workunit.client.0.smithi006.stdout:2/805: getdents d13/d34 0 2024-03-22T06:15:12.995 INFO:tasks.workunit.client.0.smithi006.stdout:2/806: dread d13/d20/d26/d19/f3e [0,4194304] 0 2024-03-22T06:15:12.997 INFO:tasks.workunit.client.0.smithi006.stdout:2/807: rename d13/d53/f126 to d13/d34/d4d/f12c 0 2024-03-22T06:15:12.997 INFO:tasks.workunit.client.0.smithi006.stdout:2/808: chown d13/d20/d26/la4 4947943 1 2024-03-22T06:15:13.004 INFO:tasks.workunit.client.0.smithi006.stdout:2/809: read d13/d34/d4d/d4e/d75/d62/fbd [1118747,82544] 0 2024-03-22T06:15:13.006 INFO:tasks.workunit.client.0.smithi006.stdout:2/810: mkdir d13/d12d 0 2024-03-22T06:15:13.021 INFO:tasks.workunit.client.0.smithi006.stdout:2/811: dwrite d13/d34/d4d/d4e/d75/da5/d82/dc3/ff6 [0,4194304] 0 2024-03-22T06:15:13.025 INFO:tasks.workunit.client.0.smithi006.stdout:6/671: truncate d6/d1d/d99/fc2 1664333 0 2024-03-22T06:15:13.027 INFO:tasks.workunit.client.0.smithi006.stdout:2/812: symlink d13/d34/d4d/d4e/d75/da5/d82/dc3/l12e 0 2024-03-22T06:15:13.030 INFO:tasks.workunit.client.0.smithi006.stdout:6/672: mknod d6/d1d/d5c/cd3 0 2024-03-22T06:15:13.031 INFO:tasks.workunit.client.0.smithi006.stdout:6/673: write d6/d7c/d92/fb9 [745749,126710] 0 2024-03-22T06:15:13.032 INFO:tasks.workunit.client.0.smithi006.stdout:2/813: creat d13/d20/d26/dca/f12f x:0 0 0 2024-03-22T06:15:13.035 INFO:tasks.workunit.client.0.smithi006.stdout:6/674: unlink d6/d6f/f6b 0 2024-03-22T06:15:13.036 INFO:tasks.workunit.client.0.smithi006.stdout:2/814: mknod d13/d20/d26/d19/d99/c130 0 2024-03-22T06:15:13.038 INFO:tasks.workunit.client.0.smithi006.stdout:6/675: symlink d6/d6f/ld4 0 2024-03-22T06:15:13.040 INFO:tasks.workunit.client.0.smithi006.stdout:2/815: mkdir d13/d53/d6b/dd8/d131 0 2024-03-22T06:15:13.043 INFO:tasks.workunit.client.0.smithi006.stdout:6/676: truncate d6/d1d/d5c/d81/f7e 203525 0 2024-03-22T06:15:13.046 INFO:tasks.workunit.client.0.smithi006.stdout:2/816: symlink d13/d20/d26/db9/l132 0 2024-03-22T06:15:13.047 INFO:tasks.workunit.client.0.smithi006.stdout:2/817: fsync d13/d53/f4a 0 2024-03-22T06:15:13.050 INFO:tasks.workunit.client.0.smithi006.stdout:2/818: creat d13/d53/dc5/d10e/f133 x:0 0 0 2024-03-22T06:15:13.050 INFO:tasks.workunit.client.0.smithi006.stdout:2/819: write d13/d69/ffc [205821,101696] 0 2024-03-22T06:15:13.052 INFO:tasks.workunit.client.0.smithi006.stdout:2/820: symlink d13/d53/d6b/dd8/d131/l134 0 2024-03-22T06:15:13.060 INFO:tasks.workunit.client.0.smithi006.stdout:6/677: dwrite d6/f66 [0,4194304] 0 2024-03-22T06:15:13.060 INFO:tasks.workunit.client.0.smithi006.stdout:6/678: readlink d6/d7c/d83/l9d 0 2024-03-22T06:15:13.065 INFO:tasks.workunit.client.0.smithi006.stdout:6/679: symlink d6/d1b/dae/dbc/ld5 0 2024-03-22T06:15:13.070 INFO:tasks.workunit.client.0.smithi006.stdout:2/821: dwrite d13/d53/d6b/dd8/fe6 [0,4194304] 0 2024-03-22T06:15:13.072 INFO:tasks.workunit.client.0.smithi006.stdout:6/680: dread d6/d24/f31 [0,4194304] 0 2024-03-22T06:15:13.078 INFO:tasks.workunit.client.0.smithi006.stdout:2/822: mknod d13/d69/d10f/c135 0 2024-03-22T06:15:13.079 INFO:tasks.workunit.client.0.smithi006.stdout:2/823: stat d13/d20/d35/d65/lf3 0 2024-03-22T06:15:13.079 INFO:tasks.workunit.client.0.smithi006.stdout:2/824: dread - d13/d20/d26/d19/ffb zero size 2024-03-22T06:15:13.079 INFO:tasks.workunit.client.0.smithi006.stdout:2/825: truncate d13/d20/d35/d65/fd3 1493698 0 2024-03-22T06:15:13.079 INFO:tasks.workunit.client.0.smithi006.stdout:2/826: chown d13/d20/d26/l8e 13436 1 2024-03-22T06:15:13.082 INFO:tasks.workunit.client.0.smithi006.stdout:2/827: link d13/d34/d4d/d4e/d75/c2b d13/d20/d26/c136 0 2024-03-22T06:15:13.085 INFO:tasks.workunit.client.0.smithi006.stdout:2/828: rename d13/d34/d6e/ccd to d13/d34/d4d/d4e/d75/dcc/c137 0 2024-03-22T06:15:13.086 INFO:tasks.workunit.client.0.smithi006.stdout:6/681: dwrite d6/f51 [0,4194304] 0 2024-03-22T06:15:13.089 INFO:tasks.workunit.client.0.smithi006.stdout:2/829: link d13/d69/f128 d13/d34/d4d/f138 0 2024-03-22T06:15:13.097 INFO:tasks.workunit.client.0.smithi006.stdout:6/682: read d6/d1b/d2c/f8c [3794774,17812] 0 2024-03-22T06:15:13.097 INFO:tasks.workunit.client.0.smithi006.stdout:2/830: write fe [6831160,42573] 0 2024-03-22T06:15:13.101 INFO:tasks.workunit.client.0.smithi006.stdout:2/831: rmdir d13/d20/da6 39 2024-03-22T06:15:13.102 INFO:tasks.workunit.client.0.smithi006.stdout:2/832: readlink d13/d20/d35/d65/lf3 0 2024-03-22T06:15:13.102 INFO:tasks.workunit.client.0.smithi006.stdout:2/833: write d13/d53/f4a [465930,64736] 0 2024-03-22T06:15:13.102 INFO:tasks.workunit.client.0.smithi006.stdout:2/834: creat d13/d34/d4d/d4e/d105/f139 x:0 0 0 2024-03-22T06:15:13.105 INFO:tasks.workunit.client.0.smithi006.stdout:2/835: link d13/d20/f42 d13/d34/d4d/d4e/d105/f13a 0 2024-03-22T06:15:13.105 INFO:tasks.workunit.client.0.smithi006.stdout:6/683: read d6/d1b/d2c/f95 [3879363,64933] 0 2024-03-22T06:15:13.105 INFO:tasks.workunit.client.0.smithi006.stdout:6/684: dread - d6/d1d/d99/dcd/fd0 zero size 2024-03-22T06:15:13.105 INFO:tasks.workunit.client.0.smithi006.stdout:6/685: dread - d6/f9c zero size 2024-03-22T06:15:13.110 INFO:tasks.workunit.client.0.smithi006.stdout:2/836: mkdir d13/d53/dc5/d10e/d13b 0 2024-03-22T06:15:13.110 INFO:tasks.workunit.client.0.smithi006.stdout:2/837: write d13/d20/d26/dca/dea/ffe [753922,118461] 0 2024-03-22T06:15:13.119 INFO:tasks.workunit.client.0.smithi006.stdout:2/838: mknod d13/d20/d26/d19/c13c 0 2024-03-22T06:15:13.126 INFO:tasks.workunit.client.0.smithi006.stdout:2/839: dread d13/d34/d4d/d4e/d75/faf [0,4194304] 0 2024-03-22T06:15:13.131 INFO:tasks.workunit.client.0.smithi006.stdout:2/840: mknod d13/d53/c13d 0 2024-03-22T06:15:13.131 INFO:tasks.workunit.client.0.smithi006.stdout:2/841: truncate d13/d34/d4d/d4e/d105/f139 979862 0 2024-03-22T06:15:13.131 INFO:tasks.workunit.client.0.smithi006.stdout:2/842: write d13/d34/fe8 [162674,92242] 0 2024-03-22T06:15:13.131 INFO:tasks.workunit.client.0.smithi006.stdout:2/843: symlink d13/d34/d4d/l13e 0 2024-03-22T06:15:13.132 INFO:tasks.workunit.client.0.smithi006.stdout:2/844: write d13/d20/d26/dca/def/f113 [1290156,110977] 0 2024-03-22T06:15:13.138 INFO:tasks.workunit.client.0.smithi006.stdout:2/845: dread d13/d20/d35/d65/fbb [0,4194304] 0 2024-03-22T06:15:13.445 INFO:tasks.workunit.client.1.smithi117.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-22T06:15:13.446 INFO:tasks.workunit.client.1.smithi117.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T06:15:13.446 INFO:tasks.workunit.client.1.smithi117.stderr:+ make 2024-03-22T06:15:13.544 INFO:tasks.workunit.client.1.smithi117.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-22T06:15:13.776 INFO:tasks.workunit.client.1.smithi117.stderr:++ readlink -f fsstress 2024-03-22T06:15:13.777 INFO:tasks.workunit.client.1.smithi117.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-22T06:15:13.777 INFO:tasks.workunit.client.1.smithi117.stderr:+ popd 2024-03-22T06:15:13.777 INFO:tasks.workunit.client.1.smithi117.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T06:15:13.778 INFO:tasks.workunit.client.1.smithi117.stderr:+ popd 2024-03-22T06:15:13.778 INFO:tasks.workunit.client.1.smithi117.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-22T06:15:13.778 INFO:tasks.workunit.client.1.smithi117.stderr:++ mktemp -d -p . 2024-03-22T06:15:13.779 INFO:tasks.workunit.client.1.smithi117.stderr:+ T=./tmp.fgtQOVs9az 2024-03-22T06:15:13.779 INFO:tasks.workunit.client.1.smithi117.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.fgtQOVs9az -l 1 -n 1000 -p 10 -v 2024-03-22T06:15:13.780 INFO:tasks.workunit.client.1.smithi117.stdout:seed = 1710673150 2024-03-22T06:15:13.782 INFO:tasks.workunit.client.1.smithi117.stdout:0/0: fdatasync - no filename 2024-03-22T06:15:13.784 INFO:tasks.workunit.client.1.smithi117.stdout:1/0: symlink l0 0 2024-03-22T06:15:13.784 INFO:tasks.workunit.client.1.smithi117.stdout:1/1: write - no filename 2024-03-22T06:15:13.784 INFO:tasks.workunit.client.1.smithi117.stdout:1/2: fdatasync - no filename 2024-03-22T06:15:13.784 INFO:tasks.workunit.client.1.smithi117.stdout:1/3: write - no filename 2024-03-22T06:15:13.785 INFO:tasks.workunit.client.1.smithi117.stdout:2/0: mknod c0 0 2024-03-22T06:15:13.909 INFO:tasks.workunit.client.1.smithi117.stdout:2/1: write - no filename 2024-03-22T06:15:13.909 INFO:tasks.workunit.client.1.smithi117.stdout:4/0: rename - no filename 2024-03-22T06:15:13.909 INFO:tasks.workunit.client.1.smithi117.stdout:4/1: link - no file 2024-03-22T06:15:13.909 INFO:tasks.workunit.client.1.smithi117.stdout:4/2: write - no filename 2024-03-22T06:15:13.909 INFO:tasks.workunit.client.1.smithi117.stdout:4/3: read - no filename 2024-03-22T06:15:13.910 INFO:tasks.workunit.client.1.smithi117.stdout:5/0: chown . 40 1 2024-03-22T06:15:13.910 INFO:tasks.workunit.client.1.smithi117.stdout:7/0: stat - no entries 2024-03-22T06:15:13.910 INFO:tasks.workunit.client.1.smithi117.stdout:8/0: chown . 402855449 1 2024-03-22T06:15:13.910 INFO:tasks.workunit.client.1.smithi117.stdout:8/1: fdatasync - no filename 2024-03-22T06:15:13.910 INFO:tasks.workunit.client.1.smithi117.stdout:8/2: dwrite - no filename 2024-03-22T06:15:13.910 INFO:tasks.workunit.client.1.smithi117.stdout:8/3: dwrite - no filename 2024-03-22T06:15:13.910 INFO:tasks.workunit.client.1.smithi117.stdout:8/4: write - no filename 2024-03-22T06:15:13.910 INFO:tasks.workunit.client.1.smithi117.stdout:8/5: readlink - no filename 2024-03-22T06:15:13.916 INFO:tasks.workunit.client.1.smithi117.stdout:1/4: getdents . 0 2024-03-22T06:15:14.132 INFO:tasks.workunit.client.1.smithi117.stdout:2/2: unlink c0 0 2024-03-22T06:15:14.132 INFO:tasks.workunit.client.1.smithi117.stdout:2/3: write - no filename 2024-03-22T06:15:14.132 INFO:tasks.workunit.client.1.smithi117.stdout:2/4: truncate - no filename 2024-03-22T06:15:14.132 INFO:tasks.workunit.client.1.smithi117.stdout:2/5: link - no file 2024-03-22T06:15:14.132 INFO:tasks.workunit.client.1.smithi117.stdout:2/6: dread - no filename 2024-03-22T06:15:14.132 INFO:tasks.workunit.client.1.smithi117.stdout:2/7: readlink - no filename 2024-03-22T06:15:14.132 INFO:tasks.workunit.client.1.smithi117.stdout:4/4: mkdir d0 0 2024-03-22T06:15:14.133 INFO:tasks.workunit.client.1.smithi117.stdout:4/5: rename d0 to d0/d1 22 2024-03-22T06:15:14.133 INFO:tasks.workunit.client.1.smithi117.stdout:6/0: creat f0 x:0 0 0 2024-03-22T06:15:14.134 INFO:tasks.workunit.client.1.smithi117.stdout:7/1: creat f0 x:0 0 0 2024-03-22T06:15:14.134 INFO:tasks.workunit.client.1.smithi117.stdout:7/2: chown f0 1054 1 2024-03-22T06:15:14.134 INFO:tasks.workunit.client.1.smithi117.stdout:9/0: creat f0 x:0 0 0 2024-03-22T06:15:14.135 INFO:tasks.workunit.client.1.smithi117.stdout:8/6: creat f0 x:0 0 0 2024-03-22T06:15:14.142 INFO:tasks.workunit.client.1.smithi117.stdout:1/5: mkdir d1 0 2024-03-22T06:15:14.145 INFO:tasks.workunit.client.1.smithi117.stdout:2/8: creat f1 x:0 0 0 2024-03-22T06:15:14.145 INFO:tasks.workunit.client.1.smithi117.stdout:4/6: creat d0/f2 x:0 0 0 2024-03-22T06:15:14.146 INFO:tasks.workunit.client.1.smithi117.stdout:6/1: link f0 f1 0 2024-03-22T06:15:14.149 INFO:tasks.workunit.client.1.smithi117.stdout:1/6: stat l0 0 2024-03-22T06:15:14.150 INFO:tasks.workunit.client.1.smithi117.stdout:2/9: mkdir d2 0 2024-03-22T06:15:14.152 INFO:tasks.workunit.client.1.smithi117.stdout:1/7: creat d1/f2 x:0 0 0 2024-03-22T06:15:14.153 INFO:tasks.workunit.client.1.smithi117.stdout:2/10: creat d2/f3 x:0 0 0 2024-03-22T06:15:14.153 INFO:tasks.workunit.client.1.smithi117.stdout:2/11: dread - d2/f3 zero size 2024-03-22T06:15:14.167 INFO:tasks.workunit.client.1.smithi117.stdout:7/3: dwrite f0 [0,4194304] 0 2024-03-22T06:15:14.170 INFO:tasks.workunit.client.1.smithi117.stdout:7/4: symlink l1 0 2024-03-22T06:15:14.171 INFO:tasks.workunit.client.1.smithi117.stdout:7/5: mknod c2 0 2024-03-22T06:15:14.171 INFO:tasks.workunit.client.1.smithi117.stdout:7/6: creat f3 x:0 0 0 2024-03-22T06:15:14.171 INFO:tasks.workunit.client.1.smithi117.stdout:7/7: chown f0 21794087 1 2024-03-22T06:15:14.180 INFO:tasks.workunit.client.1.smithi117.stdout:7/8: write f0 [2214258,33095] 0 2024-03-22T06:15:14.181 INFO:tasks.workunit.client.1.smithi117.stdout:7/9: link f0 f4 0 2024-03-22T06:15:14.183 INFO:tasks.workunit.client.1.smithi117.stdout:4/7: dwrite d0/f2 [0,4194304] 0 2024-03-22T06:15:14.184 INFO:tasks.workunit.client.1.smithi117.stdout:8/7: dwrite f0 [0,4194304] 0 2024-03-22T06:15:14.184 INFO:tasks.workunit.client.1.smithi117.stdout:1/8: dwrite d1/f2 [0,4194304] 0 2024-03-22T06:15:14.184 INFO:tasks.workunit.client.1.smithi117.stdout:2/12: dwrite f1 [0,4194304] 0 2024-03-22T06:15:14.184 INFO:tasks.workunit.client.1.smithi117.stdout:6/2: dwrite f1 [0,4194304] 0 2024-03-22T06:15:14.184 INFO:tasks.workunit.client.1.smithi117.stdout:2/13: write d2/f3 [457671,101314] 0 2024-03-22T06:15:14.189 INFO:tasks.workunit.client.1.smithi117.stdout:7/10: dread f4 [0,4194304] 0 2024-03-22T06:15:14.190 INFO:tasks.workunit.client.1.smithi117.stdout:1/9: symlink d1/l3 0 2024-03-22T06:15:14.195 INFO:tasks.workunit.client.1.smithi117.stdout:6/3: symlink l2 0 2024-03-22T06:15:14.195 INFO:tasks.workunit.client.1.smithi117.stdout:4/8: read d0/f2 [3318783,126070] 0 2024-03-22T06:15:14.197 INFO:tasks.workunit.client.1.smithi117.stdout:1/10: unlink d1/f2 0 2024-03-22T06:15:14.197 INFO:tasks.workunit.client.1.smithi117.stdout:1/11: fsync - no filename 2024-03-22T06:15:14.197 INFO:tasks.workunit.client.1.smithi117.stdout:1/12: dread - no filename 2024-03-22T06:15:14.197 INFO:tasks.workunit.client.1.smithi117.stdout:1/13: chown l0 97017 1 2024-03-22T06:15:14.198 INFO:tasks.workunit.client.1.smithi117.stdout:6/4: mkdir d3 0 2024-03-22T06:15:14.198 INFO:tasks.workunit.client.1.smithi117.stdout:6/5: getdents d3 0 2024-03-22T06:15:14.199 INFO:tasks.workunit.client.1.smithi117.stdout:4/9: link d0/f2 d0/f3 0 2024-03-22T06:15:14.200 INFO:tasks.workunit.client.1.smithi117.stdout:1/14: unlink d1/l3 0 2024-03-22T06:15:14.200 INFO:tasks.workunit.client.1.smithi117.stdout:1/15: dwrite - no filename 2024-03-22T06:15:14.201 INFO:tasks.workunit.client.1.smithi117.stdout:6/6: creat d3/f4 x:0 0 0 2024-03-22T06:15:14.201 INFO:tasks.workunit.client.1.smithi117.stdout:6/7: fdatasync f0 0 2024-03-22T06:15:14.202 INFO:tasks.workunit.client.1.smithi117.stdout:4/10: dread d0/f3 [0,4194304] 0 2024-03-22T06:15:14.202 INFO:tasks.workunit.client.1.smithi117.stdout:1/16: link l0 d1/l4 0 2024-03-22T06:15:14.203 INFO:tasks.workunit.client.1.smithi117.stdout:1/17: stat l0 0 2024-03-22T06:15:14.203 INFO:tasks.workunit.client.1.smithi117.stdout:1/18: chown l0 520 1 2024-03-22T06:15:14.203 INFO:tasks.workunit.client.1.smithi117.stdout:1/19: chown d1/l4 5 1 2024-03-22T06:15:14.203 INFO:tasks.workunit.client.1.smithi117.stdout:1/20: dwrite - no filename 2024-03-22T06:15:14.203 INFO:tasks.workunit.client.1.smithi117.stdout:1/21: dread - no filename 2024-03-22T06:15:14.205 INFO:tasks.workunit.client.1.smithi117.stdout:4/11: dread d0/f2 [0,4194304] 0 2024-03-22T06:15:14.206 INFO:tasks.workunit.client.1.smithi117.stdout:4/12: mkdir d0/d4 0 2024-03-22T06:15:14.206 INFO:tasks.workunit.client.1.smithi117.stdout:4/13: write d0/f3 [3244797,126400] 0 2024-03-22T06:15:14.208 INFO:tasks.workunit.client.1.smithi117.stdout:2/14: dwrite f1 [0,4194304] 0 2024-03-22T06:15:14.208 INFO:tasks.workunit.client.1.smithi117.stdout:2/15: creat d2/f4 x:0 0 0 2024-03-22T06:15:14.210 INFO:tasks.workunit.client.1.smithi117.stdout:2/16: mkdir d2/d5 0 2024-03-22T06:15:14.210 INFO:tasks.workunit.client.1.smithi117.stdout:7/11: dwrite f4 [0,4194304] 0 2024-03-22T06:15:14.214 INFO:tasks.workunit.client.1.smithi117.stdout:6/8: write f0 [332422,34344] 0 2024-03-22T06:15:14.214 INFO:tasks.workunit.client.1.smithi117.stdout:6/9: write f1 [901672,15755] 0 2024-03-22T06:15:14.216 INFO:tasks.workunit.client.1.smithi117.stdout:6/10: rename f1 to d3/f5 0 2024-03-22T06:15:14.216 INFO:tasks.workunit.client.1.smithi117.stdout:6/11: write d3/f4 [245594,84207] 0 2024-03-22T06:15:14.216 INFO:tasks.workunit.client.1.smithi117.stdout:6/12: truncate d3/f4 389340 0 2024-03-22T06:15:14.218 INFO:tasks.workunit.client.1.smithi117.stdout:6/13: unlink f0 0 2024-03-22T06:15:14.240 INFO:tasks.workunit.client.1.smithi117.stdout:7/12: dwrite f0 [0,4194304] 0 2024-03-22T06:15:14.240 INFO:tasks.workunit.client.1.smithi117.stdout:7/13: readlink l1 0 2024-03-22T06:15:14.241 INFO:tasks.workunit.client.1.smithi117.stdout:4/14: dwrite d0/f2 [0,4194304] 0 2024-03-22T06:15:14.241 INFO:tasks.workunit.client.1.smithi117.stdout:2/17: dwrite f1 [0,4194304] 0 2024-03-22T06:15:14.241 INFO:tasks.workunit.client.1.smithi117.stdout:6/14: fdatasync d3/f4 0 2024-03-22T06:15:14.241 INFO:tasks.workunit.client.1.smithi117.stdout:7/14: rename f3 to f5 0 2024-03-22T06:15:14.250 INFO:tasks.workunit.client.1.smithi117.stdout:4/15: write d0/f2 [1674546,98510] 0 2024-03-22T06:15:14.250 INFO:tasks.workunit.client.1.smithi117.stdout:7/15: write f4 [1206701,100460] 0 2024-03-22T06:15:14.251 INFO:tasks.workunit.client.1.smithi117.stdout:4/16: creat d0/d4/f5 x:0 0 0 2024-03-22T06:15:14.251 INFO:tasks.workunit.client.1.smithi117.stdout:4/17: chown d0/d4/f5 37389253 1 2024-03-22T06:15:14.272 INFO:tasks.workunit.client.1.smithi117.stdout:2/18: dwrite d2/f3 [0,4194304] 0 2024-03-22T06:15:14.272 INFO:tasks.workunit.client.1.smithi117.stdout:2/19: chown d2 201146483 1 2024-03-22T06:15:14.272 INFO:tasks.workunit.client.1.smithi117.stdout:7/16: dwrite f5 [0,4194304] 0 2024-03-22T06:15:14.273 INFO:tasks.workunit.client.1.smithi117.stdout:2/20: mknod d2/c6 0 2024-03-22T06:15:14.276 INFO:tasks.workunit.client.1.smithi117.stdout:2/21: unlink d2/c6 0 2024-03-22T06:15:14.277 INFO:tasks.workunit.client.1.smithi117.stdout:2/22: mknod d2/d5/c7 0 2024-03-22T06:15:14.281 INFO:tasks.workunit.client.0.smithi006.stdout:0/565: sync 2024-03-22T06:15:14.281 INFO:tasks.workunit.client.0.smithi006.stdout:0/566: creat db/dc/d38/fc1 x:0 0 0 2024-03-22T06:15:14.288 INFO:tasks.workunit.client.1.smithi117.stdout:2/23: dread d2/f3 [0,4194304] 0 2024-03-22T06:15:14.290 INFO:tasks.workunit.client.1.smithi117.stdout:2/24: mkdir d2/d5/d8 0 2024-03-22T06:15:14.293 INFO:tasks.workunit.client.0.smithi006.stdout:0/567: dread db/d3d/d4e/f54 [0,4194304] 0 2024-03-22T06:15:14.297 INFO:tasks.workunit.client.1.smithi117.stdout:2/25: write d2/f3 [3448037,45719] 0 2024-03-22T06:15:14.297 INFO:tasks.workunit.client.1.smithi117.stdout:2/26: creat d2/f9 x:0 0 0 2024-03-22T06:15:14.298 INFO:tasks.workunit.client.0.smithi006.stdout:0/568: dread db/d10/d33/f4d [0,4194304] 0 2024-03-22T06:15:14.298 INFO:tasks.workunit.client.0.smithi006.stdout:0/569: write db/d10/d35/d6c/fbb [94470,118836] 0 2024-03-22T06:15:14.299 INFO:tasks.workunit.client.1.smithi117.stdout:2/27: symlink d2/d5/d8/la 0 2024-03-22T06:15:14.301 INFO:tasks.workunit.client.0.smithi006.stdout:0/570: mkdir db/d10/dc2 0 2024-03-22T06:15:14.310 INFO:tasks.workunit.client.0.smithi006.stdout:0/571: dread db/d10/d33/f91 [0,4194304] 0 2024-03-22T06:15:14.311 INFO:tasks.workunit.client.0.smithi006.stdout:0/572: truncate db/d10/d35/f53 1042343 0 2024-03-22T06:15:14.314 INFO:tasks.workunit.client.0.smithi006.stdout:0/573: rename db/d3d/d4e/f65 to db/dc/d38/d50/fc3 0 2024-03-22T06:15:14.315 INFO:tasks.workunit.client.0.smithi006.stdout:0/574: mkdir db/d10/d35/d7b/da1/da6/dc4 0 2024-03-22T06:15:14.316 INFO:tasks.workunit.client.0.smithi006.stdout:0/575: mkdir db/d10/dc2/dc5 0 2024-03-22T06:15:14.316 INFO:tasks.workunit.client.1.smithi117.stdout:2/28: dwrite d2/f9 [0,4194304] 0 2024-03-22T06:15:14.317 INFO:tasks.workunit.client.0.smithi006.stdout:0/576: mknod db/dc/d38/cc6 0 2024-03-22T06:15:14.317 INFO:tasks.workunit.client.0.smithi006.stdout:0/577: chown la 15 1 2024-03-22T06:15:14.318 INFO:tasks.workunit.client.0.smithi006.stdout:0/578: creat db/dc/fc7 x:0 0 0 2024-03-22T06:15:14.322 INFO:tasks.workunit.client.0.smithi006.stdout:0/579: link db/d10/d35/d7b/l95 db/d3d/d4e/d76/lc8 0 2024-03-22T06:15:14.322 INFO:tasks.workunit.client.0.smithi006.stdout:0/580: chown db/d10/d35/d6c/c97 508 1 2024-03-22T06:15:14.324 INFO:tasks.workunit.client.0.smithi006.stdout:0/581: symlink db/d10/d33/d9d/lc9 0 2024-03-22T06:15:14.324 INFO:tasks.workunit.client.0.smithi006.stdout:0/582: fdatasync db/d3d/d4e/f7d 0 2024-03-22T06:15:14.325 INFO:tasks.workunit.client.0.smithi006.stdout:0/583: symlink db/d10/d35/d7b/da1/lca 0 2024-03-22T06:15:14.325 INFO:tasks.workunit.client.0.smithi006.stdout:0/584: fdatasync db/d10/d33/f42 0 2024-03-22T06:15:14.328 INFO:tasks.workunit.client.0.smithi006.stdout:0/585: mknod db/d10/ccb 0 2024-03-22T06:15:14.350 INFO:tasks.workunit.client.1.smithi117.stdout:9/1: getdents . 0 2024-03-22T06:15:14.351 INFO:tasks.workunit.client.1.smithi117.stdout:8/8: fsync f0 0 2024-03-22T06:15:14.351 INFO:tasks.workunit.client.1.smithi117.stdout:8/9: chown f0 28 1 2024-03-22T06:15:14.352 INFO:tasks.workunit.client.1.smithi117.stdout:9/2: rename f0 to f1 0 2024-03-22T06:15:14.353 INFO:tasks.workunit.client.1.smithi117.stdout:8/10: mkdir d1 0 2024-03-22T06:15:14.354 INFO:tasks.workunit.client.1.smithi117.stdout:8/11: mkdir d1/d2 0 2024-03-22T06:15:14.354 INFO:tasks.workunit.client.1.smithi117.stdout:8/12: mknod d1/c3 0 2024-03-22T06:15:14.355 INFO:tasks.workunit.client.1.smithi117.stdout:8/13: symlink d1/d2/l4 0 2024-03-22T06:15:14.362 INFO:tasks.workunit.client.1.smithi117.stdout:8/14: write f0 [2620413,26395] 0 2024-03-22T06:15:14.363 INFO:tasks.workunit.client.1.smithi117.stdout:8/15: rename d1/c3 to d1/c5 0 2024-03-22T06:15:14.363 INFO:tasks.workunit.client.1.smithi117.stdout:8/16: truncate f0 5068993 0 2024-03-22T06:15:14.370 INFO:tasks.workunit.client.1.smithi117.stdout:8/17: dread f0 [0,4194304] 0 2024-03-22T06:15:14.371 INFO:tasks.workunit.client.1.smithi117.stdout:8/18: mknod d1/d2/c6 0 2024-03-22T06:15:14.371 INFO:tasks.workunit.client.1.smithi117.stdout:8/19: chown d1/d2 33215 1 2024-03-22T06:15:14.373 INFO:tasks.workunit.client.1.smithi117.stdout:8/20: creat d1/d2/f7 x:0 0 0 2024-03-22T06:15:14.374 INFO:tasks.workunit.client.1.smithi117.stdout:1/22: rmdir d1 39 2024-03-22T06:15:14.374 INFO:tasks.workunit.client.1.smithi117.stdout:1/23: write - no filename 2024-03-22T06:15:14.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/21: symlink d1/l8 0 2024-03-22T06:15:14.377 INFO:tasks.workunit.client.1.smithi117.stdout:1/24: creat d1/f5 x:0 0 0 2024-03-22T06:15:14.377 INFO:tasks.workunit.client.1.smithi117.stdout:1/25: write d1/f5 [217579,129998] 0 2024-03-22T06:15:14.378 INFO:tasks.workunit.client.1.smithi117.stdout:8/22: mkdir d1/d2/d9 0 2024-03-22T06:15:14.378 INFO:tasks.workunit.client.1.smithi117.stdout:8/23: dread - d1/d2/f7 zero size 2024-03-22T06:15:14.379 INFO:tasks.workunit.client.1.smithi117.stdout:1/26: mknod d1/c6 0 2024-03-22T06:15:14.379 INFO:tasks.workunit.client.1.smithi117.stdout:1/27: creat d1/f7 x:0 0 0 2024-03-22T06:15:14.380 INFO:tasks.workunit.client.1.smithi117.stdout:6/15: getdents d3 0 2024-03-22T06:15:14.380 INFO:tasks.workunit.client.1.smithi117.stdout:6/16: write d3/f4 [1381733,92435] 0 2024-03-22T06:15:14.381 INFO:tasks.workunit.client.1.smithi117.stdout:8/24: dread f0 [0,4194304] 0 2024-03-22T06:15:14.381 INFO:tasks.workunit.client.1.smithi117.stdout:7/17: dwrite f4 [4194304,4194304] 0 2024-03-22T06:15:14.382 INFO:tasks.workunit.client.1.smithi117.stdout:2/29: dwrite f1 [4194304,4194304] 0 2024-03-22T06:15:14.382 INFO:tasks.workunit.client.1.smithi117.stdout:8/25: write f0 [4983730,108440] 0 2024-03-22T06:15:14.384 INFO:tasks.workunit.client.1.smithi117.stdout:4/18: rmdir d0 39 2024-03-22T06:15:14.384 INFO:tasks.workunit.client.1.smithi117.stdout:4/19: truncate d0/d4/f5 953157 0 2024-03-22T06:15:14.385 INFO:tasks.workunit.client.1.smithi117.stdout:4/20: truncate d0/d4/f5 1207878 0 2024-03-22T06:15:14.385 INFO:tasks.workunit.client.1.smithi117.stdout:4/21: write d0/f3 [850017,20162] 0 2024-03-22T06:15:14.385 INFO:tasks.workunit.client.1.smithi117.stdout:4/22: chown d0 6 1 2024-03-22T06:15:14.388 INFO:tasks.workunit.client.1.smithi117.stdout:6/17: mknod d3/c6 0 2024-03-22T06:15:14.390 INFO:tasks.workunit.client.1.smithi117.stdout:2/30: mkdir d2/d5/db 0 2024-03-22T06:15:14.391 INFO:tasks.workunit.client.1.smithi117.stdout:2/31: fdatasync d2/f4 0 2024-03-22T06:15:14.392 INFO:tasks.workunit.client.1.smithi117.stdout:8/26: rmdir d1 39 2024-03-22T06:15:14.393 INFO:tasks.workunit.client.1.smithi117.stdout:1/28: link d1/f7 d1/f8 0 2024-03-22T06:15:14.394 INFO:tasks.workunit.client.1.smithi117.stdout:6/18: chown d3/f4 111 1 2024-03-22T06:15:14.396 INFO:tasks.workunit.client.1.smithi117.stdout:7/18: write f5 [215558,41131] 0 2024-03-22T06:15:14.396 INFO:tasks.workunit.client.1.smithi117.stdout:7/19: creat f6 x:0 0 0 2024-03-22T06:15:14.397 INFO:tasks.workunit.client.1.smithi117.stdout:7/20: rmdir - no directory 2024-03-22T06:15:14.397 INFO:tasks.workunit.client.1.smithi117.stdout:7/21: readlink l1 0 2024-03-22T06:15:14.409 INFO:tasks.workunit.client.1.smithi117.stdout:9/3: rename f1 to f2 0 2024-03-22T06:15:14.410 INFO:tasks.workunit.client.1.smithi117.stdout:9/4: mknod c3 0 2024-03-22T06:15:14.412 INFO:tasks.workunit.client.1.smithi117.stdout:9/5: unlink c3 0 2024-03-22T06:15:14.420 INFO:tasks.workunit.client.1.smithi117.stdout:6/19: mkdir d3/d7 0 2024-03-22T06:15:14.426 INFO:tasks.workunit.client.1.smithi117.stdout:4/23: dwrite d0/d4/f5 [0,4194304] 0 2024-03-22T06:15:14.427 INFO:tasks.workunit.client.1.smithi117.stdout:4/24: stat d0/f2 0 2024-03-22T06:15:14.427 INFO:tasks.workunit.client.1.smithi117.stdout:7/22: dwrite f6 [0,4194304] 0 2024-03-22T06:15:14.427 INFO:tasks.workunit.client.1.smithi117.stdout:8/27: dwrite f0 [4194304,4194304] 0 2024-03-22T06:15:14.427 INFO:tasks.workunit.client.1.smithi117.stdout:8/28: read f0 [2149249,28682] 0 2024-03-22T06:15:14.427 INFO:tasks.workunit.client.1.smithi117.stdout:8/29: read - d1/d2/f7 zero size 2024-03-22T06:15:14.427 INFO:tasks.workunit.client.1.smithi117.stdout:8/30: chown d1/c5 94242 1 2024-03-22T06:15:14.427 INFO:tasks.workunit.client.1.smithi117.stdout:1/29: dwrite d1/f8 [0,4194304] 0 2024-03-22T06:15:14.428 INFO:tasks.workunit.client.1.smithi117.stdout:4/25: getdents d0 0 2024-03-22T06:15:14.433 INFO:tasks.workunit.client.1.smithi117.stdout:9/6: getdents . 0 2024-03-22T06:15:14.433 INFO:tasks.workunit.client.1.smithi117.stdout:1/30: dread d1/f5 [0,4194304] 0 2024-03-22T06:15:14.433 INFO:tasks.workunit.client.1.smithi117.stdout:1/31: fsync d1/f7 0 2024-03-22T06:15:14.438 INFO:tasks.workunit.client.1.smithi117.stdout:4/26: creat d0/d4/f6 x:0 0 0 2024-03-22T06:15:14.438 INFO:tasks.workunit.client.1.smithi117.stdout:4/27: creat d0/d4/f7 x:0 0 0 2024-03-22T06:15:14.438 INFO:tasks.workunit.client.1.smithi117.stdout:4/28: dread - d0/d4/f6 zero size 2024-03-22T06:15:14.438 INFO:tasks.workunit.client.1.smithi117.stdout:4/29: getdents d0 0 2024-03-22T06:15:14.444 INFO:tasks.workunit.client.1.smithi117.stdout:1/32: mkdir d1/d9 0 2024-03-22T06:15:14.447 INFO:tasks.workunit.client.1.smithi117.stdout:7/23: truncate f0 2973669 0 2024-03-22T06:15:14.448 INFO:tasks.workunit.client.1.smithi117.stdout:4/30: rename d0/d4/f5 to d0/f8 0 2024-03-22T06:15:14.448 INFO:tasks.workunit.client.1.smithi117.stdout:4/31: read - d0/d4/f6 zero size 2024-03-22T06:15:14.449 INFO:tasks.workunit.client.1.smithi117.stdout:1/33: rmdir d1/d9 0 2024-03-22T06:15:14.463 INFO:tasks.workunit.client.1.smithi117.stdout:9/7: dwrite f2 [0,4194304] 0 2024-03-22T06:15:14.463 INFO:tasks.workunit.client.1.smithi117.stdout:9/8: chown f2 0 1 2024-03-22T06:15:14.463 INFO:tasks.workunit.client.1.smithi117.stdout:8/31: truncate f0 4707264 0 2024-03-22T06:15:14.463 INFO:tasks.workunit.client.1.smithi117.stdout:8/32: creat d1/d2/fa x:0 0 0 2024-03-22T06:15:14.464 INFO:tasks.workunit.client.1.smithi117.stdout:9/9: mkdir d4 0 2024-03-22T06:15:14.466 INFO:tasks.workunit.client.1.smithi117.stdout:8/33: symlink d1/lb 0 2024-03-22T06:15:14.470 INFO:tasks.workunit.client.1.smithi117.stdout:9/10: dread f2 [0,4194304] 0 2024-03-22T06:15:14.471 INFO:tasks.workunit.client.1.smithi117.stdout:9/11: rename d4 to d4/d5 22 2024-03-22T06:15:14.471 INFO:tasks.workunit.client.0.smithi006.stdout:4/835: sync 2024-03-22T06:15:14.471 INFO:tasks.workunit.client.0.smithi006.stdout:1/720: sync 2024-03-22T06:15:14.471 INFO:tasks.workunit.client.0.smithi006.stdout:5/856: sync 2024-03-22T06:15:14.471 INFO:tasks.workunit.client.0.smithi006.stdout:3/722: sync 2024-03-22T06:15:14.472 INFO:tasks.workunit.client.0.smithi006.stdout:3/723: readlink d0/d8/d1c/d49/lda 0 2024-03-22T06:15:14.472 INFO:tasks.workunit.client.1.smithi117.stdout:9/12: creat d4/f6 x:0 0 0 2024-03-22T06:15:14.472 INFO:tasks.workunit.client.1.smithi117.stdout:9/13: dread - d4/f6 zero size 2024-03-22T06:15:14.473 INFO:tasks.workunit.client.1.smithi117.stdout:9/14: chown d4 89126 1 2024-03-22T06:15:14.473 INFO:tasks.workunit.client.1.smithi117.stdout:9/15: chown d4 25 1 2024-03-22T06:15:14.473 INFO:tasks.workunit.client.1.smithi117.stdout:9/16: truncate d4/f6 269202 0 2024-03-22T06:15:14.473 INFO:tasks.workunit.client.1.smithi117.stdout:9/17: write d4/f6 [740351,107580] 0 2024-03-22T06:15:14.474 INFO:tasks.workunit.client.0.smithi006.stdout:5/857: unlink d0/db/d99/f2b 0 2024-03-22T06:15:14.476 INFO:tasks.workunit.client.0.smithi006.stdout:4/836: symlink d6/d1b/d46/l121 0 2024-03-22T06:15:14.476 INFO:tasks.workunit.client.0.smithi006.stdout:1/721: dread d8/fd8 [0,4194304] 0 2024-03-22T06:15:14.477 INFO:tasks.workunit.client.1.smithi117.stdout:9/18: mkdir d4/d7 0 2024-03-22T06:15:14.477 INFO:tasks.workunit.client.0.smithi006.stdout:3/724: link d0/d8/d1c/l55 d0/d8/d1c/d49/lf1 0 2024-03-22T06:15:14.479 INFO:tasks.workunit.client.0.smithi006.stdout:1/722: mkdir d8/d9/d13/d24/d32/d34/daa/df4 0 2024-03-22T06:15:14.479 INFO:tasks.workunit.client.0.smithi006.stdout:1/723: chown d8/d9/d10/d7c/d96/la0 4453968 1 2024-03-22T06:15:14.479 INFO:tasks.workunit.client.1.smithi117.stdout:9/19: mkdir d4/d8 0 2024-03-22T06:15:14.481 INFO:tasks.workunit.client.0.smithi006.stdout:3/725: mknod d0/d8/d1c/d3e/d5f/d90/da7/d97/cf2 0 2024-03-22T06:15:14.482 INFO:tasks.workunit.client.0.smithi006.stdout:3/726: write d0/d8/d1c/d3e/d5f/d90/da7/de7/d4c/fa4 [758099,61077] 0 2024-03-22T06:15:14.482 INFO:tasks.workunit.client.0.smithi006.stdout:3/727: dread - d0/d8/d1c/d3e/d68/f75 zero size 2024-03-22T06:15:14.484 INFO:tasks.workunit.client.0.smithi006.stdout:3/728: rename d0/d8/d9/cc5 to d0/d8/d1c/d49/d79/cf3 0 2024-03-22T06:15:14.485 INFO:tasks.workunit.client.1.smithi117.stdout:8/34: dwrite d1/d2/f7 [0,4194304] 0 2024-03-22T06:15:14.486 INFO:tasks.workunit.client.0.smithi006.stdout:5/858: dread d0/db/d16/d1c/d34/d35/d85/dda/fee [0,4194304] 0 2024-03-22T06:15:14.486 INFO:tasks.workunit.client.0.smithi006.stdout:5/859: fdatasync d0/db/d50/db9/de1/f7d 0 2024-03-22T06:15:14.486 INFO:tasks.workunit.client.0.smithi006.stdout:3/729: truncate d0/d7e/f80 1854737 0 2024-03-22T06:15:14.489 INFO:tasks.workunit.client.0.smithi006.stdout:3/730: link d0/d8/d1c/d3e/d5f/d90/da7/de7/fe4 d0/d8/d1c/d3e/d5f/d83/d8c/ff4 0 2024-03-22T06:15:14.490 INFO:tasks.workunit.client.0.smithi006.stdout:3/731: symlink d0/d4/dbb/lf5 0 2024-03-22T06:15:14.491 INFO:tasks.workunit.client.0.smithi006.stdout:3/732: symlink d0/d8/d1c/d3e/d5f/d90/da7/lf6 0 2024-03-22T06:15:14.495 INFO:tasks.workunit.client.1.smithi117.stdout:8/35: write d1/d2/f7 [2666368,45469] 0 2024-03-22T06:15:14.497 INFO:tasks.workunit.client.0.smithi006.stdout:1/724: dwrite d8/d9/d13/f28 [4194304,4194304] 0 2024-03-22T06:15:14.498 INFO:tasks.workunit.client.1.smithi117.stdout:9/20: write f2 [675224,80727] 0 2024-03-22T06:15:14.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:14 smithi117 ceph-mon[36514]: pgmap v20: 65 pgs: 65 active+clean; 1.6 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 70 MiB/s rd, 96 MiB/s wr, 762 op/s 2024-03-22T06:15:14.506 INFO:tasks.workunit.client.1.smithi117.stdout:9/21: getdents d4 0 2024-03-22T06:15:14.507 INFO:tasks.workunit.client.1.smithi117.stdout:9/22: unlink d4/f6 0 2024-03-22T06:15:14.507 INFO:tasks.workunit.client.1.smithi117.stdout:9/23: readlink - no filename 2024-03-22T06:15:14.510 INFO:tasks.workunit.client.1.smithi117.stdout:9/24: symlink d4/l9 0 2024-03-22T06:15:14.512 INFO:tasks.workunit.client.1.smithi117.stdout:9/25: mknod d4/ca 0 2024-03-22T06:15:14.512 INFO:tasks.workunit.client.1.smithi117.stdout:9/26: readlink d4/l9 0 2024-03-22T06:15:14.514 INFO:tasks.workunit.client.0.smithi006.stdout:3/733: rename d0/d8/d1c/d3e to d0/d4/dbb/df7 0 2024-03-22T06:15:14.514 INFO:tasks.workunit.client.0.smithi006.stdout:3/734: creat d0/d4/d31/d98/dab/ff8 x:0 0 0 2024-03-22T06:15:14.515 INFO:tasks.workunit.client.0.smithi006.stdout:3/735: chown d0/fcf 1470839 1 2024-03-22T06:15:14.515 INFO:tasks.workunit.client.1.smithi117.stdout:9/27: symlink d4/d8/lb 0 2024-03-22T06:15:14.517 INFO:tasks.workunit.client.0.smithi006.stdout:3/736: rename d0/d4/dbb/df7/d68/le8 to d0/d4/dbb/df7/d5f/d90/da7/de7/lf9 0 2024-03-22T06:15:14.526 INFO:tasks.workunit.client.1.smithi117.stdout:7/24: dread f0 [0,4194304] 0 2024-03-22T06:15:14.526 INFO:tasks.workunit.client.1.smithi117.stdout:9/28: dread f2 [0,4194304] 0 2024-03-22T06:15:14.526 INFO:tasks.workunit.client.1.smithi117.stdout:7/25: chown f6 1282084 1 2024-03-22T06:15:14.528 INFO:tasks.workunit.client.1.smithi117.stdout:7/26: rename f0 to f7 0 2024-03-22T06:15:14.529 INFO:tasks.workunit.client.1.smithi117.stdout:7/27: mkdir d8 0 2024-03-22T06:15:14.531 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:14 smithi006 ceph-mon[29301]: pgmap v20: 65 pgs: 65 active+clean; 1.6 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 70 MiB/s rd, 96 MiB/s wr, 762 op/s 2024-03-22T06:15:14.533 INFO:tasks.workunit.client.1.smithi117.stdout:9/29: dread f2 [0,4194304] 0 2024-03-22T06:15:14.536 INFO:tasks.workunit.client.1.smithi117.stdout:4/32: fsync d0/f3 0 2024-03-22T06:15:14.536 INFO:tasks.workunit.client.1.smithi117.stdout:4/33: chown d0/f3 1620368290 1 2024-03-22T06:15:14.536 INFO:tasks.workunit.client.1.smithi117.stdout:8/36: dwrite f0 [0,4194304] 0 2024-03-22T06:15:14.536 INFO:tasks.workunit.client.1.smithi117.stdout:6/20: dwrite d3/f5 [0,4194304] 0 2024-03-22T06:15:14.537 INFO:tasks.workunit.client.1.smithi117.stdout:2/32: dwrite d2/f3 [0,4194304] 0 2024-03-22T06:15:14.541 INFO:tasks.workunit.client.1.smithi117.stdout:4/34: rename d0/d4/f6 to d0/f9 0 2024-03-22T06:15:14.541 INFO:tasks.workunit.client.1.smithi117.stdout:4/35: write d0/f8 [4875409,83968] 0 2024-03-22T06:15:14.542 INFO:tasks.workunit.client.1.smithi117.stdout:8/37: symlink d1/d2/d9/lc 0 2024-03-22T06:15:14.547 INFO:tasks.workunit.client.1.smithi117.stdout:7/28: dwrite f4 [0,4194304] 0 2024-03-22T06:15:14.547 INFO:tasks.workunit.client.1.smithi117.stdout:6/21: truncate d3/f5 4802326 0 2024-03-22T06:15:14.547 INFO:tasks.workunit.client.1.smithi117.stdout:6/22: readlink l2 0 2024-03-22T06:15:14.547 INFO:tasks.workunit.client.1.smithi117.stdout:6/23: write d3/f4 [2263080,130216] 0 2024-03-22T06:15:14.553 INFO:tasks.workunit.client.1.smithi117.stdout:8/38: link d1/d2/l4 d1/d2/d9/ld 0 2024-03-22T06:15:14.554 INFO:tasks.workunit.client.1.smithi117.stdout:7/29: mkdir d8/d9 0 2024-03-22T06:15:14.556 INFO:tasks.workunit.client.1.smithi117.stdout:7/30: mkdir d8/d9/da 0 2024-03-22T06:15:14.557 INFO:tasks.workunit.client.1.smithi117.stdout:7/31: creat d8/fb x:0 0 0 2024-03-22T06:15:14.558 INFO:tasks.workunit.client.1.smithi117.stdout:7/32: chown f6 2743 1 2024-03-22T06:15:14.558 INFO:tasks.workunit.client.1.smithi117.stdout:7/33: mknod d8/cc 0 2024-03-22T06:15:14.560 INFO:tasks.workunit.client.1.smithi117.stdout:4/36: rename d0/f9 to d0/d4/fa 0 2024-03-22T06:15:14.560 INFO:tasks.workunit.client.1.smithi117.stdout:4/37: creat d0/d4/fb x:0 0 0 2024-03-22T06:15:14.566 INFO:tasks.workunit.client.1.smithi117.stdout:9/30: dwrite f2 [0,4194304] 0 2024-03-22T06:15:14.566 INFO:tasks.workunit.client.1.smithi117.stdout:9/31: fsync f2 0 2024-03-22T06:15:14.566 INFO:tasks.workunit.client.1.smithi117.stdout:2/33: dwrite d2/f9 [0,4194304] 0 2024-03-22T06:15:14.566 INFO:tasks.workunit.client.1.smithi117.stdout:2/34: chown d2 582087 1 2024-03-22T06:15:14.566 INFO:tasks.workunit.client.1.smithi117.stdout:2/35: creat d2/fc x:0 0 0 2024-03-22T06:15:14.567 INFO:tasks.workunit.client.1.smithi117.stdout:8/39: rmdir d1/d2 39 2024-03-22T06:15:14.567 INFO:tasks.workunit.client.1.smithi117.stdout:8/40: truncate d1/d2/fa 55582 0 2024-03-22T06:15:14.569 INFO:tasks.workunit.client.1.smithi117.stdout:9/32: link d4/d8/lb d4/d8/lc 0 2024-03-22T06:15:14.569 INFO:tasks.workunit.client.1.smithi117.stdout:7/34: dread f7 [0,4194304] 0 2024-03-22T06:15:14.572 INFO:tasks.workunit.client.1.smithi117.stdout:2/36: rename d2/d5/c7 to d2/cd 0 2024-03-22T06:15:14.575 INFO:tasks.workunit.client.1.smithi117.stdout:9/33: creat d4/d7/fd x:0 0 0 2024-03-22T06:15:14.576 INFO:tasks.workunit.client.1.smithi117.stdout:4/38: rmdir d0/d4 39 2024-03-22T06:15:14.579 INFO:tasks.workunit.client.1.smithi117.stdout:4/39: creat d0/d4/fc x:0 0 0 2024-03-22T06:15:14.580 INFO:tasks.workunit.client.1.smithi117.stdout:4/40: write d0/f2 [785189,64801] 0 2024-03-22T06:15:14.580 INFO:tasks.workunit.client.1.smithi117.stdout:4/41: mknod d0/d4/cd 0 2024-03-22T06:15:14.582 INFO:tasks.workunit.client.1.smithi117.stdout:4/42: rename d0/f8 to d0/d4/fe 0 2024-03-22T06:15:14.582 INFO:tasks.workunit.client.1.smithi117.stdout:4/43: fdatasync d0/d4/fa 0 2024-03-22T06:15:14.582 INFO:tasks.workunit.client.1.smithi117.stdout:4/44: rename d0 to d0/df 22 2024-03-22T06:15:14.582 INFO:tasks.workunit.client.1.smithi117.stdout:2/37: dread f1 [4194304,4194304] 0 2024-03-22T06:15:14.583 INFO:tasks.workunit.client.1.smithi117.stdout:4/45: creat d0/f10 x:0 0 0 2024-03-22T06:15:14.586 INFO:tasks.workunit.client.1.smithi117.stdout:4/46: rename d0/d4/fc to d0/f11 0 2024-03-22T06:15:14.587 INFO:tasks.workunit.client.1.smithi117.stdout:4/47: chown d0/f10 7881177 1 2024-03-22T06:15:14.587 INFO:tasks.workunit.client.1.smithi117.stdout:4/48: write d0/d4/f7 [628150,34622] 0 2024-03-22T06:15:14.587 INFO:tasks.workunit.client.1.smithi117.stdout:8/41: dwrite f0 [0,4194304] 0 2024-03-22T06:15:14.589 INFO:tasks.workunit.client.1.smithi117.stdout:8/42: symlink d1/le 0 2024-03-22T06:15:14.594 INFO:tasks.workunit.client.1.smithi117.stdout:7/35: dwrite f7 [0,4194304] 0 2024-03-22T06:15:14.595 INFO:tasks.workunit.client.1.smithi117.stdout:2/38: read d2/f9 [2018699,123605] 0 2024-03-22T06:15:14.595 INFO:tasks.workunit.client.1.smithi117.stdout:2/39: chown d2/fc 190225385 1 2024-03-22T06:15:14.599 INFO:tasks.workunit.client.1.smithi117.stdout:2/40: symlink d2/d5/db/le 0 2024-03-22T06:15:14.599 INFO:tasks.workunit.client.1.smithi117.stdout:9/34: dwrite f2 [0,4194304] 0 2024-03-22T06:15:14.614 INFO:tasks.workunit.client.1.smithi117.stdout:4/49: dwrite d0/f2 [0,4194304] 0 2024-03-22T06:15:14.615 INFO:tasks.workunit.client.1.smithi117.stdout:4/50: truncate d0/d4/f7 1384058 0 2024-03-22T06:15:14.615 INFO:tasks.workunit.client.1.smithi117.stdout:9/35: write f2 [3812727,4583] 0 2024-03-22T06:15:14.615 INFO:tasks.workunit.client.1.smithi117.stdout:8/43: dwrite d1/d2/f7 [4194304,4194304] 0 2024-03-22T06:15:14.615 INFO:tasks.workunit.client.1.smithi117.stdout:9/36: stat d4 0 2024-03-22T06:15:14.618 INFO:tasks.workunit.client.1.smithi117.stdout:9/37: mknod d4/d7/ce 0 2024-03-22T06:15:14.619 INFO:tasks.workunit.client.1.smithi117.stdout:9/38: write d4/d7/fd [22486,52028] 0 2024-03-22T06:15:14.619 INFO:tasks.workunit.client.1.smithi117.stdout:9/39: creat d4/d7/ff x:0 0 0 2024-03-22T06:15:14.619 INFO:tasks.workunit.client.1.smithi117.stdout:9/40: write d4/d7/ff [472311,116120] 0 2024-03-22T06:15:14.620 INFO:tasks.workunit.client.1.smithi117.stdout:9/41: mkdir d4/d7/d10 0 2024-03-22T06:15:14.621 INFO:tasks.workunit.client.1.smithi117.stdout:9/42: stat d4/d7/ff 0 2024-03-22T06:15:14.621 INFO:tasks.workunit.client.1.smithi117.stdout:9/43: truncate d4/d7/ff 722219 0 2024-03-22T06:15:14.621 INFO:tasks.workunit.client.1.smithi117.stdout:9/44: rmdir d4/d8 39 2024-03-22T06:15:14.622 INFO:tasks.workunit.client.1.smithi117.stdout:9/45: unlink d4/d8/lc 0 2024-03-22T06:15:14.623 INFO:tasks.workunit.client.1.smithi117.stdout:9/46: creat d4/f11 x:0 0 0 2024-03-22T06:15:14.624 INFO:tasks.workunit.client.1.smithi117.stdout:9/47: symlink d4/l12 0 2024-03-22T06:15:14.625 INFO:tasks.workunit.client.1.smithi117.stdout:8/44: write f0 [1984890,125451] 0 2024-03-22T06:15:14.627 INFO:tasks.workunit.client.1.smithi117.stdout:8/45: rename d1/le to d1/lf 0 2024-03-22T06:15:14.628 INFO:tasks.workunit.client.1.smithi117.stdout:4/51: dwrite d0/f11 [0,4194304] 0 2024-03-22T06:15:14.629 INFO:tasks.workunit.client.1.smithi117.stdout:8/46: rename d1/lb to d1/d2/l10 0 2024-03-22T06:15:14.629 INFO:tasks.workunit.client.1.smithi117.stdout:8/47: write d1/d2/fa [425583,72106] 0 2024-03-22T06:15:14.634 INFO:tasks.workunit.client.1.smithi117.stdout:4/52: rename d0/f2 to d0/d4/f12 0 2024-03-22T06:15:14.634 INFO:tasks.workunit.client.1.smithi117.stdout:4/53: stat d0 0 2024-03-22T06:15:14.636 INFO:tasks.workunit.client.1.smithi117.stdout:4/54: symlink d0/d4/l13 0 2024-03-22T06:15:14.637 INFO:tasks.workunit.client.1.smithi117.stdout:4/55: symlink d0/l14 0 2024-03-22T06:15:14.638 INFO:tasks.workunit.client.1.smithi117.stdout:8/48: write d1/d2/f7 [6444777,84446] 0 2024-03-22T06:15:14.640 INFO:tasks.workunit.client.1.smithi117.stdout:8/49: mkdir d1/d2/d11 0 2024-03-22T06:15:14.642 INFO:tasks.workunit.client.1.smithi117.stdout:9/48: dwrite d4/f11 [0,4194304] 0 2024-03-22T06:15:14.642 INFO:tasks.workunit.client.1.smithi117.stdout:9/49: readlink d4/l9 0 2024-03-22T06:15:14.642 INFO:tasks.workunit.client.1.smithi117.stdout:8/50: creat d1/d2/d11/f12 x:0 0 0 2024-03-22T06:15:14.646 INFO:tasks.workunit.client.1.smithi117.stdout:9/50: unlink d4/ca 0 2024-03-22T06:15:14.654 INFO:tasks.workunit.client.1.smithi117.stdout:4/56: dwrite d0/f10 [0,4194304] 0 2024-03-22T06:15:14.657 INFO:tasks.workunit.client.0.smithi006.stdout:2/846: sync 2024-03-22T06:15:14.657 INFO:tasks.workunit.client.0.smithi006.stdout:8/863: sync 2024-03-22T06:15:14.658 INFO:tasks.workunit.client.0.smithi006.stdout:8/864: creat d9/db/dc/de/d3e/f13d x:0 0 0 2024-03-22T06:15:14.658 INFO:tasks.workunit.client.0.smithi006.stdout:7/771: sync 2024-03-22T06:15:14.658 INFO:tasks.workunit.client.0.smithi006.stdout:0/586: sync 2024-03-22T06:15:14.658 INFO:tasks.workunit.client.0.smithi006.stdout:8/865: chown d9/db/d35/dd2/d4a/d4d/l113 2 1 2024-03-22T06:15:14.658 INFO:tasks.workunit.client.0.smithi006.stdout:6/686: sync 2024-03-22T06:15:14.658 INFO:tasks.workunit.client.0.smithi006.stdout:9/660: sync 2024-03-22T06:15:14.658 INFO:tasks.workunit.client.0.smithi006.stdout:6/687: readlink d6/d1d/d5c/d89/l9a 0 2024-03-22T06:15:14.658 INFO:tasks.workunit.client.0.smithi006.stdout:6/688: creat d6/d1b/d2c/fd6 x:0 0 0 2024-03-22T06:15:14.662 INFO:tasks.workunit.client.0.smithi006.stdout:8/866: mknod d9/db/d12/d66/d6c/d80/c13e 0 2024-03-22T06:15:14.663 INFO:tasks.workunit.client.1.smithi117.stdout:4/57: dread d0/f3 [0,4194304] 0 2024-03-22T06:15:14.664 INFO:tasks.workunit.client.1.smithi117.stdout:9/51: write d4/f11 [2780908,4272] 0 2024-03-22T06:15:14.664 INFO:tasks.workunit.client.1.smithi117.stdout:9/52: read d4/d7/fd [54704,11315] 0 2024-03-22T06:15:14.664 INFO:tasks.workunit.client.0.smithi006.stdout:0/587: unlink db/dc/l15 0 2024-03-22T06:15:14.666 INFO:tasks.workunit.client.0.smithi006.stdout:8/867: creat d9/db/d35/dd2/dc4/f13f x:0 0 0 2024-03-22T06:15:14.667 INFO:tasks.workunit.client.0.smithi006.stdout:9/661: getdents d8/d10/d20/d45/db9 0 2024-03-22T06:15:14.668 INFO:tasks.workunit.client.0.smithi006.stdout:8/868: symlink d9/d5e/l140 0 2024-03-22T06:15:14.668 INFO:tasks.workunit.client.0.smithi006.stdout:8/869: read - d9/db/d35/dd2/dc4/ddd/f10d zero size 2024-03-22T06:15:14.668 INFO:tasks.workunit.client.0.smithi006.stdout:6/689: dread d6/f42 [0,4194304] 0 2024-03-22T06:15:14.668 INFO:tasks.workunit.client.0.smithi006.stdout:6/690: dread - d6/d24/f58 zero size 2024-03-22T06:15:14.670 INFO:tasks.workunit.client.0.smithi006.stdout:2/847: write d13/d34/d4d/d4e/d75/d62/f9c [217810,14666] 0 2024-03-22T06:15:14.671 INFO:tasks.workunit.client.0.smithi006.stdout:7/772: write d5/d1e/d63/d85/fad [2109730,30231] 0 2024-03-22T06:15:14.671 INFO:tasks.workunit.client.0.smithi006.stdout:9/662: symlink d8/d10/d23/ld7 0 2024-03-22T06:15:14.671 INFO:tasks.workunit.client.1.smithi117.stdout:8/51: dwrite d1/d2/fa [0,4194304] 0 2024-03-22T06:15:14.674 INFO:tasks.workunit.client.1.smithi117.stdout:4/58: dread d0/f11 [0,4194304] 0 2024-03-22T06:15:14.674 INFO:tasks.workunit.client.1.smithi117.stdout:4/59: creat d0/d4/f15 x:0 0 0 2024-03-22T06:15:14.674 INFO:tasks.workunit.client.0.smithi006.stdout:0/588: write db/d10/d2a/d41/f82 [1313035,82597] 0 2024-03-22T06:15:14.681 INFO:tasks.workunit.client.0.smithi006.stdout:9/663: write d8/d10/d20/f2c [220314,94156] 0 2024-03-22T06:15:14.683 INFO:tasks.workunit.client.0.smithi006.stdout:9/664: dread d8/fb8 [0,4194304] 0 2024-03-22T06:15:14.694 INFO:tasks.workunit.client.0.smithi006.stdout:9/665: read d8/db3/d50/f8e [4065083,15067] 0 2024-03-22T06:15:14.694 INFO:tasks.workunit.client.0.smithi006.stdout:9/666: truncate d8/d10/d23/f7d 9024906 0 2024-03-22T06:15:14.694 INFO:tasks.workunit.client.0.smithi006.stdout:9/667: stat d8/d10 0 2024-03-22T06:15:14.695 INFO:tasks.workunit.client.0.smithi006.stdout:9/668: creat d8/d10/d20/d45/d4d/fd8 x:0 0 0 2024-03-22T06:15:14.699 INFO:tasks.workunit.client.1.smithi117.stdout:8/52: dwrite f0 [0,4194304] 0 2024-03-22T06:15:14.700 INFO:tasks.workunit.client.0.smithi006.stdout:0/589: dwrite db/d3d/d4e/f6f [0,4194304] 0 2024-03-22T06:15:14.718 INFO:tasks.workunit.client.0.smithi006.stdout:0/590: dwrite db/d10/f2f [4194304,4194304] 0 2024-03-22T06:15:14.738 INFO:tasks.workunit.client.0.smithi006.stdout:8/870: truncate f2 2329269 0 2024-03-22T06:15:14.738 INFO:tasks.workunit.client.0.smithi006.stdout:8/871: fsync d9/db/dc/de/d3e/f13d 0 2024-03-22T06:15:14.738 INFO:tasks.workunit.client.0.smithi006.stdout:8/872: dread - d9/db/d12/d1e/fa8 zero size 2024-03-22T06:15:14.738 INFO:tasks.workunit.client.0.smithi006.stdout:6/691: mknod d6/d1d/cd7 0 2024-03-22T06:15:14.738 INFO:tasks.workunit.client.0.smithi006.stdout:6/692: readlink d6/d6f/ld4 0 2024-03-22T06:15:14.738 INFO:tasks.workunit.client.0.smithi006.stdout:6/693: chown d6/d1d/c85 17040610 1 2024-03-22T06:15:14.741 INFO:tasks.workunit.client.0.smithi006.stdout:2/848: mknod d13/d34/d4d/d4e/d75/da5/d82/c13f 0 2024-03-22T06:15:14.743 INFO:tasks.workunit.client.1.smithi117.stdout:9/53: truncate d4/d7/ff 273711 0 2024-03-22T06:15:14.744 INFO:tasks.workunit.client.1.smithi117.stdout:9/54: read f2 [3697366,67105] 0 2024-03-22T06:15:14.747 INFO:tasks.workunit.client.0.smithi006.stdout:6/694: dread d6/f18 [0,4194304] 0 2024-03-22T06:15:14.747 INFO:tasks.workunit.client.0.smithi006.stdout:6/695: chown d6/d1d/d5c/d89/fb0 124421444 1 2024-03-22T06:15:14.750 INFO:tasks.workunit.client.1.smithi117.stdout:4/60: mknod d0/d4/c16 0 2024-03-22T06:15:14.750 INFO:tasks.workunit.client.1.smithi117.stdout:4/61: chown d0/f11 461 1 2024-03-22T06:15:14.750 INFO:tasks.workunit.client.0.smithi006.stdout:6/696: dread d6/f51 [4194304,4194304] 0 2024-03-22T06:15:14.752 INFO:tasks.workunit.client.1.smithi117.stdout:8/53: creat d1/d2/d9/f13 x:0 0 0 2024-03-22T06:15:14.759 INFO:tasks.workunit.client.0.smithi006.stdout:6/697: dread d6/d1b/d2c/f72 [0,4194304] 0 2024-03-22T06:15:14.765 INFO:tasks.workunit.client.1.smithi117.stdout:9/55: creat d4/d8/f13 x:0 0 0 2024-03-22T06:15:14.773 INFO:tasks.workunit.client.1.smithi117.stdout:8/54: dwrite d1/d2/fa [0,4194304] 0 2024-03-22T06:15:14.781 INFO:tasks.workunit.client.1.smithi117.stdout:8/55: dread d1/d2/f7 [4194304,4194304] 0 2024-03-22T06:15:14.786 INFO:tasks.workunit.client.0.smithi006.stdout:0/591: read db/dc/f21 [3015698,8939] 0 2024-03-22T06:15:14.804 INFO:tasks.workunit.client.0.smithi006.stdout:8/873: getdents d9/db/d35/dd2/d4a 0 2024-03-22T06:15:14.804 INFO:tasks.workunit.client.0.smithi006.stdout:8/874: fsync d9/db/d12/f12a 0 2024-03-22T06:15:14.807 INFO:tasks.workunit.client.0.smithi006.stdout:2/849: mknod d13/d20/d26/c140 0 2024-03-22T06:15:14.813 INFO:tasks.workunit.client.0.smithi006.stdout:6/698: rename d6/d24/d44/fa0 to d6/d1d/d5c/d81/dab/fd8 0 2024-03-22T06:15:14.828 INFO:tasks.workunit.client.0.smithi006.stdout:2/850: creat d13/d69/d10f/deb/f141 x:0 0 0 2024-03-22T06:15:14.831 INFO:tasks.workunit.client.0.smithi006.stdout:6/699: symlink d6/d1b/d2c/ld9 0 2024-03-22T06:15:14.834 INFO:tasks.workunit.client.0.smithi006.stdout:8/875: getdents d9/db/d35/dd2/d4a/da7 0 2024-03-22T06:15:14.834 INFO:tasks.workunit.client.0.smithi006.stdout:8/876: truncate d9/d14/dcc/f133 46947 0 2024-03-22T06:15:14.835 INFO:tasks.workunit.client.0.smithi006.stdout:0/592: dwrite db/d10/f14 [0,4194304] 0 2024-03-22T06:15:14.837 INFO:tasks.workunit.client.0.smithi006.stdout:2/851: mkdir d13/d34/d4d/d4e/d75/da5/d82/dda/d142 0 2024-03-22T06:15:14.843 INFO:tasks.workunit.client.0.smithi006.stdout:8/877: dread d9/db/d35/dd2/d4a/d4d/f8d [0,4194304] 0 2024-03-22T06:15:14.843 INFO:tasks.workunit.client.0.smithi006.stdout:8/878: truncate d9/db/d12/d66/d10c/d122/f138 95139 0 2024-03-22T06:15:14.843 INFO:tasks.workunit.client.0.smithi006.stdout:8/879: read d9/fb4 [2487366,95058] 0 2024-03-22T06:15:14.844 INFO:tasks.workunit.client.0.smithi006.stdout:6/700: dwrite d6/d7c/d92/faa [4194304,4194304] 0 2024-03-22T06:15:14.844 INFO:tasks.workunit.client.0.smithi006.stdout:6/701: fsync d6/d1b/f97 0 2024-03-22T06:15:14.851 INFO:tasks.workunit.client.0.smithi006.stdout:8/880: mknod d9/d14/c141 0 2024-03-22T06:15:14.856 INFO:tasks.workunit.client.0.smithi006.stdout:0/593: dread db/dc/d38/f3a [0,4194304] 0 2024-03-22T06:15:14.857 INFO:tasks.workunit.client.0.smithi006.stdout:6/702: creat d6/d1b/dae/fda x:0 0 0 2024-03-22T06:15:14.857 INFO:tasks.workunit.client.0.smithi006.stdout:6/703: write d6/d1d/d5c/d81/f65 [608673,88995] 0 2024-03-22T06:15:14.857 INFO:tasks.workunit.client.0.smithi006.stdout:2/852: read d13/d20/f4b [543006,3375] 0 2024-03-22T06:15:14.858 INFO:tasks.workunit.client.0.smithi006.stdout:2/853: chown d13/d53/d6b 27 1 2024-03-22T06:15:14.861 INFO:tasks.workunit.client.0.smithi006.stdout:6/704: mknod d6/d1d/d5c/d81/d8a/dac/cdb 0 2024-03-22T06:15:14.863 INFO:tasks.workunit.client.0.smithi006.stdout:2/854: truncate d13/d34/d4d/d4e/d75/d62/f100 2602519 0 2024-03-22T06:15:14.865 INFO:tasks.workunit.client.0.smithi006.stdout:0/594: link db/d3d/d4e/c9e db/d10/d35/d7b/da1/da6/dc4/ccc 0 2024-03-22T06:15:14.865 INFO:tasks.workunit.client.0.smithi006.stdout:0/595: chown db/d10/d2a/f5e 3 1 2024-03-22T06:15:14.866 INFO:tasks.workunit.client.0.smithi006.stdout:8/881: dwrite d9/db/d35/dd2/d76/db3/d132/f8e [0,4194304] 0 2024-03-22T06:15:14.870 INFO:tasks.workunit.client.0.smithi006.stdout:2/855: mkdir d13/d20/d143 0 2024-03-22T06:15:14.884 INFO:tasks.workunit.client.0.smithi006.stdout:2/856: creat d13/d53/d6b/dd8/d11f/f144 x:0 0 0 2024-03-22T06:15:14.886 INFO:tasks.workunit.client.0.smithi006.stdout:2/857: rmdir d13/d34/d4d/d4e/d75/dcc 39 2024-03-22T06:15:14.894 INFO:tasks.workunit.client.0.smithi006.stdout:0/596: dwrite db/d3d/d4e/f54 [0,4194304] 0 2024-03-22T06:15:14.894 INFO:tasks.workunit.client.0.smithi006.stdout:8/882: dwrite d9/db/d12/f110 [0,4194304] 0 2024-03-22T06:15:14.896 INFO:tasks.workunit.client.0.smithi006.stdout:0/597: mkdir db/d10/d33/d9d/dcd 0 2024-03-22T06:15:14.897 INFO:tasks.workunit.client.0.smithi006.stdout:0/598: dread - db/d10/d33/f67 zero size 2024-03-22T06:15:14.907 INFO:tasks.workunit.client.0.smithi006.stdout:2/858: dwrite d13/d53/dc5/fd7 [4194304,4194304] 0 2024-03-22T06:15:14.907 INFO:tasks.workunit.client.0.smithi006.stdout:8/883: mknod d9/db/dc/c142 0 2024-03-22T06:15:14.910 INFO:tasks.workunit.client.0.smithi006.stdout:0/599: mkdir db/d10/d33/dce 0 2024-03-22T06:15:14.910 INFO:tasks.workunit.client.0.smithi006.stdout:0/600: fdatasync db/dc/fa9 0 2024-03-22T06:15:14.918 INFO:tasks.workunit.client.0.smithi006.stdout:2/859: rename d13/d20/d26/d19/f73 to d13/d53/f145 0 2024-03-22T06:15:14.919 INFO:tasks.workunit.client.0.smithi006.stdout:8/884: truncate d9/db/d12/d66/d10c/d125/fe7 685622 0 2024-03-22T06:15:14.922 INFO:tasks.workunit.client.0.smithi006.stdout:0/601: dwrite db/dc/d38/fc1 [0,4194304] 0 2024-03-22T06:15:14.923 INFO:tasks.workunit.client.0.smithi006.stdout:8/885: creat d9/db/d12/d1e/d11e/f143 x:0 0 0 2024-03-22T06:15:14.923 INFO:tasks.workunit.client.0.smithi006.stdout:8/886: chown d9/db/d35/dd2/d76/db3/dc3/d89/db8/d130 367041 1 2024-03-22T06:15:14.923 INFO:tasks.workunit.client.0.smithi006.stdout:8/887: stat d9/d5e/f69 0 2024-03-22T06:15:14.931 INFO:tasks.workunit.client.0.smithi006.stdout:0/602: symlink db/d3d/d4e/lcf 0 2024-03-22T06:15:14.931 INFO:tasks.workunit.client.0.smithi006.stdout:8/888: getdents d9/db/d12/d66/d6c/d71 0 2024-03-22T06:15:14.934 INFO:tasks.workunit.client.0.smithi006.stdout:0/603: link db/d3d/d4e/l60 db/d10/d33/dce/ld0 0 2024-03-22T06:15:14.938 INFO:tasks.workunit.client.0.smithi006.stdout:8/889: dread d9/f21 [0,4194304] 0 2024-03-22T06:15:14.939 INFO:tasks.workunit.client.0.smithi006.stdout:0/604: rename db/dc/d37/c3b to db/dc/d37/cd1 0 2024-03-22T06:15:14.944 INFO:tasks.workunit.client.0.smithi006.stdout:8/890: truncate d9/db/d12/d1e/f30 1182148 0 2024-03-22T06:15:14.946 INFO:tasks.workunit.client.0.smithi006.stdout:0/605: rmdir db/d10/d33/d9d 39 2024-03-22T06:15:14.951 INFO:tasks.workunit.client.0.smithi006.stdout:8/891: rename d9/db/d12/d66/d6c/d80/fa4 to d9/db/d35/dd2/d76/dca/dee/f144 0 2024-03-22T06:15:14.951 INFO:tasks.workunit.client.0.smithi006.stdout:8/892: write d9/db/d12/f12a [444871,108454] 0 2024-03-22T06:15:14.951 INFO:tasks.workunit.client.0.smithi006.stdout:8/893: write d9/db/d12/d66/f9b [977307,16642] 0 2024-03-22T06:15:14.955 INFO:tasks.workunit.client.0.smithi006.stdout:0/606: creat db/d10/d35/d7b/da1/da6/fd2 x:0 0 0 2024-03-22T06:15:14.960 INFO:tasks.workunit.client.0.smithi006.stdout:0/607: creat db/d10/d35/fd3 x:0 0 0 2024-03-22T06:15:14.968 INFO:tasks.workunit.client.0.smithi006.stdout:0/608: write db/d3d/d4e/f6f [3277175,97132] 0 2024-03-22T06:15:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:0/609: truncate db/d3d/d4e/f9b 3362514 0 2024-03-22T06:15:14.975 INFO:tasks.workunit.client.0.smithi006.stdout:0/610: creat db/d10/d33/d9d/dcd/fd4 x:0 0 0 2024-03-22T06:15:14.982 INFO:tasks.workunit.client.0.smithi006.stdout:7/773: rmdir d5/d1e/d63/d85 39 2024-03-22T06:15:14.985 INFO:tasks.workunit.client.0.smithi006.stdout:7/774: rename d5/d1e/d29/d10/f3d to d5/d1e/d63/d85/da1/ffd 0 2024-03-22T06:15:14.994 INFO:tasks.workunit.client.0.smithi006.stdout:6/705: rmdir d6/d1b/d2c 39 2024-03-22T06:15:14.999 INFO:tasks.workunit.client.0.smithi006.stdout:6/706: rename d6/d1d/l87 to d6/d1b/dae/dbc/ldc 0 2024-03-22T06:15:14.999 INFO:tasks.workunit.client.0.smithi006.stdout:6/707: fsync d6/d1b/f82 0 2024-03-22T06:15:14.999 INFO:tasks.workunit.client.0.smithi006.stdout:6/708: chown d6/fc1 1721465 1 2024-03-22T06:15:14.999 INFO:tasks.workunit.client.0.smithi006.stdout:6/709: write d6/d1d/fce [856024,39186] 0 2024-03-22T06:15:14.999 INFO:tasks.workunit.client.0.smithi006.stdout:6/710: chown d6/d1b 248 1 2024-03-22T06:15:15.007 INFO:tasks.workunit.client.0.smithi006.stdout:7/775: dwrite d5/d1e/d29/d1f/d2d/d41/f7b [0,4194304] 0 2024-03-22T06:15:15.022 INFO:tasks.workunit.client.0.smithi006.stdout:6/711: dwrite d6/d1b/dae/fb4 [0,4194304] 0 2024-03-22T06:15:15.022 INFO:tasks.workunit.client.0.smithi006.stdout:6/712: creat d6/d1d/fdd x:0 0 0 2024-03-22T06:15:15.030 INFO:tasks.workunit.client.0.smithi006.stdout:6/713: creat d6/d6f/d94/fde x:0 0 0 2024-03-22T06:15:15.030 INFO:tasks.workunit.client.0.smithi006.stdout:6/714: chown d6/d1d/cd7 4383052 1 2024-03-22T06:15:15.032 INFO:tasks.workunit.client.0.smithi006.stdout:6/715: readlink d6/d1b/d2c/l36 0 2024-03-22T06:15:15.034 INFO:tasks.workunit.client.0.smithi006.stdout:6/716: symlink d6/d1d/d5c/d81/d8a/dac/ldf 0 2024-03-22T06:15:15.036 INFO:tasks.workunit.client.0.smithi006.stdout:6/717: symlink d6/d1d/d5c/le0 0 2024-03-22T06:15:15.036 INFO:tasks.workunit.client.0.smithi006.stdout:6/718: readlink d6/d1d/d5c/le0 0 2024-03-22T06:15:15.036 INFO:tasks.workunit.client.0.smithi006.stdout:6/719: fsync d6/d1b/d2c/f59 0 2024-03-22T06:15:15.038 INFO:tasks.workunit.client.0.smithi006.stdout:6/720: truncate d6/d6f/f6e 3764793 0 2024-03-22T06:15:15.038 INFO:tasks.workunit.client.0.smithi006.stdout:6/721: chown d6/d1b/d2c/f7d 167054 1 2024-03-22T06:15:15.059 INFO:tasks.workunit.client.0.smithi006.stdout:6/722: dread d6/d7c/d92/fad [0,4194304] 0 2024-03-22T06:15:15.075 INFO:tasks.workunit.client.0.smithi006.stdout:6/723: dwrite d6/d1d/f20 [0,4194304] 0 2024-03-22T06:15:15.077 INFO:tasks.workunit.client.0.smithi006.stdout:6/724: creat d6/d1d/d32/fe1 x:0 0 0 2024-03-22T06:15:15.079 INFO:tasks.workunit.client.0.smithi006.stdout:9/669: unlink d8/d10/d20/d45/f9b 0 2024-03-22T06:15:15.082 INFO:tasks.workunit.client.0.smithi006.stdout:6/725: link d6/d24/f58 d6/d1d/dc6/fe2 0 2024-03-22T06:15:15.084 INFO:tasks.workunit.client.0.smithi006.stdout:9/670: mkdir d8/d1f/dd9 0 2024-03-22T06:15:15.089 INFO:tasks.workunit.client.0.smithi006.stdout:6/726: rename d6/d77/db7/cc0 to d6/d1d/d99/dcd/ce3 0 2024-03-22T06:15:15.091 INFO:tasks.workunit.client.0.smithi006.stdout:6/727: creat d6/d1d/d99/fe4 x:0 0 0 2024-03-22T06:15:15.096 INFO:tasks.workunit.client.0.smithi006.stdout:8/894: link d9/db/d35/dd2/dc4/d104/f124 d9/db/d35/dd2/d76/f145 0 2024-03-22T06:15:15.096 INFO:tasks.workunit.client.0.smithi006.stdout:6/728: read d6/d1b/f2a [3647417,72105] 0 2024-03-22T06:15:15.101 INFO:tasks.workunit.client.0.smithi006.stdout:8/895: dread d9/db/dc/de/f22 [0,4194304] 0 2024-03-22T06:15:15.104 INFO:tasks.workunit.client.0.smithi006.stdout:9/671: dwrite d8/d10/f17 [0,4194304] 0 2024-03-22T06:15:15.105 INFO:tasks.workunit.client.0.smithi006.stdout:6/729: write d6/d7c/d83/fa2 [1478525,108161] 0 2024-03-22T06:15:15.115 INFO:tasks.workunit.client.0.smithi006.stdout:9/672: dread d8/db3/d50/f8e [0,4194304] 0 2024-03-22T06:15:15.115 INFO:tasks.workunit.client.0.smithi006.stdout:0/611: write db/d10/d2a/f5e [1361634,130312] 0 2024-03-22T06:15:15.115 INFO:tasks.workunit.client.0.smithi006.stdout:8/896: dread d9/db/d35/fbb [0,4194304] 0 2024-03-22T06:15:15.115 INFO:tasks.workunit.client.0.smithi006.stdout:8/897: truncate d9/db/d35/dd2/d76/f88 274237 0 2024-03-22T06:15:15.118 INFO:tasks.workunit.client.0.smithi006.stdout:0/612: rename db/dc/f21 to db/d10/d35/d7b/da1/fd5 0 2024-03-22T06:15:15.124 INFO:tasks.workunit.client.0.smithi006.stdout:0/613: truncate db/d10/f1b 1026209 0 2024-03-22T06:15:15.124 INFO:tasks.workunit.client.0.smithi006.stdout:0/614: chown db/d10/d35/d7b/da1/da6/fd2 3534 1 2024-03-22T06:15:15.124 INFO:tasks.workunit.client.0.smithi006.stdout:9/673: write d8/d10/d20/d45/d4d/f85 [1630392,124476] 0 2024-03-22T06:15:15.127 INFO:tasks.workunit.client.0.smithi006.stdout:0/615: rename db/dc/fa9 to db/d10/dc2/fd6 0 2024-03-22T06:15:15.127 INFO:tasks.workunit.client.0.smithi006.stdout:0/616: write db/d10/d35/d7b/da1/f8b [393026,76611] 0 2024-03-22T06:15:15.130 INFO:tasks.workunit.client.0.smithi006.stdout:8/898: dwrite d9/db/d35/da9/faa [0,4194304] 0 2024-03-22T06:15:15.131 INFO:tasks.workunit.client.0.smithi006.stdout:9/674: link d8/d10/d28/dba/cca d8/d10/d28/cda 0 2024-03-22T06:15:15.133 INFO:tasks.workunit.client.0.smithi006.stdout:0/617: creat db/d10/d33/db4/fd7 x:0 0 0 2024-03-22T06:15:15.139 INFO:tasks.workunit.client.0.smithi006.stdout:0/618: rename db/dc/d38/d50/l89 to db/d10/d35/d7b/ld8 0 2024-03-22T06:15:15.144 INFO:tasks.workunit.client.0.smithi006.stdout:9/675: write d8/d10/f2e [1372871,35398] 0 2024-03-22T06:15:15.144 INFO:tasks.workunit.client.0.smithi006.stdout:9/676: fsync d8/db3/d50/fa2 0 2024-03-22T06:15:15.144 INFO:tasks.workunit.client.0.smithi006.stdout:9/677: readlink d8/db3/la1 0 2024-03-22T06:15:15.147 INFO:tasks.workunit.client.0.smithi006.stdout:8/899: dwrite d9/db/d35/dd2/dc4/f119 [0,4194304] 0 2024-03-22T06:15:15.148 INFO:tasks.workunit.client.0.smithi006.stdout:8/900: chown d9/d5e/fc2 658 1 2024-03-22T06:15:15.154 INFO:tasks.workunit.client.0.smithi006.stdout:8/901: getdents d9/db/d35/da9/df6 0 2024-03-22T06:15:15.157 INFO:tasks.workunit.client.0.smithi006.stdout:9/678: dwrite d8/d10/d82/fc5 [0,4194304] 0 2024-03-22T06:15:15.157 INFO:tasks.workunit.client.0.smithi006.stdout:8/902: rename d9/db/dc/de/d3e/f10f to d9/db/d12/d7e/f146 0 2024-03-22T06:15:15.163 INFO:tasks.workunit.client.0.smithi006.stdout:9/679: rename d8/d10/d20/d45/fcd to d8/d10/d28/fdb 0 2024-03-22T06:15:15.165 INFO:tasks.workunit.client.0.smithi006.stdout:8/903: rename d9/db/dc/de/c24 to d9/db/d35/dd2/d76/db3/dc3/d12d/c147 0 2024-03-22T06:15:15.167 INFO:tasks.workunit.client.0.smithi006.stdout:9/680: mknod d8/d10/d28/dc2/cdc 0 2024-03-22T06:15:15.167 INFO:tasks.workunit.client.0.smithi006.stdout:9/681: chown d8/d10/d28/fb6 62 1 2024-03-22T06:15:15.167 INFO:tasks.workunit.client.0.smithi006.stdout:9/682: chown d8/d10/d28/dc2/cc9 293007013 1 2024-03-22T06:15:15.169 INFO:tasks.workunit.client.0.smithi006.stdout:8/904: mknod d9/db/d12/d66/d6c/c148 0 2024-03-22T06:15:15.169 INFO:tasks.workunit.client.0.smithi006.stdout:8/905: write d9/db/dc/f1f [2454875,95359] 0 2024-03-22T06:15:15.171 INFO:tasks.workunit.client.0.smithi006.stdout:9/683: creat d8/d10/d20/d45/db9/fdd x:0 0 0 2024-03-22T06:15:15.171 INFO:tasks.workunit.client.0.smithi006.stdout:9/684: fdatasync d8/fb 0 2024-03-22T06:15:15.190 INFO:tasks.workunit.client.0.smithi006.stdout:8/906: dwrite d9/db/d12/d66/d10c/d125/da3/fb2 [4194304,4194304] 0 2024-03-22T06:15:15.190 INFO:tasks.workunit.client.0.smithi006.stdout:8/907: truncate d9/db/d35/dd2/d76/db3/dc3/d89/db8/ff5 681553 0 2024-03-22T06:15:15.193 INFO:tasks.workunit.client.0.smithi006.stdout:0/619: mknod db/cd9 0 2024-03-22T06:15:15.193 INFO:tasks.workunit.client.0.smithi006.stdout:0/620: truncate db/d10/fb5 502005 0 2024-03-22T06:15:15.196 INFO:tasks.workunit.client.0.smithi006.stdout:0/621: dread - db/d87/f9f zero size 2024-03-22T06:15:15.196 INFO:tasks.workunit.client.0.smithi006.stdout:0/622: stat db/d3d/d4e 0 2024-03-22T06:15:15.304 INFO:tasks.workunit.client.0.smithi006.stdout:6/730: dwrite d6/d1d/d32/f41 [0,4194304] 0 2024-03-22T06:15:15.306 INFO:tasks.workunit.client.0.smithi006.stdout:6/731: rmdir d6/d1d/d5c 39 2024-03-22T06:15:15.309 INFO:tasks.workunit.client.0.smithi006.stdout:6/732: read d6/d1d/d99/fc2 [1280535,91933] 0 2024-03-22T06:15:15.311 INFO:tasks.workunit.client.0.smithi006.stdout:6/733: mknod d6/d1d/ce5 0 2024-03-22T06:15:15.311 INFO:tasks.workunit.client.0.smithi006.stdout:6/734: chown d6/d6f/la4 1491 1 2024-03-22T06:15:15.313 INFO:tasks.workunit.client.0.smithi006.stdout:6/735: getdents d6/d1d/d5c/d81 0 2024-03-22T06:15:15.313 INFO:tasks.workunit.client.0.smithi006.stdout:6/736: write d6/d1d/d99/fb1 [692147,85272] 0 2024-03-22T06:15:15.314 INFO:tasks.workunit.client.0.smithi006.stdout:6/737: write d6/f42 [4787976,73196] 0 2024-03-22T06:15:15.317 INFO:tasks.workunit.client.0.smithi006.stdout:6/738: link d6/d1d/d32/l3f d6/d24/le6 0 2024-03-22T06:15:15.319 INFO:tasks.workunit.client.0.smithi006.stdout:6/739: symlink d6/d1d/d5c/d81/d8a/dac/le7 0 2024-03-22T06:15:15.322 INFO:tasks.workunit.client.0.smithi006.stdout:6/740: dread d6/d1d/d5c/d81/dab/fd8 [0,4194304] 0 2024-03-22T06:15:15.322 INFO:tasks.workunit.client.0.smithi006.stdout:6/741: fdatasync d6/f16 0 2024-03-22T06:15:15.322 INFO:tasks.workunit.client.0.smithi006.stdout:6/742: write d6/d24/fa8 [666176,11051] 0 2024-03-22T06:15:15.324 INFO:tasks.workunit.client.0.smithi006.stdout:6/743: getdents d6/d1b/dae/dbc 0 2024-03-22T06:15:15.324 INFO:tasks.workunit.client.0.smithi006.stdout:6/744: chown d6/d1d/d5c/le0 176632 1 2024-03-22T06:15:15.327 INFO:tasks.workunit.client.0.smithi006.stdout:6/745: symlink d6/d1d/d5c/d89/le8 0 2024-03-22T06:15:15.327 INFO:tasks.workunit.client.0.smithi006.stdout:6/746: dread - d6/d1b/d2c/fd1 zero size 2024-03-22T06:15:15.327 INFO:tasks.workunit.client.0.smithi006.stdout:6/747: readlink d6/l1a 0 2024-03-22T06:15:15.327 INFO:tasks.workunit.client.0.smithi006.stdout:6/748: truncate d6/d1d/d32/f45 1523818 0 2024-03-22T06:15:15.329 INFO:tasks.workunit.client.0.smithi006.stdout:6/749: getdents d6/d1d/dc6 0 2024-03-22T06:15:15.329 INFO:tasks.workunit.client.0.smithi006.stdout:3/737: sync 2024-03-22T06:15:15.330 INFO:tasks.workunit.client.0.smithi006.stdout:5/860: sync 2024-03-22T06:15:15.330 INFO:tasks.workunit.client.0.smithi006.stdout:4/837: sync 2024-03-22T06:15:15.330 INFO:tasks.workunit.client.0.smithi006.stdout:1/725: sync 2024-03-22T06:15:15.330 INFO:tasks.workunit.client.0.smithi006.stdout:1/726: chown d8/cf 3019 1 2024-03-22T06:15:15.334 INFO:tasks.workunit.client.0.smithi006.stdout:6/750: dread d6/d1b/d2c/f8c [0,4194304] 0 2024-03-22T06:15:15.334 INFO:tasks.workunit.client.0.smithi006.stdout:6/751: readlink d6/d6f/l8f 0 2024-03-22T06:15:15.336 INFO:tasks.workunit.client.0.smithi006.stdout:4/838: mknod d6/d13/d3b/c122 0 2024-03-22T06:15:15.336 INFO:tasks.workunit.client.0.smithi006.stdout:4/839: write d6/d1b/d20/d63/d93/d9a/d8f/da1/db5/f9b [597411,24045] 0 2024-03-22T06:15:15.336 INFO:tasks.workunit.client.0.smithi006.stdout:4/840: chown d6/d1b/d20/d63/d93/d9a/d8f/da1 1998185 1 2024-03-22T06:15:15.361 INFO:tasks.workunit.client.0.smithi006.stdout:3/738: dwrite d0/d4/dbb/df7/d5f/d90/da7/de7/f7f [0,4194304] 0 2024-03-22T06:15:15.363 INFO:tasks.workunit.client.0.smithi006.stdout:3/739: rename d0/d8/d88/ddf/dec to d0/d8/d88/dfa 0 2024-03-22T06:15:15.364 INFO:tasks.workunit.client.0.smithi006.stdout:3/740: creat d0/d8/d1c/d49/d79/ffb x:0 0 0 2024-03-22T06:15:15.366 INFO:tasks.workunit.client.0.smithi006.stdout:5/861: dwrite d0/db/d16/d1c/d34/d35/d85/dda/fd3 [0,4194304] 0 2024-03-22T06:15:15.368 INFO:tasks.workunit.client.0.smithi006.stdout:1/727: dwrite d8/d9/d13/d24/d32/d34/fb3 [0,4194304] 0 2024-03-22T06:15:15.375 INFO:tasks.workunit.client.0.smithi006.stdout:5/862: rename d0/db/d50/d82/db5 to d0/db/d50/d96/d106 0 2024-03-22T06:15:15.378 INFO:tasks.workunit.client.0.smithi006.stdout:3/741: truncate d0/d7e/f80 1205775 0 2024-03-22T06:15:15.378 INFO:tasks.workunit.client.0.smithi006.stdout:3/742: fdatasync d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/f30 0 2024-03-22T06:15:15.379 INFO:tasks.workunit.client.0.smithi006.stdout:1/728: dread d8/d9/d10/f19 [0,4194304] 0 2024-03-22T06:15:15.379 INFO:tasks.workunit.client.0.smithi006.stdout:1/729: stat l4 0 2024-03-22T06:15:15.379 INFO:tasks.workunit.client.0.smithi006.stdout:1/730: chown d8/d9/d13/d39/d60/d8f/dea 920 1 2024-03-22T06:15:15.383 INFO:tasks.workunit.client.0.smithi006.stdout:1/731: truncate d8/d9/d10/f19 1023061 0 2024-03-22T06:15:15.386 INFO:tasks.workunit.client.0.smithi006.stdout:3/743: link d0/cd d0/cfc 0 2024-03-22T06:15:15.388 INFO:tasks.workunit.client.0.smithi006.stdout:1/732: mknod d8/d9/d10/d7c/cf5 0 2024-03-22T06:15:15.390 INFO:tasks.workunit.client.0.smithi006.stdout:3/744: rename d0/d8/d88/dfa/dcb/fe6 to d0/d8/d88/ffd 0 2024-03-22T06:15:15.393 INFO:tasks.workunit.client.0.smithi006.stdout:1/733: link d8/d9/d10/d7c/f82 d8/d71/dcd/ff6 0 2024-03-22T06:15:15.411 INFO:tasks.workunit.client.0.smithi006.stdout:3/745: dwrite d0/d4/dbb/df7/f60 [0,4194304] 0 2024-03-22T06:15:15.413 INFO:tasks.workunit.client.0.smithi006.stdout:3/746: creat d0/d4/d31/ffe x:0 0 0 2024-03-22T06:15:15.417 INFO:tasks.workunit.client.0.smithi006.stdout:1/734: dwrite d8/d9/f1e [0,4194304] 0 2024-03-22T06:15:15.421 INFO:tasks.workunit.client.0.smithi006.stdout:1/735: mknod d8/d9/d13/d24/d32/dc7/cf7 0 2024-03-22T06:15:15.421 INFO:tasks.workunit.client.0.smithi006.stdout:3/747: dread d0/d8/d1c/f4f [0,4194304] 0 2024-03-22T06:15:15.423 INFO:tasks.workunit.client.0.smithi006.stdout:1/736: creat d8/d9/d3a/d40/d59/db1/ff8 x:0 0 0 2024-03-22T06:15:15.425 INFO:tasks.workunit.client.0.smithi006.stdout:3/748: mknod d0/d8/d1c/d49/cff 0 2024-03-22T06:15:15.425 INFO:tasks.workunit.client.0.smithi006.stdout:3/749: fsync d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/f35 0 2024-03-22T06:15:15.427 INFO:tasks.workunit.client.0.smithi006.stdout:1/737: rename d8/d33/lc4 to d8/d9/d13/lf9 0 2024-03-22T06:15:15.427 INFO:tasks.workunit.client.0.smithi006.stdout:1/738: stat d8/fa2 0 2024-03-22T06:15:15.451 INFO:tasks.workunit.client.0.smithi006.stdout:3/750: dwrite d0/d4/fa9 [4194304,4194304] 0 2024-03-22T06:15:15.451 INFO:tasks.workunit.client.0.smithi006.stdout:3/751: write d0/d8/d1c/d49/d79/ffb [114047,95106] 0 2024-03-22T06:15:15.451 INFO:tasks.workunit.client.0.smithi006.stdout:1/739: dwrite d8/d9/d3a/d40/d44/f76 [0,4194304] 0 2024-03-22T06:15:15.455 INFO:tasks.workunit.client.0.smithi006.stdout:3/752: truncate d0/d8/f1a 1661665 0 2024-03-22T06:15:15.455 INFO:tasks.workunit.client.0.smithi006.stdout:3/753: fsync d0/d4/d31/d98/dab/ff8 0 2024-03-22T06:15:15.455 INFO:tasks.workunit.client.0.smithi006.stdout:3/754: write d0/d4/d53/de0/fed [785304,95653] 0 2024-03-22T06:15:15.455 INFO:tasks.workunit.client.0.smithi006.stdout:3/755: read d0/d8/d88/dfa/f86 [1711332,5620] 0 2024-03-22T06:15:15.455 INFO:tasks.workunit.client.0.smithi006.stdout:3/756: chown d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/fa4 94624 1 2024-03-22T06:15:15.465 INFO:tasks.workunit.client.0.smithi006.stdout:1/740: symlink d8/d9/d3a/lfa 0 2024-03-22T06:15:15.477 INFO:tasks.workunit.client.0.smithi006.stdout:3/757: dwrite d0/d4/d31/fcc [0,4194304] 0 2024-03-22T06:15:15.481 INFO:tasks.workunit.client.0.smithi006.stdout:1/741: dwrite d8/d9/d3a/d40/d44/d80/fab [0,4194304] 0 2024-03-22T06:15:15.483 INFO:tasks.workunit.client.0.smithi006.stdout:3/758: creat d0/d8/d1c/d72/f100 x:0 0 0 2024-03-22T06:15:15.485 INFO:tasks.workunit.client.0.smithi006.stdout:3/759: dread d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/fa4 [0,4194304] 0 2024-03-22T06:15:15.485 INFO:tasks.workunit.client.0.smithi006.stdout:3/760: chown d0/d4/dbb/df7/d5f/d83/d8c 11259 1 2024-03-22T06:15:15.487 INFO:tasks.workunit.client.0.smithi006.stdout:1/742: rename d8/d9/d3a/d18/dd2/fe3 to d8/d9/d13/d24/d32/ffb 0 2024-03-22T06:15:15.491 INFO:tasks.workunit.client.0.smithi006.stdout:3/761: link d0/d8/d88/dfa/f5b d0/d8/d88/ddf/df0/f101 0 2024-03-22T06:15:15.492 INFO:tasks.workunit.client.0.smithi006.stdout:4/841: dwrite d6/d1b/d20/d21/d2a/dc9/fe0 [0,4194304] 0 2024-03-22T06:15:15.494 INFO:tasks.workunit.client.0.smithi006.stdout:1/743: rename d8/d9/d10/d7c/fa6 to d8/d9/d10/ffc 0 2024-03-22T06:15:15.494 INFO:tasks.workunit.client.0.smithi006.stdout:1/744: write d8/d9/d13/d24/d32/d34/f77 [4285223,35167] 0 2024-03-22T06:15:15.496 INFO:tasks.workunit.client.0.smithi006.stdout:3/762: mkdir d0/d8/d88/ddf/d102 0 2024-03-22T06:15:15.496 INFO:tasks.workunit.client.0.smithi006.stdout:3/763: chown d0/d8/d1c/d89 61057621 1 2024-03-22T06:15:15.503 INFO:tasks.workunit.client.0.smithi006.stdout:1/745: rmdir d8/d9/d13/d39/d94/db5 39 2024-03-22T06:15:15.508 INFO:tasks.workunit.client.0.smithi006.stdout:1/746: creat d8/d9/d13/d39/d60/d8f/dea/ffd x:0 0 0 2024-03-22T06:15:15.508 INFO:tasks.workunit.client.0.smithi006.stdout:1/747: dread - d8/d9/d13/d24/d32/ffb zero size 2024-03-22T06:15:15.513 INFO:tasks.workunit.client.0.smithi006.stdout:3/764: dwrite d0/d8/d88/ddf/df0/f101 [0,4194304] 0 2024-03-22T06:15:15.514 INFO:tasks.workunit.client.0.smithi006.stdout:3/765: read - d0/d8/d1c/d49/d79/f99 zero size 2024-03-22T06:15:15.514 INFO:tasks.workunit.client.0.smithi006.stdout:3/766: fsync d0/d8/d9/db1/fbc 0 2024-03-22T06:15:15.525 INFO:tasks.workunit.client.0.smithi006.stdout:3/767: write d0/d4/dbb/df7/d5f/d90/da7/d97/fa5 [668175,5040] 0 2024-03-22T06:15:15.528 INFO:tasks.workunit.client.0.smithi006.stdout:5/863: dwrite d0/db/d76/db4/fc3 [0,4194304] 0 2024-03-22T06:15:15.528 INFO:tasks.workunit.client.0.smithi006.stdout:1/748: dwrite d8/d9/d10/f19 [0,4194304] 0 2024-03-22T06:15:15.536 INFO:tasks.workunit.client.0.smithi006.stdout:5/864: creat d0/db/d50/d82/f107 x:0 0 0 2024-03-22T06:15:15.541 INFO:tasks.workunit.client.0.smithi006.stdout:5/865: creat d0/db/d76/db4/f108 x:0 0 0 2024-03-22T06:15:15.541 INFO:tasks.workunit.client.0.smithi006.stdout:5/866: creat d0/db/d99/f109 x:0 0 0 2024-03-22T06:15:15.542 INFO:tasks.workunit.client.0.smithi006.stdout:3/768: rmdir d0/d4/d31 39 2024-03-22T06:15:15.544 INFO:tasks.workunit.client.0.smithi006.stdout:1/749: dwrite d8/d9/d13/d39/fc9 [0,4194304] 0 2024-03-22T06:15:15.545 INFO:tasks.workunit.client.0.smithi006.stdout:5/867: rmdir d0/db/d50/d96/d106 39 2024-03-22T06:15:15.547 INFO:tasks.workunit.client.0.smithi006.stdout:3/769: truncate d0/d4/dbb/df7/d5f/d90/da7/d97/fb7 1860545 0 2024-03-22T06:15:15.554 INFO:tasks.workunit.client.0.smithi006.stdout:5/868: mknod d0/db/c10a 0 2024-03-22T06:15:15.554 INFO:tasks.workunit.client.0.smithi006.stdout:5/869: chown d0/db/d50/db9/de1/lb8 0 1 2024-03-22T06:15:15.557 INFO:tasks.workunit.client.0.smithi006.stdout:5/870: rmdir d0/db/d16/d43/db1 0 2024-03-22T06:15:15.557 INFO:tasks.workunit.client.0.smithi006.stdout:5/871: fdatasync d0/db/dc/f8f 0 2024-03-22T06:15:15.567 INFO:tasks.workunit.client.0.smithi006.stdout:3/770: dwrite d0/d4/fa9 [0,4194304] 0 2024-03-22T06:15:15.570 INFO:tasks.workunit.client.0.smithi006.stdout:3/771: rename d0/d8/d88/dfa/fb9 to d0/d4/dbb/df7/d68/f103 0 2024-03-22T06:15:15.573 INFO:tasks.workunit.client.0.smithi006.stdout:3/772: link d0/d8/d9/db1/lc7 d0/d4/dbb/df7/l104 0 2024-03-22T06:15:15.575 INFO:tasks.workunit.client.0.smithi006.stdout:3/773: link d0/d4/d31/cc3 d0/d8/db3/c105 0 2024-03-22T06:15:15.577 INFO:tasks.workunit.client.0.smithi006.stdout:5/872: dwrite d0/db/d50/d96/f2d [4194304,4194304] 0 2024-03-22T06:15:15.578 INFO:tasks.workunit.client.0.smithi006.stdout:3/774: dread d0/d4/d3a/f7c [0,4194304] 0 2024-03-22T06:15:15.590 INFO:tasks.workunit.client.0.smithi006.stdout:3/775: read d0/d4/dbb/df7/d5f/d83/d8c/ff4 [2102773,113756] 0 2024-03-22T06:15:15.591 INFO:tasks.workunit.client.0.smithi006.stdout:2/860: sync 2024-03-22T06:15:15.608 INFO:tasks.workunit.client.0.smithi006.stdout:5/873: dwrite d0/db/d16/d1c/d34/d35/d85/f47 [4194304,4194304] 0 2024-03-22T06:15:15.622 INFO:tasks.workunit.client.0.smithi006.stdout:2/861: dwrite d13/d34/d4d/d4e/d75/da5/d82/f95 [0,4194304] 0 2024-03-22T06:15:15.623 INFO:tasks.workunit.client.0.smithi006.stdout:3/776: dwrite d0/f56 [0,4194304] 0 2024-03-22T06:15:15.623 INFO:tasks.workunit.client.0.smithi006.stdout:3/777: dread - d0/d4/d31/ffe zero size 2024-03-22T06:15:15.625 INFO:tasks.workunit.client.0.smithi006.stdout:5/874: dwrite d0/db/d16/d43/ddc/fdf [0,4194304] 0 2024-03-22T06:15:15.630 INFO:tasks.workunit.client.0.smithi006.stdout:2/862: rmdir d13/d20/da6/dba 39 2024-03-22T06:15:15.630 INFO:tasks.workunit.client.0.smithi006.stdout:2/863: stat d13/d20/d35/d104/d129 0 2024-03-22T06:15:15.635 INFO:tasks.workunit.client.0.smithi006.stdout:3/778: symlink d0/d4/dbb/df7/d5f/d90/da7/de7/l106 0 2024-03-22T06:15:15.635 INFO:tasks.workunit.client.0.smithi006.stdout:3/779: chown d0/d4/dbb/df7/d5f/d83/fd2 0 1 2024-03-22T06:15:15.638 INFO:tasks.workunit.client.0.smithi006.stdout:2/864: symlink d13/l146 0 2024-03-22T06:15:15.638 INFO:tasks.workunit.client.0.smithi006.stdout:2/865: chown d13/d53/dc5/fd5 10659047 1 2024-03-22T06:15:15.640 INFO:tasks.workunit.client.0.smithi006.stdout:3/780: rmdir d0/d4/dbb/df7/d5f/d83 39 2024-03-22T06:15:15.641 INFO:tasks.workunit.client.0.smithi006.stdout:2/866: creat d13/d20/da6/dba/f147 x:0 0 0 2024-03-22T06:15:15.642 INFO:tasks.workunit.client.0.smithi006.stdout:5/875: dwrite d0/fd9 [0,4194304] 0 2024-03-22T06:15:15.642 INFO:tasks.workunit.client.0.smithi006.stdout:5/876: read - d0/db/d16/d1c/d34/d35/d85/d5e/dfb/ffd zero size 2024-03-22T06:15:15.643 INFO:tasks.workunit.client.0.smithi006.stdout:3/781: rename d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/c43 to d0/d8/d1c/d49/de5/c107 0 2024-03-22T06:15:15.650 INFO:tasks.workunit.client.0.smithi006.stdout:2/867: dread d13/d34/d4d/d4e/d75/d62/fd6 [0,4194304] 0 2024-03-22T06:15:15.650 INFO:tasks.workunit.client.0.smithi006.stdout:2/868: write d13/d20/d26/fab [452669,103389] 0 2024-03-22T06:15:15.650 INFO:tasks.workunit.client.0.smithi006.stdout:2/869: read d13/d69/ffc [204994,63977] 0 2024-03-22T06:15:15.650 INFO:tasks.workunit.client.0.smithi006.stdout:2/870: dread - d13/d20/d26/dca/def/f118 zero size 2024-03-22T06:15:15.652 INFO:tasks.workunit.client.0.smithi006.stdout:3/782: mkdir d0/d8/d88/dfa/dcb/d108 0 2024-03-22T06:15:15.652 INFO:tasks.workunit.client.0.smithi006.stdout:3/783: stat d0/d4/dbb/df7/d5f/d90/da7/de7/d4c 0 2024-03-22T06:15:15.652 INFO:tasks.workunit.client.0.smithi006.stdout:3/784: write d0/d4/fc9 [277687,101444] 0 2024-03-22T06:15:15.653 INFO:tasks.workunit.client.0.smithi006.stdout:2/871: symlink d13/d34/d4d/d4e/d75/da5/d6c/l148 0 2024-03-22T06:15:15.655 INFO:tasks.workunit.client.0.smithi006.stdout:3/785: creat d0/d8/d88/dfa/dcb/d108/f109 x:0 0 0 2024-03-22T06:15:15.657 INFO:tasks.workunit.client.0.smithi006.stdout:3/786: mkdir d0/d4/dbb/df7/d5f/d90/d10a 0 2024-03-22T06:15:15.660 INFO:tasks.workunit.client.0.smithi006.stdout:3/787: stat d0/d8/d1c/d49/lf1 0 2024-03-22T06:15:15.665 INFO:tasks.workunit.client.0.smithi006.stdout:2/872: dread d13/d34/d4d/d4e/d75/da5/d82/dc3/ff6 [0,4194304] 0 2024-03-22T06:15:15.665 INFO:tasks.workunit.client.0.smithi006.stdout:3/788: creat d0/d4/d31/d98/f10b x:0 0 0 2024-03-22T06:15:15.672 INFO:tasks.workunit.client.0.smithi006.stdout:3/789: rmdir d0/d4/d3a 39 2024-03-22T06:15:15.677 INFO:tasks.workunit.client.0.smithi006.stdout:2/873: dwrite d13/d53/dc5/d10e/f133 [0,4194304] 0 2024-03-22T06:15:15.733 INFO:tasks.workunit.client.0.smithi006.stdout:9/685: sync 2024-03-22T06:15:15.734 INFO:tasks.workunit.client.0.smithi006.stdout:7/776: sync 2024-03-22T06:15:15.734 INFO:tasks.workunit.client.0.smithi006.stdout:9/686: fsync d8/d10/d28/d39/f93 0 2024-03-22T06:15:15.734 INFO:tasks.workunit.client.0.smithi006.stdout:9/687: write d8/d1f/d34/f92 [421753,4834] 0 2024-03-22T06:15:15.735 INFO:tasks.workunit.client.0.smithi006.stdout:7/777: link d5/d1e/d29/d59/d5c/c79 d5/d1e/d63/d85/da1/cfe 0 2024-03-22T06:15:15.737 INFO:tasks.workunit.client.0.smithi006.stdout:9/688: rename d8/d10/l54 to d8/d10/d20/d45/db9/lde 0 2024-03-22T06:15:15.740 INFO:tasks.workunit.client.0.smithi006.stdout:7/778: creat d5/d1e/d29/d1f/d2d/d61/daa/fff x:0 0 0 2024-03-22T06:15:15.740 INFO:tasks.workunit.client.0.smithi006.stdout:7/779: chown d5/d1e/d3f/f45 75275 1 2024-03-22T06:15:15.742 INFO:tasks.workunit.client.0.smithi006.stdout:9/689: creat d8/d10/d28/d39/fdf x:0 0 0 2024-03-22T06:15:15.746 INFO:tasks.workunit.client.0.smithi006.stdout:9/690: truncate d8/d10/d47/f97 895603 0 2024-03-22T06:15:15.747 INFO:tasks.workunit.client.0.smithi006.stdout:9/691: dread - d8/d1f/fbe zero size 2024-03-22T06:15:15.749 INFO:tasks.workunit.client.0.smithi006.stdout:7/780: getdents d5/d1e/d63/d85/d80 0 2024-03-22T06:15:15.749 INFO:tasks.workunit.client.0.smithi006.stdout:7/781: truncate d5/d1e/d29/d1f/de4/ff1 666892 0 2024-03-22T06:15:15.751 INFO:tasks.workunit.client.0.smithi006.stdout:7/782: mkdir d5/d1e/d29/d66/d91/ddc/d100 0 2024-03-22T06:15:15.752 INFO:tasks.workunit.client.0.smithi006.stdout:9/692: dread d8/db3/d50/f8e [0,4194304] 0 2024-03-22T06:15:15.761 INFO:tasks.workunit.client.0.smithi006.stdout:9/693: rename d8/d10/d23/d3e to d8/d10/d20/d45/db9/de0 0 2024-03-22T06:15:15.763 INFO:tasks.workunit.client.0.smithi006.stdout:9/694: truncate d8/d10/f2e 3599385 0 2024-03-22T06:15:15.766 INFO:tasks.workunit.client.0.smithi006.stdout:9/695: link d8/d10/d28/dc2/cdc d8/d10/d82/ce1 0 2024-03-22T06:15:15.768 INFO:tasks.workunit.client.0.smithi006.stdout:3/790: read d0/d4/dbb/df7/d5f/d90/da7/d97/fb7 [402348,75658] 0 2024-03-22T06:15:15.769 INFO:tasks.workunit.client.0.smithi006.stdout:9/696: creat d8/db3/d50/fe2 x:0 0 0 2024-03-22T06:15:15.771 INFO:tasks.workunit.client.0.smithi006.stdout:3/791: link d0/d4/d31/d98/f10b d0/d8/d1c/d49/d79/f10c 0 2024-03-22T06:15:15.772 INFO:tasks.workunit.client.0.smithi006.stdout:9/697: link d8/d10/d11/cbb d8/db3/ce3 0 2024-03-22T06:15:15.774 INFO:tasks.workunit.client.0.smithi006.stdout:3/792: link d0/d8/d1c/c51 d0/d8/d88/dfa/d58/c10d 0 2024-03-22T06:15:15.777 INFO:tasks.workunit.client.0.smithi006.stdout:2/874: write d13/d20/f57 [1008254,86747] 0 2024-03-22T06:15:15.777 INFO:tasks.workunit.client.0.smithi006.stdout:2/875: fsync d13/d20/d35/d65/f108 0 2024-03-22T06:15:15.781 INFO:tasks.workunit.client.0.smithi006.stdout:2/876: symlink d13/d20/d26/d19/d99/l149 0 2024-03-22T06:15:15.782 INFO:tasks.workunit.client.0.smithi006.stdout:2/877: dread - d13/d20/d35/fa2 zero size 2024-03-22T06:15:15.785 INFO:tasks.workunit.client.0.smithi006.stdout:2/878: mknod d13/d34/d4d/d4e/d75/dcc/c14a 0 2024-03-22T06:15:15.785 INFO:tasks.workunit.client.0.smithi006.stdout:2/879: fsync d13/d20/d26/dca/def/f113 0 2024-03-22T06:15:15.785 INFO:tasks.workunit.client.0.smithi006.stdout:2/880: stat d13/d20/d26/dca/l114 0 2024-03-22T06:15:15.788 INFO:tasks.workunit.client.0.smithi006.stdout:2/881: creat d13/d34/d4d/d4e/d75/f14b x:0 0 0 2024-03-22T06:15:15.795 INFO:tasks.workunit.client.0.smithi006.stdout:2/882: dread d13/d34/d4d/d4e/d75/f9a [0,4194304] 0 2024-03-22T06:15:15.798 INFO:tasks.workunit.client.0.smithi006.stdout:2/883: creat d13/d34/d4d/d4e/d75/d62/f14c x:0 0 0 2024-03-22T06:15:15.807 INFO:tasks.workunit.client.0.smithi006.stdout:2/884: dread f12 [0,4194304] 0 2024-03-22T06:15:15.809 INFO:tasks.workunit.client.0.smithi006.stdout:2/885: rmdir d13/d20/d35/d65 39 2024-03-22T06:15:15.809 INFO:tasks.workunit.client.0.smithi006.stdout:2/886: creat d13/d53/d6b/dd8/d11f/f14d x:0 0 0 2024-03-22T06:15:15.812 INFO:tasks.workunit.client.0.smithi006.stdout:2/887: mknod d13/d34/d6e/c14e 0 2024-03-22T06:15:15.812 INFO:tasks.workunit.client.0.smithi006.stdout:2/888: write d13/d34/d6e/fdf [236016,7430] 0 2024-03-22T06:15:15.814 INFO:tasks.workunit.client.0.smithi006.stdout:2/889: mkdir d13/d34/d4d/d4e/d75/d62/d14f 0 2024-03-22T06:15:15.814 INFO:tasks.workunit.client.0.smithi006.stdout:2/890: creat d13/d34/d4d/d4e/de3/f150 x:0 0 0 2024-03-22T06:15:15.818 INFO:tasks.workunit.client.0.smithi006.stdout:2/891: unlink d13/d20/d26/c49 0 2024-03-22T06:15:15.821 INFO:tasks.workunit.client.0.smithi006.stdout:2/892: rename d13/d20/d26/d19/fb3 to d13/d34/d4d/d4e/d75/da5/d6c/f151 0 2024-03-22T06:15:15.821 INFO:tasks.workunit.client.0.smithi006.stdout:2/893: chown d13/d34/d4d/d4e/de3/f150 823264 1 2024-03-22T06:15:15.853 INFO:tasks.workunit.client.0.smithi006.stdout:8/908: sync 2024-03-22T06:15:15.854 INFO:tasks.workunit.client.0.smithi006.stdout:0/623: sync 2024-03-22T06:15:15.855 INFO:tasks.workunit.client.0.smithi006.stdout:8/909: truncate d9/d14/dcc/f11c 829433 0 2024-03-22T06:15:15.857 INFO:tasks.workunit.client.0.smithi006.stdout:0/624: symlink db/d10/d33/d9d/lda 0 2024-03-22T06:15:15.862 INFO:tasks.workunit.client.0.smithi006.stdout:8/910: rename d9/db/d12/d66/f131 to d9/db/d35/dd2/d76/db3/dc3/d89/db8/f149 0 2024-03-22T06:15:15.862 INFO:tasks.workunit.client.0.smithi006.stdout:8/911: chown d9/db/d35/dd2/d76/db3/d132/l72 5973 1 2024-03-22T06:15:15.876 INFO:tasks.workunit.client.0.smithi006.stdout:3/793: write d0/d4/d3a/f7c [79738,82815] 0 2024-03-22T06:15:15.880 INFO:tasks.workunit.client.0.smithi006.stdout:0/625: dwrite db/d3d/d4e/f7d [0,4194304] 0 2024-03-22T06:15:15.880 INFO:tasks.workunit.client.0.smithi006.stdout:8/912: dwrite d9/db/d35/dd2/d4a/f8a [0,4194304] 0 2024-03-22T06:15:15.880 INFO:tasks.workunit.client.0.smithi006.stdout:8/913: write d9/fa [3590764,4243] 0 2024-03-22T06:15:15.957 INFO:tasks.workunit.client.0.smithi006.stdout:3/794: rename d0/d4/d31/cc3 to d0/d8/d88/dfa/dcb/c10e 0 2024-03-22T06:15:15.965 INFO:tasks.workunit.client.0.smithi006.stdout:0/626: creat db/d3d/fdb x:0 0 0 2024-03-22T06:15:15.967 INFO:tasks.workunit.client.0.smithi006.stdout:3/795: write d0/d8/d1c/d49/f5e [6470449,44924] 0 2024-03-22T06:15:15.968 INFO:tasks.workunit.client.0.smithi006.stdout:0/627: write db/dc/d38/d50/fc3 [156675,28499] 0 2024-03-22T06:15:15.974 INFO:tasks.workunit.client.0.smithi006.stdout:8/914: getdents d9/db/d12/d66/d6c/d92 0 2024-03-22T06:15:15.974 INFO:tasks.workunit.client.0.smithi006.stdout:8/915: fdatasync d9/fdc 0 2024-03-22T06:15:15.976 INFO:tasks.workunit.client.0.smithi006.stdout:8/916: read d9/db/d12/f26 [799879,100001] 0 2024-03-22T06:15:15.978 INFO:tasks.workunit.client.0.smithi006.stdout:8/917: dread d9/db/d35/dd2/f41 [0,4194304] 0 2024-03-22T06:15:15.978 INFO:tasks.workunit.client.0.smithi006.stdout:8/918: truncate d9/d14/dcc/f133 340821 0 2024-03-22T06:15:15.979 INFO:tasks.workunit.client.0.smithi006.stdout:0/628: getdents db/d10/d33 0 2024-03-22T06:15:15.983 INFO:tasks.workunit.client.0.smithi006.stdout:8/919: symlink d9/db/d35/dd2/d76/db3/dc3/d89/l14a 0 2024-03-22T06:15:15.985 INFO:tasks.workunit.client.0.smithi006.stdout:3/796: dwrite d0/f3 [0,4194304] 0 2024-03-22T06:15:15.985 INFO:tasks.workunit.client.0.smithi006.stdout:3/797: chown d0/d8/d1c/d49 205205844 1 2024-03-22T06:15:15.985 INFO:tasks.workunit.client.0.smithi006.stdout:3/798: write d0/d8/d1c/f4f [9238407,6126] 0 2024-03-22T06:15:15.986 INFO:tasks.workunit.client.0.smithi006.stdout:3/799: read d0/d4/dbb/df7/d5f/d90/da7/de7/f25 [657168,113601] 0 2024-03-22T06:15:15.986 INFO:tasks.workunit.client.0.smithi006.stdout:0/629: mknod db/d10/d33/db4/cdc 0 2024-03-22T06:15:15.987 INFO:tasks.workunit.client.0.smithi006.stdout:8/920: symlink d9/db/d35/dd2/d76/db3/dc3/d89/db8/d130/l14b 0 2024-03-22T06:15:15.990 INFO:tasks.workunit.client.0.smithi006.stdout:3/800: truncate d0/d4/d31/fcc 893038 0 2024-03-22T06:15:15.990 INFO:tasks.workunit.client.0.smithi006.stdout:3/801: truncate d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/fa2 222389 0 2024-03-22T06:15:15.990 INFO:tasks.workunit.client.0.smithi006.stdout:3/802: write d0/d4/d31/d98/dab/ff8 [361813,107203] 0 2024-03-22T06:15:15.992 INFO:tasks.workunit.client.0.smithi006.stdout:0/630: mkdir db/d87/ddd 0 2024-03-22T06:15:15.992 INFO:tasks.workunit.client.0.smithi006.stdout:0/631: dread - db/dc/d38/fc0 zero size 2024-03-22T06:15:15.992 INFO:tasks.workunit.client.0.smithi006.stdout:0/632: truncate db/d10/d33/d9d/dcd/fd4 475187 0 2024-03-22T06:15:15.992 INFO:tasks.workunit.client.0.smithi006.stdout:0/633: stat db/dc/d38/fc1 0 2024-03-22T06:15:15.994 INFO:tasks.workunit.client.0.smithi006.stdout:8/921: symlink d9/db/d35/dd2/d76/l14c 0 2024-03-22T06:15:15.995 INFO:tasks.workunit.client.0.smithi006.stdout:3/803: symlink d0/d4/dbb/df7/d5f/d90/da7/d97/l10f 0 2024-03-22T06:15:15.999 INFO:tasks.workunit.client.0.smithi006.stdout:8/922: unlink d9/db/d12/d66/d6c/d80/df0/ffb 0 2024-03-22T06:15:16.002 INFO:tasks.workunit.client.0.smithi006.stdout:8/923: write d9/db/d35/da9/df6/f105 [534276,65587] 0 2024-03-22T06:15:16.002 INFO:tasks.workunit.client.0.smithi006.stdout:3/804: creat d0/d8/d88/dfa/de2/f110 x:0 0 0 2024-03-22T06:15:16.002 INFO:tasks.workunit.client.0.smithi006.stdout:3/805: write d0/d4/d31/ffe [998536,85911] 0 2024-03-22T06:15:16.002 INFO:tasks.workunit.client.0.smithi006.stdout:3/806: chown d0/d4/dbb/df7/d5f/d90 3 1 2024-03-22T06:15:16.009 INFO:tasks.workunit.client.0.smithi006.stdout:8/924: read d9/db/d35/da9/faa [2007887,118192] 0 2024-03-22T06:15:16.014 INFO:tasks.workunit.client.0.smithi006.stdout:3/807: dread d0/d4/dbb/df7/d5f/da3/fc8 [0,4194304] 0 2024-03-22T06:15:16.021 INFO:tasks.workunit.client.0.smithi006.stdout:3/808: rename d0/cd to d0/d8/d88/dfa/d58/c111 0 2024-03-22T06:15:16.023 INFO:tasks.workunit.client.0.smithi006.stdout:8/925: getdents d9/db/d35/dd2/dc4 0 2024-03-22T06:15:16.024 INFO:tasks.workunit.client.0.smithi006.stdout:3/809: mkdir d0/d8/d88/ddf/df0/d112 0 2024-03-22T06:15:16.024 INFO:tasks.workunit.client.0.smithi006.stdout:3/810: dread - d0/d8/d88/ffd zero size 2024-03-22T06:15:16.026 INFO:tasks.workunit.client.0.smithi006.stdout:8/926: mknod d9/db/d12/d1e/d11e/c14d 0 2024-03-22T06:15:16.027 INFO:tasks.workunit.client.0.smithi006.stdout:3/811: symlink d0/d8/d88/ddf/d102/l113 0 2024-03-22T06:15:16.027 INFO:tasks.workunit.client.0.smithi006.stdout:3/812: chown d0/d8/d88/ddf 18827 1 2024-03-22T06:15:16.029 INFO:tasks.workunit.client.0.smithi006.stdout:8/927: mknod d9/db/d12/d66/d10c/d122/c14e 0 2024-03-22T06:15:16.031 INFO:tasks.workunit.client.0.smithi006.stdout:3/813: rmdir d0/d4/dbb/df7/d5f/d83/d8c 39 2024-03-22T06:15:16.032 INFO:tasks.workunit.client.0.smithi006.stdout:8/928: symlink d9/db/d35/dd2/d76/db3/dc3/d12d/l14f 0 2024-03-22T06:15:16.035 INFO:tasks.workunit.client.0.smithi006.stdout:3/814: getdents d0/d8/d1c/d49/de5 0 2024-03-22T06:15:16.040 INFO:tasks.workunit.client.0.smithi006.stdout:3/815: creat d0/d4/d31/f114 x:0 0 0 2024-03-22T06:15:16.043 INFO:tasks.workunit.client.0.smithi006.stdout:3/816: unlink d0/d4/dbb/df7/l8b 0 2024-03-22T06:15:16.043 INFO:tasks.workunit.client.0.smithi006.stdout:3/817: dread - d0/d4/dbb/df7/d68/f75 zero size 2024-03-22T06:15:16.046 INFO:tasks.workunit.client.0.smithi006.stdout:3/818: creat d0/d4/d3a/f115 x:0 0 0 2024-03-22T06:15:16.046 INFO:tasks.workunit.client.0.smithi006.stdout:3/819: stat d0/d4 0 2024-03-22T06:15:16.046 INFO:tasks.workunit.client.0.smithi006.stdout:3/820: chown d0/d8/d1c/d49/d79/cf3 54 1 2024-03-22T06:15:16.046 INFO:tasks.workunit.client.0.smithi006.stdout:3/821: stat d0/d4/d3a/f7c 0 2024-03-22T06:15:16.049 INFO:tasks.workunit.client.0.smithi006.stdout:3/822: rename d0/d8/d88/dfa/dcb/c10e to d0/d4/dbb/df7/d5f/d90/d10a/c116 0 2024-03-22T06:15:16.049 INFO:tasks.workunit.client.0.smithi006.stdout:3/823: fdatasync d0/d4/d31/d98/dab/fd5 0 2024-03-22T06:15:16.049 INFO:tasks.workunit.client.0.smithi006.stdout:3/824: stat d0/d8/d1c/d72/c77 0 2024-03-22T06:15:16.051 INFO:tasks.workunit.client.0.smithi006.stdout:3/825: creat d0/d4/d53/f117 x:0 0 0 2024-03-22T06:15:16.052 INFO:tasks.workunit.client.0.smithi006.stdout:3/826: truncate d0/d4/dbb/df7/d68/f103 525795 0 2024-03-22T06:15:16.053 INFO:tasks.workunit.client.0.smithi006.stdout:3/827: write d0/d8/d1c/f4f [9151270,20406] 0 2024-03-22T06:15:16.077 INFO:tasks.workunit.client.0.smithi006.stdout:3/828: dwrite d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/fa2 [0,4194304] 0 2024-03-22T06:15:16.077 INFO:tasks.workunit.client.0.smithi006.stdout:3/829: creat d0/d4/d3a/f118 x:0 0 0 2024-03-22T06:15:16.083 INFO:tasks.workunit.client.0.smithi006.stdout:3/830: mknod d0/d8/d9/c119 0 2024-03-22T06:15:16.088 INFO:tasks.workunit.client.0.smithi006.stdout:3/831: unlink d0/d8/d88/dfa/d58/l66 0 2024-03-22T06:15:16.088 INFO:tasks.workunit.client.0.smithi006.stdout:3/832: write d0/d4/d53/f117 [865620,70577] 0 2024-03-22T06:15:16.090 INFO:tasks.workunit.client.0.smithi006.stdout:9/698: truncate d8/d10/d47/f97 567136 0 2024-03-22T06:15:16.092 INFO:tasks.workunit.client.0.smithi006.stdout:3/833: link d0/d4/d31/d98/f10b d0/d4/dbb/df7/d5f/d90/d10a/f11a 0 2024-03-22T06:15:16.095 INFO:tasks.workunit.client.0.smithi006.stdout:9/699: rename d8/d10/c31 to d8/d10/d20/d45/db9/ce4 0 2024-03-22T06:15:16.097 INFO:tasks.workunit.client.0.smithi006.stdout:3/834: mknod d0/d4/dbb/df7/c11b 0 2024-03-22T06:15:16.099 INFO:tasks.workunit.client.0.smithi006.stdout:9/700: creat d8/d1f/fe5 x:0 0 0 2024-03-22T06:15:16.099 INFO:tasks.workunit.client.0.smithi006.stdout:9/701: creat d8/d10/d47/dbc/fe6 x:0 0 0 2024-03-22T06:15:16.100 INFO:tasks.workunit.client.0.smithi006.stdout:3/835: creat d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/f11c x:0 0 0 2024-03-22T06:15:16.101 INFO:tasks.workunit.client.0.smithi006.stdout:3/836: creat d0/d4/d31/f11d x:0 0 0 2024-03-22T06:15:16.103 INFO:tasks.workunit.client.0.smithi006.stdout:0/634: dwrite db/d10/d33/f57 [0,4194304] 0 2024-03-22T06:15:16.105 INFO:tasks.workunit.client.0.smithi006.stdout:9/702: symlink d8/d10/d28/dc2/le7 0 2024-03-22T06:15:16.111 INFO:tasks.workunit.client.0.smithi006.stdout:0/635: write db/d10/d2a/f51 [2515828,99404] 0 2024-03-22T06:15:16.114 INFO:tasks.workunit.client.0.smithi006.stdout:9/703: symlink d8/d10/d20/d45/db9/de0/le8 0 2024-03-22T06:15:16.117 INFO:tasks.workunit.client.0.smithi006.stdout:9/704: rename d8/db3/d50/d38/c3b to d8/d10/d20/d45/d49/ce9 0 2024-03-22T06:15:16.117 INFO:tasks.workunit.client.0.smithi006.stdout:9/705: write d8/d10/d20/d45/d49/f8c [748061,198] 0 2024-03-22T06:15:16.118 INFO:tasks.workunit.client.0.smithi006.stdout:9/706: write d8/d10/d28/d39/fb7 [2415884,22279] 0 2024-03-22T06:15:16.122 INFO:tasks.workunit.client.0.smithi006.stdout:0/636: dread db/d10/f14 [0,4194304] 0 2024-03-22T06:15:16.122 INFO:tasks.workunit.client.0.smithi006.stdout:3/837: dwrite d0/d4/dbb/df7/d5f/da3/f47 [0,4194304] 0 2024-03-22T06:15:16.122 INFO:tasks.workunit.client.0.smithi006.stdout:3/838: stat d0/d4/dbb/df7/d5f/d90/da7/de7/f25 0 2024-03-22T06:15:16.126 INFO:tasks.workunit.client.0.smithi006.stdout:9/707: dread f4 [0,4194304] 0 2024-03-22T06:15:16.126 INFO:tasks.workunit.client.0.smithi006.stdout:9/708: fdatasync d8/fa 0 2024-03-22T06:15:16.127 INFO:tasks.workunit.client.0.smithi006.stdout:0/637: mknod db/d10/d35/cde 0 2024-03-22T06:15:16.127 INFO:tasks.workunit.client.0.smithi006.stdout:0/638: chown db/dc/d38/c58 30 1 2024-03-22T06:15:16.127 INFO:tasks.workunit.client.0.smithi006.stdout:0/639: chown db/d10/d35/d6c/fbb 58549898 1 2024-03-22T06:15:16.131 INFO:tasks.workunit.client.0.smithi006.stdout:3/839: dread d0/d4/d31/f3b [0,4194304] 0 2024-03-22T06:15:16.131 INFO:tasks.workunit.client.0.smithi006.stdout:3/840: stat d0/d4/d53/de0 0 2024-03-22T06:15:16.131 INFO:tasks.workunit.client.0.smithi006.stdout:9/709: symlink d8/db3/lea 0 2024-03-22T06:15:16.131 INFO:tasks.workunit.client.0.smithi006.stdout:9/710: readlink d8/db3/d50/d38/lc1 0 2024-03-22T06:15:16.132 INFO:tasks.workunit.client.0.smithi006.stdout:9/711: fsync d8/d10/d28/fa7 0 2024-03-22T06:15:16.132 INFO:tasks.workunit.client.0.smithi006.stdout:9/712: write d8/d10/d11/f16 [2418387,124260] 0 2024-03-22T06:15:16.150 INFO:tasks.workunit.client.0.smithi006.stdout:3/841: creat d0/d4/dbb/df7/d5f/d83/f11e x:0 0 0 2024-03-22T06:15:16.151 INFO:tasks.workunit.client.0.smithi006.stdout:8/929: dwrite d9/db/d35/dd2/d76/f145 [0,4194304] 0 2024-03-22T06:15:16.154 INFO:tasks.workunit.client.0.smithi006.stdout:9/713: unlink d8/d10/d20/d45/db9/lde 0 2024-03-22T06:15:16.155 INFO:tasks.workunit.client.0.smithi006.stdout:3/842: rmdir d0/d8/d9/db1 39 2024-03-22T06:15:16.160 INFO:tasks.workunit.client.0.smithi006.stdout:8/930: creat d9/db/d12/f150 x:0 0 0 2024-03-22T06:15:16.161 INFO:tasks.workunit.client.0.smithi006.stdout:9/714: symlink d8/d10/d82/leb 0 2024-03-22T06:15:16.163 INFO:tasks.workunit.client.0.smithi006.stdout:3/843: creat d0/d8/d6a/f11f x:0 0 0 2024-03-22T06:15:16.164 INFO:tasks.workunit.client.0.smithi006.stdout:8/931: creat d9/db/d35/dd2/d76/db3/dc3/f151 x:0 0 0 2024-03-22T06:15:16.167 INFO:tasks.workunit.client.0.smithi006.stdout:9/715: unlink d8/d1f/d34/f92 0 2024-03-22T06:15:16.167 INFO:tasks.workunit.client.0.smithi006.stdout:9/716: chown d8/d10/d28/fdb 104524 1 2024-03-22T06:15:16.170 INFO:tasks.workunit.client.0.smithi006.stdout:9/717: creat d8/d10/d20/d45/d49/fec x:0 0 0 2024-03-22T06:15:16.187 INFO:tasks.workunit.client.0.smithi006.stdout:3/844: dwrite d0/d4/d53/de0/fed [0,4194304] 0 2024-03-22T06:15:16.187 INFO:tasks.workunit.client.0.smithi006.stdout:9/718: dwrite d8/d10/d28/dba/fc4 [0,4194304] 0 2024-03-22T06:15:16.187 INFO:tasks.workunit.client.0.smithi006.stdout:9/719: creat d8/d10/da6/fed x:0 0 0 2024-03-22T06:15:16.187 INFO:tasks.workunit.client.0.smithi006.stdout:9/720: stat d8/d10/d28/fb6 0 2024-03-22T06:15:16.187 INFO:tasks.workunit.client.0.smithi006.stdout:9/721: chown f1 425 1 2024-03-22T06:15:16.194 INFO:tasks.workunit.client.0.smithi006.stdout:4/842: sync 2024-03-22T06:15:16.194 INFO:tasks.workunit.client.0.smithi006.stdout:6/752: sync 2024-03-22T06:15:16.194 INFO:tasks.workunit.client.0.smithi006.stdout:1/750: sync 2024-03-22T06:15:16.195 INFO:tasks.workunit.client.0.smithi006.stdout:2/894: sync 2024-03-22T06:15:16.195 INFO:tasks.workunit.client.0.smithi006.stdout:7/783: sync 2024-03-22T06:15:16.195 INFO:tasks.workunit.client.0.smithi006.stdout:5/877: sync 2024-03-22T06:15:16.195 INFO:tasks.workunit.client.0.smithi006.stdout:2/895: write d13/d69/ffc [791009,68806] 0 2024-03-22T06:15:16.195 INFO:tasks.workunit.client.0.smithi006.stdout:2/896: readlink d13/d34/d4d/d4e/d75/da5/l55 0 2024-03-22T06:15:16.195 INFO:tasks.workunit.client.0.smithi006.stdout:2/897: fsync d13/d20/d35/d65/fbb 0 2024-03-22T06:15:16.195 INFO:tasks.workunit.client.0.smithi006.stdout:2/898: write fe [5423277,69684] 0 2024-03-22T06:15:16.195 INFO:tasks.workunit.client.0.smithi006.stdout:9/722: dread d8/d1f/f68 [0,4194304] 0 2024-03-22T06:15:16.198 INFO:tasks.workunit.client.0.smithi006.stdout:6/753: dread - d6/d1b/dae/fb6 zero size 2024-03-22T06:15:16.200 INFO:tasks.workunit.client.0.smithi006.stdout:1/751: creat d8/d9/d3a/d18/ffe x:0 0 0 2024-03-22T06:15:16.202 INFO:tasks.workunit.client.0.smithi006.stdout:7/784: creat d5/d1e/d63/d85/dca/f101 x:0 0 0 2024-03-22T06:15:16.204 INFO:tasks.workunit.client.0.smithi006.stdout:4/843: dread f3 [4194304,4194304] 0 2024-03-22T06:15:16.206 INFO:tasks.workunit.client.0.smithi006.stdout:5/878: mkdir d0/db/d76/db4/dfe/d10b 0 2024-03-22T06:15:16.209 INFO:tasks.workunit.client.0.smithi006.stdout:2/899: rename d13/d34/f58 to d13/d53/f152 0 2024-03-22T06:15:16.212 INFO:tasks.workunit.client.0.smithi006.stdout:9/723: unlink d8/d10/d28/d39/fb7 0 2024-03-22T06:15:16.213 INFO:tasks.workunit.client.0.smithi006.stdout:9/724: fsync d8/d10/d28/f57 0 2024-03-22T06:15:16.213 INFO:tasks.workunit.client.0.smithi006.stdout:9/725: chown d8/d10/d28/dc2/cdc 15620745 1 2024-03-22T06:15:16.213 INFO:tasks.workunit.client.0.smithi006.stdout:9/726: fsync d8/d10/d23/f40 0 2024-03-22T06:15:16.214 INFO:tasks.workunit.client.0.smithi006.stdout:5/879: dread d0/f1a [0,4194304] 0 2024-03-22T06:15:16.214 INFO:tasks.workunit.client.0.smithi006.stdout:5/880: chown d0/db/d16/d1c/d34/d35/d85/dda/d65/fb2 35683226 1 2024-03-22T06:15:16.218 INFO:tasks.workunit.client.0.smithi006.stdout:4/844: creat d6/d1b/d20/d21/d11d/f123 x:0 0 0 2024-03-22T06:15:16.219 INFO:tasks.workunit.client.0.smithi006.stdout:7/785: dwrite d5/d1e/d29/d59/f9d [4194304,4194304] 0 2024-03-22T06:15:16.219 INFO:tasks.workunit.client.0.smithi006.stdout:7/786: chown d5/d1e/d29/d1f/l57 6364 1 2024-03-22T06:15:16.219 INFO:tasks.workunit.client.0.smithi006.stdout:7/787: fdatasync d5/d1e/d3f/f45 0 2024-03-22T06:15:16.219 INFO:tasks.workunit.client.0.smithi006.stdout:7/788: truncate d5/d1e/d29/d1f/d2d/d41/fd2 1019040 0 2024-03-22T06:15:16.219 INFO:tasks.workunit.client.0.smithi006.stdout:7/789: getdents d5/d1e/d29/d1f/d2d/df7/ddd 0 2024-03-22T06:15:16.220 INFO:tasks.workunit.client.0.smithi006.stdout:2/900: creat d13/d20/d35/d104/f153 x:0 0 0 2024-03-22T06:15:16.220 INFO:tasks.workunit.client.0.smithi006.stdout:2/901: creat d13/d20/d26/dca/def/f154 x:0 0 0 2024-03-22T06:15:16.220 INFO:tasks.workunit.client.0.smithi006.stdout:2/902: truncate d13/d34/d4d/d4e/d75/da5/d82/fae 1166425 0 2024-03-22T06:15:16.222 INFO:tasks.workunit.client.0.smithi006.stdout:9/727: unlink d8/d10/d20/d45/db9/de0/f9a 0 2024-03-22T06:15:16.225 INFO:tasks.workunit.client.0.smithi006.stdout:5/881: symlink d0/db/d16/d1c/d34/d35/d85/dda/d65/d77/l10c 0 2024-03-22T06:15:16.225 INFO:tasks.workunit.client.0.smithi006.stdout:5/882: readlink d0/db/d16/d70/lfa 0 2024-03-22T06:15:16.226 INFO:tasks.workunit.client.0.smithi006.stdout:1/752: symlink d8/d9/d13/d24/lff 0 2024-03-22T06:15:16.226 INFO:tasks.workunit.client.0.smithi006.stdout:1/753: fsync d8/d9/d13/d39/d60/fec 0 2024-03-22T06:15:16.227 INFO:tasks.workunit.client.0.smithi006.stdout:4/845: mknod d6/d1b/d20/d21/d2a/d5c/c124 0 2024-03-22T06:15:16.234 INFO:tasks.workunit.client.0.smithi006.stdout:2/903: mknod d13/d34/d4d/d4e/d75/d62/c155 0 2024-03-22T06:15:16.238 INFO:tasks.workunit.client.0.smithi006.stdout:5/883: creat d0/db/d76/db4/dfe/f10d x:0 0 0 2024-03-22T06:15:16.238 INFO:tasks.workunit.client.0.smithi006.stdout:5/884: dread - d0/db/d50/d58/fc7 zero size 2024-03-22T06:15:16.238 INFO:tasks.workunit.client.0.smithi006.stdout:5/885: truncate d0/db/d16/d1c/d34/d35/d85/dda/d65/fb2 961018 0 2024-03-22T06:15:16.239 INFO:tasks.workunit.client.0.smithi006.stdout:1/754: truncate d8/d9/d13/f6d 1975960 0 2024-03-22T06:15:16.245 INFO:tasks.workunit.client.0.smithi006.stdout:2/904: creat d13/d34/d4d/d4e/d75/d62/d14f/f156 x:0 0 0 2024-03-22T06:15:16.245 INFO:tasks.workunit.client.0.smithi006.stdout:2/905: creat d13/d53/dc5/f157 x:0 0 0 2024-03-22T06:15:16.245 INFO:tasks.workunit.client.0.smithi006.stdout:2/906: write d13/d69/f128 [1015322,84701] 0 2024-03-22T06:15:16.249 INFO:tasks.workunit.client.0.smithi006.stdout:2/907: rmdir d13/d34/d4d/d4e/d75/da5/d82 39 2024-03-22T06:15:16.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:16 smithi117 ceph-mon[36514]: pgmap v21: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 80 MiB/s rd, 102 MiB/s wr, 805 op/s 2024-03-22T06:15:16.276 INFO:tasks.workunit.client.0.smithi006.stdout:9/728: fsync d8/d10/da6/fed 0 2024-03-22T06:15:16.278 INFO:tasks.workunit.client.0.smithi006.stdout:9/729: creat d8/d10/d11/fee x:0 0 0 2024-03-22T06:15:16.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:16 smithi006 ceph-mon[29301]: pgmap v21: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 80 MiB/s rd, 102 MiB/s wr, 805 op/s 2024-03-22T06:15:16.282 INFO:tasks.workunit.client.0.smithi006.stdout:9/730: mknod d8/d10/d20/d45/db9/de0/cef 0 2024-03-22T06:15:16.294 INFO:tasks.workunit.client.0.smithi006.stdout:9/731: dread d8/d10/d23/f40 [0,4194304] 0 2024-03-22T06:15:16.294 INFO:tasks.workunit.client.0.smithi006.stdout:9/732: dread - d8/d10/d47/dbc/fcc zero size 2024-03-22T06:15:16.294 INFO:tasks.workunit.client.0.smithi006.stdout:9/733: chown d8/d10/d20/d45/d49/fec 1 1 2024-03-22T06:15:16.296 INFO:tasks.workunit.client.0.smithi006.stdout:9/734: write d8/d10/d28/f44 [4868855,116703] 0 2024-03-22T06:15:16.296 INFO:tasks.workunit.client.0.smithi006.stdout:9/735: chown d8/d10/d20/d45/d49/l78 1753117 1 2024-03-22T06:15:16.296 INFO:tasks.workunit.client.0.smithi006.stdout:9/736: write d8/db3/d50/fa2 [860587,21828] 0 2024-03-22T06:15:16.296 INFO:tasks.workunit.client.0.smithi006.stdout:9/737: chown d8/d10/d11/dc0/ld4 29 1 2024-03-22T06:15:16.299 INFO:tasks.workunit.client.0.smithi006.stdout:1/755: dwrite d8/d9/d3a/d18/fc3 [0,4194304] 0 2024-03-22T06:15:16.299 INFO:tasks.workunit.client.0.smithi006.stdout:4/846: dwrite d6/d1b/d20/d21/d2a/d43/fe4 [0,4194304] 0 2024-03-22T06:15:16.299 INFO:tasks.workunit.client.0.smithi006.stdout:4/847: readlink d6/d1b/d20/d6b/d60/ldc 0 2024-03-22T06:15:16.300 INFO:tasks.workunit.client.0.smithi006.stdout:5/886: dwrite d0/db/d16/d1c/d34/d4d/f6b [0,4194304] 0 2024-03-22T06:15:16.300 INFO:tasks.workunit.client.0.smithi006.stdout:2/908: dwrite d13/d34/d4d/d4e/de3/f127 [0,4194304] 0 2024-03-22T06:15:16.300 INFO:tasks.workunit.client.0.smithi006.stdout:9/738: creat d8/ff0 x:0 0 0 2024-03-22T06:15:16.318 INFO:tasks.workunit.client.0.smithi006.stdout:6/754: dwrite d6/d1b/f3c [0,4194304] 0 2024-03-22T06:15:16.322 INFO:tasks.workunit.client.0.smithi006.stdout:2/909: dread d13/d20/d35/d65/f108 [0,4194304] 0 2024-03-22T06:15:16.322 INFO:tasks.workunit.client.0.smithi006.stdout:2/910: creat d13/d53/dc5/d10e/f158 x:0 0 0 2024-03-22T06:15:16.322 INFO:tasks.workunit.client.0.smithi006.stdout:9/739: dwrite d8/d10/d20/f2c [0,4194304] 0 2024-03-22T06:15:16.323 INFO:tasks.workunit.client.0.smithi006.stdout:4/848: dwrite d6/d1b/d20/f22 [4194304,4194304] 0 2024-03-22T06:15:16.324 INFO:tasks.workunit.client.0.smithi006.stdout:1/756: rmdir d8/d9/d3a/d40/d44/d80 39 2024-03-22T06:15:16.324 INFO:tasks.workunit.client.0.smithi006.stdout:1/757: chown d8/d71/cde 117040566 1 2024-03-22T06:15:16.324 INFO:tasks.workunit.client.0.smithi006.stdout:1/758: stat d8/d9/d3a/d40/d44/d80 0 2024-03-22T06:15:16.330 INFO:tasks.workunit.client.0.smithi006.stdout:6/755: mkdir d6/d1d/d99/de9 0 2024-03-22T06:15:16.332 INFO:tasks.workunit.client.0.smithi006.stdout:1/759: dread d8/d33/f9f [0,4194304] 0 2024-03-22T06:15:16.332 INFO:tasks.workunit.client.0.smithi006.stdout:9/740: write d8/d10/d20/d45/db9/de0/f5a [1431065,39658] 0 2024-03-22T06:15:16.333 INFO:tasks.workunit.client.0.smithi006.stdout:9/741: stat d8/d10/d28/f57 0 2024-03-22T06:15:16.333 INFO:tasks.workunit.client.0.smithi006.stdout:9/742: creat d8/db3/d50/ff1 x:0 0 0 2024-03-22T06:15:16.333 INFO:tasks.workunit.client.0.smithi006.stdout:6/756: dread d6/d24/fa8 [0,4194304] 0 2024-03-22T06:15:16.335 INFO:tasks.workunit.client.0.smithi006.stdout:5/887: write d0/db/d16/f46 [1375647,50325] 0 2024-03-22T06:15:16.335 INFO:tasks.workunit.client.0.smithi006.stdout:5/888: chown d0/db/d16/d1c/d34/d35/d85/d5e/d66/f7b 11440 1 2024-03-22T06:15:16.335 INFO:tasks.workunit.client.0.smithi006.stdout:5/889: write d0/db/d50/db9/fbe [99542,41555] 0 2024-03-22T06:15:16.335 INFO:tasks.workunit.client.0.smithi006.stdout:5/890: stat d0/db/d76/db4/f108 0 2024-03-22T06:15:16.335 INFO:tasks.workunit.client.0.smithi006.stdout:7/790: dwrite d5/d1e/d29/d1f/de4/ff1 [0,4194304] 0 2024-03-22T06:15:16.335 INFO:tasks.workunit.client.0.smithi006.stdout:5/891: truncate d0/db/d50/d82/f107 1028890 0 2024-03-22T06:15:16.335 INFO:tasks.workunit.client.0.smithi006.stdout:7/791: chown d5/d1e/d29/d59/f7a 358281 1 2024-03-22T06:15:16.337 INFO:tasks.workunit.client.0.smithi006.stdout:2/911: mkdir d13/d20/d159 0 2024-03-22T06:15:16.337 INFO:tasks.workunit.client.0.smithi006.stdout:2/912: write d13/d20/f4b [2591315,121810] 0 2024-03-22T06:15:16.342 INFO:tasks.workunit.client.0.smithi006.stdout:9/743: dread d8/fb [0,4194304] 0 2024-03-22T06:15:16.343 INFO:tasks.workunit.client.0.smithi006.stdout:4/849: link d6/d1b/d20/d6b/d88/ccf d6/d1b/d46/c125 0 2024-03-22T06:15:16.345 INFO:tasks.workunit.client.1.smithi117.stdout:4/62: rename d0/f10 to d0/d4/f17 0 2024-03-22T06:15:16.346 INFO:tasks.workunit.client.0.smithi006.stdout:1/760: dread - d8/d9/d13/d24/fc8 zero size 2024-03-22T06:15:16.350 INFO:tasks.workunit.client.0.smithi006.stdout:6/757: mknod d6/d1b/d2c/d4a/cea 0 2024-03-22T06:15:16.351 INFO:tasks.workunit.client.1.smithi117.stdout:8/56: mknod d1/d2/d11/c14 0 2024-03-22T06:15:16.351 INFO:tasks.workunit.client.1.smithi117.stdout:8/57: dread - d1/d2/d9/f13 zero size 2024-03-22T06:15:16.356 INFO:tasks.workunit.client.0.smithi006.stdout:2/913: symlink d13/d20/d35/d104/l15a 0 2024-03-22T06:15:16.356 INFO:tasks.workunit.client.0.smithi006.stdout:2/914: read d13/d34/d4d/d4e/d75/d62/f9c [366531,94929] 0 2024-03-22T06:15:16.356 INFO:tasks.workunit.client.0.smithi006.stdout:2/915: fdatasync d13/d69/d10f/deb/f141 0 2024-03-22T06:15:16.357 INFO:tasks.workunit.client.0.smithi006.stdout:9/744: dwrite d8/d10/d28/f57 [4194304,4194304] 0 2024-03-22T06:15:16.360 INFO:tasks.workunit.client.1.smithi117.stdout:4/63: rename d0/l14 to d0/l18 0 2024-03-22T06:15:16.360 INFO:tasks.workunit.client.1.smithi117.stdout:4/64: write d0/d4/fa [958679,10475] 0 2024-03-22T06:15:16.363 INFO:tasks.workunit.client.1.smithi117.stdout:9/56: unlink d4/d7/ff 0 2024-03-22T06:15:16.365 INFO:tasks.workunit.client.1.smithi117.stdout:8/58: mknod d1/d2/d9/c15 0 2024-03-22T06:15:16.365 INFO:tasks.workunit.client.1.smithi117.stdout:8/59: readlink d1/d2/l4 0 2024-03-22T06:15:16.365 INFO:tasks.workunit.client.1.smithi117.stdout:8/60: readlink d1/d2/l4 0 2024-03-22T06:15:16.367 INFO:tasks.workunit.client.1.smithi117.stdout:4/65: mkdir d0/d19 0 2024-03-22T06:15:16.367 INFO:tasks.workunit.client.1.smithi117.stdout:4/66: write d0/d4/f15 [72367,107191] 0 2024-03-22T06:15:16.368 INFO:tasks.workunit.client.0.smithi006.stdout:6/758: dwrite d6/d1d/d99/fb1 [0,4194304] 0 2024-03-22T06:15:16.368 INFO:tasks.workunit.client.1.smithi117.stdout:9/57: mkdir d4/d8/d14 0 2024-03-22T06:15:16.370 INFO:tasks.workunit.client.1.smithi117.stdout:8/61: mkdir d1/d2/d16 0 2024-03-22T06:15:16.373 INFO:tasks.workunit.client.1.smithi117.stdout:4/67: truncate d0/d4/fe 2786601 0 2024-03-22T06:15:16.378 INFO:tasks.workunit.client.1.smithi117.stdout:8/62: symlink d1/d2/d9/l17 0 2024-03-22T06:15:16.378 INFO:tasks.workunit.client.1.smithi117.stdout:8/63: creat d1/d2/d9/f18 x:0 0 0 2024-03-22T06:15:16.381 INFO:tasks.workunit.client.1.smithi117.stdout:4/68: stat d0/l18 0 2024-03-22T06:15:16.385 INFO:tasks.workunit.client.1.smithi117.stdout:8/64: mkdir d1/d19 0 2024-03-22T06:15:16.387 INFO:tasks.workunit.client.1.smithi117.stdout:8/65: creat d1/f1a x:0 0 0 2024-03-22T06:15:16.387 INFO:tasks.workunit.client.1.smithi117.stdout:8/66: truncate d1/f1a 669882 0 2024-03-22T06:15:16.388 INFO:tasks.workunit.client.0.smithi006.stdout:4/850: symlink d6/d1b/d20/d6b/d7b/l126 0 2024-03-22T06:15:16.388 INFO:tasks.workunit.client.0.smithi006.stdout:1/761: creat d8/d9/d10/d7c/f100 x:0 0 0 2024-03-22T06:15:16.395 INFO:tasks.workunit.client.0.smithi006.stdout:6/759: dwrite d6/d1b/f2a [0,4194304] 0 2024-03-22T06:15:16.395 INFO:tasks.workunit.client.0.smithi006.stdout:6/760: stat d6/d1b 0 2024-03-22T06:15:16.395 INFO:tasks.workunit.client.1.smithi117.stdout:9/58: dwrite f2 [0,4194304] 0 2024-03-22T06:15:16.427 INFO:tasks.workunit.client.0.smithi006.stdout:5/892: link d0/db/d16/d1c/d34/d35/d85/d5e/dfb/ffd d0/db/f10e 0 2024-03-22T06:15:16.428 INFO:tasks.workunit.client.0.smithi006.stdout:2/916: mknod d13/d20/d26/dca/c15b 0 2024-03-22T06:15:16.428 INFO:tasks.workunit.client.0.smithi006.stdout:2/917: fdatasync d13/d34/d4d/d4e/fe0 0 2024-03-22T06:15:16.431 INFO:tasks.workunit.client.0.smithi006.stdout:9/745: rename d8/db3/d50/fa2 to d8/d1f/d34/ff2 0 2024-03-22T06:15:16.431 INFO:tasks.workunit.client.0.smithi006.stdout:9/746: fdatasync d8/d10/d23/f40 0 2024-03-22T06:15:16.434 INFO:tasks.workunit.client.0.smithi006.stdout:1/762: dwrite d8/d9/d13/d39/d60/fb2 [0,4194304] 0 2024-03-22T06:15:16.434 INFO:tasks.workunit.client.0.smithi006.stdout:1/763: chown d8/d9/d13/d39/d60/d8f/ce2 2066 1 2024-03-22T06:15:16.434 INFO:tasks.workunit.client.1.smithi117.stdout:9/59: mknod d4/d7/c15 0 2024-03-22T06:15:16.435 INFO:tasks.workunit.client.0.smithi006.stdout:4/851: creat d6/d1b/d20/d21/d2a/dc9/f127 x:0 0 0 2024-03-22T06:15:16.436 INFO:tasks.workunit.client.0.smithi006.stdout:6/761: dwrite d6/d7c/d92/fb9 [0,4194304] 0 2024-03-22T06:15:16.437 INFO:tasks.workunit.client.0.smithi006.stdout:5/893: rename d0/db/d16/f30 to d0/db/d50/d96/f10f 0 2024-03-22T06:15:16.438 INFO:tasks.workunit.client.0.smithi006.stdout:5/894: write d0/db/d50/d96/f63 [1756349,127443] 0 2024-03-22T06:15:16.441 INFO:tasks.workunit.client.0.smithi006.stdout:4/852: mkdir d6/d1b/d46/d101/d128 0 2024-03-22T06:15:16.441 INFO:tasks.workunit.client.0.smithi006.stdout:4/853: write d6/d13/f9c [422097,40686] 0 2024-03-22T06:15:16.441 INFO:tasks.workunit.client.0.smithi006.stdout:4/854: chown d6/d1b/d20/d63/d93/d9a/dfd/f11f 893805205 1 2024-03-22T06:15:16.441 INFO:tasks.workunit.client.0.smithi006.stdout:2/918: write d13/d20/d26/d19/f3e [1301982,112563] 0 2024-03-22T06:15:16.441 INFO:tasks.workunit.client.0.smithi006.stdout:2/919: truncate d13/d53/f38 1338891 0 2024-03-22T06:15:16.441 INFO:tasks.workunit.client.0.smithi006.stdout:2/920: dread - d13/d53/d6b/dd8/d11f/f144 zero size 2024-03-22T06:15:16.444 INFO:tasks.workunit.client.0.smithi006.stdout:1/764: dread d8/d9/f2d [0,4194304] 0 2024-03-22T06:15:16.447 INFO:tasks.workunit.client.0.smithi006.stdout:9/747: rename d8/d10/d23/l86 to d8/d10/d11/lf3 0 2024-03-22T06:15:16.447 INFO:tasks.workunit.client.0.smithi006.stdout:5/895: dread d0/db/d50/d96/f2d [0,4194304] 0 2024-03-22T06:15:16.450 INFO:tasks.workunit.client.1.smithi117.stdout:9/60: dread f2 [0,4194304] 0 2024-03-22T06:15:16.450 INFO:tasks.workunit.client.0.smithi006.stdout:2/921: mknod d13/d20/d35/d65/c15c 0 2024-03-22T06:15:16.450 INFO:tasks.workunit.client.0.smithi006.stdout:2/922: chown d13/d34/d4d/d4e/d75/d62/d14f/f156 167 1 2024-03-22T06:15:16.455 INFO:tasks.workunit.client.0.smithi006.stdout:9/748: symlink d8/d1f/d34/lf4 0 2024-03-22T06:15:16.455 INFO:tasks.workunit.client.0.smithi006.stdout:9/749: readlink d8/d10/d28/l51 0 2024-03-22T06:15:16.457 INFO:tasks.workunit.client.0.smithi006.stdout:2/923: symlink d13/d20/d35/d65/l15d 0 2024-03-22T06:15:16.457 INFO:tasks.workunit.client.0.smithi006.stdout:2/924: chown d13/d34/d4d/d4e/d75/da5/d6c/ce9 393279 1 2024-03-22T06:15:16.457 INFO:tasks.workunit.client.0.smithi006.stdout:2/925: readlink d13/d20/d35/d65/l15d 0 2024-03-22T06:15:16.457 INFO:tasks.workunit.client.0.smithi006.stdout:1/765: symlink d8/d9/d13/d24/l101 0 2024-03-22T06:15:16.458 INFO:tasks.workunit.client.0.smithi006.stdout:1/766: truncate d8/fd8 4477533 0 2024-03-22T06:15:16.459 INFO:tasks.workunit.client.0.smithi006.stdout:9/750: rmdir d8/d1f/dd9 0 2024-03-22T06:15:16.459 INFO:tasks.workunit.client.0.smithi006.stdout:9/751: fsync d8/d10/d20/d45/db9/fcb 0 2024-03-22T06:15:16.459 INFO:tasks.workunit.client.0.smithi006.stdout:9/752: stat d8/db3/d50/ff1 0 2024-03-22T06:15:16.460 INFO:tasks.workunit.client.0.smithi006.stdout:9/753: mkdir d8/d1f/df5 0 2024-03-22T06:15:16.464 INFO:tasks.workunit.client.0.smithi006.stdout:9/754: creat d8/d10/d20/d45/db9/de0/ff6 x:0 0 0 2024-03-22T06:15:16.466 INFO:tasks.workunit.client.0.smithi006.stdout:9/755: dread d8/d10/d28/fd1 [0,4194304] 0 2024-03-22T06:15:16.475 INFO:tasks.workunit.client.0.smithi006.stdout:5/896: dwrite d0/db/d16/d1c/d34/d35/d85/dda/fd3 [0,4194304] 0 2024-03-22T06:15:16.482 INFO:tasks.workunit.client.1.smithi117.stdout:8/67: getdents d1/d2 0 2024-03-22T06:15:16.484 INFO:tasks.workunit.client.1.smithi117.stdout:3/0: sync 2024-03-22T06:15:16.484 INFO:tasks.workunit.client.1.smithi117.stdout:2/41: sync 2024-03-22T06:15:16.484 INFO:tasks.workunit.client.1.smithi117.stdout:3/1: unlink - no file 2024-03-22T06:15:16.484 INFO:tasks.workunit.client.1.smithi117.stdout:3/2: write - no filename 2024-03-22T06:15:16.484 INFO:tasks.workunit.client.1.smithi117.stdout:1/34: sync 2024-03-22T06:15:16.484 INFO:tasks.workunit.client.1.smithi117.stdout:1/35: truncate d1/f7 5023760 0 2024-03-22T06:15:16.484 INFO:tasks.workunit.client.1.smithi117.stdout:3/3: chown . 30005614 1 2024-03-22T06:15:16.484 INFO:tasks.workunit.client.1.smithi117.stdout:3/4: write - no filename 2024-03-22T06:15:16.485 INFO:tasks.workunit.client.1.smithi117.stdout:5/1: sync 2024-03-22T06:15:16.485 INFO:tasks.workunit.client.1.smithi117.stdout:1/36: truncate d1/f5 704612 0 2024-03-22T06:15:16.485 INFO:tasks.workunit.client.1.smithi117.stdout:5/2: truncate - no filename 2024-03-22T06:15:16.485 INFO:tasks.workunit.client.1.smithi117.stdout:5/3: chown . 127515737 1 2024-03-22T06:15:16.485 INFO:tasks.workunit.client.1.smithi117.stdout:6/24: sync 2024-03-22T06:15:16.485 INFO:tasks.workunit.client.1.smithi117.stdout:0/1: sync 2024-03-22T06:15:16.485 INFO:tasks.workunit.client.1.smithi117.stdout:7/36: sync 2024-03-22T06:15:16.485 INFO:tasks.workunit.client.1.smithi117.stdout:8/68: truncate d1/d2/f7 6067808 0 2024-03-22T06:15:16.486 INFO:tasks.workunit.client.1.smithi117.stdout:1/37: symlink d1/la 0 2024-03-22T06:15:16.486 INFO:tasks.workunit.client.1.smithi117.stdout:1/38: write d1/f5 [1140575,79403] 0 2024-03-22T06:15:16.487 INFO:tasks.workunit.client.1.smithi117.stdout:3/5: mknod c0 0 2024-03-22T06:15:16.488 INFO:tasks.workunit.client.1.smithi117.stdout:5/4: symlink l0 0 2024-03-22T06:15:16.488 INFO:tasks.workunit.client.1.smithi117.stdout:5/5: dwrite - no filename 2024-03-22T06:15:16.490 INFO:tasks.workunit.client.1.smithi117.stdout:7/37: creat d8/d9/fd x:0 0 0 2024-03-22T06:15:16.490 INFO:tasks.workunit.client.1.smithi117.stdout:7/38: dread - d8/fb zero size 2024-03-22T06:15:16.490 INFO:tasks.workunit.client.1.smithi117.stdout:7/39: chown d8/cc 66298 1 2024-03-22T06:15:16.490 INFO:tasks.workunit.client.1.smithi117.stdout:7/40: chown f4 223097919 1 2024-03-22T06:15:16.490 INFO:tasks.workunit.client.1.smithi117.stdout:7/41: fsync f5 0 2024-03-22T06:15:16.490 INFO:tasks.workunit.client.1.smithi117.stdout:7/42: truncate f5 4992018 0 2024-03-22T06:15:16.490 INFO:tasks.workunit.client.1.smithi117.stdout:0/2: mknod c0 0 2024-03-22T06:15:16.491 INFO:tasks.workunit.client.1.smithi117.stdout:2/42: getdents d2 0 2024-03-22T06:15:16.492 INFO:tasks.workunit.client.1.smithi117.stdout:3/6: mknod c1 0 2024-03-22T06:15:16.493 INFO:tasks.workunit.client.1.smithi117.stdout:5/6: mknod c1 0 2024-03-22T06:15:16.494 INFO:tasks.workunit.client.1.smithi117.stdout:0/3: mkdir d1 0 2024-03-22T06:15:16.494 INFO:tasks.workunit.client.1.smithi117.stdout:0/4: dread - no filename 2024-03-22T06:15:16.495 INFO:tasks.workunit.client.1.smithi117.stdout:0/5: write - no filename 2024-03-22T06:15:16.496 INFO:tasks.workunit.client.1.smithi117.stdout:2/43: rmdir d2/d5/d8 39 2024-03-22T06:15:16.497 INFO:tasks.workunit.client.1.smithi117.stdout:3/7: creat f2 x:0 0 0 2024-03-22T06:15:16.497 INFO:tasks.workunit.client.1.smithi117.stdout:3/8: readlink - no filename 2024-03-22T06:15:16.498 INFO:tasks.workunit.client.1.smithi117.stdout:6/25: dread d3/f5 [0,4194304] 0 2024-03-22T06:15:16.499 INFO:tasks.workunit.client.1.smithi117.stdout:0/6: mknod d1/c2 0 2024-03-22T06:15:16.500 INFO:tasks.workunit.client.1.smithi117.stdout:1/39: write d1/f8 [412878,10602] 0 2024-03-22T06:15:16.500 INFO:tasks.workunit.client.0.smithi006.stdout:9/756: dwrite d8/db3/d50/d38/fa9 [0,4194304] 0 2024-03-22T06:15:16.500 INFO:tasks.workunit.client.0.smithi006.stdout:9/757: chown d8/db3/d50/ccf 2176 1 2024-03-22T06:15:16.501 INFO:tasks.workunit.client.1.smithi117.stdout:2/44: unlink d2/f9 0 2024-03-22T06:15:16.502 INFO:tasks.workunit.client.1.smithi117.stdout:3/9: mknod c3 0 2024-03-22T06:15:16.505 INFO:tasks.workunit.client.1.smithi117.stdout:0/7: mkdir d1/d3 0 2024-03-22T06:15:16.505 INFO:tasks.workunit.client.1.smithi117.stdout:0/8: chown d1 324074 1 2024-03-22T06:15:16.508 INFO:tasks.workunit.client.1.smithi117.stdout:6/26: dread d3/f5 [0,4194304] 0 2024-03-22T06:15:16.508 INFO:tasks.workunit.client.1.smithi117.stdout:6/27: readlink l2 0 2024-03-22T06:15:16.509 INFO:tasks.workunit.client.1.smithi117.stdout:2/45: symlink d2/d5/db/lf 0 2024-03-22T06:15:16.509 INFO:tasks.workunit.client.0.smithi006.stdout:2/926: dwrite d13/d20/f3d [0,4194304] 0 2024-03-22T06:15:16.509 INFO:tasks.workunit.client.0.smithi006.stdout:1/767: dwrite d8/d9/d13/d24/d32/dc7/fc6 [0,4194304] 0 2024-03-22T06:15:16.509 INFO:tasks.workunit.client.0.smithi006.stdout:5/897: dwrite d0/db/d16/d70/ff8 [0,4194304] 0 2024-03-22T06:15:16.510 INFO:tasks.workunit.client.1.smithi117.stdout:7/43: dwrite f7 [0,4194304] 0 2024-03-22T06:15:16.510 INFO:tasks.workunit.client.1.smithi117.stdout:0/9: rmdir d1 39 2024-03-22T06:15:16.510 INFO:tasks.workunit.client.1.smithi117.stdout:0/10: dwrite - no filename 2024-03-22T06:15:16.510 INFO:tasks.workunit.client.1.smithi117.stdout:0/11: dread - no filename 2024-03-22T06:15:16.510 INFO:tasks.workunit.client.1.smithi117.stdout:0/12: write - no filename 2024-03-22T06:15:16.511 INFO:tasks.workunit.client.0.smithi006.stdout:9/758: dread f1 [0,4194304] 0 2024-03-22T06:15:16.511 INFO:tasks.workunit.client.0.smithi006.stdout:2/927: creat d13/d20/d26/d19/d99/f15e x:0 0 0 2024-03-22T06:15:16.511 INFO:tasks.workunit.client.0.smithi006.stdout:2/928: write d13/d69/d10f/deb/ff4 [444741,77321] 0 2024-03-22T06:15:16.513 INFO:tasks.workunit.client.1.smithi117.stdout:2/46: getdents d2/d5/d8 0 2024-03-22T06:15:16.513 INFO:tasks.workunit.client.1.smithi117.stdout:1/40: dread d1/f5 [0,4194304] 0 2024-03-22T06:15:16.515 INFO:tasks.workunit.client.1.smithi117.stdout:6/28: dread d3/f4 [0,4194304] 0 2024-03-22T06:15:16.516 INFO:tasks.workunit.client.1.smithi117.stdout:0/13: mknod d1/d3/c4 0 2024-03-22T06:15:16.516 INFO:tasks.workunit.client.1.smithi117.stdout:0/14: dwrite - no filename 2024-03-22T06:15:16.516 INFO:tasks.workunit.client.1.smithi117.stdout:0/15: dread - no filename 2024-03-22T06:15:16.516 INFO:tasks.workunit.client.0.smithi006.stdout:6/762: truncate d6/d7c/d92/fb9 3728273 0 2024-03-22T06:15:16.517 INFO:tasks.workunit.client.0.smithi006.stdout:6/763: rename d6/d1d/d5c to d6/d1d/d5c/d81/deb 22 2024-03-22T06:15:16.517 INFO:tasks.workunit.client.0.smithi006.stdout:6/764: write d6/d1b/d2c/f4b [1153929,73146] 0 2024-03-22T06:15:16.519 INFO:tasks.workunit.client.1.smithi117.stdout:2/47: mkdir d2/d5/d8/d10 0 2024-03-22T06:15:16.519 INFO:tasks.workunit.client.1.smithi117.stdout:2/48: chown d2/d5/d8/d10 664843747 1 2024-03-22T06:15:16.519 INFO:tasks.workunit.client.1.smithi117.stdout:2/49: chown d2 1973520 1 2024-03-22T06:15:16.520 INFO:tasks.workunit.client.1.smithi117.stdout:1/41: dread d1/f7 [0,4194304] 0 2024-03-22T06:15:16.520 INFO:tasks.workunit.client.1.smithi117.stdout:1/42: stat d1/c6 0 2024-03-22T06:15:16.521 INFO:tasks.workunit.client.0.smithi006.stdout:1/768: unlink d8/d9/d3a/d40/d44/l29 0 2024-03-22T06:15:16.523 INFO:tasks.workunit.client.0.smithi006.stdout:5/898: creat d0/db/d16/d1c/d34/d35/d5d/d9d/f110 x:0 0 0 2024-03-22T06:15:16.525 INFO:tasks.workunit.client.0.smithi006.stdout:9/759: mkdir d8/d10/d82/df7 0 2024-03-22T06:15:16.525 INFO:tasks.workunit.client.0.smithi006.stdout:9/760: chown d8/d10/d20/d45/d49/fec 497 1 2024-03-22T06:15:16.527 INFO:tasks.workunit.client.1.smithi117.stdout:6/29: rename l2 to d3/d7/l8 0 2024-03-22T06:15:16.527 INFO:tasks.workunit.client.1.smithi117.stdout:6/30: chown d3/d7/l8 2956 1 2024-03-22T06:15:16.527 INFO:tasks.workunit.client.1.smithi117.stdout:7/44: dwrite d8/d9/fd [0,4194304] 0 2024-03-22T06:15:16.528 INFO:tasks.workunit.client.1.smithi117.stdout:2/50: write f1 [342791,21406] 0 2024-03-22T06:15:16.528 INFO:tasks.workunit.client.1.smithi117.stdout:2/51: chown f1 3 1 2024-03-22T06:15:16.528 INFO:tasks.workunit.client.1.smithi117.stdout:2/52: write f1 [8836713,70716] 0 2024-03-22T06:15:16.528 INFO:tasks.workunit.client.0.smithi006.stdout:2/929: symlink d13/d53/d6b/dd8/l15f 0 2024-03-22T06:15:16.530 INFO:tasks.workunit.client.0.smithi006.stdout:6/765: creat d6/d77/fec x:0 0 0 2024-03-22T06:15:16.532 INFO:tasks.workunit.client.0.smithi006.stdout:1/769: unlink d8/fa2 0 2024-03-22T06:15:16.534 INFO:tasks.workunit.client.1.smithi117.stdout:7/45: dread f7 [0,4194304] 0 2024-03-22T06:15:16.534 INFO:tasks.workunit.client.0.smithi006.stdout:9/761: dread d8/fd [0,4194304] 0 2024-03-22T06:15:16.536 INFO:tasks.workunit.client.0.smithi006.stdout:2/930: rmdir d13/d20/d35 39 2024-03-22T06:15:16.540 INFO:tasks.workunit.client.0.smithi006.stdout:6/766: unlink d6/d1b/d2c/fd6 0 2024-03-22T06:15:16.544 INFO:tasks.workunit.client.1.smithi117.stdout:6/31: symlink d3/l9 0 2024-03-22T06:15:16.547 INFO:tasks.workunit.client.0.smithi006.stdout:1/770: creat d8/d9/d13/d39/d94/db5/df1/f102 x:0 0 0 2024-03-22T06:15:16.547 INFO:tasks.workunit.client.0.smithi006.stdout:1/771: truncate d8/d9/d13/d39/d60/fec 696827 0 2024-03-22T06:15:16.548 INFO:tasks.workunit.client.1.smithi117.stdout:7/46: symlink d8/d9/le 0 2024-03-22T06:15:16.548 INFO:tasks.workunit.client.1.smithi117.stdout:2/53: dread f1 [4194304,4194304] 0 2024-03-22T06:15:16.548 INFO:tasks.workunit.client.1.smithi117.stdout:2/54: fsync f1 0 2024-03-22T06:15:16.550 INFO:tasks.workunit.client.0.smithi006.stdout:5/899: rename d0/f1a to d0/db/d76/f111 0 2024-03-22T06:15:16.551 INFO:tasks.workunit.client.1.smithi117.stdout:0/16: rmdir d1 39 2024-03-22T06:15:16.552 INFO:tasks.workunit.client.1.smithi117.stdout:6/32: mkdir d3/d7/da 0 2024-03-22T06:15:16.553 INFO:tasks.workunit.client.0.smithi006.stdout:2/931: stat d13/d34/d4d/d4e/d75/da5/c2e 0 2024-03-22T06:15:16.555 INFO:tasks.workunit.client.1.smithi117.stdout:7/47: link f6 d8/d9/ff 0 2024-03-22T06:15:16.556 INFO:tasks.workunit.client.0.smithi006.stdout:6/767: mknod d6/d1d/dc6/ced 0 2024-03-22T06:15:16.558 INFO:tasks.workunit.client.0.smithi006.stdout:1/772: symlink d8/d9/d10/d7c/l103 0 2024-03-22T06:15:16.560 INFO:tasks.workunit.client.1.smithi117.stdout:0/17: unlink d1/d3/c4 0 2024-03-22T06:15:16.560 INFO:tasks.workunit.client.1.smithi117.stdout:0/18: truncate - no filename 2024-03-22T06:15:16.561 INFO:tasks.workunit.client.0.smithi006.stdout:5/900: truncate d0/db/d16/d1c/d34/d35/d85/dda/fd3 2343278 0 2024-03-22T06:15:16.563 INFO:tasks.workunit.client.1.smithi117.stdout:6/33: rename d3/c6 to d3/d7/da/cb 0 2024-03-22T06:15:16.563 INFO:tasks.workunit.client.1.smithi117.stdout:6/34: chown d3/d7/da 960323 1 2024-03-22T06:15:16.564 INFO:tasks.workunit.client.1.smithi117.stdout:1/43: truncate d1/f7 1590242 0 2024-03-22T06:15:16.564 INFO:tasks.workunit.client.0.smithi006.stdout:2/932: symlink d13/d20/d26/dca/dea/l160 0 2024-03-22T06:15:16.566 INFO:tasks.workunit.client.1.smithi117.stdout:7/48: mkdir d8/d9/da/d10 0 2024-03-22T06:15:16.567 INFO:tasks.workunit.client.1.smithi117.stdout:0/19: mkdir d1/d3/d5 0 2024-03-22T06:15:16.568 INFO:tasks.workunit.client.1.smithi117.stdout:0/20: chown d1/c2 989122 1 2024-03-22T06:15:16.568 INFO:tasks.workunit.client.1.smithi117.stdout:0/21: chown d1/d3/d5 1863548 1 2024-03-22T06:15:16.568 INFO:tasks.workunit.client.0.smithi006.stdout:6/768: getdents d6/d1d/d99/dcd 0 2024-03-22T06:15:16.571 INFO:tasks.workunit.client.0.smithi006.stdout:1/773: write d8/d9/f67 [3567187,110951] 0 2024-03-22T06:15:16.572 INFO:tasks.workunit.client.0.smithi006.stdout:1/774: stat d8/d9/d10/d7c 0 2024-03-22T06:15:16.572 INFO:tasks.workunit.client.1.smithi117.stdout:7/49: dread f4 [0,4194304] 0 2024-03-22T06:15:16.572 INFO:tasks.workunit.client.1.smithi117.stdout:1/44: mknod d1/cb 0 2024-03-22T06:15:16.572 INFO:tasks.workunit.client.1.smithi117.stdout:6/35: write d3/f4 [382145,101840] 0 2024-03-22T06:15:16.572 INFO:tasks.workunit.client.1.smithi117.stdout:6/36: readlink d3/d7/l8 0 2024-03-22T06:15:16.572 INFO:tasks.workunit.client.1.smithi117.stdout:6/37: write d3/f5 [4648405,29944] 0 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:6/38: stat d3 0 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/22: mknod d1/d3/d5/c6 0 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/23: stat d1 0 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/24: dread - no filename 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/25: write - no filename 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/26: dwrite - no filename 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/27: write - no filename 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/28: write - no filename 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/29: dread - no filename 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/30: dwrite - no filename 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/31: dread - no filename 2024-03-22T06:15:16.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/32: write - no filename 2024-03-22T06:15:16.574 INFO:tasks.workunit.client.1.smithi117.stdout:7/50: creat d8/d9/da/d10/f11 x:0 0 0 2024-03-22T06:15:16.574 INFO:tasks.workunit.client.0.smithi006.stdout:2/933: dread d13/d20/d35/d65/f108 [0,4194304] 0 2024-03-22T06:15:16.575 INFO:tasks.workunit.client.0.smithi006.stdout:9/762: dwrite f1 [0,4194304] 0 2024-03-22T06:15:16.575 INFO:tasks.workunit.client.1.smithi117.stdout:8/69: dwrite d1/d2/f7 [4194304,4194304] 0 2024-03-22T06:15:16.577 INFO:tasks.workunit.client.1.smithi117.stdout:2/55: dwrite d2/f4 [0,4194304] 0 2024-03-22T06:15:16.578 INFO:tasks.workunit.client.1.smithi117.stdout:6/39: fsync d3/f4 0 2024-03-22T06:15:16.578 INFO:tasks.workunit.client.0.smithi006.stdout:5/901: dread d0/f5 [0,4194304] 0 2024-03-22T06:15:16.580 INFO:tasks.workunit.client.0.smithi006.stdout:2/934: rename d13/d20/d35/d65/f7b to d13/d34/d4d/d11a/f161 0 2024-03-22T06:15:16.584 INFO:tasks.workunit.client.0.smithi006.stdout:6/769: write d6/d1b/dae/fb4 [1982448,42874] 0 2024-03-22T06:15:16.584 INFO:tasks.workunit.client.0.smithi006.stdout:6/770: fsync d6/d1d/d32/f45 0 2024-03-22T06:15:16.584 INFO:tasks.workunit.client.0.smithi006.stdout:6/771: fdatasync d6/d1d/d5c/f5e 0 2024-03-22T06:15:16.584 INFO:tasks.workunit.client.0.smithi006.stdout:6/772: write d6/d1d/d32/fe1 [782213,96047] 0 2024-03-22T06:15:16.584 INFO:tasks.workunit.client.0.smithi006.stdout:6/773: fdatasync d6/f66 0 2024-03-22T06:15:16.585 INFO:tasks.workunit.client.0.smithi006.stdout:6/774: fsync d6/f11 0 2024-03-22T06:15:16.585 INFO:tasks.workunit.client.0.smithi006.stdout:6/775: dread - d6/d24/d44/fcb zero size 2024-03-22T06:15:16.586 INFO:tasks.workunit.client.0.smithi006.stdout:1/775: dread d8/d9/d13/d24/f7f [0,4194304] 0 2024-03-22T06:15:16.587 INFO:tasks.workunit.client.1.smithi117.stdout:2/56: getdents d2/d5/d8 0 2024-03-22T06:15:16.587 INFO:tasks.workunit.client.1.smithi117.stdout:2/57: fdatasync d2/f3 0 2024-03-22T06:15:16.588 INFO:tasks.workunit.client.1.smithi117.stdout:2/58: fsync d2/f3 0 2024-03-22T06:15:16.595 INFO:tasks.workunit.client.1.smithi117.stdout:6/40: write d3/f5 [149285,56257] 0 2024-03-22T06:15:16.595 INFO:tasks.workunit.client.1.smithi117.stdout:6/41: fsync d3/f4 0 2024-03-22T06:15:16.597 INFO:tasks.workunit.client.1.smithi117.stdout:7/51: dwrite f4 [4194304,4194304] 0 2024-03-22T06:15:16.597 INFO:tasks.workunit.client.1.smithi117.stdout:7/52: stat d8/d9/le 0 2024-03-22T06:15:16.603 INFO:tasks.workunit.client.1.smithi117.stdout:2/59: rmdir d2/d5/d8 39 2024-03-22T06:15:16.604 INFO:tasks.workunit.client.0.smithi006.stdout:5/902: mknod d0/db/d50/d82/c112 0 2024-03-22T06:15:16.604 INFO:tasks.workunit.client.1.smithi117.stdout:7/53: write d8/d9/ff [2148230,113090] 0 2024-03-22T06:15:16.608 INFO:tasks.workunit.client.1.smithi117.stdout:6/42: getdents d3/d7 0 2024-03-22T06:15:16.608 INFO:tasks.workunit.client.1.smithi117.stdout:6/43: fdatasync d3/f4 0 2024-03-22T06:15:16.611 INFO:tasks.workunit.client.0.smithi006.stdout:2/935: link d13/d69/c120 d13/d69/d10f/c162 0 2024-03-22T06:15:16.614 INFO:tasks.workunit.client.1.smithi117.stdout:2/60: dread d2/f4 [0,4194304] 0 2024-03-22T06:15:16.617 INFO:tasks.workunit.client.1.smithi117.stdout:0/33: rmdir d1/d3 39 2024-03-22T06:15:16.619 INFO:tasks.workunit.client.1.smithi117.stdout:6/44: creat d3/d7/da/fc x:0 0 0 2024-03-22T06:15:16.619 INFO:tasks.workunit.client.1.smithi117.stdout:6/45: readlink d3/l9 0 2024-03-22T06:15:16.620 INFO:tasks.workunit.client.0.smithi006.stdout:6/776: truncate d6/d7c/d92/faa 5073498 0 2024-03-22T06:15:16.620 INFO:tasks.workunit.client.0.smithi006.stdout:6/777: getdents d6/d7c/d83 0 2024-03-22T06:15:16.620 INFO:tasks.workunit.client.1.smithi117.stdout:7/54: dwrite f5 [4194304,4194304] 0 2024-03-22T06:15:16.623 INFO:tasks.workunit.client.1.smithi117.stdout:0/34: mkdir d1/d7 0 2024-03-22T06:15:16.627 INFO:tasks.workunit.client.1.smithi117.stdout:6/46: creat d3/d7/fd x:0 0 0 2024-03-22T06:15:16.628 INFO:tasks.workunit.client.1.smithi117.stdout:1/45: dwrite d1/f8 [0,4194304] 0 2024-03-22T06:15:16.628 INFO:tasks.workunit.client.1.smithi117.stdout:1/46: readlink l0 0 2024-03-22T06:15:16.628 INFO:tasks.workunit.client.1.smithi117.stdout:1/47: fdatasync d1/f7 0 2024-03-22T06:15:16.628 INFO:tasks.workunit.client.1.smithi117.stdout:1/48: readlink d1/la 0 2024-03-22T06:15:16.628 INFO:tasks.workunit.client.1.smithi117.stdout:1/49: truncate d1/f5 1899056 0 2024-03-22T06:15:16.628 INFO:tasks.workunit.client.1.smithi117.stdout:1/50: chown d1/c6 231811 1 2024-03-22T06:15:16.628 INFO:tasks.workunit.client.1.smithi117.stdout:1/51: stat d1/la 0 2024-03-22T06:15:16.628 INFO:tasks.workunit.client.1.smithi117.stdout:1/52: creat d1/fc x:0 0 0 2024-03-22T06:15:16.629 INFO:tasks.workunit.client.1.smithi117.stdout:1/53: creat d1/fd x:0 0 0 2024-03-22T06:15:16.629 INFO:tasks.workunit.client.0.smithi006.stdout:5/903: mkdir d0/db/d50/d96/d52/d113 0 2024-03-22T06:15:16.629 INFO:tasks.workunit.client.0.smithi006.stdout:5/904: chown d0/db/d50/d58/df7/cf9 12 1 2024-03-22T06:15:16.631 INFO:tasks.workunit.client.0.smithi006.stdout:2/936: unlink d13/d53/d6b/fb5 0 2024-03-22T06:15:16.632 INFO:tasks.workunit.client.1.smithi117.stdout:7/55: getdents d8/d9/da 0 2024-03-22T06:15:16.632 INFO:tasks.workunit.client.1.smithi117.stdout:7/56: chown c2 160820763 1 2024-03-22T06:15:16.635 INFO:tasks.workunit.client.1.smithi117.stdout:0/35: unlink d1/d3/d5/c6 0 2024-03-22T06:15:16.635 INFO:tasks.workunit.client.1.smithi117.stdout:0/36: write - no filename 2024-03-22T06:15:16.635 INFO:tasks.workunit.client.1.smithi117.stdout:0/37: chown d1 129575438 1 2024-03-22T06:15:16.635 INFO:tasks.workunit.client.1.smithi117.stdout:0/38: chown d1/d7 49441766 1 2024-03-22T06:15:16.635 INFO:tasks.workunit.client.1.smithi117.stdout:0/39: stat d1/d3/d5 0 2024-03-22T06:15:16.635 INFO:tasks.workunit.client.1.smithi117.stdout:0/40: dread - no filename 2024-03-22T06:15:16.639 INFO:tasks.workunit.client.1.smithi117.stdout:0/41: mkdir d1/d8 0 2024-03-22T06:15:16.639 INFO:tasks.workunit.client.1.smithi117.stdout:0/42: dread - no filename 2024-03-22T06:15:16.639 INFO:tasks.workunit.client.1.smithi117.stdout:0/43: dread - no filename 2024-03-22T06:15:16.640 INFO:tasks.workunit.client.0.smithi006.stdout:5/905: write d0/db/d50/d96/f10f [4097527,14189] 0 2024-03-22T06:15:16.641 INFO:tasks.workunit.client.0.smithi006.stdout:0/640: sync 2024-03-22T06:15:16.641 INFO:tasks.workunit.client.0.smithi006.stdout:0/641: fdatasync db/d87/f9f 0 2024-03-22T06:15:16.642 INFO:tasks.workunit.client.0.smithi006.stdout:6/778: dwrite d6/d24/d44/f4d [0,4194304] 0 2024-03-22T06:15:16.644 INFO:tasks.workunit.client.1.smithi117.stdout:0/44: mknod d1/c9 0 2024-03-22T06:15:16.646 INFO:tasks.workunit.client.0.smithi006.stdout:5/906: symlink d0/db/d50/d82/d103/d92/db7/l114 0 2024-03-22T06:15:16.646 INFO:tasks.workunit.client.1.smithi117.stdout:4/69: sync 2024-03-22T06:15:16.648 INFO:tasks.workunit.client.1.smithi117.stdout:0/45: link d1/c2 d1/ca 0 2024-03-22T06:15:16.648 INFO:tasks.workunit.client.1.smithi117.stdout:0/46: write - no filename 2024-03-22T06:15:16.648 INFO:tasks.workunit.client.1.smithi117.stdout:0/47: chown d1/ca 31341 1 2024-03-22T06:15:16.650 INFO:tasks.workunit.client.1.smithi117.stdout:4/70: truncate d0/d4/f17 1176910 0 2024-03-22T06:15:16.650 INFO:tasks.workunit.client.1.smithi117.stdout:4/71: write d0/d4/fe [3089119,48099] 0 2024-03-22T06:15:16.652 INFO:tasks.workunit.client.0.smithi006.stdout:2/937: dwrite d13/d53/f38 [0,4194304] 0 2024-03-22T06:15:16.652 INFO:tasks.workunit.client.0.smithi006.stdout:2/938: write d13/d20/d26/d19/f46 [1464496,103978] 0 2024-03-22T06:15:16.652 INFO:tasks.workunit.client.0.smithi006.stdout:2/939: chown d13/d34/d4d/d4e/d75/d62/fb6 211309 1 2024-03-22T06:15:16.654 INFO:tasks.workunit.client.0.smithi006.stdout:5/907: unlink d0/db/d16/d1c/d34/d4d/fd7 0 2024-03-22T06:15:16.657 INFO:tasks.workunit.client.1.smithi117.stdout:4/72: rename d0/d4/f15 to d0/d19/f1a 0 2024-03-22T06:15:16.657 INFO:tasks.workunit.client.0.smithi006.stdout:0/642: rmdir db/d10/d33/db4 39 2024-03-22T06:15:16.658 INFO:tasks.workunit.client.0.smithi006.stdout:0/643: write db/d10/d35/d6c/fbb [1055829,15427] 0 2024-03-22T06:15:16.664 INFO:tasks.workunit.client.1.smithi117.stdout:4/73: rmdir d0/d19 39 2024-03-22T06:15:16.665 INFO:tasks.workunit.client.1.smithi117.stdout:4/74: readlink d0/d4/l13 0 2024-03-22T06:15:16.665 INFO:tasks.workunit.client.1.smithi117.stdout:4/75: creat d0/d4/f1b x:0 0 0 2024-03-22T06:15:16.665 INFO:tasks.workunit.client.1.smithi117.stdout:4/76: fdatasync d0/d4/fa 0 2024-03-22T06:15:16.665 INFO:tasks.workunit.client.1.smithi117.stdout:4/77: stat d0/d4/cd 0 2024-03-22T06:15:16.666 INFO:tasks.workunit.client.0.smithi006.stdout:0/644: write db/dc/d38/fc1 [4029710,71945] 0 2024-03-22T06:15:16.667 INFO:tasks.workunit.client.1.smithi117.stdout:4/78: mkdir d0/d19/d1c 0 2024-03-22T06:15:16.673 INFO:tasks.workunit.client.0.smithi006.stdout:0/645: unlink db/d3d/d4e/f6f 0 2024-03-22T06:15:16.674 INFO:tasks.workunit.client.1.smithi117.stdout:1/54: truncate d1/f8 3983521 0 2024-03-22T06:15:16.674 INFO:tasks.workunit.client.1.smithi117.stdout:1/55: fdatasync d1/f8 0 2024-03-22T06:15:16.676 INFO:tasks.workunit.client.1.smithi117.stdout:0/48: rmdir d1/d3/d5 0 2024-03-22T06:15:16.678 INFO:tasks.workunit.client.1.smithi117.stdout:1/56: rename d1/f5 to d1/fe 0 2024-03-22T06:15:16.678 INFO:tasks.workunit.client.1.smithi117.stdout:1/57: truncate d1/fd 215300 0 2024-03-22T06:15:16.680 INFO:tasks.workunit.client.1.smithi117.stdout:0/49: creat d1/d7/fb x:0 0 0 2024-03-22T06:15:16.680 INFO:tasks.workunit.client.1.smithi117.stdout:0/50: truncate d1/d7/fb 851538 0 2024-03-22T06:15:16.687 INFO:tasks.workunit.client.0.smithi006.stdout:2/940: dwrite d13/d53/dc5/ffa [0,4194304] 0 2024-03-22T06:15:16.687 INFO:tasks.workunit.client.0.smithi006.stdout:5/908: dwrite d0/db/d16/d1c/d34/f4b [0,4194304] 0 2024-03-22T06:15:16.687 INFO:tasks.workunit.client.0.smithi006.stdout:5/909: write d0/db/d16/d1c/d34/d35/d85/d5e/d66/f9b [944814,122394] 0 2024-03-22T06:15:16.687 INFO:tasks.workunit.client.0.smithi006.stdout:5/910: dread - d0/db/d76/db4/f108 zero size 2024-03-22T06:15:16.688 INFO:tasks.workunit.client.0.smithi006.stdout:2/941: mkdir d13/d20/d26/d19/d163 0 2024-03-22T06:15:16.688 INFO:tasks.workunit.client.0.smithi006.stdout:1/776: dwrite d8/d9/d10/d7c/f82 [0,4194304] 0 2024-03-22T06:15:16.690 INFO:tasks.workunit.client.1.smithi117.stdout:4/79: getdents d0/d4 0 2024-03-22T06:15:16.690 INFO:tasks.workunit.client.1.smithi117.stdout:4/80: fsync d0/f3 0 2024-03-22T06:15:16.695 INFO:tasks.workunit.client.1.smithi117.stdout:0/51: dwrite d1/d7/fb [0,4194304] 0 2024-03-22T06:15:16.695 INFO:tasks.workunit.client.0.smithi006.stdout:2/942: creat d13/d34/d4d/d4e/d75/da5/d82/f164 x:0 0 0 2024-03-22T06:15:16.701 INFO:tasks.workunit.client.0.smithi006.stdout:5/911: rmdir d0/db/d16/d1c/d34/d35/d85/d5e/d66 39 2024-03-22T06:15:16.718 INFO:tasks.workunit.client.1.smithi117.stdout:4/81: dwrite d0/d4/fb [0,4194304] 0 2024-03-22T06:15:16.724 INFO:tasks.workunit.client.1.smithi117.stdout:0/52: mkdir d1/d8/dc 0 2024-03-22T06:15:16.724 INFO:tasks.workunit.client.1.smithi117.stdout:0/53: truncate d1/d7/fb 4428699 0 2024-03-22T06:15:16.724 INFO:tasks.workunit.client.1.smithi117.stdout:0/54: fdatasync d1/d7/fb 0 2024-03-22T06:15:16.725 INFO:tasks.workunit.client.0.smithi006.stdout:1/777: rename d8/d9/d13/d39/d60/fec to d8/d9/d13/d24/f104 0 2024-03-22T06:15:16.727 INFO:tasks.workunit.client.0.smithi006.stdout:2/943: dwrite d13/d53/d6b/dd8/d11f/f14d [0,4194304] 0 2024-03-22T06:15:16.729 INFO:tasks.workunit.client.0.smithi006.stdout:5/912: mkdir d0/d115 0 2024-03-22T06:15:16.731 INFO:tasks.workunit.client.0.smithi006.stdout:1/778: mknod d8/d9/d10/c105 0 2024-03-22T06:15:16.731 INFO:tasks.workunit.client.0.smithi006.stdout:1/779: write d8/d9/d3a/d18/ffe [642786,50348] 0 2024-03-22T06:15:16.735 INFO:tasks.workunit.client.1.smithi117.stdout:4/82: dwrite d0/d4/f17 [0,4194304] 0 2024-03-22T06:15:16.735 INFO:tasks.workunit.client.1.smithi117.stdout:4/83: readlink d0/d4/l13 0 2024-03-22T06:15:16.735 INFO:tasks.workunit.client.1.smithi117.stdout:4/84: chown d0/d4/fa 2 1 2024-03-22T06:15:16.736 INFO:tasks.workunit.client.0.smithi006.stdout:5/913: rename d0/db/d16/d1c/c105 to d0/db/d16/d1c/d34/d35/d85/dda/d65/dd4/c116 0 2024-03-22T06:15:16.738 INFO:tasks.workunit.client.0.smithi006.stdout:1/780: rmdir d8/d9/d13/d39/d60 39 2024-03-22T06:15:16.739 INFO:tasks.workunit.client.1.smithi117.stdout:0/55: getdents d1/d7 0 2024-03-22T06:15:16.742 INFO:tasks.workunit.client.0.smithi006.stdout:5/914: creat d0/db/d16/d1c/d34/d35/d85/d5e/d66/f117 x:0 0 0 2024-03-22T06:15:16.742 INFO:tasks.workunit.client.0.smithi006.stdout:5/915: truncate d0/fa4 73883 0 2024-03-22T06:15:16.742 INFO:tasks.workunit.client.0.smithi006.stdout:5/916: read d0/db/d99/fc4 [436163,94528] 0 2024-03-22T06:15:16.742 INFO:tasks.workunit.client.0.smithi006.stdout:5/917: write d0/db/f1e [745155,102884] 0 2024-03-22T06:15:16.743 INFO:tasks.workunit.client.1.smithi117.stdout:0/56: unlink d1/ca 0 2024-03-22T06:15:16.744 INFO:tasks.workunit.client.0.smithi006.stdout:1/781: link d8/d9/d13/d39/ld9 d8/l106 0 2024-03-22T06:15:16.745 INFO:tasks.workunit.client.0.smithi006.stdout:2/944: dwrite d13/d20/d35/d65/f108 [0,4194304] 0 2024-03-22T06:15:16.745 INFO:tasks.workunit.client.0.smithi006.stdout:2/945: creat d13/d20/d26/dca/f165 x:0 0 0 2024-03-22T06:15:16.746 INFO:tasks.workunit.client.0.smithi006.stdout:2/946: stat d13/d20/d26/l8e 0 2024-03-22T06:15:16.746 INFO:tasks.workunit.client.0.smithi006.stdout:2/947: write d13/d34/d4d/d4e/d75/da5/f5f [3796040,88665] 0 2024-03-22T06:15:16.746 INFO:tasks.workunit.client.0.smithi006.stdout:2/948: write d13/d20/d26/f15 [1427426,26415] 0 2024-03-22T06:15:16.746 INFO:tasks.workunit.client.0.smithi006.stdout:2/949: fsync d13/d53/d6b/f7e 0 2024-03-22T06:15:16.748 INFO:tasks.workunit.client.0.smithi006.stdout:5/918: creat d0/db/d50/d58/df7/f118 x:0 0 0 2024-03-22T06:15:16.748 INFO:tasks.workunit.client.0.smithi006.stdout:2/950: write d13/d53/dc5/fd5 [98710,23736] 0 2024-03-22T06:15:16.749 INFO:tasks.workunit.client.1.smithi117.stdout:0/57: link d1/c2 d1/d8/dc/cd 0 2024-03-22T06:15:16.749 INFO:tasks.workunit.client.1.smithi117.stdout:9/61: sync 2024-03-22T06:15:16.749 INFO:tasks.workunit.client.1.smithi117.stdout:9/62: creat d4/d7/f16 x:0 0 0 2024-03-22T06:15:16.753 INFO:tasks.workunit.client.1.smithi117.stdout:0/58: mkdir d1/d7/de 0 2024-03-22T06:15:16.757 INFO:tasks.workunit.client.1.smithi117.stdout:9/63: creat d4/f17 x:0 0 0 2024-03-22T06:15:16.759 INFO:tasks.workunit.client.0.smithi006.stdout:2/951: link d13/d20/fcb d13/d12d/f166 0 2024-03-22T06:15:16.759 INFO:tasks.workunit.client.0.smithi006.stdout:2/952: chown d13/d20/da6/ca8 220425332 1 2024-03-22T06:15:16.761 INFO:tasks.workunit.client.1.smithi117.stdout:0/59: creat d1/d7/ff x:0 0 0 2024-03-22T06:15:16.762 INFO:tasks.workunit.client.1.smithi117.stdout:9/64: truncate f2 458044 0 2024-03-22T06:15:16.762 INFO:tasks.workunit.client.1.smithi117.stdout:9/65: chown d4/f11 10355548 1 2024-03-22T06:15:16.763 INFO:tasks.workunit.client.1.smithi117.stdout:0/60: mkdir d1/d10 0 2024-03-22T06:15:16.764 INFO:tasks.workunit.client.0.smithi006.stdout:5/919: dwrite d0/db/d16/d70/f75 [0,4194304] 0 2024-03-22T06:15:16.764 INFO:tasks.workunit.client.0.smithi006.stdout:5/920: write d0/db/d76/db4/f108 [817297,7511] 0 2024-03-22T06:15:16.764 INFO:tasks.workunit.client.0.smithi006.stdout:5/921: dread - d0/d98/fe0 zero size 2024-03-22T06:15:16.764 INFO:tasks.workunit.client.0.smithi006.stdout:5/922: stat d0/db/d50/d58/f6f 0 2024-03-22T06:15:16.765 INFO:tasks.workunit.client.1.smithi117.stdout:0/61: creat d1/d10/f11 x:0 0 0 2024-03-22T06:15:16.766 INFO:tasks.workunit.client.1.smithi117.stdout:9/66: mknod d4/c18 0 2024-03-22T06:15:16.766 INFO:tasks.workunit.client.1.smithi117.stdout:9/67: creat d4/d7/f19 x:0 0 0 2024-03-22T06:15:16.767 INFO:tasks.workunit.client.0.smithi006.stdout:5/923: symlink d0/db/d16/d1c/d34/d35/d85/d5e/d66/l119 0 2024-03-22T06:15:16.768 INFO:tasks.workunit.client.0.smithi006.stdout:5/924: mkdir d0/de8/d11a 0 2024-03-22T06:15:16.770 INFO:tasks.workunit.client.1.smithi117.stdout:9/68: rename d4/f11 to d4/d8/f1a 0 2024-03-22T06:15:16.770 INFO:tasks.workunit.client.1.smithi117.stdout:9/69: creat d4/d7/f1b x:0 0 0 2024-03-22T06:15:16.773 INFO:tasks.workunit.client.0.smithi006.stdout:2/953: dwrite d13/d20/d26/dca/f12f [0,4194304] 0 2024-03-22T06:15:16.779 INFO:tasks.workunit.client.0.smithi006.stdout:2/954: rmdir d13/d34/d4d/d4e/de3 39 2024-03-22T06:15:16.781 INFO:tasks.workunit.client.0.smithi006.stdout:2/955: creat d13/d34/d4d/d4e/d75/da5/d10a/f167 x:0 0 0 2024-03-22T06:15:16.781 INFO:tasks.workunit.client.0.smithi006.stdout:2/956: stat d13/d34/d4d/fdb 0 2024-03-22T06:15:16.782 INFO:tasks.workunit.client.0.smithi006.stdout:2/957: creat d13/d34/d4d/d4e/d75/da5/d6c/f168 x:0 0 0 2024-03-22T06:15:16.785 INFO:tasks.workunit.client.1.smithi117.stdout:0/62: getdents d1/d7 0 2024-03-22T06:15:16.785 INFO:tasks.workunit.client.1.smithi117.stdout:0/63: getdents d1/d7/de 0 2024-03-22T06:15:16.785 INFO:tasks.workunit.client.1.smithi117.stdout:0/64: chown d1/d8/dc 0 1 2024-03-22T06:15:16.786 INFO:tasks.workunit.client.0.smithi006.stdout:2/958: dread d13/d34/d4d/d4e/d75/d62/f100 [0,4194304] 0 2024-03-22T06:15:16.787 INFO:tasks.workunit.client.1.smithi117.stdout:0/65: creat d1/d3/f12 x:0 0 0 2024-03-22T06:15:16.788 INFO:tasks.workunit.client.0.smithi006.stdout:2/959: mknod d13/d20/d35/d104/c169 0 2024-03-22T06:15:16.790 INFO:tasks.workunit.client.1.smithi117.stdout:0/66: truncate d1/d7/fb 1344189 0 2024-03-22T06:15:16.791 INFO:tasks.workunit.client.0.smithi006.stdout:2/960: getdents d13/d53/dc5/d10e 0 2024-03-22T06:15:16.793 INFO:tasks.workunit.client.0.smithi006.stdout:2/961: rmdir d13/d20/d159 0 2024-03-22T06:15:16.793 INFO:tasks.workunit.client.0.smithi006.stdout:2/962: fsync d13/d34/d4d/fdb 0 2024-03-22T06:15:16.794 INFO:tasks.workunit.client.1.smithi117.stdout:9/70: dread f2 [0,4194304] 0 2024-03-22T06:15:16.794 INFO:tasks.workunit.client.1.smithi117.stdout:9/71: creat d4/d7/f1c x:0 0 0 2024-03-22T06:15:16.794 INFO:tasks.workunit.client.1.smithi117.stdout:9/72: read - d4/d7/f1b zero size 2024-03-22T06:15:16.794 INFO:tasks.workunit.client.0.smithi006.stdout:2/963: link d13/f23 d13/d20/da6/f16a 0 2024-03-22T06:15:16.797 INFO:tasks.workunit.client.0.smithi006.stdout:2/964: unlink d13/d53/l9e 0 2024-03-22T06:15:16.800 INFO:tasks.workunit.client.0.smithi006.stdout:2/965: symlink d13/d34/d4d/d4e/d75/dcc/l16b 0 2024-03-22T06:15:16.805 INFO:tasks.workunit.client.0.smithi006.stdout:2/966: unlink d13/d20/d26/dca/def/f118 0 2024-03-22T06:15:16.805 INFO:tasks.workunit.client.0.smithi006.stdout:2/967: write d13/d34/d4d/d4e/d75/d62/f9c [275280,34823] 0 2024-03-22T06:15:16.806 INFO:tasks.workunit.client.0.smithi006.stdout:2/968: chown d13/d69/c111 13723 1 2024-03-22T06:15:16.810 INFO:tasks.workunit.client.1.smithi117.stdout:9/73: rmdir d4 39 2024-03-22T06:15:16.810 INFO:tasks.workunit.client.1.smithi117.stdout:9/74: dread - d4/f17 zero size 2024-03-22T06:15:16.811 INFO:tasks.workunit.client.1.smithi117.stdout:9/75: symlink d4/d7/d10/l1d 0 2024-03-22T06:15:16.811 INFO:tasks.workunit.client.1.smithi117.stdout:0/67: write d1/d7/fb [510765,25572] 0 2024-03-22T06:15:16.812 INFO:tasks.workunit.client.0.smithi006.stdout:2/969: dread d13/d34/d6e/f9b [0,4194304] 0 2024-03-22T06:15:16.813 INFO:tasks.workunit.client.1.smithi117.stdout:9/76: creat d4/f1e x:0 0 0 2024-03-22T06:15:16.813 INFO:tasks.workunit.client.1.smithi117.stdout:9/77: truncate d4/d7/f16 729892 0 2024-03-22T06:15:16.813 INFO:tasks.workunit.client.1.smithi117.stdout:9/78: write d4/d7/f1c [358786,5408] 0 2024-03-22T06:15:16.814 INFO:tasks.workunit.client.1.smithi117.stdout:3/10: sync 2024-03-22T06:15:16.814 INFO:tasks.workunit.client.1.smithi117.stdout:5/7: sync 2024-03-22T06:15:16.814 INFO:tasks.workunit.client.1.smithi117.stdout:8/70: sync 2024-03-22T06:15:16.814 INFO:tasks.workunit.client.1.smithi117.stdout:3/11: dread - f2 zero size 2024-03-22T06:15:16.814 INFO:tasks.workunit.client.1.smithi117.stdout:3/12: write f2 [588913,70482] 0 2024-03-22T06:15:16.814 INFO:tasks.workunit.client.1.smithi117.stdout:3/13: write f2 [1382021,107574] 0 2024-03-22T06:15:16.814 INFO:tasks.workunit.client.1.smithi117.stdout:0/68: symlink d1/d7/l13 0 2024-03-22T06:15:16.815 INFO:tasks.workunit.client.0.smithi006.stdout:2/970: unlink d13/d34/d4d/d4e/d75/da5/l9f 0 2024-03-22T06:15:16.815 INFO:tasks.workunit.client.0.smithi006.stdout:2/971: fdatasync d13/d34/d4d/d4e/fe0 0 2024-03-22T06:15:16.818 INFO:tasks.workunit.client.1.smithi117.stdout:9/79: unlink d4/d7/d10/l1d 0 2024-03-22T06:15:16.821 INFO:tasks.workunit.client.1.smithi117.stdout:5/8: creat f2 x:0 0 0 2024-03-22T06:15:16.821 INFO:tasks.workunit.client.1.smithi117.stdout:5/9: write f2 [123853,34999] 0 2024-03-22T06:15:16.821 INFO:tasks.workunit.client.1.smithi117.stdout:5/10: creat f3 x:0 0 0 2024-03-22T06:15:16.821 INFO:tasks.workunit.client.1.smithi117.stdout:3/14: link c1 c4 0 2024-03-22T06:15:16.822 INFO:tasks.workunit.client.1.smithi117.stdout:3/15: readlink - no filename 2024-03-22T06:15:16.822 INFO:tasks.workunit.client.1.smithi117.stdout:3/16: creat f5 x:0 0 0 2024-03-22T06:15:16.822 INFO:tasks.workunit.client.1.smithi117.stdout:3/17: truncate f5 145220 0 2024-03-22T06:15:16.823 INFO:tasks.workunit.client.1.smithi117.stdout:0/69: mkdir d1/d7/de/d14 0 2024-03-22T06:15:16.823 INFO:tasks.workunit.client.1.smithi117.stdout:0/70: chown d1/d7/de/d14 0 1 2024-03-22T06:15:16.826 INFO:tasks.workunit.client.1.smithi117.stdout:3/18: unlink c1 0 2024-03-22T06:15:16.826 INFO:tasks.workunit.client.1.smithi117.stdout:5/11: rename f3 to f4 0 2024-03-22T06:15:16.827 INFO:tasks.workunit.client.1.smithi117.stdout:3/19: rename f5 to f6 0 2024-03-22T06:15:16.827 INFO:tasks.workunit.client.1.smithi117.stdout:3/20: creat f7 x:0 0 0 2024-03-22T06:15:16.827 INFO:tasks.workunit.client.1.smithi117.stdout:3/21: truncate f6 964995 0 2024-03-22T06:15:16.827 INFO:tasks.workunit.client.1.smithi117.stdout:3/22: creat f8 x:0 0 0 2024-03-22T06:15:16.829 INFO:tasks.workunit.client.1.smithi117.stdout:3/23: mkdir d9 0 2024-03-22T06:15:16.852 INFO:tasks.workunit.client.1.smithi117.stdout:8/71: dwrite d1/d2/d9/f13 [0,4194304] 0 2024-03-22T06:15:16.854 INFO:tasks.workunit.client.1.smithi117.stdout:8/72: mknod d1/d2/c1b 0 2024-03-22T06:15:16.855 INFO:tasks.workunit.client.1.smithi117.stdout:9/80: dwrite d4/d7/fd [0,4194304] 0 2024-03-22T06:15:16.855 INFO:tasks.workunit.client.1.smithi117.stdout:9/81: fdatasync d4/d8/f13 0 2024-03-22T06:15:16.855 INFO:tasks.workunit.client.1.smithi117.stdout:0/71: dwrite d1/d3/f12 [0,4194304] 0 2024-03-22T06:15:16.855 INFO:tasks.workunit.client.1.smithi117.stdout:0/72: stat d1/d3 0 2024-03-22T06:15:16.856 INFO:tasks.workunit.client.0.smithi006.stdout:2/972: dwrite d13/d34/d4d/d4e/d75/da5/d82/dc3/ff6 [0,4194304] 0 2024-03-22T06:15:16.856 INFO:tasks.workunit.client.0.smithi006.stdout:2/973: write d13/d53/fc8 [582349,60102] 0 2024-03-22T06:15:16.856 INFO:tasks.workunit.client.1.smithi117.stdout:2/61: sync 2024-03-22T06:15:16.856 INFO:tasks.workunit.client.1.smithi117.stdout:4/85: sync 2024-03-22T06:15:16.857 INFO:tasks.workunit.client.1.smithi117.stdout:1/58: sync 2024-03-22T06:15:16.857 INFO:tasks.workunit.client.1.smithi117.stdout:6/47: sync 2024-03-22T06:15:16.857 INFO:tasks.workunit.client.1.smithi117.stdout:6/48: dread - d3/d7/fd zero size 2024-03-22T06:15:16.857 INFO:tasks.workunit.client.1.smithi117.stdout:6/49: chown d3/l9 770695 1 2024-03-22T06:15:16.857 INFO:tasks.workunit.client.1.smithi117.stdout:7/57: sync 2024-03-22T06:15:16.857 INFO:tasks.workunit.client.1.smithi117.stdout:1/59: creat d1/ff x:0 0 0 2024-03-22T06:15:16.857 INFO:tasks.workunit.client.1.smithi117.stdout:7/58: readlink l1 0 2024-03-22T06:15:16.857 INFO:tasks.workunit.client.1.smithi117.stdout:7/59: stat l1 0 2024-03-22T06:15:16.862 INFO:tasks.workunit.client.1.smithi117.stdout:1/60: dread d1/fe [0,4194304] 0 2024-03-22T06:15:16.865 INFO:tasks.workunit.client.1.smithi117.stdout:5/12: getdents . 0 2024-03-22T06:15:16.869 INFO:tasks.workunit.client.1.smithi117.stdout:4/86: write d0/f11 [426449,19879] 0 2024-03-22T06:15:16.870 INFO:tasks.workunit.client.1.smithi117.stdout:4/87: chown d0/d4/cd 130902 1 2024-03-22T06:15:16.871 INFO:tasks.workunit.client.1.smithi117.stdout:8/73: write f0 [754933,14303] 0 2024-03-22T06:15:16.871 INFO:tasks.workunit.client.1.smithi117.stdout:1/61: write d1/fd [156560,23022] 0 2024-03-22T06:15:16.871 INFO:tasks.workunit.client.1.smithi117.stdout:1/62: write d1/fc [432165,45353] 0 2024-03-22T06:15:16.872 INFO:tasks.workunit.client.1.smithi117.stdout:0/73: symlink d1/l15 0 2024-03-22T06:15:16.874 INFO:tasks.workunit.client.1.smithi117.stdout:6/50: mkdir d3/d7/da/de 0 2024-03-22T06:15:16.875 INFO:tasks.workunit.client.1.smithi117.stdout:7/60: unlink f4 0 2024-03-22T06:15:16.877 INFO:tasks.workunit.client.1.smithi117.stdout:3/24: dwrite f2 [0,4194304] 0 2024-03-22T06:15:16.879 INFO:tasks.workunit.client.1.smithi117.stdout:1/63: dread d1/fd [0,4194304] 0 2024-03-22T06:15:16.880 INFO:tasks.workunit.client.1.smithi117.stdout:1/64: truncate d1/fd 1134401 0 2024-03-22T06:15:16.883 INFO:tasks.workunit.client.1.smithi117.stdout:8/74: unlink d1/f1a 0 2024-03-22T06:15:16.884 INFO:tasks.workunit.client.1.smithi117.stdout:0/74: mkdir d1/d7/de/d16 0 2024-03-22T06:15:16.885 INFO:tasks.workunit.client.1.smithi117.stdout:0/75: write d1/d10/f11 [227860,109721] 0 2024-03-22T06:15:16.885 INFO:tasks.workunit.client.0.smithi006.stdout:8/932: sync 2024-03-22T06:15:16.887 INFO:tasks.workunit.client.1.smithi117.stdout:3/25: dread - f8 zero size 2024-03-22T06:15:16.888 INFO:tasks.workunit.client.1.smithi117.stdout:1/65: symlink d1/l10 0 2024-03-22T06:15:16.888 INFO:tasks.workunit.client.1.smithi117.stdout:4/88: rmdir d0 39 2024-03-22T06:15:16.888 INFO:tasks.workunit.client.1.smithi117.stdout:4/89: fsync d0/f3 0 2024-03-22T06:15:16.889 INFO:tasks.workunit.client.1.smithi117.stdout:0/76: rmdir d1/d7/de/d16 0 2024-03-22T06:15:16.890 INFO:tasks.workunit.client.1.smithi117.stdout:6/51: mkdir d3/df 0 2024-03-22T06:15:16.891 INFO:tasks.workunit.client.1.smithi117.stdout:7/61: truncate f7 6435660 0 2024-03-22T06:15:16.893 INFO:tasks.workunit.client.1.smithi117.stdout:1/66: mknod d1/c11 0 2024-03-22T06:15:16.894 INFO:tasks.workunit.client.1.smithi117.stdout:0/77: symlink d1/d7/l17 0 2024-03-22T06:15:16.895 INFO:tasks.workunit.client.1.smithi117.stdout:6/52: mknod d3/d7/da/c10 0 2024-03-22T06:15:16.896 INFO:tasks.workunit.client.1.smithi117.stdout:7/62: link d8/d9/le d8/d9/l12 0 2024-03-22T06:15:16.896 INFO:tasks.workunit.client.1.smithi117.stdout:7/63: fdatasync f5 0 2024-03-22T06:15:16.896 INFO:tasks.workunit.client.1.smithi117.stdout:9/82: dwrite f2 [0,4194304] 0 2024-03-22T06:15:16.896 INFO:tasks.workunit.client.1.smithi117.stdout:7/64: read f5 [2151879,82258] 0 2024-03-22T06:15:16.896 INFO:tasks.workunit.client.1.smithi117.stdout:8/75: dread d1/d2/fa [0,4194304] 0 2024-03-22T06:15:16.897 INFO:tasks.workunit.client.0.smithi006.stdout:8/933: read d9/db/d35/da9/fc8 [2151300,8695] 0 2024-03-22T06:15:16.897 INFO:tasks.workunit.client.1.smithi117.stdout:3/26: creat d9/fa x:0 0 0 2024-03-22T06:15:16.899 INFO:tasks.workunit.client.1.smithi117.stdout:1/67: rename d1/cb to d1/c12 0 2024-03-22T06:15:16.899 INFO:tasks.workunit.client.1.smithi117.stdout:1/68: creat d1/f13 x:0 0 0 2024-03-22T06:15:16.899 INFO:tasks.workunit.client.1.smithi117.stdout:5/13: dwrite f4 [0,4194304] 0 2024-03-22T06:15:16.899 INFO:tasks.workunit.client.1.smithi117.stdout:2/62: dwrite f1 [0,4194304] 0 2024-03-22T06:15:16.900 INFO:tasks.workunit.client.1.smithi117.stdout:2/63: write d2/fc [974981,96] 0 2024-03-22T06:15:16.901 INFO:tasks.workunit.client.1.smithi117.stdout:0/78: mkdir d1/d10/d18 0 2024-03-22T06:15:16.901 INFO:tasks.workunit.client.1.smithi117.stdout:0/79: chown d1/d10/d18 117716 1 2024-03-22T06:15:16.901 INFO:tasks.workunit.client.1.smithi117.stdout:0/80: fdatasync d1/d3/f12 0 2024-03-22T06:15:16.901 INFO:tasks.workunit.client.1.smithi117.stdout:4/90: write d0/d4/f17 [782226,57204] 0 2024-03-22T06:15:16.902 INFO:tasks.workunit.client.0.smithi006.stdout:2/974: dwrite d13/d34/d4d/fdb [0,4194304] 0 2024-03-22T06:15:16.902 INFO:tasks.workunit.client.0.smithi006.stdout:2/975: chown d13/d53/d6b/dd8/d131/l134 240409558 1 2024-03-22T06:15:16.906 INFO:tasks.workunit.client.0.smithi006.stdout:8/934: unlink d9/db/d12/d66/d10c/d122/f138 0 2024-03-22T06:15:16.907 INFO:tasks.workunit.client.1.smithi117.stdout:9/83: mknod d4/d7/d10/c1f 0 2024-03-22T06:15:16.912 INFO:tasks.workunit.client.0.smithi006.stdout:2/976: mkdir d13/d20/d26/d19/d99/d16c 0 2024-03-22T06:15:16.912 INFO:tasks.workunit.client.0.smithi006.stdout:2/977: chown d13/d34/d4d/d4e/d75/c3b 1 1 2024-03-22T06:15:16.912 INFO:tasks.workunit.client.0.smithi006.stdout:2/978: write d13/d53/dc5/fde [428038,128624] 0 2024-03-22T06:15:16.913 INFO:tasks.workunit.client.0.smithi006.stdout:8/935: creat d9/db/d12/d66/d6c/f152 x:0 0 0 2024-03-22T06:15:16.914 INFO:tasks.workunit.client.1.smithi117.stdout:1/69: mkdir d1/d14 0 2024-03-22T06:15:16.915 INFO:tasks.workunit.client.1.smithi117.stdout:2/64: dread f1 [0,4194304] 0 2024-03-22T06:15:16.915 INFO:tasks.workunit.client.1.smithi117.stdout:2/65: fsync d2/f4 0 2024-03-22T06:15:16.916 INFO:tasks.workunit.client.0.smithi006.stdout:8/936: mkdir d9/db/d12/d66/d6c/d92/d153 0 2024-03-22T06:15:16.916 INFO:tasks.workunit.client.1.smithi117.stdout:8/76: dread d1/d2/f7 [0,4194304] 0 2024-03-22T06:15:16.918 INFO:tasks.workunit.client.1.smithi117.stdout:6/53: dwrite d3/d7/da/fc [0,4194304] 0 2024-03-22T06:15:16.918 INFO:tasks.workunit.client.1.smithi117.stdout:6/54: creat d3/d7/f11 x:0 0 0 2024-03-22T06:15:16.918 INFO:tasks.workunit.client.1.smithi117.stdout:7/65: dwrite d8/d9/ff [0,4194304] 0 2024-03-22T06:15:16.918 INFO:tasks.workunit.client.1.smithi117.stdout:5/14: rename l0 to l5 0 2024-03-22T06:15:16.919 INFO:tasks.workunit.client.0.smithi006.stdout:8/937: truncate d9/db/d12/d1e/f51 1656225 0 2024-03-22T06:15:16.922 INFO:tasks.workunit.client.1.smithi117.stdout:0/81: creat d1/f19 x:0 0 0 2024-03-22T06:15:16.922 INFO:tasks.workunit.client.1.smithi117.stdout:0/82: chown d1/c2 44 1 2024-03-22T06:15:16.922 INFO:tasks.workunit.client.1.smithi117.stdout:0/83: truncate d1/d7/ff 875422 0 2024-03-22T06:15:16.923 INFO:tasks.workunit.client.0.smithi006.stdout:8/938: creat d9/db/d12/d66/d10c/d125/da3/f154 x:0 0 0 2024-03-22T06:15:16.924 INFO:tasks.workunit.client.0.smithi006.stdout:8/939: dread - d9/db/d35/dd2/d76/db3/d132/f97 zero size 2024-03-22T06:15:16.925 INFO:tasks.workunit.client.1.smithi117.stdout:4/91: creat d0/f1d x:0 0 0 2024-03-22T06:15:16.925 INFO:tasks.workunit.client.1.smithi117.stdout:4/92: write d0/f3 [5093286,63259] 0 2024-03-22T06:15:16.926 INFO:tasks.workunit.client.0.smithi006.stdout:2/979: dread d13/d53/dc5/fd7 [4194304,4194304] 0 2024-03-22T06:15:16.926 INFO:tasks.workunit.client.0.smithi006.stdout:2/980: truncate d13/d53/d6b/f7e 535651 0 2024-03-22T06:15:16.928 INFO:tasks.workunit.client.1.smithi117.stdout:9/84: link d4/d7/d10/c1f d4/d7/c20 0 2024-03-22T06:15:16.929 INFO:tasks.workunit.client.0.smithi006.stdout:2/981: unlink d13/l44 0 2024-03-22T06:15:16.929 INFO:tasks.workunit.client.0.smithi006.stdout:2/982: creat d13/d20/da6/dba/f16d x:0 0 0 2024-03-22T06:15:16.930 INFO:tasks.workunit.client.1.smithi117.stdout:0/84: fdatasync d1/d10/f11 0 2024-03-22T06:15:16.931 INFO:tasks.workunit.client.1.smithi117.stdout:0/85: rename d1/d8 to d1/d8/dc/d1a 22 2024-03-22T06:15:16.931 INFO:tasks.workunit.client.1.smithi117.stdout:7/66: dread d8/d9/ff [0,4194304] 0 2024-03-22T06:15:16.931 INFO:tasks.workunit.client.1.smithi117.stdout:7/67: creat d8/d9/da/d10/f13 x:0 0 0 2024-03-22T06:15:16.931 INFO:tasks.workunit.client.1.smithi117.stdout:7/68: chown d8/d9/fd 499214243 1 2024-03-22T06:15:16.931 INFO:tasks.workunit.client.1.smithi117.stdout:7/69: creat d8/d9/f14 x:0 0 0 2024-03-22T06:15:16.931 INFO:tasks.workunit.client.1.smithi117.stdout:7/70: readlink l1 0 2024-03-22T06:15:16.931 INFO:tasks.workunit.client.1.smithi117.stdout:1/70: rename d1/ff to d1/f15 0 2024-03-22T06:15:16.932 INFO:tasks.workunit.client.1.smithi117.stdout:1/71: chown d1/f8 261002980 1 2024-03-22T06:15:16.932 INFO:tasks.workunit.client.1.smithi117.stdout:1/72: fsync d1/fe 0 2024-03-22T06:15:16.932 INFO:tasks.workunit.client.1.smithi117.stdout:1/73: dread - d1/f13 zero size 2024-03-22T06:15:16.932 INFO:tasks.workunit.client.1.smithi117.stdout:1/74: stat d1/fd 0 2024-03-22T06:15:16.932 INFO:tasks.workunit.client.1.smithi117.stdout:1/75: chown d1/f8 778 1 2024-03-22T06:15:16.933 INFO:tasks.workunit.client.0.smithi006.stdout:8/940: rename d9/db/d12/d66/f7f to d9/db/f155 0 2024-03-22T06:15:16.933 INFO:tasks.workunit.client.0.smithi006.stdout:8/941: chown d9/db/dc/de/d3e/c91 183571184 1 2024-03-22T06:15:16.933 INFO:tasks.workunit.client.0.smithi006.stdout:8/942: truncate d9/db/d12/d66/d6c/d71/f101 1537524 0 2024-03-22T06:15:16.933 INFO:tasks.workunit.client.0.smithi006.stdout:8/943: creat d9/db/d35/dd2/d76/db3/d132/f156 x:0 0 0 2024-03-22T06:15:16.933 INFO:tasks.workunit.client.1.smithi117.stdout:2/66: mkdir d2/d5/d8/d10/d11 0 2024-03-22T06:15:16.935 INFO:tasks.workunit.client.1.smithi117.stdout:8/77: link d1/l8 d1/d19/l1c 0 2024-03-22T06:15:16.937 INFO:tasks.workunit.client.0.smithi006.stdout:2/983: rename d13/d20/d26/dca/lf1 to d13/d53/d6b/dd8/l16e 0 2024-03-22T06:15:16.937 INFO:tasks.workunit.client.0.smithi006.stdout:2/984: stat d13/d34/d4d/d4e/d75/da5/f47 0 2024-03-22T06:15:16.942 INFO:tasks.workunit.client.0.smithi006.stdout:8/944: symlink d9/db/dc/de/d3e/l157 0 2024-03-22T06:15:16.946 INFO:tasks.workunit.client.1.smithi117.stdout:6/55: mknod d3/df/c12 0 2024-03-22T06:15:16.946 INFO:tasks.workunit.client.1.smithi117.stdout:6/56: readlink d3/l9 0 2024-03-22T06:15:16.946 INFO:tasks.workunit.client.1.smithi117.stdout:6/57: chown d3/df 64517 1 2024-03-22T06:15:16.948 INFO:tasks.workunit.client.1.smithi117.stdout:5/15: creat f6 x:0 0 0 2024-03-22T06:15:16.948 INFO:tasks.workunit.client.1.smithi117.stdout:5/16: creat f7 x:0 0 0 2024-03-22T06:15:16.949 INFO:tasks.workunit.client.1.smithi117.stdout:4/93: mknod d0/c1e 0 2024-03-22T06:15:16.952 INFO:tasks.workunit.client.1.smithi117.stdout:2/67: read d2/f3 [1957040,111133] 0 2024-03-22T06:15:16.953 INFO:tasks.workunit.client.0.smithi006.stdout:2/985: mkdir d13/d20/d16f 0 2024-03-22T06:15:16.953 INFO:tasks.workunit.client.1.smithi117.stdout:9/85: dwrite d4/d7/f16 [0,4194304] 0 2024-03-22T06:15:16.953 INFO:tasks.workunit.client.1.smithi117.stdout:9/86: read - d4/d8/f13 zero size 2024-03-22T06:15:16.954 INFO:tasks.workunit.client.0.smithi006.stdout:8/945: write d9/d14/d47/f5c [2750240,42030] 0 2024-03-22T06:15:16.954 INFO:tasks.workunit.client.0.smithi006.stdout:8/946: write d9/db/dc/fd [190922,79466] 0 2024-03-22T06:15:16.954 INFO:tasks.workunit.client.0.smithi006.stdout:8/947: write d9/db/dc/f1f [3869004,41584] 0 2024-03-22T06:15:16.956 INFO:tasks.workunit.client.1.smithi117.stdout:7/71: symlink d8/d9/da/d10/l15 0 2024-03-22T06:15:16.960 INFO:tasks.workunit.client.0.smithi006.stdout:2/986: rmdir d13/d34/d4d/d4e/d75 39 2024-03-22T06:15:16.960 INFO:tasks.workunit.client.1.smithi117.stdout:5/17: dread f4 [0,4194304] 0 2024-03-22T06:15:16.962 INFO:tasks.workunit.client.1.smithi117.stdout:6/58: mknod d3/c13 0 2024-03-22T06:15:16.963 INFO:tasks.workunit.client.0.smithi006.stdout:8/948: mknod d9/db/d35/dd2/c158 0 2024-03-22T06:15:16.964 INFO:tasks.workunit.client.1.smithi117.stdout:3/27: dwrite f2 [0,4194304] 0 2024-03-22T06:15:16.964 INFO:tasks.workunit.client.1.smithi117.stdout:3/28: creat d9/fb x:0 0 0 2024-03-22T06:15:16.966 INFO:tasks.workunit.client.0.smithi006.stdout:8/949: truncate d9/db/dc/de/d3e/f56 1405089 0 2024-03-22T06:15:16.968 INFO:tasks.workunit.client.1.smithi117.stdout:9/87: dread f2 [0,4194304] 0 2024-03-22T06:15:16.968 INFO:tasks.workunit.client.1.smithi117.stdout:9/88: fsync d4/d7/f1c 0 2024-03-22T06:15:16.968 INFO:tasks.workunit.client.1.smithi117.stdout:8/78: dwrite d1/d2/d9/f13 [4194304,4194304] 0 2024-03-22T06:15:16.969 INFO:tasks.workunit.client.1.smithi117.stdout:0/86: dwrite d1/d3/f12 [0,4194304] 0 2024-03-22T06:15:16.973 INFO:tasks.workunit.client.0.smithi006.stdout:2/987: dread d13/d20/f42 [0,4194304] 0 2024-03-22T06:15:16.975 INFO:tasks.workunit.client.1.smithi117.stdout:0/87: dread d1/d10/f11 [0,4194304] 0 2024-03-22T06:15:16.981 INFO:tasks.workunit.client.0.smithi006.stdout:8/950: creat d9/db/d12/d66/d10c/d122/f159 x:0 0 0 2024-03-22T06:15:16.987 INFO:tasks.workunit.client.0.smithi006.stdout:2/988: getdents d13/d34/d4d/d4e/d75/da5 0 2024-03-22T06:15:16.988 INFO:tasks.workunit.client.1.smithi117.stdout:0/88: dread d1/d3/f12 [0,4194304] 0 2024-03-22T06:15:16.989 INFO:tasks.workunit.client.0.smithi006.stdout:8/951: rmdir d9/db/d35/dd2/d76/db3 39 2024-03-22T06:15:16.992 INFO:tasks.workunit.client.0.smithi006.stdout:2/989: rename d13/d34/d4d/d4e/d75/d62/f66 to d13/d53/d6b/dd8/f170 0 2024-03-22T06:15:16.994 INFO:tasks.workunit.client.1.smithi117.stdout:0/89: dread d1/d7/fb [0,4194304] 0 2024-03-22T06:15:16.996 INFO:tasks.workunit.client.1.smithi117.stdout:9/89: truncate d4/d7/fd 1088993 0 2024-03-22T06:15:16.996 INFO:tasks.workunit.client.1.smithi117.stdout:9/90: truncate d4/d8/f1a 5118522 0 2024-03-22T06:15:16.996 INFO:tasks.workunit.client.1.smithi117.stdout:9/91: chown d4/l12 40251 1 2024-03-22T06:15:16.996 INFO:tasks.workunit.client.1.smithi117.stdout:9/92: chown d4/d7/f19 2352 1 2024-03-22T06:15:16.997 INFO:tasks.workunit.client.1.smithi117.stdout:7/72: dwrite f5 [0,4194304] 0 2024-03-22T06:15:16.997 INFO:tasks.workunit.client.1.smithi117.stdout:7/73: creat d8/f16 x:0 0 0 2024-03-22T06:15:16.997 INFO:tasks.workunit.client.0.smithi006.stdout:2/990: mkdir d13/d34/d171 0 2024-03-22T06:15:16.999 INFO:tasks.workunit.client.1.smithi117.stdout:8/79: link f0 d1/d2/f1d 0 2024-03-22T06:15:17.000 INFO:tasks.workunit.client.1.smithi117.stdout:8/80: write d1/d2/d11/f12 [849697,33613] 0 2024-03-22T06:15:17.000 INFO:tasks.workunit.client.1.smithi117.stdout:8/81: creat d1/f1e x:0 0 0 2024-03-22T06:15:17.004 INFO:tasks.workunit.client.1.smithi117.stdout:5/18: getdents . 0 2024-03-22T06:15:17.004 INFO:tasks.workunit.client.1.smithi117.stdout:5/19: truncate f2 321221 0 2024-03-22T06:15:17.005 INFO:tasks.workunit.client.1.smithi117.stdout:5/20: fdatasync f2 0 2024-03-22T06:15:17.005 INFO:tasks.workunit.client.1.smithi117.stdout:6/59: dwrite d3/d7/fd [0,4194304] 0 2024-03-22T06:15:17.005 INFO:tasks.workunit.client.1.smithi117.stdout:1/76: dwrite d1/fc [0,4194304] 0 2024-03-22T06:15:17.006 INFO:tasks.workunit.client.1.smithi117.stdout:3/29: dwrite f8 [0,4194304] 0 2024-03-22T06:15:17.006 INFO:tasks.workunit.client.0.smithi006.stdout:8/952: dwrite d9/db/d35/dd2/d4a/d4d/f75 [0,4194304] 0 2024-03-22T06:15:17.006 INFO:tasks.workunit.client.0.smithi006.stdout:8/953: chown d9/db/d35/da9/df6/ff7 416131 1 2024-03-22T06:15:17.007 INFO:tasks.workunit.client.1.smithi117.stdout:7/74: write f7 [4338507,95603] 0 2024-03-22T06:15:17.007 INFO:tasks.workunit.client.0.smithi006.stdout:2/991: dread fe [0,4194304] 0 2024-03-22T06:15:17.008 INFO:tasks.workunit.client.0.smithi006.stdout:8/954: mknod d9/db/d12/d137/c15a 0 2024-03-22T06:15:17.012 INFO:tasks.workunit.client.0.smithi006.stdout:2/992: mknod d13/d20/d26/c172 0 2024-03-22T06:15:17.012 INFO:tasks.workunit.client.0.smithi006.stdout:2/993: fdatasync d13/d34/d4d/d4e/d105/f139 0 2024-03-22T06:15:17.015 INFO:tasks.workunit.client.0.smithi006.stdout:3/845: sync 2024-03-22T06:15:17.019 INFO:tasks.workunit.client.0.smithi006.stdout:2/994: link d13/d20/d35/d65/f108 d13/d34/f173 0 2024-03-22T06:15:17.019 INFO:tasks.workunit.client.0.smithi006.stdout:2/995: chown d13/d53/d6b/f97 261100491 1 2024-03-22T06:15:17.019 INFO:tasks.workunit.client.0.smithi006.stdout:2/996: stat d13/d20/d26/fab 0 2024-03-22T06:15:17.019 INFO:tasks.workunit.client.0.smithi006.stdout:2/997: chown d13/d34/d4d/d4e/d75/da5/d82/f164 969 1 2024-03-22T06:15:17.020 INFO:tasks.workunit.client.1.smithi117.stdout:5/21: symlink l8 0 2024-03-22T06:15:17.020 INFO:tasks.workunit.client.1.smithi117.stdout:5/22: dread - f6 zero size 2024-03-22T06:15:17.020 INFO:tasks.workunit.client.1.smithi117.stdout:5/23: write f7 [508683,82085] 0 2024-03-22T06:15:17.024 INFO:tasks.workunit.client.1.smithi117.stdout:6/60: mknod d3/d7/da/de/c14 0 2024-03-22T06:15:17.025 INFO:tasks.workunit.client.1.smithi117.stdout:9/93: truncate d4/d7/fd 2101151 0 2024-03-22T06:15:17.025 INFO:tasks.workunit.client.1.smithi117.stdout:9/94: truncate d4/d7/f1b 922268 0 2024-03-22T06:15:17.027 INFO:tasks.workunit.client.0.smithi006.stdout:3/846: rmdir d0/d4/dbb/df7/d5f/d90/da7/d97 39 2024-03-22T06:15:17.030 INFO:tasks.workunit.client.0.smithi006.stdout:2/998: rename d13/d20/d35/l7c to d13/d53/d6b/l174 0 2024-03-22T06:15:17.030 INFO:tasks.workunit.client.0.smithi006.stdout:2/999: dread - d13/d34/d4d/d4e/d75/da5/d6c/f151 zero size 2024-03-22T06:15:17.031 INFO:tasks.workunit.client.1.smithi117.stdout:7/75: dwrite d8/d9/da/d10/f13 [0,4194304] 0 2024-03-22T06:15:17.034 INFO:tasks.workunit.client.0.smithi006.stdout:3/847: rename d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/f35 to d0/d8/d88/dfa/d58/f120 0 2024-03-22T06:15:17.038 INFO:tasks.workunit.client.0.smithi006.stdout:3/848: link d0/d4/dbb/df7/d5f/f81 d0/d8/d88/dfa/d58/f121 0 2024-03-22T06:15:17.040 INFO:tasks.workunit.client.0.smithi006.stdout:3/849: dread d0/d8/d1c/d49/f5e [8388608,4194304] 0 2024-03-22T06:15:17.043 INFO:tasks.workunit.client.0.smithi006.stdout:3/850: getdents d0/d8/d1c 0 2024-03-22T06:15:17.043 INFO:tasks.workunit.client.0.smithi006.stdout:3/851: write d0/d4/d31/d98/f10b [569951,42011] 0 2024-03-22T06:15:17.043 INFO:tasks.workunit.client.0.smithi006.stdout:3/852: stat d0/f3 0 2024-03-22T06:15:17.045 INFO:tasks.workunit.client.1.smithi117.stdout:0/90: truncate d1/d3/f12 2720591 0 2024-03-22T06:15:17.047 INFO:tasks.workunit.client.1.smithi117.stdout:5/24: rename l8 to l9 0 2024-03-22T06:15:17.047 INFO:tasks.workunit.client.1.smithi117.stdout:5/25: chown f2 1 1 2024-03-22T06:15:17.050 INFO:tasks.workunit.client.1.smithi117.stdout:9/95: unlink d4/f17 0 2024-03-22T06:15:17.050 INFO:tasks.workunit.client.1.smithi117.stdout:1/77: getdents d1 0 2024-03-22T06:15:17.051 INFO:tasks.workunit.client.1.smithi117.stdout:1/78: chown d1/la 14333565 1 2024-03-22T06:15:17.051 INFO:tasks.workunit.client.0.smithi006.stdout:8/955: write d9/db/d35/fbb [1114200,123379] 0 2024-03-22T06:15:17.053 INFO:tasks.workunit.client.1.smithi117.stdout:3/30: rename c3 to d9/cc 0 2024-03-22T06:15:17.054 INFO:tasks.workunit.client.1.smithi117.stdout:6/61: mknod d3/c15 0 2024-03-22T06:15:17.054 INFO:tasks.workunit.client.0.smithi006.stdout:4/855: sync 2024-03-22T06:15:17.054 INFO:tasks.workunit.client.0.smithi006.stdout:7/792: sync 2024-03-22T06:15:17.057 INFO:tasks.workunit.client.0.smithi006.stdout:8/956: rename d9/db/d12/d1e/fa8 to d9/db/d35/dd2/dad/f15b 0 2024-03-22T06:15:17.060 INFO:tasks.workunit.client.1.smithi117.stdout:7/76: dwrite d8/f16 [0,4194304] 0 2024-03-22T06:15:17.060 INFO:tasks.workunit.client.1.smithi117.stdout:7/77: read - d8/d9/da/d10/f11 zero size 2024-03-22T06:15:17.061 INFO:tasks.workunit.client.1.smithi117.stdout:1/79: unlink d1/fe 0 2024-03-22T06:15:17.063 INFO:tasks.workunit.client.0.smithi006.stdout:3/853: dwrite d0/d8/d88/dfa/de2/f110 [0,4194304] 0 2024-03-22T06:15:17.065 INFO:tasks.workunit.client.0.smithi006.stdout:4/856: dread d6/d1b/d20/d21/d2a/d5c/f6c [0,4194304] 0 2024-03-22T06:15:17.065 INFO:tasks.workunit.client.0.smithi006.stdout:7/793: mknod d5/d1e/d29/d66/d91/ddc/d100/c102 0 2024-03-22T06:15:17.065 INFO:tasks.workunit.client.0.smithi006.stdout:7/794: creat d5/d1e/d29/db9/f103 x:0 0 0 2024-03-22T06:15:17.067 INFO:tasks.workunit.client.1.smithi117.stdout:0/91: rename d1/d7/l17 to d1/d10/d18/l1b 0 2024-03-22T06:15:17.067 INFO:tasks.workunit.client.1.smithi117.stdout:0/92: creat d1/f1c x:0 0 0 2024-03-22T06:15:17.069 INFO:tasks.workunit.client.1.smithi117.stdout:3/31: symlink d9/ld 0 2024-03-22T06:15:17.069 INFO:tasks.workunit.client.1.smithi117.stdout:3/32: rename d9 to d9/de 22 2024-03-22T06:15:17.069 INFO:tasks.workunit.client.1.smithi117.stdout:3/33: truncate d9/fa 840528 0 2024-03-22T06:15:17.072 INFO:tasks.workunit.client.1.smithi117.stdout:6/62: mkdir d3/d7/da/d16 0 2024-03-22T06:15:17.072 INFO:tasks.workunit.client.1.smithi117.stdout:7/78: dread d8/f16 [0,4194304] 0 2024-03-22T06:15:17.073 INFO:tasks.workunit.client.1.smithi117.stdout:2/68: sync 2024-03-22T06:15:17.075 INFO:tasks.workunit.client.1.smithi117.stdout:9/96: getdents d4 0 2024-03-22T06:15:17.075 INFO:tasks.workunit.client.1.smithi117.stdout:9/97: stat d4/d7/f1c 0 2024-03-22T06:15:17.076 INFO:tasks.workunit.client.0.smithi006.stdout:4/857: write d6/d1b/d20/d21/d2a/d43/fe4 [1853695,34696] 0 2024-03-22T06:15:17.076 INFO:tasks.workunit.client.0.smithi006.stdout:4/858: read - d6/d1b/d20/d63/d93/fdf zero size 2024-03-22T06:15:17.079 INFO:tasks.workunit.client.0.smithi006.stdout:3/854: unlink d0/d4/dbb/df7/d5f/d90/da7/de7/f25 0 2024-03-22T06:15:17.081 INFO:tasks.workunit.client.1.smithi117.stdout:4/94: sync 2024-03-22T06:15:17.082 INFO:tasks.workunit.client.0.smithi006.stdout:7/795: creat d5/d1e/d63/d85/d80/f104 x:0 0 0 2024-03-22T06:15:17.082 INFO:tasks.workunit.client.0.smithi006.stdout:8/957: dwrite d9/db/d35/da9/fc8 [0,4194304] 0 2024-03-22T06:15:17.083 INFO:tasks.workunit.client.1.smithi117.stdout:5/26: dwrite f6 [0,4194304] 0 2024-03-22T06:15:17.083 INFO:tasks.workunit.client.1.smithi117.stdout:0/93: mknod d1/d8/c1d 0 2024-03-22T06:15:17.085 INFO:tasks.workunit.client.1.smithi117.stdout:3/34: symlink d9/lf 0 2024-03-22T06:15:17.086 INFO:tasks.workunit.client.0.smithi006.stdout:4/859: mknod d6/d1b/d20/d6b/d60/db1/c129 0 2024-03-22T06:15:17.087 INFO:tasks.workunit.client.0.smithi006.stdout:4/860: fdatasync d6/d1b/d20/d21/d9f/f11b 0 2024-03-22T06:15:17.087 INFO:tasks.workunit.client.1.smithi117.stdout:7/79: read d8/d9/fd [2164616,11422] 0 2024-03-22T06:15:17.087 INFO:tasks.workunit.client.1.smithi117.stdout:6/63: rmdir d3/d7/da/de 39 2024-03-22T06:15:17.088 INFO:tasks.workunit.client.0.smithi006.stdout:3/855: mknod d0/d4/dbb/df7/d5f/d83/c122 0 2024-03-22T06:15:17.089 INFO:tasks.workunit.client.1.smithi117.stdout:5/27: fsync f7 0 2024-03-22T06:15:17.089 INFO:tasks.workunit.client.1.smithi117.stdout:5/28: fdatasync f4 0 2024-03-22T06:15:17.094 INFO:tasks.workunit.client.0.smithi006.stdout:8/958: dread d9/db/d35/f3c [0,4194304] 0 2024-03-22T06:15:17.094 INFO:tasks.workunit.client.1.smithi117.stdout:4/95: write d0/f3 [645607,98700] 0 2024-03-22T06:15:17.094 INFO:tasks.workunit.client.1.smithi117.stdout:4/96: write d0/d4/f12 [5705732,88778] 0 2024-03-22T06:15:17.094 INFO:tasks.workunit.client.1.smithi117.stdout:9/98: creat d4/f21 x:0 0 0 2024-03-22T06:15:17.094 INFO:tasks.workunit.client.1.smithi117.stdout:1/80: mkdir d1/d14/d16 0 2024-03-22T06:15:17.094 INFO:tasks.workunit.client.1.smithi117.stdout:2/69: dwrite d2/fc [0,4194304] 0 2024-03-22T06:15:17.095 INFO:tasks.workunit.client.0.smithi006.stdout:7/796: truncate d5/d1e/d29/d1f/d2d/d61/daa/fc0 3049859 0 2024-03-22T06:15:17.096 INFO:tasks.workunit.client.0.smithi006.stdout:4/861: dread d6/d1b/d20/d21/d77/fa6 [0,4194304] 0 2024-03-22T06:15:17.099 INFO:tasks.workunit.client.0.smithi006.stdout:3/856: mknod d0/d8/d9/c123 0 2024-03-22T06:15:17.099 INFO:tasks.workunit.client.1.smithi117.stdout:8/82: sync 2024-03-22T06:15:17.100 INFO:tasks.workunit.client.1.smithi117.stdout:0/94: symlink d1/d10/d18/l1e 0 2024-03-22T06:15:17.100 INFO:tasks.workunit.client.1.smithi117.stdout:5/29: dread f6 [0,4194304] 0 2024-03-22T06:15:17.101 INFO:tasks.workunit.client.0.smithi006.stdout:8/959: mkdir d9/db/dc/de/d3e/d15c 0 2024-03-22T06:15:17.101 INFO:tasks.workunit.client.0.smithi006.stdout:8/960: chown d9/d5e/l140 12 1 2024-03-22T06:15:17.102 INFO:tasks.workunit.client.0.smithi006.stdout:8/961: read d9/db/d35/fbb [2007081,103544] 0 2024-03-22T06:15:17.103 INFO:tasks.workunit.client.1.smithi117.stdout:5/30: write f4 [1553921,127948] 0 2024-03-22T06:15:17.103 INFO:tasks.workunit.client.1.smithi117.stdout:6/64: mkdir d3/df/d17 0 2024-03-22T06:15:17.104 INFO:tasks.workunit.client.1.smithi117.stdout:2/70: dread f1 [4194304,4194304] 0 2024-03-22T06:15:17.104 INFO:tasks.workunit.client.1.smithi117.stdout:2/71: readlink d2/d5/db/lf 0 2024-03-22T06:15:17.106 INFO:tasks.workunit.client.0.smithi006.stdout:4/862: mknod d6/d1b/d20/d6b/d7b/c12a 0 2024-03-22T06:15:17.106 INFO:tasks.workunit.client.0.smithi006.stdout:4/863: fdatasync d6/f4a 0 2024-03-22T06:15:17.106 INFO:tasks.workunit.client.0.smithi006.stdout:4/864: chown d6/d1b/d20/d21/d2a/d43/d4b/lb7 142604 1 2024-03-22T06:15:17.107 INFO:tasks.workunit.client.0.smithi006.stdout:3/857: getdents d0/d4/dbb/df7/d5f/d83/d8c 0 2024-03-22T06:15:17.107 INFO:tasks.workunit.client.1.smithi117.stdout:9/99: dread d4/d8/f1a [4194304,4194304] 0 2024-03-22T06:15:17.109 INFO:tasks.workunit.client.1.smithi117.stdout:4/97: symlink d0/l1f 0 2024-03-22T06:15:17.112 INFO:tasks.workunit.client.1.smithi117.stdout:6/65: write d3/d7/fd [515088,101752] 0 2024-03-22T06:15:17.112 INFO:tasks.workunit.client.0.smithi006.stdout:8/962: dread d9/db/d12/f90 [0,4194304] 0 2024-03-22T06:15:17.114 INFO:tasks.workunit.client.1.smithi117.stdout:1/81: symlink d1/d14/d16/l17 0 2024-03-22T06:15:17.117 INFO:tasks.workunit.client.0.smithi006.stdout:4/865: rename d6/d1b/d20/d63/d93/f102 to d6/d1b/d20/d21/f12b 0 2024-03-22T06:15:17.118 INFO:tasks.workunit.client.1.smithi117.stdout:4/98: dread d0/f11 [0,4194304] 0 2024-03-22T06:15:17.120 INFO:tasks.workunit.client.0.smithi006.stdout:3/858: getdents d0/d8 0 2024-03-22T06:15:17.122 INFO:tasks.workunit.client.0.smithi006.stdout:8/963: mkdir d9/db/d35/dd2/d76/dca/d15d 0 2024-03-22T06:15:17.124 INFO:tasks.workunit.client.1.smithi117.stdout:8/83: mkdir d1/d1f 0 2024-03-22T06:15:17.124 INFO:tasks.workunit.client.1.smithi117.stdout:8/84: readlink d1/d2/l10 0 2024-03-22T06:15:17.124 INFO:tasks.workunit.client.1.smithi117.stdout:0/95: mkdir d1/d7/de/d1f 0 2024-03-22T06:15:17.127 INFO:tasks.workunit.client.1.smithi117.stdout:4/99: write d0/d4/fb [552090,129805] 0 2024-03-22T06:15:17.129 INFO:tasks.workunit.client.1.smithi117.stdout:7/80: rmdir d8/d9/da/d10 39 2024-03-22T06:15:17.131 INFO:tasks.workunit.client.1.smithi117.stdout:5/31: creat fa x:0 0 0 2024-03-22T06:15:17.134 INFO:tasks.workunit.client.0.smithi006.stdout:4/866: dwrite d6/d1b/d20/fe6 [0,4194304] 0 2024-03-22T06:15:17.134 INFO:tasks.workunit.client.1.smithi117.stdout:8/85: dread d1/d2/fa [0,4194304] 0 2024-03-22T06:15:17.135 INFO:tasks.workunit.client.1.smithi117.stdout:2/72: symlink d2/l12 0 2024-03-22T06:15:17.135 INFO:tasks.workunit.client.1.smithi117.stdout:2/73: chown d2/d5/db/lf 437 1 2024-03-22T06:15:17.139 INFO:tasks.workunit.client.0.smithi006.stdout:3/859: rmdir d0/d8/d88/ddd 0 2024-03-22T06:15:17.139 INFO:tasks.workunit.client.0.smithi006.stdout:3/860: readlink d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/ldc 0 2024-03-22T06:15:17.139 INFO:tasks.workunit.client.0.smithi006.stdout:3/861: truncate d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/f11c 964850 0 2024-03-22T06:15:17.143 INFO:tasks.workunit.client.1.smithi117.stdout:6/66: symlink d3/d7/l18 0 2024-03-22T06:15:17.144 INFO:tasks.workunit.client.0.smithi006.stdout:3/862: creat d0/f124 x:0 0 0 2024-03-22T06:15:17.144 INFO:tasks.workunit.client.0.smithi006.stdout:3/863: write d0/d4/dbb/df7/d5f/d90/da7/de7/d33/f4d [2927049,18954] 0 2024-03-22T06:15:17.144 INFO:tasks.workunit.client.0.smithi006.stdout:3/864: fsync d0/d4/dbb/df7/d5f/d90/da7/de7/d33/f44 0 2024-03-22T06:15:17.145 INFO:tasks.workunit.client.1.smithi117.stdout:8/86: dread d1/d2/d9/f13 [4194304,4194304] 0 2024-03-22T06:15:17.149 INFO:tasks.workunit.client.0.smithi006.stdout:3/865: creat d0/d4/dbb/df7/f125 x:0 0 0 2024-03-22T06:15:17.150 INFO:tasks.workunit.client.1.smithi117.stdout:5/32: rename c1 to cb 0 2024-03-22T06:15:17.152 INFO:tasks.workunit.client.1.smithi117.stdout:6/67: creat d3/d7/da/de/f19 x:0 0 0 2024-03-22T06:15:17.154 INFO:tasks.workunit.client.0.smithi006.stdout:3/866: rename d0/d4/d31/l9e to d0/d8/d1c/l126 0 2024-03-22T06:15:17.155 INFO:tasks.workunit.client.1.smithi117.stdout:1/82: symlink d1/l18 0 2024-03-22T06:15:17.155 INFO:tasks.workunit.client.1.smithi117.stdout:7/81: rmdir d8/d9 39 2024-03-22T06:15:17.156 INFO:tasks.workunit.client.1.smithi117.stdout:9/100: dread d4/d7/f16 [0,4194304] 0 2024-03-22T06:15:17.158 INFO:tasks.workunit.client.1.smithi117.stdout:1/83: creat d1/d14/f19 x:0 0 0 2024-03-22T06:15:17.159 INFO:tasks.workunit.client.0.smithi006.stdout:8/964: dwrite d9/db/f10e [0,4194304] 0 2024-03-22T06:15:17.159 INFO:tasks.workunit.client.1.smithi117.stdout:7/82: rmdir d8/d9 39 2024-03-22T06:15:17.160 INFO:tasks.workunit.client.1.smithi117.stdout:0/96: dwrite d1/d7/fb [0,4194304] 0 2024-03-22T06:15:17.162 INFO:tasks.workunit.client.1.smithi117.stdout:8/87: dwrite d1/d2/d9/f18 [0,4194304] 0 2024-03-22T06:15:17.166 INFO:tasks.workunit.client.0.smithi006.stdout:8/965: mkdir d9/db/d12/d66/d10c/d125/da3/d15e 0 2024-03-22T06:15:17.166 INFO:tasks.workunit.client.0.smithi006.stdout:8/966: readlink d9/db/d35/dd2/d76/dca/lec 0 2024-03-22T06:15:17.169 INFO:tasks.workunit.client.1.smithi117.stdout:6/68: dread d3/d7/da/fc [0,4194304] 0 2024-03-22T06:15:17.171 INFO:tasks.workunit.client.1.smithi117.stdout:8/88: unlink d1/d2/d11/f12 0 2024-03-22T06:15:17.174 INFO:tasks.workunit.client.0.smithi006.stdout:8/967: dread d9/d14/dcc/f11c [0,4194304] 0 2024-03-22T06:15:17.174 INFO:tasks.workunit.client.1.smithi117.stdout:1/84: dread d1/fc [0,4194304] 0 2024-03-22T06:15:17.174 INFO:tasks.workunit.client.1.smithi117.stdout:6/69: symlink d3/df/d17/l1a 0 2024-03-22T06:15:17.174 INFO:tasks.workunit.client.1.smithi117.stdout:6/70: fsync d3/f5 0 2024-03-22T06:15:17.174 INFO:tasks.workunit.client.1.smithi117.stdout:6/71: creat d3/d7/f1b x:0 0 0 2024-03-22T06:15:17.174 INFO:tasks.workunit.client.1.smithi117.stdout:6/72: fdatasync d3/d7/fd 0 2024-03-22T06:15:17.176 INFO:tasks.workunit.client.1.smithi117.stdout:7/83: dread f6 [0,4194304] 0 2024-03-22T06:15:17.176 INFO:tasks.workunit.client.1.smithi117.stdout:8/89: mknod d1/d2/d9/c20 0 2024-03-22T06:15:17.176 INFO:tasks.workunit.client.1.smithi117.stdout:8/90: dread - d1/f1e zero size 2024-03-22T06:15:17.176 INFO:tasks.workunit.client.1.smithi117.stdout:0/97: read d1/d7/fb [1563142,62256] 0 2024-03-22T06:15:17.176 INFO:tasks.workunit.client.1.smithi117.stdout:0/98: chown d1/c9 268282513 1 2024-03-22T06:15:17.178 INFO:tasks.workunit.client.1.smithi117.stdout:6/73: symlink d3/d7/da/de/l1c 0 2024-03-22T06:15:17.180 INFO:tasks.workunit.client.1.smithi117.stdout:8/91: mknod d1/d2/d16/c21 0 2024-03-22T06:15:17.180 INFO:tasks.workunit.client.1.smithi117.stdout:8/92: write d1/d2/f1d [4914944,111970] 0 2024-03-22T06:15:17.180 INFO:tasks.workunit.client.1.smithi117.stdout:8/93: write d1/f1e [841221,30617] 0 2024-03-22T06:15:17.180 INFO:tasks.workunit.client.1.smithi117.stdout:0/99: truncate d1/d3/f12 473991 0 2024-03-22T06:15:17.181 INFO:tasks.workunit.client.1.smithi117.stdout:0/100: write d1/d10/f11 [74259,117971] 0 2024-03-22T06:15:17.181 INFO:tasks.workunit.client.1.smithi117.stdout:1/85: rename d1/f15 to d1/d14/d16/f1a 0 2024-03-22T06:15:17.182 INFO:tasks.workunit.client.1.smithi117.stdout:8/94: stat d1/d19/l1c 0 2024-03-22T06:15:17.183 INFO:tasks.workunit.client.1.smithi117.stdout:0/101: symlink d1/d3/l20 0 2024-03-22T06:15:17.184 INFO:tasks.workunit.client.1.smithi117.stdout:8/95: mknod d1/c22 0 2024-03-22T06:15:17.185 INFO:tasks.workunit.client.1.smithi117.stdout:6/74: dread d3/d7/fd [0,4194304] 0 2024-03-22T06:15:17.186 INFO:tasks.workunit.client.1.smithi117.stdout:6/75: truncate d3/f4 1148040 0 2024-03-22T06:15:17.187 INFO:tasks.workunit.client.1.smithi117.stdout:0/102: dread d1/d7/fb [0,4194304] 0 2024-03-22T06:15:17.189 INFO:tasks.workunit.client.1.smithi117.stdout:0/103: link d1/c9 d1/d10/c21 0 2024-03-22T06:15:17.190 INFO:tasks.workunit.client.1.smithi117.stdout:6/76: rename d3/d7/da/de/l1c to d3/l1d 0 2024-03-22T06:15:17.191 INFO:tasks.workunit.client.0.smithi006.stdout:8/968: dread d9/db/f7b [0,4194304] 0 2024-03-22T06:15:17.192 INFO:tasks.workunit.client.0.smithi006.stdout:8/969: write d9/db/d12/d1e/f6d [1041917,8604] 0 2024-03-22T06:15:17.192 INFO:tasks.workunit.client.1.smithi117.stdout:6/77: symlink d3/d7/da/l1e 0 2024-03-22T06:15:17.194 INFO:tasks.workunit.client.0.smithi006.stdout:8/970: link d9/db/d35/dd2/d76/db3/d132/f136 d9/db/d12/d66/d6c/d80/df0/f15f 0 2024-03-22T06:15:17.194 INFO:tasks.workunit.client.0.smithi006.stdout:8/971: chown d9/db/d12/d66/d10c/d125/da3/fc0 0 1 2024-03-22T06:15:17.195 INFO:tasks.workunit.client.1.smithi117.stdout:6/78: creat d3/f1f x:0 0 0 2024-03-22T06:15:17.197 INFO:tasks.workunit.client.1.smithi117.stdout:6/79: creat d3/f20 x:0 0 0 2024-03-22T06:15:17.197 INFO:tasks.workunit.client.1.smithi117.stdout:6/80: chown d3/d7/da/fc 301 1 2024-03-22T06:15:17.197 INFO:tasks.workunit.client.1.smithi117.stdout:6/81: write d3/f5 [5266641,77465] 0 2024-03-22T06:15:17.197 INFO:tasks.workunit.client.1.smithi117.stdout:6/82: readlink d3/d7/l8 0 2024-03-22T06:15:17.197 INFO:tasks.workunit.client.1.smithi117.stdout:8/96: write d1/d2/d9/f13 [5359774,51496] 0 2024-03-22T06:15:17.198 INFO:tasks.workunit.client.1.smithi117.stdout:6/83: symlink d3/d7/l21 0 2024-03-22T06:15:17.199 INFO:tasks.workunit.client.1.smithi117.stdout:4/100: dwrite d0/f3 [4194304,4194304] 0 2024-03-22T06:15:17.199 INFO:tasks.workunit.client.1.smithi117.stdout:4/101: write d0/d19/f1a [118811,52701] 0 2024-03-22T06:15:17.199 INFO:tasks.workunit.client.1.smithi117.stdout:6/84: creat d3/d7/da/d16/f22 x:0 0 0 2024-03-22T06:15:17.199 INFO:tasks.workunit.client.1.smithi117.stdout:4/102: chown d0/d19 40 1 2024-03-22T06:15:17.200 INFO:tasks.workunit.client.1.smithi117.stdout:4/103: chown d0/f11 26 1 2024-03-22T06:15:17.200 INFO:tasks.workunit.client.1.smithi117.stdout:6/85: chown d3/d7/f1b 0 1 2024-03-22T06:15:17.200 INFO:tasks.workunit.client.1.smithi117.stdout:6/86: chown d3/df/d17 3228 1 2024-03-22T06:15:17.200 INFO:tasks.workunit.client.1.smithi117.stdout:6/87: creat d3/d7/da/de/f23 x:0 0 0 2024-03-22T06:15:17.200 INFO:tasks.workunit.client.1.smithi117.stdout:6/88: readlink d3/l1d 0 2024-03-22T06:15:17.200 INFO:tasks.workunit.client.1.smithi117.stdout:6/89: write d3/d7/fd [4495398,3073] 0 2024-03-22T06:15:17.201 INFO:tasks.workunit.client.1.smithi117.stdout:4/104: mknod d0/d19/c20 0 2024-03-22T06:15:17.201 INFO:tasks.workunit.client.1.smithi117.stdout:5/33: dwrite fa [0,4194304] 0 2024-03-22T06:15:17.201 INFO:tasks.workunit.client.1.smithi117.stdout:1/86: write d1/f8 [654085,129285] 0 2024-03-22T06:15:17.201 INFO:tasks.workunit.client.1.smithi117.stdout:5/34: write f4 [4374016,74777] 0 2024-03-22T06:15:17.201 INFO:tasks.workunit.client.1.smithi117.stdout:9/101: dwrite d4/d8/f1a [4194304,4194304] 0 2024-03-22T06:15:17.202 INFO:tasks.workunit.client.0.smithi006.stdout:3/867: dwrite d0/d8/d9/db1/fbc [0,4194304] 0 2024-03-22T06:15:17.202 INFO:tasks.workunit.client.1.smithi117.stdout:6/90: mkdir d3/d7/da/d16/d24 0 2024-03-22T06:15:17.202 INFO:tasks.workunit.client.1.smithi117.stdout:6/91: fdatasync d3/d7/f11 0 2024-03-22T06:15:17.202 INFO:tasks.workunit.client.1.smithi117.stdout:7/84: dwrite d8/d9/ff [4194304,4194304] 0 2024-03-22T06:15:17.203 INFO:tasks.workunit.client.1.smithi117.stdout:4/105: mknod d0/c21 0 2024-03-22T06:15:17.203 INFO:tasks.workunit.client.1.smithi117.stdout:4/106: truncate d0/d19/f1a 1001701 0 2024-03-22T06:15:17.211 INFO:tasks.workunit.client.0.smithi006.stdout:3/868: write d0/f56 [1414395,111554] 0 2024-03-22T06:15:17.211 INFO:tasks.workunit.client.1.smithi117.stdout:1/87: creat d1/f1b x:0 0 0 2024-03-22T06:15:17.215 INFO:tasks.workunit.client.0.smithi006.stdout:3/869: mknod d0/d4/dbb/df7/c127 0 2024-03-22T06:15:17.216 INFO:tasks.workunit.client.1.smithi117.stdout:9/102: write d4/f1e [970616,98080] 0 2024-03-22T06:15:17.217 INFO:tasks.workunit.client.0.smithi006.stdout:5/925: sync 2024-03-22T06:15:17.217 INFO:tasks.workunit.client.0.smithi006.stdout:5/926: read d0/db/d16/f46 [2307779,93410] 0 2024-03-22T06:15:17.218 INFO:tasks.workunit.client.1.smithi117.stdout:0/104: write d1/d3/f12 [1337149,127770] 0 2024-03-22T06:15:17.218 INFO:tasks.workunit.client.0.smithi006.stdout:3/870: creat d0/d4/dbb/df7/d5f/da3/f128 x:0 0 0 2024-03-22T06:15:17.219 INFO:tasks.workunit.client.0.smithi006.stdout:3/871: creat d0/d4/d31/f129 x:0 0 0 2024-03-22T06:15:17.219 INFO:tasks.workunit.client.0.smithi006.stdout:3/872: creat d0/d4/d31/f12a x:0 0 0 2024-03-22T06:15:17.221 INFO:tasks.workunit.client.0.smithi006.stdout:5/927: unlink d0/fa4 0 2024-03-22T06:15:17.222 INFO:tasks.workunit.client.0.smithi006.stdout:0/646: sync 2024-03-22T06:15:17.222 INFO:tasks.workunit.client.0.smithi006.stdout:6/779: sync 2024-03-22T06:15:17.222 INFO:tasks.workunit.client.0.smithi006.stdout:0/647: chown db/d10/d33/fb0 13842 1 2024-03-22T06:15:17.222 INFO:tasks.workunit.client.0.smithi006.stdout:0/648: truncate db/dc/d38/fc1 5162476 0 2024-03-22T06:15:17.223 INFO:tasks.workunit.client.0.smithi006.stdout:9/763: sync 2024-03-22T06:15:17.223 INFO:tasks.workunit.client.0.smithi006.stdout:1/782: sync 2024-03-22T06:15:17.223 INFO:tasks.workunit.client.1.smithi117.stdout:8/97: dwrite d1/d2/f1d [4194304,4194304] 0 2024-03-22T06:15:17.223 INFO:tasks.workunit.client.1.smithi117.stdout:5/35: dwrite f7 [0,4194304] 0 2024-03-22T06:15:17.226 INFO:tasks.workunit.client.0.smithi006.stdout:3/873: symlink d0/d4/dbb/df7/d5f/l12b 0 2024-03-22T06:15:17.229 INFO:tasks.workunit.client.0.smithi006.stdout:0/649: dread db/d10/d2a/d41/f7f [0,4194304] 0 2024-03-22T06:15:17.229 INFO:tasks.workunit.client.0.smithi006.stdout:0/650: chown db/d3d 24498118 1 2024-03-22T06:15:17.230 INFO:tasks.workunit.client.0.smithi006.stdout:0/651: write db/d10/f1b [700763,23184] 0 2024-03-22T06:15:17.230 INFO:tasks.workunit.client.0.smithi006.stdout:0/652: dread db/dc/d37/fac [0,4194304] 0 2024-03-22T06:15:17.230 INFO:tasks.workunit.client.0.smithi006.stdout:0/653: fsync db/d87/f9f 0 2024-03-22T06:15:17.249 INFO:tasks.workunit.client.0.smithi006.stdout:5/928: rmdir d0/db/d16/d1c/d34 39 2024-03-22T06:15:17.250 INFO:tasks.workunit.client.0.smithi006.stdout:6/780: symlink d6/d6f/lee 0 2024-03-22T06:15:17.250 INFO:tasks.workunit.client.0.smithi006.stdout:6/781: dread - d6/d6f/d94/fde zero size 2024-03-22T06:15:17.252 INFO:tasks.workunit.client.0.smithi006.stdout:9/764: link d8/d1f/fe5 d8/d1f/d34/ff8 0 2024-03-22T06:15:17.253 INFO:tasks.workunit.client.0.smithi006.stdout:1/783: getdents d8/d71/dcd 0 2024-03-22T06:15:17.255 INFO:tasks.workunit.client.0.smithi006.stdout:3/874: mkdir d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d12c 0 2024-03-22T06:15:17.255 INFO:tasks.workunit.client.0.smithi006.stdout:3/875: fdatasync d0/d4/dbb/df7/f60 0 2024-03-22T06:15:17.259 INFO:tasks.workunit.client.0.smithi006.stdout:0/654: rename db/dc/d38/d50/ca5 to db/d10/d33/db4/cdf 0 2024-03-22T06:15:17.259 INFO:tasks.workunit.client.0.smithi006.stdout:0/655: chown db/dc/d38/d50/fc3 5180 1 2024-03-22T06:15:17.259 INFO:tasks.workunit.client.0.smithi006.stdout:0/656: readlink db/d87/l6d 0 2024-03-22T06:15:17.259 INFO:tasks.workunit.client.0.smithi006.stdout:0/657: creat db/d3d/fe0 x:0 0 0 2024-03-22T06:15:17.260 INFO:tasks.workunit.client.1.smithi117.stdout:7/85: symlink d8/d9/da/l17 0 2024-03-22T06:15:17.261 INFO:tasks.workunit.client.0.smithi006.stdout:1/784: dread d8/d9/d3a/d40/d44/d80/f88 [0,4194304] 0 2024-03-22T06:15:17.264 INFO:tasks.workunit.client.0.smithi006.stdout:5/929: creat d0/db/dc/f11b x:0 0 0 2024-03-22T06:15:17.266 INFO:tasks.workunit.client.0.smithi006.stdout:3/876: dread d0/d4/dbb/df7/d5f/f92 [0,4194304] 0 2024-03-22T06:15:17.268 INFO:tasks.workunit.client.1.smithi117.stdout:4/107: rename d0/d4/fa to d0/d19/f22 0 2024-03-22T06:15:17.270 INFO:tasks.workunit.client.0.smithi006.stdout:6/782: symlink d6/d1d/d5c/dc4/lef 0 2024-03-22T06:15:17.270 INFO:tasks.workunit.client.0.smithi006.stdout:6/783: fsync d6/d24/d44/f62 0 2024-03-22T06:15:17.271 INFO:tasks.workunit.client.1.smithi117.stdout:1/88: symlink d1/d14/l1c 0 2024-03-22T06:15:17.271 INFO:tasks.workunit.client.1.smithi117.stdout:1/89: rename d1 to d1/d14/d16/d1d 22 2024-03-22T06:15:17.274 INFO:tasks.workunit.client.0.smithi006.stdout:9/765: link d8/db3/d50/f69 d8/d10/d28/d39/ff9 0 2024-03-22T06:15:17.275 INFO:tasks.workunit.client.1.smithi117.stdout:9/103: getdents d4/d7 0 2024-03-22T06:15:17.275 INFO:tasks.workunit.client.1.smithi117.stdout:9/104: write f2 [4375631,83090] 0 2024-03-22T06:15:17.276 INFO:tasks.workunit.client.1.smithi117.stdout:6/92: rmdir d3/d7/da/d16 39 2024-03-22T06:15:17.276 INFO:tasks.workunit.client.1.smithi117.stdout:6/93: dread - d3/d7/f1b zero size 2024-03-22T06:15:17.277 INFO:tasks.workunit.client.1.smithi117.stdout:4/108: write d0/f3 [5810505,53309] 0 2024-03-22T06:15:17.277 INFO:tasks.workunit.client.1.smithi117.stdout:4/109: creat d0/f23 x:0 0 0 2024-03-22T06:15:17.278 INFO:tasks.workunit.client.1.smithi117.stdout:0/105: mknod d1/d8/c22 0 2024-03-22T06:15:17.279 INFO:tasks.workunit.client.0.smithi006.stdout:5/930: creat d0/de8/f11c x:0 0 0 2024-03-22T06:15:17.279 INFO:tasks.workunit.client.0.smithi006.stdout:5/931: write d0/db/d50/db9/fbe [720040,4479] 0 2024-03-22T06:15:17.280 INFO:tasks.workunit.client.0.smithi006.stdout:3/877: symlink d0/d8/d88/dfa/l12d 0 2024-03-22T06:15:17.282 INFO:tasks.workunit.client.0.smithi006.stdout:6/784: chown d6/c9 1182691 1 2024-03-22T06:15:17.282 INFO:tasks.workunit.client.0.smithi006.stdout:6/785: readlink d6/d6f/l8f 0 2024-03-22T06:15:17.282 INFO:tasks.workunit.client.0.smithi006.stdout:1/785: dwrite d8/d33/f9f [0,4194304] 0 2024-03-22T06:15:17.284 INFO:tasks.workunit.client.0.smithi006.stdout:9/766: mkdir d8/d1f/d34/dfa 0 2024-03-22T06:15:17.285 INFO:tasks.workunit.client.0.smithi006.stdout:0/658: rmdir db/d3d/d4e 39 2024-03-22T06:15:17.285 INFO:tasks.workunit.client.0.smithi006.stdout:0/659: read db/d10/f1b [117945,100768] 0 2024-03-22T06:15:17.286 INFO:tasks.workunit.client.0.smithi006.stdout:0/660: stat db/d3d/c59 0 2024-03-22T06:15:17.286 INFO:tasks.workunit.client.1.smithi117.stdout:3/35: sync 2024-03-22T06:15:17.287 INFO:tasks.workunit.client.0.smithi006.stdout:5/932: rename d0/db/d76/f111 to d0/d115/f11d 0 2024-03-22T06:15:17.290 INFO:tasks.workunit.client.1.smithi117.stdout:7/86: dwrite d8/f16 [0,4194304] 0 2024-03-22T06:15:17.291 INFO:tasks.workunit.client.0.smithi006.stdout:3/878: mknod d0/d8/d88/ddf/d102/c12e 0 2024-03-22T06:15:17.293 INFO:tasks.workunit.client.0.smithi006.stdout:6/786: truncate d6/d1b/d2c/f59 7674186 0 2024-03-22T06:15:17.294 INFO:tasks.workunit.client.1.smithi117.stdout:5/36: rename l9 to lc 0 2024-03-22T06:15:17.294 INFO:tasks.workunit.client.1.smithi117.stdout:5/37: truncate f7 5163015 0 2024-03-22T06:15:17.298 INFO:tasks.workunit.client.1.smithi117.stdout:8/98: symlink d1/l23 0 2024-03-22T06:15:17.300 INFO:tasks.workunit.client.0.smithi006.stdout:1/786: creat d8/d9/d13/d39/f107 x:0 0 0 2024-03-22T06:15:17.301 INFO:tasks.workunit.client.1.smithi117.stdout:9/105: creat d4/d8/f22 x:0 0 0 2024-03-22T06:15:17.302 INFO:tasks.workunit.client.0.smithi006.stdout:9/767: creat d8/d10/d47/ffb x:0 0 0 2024-03-22T06:15:17.303 INFO:tasks.workunit.client.1.smithi117.stdout:6/94: symlink d3/d7/l25 0 2024-03-22T06:15:17.303 INFO:tasks.workunit.client.1.smithi117.stdout:5/38: write fa [54945,38083] 0 2024-03-22T06:15:17.303 INFO:tasks.workunit.client.1.smithi117.stdout:6/95: dread - d3/f1f zero size 2024-03-22T06:15:17.304 INFO:tasks.workunit.client.0.smithi006.stdout:0/661: stat db/dc/d37/cd1 0 2024-03-22T06:15:17.304 INFO:tasks.workunit.client.0.smithi006.stdout:0/662: chown db/d10/f14 0 1 2024-03-22T06:15:17.305 INFO:tasks.workunit.client.1.smithi117.stdout:7/87: dwrite f5 [4194304,4194304] 0 2024-03-22T06:15:17.305 INFO:tasks.workunit.client.1.smithi117.stdout:7/88: write d8/d9/fd [538835,111552] 0 2024-03-22T06:15:17.305 INFO:tasks.workunit.client.1.smithi117.stdout:7/89: chown d8 2652 1 2024-03-22T06:15:17.306 INFO:tasks.workunit.client.1.smithi117.stdout:4/110: mknod d0/d4/c24 0 2024-03-22T06:15:17.308 INFO:tasks.workunit.client.1.smithi117.stdout:0/106: creat d1/d7/de/f23 x:0 0 0 2024-03-22T06:15:17.308 INFO:tasks.workunit.client.1.smithi117.stdout:0/107: dread - d1/f19 zero size 2024-03-22T06:15:17.312 INFO:tasks.workunit.client.1.smithi117.stdout:2/74: sync 2024-03-22T06:15:17.312 INFO:tasks.workunit.client.1.smithi117.stdout:2/75: readlink d2/d5/db/lf 0 2024-03-22T06:15:17.315 INFO:tasks.workunit.client.0.smithi006.stdout:3/879: mknod d0/d8/d9/c12f 0 2024-03-22T06:15:17.317 INFO:tasks.workunit.client.1.smithi117.stdout:8/99: rename d1/d2/d9/c20 to d1/d2/d11/c24 0 2024-03-22T06:15:17.319 INFO:tasks.workunit.client.0.smithi006.stdout:6/787: mkdir d6/d24/df0 0 2024-03-22T06:15:17.321 INFO:tasks.workunit.client.1.smithi117.stdout:1/90: truncate d1/f7 2715266 0 2024-03-22T06:15:17.323 INFO:tasks.workunit.client.0.smithi006.stdout:1/787: rename d8/d9/cbd to d8/d9/d3a/d40/da1/c108 0 2024-03-22T06:15:17.323 INFO:tasks.workunit.client.0.smithi006.stdout:1/788: write d8/d9/d13/d39/d60/f6e [891423,1779] 0 2024-03-22T06:15:17.326 INFO:tasks.workunit.client.1.smithi117.stdout:9/106: rmdir d4/d8/d14 0 2024-03-22T06:15:17.326 INFO:tasks.workunit.client.1.smithi117.stdout:9/107: write d4/d8/f13 [361867,75472] 0 2024-03-22T06:15:17.327 INFO:tasks.workunit.client.0.smithi006.stdout:9/768: mkdir d8/d10/d47/dfc 0 2024-03-22T06:15:17.329 INFO:tasks.workunit.client.1.smithi117.stdout:5/39: mkdir dd 0 2024-03-22T06:15:17.329 INFO:tasks.workunit.client.0.smithi006.stdout:0/663: creat db/d10/d33/d9d/fe1 x:0 0 0 2024-03-22T06:15:17.330 INFO:tasks.workunit.client.0.smithi006.stdout:3/880: symlink d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d12c/l130 0 2024-03-22T06:15:17.330 INFO:tasks.workunit.client.0.smithi006.stdout:3/881: fsync d0/d4/dbb/df7/d5f/d90/da7/d97/fb7 0 2024-03-22T06:15:17.331 INFO:tasks.workunit.client.0.smithi006.stdout:9/769: mknod d8/d1f/d34/dfa/cfd 0 2024-03-22T06:15:17.332 INFO:tasks.workunit.client.1.smithi117.stdout:2/76: truncate d2/f4 270248 0 2024-03-22T06:15:17.332 INFO:tasks.workunit.client.0.smithi006.stdout:0/664: unlink db/dc/d38/cc6 0 2024-03-22T06:15:17.333 INFO:tasks.workunit.client.1.smithi117.stdout:8/100: symlink d1/d2/l25 0 2024-03-22T06:15:17.334 INFO:tasks.workunit.client.1.smithi117.stdout:1/91: chown d1/c12 99677 1 2024-03-22T06:15:17.335 INFO:tasks.workunit.client.0.smithi006.stdout:3/882: symlink d0/d4/dbb/df7/d68/l131 0 2024-03-22T06:15:17.336 INFO:tasks.workunit.client.1.smithi117.stdout:9/108: creat d4/d7/f23 x:0 0 0 2024-03-22T06:15:17.338 INFO:tasks.workunit.client.1.smithi117.stdout:5/40: creat dd/fe x:0 0 0 2024-03-22T06:15:17.338 INFO:tasks.workunit.client.1.smithi117.stdout:5/41: write dd/fe [902526,17598] 0 2024-03-22T06:15:17.339 INFO:tasks.workunit.client.1.smithi117.stdout:7/90: rmdir d8 39 2024-03-22T06:15:17.339 INFO:tasks.workunit.client.0.smithi006.stdout:3/883: mknod d0/d8/d88/dfa/dcb/c132 0 2024-03-22T06:15:17.346 INFO:tasks.workunit.client.1.smithi117.stdout:2/77: write f1 [1067082,88028] 0 2024-03-22T06:15:17.348 INFO:tasks.workunit.client.1.smithi117.stdout:2/78: mkdir d2/d5/d13 0 2024-03-22T06:15:17.350 INFO:tasks.workunit.client.1.smithi117.stdout:6/96: dwrite d3/d7/da/fc [0,4194304] 0 2024-03-22T06:15:17.350 INFO:tasks.workunit.client.1.smithi117.stdout:7/91: dread d8/f16 [0,4194304] 0 2024-03-22T06:15:17.352 INFO:tasks.workunit.client.1.smithi117.stdout:6/97: mknod d3/d7/da/c26 0 2024-03-22T06:15:17.352 INFO:tasks.workunit.client.1.smithi117.stdout:6/98: chown d3/d7/f1b 1 1 2024-03-22T06:15:17.352 INFO:tasks.workunit.client.1.smithi117.stdout:6/99: fdatasync d3/d7/da/de/f19 0 2024-03-22T06:15:17.352 INFO:tasks.workunit.client.1.smithi117.stdout:7/92: symlink d8/d9/l18 0 2024-03-22T06:15:17.353 INFO:tasks.workunit.client.1.smithi117.stdout:6/100: mkdir d3/df/d17/d27 0 2024-03-22T06:15:17.353 INFO:tasks.workunit.client.1.smithi117.stdout:8/101: write d1/d2/d9/f13 [2579884,18291] 0 2024-03-22T06:15:17.354 INFO:tasks.workunit.client.1.smithi117.stdout:8/102: readlink d1/d2/d9/lc 0 2024-03-22T06:15:17.354 INFO:tasks.workunit.client.1.smithi117.stdout:4/111: dwrite d0/d4/f7 [0,4194304] 0 2024-03-22T06:15:17.355 INFO:tasks.workunit.client.1.smithi117.stdout:6/101: unlink d3/d7/da/c10 0 2024-03-22T06:15:17.364 INFO:tasks.workunit.client.1.smithi117.stdout:0/108: dwrite d1/d10/f11 [0,4194304] 0 2024-03-22T06:15:17.365 INFO:tasks.workunit.client.0.smithi006.stdout:6/788: dwrite d6/d1b/d2c/f72 [4194304,4194304] 0 2024-03-22T06:15:17.365 INFO:tasks.workunit.client.0.smithi006.stdout:1/789: dwrite d8/d9/d3a/d40/d44/f76 [0,4194304] 0 2024-03-22T06:15:17.366 INFO:tasks.workunit.client.1.smithi117.stdout:9/109: dwrite d4/d7/f19 [0,4194304] 0 2024-03-22T06:15:17.366 INFO:tasks.workunit.client.1.smithi117.stdout:9/110: creat d4/d7/f24 x:0 0 0 2024-03-22T06:15:17.366 INFO:tasks.workunit.client.1.smithi117.stdout:1/92: fdatasync d1/f8 0 2024-03-22T06:15:17.366 INFO:tasks.workunit.client.1.smithi117.stdout:9/111: creat d4/d8/f25 x:0 0 0 2024-03-22T06:15:17.368 INFO:tasks.workunit.client.0.smithi006.stdout:3/884: dread d0/d4/dbb/df7/f96 [0,4194304] 0 2024-03-22T06:15:17.369 INFO:tasks.workunit.client.1.smithi117.stdout:3/36: dwrite f7 [0,4194304] 0 2024-03-22T06:15:17.374 INFO:tasks.workunit.client.0.smithi006.stdout:1/790: creat d8/d9/d13/d24/d32/dc7/f109 x:0 0 0 2024-03-22T06:15:17.374 INFO:tasks.workunit.client.0.smithi006.stdout:1/791: chown d8/d9/d3a/d18/f37 25616 1 2024-03-22T06:15:17.376 INFO:tasks.workunit.client.0.smithi006.stdout:3/885: mknod d0/d4/dbb/df7/d5f/d90/da7/de7/d33/c133 0 2024-03-22T06:15:17.376 INFO:tasks.workunit.client.0.smithi006.stdout:3/886: creat d0/d4/d31/d98/f134 x:0 0 0 2024-03-22T06:15:17.376 INFO:tasks.workunit.client.0.smithi006.stdout:3/887: chown d0/d4/dbb/df7/d5f/da3/f47 0 1 2024-03-22T06:15:17.378 INFO:tasks.workunit.client.1.smithi117.stdout:0/109: fsync d1/d3/f12 0 2024-03-22T06:15:17.384 INFO:tasks.workunit.client.1.smithi117.stdout:3/37: dread f7 [0,4194304] 0 2024-03-22T06:15:17.397 INFO:tasks.workunit.client.1.smithi117.stdout:7/93: dwrite d8/d9/ff [0,4194304] 0 2024-03-22T06:15:17.414 INFO:tasks.workunit.client.1.smithi117.stdout:4/112: dwrite d0/f11 [0,4194304] 0 2024-03-22T06:15:17.415 INFO:tasks.workunit.client.1.smithi117.stdout:4/113: write d0/f3 [2187462,71634] 0 2024-03-22T06:15:17.415 INFO:tasks.workunit.client.1.smithi117.stdout:4/114: chown d0/d4/f17 219285960 1 2024-03-22T06:15:17.417 INFO:tasks.workunit.client.1.smithi117.stdout:5/42: dwrite f4 [0,4194304] 0 2024-03-22T06:15:17.417 INFO:tasks.workunit.client.1.smithi117.stdout:1/93: dwrite d1/d14/f19 [0,4194304] 0 2024-03-22T06:15:17.418 INFO:tasks.workunit.client.1.smithi117.stdout:1/94: chown d1/d14/f19 1 1 2024-03-22T06:15:17.418 INFO:tasks.workunit.client.1.smithi117.stdout:1/95: write d1/f7 [1860501,35540] 0 2024-03-22T06:15:17.418 INFO:tasks.workunit.client.1.smithi117.stdout:1/96: write d1/d14/d16/f1a [968828,11185] 0 2024-03-22T06:15:17.418 INFO:tasks.workunit.client.1.smithi117.stdout:9/112: dwrite d4/d7/f19 [0,4194304] 0 2024-03-22T06:15:17.418 INFO:tasks.workunit.client.1.smithi117.stdout:9/113: truncate d4/d8/f22 831626 0 2024-03-22T06:15:17.418 INFO:tasks.workunit.client.1.smithi117.stdout:2/79: dwrite f1 [4194304,4194304] 0 2024-03-22T06:15:17.419 INFO:tasks.workunit.client.1.smithi117.stdout:1/97: write d1/d14/d16/f1a [292698,96111] 0 2024-03-22T06:15:17.419 INFO:tasks.workunit.client.1.smithi117.stdout:6/102: dwrite d3/d7/da/d16/f22 [0,4194304] 0 2024-03-22T06:15:17.419 INFO:tasks.workunit.client.0.smithi006.stdout:6/789: dwrite d6/d1d/f71 [4194304,4194304] 0 2024-03-22T06:15:17.419 INFO:tasks.workunit.client.0.smithi006.stdout:6/790: creat d6/d1d/d99/ff1 x:0 0 0 2024-03-22T06:15:17.419 INFO:tasks.workunit.client.0.smithi006.stdout:6/791: write d6/d24/fa8 [1478298,124670] 0 2024-03-22T06:15:17.433 INFO:tasks.workunit.client.1.smithi117.stdout:9/114: write f2 [1981308,11978] 0 2024-03-22T06:15:17.433 INFO:tasks.workunit.client.1.smithi117.stdout:9/115: creat d4/d7/f26 x:0 0 0 2024-03-22T06:15:17.433 INFO:tasks.workunit.client.1.smithi117.stdout:9/116: chown d4/d7/f16 3888 1 2024-03-22T06:15:17.433 INFO:tasks.workunit.client.1.smithi117.stdout:9/117: read - d4/d7/f23 zero size 2024-03-22T06:15:17.442 INFO:tasks.workunit.client.1.smithi117.stdout:5/43: dwrite f6 [0,4194304] 0 2024-03-22T06:15:17.442 INFO:tasks.workunit.client.1.smithi117.stdout:5/44: chown f6 689262 1 2024-03-22T06:15:17.444 INFO:tasks.workunit.client.1.smithi117.stdout:5/45: dread f2 [0,4194304] 0 2024-03-22T06:15:17.471 INFO:tasks.workunit.client.0.smithi006.stdout:1/792: rename d8/d9/d3a/d40 to d8/d9/d13/d39/d60/d8f/d10a 0 2024-03-22T06:15:17.473 INFO:tasks.workunit.client.0.smithi006.stdout:9/770: getdents d8/d1f/d34/dfa 0 2024-03-22T06:15:17.474 INFO:tasks.workunit.client.1.smithi117.stdout:3/38: symlink d9/l10 0 2024-03-22T06:15:17.475 INFO:tasks.workunit.client.1.smithi117.stdout:3/39: creat d9/f11 x:0 0 0 2024-03-22T06:15:17.475 INFO:tasks.workunit.client.1.smithi117.stdout:3/40: write d9/f11 [937913,107705] 0 2024-03-22T06:15:17.480 INFO:tasks.workunit.client.0.smithi006.stdout:9/771: dread d8/db3/d50/f61 [0,4194304] 0 2024-03-22T06:15:17.482 INFO:tasks.workunit.client.1.smithi117.stdout:7/94: creat d8/d9/da/d10/f19 x:0 0 0 2024-03-22T06:15:17.484 INFO:tasks.workunit.client.0.smithi006.stdout:6/792: symlink d6/d1d/d5c/d81/d8a/lf2 0 2024-03-22T06:15:17.484 INFO:tasks.workunit.client.0.smithi006.stdout:6/793: write d6/d1d/d99/fbe [258239,94950] 0 2024-03-22T06:15:17.487 INFO:tasks.workunit.client.1.smithi117.stdout:4/115: rename d0/d4/fb to d0/f25 0 2024-03-22T06:15:17.494 INFO:tasks.workunit.client.0.smithi006.stdout:1/793: creat d8/d9/d13/f10b x:0 0 0 2024-03-22T06:15:17.494 INFO:tasks.workunit.client.1.smithi117.stdout:7/95: dread f5 [0,4194304] 0 2024-03-22T06:15:17.496 INFO:tasks.workunit.client.0.smithi006.stdout:0/665: dwrite db/d10/d33/d9d/fa2 [0,4194304] 0 2024-03-22T06:15:17.497 INFO:tasks.workunit.client.0.smithi006.stdout:9/772: mkdir d8/db3/d50/d91/dfe 0 2024-03-22T06:15:17.497 INFO:tasks.workunit.client.0.smithi006.stdout:9/773: write d8/db3/d50/ff1 [771540,91810] 0 2024-03-22T06:15:17.497 INFO:tasks.workunit.client.0.smithi006.stdout:9/774: readlink d8/d10/d28/dc2/le7 0 2024-03-22T06:15:17.501 INFO:tasks.workunit.client.1.smithi117.stdout:2/80: mkdir d2/d14 0 2024-03-22T06:15:17.502 INFO:tasks.workunit.client.0.smithi006.stdout:0/666: dread - db/f96 zero size 2024-03-22T06:15:17.503 INFO:tasks.workunit.client.0.smithi006.stdout:0/667: creat db/d10/d33/d9d/dcd/fe2 x:0 0 0 2024-03-22T06:15:17.504 INFO:tasks.workunit.client.1.smithi117.stdout:4/116: dwrite d0/f25 [0,4194304] 0 2024-03-22T06:15:17.504 INFO:tasks.workunit.client.0.smithi006.stdout:9/775: truncate d8/d10/f17 2661204 0 2024-03-22T06:15:17.507 INFO:tasks.workunit.client.0.smithi006.stdout:6/794: getdents d6/d7c/d92 0 2024-03-22T06:15:17.507 INFO:tasks.workunit.client.0.smithi006.stdout:1/794: write d8/d9/d13/d24/d32/d34/fb3 [3672169,81747] 0 2024-03-22T06:15:17.511 INFO:tasks.workunit.client.1.smithi117.stdout:9/118: chown d4/l12 915149 1 2024-03-22T06:15:17.514 INFO:tasks.workunit.client.0.smithi006.stdout:0/668: mknod db/d87/ce3 0 2024-03-22T06:15:17.518 INFO:tasks.workunit.client.0.smithi006.stdout:1/795: mkdir d8/d9/d13/d39/d60/d8f/d10a/d10c 0 2024-03-22T06:15:17.518 INFO:tasks.workunit.client.0.smithi006.stdout:1/796: truncate d8/d9/d13/d39/d60/fd4 445841 0 2024-03-22T06:15:17.520 INFO:tasks.workunit.client.0.smithi006.stdout:1/797: rmdir d8/d9/d13/d24/d32/d34/daa 39 2024-03-22T06:15:17.523 INFO:tasks.workunit.client.0.smithi006.stdout:1/798: symlink d8/d71/dcd/l10d 0 2024-03-22T06:15:17.526 INFO:tasks.workunit.client.0.smithi006.stdout:1/799: truncate d8/d9/d13/f28 7604817 0 2024-03-22T06:15:17.526 INFO:tasks.workunit.client.0.smithi006.stdout:1/800: dread - d8/d9/d13/d39/d60/d8f/dea/ffd zero size 2024-03-22T06:15:17.527 INFO:tasks.workunit.client.0.smithi006.stdout:9/776: dwrite d8/d10/d20/fa0 [0,4194304] 0 2024-03-22T06:15:17.529 INFO:tasks.workunit.client.0.smithi006.stdout:1/801: truncate d8/d9/d3a/f92 1262644 0 2024-03-22T06:15:17.534 INFO:tasks.workunit.client.0.smithi006.stdout:6/795: dwrite d6/d24/d44/f5f [4194304,4194304] 0 2024-03-22T06:15:17.536 INFO:tasks.workunit.client.1.smithi117.stdout:3/41: fsync f6 0 2024-03-22T06:15:17.536 INFO:tasks.workunit.client.1.smithi117.stdout:3/42: truncate d9/fb 215410 0 2024-03-22T06:15:17.538 INFO:tasks.workunit.client.1.smithi117.stdout:7/96: mkdir d8/d9/d1a 0 2024-03-22T06:15:17.539 INFO:tasks.workunit.client.0.smithi006.stdout:9/777: read d8/d10/d28/f3d [2921374,35592] 0 2024-03-22T06:15:17.539 INFO:tasks.workunit.client.0.smithi006.stdout:6/796: getdents d6/d1d 0 2024-03-22T06:15:17.539 INFO:tasks.workunit.client.0.smithi006.stdout:6/797: dread - d6/d1d/dc6/fe2 zero size 2024-03-22T06:15:17.541 INFO:tasks.workunit.client.1.smithi117.stdout:2/81: creat d2/d5/d8/d10/d11/f15 x:0 0 0 2024-03-22T06:15:17.541 INFO:tasks.workunit.client.0.smithi006.stdout:0/669: dwrite db/d10/d35/fd3 [0,4194304] 0 2024-03-22T06:15:17.542 INFO:tasks.workunit.client.0.smithi006.stdout:1/802: dread d8/d33/f4d [0,4194304] 0 2024-03-22T06:15:17.542 INFO:tasks.workunit.client.0.smithi006.stdout:0/670: chown db/d87/faf 7355 1 2024-03-22T06:15:17.542 INFO:tasks.workunit.client.0.smithi006.stdout:0/671: fsync db/d3d/fe0 0 2024-03-22T06:15:17.542 INFO:tasks.workunit.client.0.smithi006.stdout:0/672: stat db/dc/f44 0 2024-03-22T06:15:17.542 INFO:tasks.workunit.client.0.smithi006.stdout:0/673: readlink db/d10/d35/d7b/l95 0 2024-03-22T06:15:17.542 INFO:tasks.workunit.client.0.smithi006.stdout:9/778: creat d8/db3/d50/d91/dfe/fff x:0 0 0 2024-03-22T06:15:17.543 INFO:tasks.workunit.client.1.smithi117.stdout:1/98: mknod d1/c1e 0 2024-03-22T06:15:17.547 INFO:tasks.workunit.client.1.smithi117.stdout:4/117: link d0/d4/cd d0/d19/d1c/c26 0 2024-03-22T06:15:17.547 INFO:tasks.workunit.client.1.smithi117.stdout:4/118: creat d0/f27 x:0 0 0 2024-03-22T06:15:17.547 INFO:tasks.workunit.client.0.smithi006.stdout:6/798: mkdir d6/d24/d44/df3 0 2024-03-22T06:15:17.549 INFO:tasks.workunit.client.1.smithi117.stdout:6/103: getdents d3/d7/da 0 2024-03-22T06:15:17.549 INFO:tasks.workunit.client.1.smithi117.stdout:6/104: fdatasync d3/f20 0 2024-03-22T06:15:17.549 INFO:tasks.workunit.client.1.smithi117.stdout:6/105: readlink d3/d7/l8 0 2024-03-22T06:15:17.552 INFO:tasks.workunit.client.1.smithi117.stdout:9/119: mkdir d4/d7/d27 0 2024-03-22T06:15:17.552 INFO:tasks.workunit.client.1.smithi117.stdout:1/99: dread d1/f7 [0,4194304] 0 2024-03-22T06:15:17.552 INFO:tasks.workunit.client.1.smithi117.stdout:1/100: chown d1/f1b 20 1 2024-03-22T06:15:17.552 INFO:tasks.workunit.client.1.smithi117.stdout:9/120: dread - d4/d7/f23 zero size 2024-03-22T06:15:17.556 INFO:tasks.workunit.client.1.smithi117.stdout:3/43: unlink d9/l10 0 2024-03-22T06:15:17.556 INFO:tasks.workunit.client.1.smithi117.stdout:3/44: write d9/fa [90225,72338] 0 2024-03-22T06:15:17.557 INFO:tasks.workunit.client.1.smithi117.stdout:7/97: symlink d8/d9/d1a/l1b 0 2024-03-22T06:15:17.559 INFO:tasks.workunit.client.0.smithi006.stdout:1/803: symlink d8/d9/l10e 0 2024-03-22T06:15:17.560 INFO:tasks.workunit.client.1.smithi117.stdout:2/82: mknod d2/c16 0 2024-03-22T06:15:17.562 INFO:tasks.workunit.client.0.smithi006.stdout:0/674: link db/d10/d33/db4/cdc db/d10/d33/d9d/ce4 0 2024-03-22T06:15:17.562 INFO:tasks.workunit.client.0.smithi006.stdout:0/675: chown db/d10/d35/d7b/da1/fd5 0 1 2024-03-22T06:15:17.564 INFO:tasks.workunit.client.0.smithi006.stdout:9/779: mknod d8/c100 0 2024-03-22T06:15:17.564 INFO:tasks.workunit.client.0.smithi006.stdout:9/780: chown d8/d10/d28/f9d 1551 1 2024-03-22T06:15:17.566 INFO:tasks.workunit.client.0.smithi006.stdout:6/799: rename d6/d1d/d32/f67 to d6/d24/ff4 0 2024-03-22T06:15:17.567 INFO:tasks.workunit.client.1.smithi117.stdout:6/106: dwrite d3/d7/da/de/f19 [0,4194304] 0 2024-03-22T06:15:17.568 INFO:tasks.workunit.client.1.smithi117.stdout:8/103: sync 2024-03-22T06:15:17.568 INFO:tasks.workunit.client.1.smithi117.stdout:0/110: sync 2024-03-22T06:15:17.569 INFO:tasks.workunit.client.1.smithi117.stdout:9/121: creat d4/d7/d10/f28 x:0 0 0 2024-03-22T06:15:17.569 INFO:tasks.workunit.client.1.smithi117.stdout:2/83: dread d2/f4 [0,4194304] 0 2024-03-22T06:15:17.569 INFO:tasks.workunit.client.1.smithi117.stdout:2/84: readlink d2/d5/db/lf 0 2024-03-22T06:15:17.569 INFO:tasks.workunit.client.1.smithi117.stdout:9/122: write d4/f21 [502158,74751] 0 2024-03-22T06:15:17.570 INFO:tasks.workunit.client.0.smithi006.stdout:1/804: symlink d8/d9/d13/d39/d60/d8f/l10f 0 2024-03-22T06:15:17.570 INFO:tasks.workunit.client.0.smithi006.stdout:1/805: fsync d8/d9/d3a/d18/ffe 0 2024-03-22T06:15:17.571 INFO:tasks.workunit.client.1.smithi117.stdout:5/46: truncate f6 1654874 0 2024-03-22T06:15:17.571 INFO:tasks.workunit.client.1.smithi117.stdout:5/47: stat f6 0 2024-03-22T06:15:17.571 INFO:tasks.workunit.client.1.smithi117.stdout:5/48: readlink lc 0 2024-03-22T06:15:17.572 INFO:tasks.workunit.client.1.smithi117.stdout:3/45: mknod d9/c12 0 2024-03-22T06:15:17.574 INFO:tasks.workunit.client.0.smithi006.stdout:9/781: mknod d8/d10/d20/c101 0 2024-03-22T06:15:17.574 INFO:tasks.workunit.client.0.smithi006.stdout:9/782: chown d8/db3/lb1 57 1 2024-03-22T06:15:17.575 INFO:tasks.workunit.client.0.smithi006.stdout:0/676: read db/d10/d35/fd3 [2017844,47057] 0 2024-03-22T06:15:17.575 INFO:tasks.workunit.client.1.smithi117.stdout:7/98: mkdir d8/d9/da/d10/d1c 0 2024-03-22T06:15:17.578 INFO:tasks.workunit.client.0.smithi006.stdout:6/800: symlink d6/d1d/d99/dcd/lf5 0 2024-03-22T06:15:17.578 INFO:tasks.workunit.client.1.smithi117.stdout:9/123: dread d4/d7/fd [0,4194304] 0 2024-03-22T06:15:17.579 INFO:tasks.workunit.client.0.smithi006.stdout:1/806: creat d8/d9/d13/d39/d60/d8f/d10a/d44/f110 x:0 0 0 2024-03-22T06:15:17.580 INFO:tasks.workunit.client.1.smithi117.stdout:2/85: dread f1 [0,4194304] 0 2024-03-22T06:15:17.580 INFO:tasks.workunit.client.1.smithi117.stdout:1/101: dwrite d1/f7 [0,4194304] 0 2024-03-22T06:15:17.581 INFO:tasks.workunit.client.1.smithi117.stdout:6/107: mkdir d3/d7/d28 0 2024-03-22T06:15:17.581 INFO:tasks.workunit.client.1.smithi117.stdout:6/108: readlink d3/d7/l8 0 2024-03-22T06:15:17.584 INFO:tasks.workunit.client.0.smithi006.stdout:9/783: read f7 [3695777,13372] 0 2024-03-22T06:15:17.586 INFO:tasks.workunit.client.1.smithi117.stdout:0/111: mkdir d1/d3/d24 0 2024-03-22T06:15:17.588 INFO:tasks.workunit.client.1.smithi117.stdout:5/49: mkdir dd/df 0 2024-03-22T06:15:17.589 INFO:tasks.workunit.client.0.smithi006.stdout:0/677: symlink db/dc/d37/le5 0 2024-03-22T06:15:17.590 INFO:tasks.workunit.client.1.smithi117.stdout:3/46: rename d9/ld to d9/l13 0 2024-03-22T06:15:17.594 INFO:tasks.workunit.client.1.smithi117.stdout:7/99: rmdir d8/d9/da/d10/d1c 0 2024-03-22T06:15:17.597 INFO:tasks.workunit.client.0.smithi006.stdout:9/784: link d8/d10/d20/d45/d4d/f85 d8/d10/d20/d45/d4d/f102 0 2024-03-22T06:15:17.600 INFO:tasks.workunit.client.0.smithi006.stdout:0/678: symlink db/d10/d35/d7b/da1/da6/dc4/le6 0 2024-03-22T06:15:17.601 INFO:tasks.workunit.client.1.smithi117.stdout:3/47: write f8 [3482347,88084] 0 2024-03-22T06:15:17.601 INFO:tasks.workunit.client.1.smithi117.stdout:2/86: symlink d2/l17 0 2024-03-22T06:15:17.601 INFO:tasks.workunit.client.1.smithi117.stdout:2/87: stat d2/l12 0 2024-03-22T06:15:17.602 INFO:tasks.workunit.client.1.smithi117.stdout:2/88: chown d2/d5 7 1 2024-03-22T06:15:17.602 INFO:tasks.workunit.client.0.smithi006.stdout:6/801: symlink d6/lf6 0 2024-03-22T06:15:17.602 INFO:tasks.workunit.client.1.smithi117.stdout:4/119: rmdir d0 39 2024-03-22T06:15:17.605 INFO:tasks.workunit.client.1.smithi117.stdout:7/100: write f6 [6987762,48562] 0 2024-03-22T06:15:17.605 INFO:tasks.workunit.client.1.smithi117.stdout:7/101: dread - d8/fb zero size 2024-03-22T06:15:17.605 INFO:tasks.workunit.client.1.smithi117.stdout:7/102: chown d8 27934 1 2024-03-22T06:15:17.607 INFO:tasks.workunit.client.1.smithi117.stdout:8/104: rmdir d1 39 2024-03-22T06:15:17.607 INFO:tasks.workunit.client.1.smithi117.stdout:8/105: chown d1/d2/d9/l17 14056861 1 2024-03-22T06:15:17.609 INFO:tasks.workunit.client.1.smithi117.stdout:0/112: creat d1/d8/dc/f25 x:0 0 0 2024-03-22T06:15:17.609 INFO:tasks.workunit.client.1.smithi117.stdout:5/50: mkdir dd/df/d10 0 2024-03-22T06:15:17.610 INFO:tasks.workunit.client.1.smithi117.stdout:1/102: mkdir d1/d1f 0 2024-03-22T06:15:17.613 INFO:tasks.workunit.client.0.smithi006.stdout:0/679: dread db/d10/d2a/d41/f7f [0,4194304] 0 2024-03-22T06:15:17.614 INFO:tasks.workunit.client.0.smithi006.stdout:0/680: chown db/d3d/d4e/f90 249917715 1 2024-03-22T06:15:17.615 INFO:tasks.workunit.client.0.smithi006.stdout:7/797: sync 2024-03-22T06:15:17.615 INFO:tasks.workunit.client.0.smithi006.stdout:7/798: stat d5/d1e/d29/d59/d5c 0 2024-03-22T06:15:17.615 INFO:tasks.workunit.client.0.smithi006.stdout:7/799: chown d5/d1e/l97 15212174 1 2024-03-22T06:15:17.615 INFO:tasks.workunit.client.0.smithi006.stdout:7/800: write d5/d1e/d3f/f48 [940321,22574] 0 2024-03-22T06:15:17.616 INFO:tasks.workunit.client.1.smithi117.stdout:2/89: creat d2/d14/f18 x:0 0 0 2024-03-22T06:15:17.617 INFO:tasks.workunit.client.0.smithi006.stdout:6/802: dread d6/d1d/d5c/d81/d8a/dac/fbb [0,4194304] 0 2024-03-22T06:15:17.617 INFO:tasks.workunit.client.0.smithi006.stdout:4/867: sync 2024-03-22T06:15:17.617 INFO:tasks.workunit.client.0.smithi006.stdout:5/933: sync 2024-03-22T06:15:17.617 INFO:tasks.workunit.client.0.smithi006.stdout:8/972: sync 2024-03-22T06:15:17.617 INFO:tasks.workunit.client.0.smithi006.stdout:3/888: sync 2024-03-22T06:15:17.617 INFO:tasks.workunit.client.0.smithi006.stdout:8/973: fsync d9/db/d12/d66/d6c/f63 0 2024-03-22T06:15:17.617 INFO:tasks.workunit.client.0.smithi006.stdout:4/868: write d6/f10e [341583,89952] 0 2024-03-22T06:15:17.617 INFO:tasks.workunit.client.0.smithi006.stdout:4/869: fsync d6/d1b/d20/d21/d11d/f123 0 2024-03-22T06:15:17.618 INFO:tasks.workunit.client.1.smithi117.stdout:4/120: getdents d0/d19 0 2024-03-22T06:15:17.619 INFO:tasks.workunit.client.1.smithi117.stdout:7/103: write f5 [6607097,129880] 0 2024-03-22T06:15:17.619 INFO:tasks.workunit.client.1.smithi117.stdout:9/124: dwrite d4/d7/f1b [0,4194304] 0 2024-03-22T06:15:17.619 INFO:tasks.workunit.client.0.smithi006.stdout:0/681: creat db/d3d/d4e/d76/fe7 x:0 0 0 2024-03-22T06:15:17.621 INFO:tasks.workunit.client.1.smithi117.stdout:0/113: rmdir d1/d10/d18 39 2024-03-22T06:15:17.622 INFO:tasks.workunit.client.1.smithi117.stdout:0/114: write d1/d3/f12 [1628527,59650] 0 2024-03-22T06:15:17.622 INFO:tasks.workunit.client.1.smithi117.stdout:6/109: dwrite d3/d7/da/d16/f22 [0,4194304] 0 2024-03-22T06:15:17.624 INFO:tasks.workunit.client.1.smithi117.stdout:5/51: mkdir dd/d11 0 2024-03-22T06:15:17.626 INFO:tasks.workunit.client.1.smithi117.stdout:1/103: write d1/f7 [1545371,106953] 0 2024-03-22T06:15:17.626 INFO:tasks.workunit.client.1.smithi117.stdout:2/90: unlink d2/cd 0 2024-03-22T06:15:17.627 INFO:tasks.workunit.client.0.smithi006.stdout:5/934: mkdir d0/db/d50/d96/d52/d113/d11e 0 2024-03-22T06:15:17.627 INFO:tasks.workunit.client.0.smithi006.stdout:5/935: chown d0/db/d16/d1c/d34/f53 655168 1 2024-03-22T06:15:17.627 INFO:tasks.workunit.client.0.smithi006.stdout:5/936: dread - d0/db/d16/d43/fe9 zero size 2024-03-22T06:15:17.628 INFO:tasks.workunit.client.1.smithi117.stdout:4/121: mkdir d0/d19/d28 0 2024-03-22T06:15:17.629 INFO:tasks.workunit.client.0.smithi006.stdout:1/807: dwrite d8/d9/d13/d39/d60/d8f/d10a/f7d [0,4194304] 0 2024-03-22T06:15:17.630 INFO:tasks.workunit.client.0.smithi006.stdout:8/974: mknod d9/db/d12/d66/d6c/c160 0 2024-03-22T06:15:17.630 INFO:tasks.workunit.client.0.smithi006.stdout:8/975: stat d9/db/d35/f3c 0 2024-03-22T06:15:17.631 INFO:tasks.workunit.client.1.smithi117.stdout:7/104: unlink d8/d9/da/d10/f11 0 2024-03-22T06:15:17.633 INFO:tasks.workunit.client.0.smithi006.stdout:4/870: link d6/d13/f104 d6/d1b/d20/d63/d93/f12c 0 2024-03-22T06:15:17.633 INFO:tasks.workunit.client.0.smithi006.stdout:4/871: write f3 [8790744,33465] 0 2024-03-22T06:15:17.634 INFO:tasks.workunit.client.0.smithi006.stdout:0/682: rmdir db 39 2024-03-22T06:15:17.635 INFO:tasks.workunit.client.1.smithi117.stdout:6/110: write d3/d7/da/fc [3040754,19364] 0 2024-03-22T06:15:17.636 INFO:tasks.workunit.client.1.smithi117.stdout:6/111: write d3/f5 [3701160,57785] 0 2024-03-22T06:15:17.636 INFO:tasks.workunit.client.0.smithi006.stdout:7/801: rename d5/d1e/fe1 to d5/d1e/d29/d66/f105 0 2024-03-22T06:15:17.636 INFO:tasks.workunit.client.0.smithi006.stdout:7/802: stat d5/d1e/d29/d10/ca9 0 2024-03-22T06:15:17.636 INFO:tasks.workunit.client.0.smithi006.stdout:7/803: chown d5/d1e/d29/d1f/d2d/d55/d84/fa0 6481 1 2024-03-22T06:15:17.637 INFO:tasks.workunit.client.0.smithi006.stdout:7/804: getdents d5/d1e/d29/d1f/d2d/df7/ddd 0 2024-03-22T06:15:17.638 INFO:tasks.workunit.client.1.smithi117.stdout:9/125: mknod d4/d7/d27/c29 0 2024-03-22T06:15:17.638 INFO:tasks.workunit.client.1.smithi117.stdout:9/126: creat d4/d7/f2a x:0 0 0 2024-03-22T06:15:17.638 INFO:tasks.workunit.client.1.smithi117.stdout:5/52: truncate f7 4218539 0 2024-03-22T06:15:17.638 INFO:tasks.workunit.client.1.smithi117.stdout:9/127: dread d4/d8/f22 [0,4194304] 0 2024-03-22T06:15:17.639 INFO:tasks.workunit.client.1.smithi117.stdout:9/128: fsync d4/d8/f13 0 2024-03-22T06:15:17.639 INFO:tasks.workunit.client.1.smithi117.stdout:9/129: creat d4/d8/f2b x:0 0 0 2024-03-22T06:15:17.639 INFO:tasks.workunit.client.1.smithi117.stdout:9/130: write d4/d8/f22 [1486107,119714] 0 2024-03-22T06:15:17.639 INFO:tasks.workunit.client.1.smithi117.stdout:9/131: chown d4/d7/f26 63392 1 2024-03-22T06:15:17.639 INFO:tasks.workunit.client.0.smithi006.stdout:5/937: link d0/db/d16/d70/c9c d0/db/d16/d1c/dbc/dd6/c11f 0 2024-03-22T06:15:17.640 INFO:tasks.workunit.client.1.smithi117.stdout:1/104: creat d1/d1f/f20 x:0 0 0 2024-03-22T06:15:17.641 INFO:tasks.workunit.client.1.smithi117.stdout:4/122: write d0/f11 [4139892,79856] 0 2024-03-22T06:15:17.641 INFO:tasks.workunit.client.1.smithi117.stdout:2/91: creat d2/d5/d13/f19 x:0 0 0 2024-03-22T06:15:17.643 INFO:tasks.workunit.client.1.smithi117.stdout:5/53: write dd/fe [748641,19568] 0 2024-03-22T06:15:17.644 INFO:tasks.workunit.client.1.smithi117.stdout:7/105: symlink d8/d9/da/d10/l1d 0 2024-03-22T06:15:17.646 INFO:tasks.workunit.client.0.smithi006.stdout:8/976: unlink d9/db/d35/dd2/f4b 0 2024-03-22T06:15:17.646 INFO:tasks.workunit.client.0.smithi006.stdout:8/977: chown f2 164 1 2024-03-22T06:15:17.646 INFO:tasks.workunit.client.0.smithi006.stdout:1/808: dread d8/d9/d13/d24/d32/dc7/fc6 [0,4194304] 0 2024-03-22T06:15:17.647 INFO:tasks.workunit.client.0.smithi006.stdout:1/809: truncate d8/d9/d13/f6d 2187983 0 2024-03-22T06:15:17.647 INFO:tasks.workunit.client.1.smithi117.stdout:0/115: fdatasync d1/d3/f12 0 2024-03-22T06:15:17.647 INFO:tasks.workunit.client.1.smithi117.stdout:8/106: dwrite f0 [4194304,4194304] 0 2024-03-22T06:15:17.647 INFO:tasks.workunit.client.1.smithi117.stdout:0/116: read - d1/d7/de/f23 zero size 2024-03-22T06:15:17.647 INFO:tasks.workunit.client.1.smithi117.stdout:0/117: fdatasync d1/f19 0 2024-03-22T06:15:17.648 INFO:tasks.workunit.client.0.smithi006.stdout:4/872: mknod d6/d1b/d20/d6b/c12d 0 2024-03-22T06:15:17.651 INFO:tasks.workunit.client.0.smithi006.stdout:0/683: creat db/d10/d35/d7b/fe8 x:0 0 0 2024-03-22T06:15:17.651 INFO:tasks.workunit.client.1.smithi117.stdout:6/112: dread d3/d7/da/de/f19 [0,4194304] 0 2024-03-22T06:15:17.651 INFO:tasks.workunit.client.1.smithi117.stdout:3/48: dwrite f2 [0,4194304] 0 2024-03-22T06:15:17.652 INFO:tasks.workunit.client.1.smithi117.stdout:3/49: chown d9/c12 81337 1 2024-03-22T06:15:17.652 INFO:tasks.workunit.client.1.smithi117.stdout:6/113: write d3/d7/f11 [91832,73053] 0 2024-03-22T06:15:17.655 INFO:tasks.workunit.client.0.smithi006.stdout:6/803: rename d6/f13 to d6/d1d/d99/dcd/ff7 0 2024-03-22T06:15:17.657 INFO:tasks.workunit.client.0.smithi006.stdout:9/785: write d8/db3/d50/f6f [914679,1517] 0 2024-03-22T06:15:17.657 INFO:tasks.workunit.client.0.smithi006.stdout:3/889: dwrite d0/d8/d88/dfa/d58/f8f [0,4194304] 0 2024-03-22T06:15:17.658 INFO:tasks.workunit.client.1.smithi117.stdout:5/54: read f4 [2597515,106905] 0 2024-03-22T06:15:17.658 INFO:tasks.workunit.client.1.smithi117.stdout:5/55: readlink lc 0 2024-03-22T06:15:17.659 INFO:tasks.workunit.client.1.smithi117.stdout:5/56: fsync f4 0 2024-03-22T06:15:17.659 INFO:tasks.workunit.client.1.smithi117.stdout:7/106: creat d8/d9/f1e x:0 0 0 2024-03-22T06:15:17.659 INFO:tasks.workunit.client.1.smithi117.stdout:7/107: chown d8/d9/d1a/l1b 162 1 2024-03-22T06:15:17.659 INFO:tasks.workunit.client.0.smithi006.stdout:8/978: link d9/fa d9/db/d12/d66/d10c/d125/da3/f161 0 2024-03-22T06:15:17.659 INFO:tasks.workunit.client.0.smithi006.stdout:8/979: creat d9/db/d12/d66/d6c/f162 x:0 0 0 2024-03-22T06:15:17.659 INFO:tasks.workunit.client.0.smithi006.stdout:8/980: fdatasync d9/db/d35/da9/df6/ff7 0 2024-03-22T06:15:17.660 INFO:tasks.workunit.client.0.smithi006.stdout:8/981: fdatasync d9/db/d12/fb6 0 2024-03-22T06:15:17.661 INFO:tasks.workunit.client.0.smithi006.stdout:1/810: creat d8/d9/d3a/f111 x:0 0 0 2024-03-22T06:15:17.666 INFO:tasks.workunit.client.1.smithi117.stdout:8/107: mknod d1/c26 0 2024-03-22T06:15:17.668 INFO:tasks.workunit.client.0.smithi006.stdout:4/873: getdents d6/d1b/d20 0 2024-03-22T06:15:17.672 INFO:tasks.workunit.client.0.smithi006.stdout:6/804: rename d6/d1b/d2c/f7d to d6/d6f/d94/ff8 0 2024-03-22T06:15:17.672 INFO:tasks.workunit.client.0.smithi006.stdout:6/805: chown d6/d1d/d5c/d81/d8a 7 1 2024-03-22T06:15:17.672 INFO:tasks.workunit.client.0.smithi006.stdout:6/806: getdents d6/d24/df0 0 2024-03-22T06:15:17.672 INFO:tasks.workunit.client.0.smithi006.stdout:6/807: write d6/d6f/f9b [422711,120326] 0 2024-03-22T06:15:17.673 INFO:tasks.workunit.client.0.smithi006.stdout:5/938: rmdir d0/db/dc 39 2024-03-22T06:15:17.676 INFO:tasks.workunit.client.1.smithi117.stdout:3/50: mknod d9/c14 0 2024-03-22T06:15:17.678 INFO:tasks.workunit.client.0.smithi006.stdout:3/890: link d0/d4/dbb/df7/d68/cd3 d0/d8/d88/dfa/de2/c135 0 2024-03-22T06:15:17.678 INFO:tasks.workunit.client.0.smithi006.stdout:3/891: chown d0/d8/d1c/c51 303360330 1 2024-03-22T06:15:17.679 INFO:tasks.workunit.client.0.smithi006.stdout:3/892: write d0/d4/dbb/df7/d68/f6b [1304209,2125] 0 2024-03-22T06:15:17.680 INFO:tasks.workunit.client.1.smithi117.stdout:1/105: getdents d1 0 2024-03-22T06:15:17.680 INFO:tasks.workunit.client.1.smithi117.stdout:0/118: dwrite d1/d7/fb [0,4194304] 0 2024-03-22T06:15:17.682 INFO:tasks.workunit.client.1.smithi117.stdout:4/123: dwrite d0/d19/f1a [0,4194304] 0 2024-03-22T06:15:17.683 INFO:tasks.workunit.client.0.smithi006.stdout:1/811: dwrite d8/d9/d13/d39/d60/d8f/d10a/d44/d80/f88 [0,4194304] 0 2024-03-22T06:15:17.683 INFO:tasks.workunit.client.0.smithi006.stdout:1/812: write d8/d9/d3a/f92 [2067174,33836] 0 2024-03-22T06:15:17.683 INFO:tasks.workunit.client.0.smithi006.stdout:1/813: fsync d8/d9/d13/d39/f107 0 2024-03-22T06:15:17.684 INFO:tasks.workunit.client.0.smithi006.stdout:8/982: mknod d9/db/d35/dd2/d76/dca/d15d/c163 0 2024-03-22T06:15:17.684 INFO:tasks.workunit.client.0.smithi006.stdout:8/983: dread - d9/db/d35/dd2/d4a/da7/ff1 zero size 2024-03-22T06:15:17.684 INFO:tasks.workunit.client.0.smithi006.stdout:8/984: write d9/db/d12/d66/d6c/f63 [495479,56619] 0 2024-03-22T06:15:17.684 INFO:tasks.workunit.client.0.smithi006.stdout:8/985: write d9/db/d35/dd2/d4a/da7/f139 [1017977,120405] 0 2024-03-22T06:15:17.684 INFO:tasks.workunit.client.0.smithi006.stdout:8/986: creat d9/db/d12/d66/d6c/f164 x:0 0 0 2024-03-22T06:15:17.685 INFO:tasks.workunit.client.0.smithi006.stdout:8/987: dread d9/db/d12/f5d [0,4194304] 0 2024-03-22T06:15:17.687 INFO:tasks.workunit.client.0.smithi006.stdout:4/874: symlink d6/d13/l12e 0 2024-03-22T06:15:17.688 INFO:tasks.workunit.client.0.smithi006.stdout:4/875: readlink d6/d13/d3b/l81 0 2024-03-22T06:15:17.688 INFO:tasks.workunit.client.0.smithi006.stdout:4/876: fsync d6/d1b/d20/d6b/d88/fe5 0 2024-03-22T06:15:17.691 INFO:tasks.workunit.client.0.smithi006.stdout:9/786: rename d8/d10/d11/dc0 to d8/d10/d20/d103 0 2024-03-22T06:15:17.691 INFO:tasks.workunit.client.0.smithi006.stdout:9/787: read d8/d10/f2e [3267047,26832] 0 2024-03-22T06:15:17.691 INFO:tasks.workunit.client.0.smithi006.stdout:9/788: write d8/d10/d20/d45/d4d/f85 [3112067,119515] 0 2024-03-22T06:15:17.695 INFO:tasks.workunit.client.1.smithi117.stdout:5/57: rename dd/d11 to dd/df/d12 0 2024-03-22T06:15:17.696 INFO:tasks.workunit.client.0.smithi006.stdout:6/808: mknod d6/d1d/d5c/d81/d8a/cf9 0 2024-03-22T06:15:17.697 INFO:tasks.workunit.client.1.smithi117.stdout:7/108: creat d8/f1f x:0 0 0 2024-03-22T06:15:17.698 INFO:tasks.workunit.client.0.smithi006.stdout:5/939: rmdir d0/db/d16/d1c/d34/d35/d85/dda/d65 39 2024-03-22T06:15:17.701 INFO:tasks.workunit.client.0.smithi006.stdout:3/893: mkdir d0/d4/d136 0 2024-03-22T06:15:17.701 INFO:tasks.workunit.client.1.smithi117.stdout:8/108: mknod d1/d2/c27 0 2024-03-22T06:15:17.706 INFO:tasks.workunit.client.1.smithi117.stdout:3/51: mknod d9/c15 0 2024-03-22T06:15:17.707 INFO:tasks.workunit.client.1.smithi117.stdout:7/109: write d8/f16 [137013,26922] 0 2024-03-22T06:15:17.708 INFO:tasks.workunit.client.0.smithi006.stdout:1/814: creat d8/d71/dbb/f112 x:0 0 0 2024-03-22T06:15:17.712 INFO:tasks.workunit.client.1.smithi117.stdout:4/124: dwrite d0/f11 [0,4194304] 0 2024-03-22T06:15:17.712 INFO:tasks.workunit.client.0.smithi006.stdout:4/877: dwrite d6/f1c [0,4194304] 0 2024-03-22T06:15:17.718 INFO:tasks.workunit.client.1.smithi117.stdout:1/106: link d1/la d1/d14/l21 0 2024-03-22T06:15:17.737 INFO:tasks.workunit.client.1.smithi117.stdout:4/125: dwrite d0/f27 [0,4194304] 0 2024-03-22T06:15:17.737 INFO:tasks.workunit.client.1.smithi117.stdout:5/58: dwrite dd/fe [0,4194304] 0 2024-03-22T06:15:17.737 INFO:tasks.workunit.client.1.smithi117.stdout:5/59: getdents dd/df/d12 0 2024-03-22T06:15:17.738 INFO:tasks.workunit.client.1.smithi117.stdout:5/60: chown dd 32826 1 2024-03-22T06:15:17.739 INFO:tasks.workunit.client.0.smithi006.stdout:5/940: creat d0/db/d16/d1c/d34/d35/d85/d5e/d66/dc0/f120 x:0 0 0 2024-03-22T06:15:17.739 INFO:tasks.workunit.client.0.smithi006.stdout:5/941: dread - d0/db/d16/d1c/d34/d35/d85/fc1 zero size 2024-03-22T06:15:17.741 INFO:tasks.workunit.client.0.smithi006.stdout:3/894: link d0/d8/d1c/d72/f94 d0/d8/d88/ddf/d102/f137 0 2024-03-22T06:15:17.742 INFO:tasks.workunit.client.0.smithi006.stdout:3/895: dread d0/d4/dbb/df7/d5f/da3/fc8 [0,4194304] 0 2024-03-22T06:15:17.742 INFO:tasks.workunit.client.0.smithi006.stdout:3/896: creat d0/d4/dbb/df7/f138 x:0 0 0 2024-03-22T06:15:17.742 INFO:tasks.workunit.client.0.smithi006.stdout:3/897: write d0/d4/dbb/df7/d5f/d90/da7/d97/fa5 [3582560,68860] 0 2024-03-22T06:15:17.742 INFO:tasks.workunit.client.0.smithi006.stdout:3/898: dread - d0/d4/d3a/fde zero size 2024-03-22T06:15:17.747 INFO:tasks.workunit.client.0.smithi006.stdout:1/815: creat d8/d9/f113 x:0 0 0 2024-03-22T06:15:17.748 INFO:tasks.workunit.client.1.smithi117.stdout:8/109: mkdir d1/d28 0 2024-03-22T06:15:17.749 INFO:tasks.workunit.client.0.smithi006.stdout:3/899: dread d0/d4/dbb/df7/d5f/d90/da7/de7/d33/f44 [0,4194304] 0 2024-03-22T06:15:17.749 INFO:tasks.workunit.client.0.smithi006.stdout:3/900: write d0/d4/d31/fcc [1117666,71578] 0 2024-03-22T06:15:17.749 INFO:tasks.workunit.client.0.smithi006.stdout:3/901: creat d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/f139 x:0 0 0 2024-03-22T06:15:17.749 INFO:tasks.workunit.client.0.smithi006.stdout:3/902: chown d0/d7e/f80 477079787 1 2024-03-22T06:15:17.752 INFO:tasks.workunit.client.1.smithi117.stdout:3/52: rename d9/c12 to d9/c16 0 2024-03-22T06:15:17.754 INFO:tasks.workunit.client.1.smithi117.stdout:3/53: dread d9/f11 [0,4194304] 0 2024-03-22T06:15:17.754 INFO:tasks.workunit.client.1.smithi117.stdout:3/54: creat d9/f17 x:0 0 0 2024-03-22T06:15:17.756 INFO:tasks.workunit.client.0.smithi006.stdout:4/878: symlink d6/d1b/d20/d63/d93/d9a/d8f/da1/l12f 0 2024-03-22T06:15:17.756 INFO:tasks.workunit.client.0.smithi006.stdout:4/879: write d6/d1b/d20/ff0 [232077,18644] 0 2024-03-22T06:15:17.756 INFO:tasks.workunit.client.0.smithi006.stdout:4/880: write d6/d13/d3b/f6e [296678,130917] 0 2024-03-22T06:15:17.757 INFO:tasks.workunit.client.1.smithi117.stdout:7/110: rmdir d8/d9/da 39 2024-03-22T06:15:17.757 INFO:tasks.workunit.client.1.smithi117.stdout:7/111: write d8/d9/fd [2973764,90328] 0 2024-03-22T06:15:17.758 INFO:tasks.workunit.client.0.smithi006.stdout:8/988: rmdir d9/db/d12/d1e/d11e 39 2024-03-22T06:15:17.758 INFO:tasks.workunit.client.0.smithi006.stdout:8/989: creat d9/db/d35/dd2/d76/db3/dc3/f165 x:0 0 0 2024-03-22T06:15:17.759 INFO:tasks.workunit.client.1.smithi117.stdout:1/107: mkdir d1/d22 0 2024-03-22T06:15:17.759 INFO:tasks.workunit.client.1.smithi117.stdout:3/55: dread f8 [0,4194304] 0 2024-03-22T06:15:17.759 INFO:tasks.workunit.client.1.smithi117.stdout:6/114: truncate d3/d7/f11 130065 0 2024-03-22T06:15:17.761 INFO:tasks.workunit.client.0.smithi006.stdout:6/809: rename d6/d1d/d32/c4e to d6/d77/cfa 0 2024-03-22T06:15:17.761 INFO:tasks.workunit.client.0.smithi006.stdout:6/810: write d6/d24/f31 [4165009,110658] 0 2024-03-22T06:15:17.765 INFO:tasks.workunit.client.0.smithi006.stdout:5/942: link d0/db/d50/d82/d103/ff6 d0/d115/f121 0 2024-03-22T06:15:17.767 INFO:tasks.workunit.client.0.smithi006.stdout:6/811: dread d6/d1d/d5c/d89/f8b [0,4194304] 0 2024-03-22T06:15:17.767 INFO:tasks.workunit.client.0.smithi006.stdout:6/812: truncate d6/d6f/f9e 55146 0 2024-03-22T06:15:17.771 INFO:tasks.workunit.client.0.smithi006.stdout:1/816: truncate d8/d9/d13/d39/d60/fd4 1270570 0 2024-03-22T06:15:17.772 INFO:tasks.workunit.client.0.smithi006.stdout:6/813: dread d6/d1d/d32/f45 [0,4194304] 0 2024-03-22T06:15:17.775 INFO:tasks.workunit.client.1.smithi117.stdout:1/108: dwrite d1/d14/d16/f1a [0,4194304] 0 2024-03-22T06:15:17.776 INFO:tasks.workunit.client.1.smithi117.stdout:0/119: rename d1/d8 to d1/d10/d18/d26 0 2024-03-22T06:15:17.778 INFO:tasks.workunit.client.0.smithi006.stdout:3/903: write d0/d8/d88/dfa/d58/f120 [9833269,6084] 0 2024-03-22T06:15:17.781 INFO:tasks.workunit.client.0.smithi006.stdout:4/881: mkdir d6/d1b/d20/d63/d93/d9a/d130 0 2024-03-22T06:15:17.782 INFO:tasks.workunit.client.1.smithi117.stdout:7/112: mknod d8/d9/c20 0 2024-03-22T06:15:17.786 INFO:tasks.workunit.client.1.smithi117.stdout:3/56: unlink d9/lf 0 2024-03-22T06:15:17.786 INFO:tasks.workunit.client.1.smithi117.stdout:3/57: creat d9/f18 x:0 0 0 2024-03-22T06:15:17.788 INFO:tasks.workunit.client.0.smithi006.stdout:8/990: rename d9/db/d35/da9/df6/f105 to d9/db/d35/dd2/dc4/f166 0 2024-03-22T06:15:17.791 INFO:tasks.workunit.client.1.smithi117.stdout:4/126: truncate d0/f27 872469 0 2024-03-22T06:15:17.791 INFO:tasks.workunit.client.1.smithi117.stdout:4/127: fsync d0/d19/f1a 0 2024-03-22T06:15:17.792 INFO:tasks.workunit.client.0.smithi006.stdout:1/817: creat d8/d71/f114 x:0 0 0 2024-03-22T06:15:17.794 INFO:tasks.workunit.client.0.smithi006.stdout:8/991: mkdir d9/db/d12/d167 0 2024-03-22T06:15:17.795 INFO:tasks.workunit.client.0.smithi006.stdout:8/992: dread d9/db/d12/d66/d10c/d125/da3/fc0 [0,4194304] 0 2024-03-22T06:15:17.795 INFO:tasks.workunit.client.0.smithi006.stdout:8/993: truncate d9/db/d12/d66/d6c/d71/fa2 1061256 0 2024-03-22T06:15:17.795 INFO:tasks.workunit.client.0.smithi006.stdout:8/994: chown d9/db/d35/dd2/dc4/d104/c11f 21127 1 2024-03-22T06:15:17.797 INFO:tasks.workunit.client.0.smithi006.stdout:1/818: symlink d8/d71/dcd/l115 0 2024-03-22T06:15:17.798 INFO:tasks.workunit.client.1.smithi117.stdout:5/61: rename cb to dd/df/d12/c13 0 2024-03-22T06:15:17.799 INFO:tasks.workunit.client.0.smithi006.stdout:8/995: creat d9/db/d35/dd2/dc4/ddd/f168 x:0 0 0 2024-03-22T06:15:17.800 INFO:tasks.workunit.client.1.smithi117.stdout:7/113: rmdir d8/d9/d1a 39 2024-03-22T06:15:17.801 INFO:tasks.workunit.client.0.smithi006.stdout:1/819: mknod d8/d9/d3a/c116 0 2024-03-22T06:15:17.802 INFO:tasks.workunit.client.1.smithi117.stdout:4/128: mknod d0/d4/c29 0 2024-03-22T06:15:17.802 INFO:tasks.workunit.client.0.smithi006.stdout:1/820: symlink d8/d9/d13/d39/d60/l117 0 2024-03-22T06:15:17.803 INFO:tasks.workunit.client.1.smithi117.stdout:1/109: rmdir d1/d1f 39 2024-03-22T06:15:17.803 INFO:tasks.workunit.client.1.smithi117.stdout:1/110: dread - d1/f13 zero size 2024-03-22T06:15:17.804 INFO:tasks.workunit.client.1.smithi117.stdout:5/62: mknod dd/c14 0 2024-03-22T06:15:17.804 INFO:tasks.workunit.client.1.smithi117.stdout:5/63: chown dd 697755291 1 2024-03-22T06:15:17.804 INFO:tasks.workunit.client.1.smithi117.stdout:5/64: chown dd 0 1 2024-03-22T06:15:17.804 INFO:tasks.workunit.client.0.smithi006.stdout:8/996: dread d9/db/dc/de/f17 [0,4194304] 0 2024-03-22T06:15:17.805 INFO:tasks.workunit.client.0.smithi006.stdout:1/821: creat d8/d9/d13/d39/d60/d8f/d10a/d59/f118 x:0 0 0 2024-03-22T06:15:17.805 INFO:tasks.workunit.client.0.smithi006.stdout:1/822: stat d8/d71/dcd/ldf 0 2024-03-22T06:15:17.806 INFO:tasks.workunit.client.1.smithi117.stdout:4/129: mkdir d0/d4/d2a 0 2024-03-22T06:15:17.807 INFO:tasks.workunit.client.1.smithi117.stdout:1/111: mknod d1/d22/c23 0 2024-03-22T06:15:17.808 INFO:tasks.workunit.client.1.smithi117.stdout:5/65: symlink dd/df/l15 0 2024-03-22T06:15:17.808 INFO:tasks.workunit.client.1.smithi117.stdout:4/130: stat d0/d19/f22 0 2024-03-22T06:15:17.809 INFO:tasks.workunit.client.1.smithi117.stdout:4/131: creat d0/d19/d28/f2b x:0 0 0 2024-03-22T06:15:17.809 INFO:tasks.workunit.client.1.smithi117.stdout:4/132: stat d0/d4/d2a 0 2024-03-22T06:15:17.809 INFO:tasks.workunit.client.1.smithi117.stdout:4/133: dread - d0/d4/f1b zero size 2024-03-22T06:15:17.816 INFO:tasks.workunit.client.1.smithi117.stdout:0/120: dwrite d1/d7/fb [0,4194304] 0 2024-03-22T06:15:17.816 INFO:tasks.workunit.client.1.smithi117.stdout:0/121: write d1/d10/d18/d26/dc/f25 [803609,101669] 0 2024-03-22T06:15:17.816 INFO:tasks.workunit.client.1.smithi117.stdout:0/122: read - d1/d7/de/f23 zero size 2024-03-22T06:15:17.817 INFO:tasks.workunit.client.1.smithi117.stdout:0/123: fdatasync d1/d7/ff 0 2024-03-22T06:15:17.817 INFO:tasks.workunit.client.1.smithi117.stdout:0/124: stat d1/d10 0 2024-03-22T06:15:17.817 INFO:tasks.workunit.client.1.smithi117.stdout:0/125: write d1/d7/de/f23 [775511,79045] 0 2024-03-22T06:15:17.817 INFO:tasks.workunit.client.1.smithi117.stdout:0/126: creat d1/d10/f27 x:0 0 0 2024-03-22T06:15:17.821 INFO:tasks.workunit.client.0.smithi006.stdout:1/823: dread d8/d9/d13/d39/fc9 [0,4194304] 0 2024-03-22T06:15:17.821 INFO:tasks.workunit.client.0.smithi006.stdout:1/824: chown d8/d71/dcd/l10d 305562353 1 2024-03-22T06:15:17.821 INFO:tasks.workunit.client.0.smithi006.stdout:1/825: chown d8/d9/d13/lb8 1 1 2024-03-22T06:15:17.821 INFO:tasks.workunit.client.1.smithi117.stdout:3/58: write f7 [4127423,125224] 0 2024-03-22T06:15:17.821 INFO:tasks.workunit.client.1.smithi117.stdout:3/59: write d9/f11 [1176662,126037] 0 2024-03-22T06:15:17.821 INFO:tasks.workunit.client.1.smithi117.stdout:3/60: creat d9/f19 x:0 0 0 2024-03-22T06:15:17.824 INFO:tasks.workunit.client.0.smithi006.stdout:6/814: dwrite d6/d24/f38 [0,4194304] 0 2024-03-22T06:15:17.825 INFO:tasks.workunit.client.0.smithi006.stdout:6/815: mknod d6/d1d/d5c/cfb 0 2024-03-22T06:15:17.826 INFO:tasks.workunit.client.1.smithi117.stdout:5/66: dread f6 [0,4194304] 0 2024-03-22T06:15:17.826 INFO:tasks.workunit.client.0.smithi006.stdout:3/904: dwrite d0/d8/d9/db1/fbc [0,4194304] 0 2024-03-22T06:15:17.827 INFO:tasks.workunit.client.0.smithi006.stdout:5/943: dwrite d0/db/d16/d43/feb [0,4194304] 0 2024-03-22T06:15:17.827 INFO:tasks.workunit.client.0.smithi006.stdout:5/944: chown d0/db/d50/c7f 215 1 2024-03-22T06:15:17.827 INFO:tasks.workunit.client.1.smithi117.stdout:6/115: dread d3/d7/da/fc [0,4194304] 0 2024-03-22T06:15:17.829 INFO:tasks.workunit.client.1.smithi117.stdout:8/110: dwrite d1/d2/f7 [4194304,4194304] 0 2024-03-22T06:15:17.829 INFO:tasks.workunit.client.1.smithi117.stdout:1/112: dread d1/d14/f19 [0,4194304] 0 2024-03-22T06:15:17.829 INFO:tasks.workunit.client.1.smithi117.stdout:8/111: rename d1 to d1/d19/d29 22 2024-03-22T06:15:17.830 INFO:tasks.workunit.client.0.smithi006.stdout:3/905: symlink d0/d4/d3a/l13a 0 2024-03-22T06:15:17.830 INFO:tasks.workunit.client.0.smithi006.stdout:3/906: write d0/d8/d88/dfa/dcb/d108/f109 [721254,20166] 0 2024-03-22T06:15:17.831 INFO:tasks.workunit.client.1.smithi117.stdout:4/134: write d0/d4/fe [25050,12500] 0 2024-03-22T06:15:17.831 INFO:tasks.workunit.client.1.smithi117.stdout:4/135: write d0/d4/fe [3282836,78822] 0 2024-03-22T06:15:17.831 INFO:tasks.workunit.client.1.smithi117.stdout:4/136: creat d0/d19/d28/f2c x:0 0 0 2024-03-22T06:15:17.831 INFO:tasks.workunit.client.1.smithi117.stdout:4/137: chown d0/l1f 1874224942 1 2024-03-22T06:15:17.832 INFO:tasks.workunit.client.1.smithi117.stdout:6/116: creat d3/d7/da/f29 x:0 0 0 2024-03-22T06:15:17.834 INFO:tasks.workunit.client.1.smithi117.stdout:1/113: mknod d1/d1f/c24 0 2024-03-22T06:15:17.835 INFO:tasks.workunit.client.0.smithi006.stdout:4/882: dwrite f3 [4194304,4194304] 0 2024-03-22T06:15:17.835 INFO:tasks.workunit.client.0.smithi006.stdout:8/997: dwrite d9/d14/f121 [0,4194304] 0 2024-03-22T06:15:17.835 INFO:tasks.workunit.client.0.smithi006.stdout:8/998: fdatasync d9/db/f7b 0 2024-03-22T06:15:17.835 INFO:tasks.workunit.client.0.smithi006.stdout:8/999: write d9/db/d35/dd2/d4a/da7/ff1 [461013,34036] 0 2024-03-22T06:15:17.836 INFO:tasks.workunit.client.0.smithi006.stdout:1/826: write d8/d33/f5b [2888868,113668] 0 2024-03-22T06:15:17.837 INFO:tasks.workunit.client.1.smithi117.stdout:0/127: dwrite d1/d3/f12 [0,4194304] 0 2024-03-22T06:15:17.837 INFO:tasks.workunit.client.1.smithi117.stdout:0/128: truncate d1/d7/de/f23 1244802 0 2024-03-22T06:15:17.837 INFO:tasks.workunit.client.1.smithi117.stdout:0/129: fdatasync d1/d7/ff 0 2024-03-22T06:15:17.837 INFO:tasks.workunit.client.1.smithi117.stdout:0/130: rename d1/d10/d18 to d1/d10/d18/d28 22 2024-03-22T06:15:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:5/945: readlink d0/db/l33 0 2024-03-22T06:15:17.842 INFO:tasks.workunit.client.1.smithi117.stdout:4/138: creat d0/f2d x:0 0 0 2024-03-22T06:15:17.842 INFO:tasks.workunit.client.1.smithi117.stdout:4/139: creat d0/d19/d28/f2e x:0 0 0 2024-03-22T06:15:17.843 INFO:tasks.workunit.client.1.smithi117.stdout:8/112: dread d1/d2/f1d [0,4194304] 0 2024-03-22T06:15:17.844 INFO:tasks.workunit.client.1.smithi117.stdout:1/114: dread d1/d14/f19 [0,4194304] 0 2024-03-22T06:15:17.844 INFO:tasks.workunit.client.1.smithi117.stdout:1/115: read - d1/f1b zero size 2024-03-22T06:15:17.846 INFO:tasks.workunit.client.1.smithi117.stdout:9/132: sync 2024-03-22T06:15:17.846 INFO:tasks.workunit.client.1.smithi117.stdout:2/92: sync 2024-03-22T06:15:17.847 INFO:tasks.workunit.client.1.smithi117.stdout:0/131: rmdir d1/d3/d24 0 2024-03-22T06:15:17.848 INFO:tasks.workunit.client.1.smithi117.stdout:9/133: dread d4/d7/fd [0,4194304] 0 2024-03-22T06:15:17.849 INFO:tasks.workunit.client.1.smithi117.stdout:9/134: fdatasync d4/d7/f2a 0 2024-03-22T06:15:17.850 INFO:tasks.workunit.client.0.smithi006.stdout:3/907: mkdir d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d12c/d13b 0 2024-03-22T06:15:17.850 INFO:tasks.workunit.client.0.smithi006.stdout:3/908: write d0/d4/d53/f117 [917402,3748] 0 2024-03-22T06:15:17.851 INFO:tasks.workunit.client.1.smithi117.stdout:4/140: mkdir d0/d4/d2a/d2f 0 2024-03-22T06:15:17.853 INFO:tasks.workunit.client.0.smithi006.stdout:1/827: write d8/d71/dcd/ff6 [3895851,28082] 0 2024-03-22T06:15:17.853 INFO:tasks.workunit.client.0.smithi006.stdout:4/883: mkdir d6/d1b/d20/d63/d131 0 2024-03-22T06:15:17.853 INFO:tasks.workunit.client.0.smithi006.stdout:4/884: chown f1 259508935 1 2024-03-22T06:15:17.855 INFO:tasks.workunit.client.1.smithi117.stdout:6/117: mknod d3/c2a 0 2024-03-22T06:15:17.855 INFO:tasks.workunit.client.1.smithi117.stdout:1/116: dread d1/f8 [0,4194304] 0 2024-03-22T06:15:17.857 INFO:tasks.workunit.client.1.smithi117.stdout:4/141: creat d0/d4/d2a/f30 x:0 0 0 2024-03-22T06:15:17.857 INFO:tasks.workunit.client.1.smithi117.stdout:4/142: stat d0/d19/f1a 0 2024-03-22T06:15:17.859 INFO:tasks.workunit.client.1.smithi117.stdout:7/114: dwrite d8/d9/fd [4194304,4194304] 0 2024-03-22T06:15:17.859 INFO:tasks.workunit.client.1.smithi117.stdout:7/115: chown d8/fb 802114 1 2024-03-22T06:15:17.859 INFO:tasks.workunit.client.1.smithi117.stdout:7/116: write f5 [8550570,44559] 0 2024-03-22T06:15:17.861 INFO:tasks.workunit.client.0.smithi006.stdout:1/828: write d8/d9/d13/d39/d94/db5/fca [3819540,16735] 0 2024-03-22T06:15:17.862 INFO:tasks.workunit.client.1.smithi117.stdout:8/113: dread d1/d2/f1d [4194304,4194304] 0 2024-03-22T06:15:17.865 INFO:tasks.workunit.client.1.smithi117.stdout:0/132: rmdir d1/d10 39 2024-03-22T06:15:17.866 INFO:tasks.workunit.client.1.smithi117.stdout:5/67: dwrite f2 [0,4194304] 0 2024-03-22T06:15:17.867 INFO:tasks.workunit.client.1.smithi117.stdout:8/114: creat d1/d2/d11/f2a x:0 0 0 2024-03-22T06:15:17.869 INFO:tasks.workunit.client.1.smithi117.stdout:0/133: mkdir d1/d10/d18/d26/dc/d29 0 2024-03-22T06:15:17.869 INFO:tasks.workunit.client.1.smithi117.stdout:3/61: rmdir d9 39 2024-03-22T06:15:17.871 INFO:tasks.workunit.client.1.smithi117.stdout:6/118: dread d3/d7/fd [0,4194304] 0 2024-03-22T06:15:17.874 INFO:tasks.workunit.client.1.smithi117.stdout:5/68: mkdir dd/d16 0 2024-03-22T06:15:17.876 INFO:tasks.workunit.client.1.smithi117.stdout:6/119: symlink d3/df/l2b 0 2024-03-22T06:15:17.876 INFO:tasks.workunit.client.1.smithi117.stdout:6/120: stat d3/df/l2b 0 2024-03-22T06:15:17.876 INFO:tasks.workunit.client.1.smithi117.stdout:6/121: fdatasync d3/f20 0 2024-03-22T06:15:17.876 INFO:tasks.workunit.client.1.smithi117.stdout:6/122: fdatasync d3/f4 0 2024-03-22T06:15:17.877 INFO:tasks.workunit.client.1.smithi117.stdout:5/69: mknod dd/c17 0 2024-03-22T06:15:17.878 INFO:tasks.workunit.client.1.smithi117.stdout:3/62: mkdir d9/d1a 0 2024-03-22T06:15:17.879 INFO:tasks.workunit.client.1.smithi117.stdout:1/117: write d1/d14/d16/f1a [76495,123540] 0 2024-03-22T06:15:17.879 INFO:tasks.workunit.client.1.smithi117.stdout:1/118: read - d1/f1b zero size 2024-03-22T06:15:17.879 INFO:tasks.workunit.client.1.smithi117.stdout:1/119: write d1/d14/d16/f1a [1040667,117029] 0 2024-03-22T06:15:17.879 INFO:tasks.workunit.client.0.smithi006.stdout:3/909: truncate d0/fcf 426315 0 2024-03-22T06:15:17.880 INFO:tasks.workunit.client.1.smithi117.stdout:6/123: mkdir d3/d7/da/de/d2c 0 2024-03-22T06:15:17.880 INFO:tasks.workunit.client.1.smithi117.stdout:8/115: write d1/d2/fa [1555829,105942] 0 2024-03-22T06:15:17.880 INFO:tasks.workunit.client.0.smithi006.stdout:4/885: symlink d6/d1b/d20/d21/l132 0 2024-03-22T06:15:17.882 INFO:tasks.workunit.client.0.smithi006.stdout:1/829: truncate d8/d9/d3a/d18/f62 2047560 0 2024-03-22T06:15:17.882 INFO:tasks.workunit.client.0.smithi006.stdout:1/830: truncate d8/d9/f35 3116635 0 2024-03-22T06:15:17.883 INFO:tasks.workunit.client.1.smithi117.stdout:5/70: mknod dd/df/d12/c18 0 2024-03-22T06:15:17.884 INFO:tasks.workunit.client.1.smithi117.stdout:3/63: truncate f8 1966072 0 2024-03-22T06:15:17.884 INFO:tasks.workunit.client.1.smithi117.stdout:3/64: write d9/f19 [44886,96056] 0 2024-03-22T06:15:17.884 INFO:tasks.workunit.client.1.smithi117.stdout:3/65: chown d9/l13 0 1 2024-03-22T06:15:17.885 INFO:tasks.workunit.client.1.smithi117.stdout:1/120: rmdir d1/d14/d16 39 2024-03-22T06:15:17.885 INFO:tasks.workunit.client.1.smithi117.stdout:1/121: read d1/f7 [1374012,85559] 0 2024-03-22T06:15:17.886 INFO:tasks.workunit.client.0.smithi006.stdout:3/910: mkdir d0/d7e/d13c 0 2024-03-22T06:15:17.888 INFO:tasks.workunit.client.1.smithi117.stdout:6/124: mkdir d3/d7/da/d16/d24/d2d 0 2024-03-22T06:15:17.888 INFO:tasks.workunit.client.1.smithi117.stdout:0/134: dread d1/d7/de/f23 [0,4194304] 0 2024-03-22T06:15:17.888 INFO:tasks.workunit.client.1.smithi117.stdout:0/135: chown d1/f1c 7864 1 2024-03-22T06:15:17.888 INFO:tasks.workunit.client.1.smithi117.stdout:0/136: write d1/d10/f27 [623308,10160] 0 2024-03-22T06:15:17.888 INFO:tasks.workunit.client.1.smithi117.stdout:0/137: stat d1/d3 0 2024-03-22T06:15:17.889 INFO:tasks.workunit.client.1.smithi117.stdout:9/135: dwrite d4/d7/fd [0,4194304] 0 2024-03-22T06:15:17.889 INFO:tasks.workunit.client.0.smithi006.stdout:4/886: dread d6/d1b/d20/d21/d77/df1/ffb [0,4194304] 0 2024-03-22T06:15:17.889 INFO:tasks.workunit.client.0.smithi006.stdout:4/887: stat d6/d1b/d20/d6b/d60/db1 0 2024-03-22T06:15:17.890 INFO:tasks.workunit.client.1.smithi117.stdout:2/93: dwrite d2/d5/d8/d10/d11/f15 [0,4194304] 0 2024-03-22T06:15:17.892 INFO:tasks.workunit.client.1.smithi117.stdout:9/136: dread d4/f1e [0,4194304] 0 2024-03-22T06:15:17.892 INFO:tasks.workunit.client.1.smithi117.stdout:9/137: chown d4/d7/fd 5 1 2024-03-22T06:15:17.892 INFO:tasks.workunit.client.0.smithi006.stdout:1/831: dread d8/d9/d13/d39/d60/fd4 [0,4194304] 0 2024-03-22T06:15:17.893 INFO:tasks.workunit.client.1.smithi117.stdout:7/117: dwrite d8/d9/da/d10/f13 [0,4194304] 0 2024-03-22T06:15:17.893 INFO:tasks.workunit.client.1.smithi117.stdout:5/71: dread fa [0,4194304] 0 2024-03-22T06:15:17.893 INFO:tasks.workunit.client.1.smithi117.stdout:7/118: chown d8/d9/da/d10/f19 155679 1 2024-03-22T06:15:17.894 INFO:tasks.workunit.client.1.smithi117.stdout:6/125: creat d3/df/f2e x:0 0 0 2024-03-22T06:15:17.896 INFO:tasks.workunit.client.0.smithi006.stdout:1/832: creat d8/d9/d10/f119 x:0 0 0 2024-03-22T06:15:17.897 INFO:tasks.workunit.client.1.smithi117.stdout:3/66: fdatasync d9/f19 0 2024-03-22T06:15:17.898 INFO:tasks.workunit.client.1.smithi117.stdout:0/138: mknod d1/d10/d18/c2a 0 2024-03-22T06:15:17.898 INFO:tasks.workunit.client.0.smithi006.stdout:3/911: dread d0/d4/dbb/df7/d5f/d90/da7/de7/d33/f4d [0,4194304] 0 2024-03-22T06:15:17.898 INFO:tasks.workunit.client.0.smithi006.stdout:3/912: stat d0/d4/dbb/df7/d5f/fd4 0 2024-03-22T06:15:17.898 INFO:tasks.workunit.client.0.smithi006.stdout:3/913: readlink d0/d4/d31/l6c 0 2024-03-22T06:15:17.899 INFO:tasks.workunit.client.0.smithi006.stdout:4/888: getdents d6/d1b/d20/d21/d2a/dc9 0 2024-03-22T06:15:17.899 INFO:tasks.workunit.client.0.smithi006.stdout:4/889: chown d6/d1b/d20/d63/d93/f12c 730 1 2024-03-22T06:15:17.899 INFO:tasks.workunit.client.0.smithi006.stdout:4/890: stat d6/d1b/d20/d6b 0 2024-03-22T06:15:17.899 INFO:tasks.workunit.client.0.smithi006.stdout:4/891: read d6/d1b/d20/d21/d2a/dc9/fcb [1726831,88811] 0 2024-03-22T06:15:17.900 INFO:tasks.workunit.client.1.smithi117.stdout:8/116: dwrite d1/d2/d11/f2a [0,4194304] 0 2024-03-22T06:15:17.900 INFO:tasks.workunit.client.1.smithi117.stdout:8/117: read d1/d2/f7 [1306536,45305] 0 2024-03-22T06:15:17.900 INFO:tasks.workunit.client.1.smithi117.stdout:0/139: write d1/d10/f27 [552517,8727] 0 2024-03-22T06:15:17.901 INFO:tasks.workunit.client.0.smithi006.stdout:3/914: write d0/d4/d31/fcc [763168,11997] 0 2024-03-22T06:15:17.901 INFO:tasks.workunit.client.0.smithi006.stdout:3/915: truncate d0/f3 4453332 0 2024-03-22T06:15:17.901 INFO:tasks.workunit.client.0.smithi006.stdout:3/916: write d0/f56 [3324767,50201] 0 2024-03-22T06:15:17.903 INFO:tasks.workunit.client.0.smithi006.stdout:1/833: creat d8/d9/d13/d39/d60/d8f/d10a/d59/db1/f11a x:0 0 0 2024-03-22T06:15:17.904 INFO:tasks.workunit.client.0.smithi006.stdout:1/834: creat d8/d9/d3a/f11b x:0 0 0 2024-03-22T06:15:17.904 INFO:tasks.workunit.client.1.smithi117.stdout:7/119: write f7 [332642,94232] 0 2024-03-22T06:15:17.906 INFO:tasks.workunit.client.0.smithi006.stdout:4/892: mkdir d6/d1b/d20/d21/d133 0 2024-03-22T06:15:17.908 INFO:tasks.workunit.client.1.smithi117.stdout:8/118: dread d1/f1e [0,4194304] 0 2024-03-22T06:15:17.908 INFO:tasks.workunit.client.1.smithi117.stdout:8/119: chown d1/d2/d9/lc 29335217 1 2024-03-22T06:15:17.917 INFO:tasks.workunit.client.0.smithi006.stdout:3/917: creat d0/d8/d88/ddf/df0/d112/f13d x:0 0 0 2024-03-22T06:15:17.922 INFO:tasks.workunit.client.0.smithi006.stdout:1/835: creat d8/d9/d13/d24/d32/d34/f11c x:0 0 0 2024-03-22T06:15:17.925 INFO:tasks.workunit.client.0.smithi006.stdout:4/893: mkdir d6/d13/d3b/d134 0 2024-03-22T06:15:17.927 INFO:tasks.workunit.client.0.smithi006.stdout:1/836: creat d8/f11d x:0 0 0 2024-03-22T06:15:17.927 INFO:tasks.workunit.client.0.smithi006.stdout:1/837: creat d8/d33/f11e x:0 0 0 2024-03-22T06:15:17.931 INFO:tasks.workunit.client.1.smithi117.stdout:5/72: mkdir dd/d19 0 2024-03-22T06:15:17.933 INFO:tasks.workunit.client.0.smithi006.stdout:6/816: dwrite d6/d1d/d5c/d81/dab/fd8 [0,4194304] 0 2024-03-22T06:15:17.933 INFO:tasks.workunit.client.0.smithi006.stdout:6/817: truncate d6/d7c/d92/fc8 977798 0 2024-03-22T06:15:17.935 INFO:tasks.workunit.client.1.smithi117.stdout:3/67: link f6 d9/d1a/f1b 0 2024-03-22T06:15:17.935 INFO:tasks.workunit.client.1.smithi117.stdout:3/68: truncate f7 5293126 0 2024-03-22T06:15:17.935 INFO:tasks.workunit.client.0.smithi006.stdout:6/818: write d6/d1d/d5c/d89/f8b [221142,116731] 0 2024-03-22T06:15:17.936 INFO:tasks.workunit.client.0.smithi006.stdout:6/819: chown d6/d1d/dc6/fe2 16164 1 2024-03-22T06:15:17.936 INFO:tasks.workunit.client.0.smithi006.stdout:4/894: read d6/d1b/d20/d21/d2a/dc9/fcb [4527880,9496] 0 2024-03-22T06:15:17.936 INFO:tasks.workunit.client.1.smithi117.stdout:6/126: dwrite d3/f5 [0,4194304] 0 2024-03-22T06:15:17.936 INFO:tasks.workunit.client.1.smithi117.stdout:6/127: chown d3/d7/da/de 261130680 1 2024-03-22T06:15:17.943 INFO:tasks.workunit.client.0.smithi006.stdout:3/918: dwrite d0/d4/dbb/df7/f96 [4194304,4194304] 0 2024-03-22T06:15:17.946 INFO:tasks.workunit.client.1.smithi117.stdout:0/140: rename d1/d10/d18/d26/dc/d29 to d1/d3/d2b 0 2024-03-22T06:15:17.946 INFO:tasks.workunit.client.1.smithi117.stdout:8/120: rename d1/d2 to d1/d2/d9/d2b 22 2024-03-22T06:15:17.948 INFO:tasks.workunit.client.0.smithi006.stdout:4/895: creat d6/d1b/d46/d101/f135 x:0 0 0 2024-03-22T06:15:17.951 INFO:tasks.workunit.client.1.smithi117.stdout:6/128: dread d3/d7/da/fc [0,4194304] 0 2024-03-22T06:15:17.954 INFO:tasks.workunit.client.1.smithi117.stdout:7/120: link d8/d9/da/l17 d8/d9/d1a/l21 0 2024-03-22T06:15:17.955 INFO:tasks.workunit.client.0.smithi006.stdout:3/919: rmdir d0/d8/d9 39 2024-03-22T06:15:17.957 INFO:tasks.workunit.client.1.smithi117.stdout:2/94: rmdir d2/d5/d8/d10/d11 39 2024-03-22T06:15:17.960 INFO:tasks.workunit.client.0.smithi006.stdout:4/896: getdents d6/d1b/d20/d63/d93/d9a/d8f/da1 0 2024-03-22T06:15:17.962 INFO:tasks.workunit.client.0.smithi006.stdout:3/920: fsync d0/d8/d1c/d49/f67 0 2024-03-22T06:15:17.964 INFO:tasks.workunit.client.0.smithi006.stdout:4/897: link d6/d1b/d20/d63/f116 d6/d1b/d20/d6b/d60/db1/f136 0 2024-03-22T06:15:17.964 INFO:tasks.workunit.client.0.smithi006.stdout:4/898: write d6/d1b/d20/d63/d93/db8/dbc/f109 [178554,12743] 0 2024-03-22T06:15:17.966 INFO:tasks.workunit.client.0.smithi006.stdout:4/899: unlink d6/d1b/d20/d21/d2a/d43/da8/cd9 0 2024-03-22T06:15:17.967 INFO:tasks.workunit.client.0.smithi006.stdout:4/900: dread - d6/d1b/d20/d21/d2a/d43/d4b/f120 zero size 2024-03-22T06:15:17.967 INFO:tasks.workunit.client.0.smithi006.stdout:4/901: creat d6/d1b/d20/d21/d77/df1/f137 x:0 0 0 2024-03-22T06:15:17.967 INFO:tasks.workunit.client.0.smithi006.stdout:4/902: creat d6/d1b/d20/d21/d2a/d43/d4b/f138 x:0 0 0 2024-03-22T06:15:17.967 INFO:tasks.workunit.client.0.smithi006.stdout:4/903: chown d6/d13/c36 8522 1 2024-03-22T06:15:17.967 INFO:tasks.workunit.client.1.smithi117.stdout:9/138: rmdir d4 39 2024-03-22T06:15:17.972 INFO:tasks.workunit.client.1.smithi117.stdout:6/129: creat d3/d7/da/de/d2c/f2f x:0 0 0 2024-03-22T06:15:17.972 INFO:tasks.workunit.client.1.smithi117.stdout:2/95: write d2/fc [2079302,25499] 0 2024-03-22T06:15:17.972 INFO:tasks.workunit.client.1.smithi117.stdout:2/96: write d2/f4 [431656,104429] 0 2024-03-22T06:15:17.972 INFO:tasks.workunit.client.1.smithi117.stdout:2/97: dread - d2/d14/f18 zero size 2024-03-22T06:15:17.972 INFO:tasks.workunit.client.0.smithi006.stdout:4/904: rename d6/d1b/d20/d21/d77/lbe to d6/d1b/d20/d6b/d60/db1/l139 0 2024-03-22T06:15:17.974 INFO:tasks.workunit.client.1.smithi117.stdout:9/139: rename d4/d7/f23 to d4/d8/f2c 0 2024-03-22T06:15:17.974 INFO:tasks.workunit.client.1.smithi117.stdout:9/140: chown d4/d8/f2b 0 1 2024-03-22T06:15:17.975 INFO:tasks.workunit.client.1.smithi117.stdout:6/130: mkdir d3/d7/da/d16/d24/d2d/d30 0 2024-03-22T06:15:17.975 INFO:tasks.workunit.client.1.smithi117.stdout:6/131: stat d3/d7 0 2024-03-22T06:15:17.977 INFO:tasks.workunit.client.0.smithi006.stdout:4/905: rename d6/d1b/d46/f58 to d6/d1b/d20/d6b/d60/db1/f13a 0 2024-03-22T06:15:17.978 INFO:tasks.workunit.client.1.smithi117.stdout:9/141: dread d4/f21 [0,4194304] 0 2024-03-22T06:15:17.981 INFO:tasks.workunit.client.0.smithi006.stdout:1/838: dwrite d8/d9/f8d [0,4194304] 0 2024-03-22T06:15:17.981 INFO:tasks.workunit.client.0.smithi006.stdout:4/906: truncate d6/d1b/d20/d21/d77/fa6 3128846 0 2024-03-22T06:15:17.981 INFO:tasks.workunit.client.1.smithi117.stdout:3/69: dwrite f2 [0,4194304] 0 2024-03-22T06:15:17.982 INFO:tasks.workunit.client.0.smithi006.stdout:3/921: dread d0/d8/d88/dfa/f86 [0,4194304] 0 2024-03-22T06:15:17.982 INFO:tasks.workunit.client.1.smithi117.stdout:6/132: mkdir d3/df/d17/d31 0 2024-03-22T06:15:17.982 INFO:tasks.workunit.client.1.smithi117.stdout:6/133: dread - d3/d7/da/de/d2c/f2f zero size 2024-03-22T06:15:17.983 INFO:tasks.workunit.client.1.smithi117.stdout:9/142: rename d4/d7/fd to d4/d7/f2d 0 2024-03-22T06:15:17.985 INFO:tasks.workunit.client.1.smithi117.stdout:3/70: symlink d9/d1a/l1c 0 2024-03-22T06:15:17.987 INFO:tasks.workunit.client.1.smithi117.stdout:6/134: unlink d3/df/d17/l1a 0 2024-03-22T06:15:17.989 INFO:tasks.workunit.client.0.smithi006.stdout:4/907: unlink d6/d1b/d20/d63/d93/d9a/d8f/da1/ffa 0 2024-03-22T06:15:17.990 INFO:tasks.workunit.client.1.smithi117.stdout:9/143: stat d4/l9 0 2024-03-22T06:15:17.990 INFO:tasks.workunit.client.1.smithi117.stdout:9/144: stat d4/d7/f1b 0 2024-03-22T06:15:17.991 INFO:tasks.workunit.client.0.smithi006.stdout:6/820: dwrite d6/d1b/d2c/fd1 [0,4194304] 0 2024-03-22T06:15:17.991 INFO:tasks.workunit.client.0.smithi006.stdout:6/821: read d6/d1b/d2c/f8c [1474997,23011] 0 2024-03-22T06:15:17.991 INFO:tasks.workunit.client.0.smithi006.stdout:6/822: dread - d6/d24/d44/f90 zero size 2024-03-22T06:15:17.991 INFO:tasks.workunit.client.0.smithi006.stdout:3/922: mknod d0/d4/c13e 0 2024-03-22T06:15:17.992 INFO:tasks.workunit.client.1.smithi117.stdout:5/73: dwrite f2 [0,4194304] 0 2024-03-22T06:15:17.992 INFO:tasks.workunit.client.1.smithi117.stdout:8/121: dwrite d1/d2/d9/f18 [0,4194304] 0 2024-03-22T06:15:17.992 INFO:tasks.workunit.client.1.smithi117.stdout:8/122: chown d1/d2/c27 13913 1 2024-03-22T06:15:17.992 INFO:tasks.workunit.client.1.smithi117.stdout:0/141: dwrite d1/d7/ff [0,4194304] 0 2024-03-22T06:15:17.992 INFO:tasks.workunit.client.1.smithi117.stdout:2/98: dwrite d2/f3 [4194304,4194304] 0 2024-03-22T06:15:17.992 INFO:tasks.workunit.client.1.smithi117.stdout:3/71: getdents d9 0 2024-03-22T06:15:17.992 INFO:tasks.workunit.client.1.smithi117.stdout:3/72: write d9/f11 [1226742,112258] 0 2024-03-22T06:15:17.993 INFO:tasks.workunit.client.0.smithi006.stdout:4/908: fsync d6/d1b/d20/d63/d93/db8/dbc/f109 0 2024-03-22T06:15:17.993 INFO:tasks.workunit.client.1.smithi117.stdout:3/73: dread d9/fa [0,4194304] 0 2024-03-22T06:15:17.993 INFO:tasks.workunit.client.1.smithi117.stdout:6/135: mkdir d3/d7/da/d32 0 2024-03-22T06:15:17.994 INFO:tasks.workunit.client.1.smithi117.stdout:6/136: write d3/f4 [1519980,118558] 0 2024-03-22T06:15:17.995 INFO:tasks.workunit.client.1.smithi117.stdout:9/145: unlink d4/d8/f2b 0 2024-03-22T06:15:17.995 INFO:tasks.workunit.client.1.smithi117.stdout:9/146: dread - d4/d7/f2a zero size 2024-03-22T06:15:17.995 INFO:tasks.workunit.client.1.smithi117.stdout:9/147: truncate d4/d8/f13 687598 0 2024-03-22T06:15:17.998 INFO:tasks.workunit.client.0.smithi006.stdout:6/823: unlink d6/d1d/d32/c49 0 2024-03-22T06:15:17.998 INFO:tasks.workunit.client.0.smithi006.stdout:6/824: getdents d6/d24/d44/df3 0 2024-03-22T06:15:17.998 INFO:tasks.workunit.client.0.smithi006.stdout:6/825: write d6/d1b/d2c/f72 [4181762,120422] 0 2024-03-22T06:15:18.000 INFO:tasks.workunit.client.1.smithi117.stdout:5/74: unlink dd/c17 0 2024-03-22T06:15:18.001 INFO:tasks.workunit.client.0.smithi006.stdout:3/923: rmdir d0/d8/d9 39 2024-03-22T06:15:18.001 INFO:tasks.workunit.client.0.smithi006.stdout:3/924: dread - d0/d8/d1c/f76 zero size 2024-03-22T06:15:18.009 INFO:tasks.workunit.client.1.smithi117.stdout:8/123: symlink d1/l2c 0 2024-03-22T06:15:18.012 INFO:tasks.workunit.client.0.smithi006.stdout:3/925: symlink d0/d4/dbb/df7/d5f/d90/da7/l13f 0 2024-03-22T06:15:18.014 INFO:tasks.workunit.client.0.smithi006.stdout:3/926: creat d0/f140 x:0 0 0 2024-03-22T06:15:18.016 INFO:tasks.workunit.client.0.smithi006.stdout:6/826: dwrite d6/d1d/d99/dcd/ff7 [0,4194304] 0 2024-03-22T06:15:18.016 INFO:tasks.workunit.client.0.smithi006.stdout:6/827: dread - d6/d1b/d2c/fca zero size 2024-03-22T06:15:18.016 INFO:tasks.workunit.client.0.smithi006.stdout:6/828: readlink d6/d1d/d5c/le0 0 2024-03-22T06:15:18.016 INFO:tasks.workunit.client.0.smithi006.stdout:6/829: write d6/d77/fec [899348,87476] 0 2024-03-22T06:15:18.016 INFO:tasks.workunit.client.0.smithi006.stdout:6/830: readlink d6/d1d/d32/l3f 0 2024-03-22T06:15:18.016 INFO:tasks.workunit.client.0.smithi006.stdout:6/831: write d6/d24/f54 [1086543,95092] 0 2024-03-22T06:15:18.018 INFO:tasks.workunit.client.0.smithi006.stdout:3/927: symlink d0/d8/d88/l141 0 2024-03-22T06:15:18.020 INFO:tasks.workunit.client.0.smithi006.stdout:3/928: dread d0/d4/d31/fcc [0,4194304] 0 2024-03-22T06:15:18.021 INFO:tasks.workunit.client.0.smithi006.stdout:6/832: unlink d6/d1d/d32/c70 0 2024-03-22T06:15:18.022 INFO:tasks.workunit.client.1.smithi117.stdout:2/99: getdents d2/d5/d8/d10 0 2024-03-22T06:15:18.023 INFO:tasks.workunit.client.0.smithi006.stdout:3/929: truncate d0/d8/d1c/d49/d79/f99 497806 0 2024-03-22T06:15:18.025 INFO:tasks.workunit.client.0.smithi006.stdout:3/930: dread d0/d4/d31/fcc [0,4194304] 0 2024-03-22T06:15:18.025 INFO:tasks.workunit.client.0.smithi006.stdout:3/931: write d0/d8/d88/dfa/d58/f120 [7461361,73842] 0 2024-03-22T06:15:18.026 INFO:tasks.workunit.client.1.smithi117.stdout:3/74: creat d9/f1d x:0 0 0 2024-03-22T06:15:18.027 INFO:tasks.workunit.client.0.smithi006.stdout:3/932: dread d0/d4/dbb/df7/d68/f6b [0,4194304] 0 2024-03-22T06:15:18.027 INFO:tasks.workunit.client.0.smithi006.stdout:3/933: chown d0/d4/dbb/df7/c127 1147963 1 2024-03-22T06:15:18.027 INFO:tasks.workunit.client.0.smithi006.stdout:6/833: mknod d6/d1d/d5c/cfc 0 2024-03-22T06:15:18.027 INFO:tasks.workunit.client.0.smithi006.stdout:3/934: dread d0/d4/dbb/df7/d5f/d90/da7/de7/d33/f65 [0,4194304] 0 2024-03-22T06:15:18.028 INFO:tasks.workunit.client.0.smithi006.stdout:3/935: creat d0/d8/d1c/d95/f142 x:0 0 0 2024-03-22T06:15:18.028 INFO:tasks.workunit.client.0.smithi006.stdout:3/936: write d0/d4/dbb/df7/d68/f75 [301413,72331] 0 2024-03-22T06:15:18.028 INFO:tasks.workunit.client.0.smithi006.stdout:3/937: stat d0/d8/d88/ddf/d102/l113 0 2024-03-22T06:15:18.028 INFO:tasks.workunit.client.1.smithi117.stdout:9/148: creat d4/d7/d27/f2e x:0 0 0 2024-03-22T06:15:18.029 INFO:tasks.workunit.client.0.smithi006.stdout:6/834: creat d6/d1b/dae/dbc/ffd x:0 0 0 2024-03-22T06:15:18.030 INFO:tasks.workunit.client.0.smithi006.stdout:6/835: write d6/d6f/d94/fa9 [34363,21680] 0 2024-03-22T06:15:18.031 INFO:tasks.workunit.client.1.smithi117.stdout:5/75: truncate f7 1282440 0 2024-03-22T06:15:18.031 INFO:tasks.workunit.client.1.smithi117.stdout:5/76: fdatasync fa 0 2024-03-22T06:15:18.036 INFO:tasks.workunit.client.0.smithi006.stdout:6/836: rename d6/d24/c57 to d6/d1d/d99/de9/cfe 0 2024-03-22T06:15:18.036 INFO:tasks.workunit.client.0.smithi006.stdout:6/837: write d6/d24/ff4 [527960,82086] 0 2024-03-22T06:15:18.036 INFO:tasks.workunit.client.0.smithi006.stdout:6/838: fsync d6/d24/d44/f4d 0 2024-03-22T06:15:18.040 INFO:tasks.workunit.client.0.smithi006.stdout:6/839: unlink d6/l1a 0 2024-03-22T06:15:18.042 INFO:tasks.workunit.client.1.smithi117.stdout:8/124: creat d1/d19/f2d x:0 0 0 2024-03-22T06:15:18.044 INFO:tasks.workunit.client.0.smithi006.stdout:3/938: dwrite d0/d4/dbb/df7/d5f/f7d [0,4194304] 0 2024-03-22T06:15:18.044 INFO:tasks.workunit.client.0.smithi006.stdout:3/939: chown d0/d4 59 1 2024-03-22T06:15:18.044 INFO:tasks.workunit.client.0.smithi006.stdout:3/940: fdatasync d0/d4/d3a/f115 0 2024-03-22T06:15:18.045 INFO:tasks.workunit.client.1.smithi117.stdout:0/142: rename d1/d7 to d1/d3/d2c 0 2024-03-22T06:15:18.046 INFO:tasks.workunit.client.0.smithi006.stdout:3/941: dread d0/d7e/fa0 [0,4194304] 0 2024-03-22T06:15:18.046 INFO:tasks.workunit.client.0.smithi006.stdout:3/942: creat d0/d8/d88/dfa/dcb/d108/f143 x:0 0 0 2024-03-22T06:15:18.046 INFO:tasks.workunit.client.0.smithi006.stdout:7/805: sync 2024-03-22T06:15:18.046 INFO:tasks.workunit.client.0.smithi006.stdout:7/806: fdatasync d5/d1e/d29/d1f/d2d/d41/f44 0 2024-03-22T06:15:18.050 INFO:tasks.workunit.client.1.smithi117.stdout:4/143: sync 2024-03-22T06:15:18.051 INFO:tasks.workunit.client.0.smithi006.stdout:3/943: rename d0/d4/dbb/df7/d68/fce to d0/d8/d1c/d72/f144 0 2024-03-22T06:15:18.051 INFO:tasks.workunit.client.0.smithi006.stdout:3/944: fdatasync d0/d4/dbb/df7/d5f/d83/f11e 0 2024-03-22T06:15:18.052 INFO:tasks.workunit.client.1.smithi117.stdout:6/137: getdents d3/df/d17 0 2024-03-22T06:15:18.052 INFO:tasks.workunit.client.1.smithi117.stdout:6/138: chown d3/d7/da/d16/f22 1944958 1 2024-03-22T06:15:18.054 INFO:tasks.workunit.client.0.smithi006.stdout:7/807: creat d5/d1e/d29/d1f/d2d/d41/db7/f106 x:0 0 0 2024-03-22T06:15:18.054 INFO:tasks.workunit.client.0.smithi006.stdout:7/808: chown d5/d1e/d29/d1f/d2d/d61/f6d 5 1 2024-03-22T06:15:18.054 INFO:tasks.workunit.client.1.smithi117.stdout:9/149: chown d4/c18 43585149 1 2024-03-22T06:15:18.055 INFO:tasks.workunit.client.1.smithi117.stdout:9/150: dread - d4/d8/f25 zero size 2024-03-22T06:15:18.058 INFO:tasks.workunit.client.1.smithi117.stdout:5/77: mkdir dd/d16/d1a 0 2024-03-22T06:15:18.058 INFO:tasks.workunit.client.1.smithi117.stdout:5/78: chown dd/fe 877646 1 2024-03-22T06:15:18.058 INFO:tasks.workunit.client.1.smithi117.stdout:5/79: stat dd 0 2024-03-22T06:15:18.058 INFO:tasks.workunit.client.0.smithi006.stdout:3/945: truncate d0/d8/f1a 288436 0 2024-03-22T06:15:18.059 INFO:tasks.workunit.client.0.smithi006.stdout:3/946: chown d0/d8/f17 11490 1 2024-03-22T06:15:18.060 INFO:tasks.workunit.client.1.smithi117.stdout:0/143: rename d1/d10/d18/d26/c1d to d1/d10/d18/d26/c2d 0 2024-03-22T06:15:18.060 INFO:tasks.workunit.client.1.smithi117.stdout:0/144: dread - d1/f1c zero size 2024-03-22T06:15:18.064 INFO:tasks.workunit.client.1.smithi117.stdout:3/75: truncate f2 620517 0 2024-03-22T06:15:18.064 INFO:tasks.workunit.client.1.smithi117.stdout:8/125: dwrite d1/d2/d11/f2a [0,4194304] 0 2024-03-22T06:15:18.064 INFO:tasks.workunit.client.1.smithi117.stdout:3/76: write d9/f19 [833319,53732] 0 2024-03-22T06:15:18.065 INFO:tasks.workunit.client.0.smithi006.stdout:7/809: dread d5/d1e/d29/d1f/d2d/d55/d84/fa0 [0,4194304] 0 2024-03-22T06:15:18.065 INFO:tasks.workunit.client.1.smithi117.stdout:4/144: creat d0/d4/d2a/d2f/f31 x:0 0 0 2024-03-22T06:15:18.066 INFO:tasks.workunit.client.1.smithi117.stdout:9/151: mkdir d4/d7/d10/d2f 0 2024-03-22T06:15:18.067 INFO:tasks.workunit.client.1.smithi117.stdout:9/152: chown d4/d8 146 1 2024-03-22T06:15:18.067 INFO:tasks.workunit.client.0.smithi006.stdout:3/947: creat d0/d8/d1c/d49/de5/f145 x:0 0 0 2024-03-22T06:15:18.067 INFO:tasks.workunit.client.0.smithi006.stdout:3/948: fsync d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/f11c 0 2024-03-22T06:15:18.067 INFO:tasks.workunit.client.0.smithi006.stdout:3/949: truncate d0/d4/dbb/df7/f96 8809031 0 2024-03-22T06:15:18.069 INFO:tasks.workunit.client.1.smithi117.stdout:9/153: dread d4/f1e [0,4194304] 0 2024-03-22T06:15:18.069 INFO:tasks.workunit.client.1.smithi117.stdout:9/154: write d4/d7/d10/f28 [270757,112774] 0 2024-03-22T06:15:18.069 INFO:tasks.workunit.client.1.smithi117.stdout:9/155: fsync f2 0 2024-03-22T06:15:18.069 INFO:tasks.workunit.client.1.smithi117.stdout:9/156: truncate d4/d8/f13 1696256 0 2024-03-22T06:15:18.069 INFO:tasks.workunit.client.1.smithi117.stdout:5/80: creat dd/df/f1b x:0 0 0 2024-03-22T06:15:18.069 INFO:tasks.workunit.client.1.smithi117.stdout:0/145: rmdir d1/d10/d18/d26/dc 39 2024-03-22T06:15:18.070 INFO:tasks.workunit.client.1.smithi117.stdout:2/100: truncate d2/f3 4371987 0 2024-03-22T06:15:18.070 INFO:tasks.workunit.client.1.smithi117.stdout:2/101: truncate d2/d14/f18 925901 0 2024-03-22T06:15:18.070 INFO:tasks.workunit.client.1.smithi117.stdout:2/102: fsync f1 0 2024-03-22T06:15:18.071 INFO:tasks.workunit.client.0.smithi006.stdout:3/950: rename d0/d4/dbb/df7/d5f/f7d to d0/d4/dbb/df7/d5f/d90/da7/d97/f146 0 2024-03-22T06:15:18.074 INFO:tasks.workunit.client.1.smithi117.stdout:8/126: mknod d1/c2e 0 2024-03-22T06:15:18.075 INFO:tasks.workunit.client.0.smithi006.stdout:7/810: write d5/d1e/d29/d1f/d2d/d41/f7b [705175,16755] 0 2024-03-22T06:15:18.075 INFO:tasks.workunit.client.0.smithi006.stdout:7/811: readlink d5/l23 0 2024-03-22T06:15:18.076 INFO:tasks.workunit.client.0.smithi006.stdout:7/812: dread - d5/d1e/d63/d85/d80/f104 zero size 2024-03-22T06:15:18.078 INFO:tasks.workunit.client.1.smithi117.stdout:4/145: mkdir d0/d32 0 2024-03-22T06:15:18.078 INFO:tasks.workunit.client.1.smithi117.stdout:4/146: creat d0/d4/d2a/d2f/f33 x:0 0 0 2024-03-22T06:15:18.080 INFO:tasks.workunit.client.1.smithi117.stdout:5/81: symlink dd/l1c 0 2024-03-22T06:15:18.080 INFO:tasks.workunit.client.1.smithi117.stdout:5/82: dread - dd/df/f1b zero size 2024-03-22T06:15:18.080 INFO:tasks.workunit.client.1.smithi117.stdout:1/122: sync 2024-03-22T06:15:18.081 INFO:tasks.workunit.client.1.smithi117.stdout:2/103: symlink d2/d5/d8/l1a 0 2024-03-22T06:15:18.082 INFO:tasks.workunit.client.1.smithi117.stdout:2/104: fdatasync f1 0 2024-03-22T06:15:18.082 INFO:tasks.workunit.client.1.smithi117.stdout:6/139: dwrite d3/d7/da/de/d2c/f2f [0,4194304] 0 2024-03-22T06:15:18.082 INFO:tasks.workunit.client.1.smithi117.stdout:6/140: read - d3/df/f2e zero size 2024-03-22T06:15:18.083 INFO:tasks.workunit.client.1.smithi117.stdout:0/146: dread d1/d3/f12 [0,4194304] 0 2024-03-22T06:15:18.084 INFO:tasks.workunit.client.0.smithi006.stdout:6/840: dwrite d6/d24/f38 [0,4194304] 0 2024-03-22T06:15:18.084 INFO:tasks.workunit.client.1.smithi117.stdout:3/77: dwrite d9/d1a/f1b [0,4194304] 0 2024-03-22T06:15:18.085 INFO:tasks.workunit.client.1.smithi117.stdout:3/78: chown d9/c16 21 1 2024-03-22T06:15:18.087 INFO:tasks.workunit.client.0.smithi006.stdout:3/951: rename d0/d4/d53 to d0/d4/dbb/df7/d5f/d90/da7/de7/d147 0 2024-03-22T06:15:18.088 INFO:tasks.workunit.client.0.smithi006.stdout:3/952: creat d0/d8/d88/dfa/dcb/d108/f148 x:0 0 0 2024-03-22T06:15:18.088 INFO:tasks.workunit.client.0.smithi006.stdout:3/953: truncate d0/d4/dbb/df7/d5f/d83/fd2 115355 0 2024-03-22T06:15:18.088 INFO:tasks.workunit.client.1.smithi117.stdout:0/147: write d1/d3/f12 [115041,112181] 0 2024-03-22T06:15:18.089 INFO:tasks.workunit.client.1.smithi117.stdout:9/157: mknod d4/c30 0 2024-03-22T06:15:18.089 INFO:tasks.workunit.client.1.smithi117.stdout:9/158: truncate d4/d7/d10/f28 1286136 0 2024-03-22T06:15:18.092 INFO:tasks.workunit.client.1.smithi117.stdout:5/83: mknod dd/df/d10/c1d 0 2024-03-22T06:15:18.094 INFO:tasks.workunit.client.1.smithi117.stdout:1/123: creat d1/f25 x:0 0 0 2024-03-22T06:15:18.094 INFO:tasks.workunit.client.1.smithi117.stdout:1/124: write d1/f25 [998002,99959] 0 2024-03-22T06:15:18.095 INFO:tasks.workunit.client.1.smithi117.stdout:2/105: dread d2/d5/d8/d10/d11/f15 [0,4194304] 0 2024-03-22T06:15:18.098 INFO:tasks.workunit.client.0.smithi006.stdout:7/813: dwrite d5/d1e/d29/d1f/d2d/fda [0,4194304] 0 2024-03-22T06:15:18.098 INFO:tasks.workunit.client.0.smithi006.stdout:7/814: chown d5/d1e/d29/d1f/d2d/d41/f69 500116078 1 2024-03-22T06:15:18.098 INFO:tasks.workunit.client.1.smithi117.stdout:8/127: dwrite d1/d2/d11/f2a [0,4194304] 0 2024-03-22T06:15:18.099 INFO:tasks.workunit.client.1.smithi117.stdout:7/121: sync 2024-03-22T06:15:18.099 INFO:tasks.workunit.client.1.smithi117.stdout:7/122: creat d8/d9/da/d10/f22 x:0 0 0 2024-03-22T06:15:18.099 INFO:tasks.workunit.client.1.smithi117.stdout:7/123: chown d8/d9/da/d10 422667 1 2024-03-22T06:15:18.104 INFO:tasks.workunit.client.1.smithi117.stdout:4/147: mkdir d0/d19/d34 0 2024-03-22T06:15:18.104 INFO:tasks.workunit.client.1.smithi117.stdout:4/148: chown d0/d19/f22 204321 1 2024-03-22T06:15:18.104 INFO:tasks.workunit.client.1.smithi117.stdout:4/149: write d0/d4/d2a/f30 [347728,61814] 0 2024-03-22T06:15:18.106 INFO:tasks.workunit.client.1.smithi117.stdout:6/141: dwrite d3/d7/fd [0,4194304] 0 2024-03-22T06:15:18.107 INFO:tasks.workunit.client.1.smithi117.stdout:0/148: creat d1/d3/d2b/f2e x:0 0 0 2024-03-22T06:15:18.109 INFO:tasks.workunit.client.1.smithi117.stdout:9/159: symlink d4/d7/d10/l31 0 2024-03-22T06:15:18.111 INFO:tasks.workunit.client.1.smithi117.stdout:5/84: rename dd/d19 to dd/df/d1e 0 2024-03-22T06:15:18.113 INFO:tasks.workunit.client.1.smithi117.stdout:1/125: truncate d1/f8 3759223 0 2024-03-22T06:15:18.115 INFO:tasks.workunit.client.1.smithi117.stdout:8/128: symlink d1/d2/d11/l2f 0 2024-03-22T06:15:18.115 INFO:tasks.workunit.client.1.smithi117.stdout:8/129: write d1/d2/f1d [2169665,3830] 0 2024-03-22T06:15:18.116 INFO:tasks.workunit.client.0.smithi006.stdout:6/841: symlink d6/d1b/d2c/lff 0 2024-03-22T06:15:18.119 INFO:tasks.workunit.client.1.smithi117.stdout:7/124: mkdir d8/d9/da/d10/d23 0 2024-03-22T06:15:18.119 INFO:tasks.workunit.client.1.smithi117.stdout:7/125: fsync d8/fb 0 2024-03-22T06:15:18.121 INFO:tasks.workunit.client.0.smithi006.stdout:3/954: unlink d0/d8/f17 0 2024-03-22T06:15:18.123 INFO:tasks.workunit.client.1.smithi117.stdout:6/142: creat d3/df/d17/f33 x:0 0 0 2024-03-22T06:15:18.123 INFO:tasks.workunit.client.1.smithi117.stdout:6/143: chown d3/d7/da/d16/d24/d2d 1 1 2024-03-22T06:15:18.124 INFO:tasks.workunit.client.1.smithi117.stdout:0/149: link d1/d10/f11 d1/d3/d2b/f2f 0 2024-03-22T06:15:18.124 INFO:tasks.workunit.client.1.smithi117.stdout:0/150: write d1/f19 [245778,49281] 0 2024-03-22T06:15:18.126 INFO:tasks.workunit.client.0.smithi006.stdout:7/815: mknod d5/d1e/d29/d1f/d2d/d41/db7/dcb/c107 0 2024-03-22T06:15:18.126 INFO:tasks.workunit.client.0.smithi006.stdout:7/816: fsync d5/d1e/d29/d1f/d2d/d41/ff5 0 2024-03-22T06:15:18.127 INFO:tasks.workunit.client.1.smithi117.stdout:5/85: mkdir dd/d16/d1a/d1f 0 2024-03-22T06:15:18.127 INFO:tasks.workunit.client.1.smithi117.stdout:5/86: chown dd/fe 0 1 2024-03-22T06:15:18.127 INFO:tasks.workunit.client.1.smithi117.stdout:5/87: write f4 [1587396,116606] 0 2024-03-22T06:15:18.127 INFO:tasks.workunit.client.1.smithi117.stdout:5/88: chown dd/d16/d1a/d1f 1655784394 1 2024-03-22T06:15:18.127 INFO:tasks.workunit.client.1.smithi117.stdout:1/126: symlink d1/d14/l26 0 2024-03-22T06:15:18.128 INFO:tasks.workunit.client.0.smithi006.stdout:3/955: mknod d0/d4/d31/d98/c149 0 2024-03-22T06:15:18.128 INFO:tasks.workunit.client.1.smithi117.stdout:8/130: write d1/d2/d11/f2a [1885875,117376] 0 2024-03-22T06:15:18.129 INFO:tasks.workunit.client.1.smithi117.stdout:0/151: creat d1/d10/d18/d26/dc/f30 x:0 0 0 2024-03-22T06:15:18.130 INFO:tasks.workunit.client.0.smithi006.stdout:7/817: creat d5/d1e/d29/d1f/d2d/d55/f108 x:0 0 0 2024-03-22T06:15:18.130 INFO:tasks.workunit.client.0.smithi006.stdout:7/818: truncate d5/d1e/d29/d66/ff6 334675 0 2024-03-22T06:15:18.131 INFO:tasks.workunit.client.1.smithi117.stdout:5/89: creat dd/df/d1e/f20 x:0 0 0 2024-03-22T06:15:18.131 INFO:tasks.workunit.client.1.smithi117.stdout:5/90: write f4 [3063264,47477] 0 2024-03-22T06:15:18.132 INFO:tasks.workunit.client.1.smithi117.stdout:1/127: mkdir d1/d22/d27 0 2024-03-22T06:15:18.132 INFO:tasks.workunit.client.1.smithi117.stdout:1/128: write d1/f13 [409703,101970] 0 2024-03-22T06:15:18.133 INFO:tasks.workunit.client.0.smithi006.stdout:3/956: truncate d0/fcf 877133 0 2024-03-22T06:15:18.134 INFO:tasks.workunit.client.1.smithi117.stdout:6/144: dread d3/d7/da/d16/f22 [0,4194304] 0 2024-03-22T06:15:18.134 INFO:tasks.workunit.client.1.smithi117.stdout:6/145: fdatasync d3/f5 0 2024-03-22T06:15:18.134 INFO:tasks.workunit.client.1.smithi117.stdout:8/131: creat d1/f30 x:0 0 0 2024-03-22T06:15:18.136 INFO:tasks.workunit.client.0.smithi006.stdout:7/819: dread d5/d1e/d29/d1f/d2d/f51 [0,4194304] 0 2024-03-22T06:15:18.136 INFO:tasks.workunit.client.0.smithi006.stdout:3/957: mknod d0/d4/dbb/df7/d5f/d90/da7/de7/d147/c14a 0 2024-03-22T06:15:18.136 INFO:tasks.workunit.client.1.smithi117.stdout:0/152: symlink d1/d10/l31 0 2024-03-22T06:15:18.136 INFO:tasks.workunit.client.1.smithi117.stdout:3/79: dwrite d9/f11 [0,4194304] 0 2024-03-22T06:15:18.137 INFO:tasks.workunit.client.1.smithi117.stdout:5/91: mknod dd/d16/d1a/c21 0 2024-03-22T06:15:18.137 INFO:tasks.workunit.client.1.smithi117.stdout:5/92: write fa [2853082,29048] 0 2024-03-22T06:15:18.138 INFO:tasks.workunit.client.0.smithi006.stdout:7/820: symlink d5/d1e/d29/d1f/d2d/df7/l109 0 2024-03-22T06:15:18.139 INFO:tasks.workunit.client.0.smithi006.stdout:3/958: truncate d0/d8/d88/ddf/df0/f101 1077978 0 2024-03-22T06:15:18.140 INFO:tasks.workunit.client.1.smithi117.stdout:2/106: dwrite d2/f3 [4194304,4194304] 0 2024-03-22T06:15:18.140 INFO:tasks.workunit.client.1.smithi117.stdout:6/146: creat d3/df/d17/d31/f34 x:0 0 0 2024-03-22T06:15:18.140 INFO:tasks.workunit.client.1.smithi117.stdout:6/147: write d3/d7/f1b [197044,60077] 0 2024-03-22T06:15:18.141 INFO:tasks.workunit.client.1.smithi117.stdout:4/150: dwrite d0/f25 [0,4194304] 0 2024-03-22T06:15:18.142 INFO:tasks.workunit.client.1.smithi117.stdout:4/151: fsync d0/d4/d2a/d2f/f33 0 2024-03-22T06:15:18.142 INFO:tasks.workunit.client.1.smithi117.stdout:4/152: dread - d0/d4/d2a/d2f/f31 zero size 2024-03-22T06:15:18.142 INFO:tasks.workunit.client.0.smithi006.stdout:7/821: creat d5/d1e/d29/d66/f10a x:0 0 0 2024-03-22T06:15:18.143 INFO:tasks.workunit.client.1.smithi117.stdout:8/132: unlink d1/d2/f7 0 2024-03-22T06:15:18.143 INFO:tasks.workunit.client.1.smithi117.stdout:8/133: chown d1/c26 6753 1 2024-03-22T06:15:18.144 INFO:tasks.workunit.client.0.smithi006.stdout:6/842: dwrite d6/f18 [0,4194304] 0 2024-03-22T06:15:18.144 INFO:tasks.workunit.client.1.smithi117.stdout:9/160: dread d4/d7/d10/f28 [0,4194304] 0 2024-03-22T06:15:18.145 INFO:tasks.workunit.client.0.smithi006.stdout:3/959: mkdir d0/d4/dbb/df7/d5f/d90/da7/de7/d147/de0/d14b 0 2024-03-22T06:15:18.148 INFO:tasks.workunit.client.1.smithi117.stdout:0/153: creat d1/d10/f32 x:0 0 0 2024-03-22T06:15:18.156 INFO:tasks.workunit.client.1.smithi117.stdout:2/107: dread d2/f3 [4194304,4194304] 0 2024-03-22T06:15:18.160 INFO:tasks.workunit.client.1.smithi117.stdout:5/93: link dd/fe dd/d16/f22 0 2024-03-22T06:15:18.160 INFO:tasks.workunit.client.1.smithi117.stdout:5/94: chown dd/df/l15 889225 1 2024-03-22T06:15:18.164 INFO:tasks.workunit.client.1.smithi117.stdout:6/148: mknod d3/d7/d28/c35 0 2024-03-22T06:15:18.177 INFO:tasks.workunit.client.1.smithi117.stdout:9/161: unlink d4/l9 0 2024-03-22T06:15:18.177 INFO:tasks.workunit.client.1.smithi117.stdout:9/162: truncate d4/d8/f25 425994 0 2024-03-22T06:15:18.177 INFO:tasks.workunit.client.1.smithi117.stdout:3/80: dwrite f7 [0,4194304] 0 2024-03-22T06:15:18.178 INFO:tasks.workunit.client.1.smithi117.stdout:2/108: mkdir d2/d1b 0 2024-03-22T06:15:18.178 INFO:tasks.workunit.client.1.smithi117.stdout:2/109: fsync d2/d14/f18 0 2024-03-22T06:15:18.178 INFO:tasks.workunit.client.1.smithi117.stdout:2/110: chown d2/fc 3 1 2024-03-22T06:15:18.178 INFO:tasks.workunit.client.1.smithi117.stdout:2/111: chown d2/d5/d8/la 0 1 2024-03-22T06:15:18.179 INFO:tasks.workunit.client.1.smithi117.stdout:0/154: dwrite d1/d3/d2c/de/f23 [0,4194304] 0 2024-03-22T06:15:18.180 INFO:tasks.workunit.client.0.smithi006.stdout:3/960: rename d0/d4/d31 to d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d14c 0 2024-03-22T06:15:18.180 INFO:tasks.workunit.client.0.smithi006.stdout:3/961: chown d0/d8/d9/f32 893453389 1 2024-03-22T06:15:18.180 INFO:tasks.workunit.client.0.smithi006.stdout:7/822: dwrite d5/d1e/d29/d1f/d2d/f67 [4194304,4194304] 0 2024-03-22T06:15:18.180 INFO:tasks.workunit.client.0.smithi006.stdout:7/823: readlink d5/d1e/l4d 0 2024-03-22T06:15:18.180 INFO:tasks.workunit.client.0.smithi006.stdout:7/824: chown d5/d1e/d29/d1f/d2d/d41/c5e 37865 1 2024-03-22T06:15:18.180 INFO:tasks.workunit.client.0.smithi006.stdout:6/843: dwrite d6/d24/d44/f90 [0,4194304] 0 2024-03-22T06:15:18.181 INFO:tasks.workunit.client.1.smithi117.stdout:2/112: write d2/f4 [18942,47319] 0 2024-03-22T06:15:18.182 INFO:tasks.workunit.client.1.smithi117.stdout:5/95: link dd/fe dd/d16/d1a/d1f/f23 0 2024-03-22T06:15:18.184 INFO:tasks.workunit.client.1.smithi117.stdout:6/149: mknod d3/d7/da/d16/d24/d2d/c36 0 2024-03-22T06:15:18.184 INFO:tasks.workunit.client.1.smithi117.stdout:6/150: chown d3/l9 3763 1 2024-03-22T06:15:18.184 INFO:tasks.workunit.client.1.smithi117.stdout:6/151: write d3/d7/da/fc [177277,67065] 0 2024-03-22T06:15:18.184 INFO:tasks.workunit.client.1.smithi117.stdout:0/155: dread d1/d10/d18/d26/dc/f25 [0,4194304] 0 2024-03-22T06:15:18.184 INFO:tasks.workunit.client.1.smithi117.stdout:0/156: truncate d1/d10/d18/d26/dc/f30 695544 0 2024-03-22T06:15:18.185 INFO:tasks.workunit.client.1.smithi117.stdout:4/153: symlink d0/d19/l35 0 2024-03-22T06:15:18.186 INFO:tasks.workunit.client.1.smithi117.stdout:4/154: read - d0/d4/d2a/d2f/f31 zero size 2024-03-22T06:15:18.186 INFO:tasks.workunit.client.1.smithi117.stdout:4/155: fsync d0/d4/fe 0 2024-03-22T06:15:18.187 INFO:tasks.workunit.client.0.smithi006.stdout:3/962: creat d0/d4/dbb/df7/d5f/d90/da7/de7/d147/de0/f14d x:0 0 0 2024-03-22T06:15:18.189 INFO:tasks.workunit.client.1.smithi117.stdout:9/163: symlink d4/d7/l32 0 2024-03-22T06:15:18.192 INFO:tasks.workunit.client.1.smithi117.stdout:3/81: rmdir d9/d1a 39 2024-03-22T06:15:18.193 INFO:tasks.workunit.client.1.smithi117.stdout:5/96: dread dd/d16/d1a/d1f/f23 [0,4194304] 0 2024-03-22T06:15:18.200 INFO:tasks.workunit.client.1.smithi117.stdout:6/152: mknod d3/d7/da/d16/d24/d2d/d30/c37 0 2024-03-22T06:15:18.200 INFO:tasks.workunit.client.1.smithi117.stdout:6/153: write d3/df/d17/f33 [876197,62942] 0 2024-03-22T06:15:18.202 INFO:tasks.workunit.client.0.smithi006.stdout:3/963: mkdir d0/d4/dbb/df7/d5f/da3/d14e 0 2024-03-22T06:15:18.203 INFO:tasks.workunit.client.1.smithi117.stdout:4/156: chown d0/d19/d1c 26213 1 2024-03-22T06:15:18.203 INFO:tasks.workunit.client.1.smithi117.stdout:4/157: getdents d0/d19/d34 0 2024-03-22T06:15:18.203 INFO:tasks.workunit.client.1.smithi117.stdout:4/158: write d0/d4/f1b [224060,94935] 0 2024-03-22T06:15:18.203 INFO:tasks.workunit.client.1.smithi117.stdout:9/164: mknod d4/d8/c33 0 2024-03-22T06:15:18.204 INFO:tasks.workunit.client.1.smithi117.stdout:9/165: fsync d4/f21 0 2024-03-22T06:15:18.204 INFO:tasks.workunit.client.0.smithi006.stdout:3/964: mknod d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d12c/d13b/c14f 0 2024-03-22T06:15:18.204 INFO:tasks.workunit.client.0.smithi006.stdout:0/684: sync 2024-03-22T06:15:18.204 INFO:tasks.workunit.client.0.smithi006.stdout:3/965: fdatasync d0/d4/dbb/df7/d5f/d83/fd2 0 2024-03-22T06:15:18.205 INFO:tasks.workunit.client.1.smithi117.stdout:3/82: creat d9/d1a/f1e x:0 0 0 2024-03-22T06:15:18.205 INFO:tasks.workunit.client.1.smithi117.stdout:3/83: write d9/f17 [1013963,111166] 0 2024-03-22T06:15:18.209 INFO:tasks.workunit.client.0.smithi006.stdout:3/966: mkdir d0/d8/d1c/d49/d150 0 2024-03-22T06:15:18.211 INFO:tasks.workunit.client.0.smithi006.stdout:3/967: link d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d12c/d13b/c14f d0/d8/d88/ddf/d102/c151 0 2024-03-22T06:15:18.212 INFO:tasks.workunit.client.1.smithi117.stdout:9/166: mknod d4/d8/c34 0 2024-03-22T06:15:18.215 INFO:tasks.workunit.client.1.smithi117.stdout:2/113: dwrite d2/fc [0,4194304] 0 2024-03-22T06:15:18.215 INFO:tasks.workunit.client.1.smithi117.stdout:2/114: stat d2/d5/d13 0 2024-03-22T06:15:18.215 INFO:tasks.workunit.client.1.smithi117.stdout:2/115: chown d2/d5/db/lf 3 1 2024-03-22T06:15:18.215 INFO:tasks.workunit.client.1.smithi117.stdout:2/116: chown d2 13 1 2024-03-22T06:15:18.215 INFO:tasks.workunit.client.1.smithi117.stdout:2/117: fsync d2/d14/f18 0 2024-03-22T06:15:18.216 INFO:tasks.workunit.client.1.smithi117.stdout:9/167: mkdir d4/d8/d35 0 2024-03-22T06:15:18.216 INFO:tasks.workunit.client.1.smithi117.stdout:9/168: fsync d4/d8/f1a 0 2024-03-22T06:15:18.216 INFO:tasks.workunit.client.0.smithi006.stdout:7/825: dwrite d5/d1e/d29/d1f/f3e [0,4194304] 0 2024-03-22T06:15:18.216 INFO:tasks.workunit.client.1.smithi117.stdout:4/159: dread d0/d4/f12 [4194304,4194304] 0 2024-03-22T06:15:18.217 INFO:tasks.workunit.client.1.smithi117.stdout:2/118: unlink d2/d5/d8/la 0 2024-03-22T06:15:18.217 INFO:tasks.workunit.client.0.smithi006.stdout:0/685: dread db/d3d/d4e/f54 [0,4194304] 0 2024-03-22T06:15:18.218 INFO:tasks.workunit.client.1.smithi117.stdout:6/154: dread d3/df/d17/f33 [0,4194304] 0 2024-03-22T06:15:18.218 INFO:tasks.workunit.client.1.smithi117.stdout:5/97: rename dd/d16/d1a/d1f to dd/d16/d24 0 2024-03-22T06:15:18.218 INFO:tasks.workunit.client.1.smithi117.stdout:5/98: chown dd/df/d10 14264790 1 2024-03-22T06:15:18.220 INFO:tasks.workunit.client.1.smithi117.stdout:3/84: dread d9/d1a/f1b [0,4194304] 0 2024-03-22T06:15:18.220 INFO:tasks.workunit.client.1.smithi117.stdout:3/85: dread - d9/f1d zero size 2024-03-22T06:15:18.221 INFO:tasks.workunit.client.1.smithi117.stdout:3/86: fdatasync d9/d1a/f1b 0 2024-03-22T06:15:18.221 INFO:tasks.workunit.client.1.smithi117.stdout:3/87: stat d9/fa 0 2024-03-22T06:15:18.221 INFO:tasks.workunit.client.1.smithi117.stdout:4/160: truncate d0/f11 3845038 0 2024-03-22T06:15:18.221 INFO:tasks.workunit.client.1.smithi117.stdout:4/161: stat d0/d19/f1a 0 2024-03-22T06:15:18.221 INFO:tasks.workunit.client.1.smithi117.stdout:3/88: creat d9/d1a/f1f x:0 0 0 2024-03-22T06:15:18.224 INFO:tasks.workunit.client.1.smithi117.stdout:5/99: creat dd/df/d10/f25 x:0 0 0 2024-03-22T06:15:18.227 INFO:tasks.workunit.client.0.smithi006.stdout:7/826: creat d5/d1e/dbc/f10b x:0 0 0 2024-03-22T06:15:18.227 INFO:tasks.workunit.client.0.smithi006.stdout:7/827: fsync d5/f56 0 2024-03-22T06:15:18.228 INFO:tasks.workunit.client.1.smithi117.stdout:9/169: write d4/d7/f1b [654663,5220] 0 2024-03-22T06:15:18.229 INFO:tasks.workunit.client.1.smithi117.stdout:4/162: link d0/d4/f17 d0/d19/d1c/f36 0 2024-03-22T06:15:18.229 INFO:tasks.workunit.client.1.smithi117.stdout:9/170: dread d4/d8/f25 [0,4194304] 0 2024-03-22T06:15:18.230 INFO:tasks.workunit.client.1.smithi117.stdout:9/171: write d4/d7/f1c [1276601,55426] 0 2024-03-22T06:15:18.230 INFO:tasks.workunit.client.1.smithi117.stdout:9/172: readlink d4/d7/l32 0 2024-03-22T06:15:18.230 INFO:tasks.workunit.client.1.smithi117.stdout:9/173: creat d4/d7/d27/f36 x:0 0 0 2024-03-22T06:15:18.230 INFO:tasks.workunit.client.1.smithi117.stdout:3/89: write f7 [265175,90466] 0 2024-03-22T06:15:18.231 INFO:tasks.workunit.client.0.smithi006.stdout:7/828: rename d5/d1e/d29/d66/d91/ddc/d100 to d5/d1e/d63/d85/da1/d10c 0 2024-03-22T06:15:18.231 INFO:tasks.workunit.client.0.smithi006.stdout:7/829: creat d5/d1e/d29/d1f/de4/f10d x:0 0 0 2024-03-22T06:15:18.231 INFO:tasks.workunit.client.0.smithi006.stdout:7/830: write d5/d1e/fcc [5172642,79720] 0 2024-03-22T06:15:18.232 INFO:tasks.workunit.client.1.smithi117.stdout:5/100: dread dd/fe [0,4194304] 0 2024-03-22T06:15:18.232 INFO:tasks.workunit.client.1.smithi117.stdout:5/101: creat dd/df/d10/f26 x:0 0 0 2024-03-22T06:15:18.233 INFO:tasks.workunit.client.1.smithi117.stdout:0/157: truncate d1/d3/d2c/de/f23 2131351 0 2024-03-22T06:15:18.233 INFO:tasks.workunit.client.1.smithi117.stdout:0/158: truncate d1/d10/d18/d26/dc/f30 1650317 0 2024-03-22T06:15:18.233 INFO:tasks.workunit.client.1.smithi117.stdout:0/159: stat d1/d10/d18/c2a 0 2024-03-22T06:15:18.234 INFO:tasks.workunit.client.0.smithi006.stdout:0/686: getdents db/d3d 0 2024-03-22T06:15:18.234 INFO:tasks.workunit.client.0.smithi006.stdout:0/687: stat db/d10/d35/d7b/da1/fd5 0 2024-03-22T06:15:18.234 INFO:tasks.workunit.client.0.smithi006.stdout:0/688: chown db/d10/d33/d9d/fe1 894 1 2024-03-22T06:15:18.236 INFO:tasks.workunit.client.1.smithi117.stdout:3/90: fdatasync d9/f17 0 2024-03-22T06:15:18.236 INFO:tasks.workunit.client.1.smithi117.stdout:2/119: dwrite d2/f4 [0,4194304] 0 2024-03-22T06:15:18.236 INFO:tasks.workunit.client.1.smithi117.stdout:2/120: rename d2 to d2/d5/d13/d1c 22 2024-03-22T06:15:18.236 INFO:tasks.workunit.client.1.smithi117.stdout:3/91: chown d9/l13 21081 1 2024-03-22T06:15:18.236 INFO:tasks.workunit.client.1.smithi117.stdout:3/92: write f6 [4433352,10863] 0 2024-03-22T06:15:18.236 INFO:tasks.workunit.client.1.smithi117.stdout:3/93: creat d9/d1a/f20 x:0 0 0 2024-03-22T06:15:18.238 INFO:tasks.workunit.client.1.smithi117.stdout:4/163: creat d0/d19/d34/f37 x:0 0 0 2024-03-22T06:15:18.239 INFO:tasks.workunit.client.1.smithi117.stdout:4/164: readlink d0/l1f 0 2024-03-22T06:15:18.239 INFO:tasks.workunit.client.1.smithi117.stdout:4/165: fsync d0/d19/f22 0 2024-03-22T06:15:18.239 INFO:tasks.workunit.client.0.smithi006.stdout:7/831: creat d5/d1e/d29/d10/f10e x:0 0 0 2024-03-22T06:15:18.241 INFO:tasks.workunit.client.1.smithi117.stdout:9/174: write d4/d8/f1a [6437966,33628] 0 2024-03-22T06:15:18.242 INFO:tasks.workunit.client.0.smithi006.stdout:0/689: creat db/fe9 x:0 0 0 2024-03-22T06:15:18.243 INFO:tasks.workunit.client.1.smithi117.stdout:1/129: truncate d1/f7 352677 0 2024-03-22T06:15:18.246 INFO:tasks.workunit.client.0.smithi006.stdout:7/832: creat d5/d1e/d29/d66/d91/ddc/dfa/f10f x:0 0 0 2024-03-22T06:15:18.250 INFO:tasks.workunit.client.0.smithi006.stdout:0/690: mkdir db/d87/dea 0 2024-03-22T06:15:18.250 INFO:tasks.workunit.client.0.smithi006.stdout:0/691: read - db/d10/d35/d7b/fe8 zero size 2024-03-22T06:15:18.251 INFO:tasks.workunit.client.1.smithi117.stdout:0/160: creat d1/d3/d2c/de/d1f/f33 x:0 0 0 2024-03-22T06:15:18.254 INFO:tasks.workunit.client.1.smithi117.stdout:9/175: write d4/d7/f19 [309046,70692] 0 2024-03-22T06:15:18.255 INFO:tasks.workunit.client.0.smithi006.stdout:0/692: rename c7 to db/d10/d35/d6c/dad/ceb 0 2024-03-22T06:15:18.258 INFO:tasks.workunit.client.0.smithi006.stdout:5/946: sync 2024-03-22T06:15:18.258 INFO:tasks.workunit.client.0.smithi006.stdout:9/789: sync 2024-03-22T06:15:18.258 INFO:tasks.workunit.client.0.smithi006.stdout:0/693: symlink db/d3d/lec 0 2024-03-22T06:15:18.259 INFO:tasks.workunit.client.1.smithi117.stdout:3/94: dwrite d9/fa [0,4194304] 0 2024-03-22T06:15:18.260 INFO:tasks.workunit.client.0.smithi006.stdout:3/968: write d0/d8/f1a [222843,49639] 0 2024-03-22T06:15:18.260 INFO:tasks.workunit.client.0.smithi006.stdout:3/969: write d0/d4/dbb/df7/d5f/d90/da7/d97/fa5 [4796986,11218] 0 2024-03-22T06:15:18.260 INFO:tasks.workunit.client.0.smithi006.stdout:3/970: creat d0/d4/d3a/f152 x:0 0 0 2024-03-22T06:15:18.263 INFO:tasks.workunit.client.0.smithi006.stdout:9/790: unlink d8/d10/d28/f57 0 2024-03-22T06:15:18.264 INFO:tasks.workunit.client.0.smithi006.stdout:9/791: truncate d8/d10/d11/fee 896662 0 2024-03-22T06:15:18.267 INFO:tasks.workunit.client.0.smithi006.stdout:0/694: link db/d10/d33/ca3 db/d10/d35/d7b/ced 0 2024-03-22T06:15:18.271 INFO:tasks.workunit.client.0.smithi006.stdout:3/971: rename d0/f56 to d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d12c/f153 0 2024-03-22T06:15:18.274 INFO:tasks.workunit.client.1.smithi117.stdout:1/130: dwrite d1/fc [4194304,4194304] 0 2024-03-22T06:15:18.276 INFO:tasks.workunit.client.0.smithi006.stdout:9/792: rename d8/d10/d28/l51 to d8/d1f/l104 0 2024-03-22T06:15:18.278 INFO:tasks.workunit.client.1.smithi117.stdout:6/155: truncate d3/df/d17/f33 305665 0 2024-03-22T06:15:18.281 INFO:tasks.workunit.client.0.smithi006.stdout:0/695: rename db/dc/d37/cd1 to db/d10/d33/dce/cee 0 2024-03-22T06:15:18.281 INFO:tasks.workunit.client.1.smithi117.stdout:2/121: dwrite d2/fc [0,4194304] 0 2024-03-22T06:15:18.281 INFO:tasks.workunit.client.0.smithi006.stdout:7/833: dwrite d5/d1e/d63/d85/dca/f101 [0,4194304] 0 2024-03-22T06:15:18.282 INFO:tasks.workunit.client.0.smithi006.stdout:7/834: dread - d5/d1e/d29/d66/fdb zero size 2024-03-22T06:15:18.282 INFO:tasks.workunit.client.1.smithi117.stdout:4/166: link d0/f23 d0/f38 0 2024-03-22T06:15:18.284 INFO:tasks.workunit.client.0.smithi006.stdout:9/793: link d8/d10/d11/c32 d8/d10/d47/dbc/c105 0 2024-03-22T06:15:18.284 INFO:tasks.workunit.client.0.smithi006.stdout:9/794: chown d8/d10/d20/d45/caa 0 1 2024-03-22T06:15:18.284 INFO:tasks.workunit.client.0.smithi006.stdout:9/795: dread - d8/d1f/fe5 zero size 2024-03-22T06:15:18.286 INFO:tasks.workunit.client.0.smithi006.stdout:0/696: mknod db/d10/d2a/d41/cef 0 2024-03-22T06:15:18.287 INFO:tasks.workunit.client.0.smithi006.stdout:0/697: dread - db/f96 zero size 2024-03-22T06:15:18.287 INFO:tasks.workunit.client.0.smithi006.stdout:5/947: dwrite d0/db/d50/f80 [0,4194304] 0 2024-03-22T06:15:18.287 INFO:tasks.workunit.client.0.smithi006.stdout:3/972: dwrite d0/d4/dbb/df7/d5f/d90/da7/de7/d147/de0/feb [0,4194304] 0 2024-03-22T06:15:18.288 INFO:tasks.workunit.client.1.smithi117.stdout:0/161: mknod d1/d10/c34 0 2024-03-22T06:15:18.288 INFO:tasks.workunit.client.1.smithi117.stdout:0/162: readlink d1/d10/d18/l1e 0 2024-03-22T06:15:18.293 INFO:tasks.workunit.client.0.smithi006.stdout:3/973: dread d0/d4/dbb/df7/d5f/d90/da7/de7/d33/f44 [0,4194304] 0 2024-03-22T06:15:18.294 INFO:tasks.workunit.client.0.smithi006.stdout:7/835: rename f2 to d5/d1e/d63/f110 0 2024-03-22T06:15:18.297 INFO:tasks.workunit.client.0.smithi006.stdout:9/796: symlink d8/d10/da6/l106 0 2024-03-22T06:15:18.301 INFO:tasks.workunit.client.1.smithi117.stdout:1/131: rename d1/f1b to d1/d22/d27/f28 0 2024-03-22T06:15:18.302 INFO:tasks.workunit.client.0.smithi006.stdout:5/948: unlink d0/db/d50/d82/d103/ff6 0 2024-03-22T06:15:18.303 INFO:tasks.workunit.client.1.smithi117.stdout:6/156: creat d3/d7/da/d32/f38 x:0 0 0 2024-03-22T06:15:18.310 INFO:tasks.workunit.client.1.smithi117.stdout:2/122: mkdir d2/d14/d1d 0 2024-03-22T06:15:18.310 INFO:tasks.workunit.client.1.smithi117.stdout:4/167: dwrite d0/f2d [0,4194304] 0 2024-03-22T06:15:18.312 INFO:tasks.workunit.client.0.smithi006.stdout:7/836: link d5/d1e/d29/d1f/d2d/d55/d84/ce5 d5/d1e/d63/d85/dac/db3/c111 0 2024-03-22T06:15:18.315 INFO:tasks.workunit.client.0.smithi006.stdout:5/949: dread d0/db/d50/d96/f2d [0,4194304] 0 2024-03-22T06:15:18.315 INFO:tasks.workunit.client.0.smithi006.stdout:5/950: stat d0/d98/f100 0 2024-03-22T06:15:18.316 INFO:tasks.workunit.client.1.smithi117.stdout:0/163: mknod d1/d10/d18/c35 0 2024-03-22T06:15:18.316 INFO:tasks.workunit.client.1.smithi117.stdout:0/164: write d1/d10/d18/d26/dc/f30 [1104511,95082] 0 2024-03-22T06:15:18.320 INFO:tasks.workunit.client.1.smithi117.stdout:8/134: sync 2024-03-22T06:15:18.320 INFO:tasks.workunit.client.1.smithi117.stdout:8/135: fdatasync d1/d19/f2d 0 2024-03-22T06:15:18.320 INFO:tasks.workunit.client.1.smithi117.stdout:8/136: fsync d1/d19/f2d 0 2024-03-22T06:15:18.320 INFO:tasks.workunit.client.1.smithi117.stdout:7/126: sync 2024-03-22T06:15:18.321 INFO:tasks.workunit.client.0.smithi006.stdout:9/797: rename d8/d10/d20/c101 to d8/d10/d20/d45/c107 0 2024-03-22T06:15:18.323 INFO:tasks.workunit.client.0.smithi006.stdout:0/698: mknod db/dc/cf0 0 2024-03-22T06:15:18.323 INFO:tasks.workunit.client.1.smithi117.stdout:2/123: dread d2/f3 [0,4194304] 0 2024-03-22T06:15:18.328 INFO:tasks.workunit.client.0.smithi006.stdout:7/837: mknod d5/d1e/d63/d85/da1/c112 0 2024-03-22T06:15:18.328 INFO:tasks.workunit.client.0.smithi006.stdout:7/838: chown d5/d1e/d29/d1f/d2d/d61/daa/lae 19 1 2024-03-22T06:15:18.329 INFO:tasks.workunit.client.1.smithi117.stdout:4/168: mknod d0/d32/c39 0 2024-03-22T06:15:18.331 INFO:tasks.workunit.client.0.smithi006.stdout:5/951: unlink d0/db/d16/d1c/d34/d4d/f6b 0 2024-03-22T06:15:18.331 INFO:tasks.workunit.client.0.smithi006.stdout:5/952: write d0/db/d50/f5b [2350619,104166] 0 2024-03-22T06:15:18.331 INFO:tasks.workunit.client.0.smithi006.stdout:5/953: truncate d0/db/d50/d96/f63 2310655 0 2024-03-22T06:15:18.332 INFO:tasks.workunit.client.1.smithi117.stdout:0/165: symlink d1/d3/d2c/de/d14/l36 0 2024-03-22T06:15:18.334 INFO:tasks.workunit.client.1.smithi117.stdout:8/137: truncate d1/d2/fa 214939 0 2024-03-22T06:15:18.335 INFO:tasks.workunit.client.0.smithi006.stdout:0/699: stat db/l8f 0 2024-03-22T06:15:18.336 INFO:tasks.workunit.client.1.smithi117.stdout:7/127: creat d8/d9/da/d10/d23/f24 x:0 0 0 2024-03-22T06:15:18.337 INFO:tasks.workunit.client.1.smithi117.stdout:3/95: truncate d9/fa 1031899 0 2024-03-22T06:15:18.337 INFO:tasks.workunit.client.1.smithi117.stdout:1/132: dwrite d1/f7 [0,4194304] 0 2024-03-22T06:15:18.338 INFO:tasks.workunit.client.0.smithi006.stdout:5/954: unlink d0/db/d50/d58/df7/f118 0 2024-03-22T06:15:18.339 INFO:tasks.workunit.client.1.smithi117.stdout:6/157: dwrite d3/f5 [0,4194304] 0 2024-03-22T06:15:18.339 INFO:tasks.workunit.client.1.smithi117.stdout:6/158: fdatasync d3/f1f 0 2024-03-22T06:15:18.340 INFO:tasks.workunit.client.0.smithi006.stdout:0/700: mknod db/d3d/d4e/d76/cf1 0 2024-03-22T06:15:18.340 INFO:tasks.workunit.client.0.smithi006.stdout:0/701: stat db/dc/d38/f4b 0 2024-03-22T06:15:18.341 INFO:tasks.workunit.client.1.smithi117.stdout:8/138: dread d1/d2/f1d [0,4194304] 0 2024-03-22T06:15:18.342 INFO:tasks.workunit.client.1.smithi117.stdout:4/169: unlink d0/d32/c39 0 2024-03-22T06:15:18.343 INFO:tasks.workunit.client.0.smithi006.stdout:5/955: mknod d0/db/d16/d1c/dbc/dd6/c122 0 2024-03-22T06:15:18.343 INFO:tasks.workunit.client.0.smithi006.stdout:5/956: chown d0/db/d50/d96 97347378 1 2024-03-22T06:15:18.343 INFO:tasks.workunit.client.0.smithi006.stdout:5/957: readlink d0/db/d16/d1c/d34/d35/d85/dda/d65/lec 0 2024-03-22T06:15:18.345 INFO:tasks.workunit.client.1.smithi117.stdout:7/128: unlink f6 0 2024-03-22T06:15:18.345 INFO:tasks.workunit.client.1.smithi117.stdout:7/129: stat d8/d9/f1e 0 2024-03-22T06:15:18.345 INFO:tasks.workunit.client.1.smithi117.stdout:7/130: dread - d8/f1f zero size 2024-03-22T06:15:18.346 INFO:tasks.workunit.client.0.smithi006.stdout:0/702: creat db/d10/d35/d7b/da1/da6/dc4/ff2 x:0 0 0 2024-03-22T06:15:18.346 INFO:tasks.workunit.client.0.smithi006.stdout:0/703: readlink db/d10/d35/d7b/l99 0 2024-03-22T06:15:18.346 INFO:tasks.workunit.client.1.smithi117.stdout:0/166: dread d1/d10/f11 [0,4194304] 0 2024-03-22T06:15:18.348 INFO:tasks.workunit.client.0.smithi006.stdout:9/798: dwrite d8/d10/d28/dba/fbd [0,4194304] 0 2024-03-22T06:15:18.348 INFO:tasks.workunit.client.1.smithi117.stdout:1/133: mknod d1/d1f/c29 0 2024-03-22T06:15:18.349 INFO:tasks.workunit.client.0.smithi006.stdout:5/958: write d0/db/f6c [1184359,57603] 0 2024-03-22T06:15:18.349 INFO:tasks.workunit.client.0.smithi006.stdout:5/959: creat d0/db/d99/f123 x:0 0 0 2024-03-22T06:15:18.352 INFO:tasks.workunit.client.0.smithi006.stdout:0/704: dread db/d10/f1b [0,4194304] 0 2024-03-22T06:15:18.352 INFO:tasks.workunit.client.0.smithi006.stdout:0/705: truncate db/d10/f14 5085776 0 2024-03-22T06:15:18.358 INFO:tasks.workunit.client.0.smithi006.stdout:9/799: unlink d8/db3/d50/f6f 0 2024-03-22T06:15:18.358 INFO:tasks.workunit.client.1.smithi117.stdout:9/176: truncate d4/d7/f19 3149013 0 2024-03-22T06:15:18.359 INFO:tasks.workunit.client.1.smithi117.stdout:9/177: read d4/d8/f13 [1285202,117045] 0 2024-03-22T06:15:18.359 INFO:tasks.workunit.client.1.smithi117.stdout:9/178: dread - d4/d7/d27/f36 zero size 2024-03-22T06:15:18.359 INFO:tasks.workunit.client.1.smithi117.stdout:9/179: write d4/d7/f26 [886000,67018] 0 2024-03-22T06:15:18.361 INFO:tasks.workunit.client.0.smithi006.stdout:0/706: truncate db/dc/d37/f77 6259197 0 2024-03-22T06:15:18.361 INFO:tasks.workunit.client.0.smithi006.stdout:0/707: chown db/d10/d33/f91 26 1 2024-03-22T06:15:18.361 INFO:tasks.workunit.client.0.smithi006.stdout:0/708: dread - db/d3d/fe0 zero size 2024-03-22T06:15:18.361 INFO:tasks.workunit.client.0.smithi006.stdout:0/709: write db/f96 [837048,123639] 0 2024-03-22T06:15:18.362 INFO:tasks.workunit.client.0.smithi006.stdout:0/710: fdatasync db/d10/d2a/d41/f82 0 2024-03-22T06:15:18.362 INFO:tasks.workunit.client.0.smithi006.stdout:0/711: dread - db/dc/d38/f66 zero size 2024-03-22T06:15:18.362 INFO:tasks.workunit.client.1.smithi117.stdout:2/124: rmdir d2/d5/d8 39 2024-03-22T06:15:18.362 INFO:tasks.workunit.client.1.smithi117.stdout:2/125: readlink d2/l12 0 2024-03-22T06:15:18.363 INFO:tasks.workunit.client.0.smithi006.stdout:9/800: rmdir d8/d10/d20/d103 39 2024-03-22T06:15:18.363 INFO:tasks.workunit.client.0.smithi006.stdout:9/801: chown d8/fb8 1797 1 2024-03-22T06:15:18.363 INFO:tasks.workunit.client.1.smithi117.stdout:4/170: rmdir d0/d4 39 2024-03-22T06:15:18.363 INFO:tasks.workunit.client.1.smithi117.stdout:4/171: write d0/d4/fe [2244876,81115] 0 2024-03-22T06:15:18.363 INFO:tasks.workunit.client.1.smithi117.stdout:4/172: write d0/f2d [4785140,6623] 0 2024-03-22T06:15:18.365 INFO:tasks.workunit.client.1.smithi117.stdout:7/131: rename d8/d9/ff to d8/d9/da/d10/f25 0 2024-03-22T06:15:18.368 INFO:tasks.workunit.client.1.smithi117.stdout:0/167: unlink d1/d10/d18/d26/dc/f30 0 2024-03-22T06:15:18.371 INFO:tasks.workunit.client.1.smithi117.stdout:1/134: creat d1/d14/d16/f2a x:0 0 0 2024-03-22T06:15:18.371 INFO:tasks.workunit.client.1.smithi117.stdout:1/135: dread - d1/d1f/f20 zero size 2024-03-22T06:15:18.374 INFO:tasks.workunit.client.1.smithi117.stdout:6/159: dwrite d3/f1f [0,4194304] 0 2024-03-22T06:15:18.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/139: dwrite d1/d2/d9/f13 [4194304,4194304] 0 2024-03-22T06:15:18.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/140: chown d1/d2/d9/ld 1417927 1 2024-03-22T06:15:18.376 INFO:tasks.workunit.client.1.smithi117.stdout:2/126: mknod d2/d5/d13/c1e 0 2024-03-22T06:15:18.376 INFO:tasks.workunit.client.1.smithi117.stdout:9/180: dread d4/d7/f16 [0,4194304] 0 2024-03-22T06:15:18.376 INFO:tasks.workunit.client.1.smithi117.stdout:9/181: write d4/d8/f2c [836048,121058] 0 2024-03-22T06:15:18.376 INFO:tasks.workunit.client.0.smithi006.stdout:7/839: dwrite d5/d1e/d29/d1f/d2d/d55/f8c [0,4194304] 0 2024-03-22T06:15:18.376 INFO:tasks.workunit.client.0.smithi006.stdout:7/840: fsync d5/d1e/d29/d1f/d2d/d61/daa/fff 0 2024-03-22T06:15:18.382 INFO:tasks.workunit.client.1.smithi117.stdout:0/168: symlink d1/d3/l37 0 2024-03-22T06:15:18.382 INFO:tasks.workunit.client.1.smithi117.stdout:9/182: dread d4/d8/f13 [0,4194304] 0 2024-03-22T06:15:18.383 INFO:tasks.workunit.client.1.smithi117.stdout:2/127: dread d2/d5/d8/d10/d11/f15 [0,4194304] 0 2024-03-22T06:15:18.383 INFO:tasks.workunit.client.1.smithi117.stdout:2/128: fsync d2/d5/d13/f19 0 2024-03-22T06:15:18.384 INFO:tasks.workunit.client.1.smithi117.stdout:2/129: creat d2/d14/f1f x:0 0 0 2024-03-22T06:15:18.390 INFO:tasks.workunit.client.0.smithi006.stdout:0/712: dread db/d87/f93 [0,4194304] 0 2024-03-22T06:15:18.393 INFO:tasks.workunit.client.1.smithi117.stdout:6/160: unlink d3/d7/l8 0 2024-03-22T06:15:18.396 INFO:tasks.workunit.client.0.smithi006.stdout:9/802: dwrite d8/d1f/d34/f3a [0,4194304] 0 2024-03-22T06:15:18.396 INFO:tasks.workunit.client.0.smithi006.stdout:9/803: truncate d8/db3/d50/fd3 64147 0 2024-03-22T06:15:18.396 INFO:tasks.workunit.client.1.smithi117.stdout:8/141: creat d1/d28/f31 x:0 0 0 2024-03-22T06:15:18.401 INFO:tasks.workunit.client.1.smithi117.stdout:0/169: creat d1/d10/d18/f38 x:0 0 0 2024-03-22T06:15:18.404 INFO:tasks.workunit.client.0.smithi006.stdout:7/841: rename d5/d1e/d29/d1f/d2d/d55/d84/fa0 to d5/d1e/d29/d1f/d2d/d55/f113 0 2024-03-22T06:15:18.404 INFO:tasks.workunit.client.0.smithi006.stdout:7/842: truncate d5/d1e/d63/d85/fa6 237626 0 2024-03-22T06:15:18.405 INFO:tasks.workunit.client.1.smithi117.stdout:5/102: sync 2024-03-22T06:15:18.405 INFO:tasks.workunit.client.1.smithi117.stdout:5/103: write dd/df/d10/f25 [868096,98400] 0 2024-03-22T06:15:18.405 INFO:tasks.workunit.client.0.smithi006.stdout:7/843: read d5/d1e/d63/d85/fab [438784,66557] 0 2024-03-22T06:15:18.406 INFO:tasks.workunit.client.1.smithi117.stdout:9/183: mkdir d4/d8/d37 0 2024-03-22T06:15:18.410 INFO:tasks.workunit.client.0.smithi006.stdout:9/804: symlink d8/d1f/df5/l108 0 2024-03-22T06:15:18.411 INFO:tasks.workunit.client.1.smithi117.stdout:6/161: link d3/d7/da/d16/d24/d2d/d30/c37 d3/d7/da/d16/d24/d2d/d30/c39 0 2024-03-22T06:15:18.413 INFO:tasks.workunit.client.1.smithi117.stdout:8/142: rename d1/l23 to d1/d19/l32 0 2024-03-22T06:15:18.413 INFO:tasks.workunit.client.1.smithi117.stdout:8/143: stat d1/d1f 0 2024-03-22T06:15:18.413 INFO:tasks.workunit.client.1.smithi117.stdout:8/144: chown d1/f1e 11058567 1 2024-03-22T06:15:18.413 INFO:tasks.workunit.client.1.smithi117.stdout:1/136: dwrite d1/d1f/f20 [0,4194304] 0 2024-03-22T06:15:18.414 INFO:tasks.workunit.client.1.smithi117.stdout:9/184: symlink d4/d7/d27/l38 0 2024-03-22T06:15:18.418 INFO:tasks.workunit.client.1.smithi117.stdout:5/104: read f6 [1267336,10676] 0 2024-03-22T06:15:18.418 INFO:tasks.workunit.client.1.smithi117.stdout:6/162: unlink d3/df/c12 0 2024-03-22T06:15:18.418 INFO:tasks.workunit.client.1.smithi117.stdout:6/163: readlink d3/l9 0 2024-03-22T06:15:18.418 INFO:tasks.workunit.client.1.smithi117.stdout:6/164: chown d3/d7/da/de/d2c/f2f 208923654 1 2024-03-22T06:15:18.418 INFO:tasks.workunit.client.1.smithi117.stdout:6/165: dread - d3/df/d17/d31/f34 zero size 2024-03-22T06:15:18.424 INFO:tasks.workunit.client.1.smithi117.stdout:1/137: symlink d1/d1f/l2b 0 2024-03-22T06:15:18.426 INFO:tasks.workunit.client.1.smithi117.stdout:9/185: creat d4/d7/d10/d2f/f39 x:0 0 0 2024-03-22T06:15:18.426 INFO:tasks.workunit.client.1.smithi117.stdout:9/186: write d4/d7/d10/f28 [1396414,60558] 0 2024-03-22T06:15:18.427 INFO:tasks.workunit.client.1.smithi117.stdout:6/166: mkdir d3/d7/da/de/d2c/d3a 0 2024-03-22T06:15:18.428 INFO:tasks.workunit.client.1.smithi117.stdout:6/167: truncate d3/df/f2e 160652 0 2024-03-22T06:15:18.428 INFO:tasks.workunit.client.1.smithi117.stdout:6/168: truncate d3/df/f2e 734832 0 2024-03-22T06:15:18.428 INFO:tasks.workunit.client.1.smithi117.stdout:7/132: dwrite d8/d9/f14 [0,4194304] 0 2024-03-22T06:15:18.428 INFO:tasks.workunit.client.1.smithi117.stdout:3/96: dwrite d9/fa [0,4194304] 0 2024-03-22T06:15:18.428 INFO:tasks.workunit.client.1.smithi117.stdout:3/97: creat d9/f21 x:0 0 0 2024-03-22T06:15:18.436 INFO:tasks.workunit.client.0.smithi006.stdout:4/909: sync 2024-03-22T06:15:18.436 INFO:tasks.workunit.client.0.smithi006.stdout:4/910: chown d6/d1b/d20/d6b/c70 207936 1 2024-03-22T06:15:18.437 INFO:tasks.workunit.client.0.smithi006.stdout:4/911: symlink d6/d1b/l13b 0 2024-03-22T06:15:18.437 INFO:tasks.workunit.client.0.smithi006.stdout:4/912: write d6/d1b/d46/d101/f135 [585551,129647] 0 2024-03-22T06:15:18.438 INFO:tasks.workunit.client.1.smithi117.stdout:4/173: truncate d0/f2d 2502443 0 2024-03-22T06:15:18.441 INFO:tasks.workunit.client.0.smithi006.stdout:4/913: mkdir d6/d1b/d46/d101/d13c 0 2024-03-22T06:15:18.443 INFO:tasks.workunit.client.1.smithi117.stdout:6/169: dread d3/d7/da/fc [0,4194304] 0 2024-03-22T06:15:18.443 INFO:tasks.workunit.client.1.smithi117.stdout:6/170: dread - d3/f20 zero size 2024-03-22T06:15:18.443 INFO:tasks.workunit.client.1.smithi117.stdout:9/187: rmdir d4/d8 39 2024-03-22T06:15:18.444 INFO:tasks.workunit.client.1.smithi117.stdout:2/130: dwrite d2/fc [4194304,4194304] 0 2024-03-22T06:15:18.445 INFO:tasks.workunit.client.0.smithi006.stdout:4/914: mknod d6/d1b/d20/d21/d77/df1/c13d 0 2024-03-22T06:15:18.445 INFO:tasks.workunit.client.0.smithi006.stdout:4/915: fsync d6/d1b/d20/d21/d2a/d43/d4b/f138 0 2024-03-22T06:15:18.445 INFO:tasks.workunit.client.0.smithi006.stdout:0/713: dwrite db/fe9 [0,4194304] 0 2024-03-22T06:15:18.447 INFO:tasks.workunit.client.1.smithi117.stdout:8/145: dwrite f0 [4194304,4194304] 0 2024-03-22T06:15:18.447 INFO:tasks.workunit.client.1.smithi117.stdout:8/146: fdatasync d1/d2/d11/f2a 0 2024-03-22T06:15:18.447 INFO:tasks.workunit.client.1.smithi117.stdout:4/174: rename d0/d19/l35 to d0/d19/d1c/l3a 0 2024-03-22T06:15:18.448 INFO:tasks.workunit.client.0.smithi006.stdout:1/839: sync 2024-03-22T06:15:18.448 INFO:tasks.workunit.client.0.smithi006.stdout:1/840: chown d8/d9/d10/d7c/d96/cb4 161046528 1 2024-03-22T06:15:18.448 INFO:tasks.workunit.client.0.smithi006.stdout:7/844: dwrite d5/d1e/d29/d66/fa3 [0,4194304] 0 2024-03-22T06:15:18.448 INFO:tasks.workunit.client.0.smithi006.stdout:9/805: dwrite d8/d10/d28/dba/fbd [0,4194304] 0 2024-03-22T06:15:18.449 INFO:tasks.workunit.client.0.smithi006.stdout:4/916: mknod d6/d1b/d20/d6b/d7b/c13e 0 2024-03-22T06:15:18.450 INFO:tasks.workunit.client.1.smithi117.stdout:6/171: mknod d3/d7/d28/c3b 0 2024-03-22T06:15:18.452 INFO:tasks.workunit.client.1.smithi117.stdout:9/188: creat d4/d7/d10/f3a x:0 0 0 2024-03-22T06:15:18.454 INFO:tasks.workunit.client.0.smithi006.stdout:7/845: write d5/d1e/d29/d10/f7e [131199,71252] 0 2024-03-22T06:15:18.454 INFO:tasks.workunit.client.1.smithi117.stdout:5/105: dwrite dd/df/f1b [0,4194304] 0 2024-03-22T06:15:18.454 INFO:tasks.workunit.client.1.smithi117.stdout:1/138: dwrite d1/d22/d27/f28 [0,4194304] 0 2024-03-22T06:15:18.455 INFO:tasks.workunit.client.0.smithi006.stdout:1/841: mknod d8/d9/d13/d24/d32/d34/daa/c11f 0 2024-03-22T06:15:18.460 INFO:tasks.workunit.client.1.smithi117.stdout:4/175: unlink d0/d4/cd 0 2024-03-22T06:15:18.462 INFO:tasks.workunit.client.0.smithi006.stdout:4/917: creat d6/d1b/d46/d101/d13c/f13f x:0 0 0 2024-03-22T06:15:18.466 INFO:tasks.workunit.client.0.smithi006.stdout:9/806: dread d8/db3/d50/f8e [0,4194304] 0 2024-03-22T06:15:18.471 INFO:tasks.workunit.client.0.smithi006.stdout:7/846: read d5/f9 [1912619,88606] 0 2024-03-22T06:15:18.471 INFO:tasks.workunit.client.0.smithi006.stdout:7/847: dread - d5/d1e/d29/d1f/d2d/d41/ff5 zero size 2024-03-22T06:15:18.475 INFO:tasks.workunit.client.1.smithi117.stdout:3/98: truncate d9/fa 4090027 0 2024-03-22T06:15:18.481 INFO:tasks.workunit.client.0.smithi006.stdout:9/807: dread d8/d1f/d34/f3a [0,4194304] 0 2024-03-22T06:15:18.484 INFO:tasks.workunit.client.0.smithi006.stdout:4/918: symlink d6/d1b/d20/d63/d93/d9a/dfd/l140 0 2024-03-22T06:15:18.485 INFO:tasks.workunit.client.0.smithi006.stdout:7/848: mkdir d5/d114 0 2024-03-22T06:15:18.486 INFO:tasks.workunit.client.0.smithi006.stdout:9/808: creat d8/db3/d50/d91/f109 x:0 0 0 2024-03-22T06:15:18.486 INFO:tasks.workunit.client.1.smithi117.stdout:6/172: getdents d3/d7/da/d16 0 2024-03-22T06:15:18.487 INFO:tasks.workunit.client.1.smithi117.stdout:6/173: dread - d3/d7/da/d32/f38 zero size 2024-03-22T06:15:18.487 INFO:tasks.workunit.client.0.smithi006.stdout:1/842: getdents d8/d9/d13/d24/d32/d34 0 2024-03-22T06:15:18.488 INFO:tasks.workunit.client.1.smithi117.stdout:9/189: mkdir d4/d7/d10/d3b 0 2024-03-22T06:15:18.490 INFO:tasks.workunit.client.0.smithi006.stdout:4/919: rename d6/fa to d6/f141 0 2024-03-22T06:15:18.490 INFO:tasks.workunit.client.0.smithi006.stdout:4/920: write d6/d1b/d20/d21/d2a/d43/d4b/f118 [1018148,127641] 0 2024-03-22T06:15:18.491 INFO:tasks.workunit.client.0.smithi006.stdout:7/849: link d5/d1e/d29/d1f/d2d/d61/daa/fc0 d5/d1e/d29/d59/f115 0 2024-03-22T06:15:18.491 INFO:tasks.workunit.client.0.smithi006.stdout:7/850: chown d5/d1e/d29/d66 1698 1 2024-03-22T06:15:18.492 INFO:tasks.workunit.client.1.smithi117.stdout:5/106: truncate f6 1150484 0 2024-03-22T06:15:18.492 INFO:tasks.workunit.client.1.smithi117.stdout:5/107: stat dd/df 0 2024-03-22T06:15:18.492 INFO:tasks.workunit.client.1.smithi117.stdout:5/108: fdatasync f4 0 2024-03-22T06:15:18.493 INFO:tasks.workunit.client.0.smithi006.stdout:1/843: unlink d8/d9/d10/d7c/d96/l9d 0 2024-03-22T06:15:18.495 INFO:tasks.workunit.client.0.smithi006.stdout:4/921: symlink d6/d1b/d20/d6b/d88/l142 0 2024-03-22T06:15:18.495 INFO:tasks.workunit.client.0.smithi006.stdout:4/922: creat d6/d1b/d20/d21/d11d/f143 x:0 0 0 2024-03-22T06:15:18.495 INFO:tasks.workunit.client.0.smithi006.stdout:4/923: write d6/d13/fe7 [182591,52336] 0 2024-03-22T06:15:18.495 INFO:tasks.workunit.client.0.smithi006.stdout:4/924: write d6/d1b/d46/d101/f135 [1008843,115395] 0 2024-03-22T06:15:18.495 INFO:tasks.workunit.client.0.smithi006.stdout:4/925: fdatasync d6/f62 0 2024-03-22T06:15:18.496 INFO:tasks.workunit.client.1.smithi117.stdout:8/147: getdents d1/d2/d11 0 2024-03-22T06:15:18.496 INFO:tasks.workunit.client.1.smithi117.stdout:8/148: truncate d1/d19/f2d 214589 0 2024-03-22T06:15:18.496 INFO:tasks.workunit.client.1.smithi117.stdout:1/139: dwrite d1/d14/d16/f1a [0,4194304] 0 2024-03-22T06:15:18.496 INFO:tasks.workunit.client.1.smithi117.stdout:1/140: fsync d1/d14/d16/f2a 0 2024-03-22T06:15:18.497 INFO:tasks.workunit.client.0.smithi006.stdout:9/809: getdents d8 0 2024-03-22T06:15:18.498 INFO:tasks.workunit.client.1.smithi117.stdout:4/176: dwrite d0/d19/d1c/f36 [0,4194304] 0 2024-03-22T06:15:18.498 INFO:tasks.workunit.client.1.smithi117.stdout:4/177: write d0/d4/d2a/d2f/f33 [450393,11522] 0 2024-03-22T06:15:18.498 INFO:tasks.workunit.client.1.smithi117.stdout:9/190: truncate d4/d7/f2d 1255361 0 2024-03-22T06:15:18.499 INFO:tasks.workunit.client.0.smithi006.stdout:4/926: creat d6/d1b/d20/d21/d77/df2/f144 x:0 0 0 2024-03-22T06:15:18.500 INFO:tasks.workunit.client.1.smithi117.stdout:6/174: read d3/f1f [3773607,66691] 0 2024-03-22T06:15:18.500 INFO:tasks.workunit.client.1.smithi117.stdout:5/109: getdents dd/df/d12 0 2024-03-22T06:15:18.501 INFO:tasks.workunit.client.0.smithi006.stdout:9/810: mknod d8/db3/d50/d91/c10a 0 2024-03-22T06:15:18.502 INFO:tasks.workunit.client.1.smithi117.stdout:7/133: dwrite d8/d9/da/d10/f25 [0,4194304] 0 2024-03-22T06:15:18.502 INFO:tasks.workunit.client.1.smithi117.stdout:3/99: dwrite d9/d1a/f1e [0,4194304] 0 2024-03-22T06:15:18.503 INFO:tasks.workunit.client.1.smithi117.stdout:1/141: symlink d1/d22/d27/l2c 0 2024-03-22T06:15:18.504 INFO:tasks.workunit.client.1.smithi117.stdout:1/142: write d1/f13 [825236,33085] 0 2024-03-22T06:15:18.505 INFO:tasks.workunit.client.0.smithi006.stdout:7/851: dread d5/d1e/d29/d1f/d2d/f51 [0,4194304] 0 2024-03-22T06:15:18.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:18 smithi117 ceph-mon[36514]: pgmap v22: 65 pgs: 65 active+clean; 1.8 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 97 MiB/s rd, 125 MiB/s wr, 935 op/s 2024-03-22T06:15:18.506 INFO:tasks.workunit.client.0.smithi006.stdout:4/927: dread d6/d1b/d20/d63/d93/fc6 [0,4194304] 0 2024-03-22T06:15:18.506 INFO:tasks.workunit.client.0.smithi006.stdout:4/928: chown d6/d13/f104 128418128 1 2024-03-22T06:15:18.507 INFO:tasks.workunit.client.0.smithi006.stdout:9/811: mkdir d8/d10/d82/df7/d10b 0 2024-03-22T06:15:18.507 INFO:tasks.workunit.client.0.smithi006.stdout:9/812: readlink d8/d1f/l94 0 2024-03-22T06:15:18.507 INFO:tasks.workunit.client.0.smithi006.stdout:9/813: truncate d8/d10/d28/d39/ff9 4464161 0 2024-03-22T06:15:18.509 INFO:tasks.workunit.client.1.smithi117.stdout:2/131: rename d2/d14 to d2/d5/d20 0 2024-03-22T06:15:18.511 INFO:tasks.workunit.client.1.smithi117.stdout:9/191: truncate d4/f1e 753487 0 2024-03-22T06:15:18.514 INFO:tasks.workunit.client.0.smithi006.stdout:7/852: dread d5/d1e/d63/d85/fab [0,4194304] 0 2024-03-22T06:15:18.514 INFO:tasks.workunit.client.1.smithi117.stdout:7/134: mkdir d8/d9/d1a/d26 0 2024-03-22T06:15:18.517 INFO:tasks.workunit.client.1.smithi117.stdout:1/143: write d1/d14/d16/f1a [2039734,38435] 0 2024-03-22T06:15:18.518 INFO:tasks.workunit.client.0.smithi006.stdout:4/929: symlink d6/d1b/d46/d101/d128/l145 0 2024-03-22T06:15:18.518 INFO:tasks.workunit.client.0.smithi006.stdout:9/814: write d8/d10/d23/f40 [532513,129339] 0 2024-03-22T06:15:18.518 INFO:tasks.workunit.client.0.smithi006.stdout:9/815: chown d8/d10/f56 11202 1 2024-03-22T06:15:18.522 INFO:tasks.workunit.client.1.smithi117.stdout:3/100: rename d9/f1d to d9/f22 0 2024-03-22T06:15:18.523 INFO:tasks.workunit.client.0.smithi006.stdout:4/930: mkdir d6/d1b/d20/d6b/d146 0 2024-03-22T06:15:18.523 INFO:tasks.workunit.client.0.smithi006.stdout:4/931: dread - d6/d1b/d20/d21/d9f/f11b zero size 2024-03-22T06:15:18.528 INFO:tasks.workunit.client.0.smithi006.stdout:4/932: rename d6/d1b/d20/d21/ca2 to d6/d1b/d20/d21/d77/da0/c147 0 2024-03-22T06:15:18.528 INFO:tasks.workunit.client.0.smithi006.stdout:4/933: chown d6/d1b/d20/d21/d2a/d43/d4b 431747 1 2024-03-22T06:15:18.531 INFO:tasks.workunit.client.0.smithi006.stdout:1/844: dwrite d8/d33/f11e [0,4194304] 0 2024-03-22T06:15:18.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:18 smithi006 ceph-mon[29301]: pgmap v22: 65 pgs: 65 active+clean; 1.8 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 97 MiB/s rd, 125 MiB/s wr, 935 op/s 2024-03-22T06:15:18.532 INFO:tasks.workunit.client.0.smithi006.stdout:4/934: dread d6/d1b/d20/d63/f116 [0,4194304] 0 2024-03-22T06:15:18.535 INFO:tasks.workunit.client.1.smithi117.stdout:4/178: rename d0/c1e to d0/d19/d28/c3b 0 2024-03-22T06:15:18.538 INFO:tasks.workunit.client.1.smithi117.stdout:6/175: dwrite d3/d7/da/d16/f22 [0,4194304] 0 2024-03-22T06:15:18.538 INFO:tasks.workunit.client.1.smithi117.stdout:6/176: read - d3/d7/da/f29 zero size 2024-03-22T06:15:18.538 INFO:tasks.workunit.client.1.smithi117.stdout:6/177: dread - d3/df/d17/d31/f34 zero size 2024-03-22T06:15:18.539 INFO:tasks.workunit.client.1.smithi117.stdout:9/192: rmdir d4/d8/d35 0 2024-03-22T06:15:18.539 INFO:tasks.workunit.client.1.smithi117.stdout:9/193: chown d4/d7/l32 9581000 1 2024-03-22T06:15:18.539 INFO:tasks.workunit.client.1.smithi117.stdout:8/149: dwrite d1/d28/f31 [0,4194304] 0 2024-03-22T06:15:18.539 INFO:tasks.workunit.client.1.smithi117.stdout:8/150: write d1/d2/fa [814250,54042] 0 2024-03-22T06:15:18.539 INFO:tasks.workunit.client.1.smithi117.stdout:8/151: write d1/d2/d11/f2a [709115,119626] 0 2024-03-22T06:15:18.545 INFO:tasks.workunit.client.1.smithi117.stdout:1/144: link d1/l4 d1/d14/d16/l2d 0 2024-03-22T06:15:18.546 INFO:tasks.workunit.client.0.smithi006.stdout:7/853: dwrite d5/d1e/d29/d1f/d2d/d55/f8c [4194304,4194304] 0 2024-03-22T06:15:18.547 INFO:tasks.workunit.client.0.smithi006.stdout:9/816: dwrite d8/d10/d11/f42 [0,4194304] 0 2024-03-22T06:15:18.547 INFO:tasks.workunit.client.0.smithi006.stdout:9/817: write d8/db3/d50/d91/f109 [1047828,122964] 0 2024-03-22T06:15:18.549 INFO:tasks.workunit.client.0.smithi006.stdout:4/935: mknod d6/c148 0 2024-03-22T06:15:18.550 INFO:tasks.workunit.client.1.smithi117.stdout:3/101: truncate f8 16793 0 2024-03-22T06:15:18.552 INFO:tasks.workunit.client.1.smithi117.stdout:2/132: rename d2/d5/d8/l1a to d2/d5/d13/l21 0 2024-03-22T06:15:18.552 INFO:tasks.workunit.client.1.smithi117.stdout:2/133: write d2/d5/d20/f18 [1642464,73739] 0 2024-03-22T06:15:18.553 INFO:tasks.workunit.client.1.smithi117.stdout:2/134: chown d2/l12 179044 1 2024-03-22T06:15:18.554 INFO:tasks.workunit.client.0.smithi006.stdout:7/854: rename d5/d1e/d29/d1f/d2d/d41/ff5 to d5/d1e/d29/d59/d5c/f116 0 2024-03-22T06:15:18.554 INFO:tasks.workunit.client.0.smithi006.stdout:1/845: dwrite d8/d9/d3a/f92 [0,4194304] 0 2024-03-22T06:15:18.555 INFO:tasks.workunit.client.0.smithi006.stdout:1/846: chown d8/d9/d13 0 1 2024-03-22T06:15:18.556 INFO:tasks.workunit.client.1.smithi117.stdout:1/145: dread d1/fc [4194304,4194304] 0 2024-03-22T06:15:18.558 INFO:tasks.workunit.client.1.smithi117.stdout:4/179: mknod d0/d4/c3c 0 2024-03-22T06:15:18.558 INFO:tasks.workunit.client.0.smithi006.stdout:6/844: sync 2024-03-22T06:15:18.563 INFO:tasks.workunit.client.1.smithi117.stdout:9/194: mknod d4/d8/c3c 0 2024-03-22T06:15:18.563 INFO:tasks.workunit.client.1.smithi117.stdout:9/195: chown d4/d7/d10/d2f/f39 29296609 1 2024-03-22T06:15:18.564 INFO:tasks.workunit.client.0.smithi006.stdout:9/818: unlink d8/f21 0 2024-03-22T06:15:18.565 INFO:tasks.workunit.client.0.smithi006.stdout:7/855: dread d5/d1e/d29/d1f/d2d/d41/f69 [0,4194304] 0 2024-03-22T06:15:18.570 INFO:tasks.workunit.client.1.smithi117.stdout:9/196: read d4/d7/f16 [2578843,113682] 0 2024-03-22T06:15:18.570 INFO:tasks.workunit.client.1.smithi117.stdout:9/197: readlink d4/d8/lb 0 2024-03-22T06:15:18.570 INFO:tasks.workunit.client.1.smithi117.stdout:9/198: write d4/d7/d10/f28 [2101046,61213] 0 2024-03-22T06:15:18.571 INFO:tasks.workunit.client.0.smithi006.stdout:6/845: rename d6/d1d/d5c/d81/d8a/dac/le7 to d6/d1b/d2c/l100 0 2024-03-22T06:15:18.573 INFO:tasks.workunit.client.1.smithi117.stdout:8/152: unlink d1/f30 0 2024-03-22T06:15:18.573 INFO:tasks.workunit.client.1.smithi117.stdout:8/153: chown d1/l8 94971222 1 2024-03-22T06:15:18.575 INFO:tasks.workunit.client.1.smithi117.stdout:7/135: mkdir d8/d9/da/d10/d23/d27 0 2024-03-22T06:15:18.575 INFO:tasks.workunit.client.0.smithi006.stdout:9/819: rmdir d8/d10/d20/d45 39 2024-03-22T06:15:18.577 INFO:tasks.workunit.client.0.smithi006.stdout:7/856: write d5/d1e/d63/d85/dca/f101 [3034876,51318] 0 2024-03-22T06:15:18.578 INFO:tasks.workunit.client.1.smithi117.stdout:2/135: creat d2/d5/d8/f22 x:0 0 0 2024-03-22T06:15:18.580 INFO:tasks.workunit.client.0.smithi006.stdout:4/936: rmdir d6/d1b/d20/d21/d77/da0 39 2024-03-22T06:15:18.580 INFO:tasks.workunit.client.1.smithi117.stdout:3/102: dwrite d9/f18 [0,4194304] 0 2024-03-22T06:15:18.581 INFO:tasks.workunit.client.0.smithi006.stdout:1/847: dwrite d8/d9/d13/d39/fc9 [0,4194304] 0 2024-03-22T06:15:18.581 INFO:tasks.workunit.client.0.smithi006.stdout:1/848: dread - d8/d9/d13/d39/d60/d8f/d10a/d59/db1/f11a zero size 2024-03-22T06:15:18.581 INFO:tasks.workunit.client.0.smithi006.stdout:1/849: readlink d8/d9/d13/d39/d60/d8f/d10a/d44/d80/le0 0 2024-03-22T06:15:18.582 INFO:tasks.workunit.client.1.smithi117.stdout:1/146: symlink d1/d22/l2e 0 2024-03-22T06:15:18.584 INFO:tasks.workunit.client.1.smithi117.stdout:0/170: sync 2024-03-22T06:15:18.585 INFO:tasks.workunit.client.0.smithi006.stdout:6/846: symlink d6/d1b/dae/dbc/l101 0 2024-03-22T06:15:18.588 INFO:tasks.workunit.client.1.smithi117.stdout:7/136: dread f7 [0,4194304] 0 2024-03-22T06:15:18.588 INFO:tasks.workunit.client.1.smithi117.stdout:8/154: write d1/d2/d9/f13 [5151171,96838] 0 2024-03-22T06:15:18.589 INFO:tasks.workunit.client.1.smithi117.stdout:4/180: rename d0/f38 to d0/d32/f3d 0 2024-03-22T06:15:18.589 INFO:tasks.workunit.client.1.smithi117.stdout:4/181: chown d0/f27 2088 1 2024-03-22T06:15:18.593 INFO:tasks.workunit.client.1.smithi117.stdout:6/178: link d3/f4 d3/f3c 0 2024-03-22T06:15:18.595 INFO:tasks.workunit.client.1.smithi117.stdout:4/182: dread d0/d4/d2a/d2f/f33 [0,4194304] 0 2024-03-22T06:15:18.595 INFO:tasks.workunit.client.0.smithi006.stdout:4/937: link d6/d1b/d20/d63/d93/lb3 d6/d13/d3b/l149 0 2024-03-22T06:15:18.596 INFO:tasks.workunit.client.1.smithi117.stdout:9/199: symlink d4/d7/l3d 0 2024-03-22T06:15:18.596 INFO:tasks.workunit.client.1.smithi117.stdout:9/200: read - d4/d7/f2a zero size 2024-03-22T06:15:18.604 INFO:tasks.workunit.client.0.smithi006.stdout:9/820: dwrite d8/d10/d11/f16 [0,4194304] 0 2024-03-22T06:15:18.606 INFO:tasks.workunit.client.0.smithi006.stdout:7/857: dwrite d5/f49 [0,4194304] 0 2024-03-22T06:15:18.610 INFO:tasks.workunit.client.0.smithi006.stdout:6/847: dwrite d6/f66 [0,4194304] 0 2024-03-22T06:15:18.610 INFO:tasks.workunit.client.0.smithi006.stdout:6/848: readlink d6/d1d/d5c/d89/lbd 0 2024-03-22T06:15:18.611 INFO:tasks.workunit.client.0.smithi006.stdout:1/850: rename d8/d9/d13/d39/d60/d8f/ce2 to d8/d9/d13/d39/d60/d8f/d10a/c120 0 2024-03-22T06:15:18.611 INFO:tasks.workunit.client.0.smithi006.stdout:1/851: creat d8/d9/d13/d39/f121 x:0 0 0 2024-03-22T06:15:18.614 INFO:tasks.workunit.client.0.smithi006.stdout:4/938: symlink d6/d1b/d20/d21/d2a/d43/da8/l14a 0 2024-03-22T06:15:18.618 INFO:tasks.workunit.client.0.smithi006.stdout:7/858: mknod d5/d1e/d29/db9/c117 0 2024-03-22T06:15:18.619 INFO:tasks.workunit.client.0.smithi006.stdout:7/859: stat d5/d1e/d29/d1f/d2d/d41 0 2024-03-22T06:15:18.619 INFO:tasks.workunit.client.1.smithi117.stdout:3/103: symlink d9/l23 0 2024-03-22T06:15:18.620 INFO:tasks.workunit.client.0.smithi006.stdout:6/849: truncate d6/d7c/d92/fc8 1025602 0 2024-03-22T06:15:18.621 INFO:tasks.workunit.client.1.smithi117.stdout:0/171: mkdir d1/d10/d18/d26/dc/d39 0 2024-03-22T06:15:18.621 INFO:tasks.workunit.client.1.smithi117.stdout:0/172: dread - d1/d10/d18/f38 zero size 2024-03-22T06:15:18.625 INFO:tasks.workunit.client.0.smithi006.stdout:4/939: rename d6/d1b/c34 to d6/d1b/d20/d21/d2a/c14b 0 2024-03-22T06:15:18.627 INFO:tasks.workunit.client.0.smithi006.stdout:6/850: dread d6/d24/fa8 [0,4194304] 0 2024-03-22T06:15:18.627 INFO:tasks.workunit.client.0.smithi006.stdout:6/851: chown d6/d1b/d2c/d4a/c61 37414287 1 2024-03-22T06:15:18.627 INFO:tasks.workunit.client.0.smithi006.stdout:6/852: fdatasync d6/f18 0 2024-03-22T06:15:18.628 INFO:tasks.workunit.client.0.smithi006.stdout:7/860: creat d5/d1e/d29/f118 x:0 0 0 2024-03-22T06:15:18.629 INFO:tasks.workunit.client.0.smithi006.stdout:9/821: dwrite d8/d10/d20/d45/d4d/f85 [0,4194304] 0 2024-03-22T06:15:18.630 INFO:tasks.workunit.client.0.smithi006.stdout:1/852: dwrite d8/d9/d13/d39/d60/d8f/d10a/d44/f110 [0,4194304] 0 2024-03-22T06:15:18.631 INFO:tasks.workunit.client.1.smithi117.stdout:2/136: rename d2/d5/db/le to d2/d5/d8/d10/l23 0 2024-03-22T06:15:18.631 INFO:tasks.workunit.client.1.smithi117.stdout:2/137: dread - d2/d5/d13/f19 zero size 2024-03-22T06:15:18.636 INFO:tasks.workunit.client.0.smithi006.stdout:7/861: mknod d5/d1e/d63/d85/c119 0 2024-03-22T06:15:18.637 INFO:tasks.workunit.client.0.smithi006.stdout:4/940: dread d6/fb [0,4194304] 0 2024-03-22T06:15:18.637 INFO:tasks.workunit.client.0.smithi006.stdout:4/941: truncate d6/d1b/d20/d21/d2a/dc9/f127 543660 0 2024-03-22T06:15:18.640 INFO:tasks.workunit.client.0.smithi006.stdout:6/853: dread d6/d1d/d99/fb1 [0,4194304] 0 2024-03-22T06:15:18.641 INFO:tasks.workunit.client.0.smithi006.stdout:9/822: dread d8/d10/d20/d45/d4d/f102 [0,4194304] 0 2024-03-22T06:15:18.642 INFO:tasks.workunit.client.1.smithi117.stdout:0/173: symlink d1/d10/d18/d26/l3a 0 2024-03-22T06:15:18.642 INFO:tasks.workunit.client.1.smithi117.stdout:0/174: chown d1/d10/d18/d26/dc/cd 2 1 2024-03-22T06:15:18.644 INFO:tasks.workunit.client.1.smithi117.stdout:8/155: truncate d1/d2/fa 432432 0 2024-03-22T06:15:18.644 INFO:tasks.workunit.client.1.smithi117.stdout:2/138: truncate d2/f3 5933847 0 2024-03-22T06:15:18.648 INFO:tasks.workunit.client.1.smithi117.stdout:4/183: symlink d0/d19/l3e 0 2024-03-22T06:15:18.649 INFO:tasks.workunit.client.1.smithi117.stdout:0/175: mkdir d1/d3/d2c/de/d1f/d3b 0 2024-03-22T06:15:18.649 INFO:tasks.workunit.client.1.smithi117.stdout:0/176: creat d1/d3/d2c/de/d1f/f3c x:0 0 0 2024-03-22T06:15:18.649 INFO:tasks.workunit.client.1.smithi117.stdout:0/177: chown d1/d3/d2b/f2f 1 1 2024-03-22T06:15:18.650 INFO:tasks.workunit.client.0.smithi006.stdout:7/862: creat d5/d1e/d29/d1f/d2d/df7/f11a x:0 0 0 2024-03-22T06:15:18.650 INFO:tasks.workunit.client.0.smithi006.stdout:7/863: read - d5/d1e/d29/d1f/de4/f10d zero size 2024-03-22T06:15:18.651 INFO:tasks.workunit.client.1.smithi117.stdout:2/139: creat d2/d5/db/f24 x:0 0 0 2024-03-22T06:15:18.651 INFO:tasks.workunit.client.1.smithi117.stdout:2/140: fsync d2/d5/d20/f1f 0 2024-03-22T06:15:18.652 INFO:tasks.workunit.client.0.smithi006.stdout:9/823: rmdir d8/d10/d82/df7 39 2024-03-22T06:15:18.652 INFO:tasks.workunit.client.0.smithi006.stdout:9/824: write d8/d10/d20/d45/d4d/fd8 [992324,66605] 0 2024-03-22T06:15:18.653 INFO:tasks.workunit.client.0.smithi006.stdout:7/864: truncate d5/d1e/fcc 3777586 0 2024-03-22T06:15:18.655 INFO:tasks.workunit.client.0.smithi006.stdout:4/942: symlink d6/d1b/d20/d21/d77/l14c 0 2024-03-22T06:15:18.655 INFO:tasks.workunit.client.1.smithi117.stdout:0/178: dread d1/d10/f27 [0,4194304] 0 2024-03-22T06:15:18.656 INFO:tasks.workunit.client.1.smithi117.stdout:0/179: chown d1/d3/f12 3 1 2024-03-22T06:15:18.656 INFO:tasks.workunit.client.1.smithi117.stdout:0/180: creat d1/f3d x:0 0 0 2024-03-22T06:15:18.657 INFO:tasks.workunit.client.0.smithi006.stdout:9/825: mkdir d8/db3/d10c 0 2024-03-22T06:15:18.657 INFO:tasks.workunit.client.0.smithi006.stdout:9/826: dread - d8/d10/d28/d39/fdf zero size 2024-03-22T06:15:18.657 INFO:tasks.workunit.client.0.smithi006.stdout:9/827: chown d8/d1f/d34/ff8 169454 1 2024-03-22T06:15:18.659 INFO:tasks.workunit.client.0.smithi006.stdout:4/943: getdents d6/d1b/d20/d21/d2a/d5c 0 2024-03-22T06:15:18.659 INFO:tasks.workunit.client.0.smithi006.stdout:4/944: truncate d6/d13/f17 6834627 0 2024-03-22T06:15:18.659 INFO:tasks.workunit.client.0.smithi006.stdout:4/945: write d6/d1b/d20/d21/d2a/d43/d4b/f138 [294496,52093] 0 2024-03-22T06:15:18.659 INFO:tasks.workunit.client.0.smithi006.stdout:4/946: dread d6/d1b/d20/d63/d93/d9a/d8f/da1/fb6 [0,4194304] 0 2024-03-22T06:15:18.659 INFO:tasks.workunit.client.0.smithi006.stdout:4/947: chown f2 1588494810 1 2024-03-22T06:15:18.660 INFO:tasks.workunit.client.0.smithi006.stdout:4/948: chown d6/d13/f104 0 1 2024-03-22T06:15:18.660 INFO:tasks.workunit.client.1.smithi117.stdout:2/141: read d2/fc [5465263,35404] 0 2024-03-22T06:15:18.660 INFO:tasks.workunit.client.1.smithi117.stdout:8/156: dread d1/d2/f1d [4194304,4194304] 0 2024-03-22T06:15:18.661 INFO:tasks.workunit.client.1.smithi117.stdout:6/179: rename d3/df to d3/d3d 0 2024-03-22T06:15:18.662 INFO:tasks.workunit.client.1.smithi117.stdout:4/184: write d0/d4/f17 [375153,11702] 0 2024-03-22T06:15:18.662 INFO:tasks.workunit.client.1.smithi117.stdout:4/185: write d0/d4/d2a/d2f/f31 [787071,50915] 0 2024-03-22T06:15:18.663 INFO:tasks.workunit.client.0.smithi006.stdout:4/949: link d6/d1b/d20/d63/d93/fc6 d6/d13/f14d 0 2024-03-22T06:15:18.663 INFO:tasks.workunit.client.0.smithi006.stdout:4/950: stat d6/d13/l56 0 2024-03-22T06:15:18.663 INFO:tasks.workunit.client.0.smithi006.stdout:4/951: getdents d6/d13/d3b/d134 0 2024-03-22T06:15:18.663 INFO:tasks.workunit.client.0.smithi006.stdout:4/952: chown d6/d1b/d20/l65 0 1 2024-03-22T06:15:18.664 INFO:tasks.workunit.client.1.smithi117.stdout:8/157: mknod d1/d28/c33 0 2024-03-22T06:15:18.664 INFO:tasks.workunit.client.1.smithi117.stdout:8/158: creat d1/f34 x:0 0 0 2024-03-22T06:15:18.664 INFO:tasks.workunit.client.1.smithi117.stdout:8/159: write d1/d2/f1d [8918269,100027] 0 2024-03-22T06:15:18.664 INFO:tasks.workunit.client.1.smithi117.stdout:8/160: fdatasync d1/d2/fa 0 2024-03-22T06:15:18.664 INFO:tasks.workunit.client.1.smithi117.stdout:8/161: chown d1/d19/l32 10089128 1 2024-03-22T06:15:18.665 INFO:tasks.workunit.client.0.smithi006.stdout:9/828: getdents d8/d10/d20/d45/d4d 0 2024-03-22T06:15:18.667 INFO:tasks.workunit.client.0.smithi006.stdout:9/829: dread d8/d10/d20/d45/d49/f8c [0,4194304] 0 2024-03-22T06:15:18.667 INFO:tasks.workunit.client.0.smithi006.stdout:9/830: stat d8/d10/d28/l26 0 2024-03-22T06:15:18.667 INFO:tasks.workunit.client.0.smithi006.stdout:9/831: fsync d8/d10/d47/ffb 0 2024-03-22T06:15:18.668 INFO:tasks.workunit.client.1.smithi117.stdout:6/180: unlink d3/f1f 0 2024-03-22T06:15:18.668 INFO:tasks.workunit.client.1.smithi117.stdout:6/181: fsync d3/d7/da/d32/f38 0 2024-03-22T06:15:18.670 INFO:tasks.workunit.client.1.smithi117.stdout:4/186: mkdir d0/d3f 0 2024-03-22T06:15:18.672 INFO:tasks.workunit.client.0.smithi006.stdout:9/832: mknod d8/d10/da6/c10d 0 2024-03-22T06:15:18.673 INFO:tasks.workunit.client.0.smithi006.stdout:4/953: rmdir d6/d1b/d20/d63/d93/d9a/d8f/da1 39 2024-03-22T06:15:18.676 INFO:tasks.workunit.client.1.smithi117.stdout:8/162: mknod d1/d2/d11/c35 0 2024-03-22T06:15:18.677 INFO:tasks.workunit.client.1.smithi117.stdout:5/110: sync 2024-03-22T06:15:18.679 INFO:tasks.workunit.client.0.smithi006.stdout:9/833: symlink d8/d1f/l10e 0 2024-03-22T06:15:18.679 INFO:tasks.workunit.client.1.smithi117.stdout:0/181: read d1/d3/d2c/ff [1243863,17429] 0 2024-03-22T06:15:18.681 INFO:tasks.workunit.client.0.smithi006.stdout:7/865: dread d5/d1e/f3a [0,4194304] 0 2024-03-22T06:15:18.681 INFO:tasks.workunit.client.1.smithi117.stdout:6/182: mkdir d3/d7/da/d16/d24/d2d/d30/d3e 0 2024-03-22T06:15:18.681 INFO:tasks.workunit.client.1.smithi117.stdout:6/183: dread - d3/d7/da/de/f23 zero size 2024-03-22T06:15:18.682 INFO:tasks.workunit.client.1.smithi117.stdout:8/163: mkdir d1/d28/d36 0 2024-03-22T06:15:18.683 INFO:tasks.workunit.client.1.smithi117.stdout:5/111: creat dd/df/d12/f27 x:0 0 0 2024-03-22T06:15:18.684 INFO:tasks.workunit.client.0.smithi006.stdout:7/866: link d5/d1e/d29/cce d5/d1e/d3f/c11b 0 2024-03-22T06:15:18.687 INFO:tasks.workunit.client.0.smithi006.stdout:7/867: creat d5/d1e/d63/f11c x:0 0 0 2024-03-22T06:15:18.687 INFO:tasks.workunit.client.0.smithi006.stdout:7/868: stat d5/d1e/d63/d85/d80/fc4 0 2024-03-22T06:15:18.687 INFO:tasks.workunit.client.1.smithi117.stdout:8/164: symlink d1/l37 0 2024-03-22T06:15:18.688 INFO:tasks.workunit.client.1.smithi117.stdout:7/137: dwrite d8/d9/da/d10/f13 [0,4194304] 0 2024-03-22T06:15:18.688 INFO:tasks.workunit.client.0.smithi006.stdout:1/853: dwrite d8/d9/d3a/d18/ffe [0,4194304] 0 2024-03-22T06:15:18.689 INFO:tasks.workunit.client.0.smithi006.stdout:1/854: chown d8/d9/d13/d24/d32 221487 1 2024-03-22T06:15:18.689 INFO:tasks.workunit.client.0.smithi006.stdout:1/855: truncate d8/d71/fed 156988 0 2024-03-22T06:15:18.690 INFO:tasks.workunit.client.1.smithi117.stdout:5/112: mknod dd/c28 0 2024-03-22T06:15:18.690 INFO:tasks.workunit.client.1.smithi117.stdout:5/113: read f4 [3027165,100152] 0 2024-03-22T06:15:18.690 INFO:tasks.workunit.client.1.smithi117.stdout:5/114: creat dd/f29 x:0 0 0 2024-03-22T06:15:18.690 INFO:tasks.workunit.client.1.smithi117.stdout:5/115: read - dd/df/d10/f26 zero size 2024-03-22T06:15:18.690 INFO:tasks.workunit.client.1.smithi117.stdout:5/116: creat dd/df/f2a x:0 0 0 2024-03-22T06:15:18.690 INFO:tasks.workunit.client.1.smithi117.stdout:5/117: chown l5 210 1 2024-03-22T06:15:18.690 INFO:tasks.workunit.client.1.smithi117.stdout:5/118: fsync dd/d16/f22 0 2024-03-22T06:15:18.691 INFO:tasks.workunit.client.0.smithi006.stdout:7/869: mknod d5/d1e/d63/d85/dca/c11d 0 2024-03-22T06:15:18.692 INFO:tasks.workunit.client.1.smithi117.stdout:0/182: getdents d1/d10 0 2024-03-22T06:15:18.698 INFO:tasks.workunit.client.1.smithi117.stdout:6/184: rename d3/d7/da/fc to d3/f3f 0 2024-03-22T06:15:18.698 INFO:tasks.workunit.client.1.smithi117.stdout:6/185: write d3/f20 [1020942,16713] 0 2024-03-22T06:15:18.699 INFO:tasks.workunit.client.0.smithi006.stdout:6/854: dwrite d6/d6f/d94/fa9 [0,4194304] 0 2024-03-22T06:15:18.700 INFO:tasks.workunit.client.1.smithi117.stdout:6/186: write d3/f4 [646332,32168] 0 2024-03-22T06:15:18.703 INFO:tasks.workunit.client.1.smithi117.stdout:5/119: dread dd/d16/f22 [0,4194304] 0 2024-03-22T06:15:18.703 INFO:tasks.workunit.client.1.smithi117.stdout:4/187: dwrite d0/d19/d28/f2c [0,4194304] 0 2024-03-22T06:15:18.705 INFO:tasks.workunit.client.0.smithi006.stdout:9/834: dread d8/db3/d50/d91/f109 [0,4194304] 0 2024-03-22T06:15:18.706 INFO:tasks.workunit.client.1.smithi117.stdout:4/188: write d0/d19/f22 [394773,77566] 0 2024-03-22T06:15:18.707 INFO:tasks.workunit.client.1.smithi117.stdout:7/138: dread d8/f16 [0,4194304] 0 2024-03-22T06:15:18.707 INFO:tasks.workunit.client.1.smithi117.stdout:2/142: dwrite d2/fc [4194304,4194304] 0 2024-03-22T06:15:18.712 INFO:tasks.workunit.client.0.smithi006.stdout:9/835: symlink d8/d10/d20/d45/d4d/l10f 0 2024-03-22T06:15:18.712 INFO:tasks.workunit.client.0.smithi006.stdout:6/855: dread d6/d7c/d83/fa2 [0,4194304] 0 2024-03-22T06:15:18.712 INFO:tasks.workunit.client.0.smithi006.stdout:4/954: dwrite d6/d1b/d20/d21/d9f/fad [4194304,4194304] 0 2024-03-22T06:15:18.712 INFO:tasks.workunit.client.0.smithi006.stdout:6/856: creat d6/d1d/d99/f102 x:0 0 0 2024-03-22T06:15:18.714 INFO:tasks.workunit.client.1.smithi117.stdout:5/120: link dd/df/f1b dd/df/f2b 0 2024-03-22T06:15:18.717 INFO:tasks.workunit.client.0.smithi006.stdout:9/836: rename d8/db3/d50/f69 to d8/db3/f110 0 2024-03-22T06:15:18.721 INFO:tasks.workunit.client.0.smithi006.stdout:1/856: dwrite d8/d9/d13/d24/fa3 [0,4194304] 0 2024-03-22T06:15:18.721 INFO:tasks.workunit.client.0.smithi006.stdout:1/857: write d8/d9/d13/d24/d32/dc7/f109 [65413,107853] 0 2024-03-22T06:15:18.721 INFO:tasks.workunit.client.0.smithi006.stdout:1/858: readlink d8/d33/lc1 0 2024-03-22T06:15:18.721 INFO:tasks.workunit.client.0.smithi006.stdout:1/859: fdatasync d8/d9/d10/d7c/f100 0 2024-03-22T06:15:18.721 INFO:tasks.workunit.client.1.smithi117.stdout:8/165: dwrite d1/d2/d9/f13 [8388608,4194304] 0 2024-03-22T06:15:18.722 INFO:tasks.workunit.client.1.smithi117.stdout:8/166: chown d1/d2/l25 456991586 1 2024-03-22T06:15:18.723 INFO:tasks.workunit.client.0.smithi006.stdout:7/870: dwrite d5/d1e/d29/d1f/de4/f10d [0,4194304] 0 2024-03-22T06:15:18.727 INFO:tasks.workunit.client.1.smithi117.stdout:7/139: creat d8/d9/da/f28 x:0 0 0 2024-03-22T06:15:18.730 INFO:tasks.workunit.client.0.smithi006.stdout:9/837: rename d8/d10/d20/d45/db9/de0/ff6 to d8/d10/d11/f111 0 2024-03-22T06:15:18.733 INFO:tasks.workunit.client.1.smithi117.stdout:2/143: unlink d2/f3 0 2024-03-22T06:15:18.736 INFO:tasks.workunit.client.1.smithi117.stdout:0/183: rename d1/d3/d2c to d1/d3/d2b/d3e 0 2024-03-22T06:15:18.739 INFO:tasks.workunit.client.0.smithi006.stdout:1/860: creat d8/d9/d13/d24/f122 x:0 0 0 2024-03-22T06:15:18.744 INFO:tasks.workunit.client.0.smithi006.stdout:4/955: dwrite d6/d1b/d20/d63/d93/fb0 [0,4194304] 0 2024-03-22T06:15:18.744 INFO:tasks.workunit.client.0.smithi006.stdout:4/956: chown f3 7327988 1 2024-03-22T06:15:18.746 INFO:tasks.workunit.client.1.smithi117.stdout:5/121: dwrite dd/df/d12/f27 [0,4194304] 0 2024-03-22T06:15:18.746 INFO:tasks.workunit.client.0.smithi006.stdout:6/857: dwrite d6/d1d/d99/fbe [0,4194304] 0 2024-03-22T06:15:18.746 INFO:tasks.workunit.client.0.smithi006.stdout:6/858: creat d6/d1b/dae/dbc/f103 x:0 0 0 2024-03-22T06:15:18.747 INFO:tasks.workunit.client.0.smithi006.stdout:7/871: rmdir d5/d1e/d63/d85/dac/db3 39 2024-03-22T06:15:18.749 INFO:tasks.workunit.client.0.smithi006.stdout:9/838: truncate d8/d10/d28/fa7 830377 0 2024-03-22T06:15:18.751 INFO:tasks.workunit.client.1.smithi117.stdout:8/167: mkdir d1/d19/d38 0 2024-03-22T06:15:18.754 INFO:tasks.workunit.client.0.smithi006.stdout:4/957: mknod d6/d1b/d20/d21/d77/da0/c14e 0 2024-03-22T06:15:18.754 INFO:tasks.workunit.client.1.smithi117.stdout:7/140: dwrite d8/d9/f14 [0,4194304] 0 2024-03-22T06:15:18.756 INFO:tasks.workunit.client.0.smithi006.stdout:1/861: dwrite d8/d9/d13/d39/d60/d8f/d10a/d44/f66 [0,4194304] 0 2024-03-22T06:15:18.757 INFO:tasks.workunit.client.1.smithi117.stdout:2/144: symlink d2/d1b/l25 0 2024-03-22T06:15:18.757 INFO:tasks.workunit.client.1.smithi117.stdout:5/122: dread f4 [0,4194304] 0 2024-03-22T06:15:18.760 INFO:tasks.workunit.client.0.smithi006.stdout:6/859: creat d6/d1b/d2c/f104 x:0 0 0 2024-03-22T06:15:18.761 INFO:tasks.workunit.client.1.smithi117.stdout:0/184: mkdir d1/d3/d2b/d3e/d3f 0 2024-03-22T06:15:18.761 INFO:tasks.workunit.client.0.smithi006.stdout:3/974: sync 2024-03-22T06:15:18.762 INFO:tasks.workunit.client.0.smithi006.stdout:5/960: sync 2024-03-22T06:15:18.762 INFO:tasks.workunit.client.0.smithi006.stdout:0/714: sync 2024-03-22T06:15:18.762 INFO:tasks.workunit.client.0.smithi006.stdout:5/961: fdatasync d0/db/d50/f80 0 2024-03-22T06:15:18.762 INFO:tasks.workunit.client.0.smithi006.stdout:0/715: dread - db/d87/faf zero size 2024-03-22T06:15:18.762 INFO:tasks.workunit.client.0.smithi006.stdout:0/716: write db/dc/d38/f66 [10592,82531] 0 2024-03-22T06:15:18.763 INFO:tasks.workunit.client.0.smithi006.stdout:7/872: creat d5/f11e x:0 0 0 2024-03-22T06:15:18.764 INFO:tasks.workunit.client.0.smithi006.stdout:9/839: mkdir d8/d10/d28/d39/d112 0 2024-03-22T06:15:18.764 INFO:tasks.workunit.client.1.smithi117.stdout:2/145: dread d2/d5/d8/d10/d11/f15 [0,4194304] 0 2024-03-22T06:15:18.768 INFO:tasks.workunit.client.1.smithi117.stdout:8/168: symlink d1/d1f/l39 0 2024-03-22T06:15:18.770 INFO:tasks.workunit.client.1.smithi117.stdout:5/123: dread fa [0,4194304] 0 2024-03-22T06:15:18.770 INFO:tasks.workunit.client.1.smithi117.stdout:5/124: chown dd/d16/d1a/c21 697 1 2024-03-22T06:15:18.772 INFO:tasks.workunit.client.0.smithi006.stdout:3/975: creat d0/d4/dbb/df7/d5f/d90/da7/de7/d147/de0/d14b/f154 x:0 0 0 2024-03-22T06:15:18.773 INFO:tasks.workunit.client.0.smithi006.stdout:1/862: write d8/d9/d13/f26 [5243290,115568] 0 2024-03-22T06:15:18.775 INFO:tasks.workunit.client.1.smithi117.stdout:7/141: chown d8/d9/da/d10/l15 31774 1 2024-03-22T06:15:18.779 INFO:tasks.workunit.client.0.smithi006.stdout:5/962: link d0/f5 d0/db/d50/d82/d103/d92/db7/dcb/f124 0 2024-03-22T06:15:18.780 INFO:tasks.workunit.client.1.smithi117.stdout:6/187: rename d3/d7/da/d16 to d3/d40 0 2024-03-22T06:15:18.782 INFO:tasks.workunit.client.1.smithi117.stdout:6/188: read d3/f20 [889208,119924] 0 2024-03-22T06:15:18.782 INFO:tasks.workunit.client.0.smithi006.stdout:0/717: rmdir db/d10 39 2024-03-22T06:15:18.783 INFO:tasks.workunit.client.1.smithi117.stdout:8/169: truncate d1/d28/f31 1850083 0 2024-03-22T06:15:18.783 INFO:tasks.workunit.client.1.smithi117.stdout:8/170: stat d1/d2/d9/f18 0 2024-03-22T06:15:18.784 INFO:tasks.workunit.client.0.smithi006.stdout:7/873: rmdir d5/d1e/d63/d85 39 2024-03-22T06:15:18.784 INFO:tasks.workunit.client.0.smithi006.stdout:7/874: chown d5/d1e/d63/d85/dca/f101 4265 1 2024-03-22T06:15:18.785 INFO:tasks.workunit.client.0.smithi006.stdout:7/875: write d5/d1e/d29/d59/f7a [889604,119352] 0 2024-03-22T06:15:18.785 INFO:tasks.workunit.client.0.smithi006.stdout:7/876: creat d5/d1e/d29/d1f/d2d/d61/daa/f11f x:0 0 0 2024-03-22T06:15:18.785 INFO:tasks.workunit.client.1.smithi117.stdout:5/125: truncate f4 2565085 0 2024-03-22T06:15:18.786 INFO:tasks.workunit.client.0.smithi006.stdout:9/840: mkdir d8/d10/d47/dfc/d113 0 2024-03-22T06:15:18.786 INFO:tasks.workunit.client.1.smithi117.stdout:6/189: stat d3/l1d 0 2024-03-22T06:15:18.788 INFO:tasks.workunit.client.1.smithi117.stdout:8/171: rmdir d1/d28/d36 0 2024-03-22T06:15:18.789 INFO:tasks.workunit.client.0.smithi006.stdout:3/976: mkdir d0/d4/dbb/df7/d5f/d83/d155 0 2024-03-22T06:15:18.791 INFO:tasks.workunit.client.1.smithi117.stdout:6/190: mknod d3/d7/da/de/d2c/d3a/c41 0 2024-03-22T06:15:18.791 INFO:tasks.workunit.client.1.smithi117.stdout:6/191: fdatasync d3/d7/f11 0 2024-03-22T06:15:18.793 INFO:tasks.workunit.client.1.smithi117.stdout:5/126: getdents dd 0 2024-03-22T06:15:18.793 INFO:tasks.workunit.client.1.smithi117.stdout:5/127: stat dd/df/d10/f25 0 2024-03-22T06:15:18.794 INFO:tasks.workunit.client.1.smithi117.stdout:1/147: sync 2024-03-22T06:15:18.794 INFO:tasks.workunit.client.1.smithi117.stdout:3/104: sync 2024-03-22T06:15:18.794 INFO:tasks.workunit.client.1.smithi117.stdout:3/105: chown d9/f22 3 1 2024-03-22T06:15:18.795 INFO:tasks.workunit.client.1.smithi117.stdout:9/201: sync 2024-03-22T06:15:18.795 INFO:tasks.workunit.client.1.smithi117.stdout:6/192: truncate d3/d3d/f2e 293371 0 2024-03-22T06:15:18.797 INFO:tasks.workunit.client.1.smithi117.stdout:5/128: rmdir dd/d16/d1a 39 2024-03-22T06:15:18.797 INFO:tasks.workunit.client.1.smithi117.stdout:5/129: chown dd/f29 140050673 1 2024-03-22T06:15:18.797 INFO:tasks.workunit.client.1.smithi117.stdout:5/130: truncate dd/d16/f22 5238732 0 2024-03-22T06:15:18.797 INFO:tasks.workunit.client.1.smithi117.stdout:5/131: creat dd/df/f2c x:0 0 0 2024-03-22T06:15:18.797 INFO:tasks.workunit.client.1.smithi117.stdout:5/132: readlink lc 0 2024-03-22T06:15:18.797 INFO:tasks.workunit.client.1.smithi117.stdout:5/133: chown dd/df/d12/c18 36649 1 2024-03-22T06:15:18.798 INFO:tasks.workunit.client.1.smithi117.stdout:1/148: truncate d1/fd 213447 0 2024-03-22T06:15:18.798 INFO:tasks.workunit.client.1.smithi117.stdout:7/142: dread d8/d9/fd [4194304,4194304] 0 2024-03-22T06:15:18.798 INFO:tasks.workunit.client.0.smithi006.stdout:7/877: stat d5/d1e/d29/l14 0 2024-03-22T06:15:18.799 INFO:tasks.workunit.client.1.smithi117.stdout:9/202: mkdir d4/d8/d3e 0 2024-03-22T06:15:18.801 INFO:tasks.workunit.client.1.smithi117.stdout:6/193: mkdir d3/d40/d24/d2d/d42 0 2024-03-22T06:15:18.801 INFO:tasks.workunit.client.1.smithi117.stdout:6/194: dread - d3/d7/da/de/f23 zero size 2024-03-22T06:15:18.801 INFO:tasks.workunit.client.1.smithi117.stdout:6/195: fdatasync d3/d7/da/d32/f38 0 2024-03-22T06:15:18.801 INFO:tasks.workunit.client.0.smithi006.stdout:9/841: mknod d8/db3/d10c/c114 0 2024-03-22T06:15:18.801 INFO:tasks.workunit.client.0.smithi006.stdout:9/842: creat d8/d10/d28/dba/f115 x:0 0 0 2024-03-22T06:15:18.802 INFO:tasks.workunit.client.0.smithi006.stdout:5/963: rmdir d0/d98 39 2024-03-22T06:15:18.803 INFO:tasks.workunit.client.1.smithi117.stdout:5/134: mknod dd/df/c2d 0 2024-03-22T06:15:18.804 INFO:tasks.workunit.client.0.smithi006.stdout:0/718: mknod db/d10/d35/d7b/da1/da6/dc4/cf3 0 2024-03-22T06:15:18.805 INFO:tasks.workunit.client.0.smithi006.stdout:0/719: write db/d10/d35/d7b/da1/da6/dc4/ff2 [561730,112471] 0 2024-03-22T06:15:18.805 INFO:tasks.workunit.client.0.smithi006.stdout:0/720: read - db/d10/d35/d7b/fe8 zero size 2024-03-22T06:15:18.805 INFO:tasks.workunit.client.0.smithi006.stdout:0/721: read db/d10/f1b [133436,126456] 0 2024-03-22T06:15:18.805 INFO:tasks.workunit.client.1.smithi117.stdout:1/149: rename d1/c11 to d1/d22/c2f 0 2024-03-22T06:15:18.807 INFO:tasks.workunit.client.1.smithi117.stdout:7/143: mknod d8/d9/d1a/c29 0 2024-03-22T06:15:18.807 INFO:tasks.workunit.client.1.smithi117.stdout:3/106: dread d9/d1a/f1e [0,4194304] 0 2024-03-22T06:15:18.807 INFO:tasks.workunit.client.1.smithi117.stdout:0/185: dwrite d1/d3/d2b/d3e/fb [0,4194304] 0 2024-03-22T06:15:18.807 INFO:tasks.workunit.client.1.smithi117.stdout:0/186: dread - d1/d10/f32 zero size 2024-03-22T06:15:18.808 INFO:tasks.workunit.client.0.smithi006.stdout:9/843: creat d8/d10/d47/dfc/f116 x:0 0 0 2024-03-22T06:15:18.811 INFO:tasks.workunit.client.0.smithi006.stdout:6/860: dwrite d6/d1b/f2d [0,4194304] 0 2024-03-22T06:15:18.811 INFO:tasks.workunit.client.0.smithi006.stdout:6/861: dread - d6/d1d/d99/dcd/fd0 zero size 2024-03-22T06:15:18.811 INFO:tasks.workunit.client.1.smithi117.stdout:6/196: creat d3/d3d/f43 x:0 0 0 2024-03-22T06:15:18.811 INFO:tasks.workunit.client.1.smithi117.stdout:6/197: fsync d3/d3d/d17/d31/f34 0 2024-03-22T06:15:18.812 INFO:tasks.workunit.client.1.smithi117.stdout:2/146: dwrite d2/d5/d8/d10/d11/f15 [0,4194304] 0 2024-03-22T06:15:18.812 INFO:tasks.workunit.client.1.smithi117.stdout:8/172: dwrite d1/d2/d11/f2a [0,4194304] 0 2024-03-22T06:15:18.812 INFO:tasks.workunit.client.0.smithi006.stdout:5/964: mknod d0/db/d50/d82/d103/d92/db7/dcb/c125 0 2024-03-22T06:15:18.814 INFO:tasks.workunit.client.1.smithi117.stdout:1/150: mkdir d1/d30 0 2024-03-22T06:15:18.816 INFO:tasks.workunit.client.1.smithi117.stdout:3/107: rename d9/cc to d9/c24 0 2024-03-22T06:15:18.816 INFO:tasks.workunit.client.1.smithi117.stdout:3/108: creat d9/f25 x:0 0 0 2024-03-22T06:15:18.816 INFO:tasks.workunit.client.1.smithi117.stdout:3/109: creat d9/f26 x:0 0 0 2024-03-22T06:15:18.818 INFO:tasks.workunit.client.0.smithi006.stdout:9/844: creat d8/d10/d20/d45/f117 x:0 0 0 2024-03-22T06:15:18.820 INFO:tasks.workunit.client.1.smithi117.stdout:7/144: mknod d8/d9/c2a 0 2024-03-22T06:15:18.822 INFO:tasks.workunit.client.0.smithi006.stdout:3/977: rename d0/d8/d1c/d89 to d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d12c/d156 0 2024-03-22T06:15:18.822 INFO:tasks.workunit.client.0.smithi006.stdout:3/978: readlink d0/d4/l85 0 2024-03-22T06:15:18.826 INFO:tasks.workunit.client.1.smithi117.stdout:9/203: rmdir d4/d7/d27 39 2024-03-22T06:15:18.826 INFO:tasks.workunit.client.1.smithi117.stdout:9/204: fdatasync d4/d8/f25 0 2024-03-22T06:15:18.828 INFO:tasks.workunit.client.1.smithi117.stdout:6/198: getdents d3/d40/d24/d2d/d30 0 2024-03-22T06:15:18.829 INFO:tasks.workunit.client.0.smithi006.stdout:7/878: dwrite d5/f9 [0,4194304] 0 2024-03-22T06:15:18.829 INFO:tasks.workunit.client.0.smithi006.stdout:7/879: dread - d5/d1e/d29/d59/d5c/fc7 zero size 2024-03-22T06:15:18.829 INFO:tasks.workunit.client.0.smithi006.stdout:7/880: write d5/d1e/d29/d1f/d2d/d41/db7/f106 [138377,21169] 0 2024-03-22T06:15:18.829 INFO:tasks.workunit.client.0.smithi006.stdout:5/965: mkdir d0/db/d50/d58/df7/d126 0 2024-03-22T06:15:18.832 INFO:tasks.workunit.client.1.smithi117.stdout:6/199: dread d3/d7/f1b [0,4194304] 0 2024-03-22T06:15:18.835 INFO:tasks.workunit.client.1.smithi117.stdout:2/147: creat d2/f26 x:0 0 0 2024-03-22T06:15:18.835 INFO:tasks.workunit.client.1.smithi117.stdout:0/187: dwrite d1/d3/d2b/d3e/fb [0,4194304] 0 2024-03-22T06:15:18.835 INFO:tasks.workunit.client.1.smithi117.stdout:1/151: dwrite d1/d14/f19 [4194304,4194304] 0 2024-03-22T06:15:18.835 INFO:tasks.workunit.client.1.smithi117.stdout:1/152: fsync d1/f8 0 2024-03-22T06:15:18.835 INFO:tasks.workunit.client.0.smithi006.stdout:9/845: mknod d8/d10/d82/df7/c118 0 2024-03-22T06:15:18.835 INFO:tasks.workunit.client.0.smithi006.stdout:9/846: chown d8/d10/d28/dba/fbd 761 1 2024-03-22T06:15:18.838 INFO:tasks.workunit.client.0.smithi006.stdout:5/966: dread d0/db/d50/d58/f91 [0,4194304] 0 2024-03-22T06:15:18.838 INFO:tasks.workunit.client.0.smithi006.stdout:0/722: rename db/d10/d2a/d41/lbc to db/d10/d35/d7b/lf4 0 2024-03-22T06:15:18.840 INFO:tasks.workunit.client.1.smithi117.stdout:8/173: mkdir d1/d3a 0 2024-03-22T06:15:18.842 INFO:tasks.workunit.client.0.smithi006.stdout:3/979: symlink d0/d4/dbb/df7/d5f/d90/da7/de7/d4c/l157 0 2024-03-22T06:15:18.846 INFO:tasks.workunit.client.0.smithi006.stdout:7/881: creat d5/d1e/d29/d66/f120 x:0 0 0 2024-03-22T06:15:18.847 INFO:tasks.workunit.client.1.smithi117.stdout:2/148: read d2/fc [8080417,42576] 0 2024-03-22T06:15:18.848 INFO:tasks.workunit.client.0.smithi006.stdout:9/847: truncate f1 3218272 0 2024-03-22T06:15:18.848 INFO:tasks.workunit.client.0.smithi006.stdout:9/848: truncate d8/d10/d20/d45/db9/de0/fae 3183744 0 2024-03-22T06:15:18.848 INFO:tasks.workunit.client.0.smithi006.stdout:4/958: dwrite d6/d13/d3b/fe2 [0,4194304] 0 2024-03-22T06:15:18.848 INFO:tasks.workunit.client.0.smithi006.stdout:4/959: write d6/f62 [1249346,113157] 0 2024-03-22T06:15:18.850 INFO:tasks.workunit.client.0.smithi006.stdout:6/862: rename d6/d24/d44/df3 to d6/d1b/dae/dbc/d105 0 2024-03-22T06:15:18.851 INFO:tasks.workunit.client.1.smithi117.stdout:7/145: mkdir d8/d9/da/d2b 0 2024-03-22T06:15:18.854 INFO:tasks.workunit.client.1.smithi117.stdout:9/205: mkdir d4/d8/d3f 0 2024-03-22T06:15:18.854 INFO:tasks.workunit.client.1.smithi117.stdout:9/206: stat d4/d8/d3f 0 2024-03-22T06:15:18.854 INFO:tasks.workunit.client.1.smithi117.stdout:9/207: write d4/d7/f2a [890539,103740] 0 2024-03-22T06:15:18.856 INFO:tasks.workunit.client.1.smithi117.stdout:6/200: creat d3/d40/d24/f44 x:0 0 0 2024-03-22T06:15:18.857 INFO:tasks.workunit.client.0.smithi006.stdout:0/723: dwrite db/dc/d38/f4b [4194304,4194304] 0 2024-03-22T06:15:18.857 INFO:tasks.workunit.client.0.smithi006.stdout:0/724: creat db/d10/d35/d7b/da1/da6/ff5 x:0 0 0 2024-03-22T06:15:18.858 INFO:tasks.workunit.client.0.smithi006.stdout:4/960: mkdir d6/d1b/d20/d21/d2a/d43/d4b/d14f 0 2024-03-22T06:15:18.859 INFO:tasks.workunit.client.1.smithi117.stdout:7/146: dread d8/f16 [0,4194304] 0 2024-03-22T06:15:18.859 INFO:tasks.workunit.client.1.smithi117.stdout:0/188: mkdir d1/d3/d2b/d3e/de/d14/d40 0 2024-03-22T06:15:18.863 INFO:tasks.workunit.client.1.smithi117.stdout:1/153: creat d1/d1f/f31 x:0 0 0 2024-03-22T06:15:18.868 INFO:tasks.workunit.client.0.smithi006.stdout:4/961: write d6/f1c [933740,107867] 0 2024-03-22T06:15:18.869 INFO:tasks.workunit.client.1.smithi117.stdout:8/174: rename d1/l37 to d1/d2/d11/l3b 0 2024-03-22T06:15:18.871 INFO:tasks.workunit.client.1.smithi117.stdout:2/149: creat d2/d5/d20/d1d/f27 x:0 0 0 2024-03-22T06:15:18.871 INFO:tasks.workunit.client.1.smithi117.stdout:2/150: chown d2/d5/d13/c1e 11274 1 2024-03-22T06:15:18.873 INFO:tasks.workunit.client.0.smithi006.stdout:5/967: rename d0/db/d50/d96 to d0/db/d99/d127 0 2024-03-22T06:15:18.874 INFO:tasks.workunit.client.1.smithi117.stdout:3/110: rmdir d9/d1a 39 2024-03-22T06:15:18.876 INFO:tasks.workunit.client.1.smithi117.stdout:0/189: creat d1/d3/d2b/d3e/de/d14/d40/f41 x:0 0 0 2024-03-22T06:15:18.877 INFO:tasks.workunit.client.0.smithi006.stdout:0/725: creat db/d10/d2a/ff6 x:0 0 0 2024-03-22T06:15:18.877 INFO:tasks.workunit.client.1.smithi117.stdout:4/189: sync 2024-03-22T06:15:18.878 INFO:tasks.workunit.client.0.smithi006.stdout:4/962: symlink d6/d1b/d20/d6b/d7b/l150 0 2024-03-22T06:15:18.878 INFO:tasks.workunit.client.0.smithi006.stdout:4/963: creat d6/d1b/d20/d6b/d88/f151 x:0 0 0 2024-03-22T06:15:18.879 INFO:tasks.workunit.client.1.smithi117.stdout:1/154: mkdir d1/d1f/d32 0 2024-03-22T06:15:18.881 INFO:tasks.workunit.client.0.smithi006.stdout:3/980: rename d0/d4/dbb/df7/d5f/l12b to d0/d8/d1c/d49/d79/l158 0 2024-03-22T06:15:18.883 INFO:tasks.workunit.client.0.smithi006.stdout:5/968: creat d0/d98/f128 x:0 0 0 2024-03-22T06:15:18.886 INFO:tasks.workunit.client.1.smithi117.stdout:8/175: creat d1/d2/f3c x:0 0 0 2024-03-22T06:15:18.890 INFO:tasks.workunit.client.1.smithi117.stdout:2/151: rename d2/d5/d20/f18 to d2/d1b/f28 0 2024-03-22T06:15:18.891 INFO:tasks.workunit.client.1.smithi117.stdout:2/152: dread - d2/d5/d20/f1f zero size 2024-03-22T06:15:18.891 INFO:tasks.workunit.client.1.smithi117.stdout:2/153: dread - d2/d5/d20/d1d/f27 zero size 2024-03-22T06:15:18.891 INFO:tasks.workunit.client.1.smithi117.stdout:6/201: dwrite d3/f20 [0,4194304] 0 2024-03-22T06:15:18.892 INFO:tasks.workunit.client.0.smithi006.stdout:4/964: mkdir d6/d1b/d20/d63/d152 0 2024-03-22T06:15:18.893 INFO:tasks.workunit.client.1.smithi117.stdout:3/111: symlink d9/l27 0 2024-03-22T06:15:18.893 INFO:tasks.workunit.client.1.smithi117.stdout:3/112: chown d9/c16 8128 1 2024-03-22T06:15:18.895 INFO:tasks.workunit.client.1.smithi117.stdout:9/208: dwrite d4/d7/f1b [4194304,4194304] 0 2024-03-22T06:15:18.895 INFO:tasks.workunit.client.0.smithi006.stdout:9/849: rename d8/d10/d20/d45/d4d/f85 to d8/d10/d47/dbc/f119 0 2024-03-22T06:15:18.895 INFO:tasks.workunit.client.0.smithi006.stdout:9/850: stat d8/d10/d20/d45/d49/fc7 0 2024-03-22T06:15:18.897 INFO:tasks.workunit.client.1.smithi117.stdout:9/209: read d4/d7/f1c [915565,130824] 0 2024-03-22T06:15:18.898 INFO:tasks.workunit.client.1.smithi117.stdout:0/190: link d1/d10/f32 d1/d3/d2b/d3e/de/d1f/d3b/f42 0 2024-03-22T06:15:18.898 INFO:tasks.workunit.client.1.smithi117.stdout:0/191: chown d1/d10/d18/d26/c2d 275781 1 2024-03-22T06:15:18.898 INFO:tasks.workunit.client.1.smithi117.stdout:0/192: chown d1/d3/d2b/d3e/de/d14 31636 1 2024-03-22T06:15:18.899 INFO:tasks.workunit.client.1.smithi117.stdout:4/190: symlink d0/d19/d34/l40 0 2024-03-22T06:15:18.904 INFO:tasks.workunit.client.1.smithi117.stdout:1/155: dwrite d1/f13 [0,4194304] 0 2024-03-22T06:15:18.906 INFO:tasks.workunit.client.1.smithi117.stdout:3/113: symlink d9/d1a/l28 0 2024-03-22T06:15:18.906 INFO:tasks.workunit.client.1.smithi117.stdout:3/114: truncate d9/f11 4597442 0 2024-03-22T06:15:18.907 INFO:tasks.workunit.client.0.smithi006.stdout:0/726: mkdir db/d10/df7 0 2024-03-22T06:15:18.907 INFO:tasks.workunit.client.0.smithi006.stdout:0/727: fdatasync db/d10/d35/d6c/fbb 0 2024-03-22T06:15:18.909 INFO:tasks.workunit.client.0.smithi006.stdout:0/728: write db/dc/d38/fb9 [1045623,61187] 0 2024-03-22T06:15:18.909 INFO:tasks.workunit.client.0.smithi006.stdout:6/863: rename d6/d1d/d32/c3b to d6/d1d/d99/de9/c106 0 2024-03-22T06:15:18.911 INFO:tasks.workunit.client.0.smithi006.stdout:9/851: rmdir d8/d10/d20/d103 39 2024-03-22T06:15:18.912 INFO:tasks.workunit.client.0.smithi006.stdout:0/729: mkdir db/d87/ddd/df8 0 2024-03-22T06:15:18.914 INFO:tasks.workunit.client.1.smithi117.stdout:0/193: rename d1/d10/d18/d26/c2d to d1/d10/d18/d26/dc/d39/c43 0 2024-03-22T06:15:18.917 INFO:tasks.workunit.client.0.smithi006.stdout:4/965: rename d6/d1b/d20/d6b/d60/db1/f13a to d6/d1b/d20/d63/d93/d9a/d8f/f153 0 2024-03-22T06:15:18.917 INFO:tasks.workunit.client.0.smithi006.stdout:4/966: chown d6/d1b/d20/d21/d77/df2/l106 15327731 1 2024-03-22T06:15:18.919 INFO:tasks.workunit.client.0.smithi006.stdout:5/969: dwrite d0/db/d16/d70/fd1 [4194304,4194304] 0 2024-03-22T06:15:18.920 INFO:tasks.workunit.client.1.smithi117.stdout:8/176: dwrite f0 [4194304,4194304] 0 2024-03-22T06:15:18.920 INFO:tasks.workunit.client.1.smithi117.stdout:1/156: read d1/f8 [2919289,109805] 0 2024-03-22T06:15:18.920 INFO:tasks.workunit.client.1.smithi117.stdout:6/202: dwrite d3/f3c [0,4194304] 0 2024-03-22T06:15:18.920 INFO:tasks.workunit.client.1.smithi117.stdout:6/203: getdents d3/d40 0 2024-03-22T06:15:18.920 INFO:tasks.workunit.client.1.smithi117.stdout:2/154: dwrite d2/d5/d13/f19 [0,4194304] 0 2024-03-22T06:15:18.920 INFO:tasks.workunit.client.1.smithi117.stdout:4/191: mknod d0/d19/d28/c41 0 2024-03-22T06:15:18.920 INFO:tasks.workunit.client.1.smithi117.stdout:4/192: chown d0/f1d 729957 1 2024-03-22T06:15:18.920 INFO:tasks.workunit.client.1.smithi117.stdout:4/193: creat d0/f42 x:0 0 0 2024-03-22T06:15:18.920 INFO:tasks.workunit.client.0.smithi006.stdout:6/864: mkdir d6/d1d/d5c/d89/d107 0 2024-03-22T06:15:18.923 INFO:tasks.workunit.client.1.smithi117.stdout:1/157: dread d1/fc [4194304,4194304] 0 2024-03-22T06:15:18.923 INFO:tasks.workunit.client.1.smithi117.stdout:1/158: write d1/f7 [1077500,39280] 0 2024-03-22T06:15:18.923 INFO:tasks.workunit.client.1.smithi117.stdout:1/159: chown d1/c6 670678002 1 2024-03-22T06:15:18.924 INFO:tasks.workunit.client.0.smithi006.stdout:0/730: mkdir db/d10/d33/dce/df9 0 2024-03-22T06:15:18.926 INFO:tasks.workunit.client.0.smithi006.stdout:4/967: mknod d6/d1b/d20/d21/d9f/c154 0 2024-03-22T06:15:18.927 INFO:tasks.workunit.client.0.smithi006.stdout:4/968: write d6/f10e [44532,102140] 0 2024-03-22T06:15:18.930 INFO:tasks.workunit.client.0.smithi006.stdout:5/970: creat d0/db/d16/d1c/d34/d4d/f129 x:0 0 0 2024-03-22T06:15:18.934 INFO:tasks.workunit.client.0.smithi006.stdout:6/865: mknod d6/d1d/d5c/dc4/c108 0 2024-03-22T06:15:18.934 INFO:tasks.workunit.client.1.smithi117.stdout:3/115: symlink d9/d1a/l29 0 2024-03-22T06:15:18.934 INFO:tasks.workunit.client.1.smithi117.stdout:3/116: chown d9/d1a/f1f 15 1 2024-03-22T06:15:18.934 INFO:tasks.workunit.client.1.smithi117.stdout:3/117: creat d9/f2a x:0 0 0 2024-03-22T06:15:18.935 INFO:tasks.workunit.client.1.smithi117.stdout:9/210: symlink d4/l40 0 2024-03-22T06:15:18.937 INFO:tasks.workunit.client.0.smithi006.stdout:4/969: rmdir d6/d1b/d20/d63/d93/d9a/d8f/da1/db5 39 2024-03-22T06:15:18.937 INFO:tasks.workunit.client.0.smithi006.stdout:9/852: write d8/d10/d28/f25 [1478224,10649] 0 2024-03-22T06:15:18.937 INFO:tasks.workunit.client.1.smithi117.stdout:9/211: read d4/f21 [300224,61655] 0 2024-03-22T06:15:18.937 INFO:tasks.workunit.client.1.smithi117.stdout:9/212: truncate d4/d7/f2d 1417972 0 2024-03-22T06:15:18.940 INFO:tasks.workunit.client.1.smithi117.stdout:6/204: link d3/d7/da/c26 d3/d40/d24/d2d/d42/c45 0 2024-03-22T06:15:18.940 INFO:tasks.workunit.client.1.smithi117.stdout:6/205: write d3/d3d/f43 [29923,117102] 0 2024-03-22T06:15:18.940 INFO:tasks.workunit.client.1.smithi117.stdout:6/206: read d3/f5 [4495182,118458] 0 2024-03-22T06:15:18.943 INFO:tasks.workunit.client.0.smithi006.stdout:6/866: rename d6/d1d/f20 to d6/d1d/d5c/d89/d107/f109 0 2024-03-22T06:15:18.946 INFO:tasks.workunit.client.0.smithi006.stdout:4/970: creat d6/d1b/d20/d21/d77/da0/f155 x:0 0 0 2024-03-22T06:15:18.948 INFO:tasks.workunit.client.1.smithi117.stdout:0/194: dwrite d1/d10/f11 [4194304,4194304] 0 2024-03-22T06:15:18.949 INFO:tasks.workunit.client.0.smithi006.stdout:3/981: write d0/fa [299627,34715] 0 2024-03-22T06:15:18.949 INFO:tasks.workunit.client.0.smithi006.stdout:3/982: dread - d0/d8/d1c/d49/de5/f145 zero size 2024-03-22T06:15:18.949 INFO:tasks.workunit.client.0.smithi006.stdout:3/983: dread - d0/d8/d88/ddf/df0/d112/f13d zero size 2024-03-22T06:15:18.951 INFO:tasks.workunit.client.0.smithi006.stdout:3/984: write d0/d4/dbb/df7/d5f/d90/da7/de7/d147/f117 [479890,76000] 0 2024-03-22T06:15:18.951 INFO:tasks.workunit.client.0.smithi006.stdout:9/853: dread d8/d10/d28/f44 [0,4194304] 0 2024-03-22T06:15:18.951 INFO:tasks.workunit.client.0.smithi006.stdout:9/854: fsync d8/d1f/d34/ff2 0 2024-03-22T06:15:18.952 INFO:tasks.workunit.client.0.smithi006.stdout:6/867: link d6/d1d/d5c/d81/f65 d6/d1d/d99/de9/f10a 0 2024-03-22T06:15:18.953 INFO:tasks.workunit.client.0.smithi006.stdout:6/868: stat d6/d24 0 2024-03-22T06:15:18.956 INFO:tasks.workunit.client.0.smithi006.stdout:4/971: rmdir d6/d1b/d20/d21/d2a/d43 39 2024-03-22T06:15:18.959 INFO:tasks.workunit.client.0.smithi006.stdout:9/855: dread d8/d10/d11/f3c [0,4194304] 0 2024-03-22T06:15:18.960 INFO:tasks.workunit.client.1.smithi117.stdout:2/155: mknod d2/d5/d8/d10/c29 0 2024-03-22T06:15:18.962 INFO:tasks.workunit.client.1.smithi117.stdout:9/213: dwrite d4/d7/d27/f2e [0,4194304] 0 2024-03-22T06:15:18.962 INFO:tasks.workunit.client.1.smithi117.stdout:6/207: dwrite d3/d7/f11 [0,4194304] 0 2024-03-22T06:15:18.964 INFO:tasks.workunit.client.0.smithi006.stdout:0/731: dwrite db/dc/d38/d50/f79 [0,4194304] 0 2024-03-22T06:15:18.964 INFO:tasks.workunit.client.0.smithi006.stdout:0/732: fdatasync db/d10/d35/d7b/da1/fd5 0 2024-03-22T06:15:18.965 INFO:tasks.workunit.client.1.smithi117.stdout:8/177: dwrite d1/d2/d9/f18 [0,4194304] 0 2024-03-22T06:15:18.965 INFO:tasks.workunit.client.1.smithi117.stdout:1/160: mkdir d1/d1f/d32/d33 0 2024-03-22T06:15:18.966 INFO:tasks.workunit.client.0.smithi006.stdout:0/733: dread db/d10/f1b [0,4194304] 0 2024-03-22T06:15:18.966 INFO:tasks.workunit.client.0.smithi006.stdout:4/972: creat d6/d1b/d20/d63/d93/d9a/d130/f156 x:0 0 0 2024-03-22T06:15:18.966 INFO:tasks.workunit.client.0.smithi006.stdout:4/973: truncate d6/d1b/d20/d21/d2a/dc9/f127 1410982 0 2024-03-22T06:15:18.968 INFO:tasks.workunit.client.1.smithi117.stdout:5/135: sync 2024-03-22T06:15:18.968 INFO:tasks.workunit.client.1.smithi117.stdout:5/136: write dd/df/f2a [769031,49090] 0 2024-03-22T06:15:18.969 INFO:tasks.workunit.client.1.smithi117.stdout:5/137: read - dd/df/d1e/f20 zero size 2024-03-22T06:15:18.971 INFO:tasks.workunit.client.0.smithi006.stdout:5/971: getdents d0/db/d16/d1c/d34/d4d 0 2024-03-22T06:15:18.972 INFO:tasks.workunit.client.1.smithi117.stdout:3/118: getdents d9/d1a 0 2024-03-22T06:15:18.973 INFO:tasks.workunit.client.1.smithi117.stdout:0/195: symlink d1/d10/d18/d26/l44 0 2024-03-22T06:15:18.974 INFO:tasks.workunit.client.0.smithi006.stdout:3/985: dwrite d0/d8/d88/dfa/de2/f110 [0,4194304] 0 2024-03-22T06:15:18.979 INFO:tasks.workunit.client.0.smithi006.stdout:6/869: dwrite d6/d1d/d99/fb1 [0,4194304] 0 2024-03-22T06:15:18.980 INFO:tasks.workunit.client.1.smithi117.stdout:7/147: sync 2024-03-22T06:15:18.981 INFO:tasks.workunit.client.1.smithi117.stdout:9/214: creat d4/d8/f41 x:0 0 0 2024-03-22T06:15:18.981 INFO:tasks.workunit.client.1.smithi117.stdout:9/215: dread - d4/d7/f24 zero size 2024-03-22T06:15:18.981 INFO:tasks.workunit.client.1.smithi117.stdout:9/216: fdatasync d4/d7/d10/f28 0 2024-03-22T06:15:18.986 INFO:tasks.workunit.client.1.smithi117.stdout:6/208: mkdir d3/d40/d46 0 2024-03-22T06:15:18.994 INFO:tasks.workunit.client.1.smithi117.stdout:8/178: rmdir d1/d2/d11 39 2024-03-22T06:15:18.994 INFO:tasks.workunit.client.0.smithi006.stdout:0/734: chown db/ca7 74880802 1 2024-03-22T06:15:18.995 INFO:tasks.workunit.client.1.smithi117.stdout:1/161: creat d1/d22/d27/f34 x:0 0 0 2024-03-22T06:15:18.995 INFO:tasks.workunit.client.1.smithi117.stdout:1/162: read - d1/d1f/f31 zero size 2024-03-22T06:15:18.996 INFO:tasks.workunit.client.0.smithi006.stdout:4/974: mkdir d6/d1b/d20/d6b/d146/d157 0 2024-03-22T06:15:18.997 INFO:tasks.workunit.client.0.smithi006.stdout:4/975: stat d6/d1b/d20/d21/d77/l14c 0 2024-03-22T06:15:18.998 INFO:tasks.workunit.client.1.smithi117.stdout:5/138: rename f6 to dd/df/d1e/f2e 0 2024-03-22T06:15:19.000 INFO:tasks.workunit.client.1.smithi117.stdout:3/119: mknod d9/c2b 0 2024-03-22T06:15:19.001 INFO:tasks.workunit.client.1.smithi117.stdout:0/196: creat d1/d3/d2b/d3e/d3f/f45 x:0 0 0 2024-03-22T06:15:19.003 INFO:tasks.workunit.client.1.smithi117.stdout:2/156: rmdir d2/d5/db 39 2024-03-22T06:15:19.006 INFO:tasks.workunit.client.1.smithi117.stdout:7/148: link d8/d9/fd d8/d9/d1a/d26/f2c 0 2024-03-22T06:15:19.008 INFO:tasks.workunit.client.0.smithi006.stdout:5/972: creat d0/db/d16/d1c/d34/d35/f12a x:0 0 0 2024-03-22T06:15:19.008 INFO:tasks.workunit.client.0.smithi006.stdout:5/973: chown d0/db/d16/d43/feb 2866 1 2024-03-22T06:15:19.008 INFO:tasks.workunit.client.0.smithi006.stdout:5/974: chown d0/db/d16/d1c/d34/d35/d85/dda/d65/dd4/fd8 6773 1 2024-03-22T06:15:19.008 INFO:tasks.workunit.client.0.smithi006.stdout:5/975: fsync d0/db/d16/d1c/f40 0 2024-03-22T06:15:19.009 INFO:tasks.workunit.client.1.smithi117.stdout:9/217: link d4/d7/d10/f28 d4/d7/d10/f42 0 2024-03-22T06:15:19.011 INFO:tasks.workunit.client.1.smithi117.stdout:8/179: stat d1/d2/d9/ld 0 2024-03-22T06:15:19.013 INFO:tasks.workunit.client.1.smithi117.stdout:5/139: read dd/df/f2b [2757074,14478] 0 2024-03-22T06:15:19.015 INFO:tasks.workunit.client.0.smithi006.stdout:0/735: creat db/d3d/ffa x:0 0 0 2024-03-22T06:15:19.016 INFO:tasks.workunit.client.0.smithi006.stdout:4/976: rmdir d6/d1b/d20/d21/d2a/d43/d4b/d14f 0 2024-03-22T06:15:19.017 INFO:tasks.workunit.client.0.smithi006.stdout:9/856: rename d8/d10/d20/d45/db9 to d8/db3/d50/d91/dfe/d11a 0 2024-03-22T06:15:19.019 INFO:tasks.workunit.client.0.smithi006.stdout:5/976: mkdir d0/db/d16/d43/ddc/d12b 0 2024-03-22T06:15:19.019 INFO:tasks.workunit.client.0.smithi006.stdout:5/977: write d0/db/d16/f46 [2520760,71530] 0 2024-03-22T06:15:19.020 INFO:tasks.workunit.client.0.smithi006.stdout:0/736: symlink db/dc/d38/lfb 0 2024-03-22T06:15:19.020 INFO:tasks.workunit.client.0.smithi006.stdout:0/737: fsync db/d10/fb5 0 2024-03-22T06:15:19.020 INFO:tasks.workunit.client.0.smithi006.stdout:5/978: dread d0/db/d16/d1c/d34/d35/d5d/d9d/fcd [0,4194304] 0 2024-03-22T06:15:19.020 INFO:tasks.workunit.client.0.smithi006.stdout:5/979: readlink d0/db/d16/d1c/d34/d35/d85/dda/d65/lec 0 2024-03-22T06:15:19.020 INFO:tasks.workunit.client.0.smithi006.stdout:0/738: write db/d10/d35/d7b/da1/da6/ff5 [145779,113081] 0 2024-03-22T06:15:19.020 INFO:tasks.workunit.client.1.smithi117.stdout:0/197: getdents d1/d3 0 2024-03-22T06:15:19.021 INFO:tasks.workunit.client.1.smithi117.stdout:1/163: rename d1/d14/d16 to d1/d30/d35 0 2024-03-22T06:15:19.022 INFO:tasks.workunit.client.1.smithi117.stdout:0/198: creat d1/d3/d2b/d3e/de/d1f/d3b/f46 x:0 0 0 2024-03-22T06:15:19.024 INFO:tasks.workunit.client.0.smithi006.stdout:5/980: link d0/db/d16/d1c/d34/ccc d0/db/d16/d43/ddc/d12b/c12c 0 2024-03-22T06:15:19.024 INFO:tasks.workunit.client.0.smithi006.stdout:5/981: readlink d0/db/d16/d1c/d34/d35/d5d/d9d/l9e 0 2024-03-22T06:15:19.025 INFO:tasks.workunit.client.0.smithi006.stdout:5/982: dread - d0/db/d16/d1c/d34/d35/f12a zero size 2024-03-22T06:15:19.026 INFO:tasks.workunit.client.1.smithi117.stdout:7/149: rename d8/d9/d1a/l1b to d8/d9/d1a/d26/l2d 0 2024-03-22T06:15:19.028 INFO:tasks.workunit.client.1.smithi117.stdout:2/157: dwrite d2/d5/d13/f19 [0,4194304] 0 2024-03-22T06:15:19.028 INFO:tasks.workunit.client.1.smithi117.stdout:1/164: mkdir d1/d30/d35/d36 0 2024-03-22T06:15:19.029 INFO:tasks.workunit.client.1.smithi117.stdout:6/209: dwrite d3/d7/da/de/d2c/f2f [0,4194304] 0 2024-03-22T06:15:19.030 INFO:tasks.workunit.client.1.smithi117.stdout:0/199: getdents d1/d3/d2b/d3e/de/d1f 0 2024-03-22T06:15:19.030 INFO:tasks.workunit.client.0.smithi006.stdout:6/870: dwrite d6/f66 [0,4194304] 0 2024-03-22T06:15:19.030 INFO:tasks.workunit.client.0.smithi006.stdout:9/857: dread d8/ff [0,4194304] 0 2024-03-22T06:15:19.030 INFO:tasks.workunit.client.0.smithi006.stdout:9/858: truncate d8/db3/d50/fe2 477821 0 2024-03-22T06:15:19.030 INFO:tasks.workunit.client.0.smithi006.stdout:9/859: readlink d8/d1f/df5/l108 0 2024-03-22T06:15:19.031 INFO:tasks.workunit.client.0.smithi006.stdout:5/983: truncate d0/db/d16/d43/ddc/fdf 1049416 0 2024-03-22T06:15:19.032 INFO:tasks.workunit.client.0.smithi006.stdout:5/984: readlink d0/db/d16/d1c/d34/d35/d85/dda/d65/lec 0 2024-03-22T06:15:19.032 INFO:tasks.workunit.client.0.smithi006.stdout:4/977: dread d6/d1b/d20/d21/d2a/d43/d4b/f103 [0,4194304] 0 2024-03-22T06:15:19.032 INFO:tasks.workunit.client.0.smithi006.stdout:3/986: dwrite d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d14c/d98/f10b [0,4194304] 0 2024-03-22T06:15:19.033 INFO:tasks.workunit.client.1.smithi117.stdout:3/120: dwrite d9/d1a/f1f [0,4194304] 0 2024-03-22T06:15:19.033 INFO:tasks.workunit.client.1.smithi117.stdout:3/121: dread - d9/f22 zero size 2024-03-22T06:15:19.035 INFO:tasks.workunit.client.1.smithi117.stdout:8/180: rename d1/lf to d1/d2/d16/l3d 0 2024-03-22T06:15:19.037 INFO:tasks.workunit.client.1.smithi117.stdout:3/122: write f6 [3051943,56036] 0 2024-03-22T06:15:19.037 INFO:tasks.workunit.client.0.smithi006.stdout:6/871: link d6/c9 d6/d6f/c10b 0 2024-03-22T06:15:19.040 INFO:tasks.workunit.client.1.smithi117.stdout:2/158: unlink d2/d5/d8/d10/l23 0 2024-03-22T06:15:19.041 INFO:tasks.workunit.client.1.smithi117.stdout:3/123: dread d9/d1a/f1e [0,4194304] 0 2024-03-22T06:15:19.043 INFO:tasks.workunit.client.0.smithi006.stdout:5/985: creat d0/db/d16/d1c/dbc/f12d x:0 0 0 2024-03-22T06:15:19.046 INFO:tasks.workunit.client.1.smithi117.stdout:1/165: unlink d1/d30/d35/f2a 0 2024-03-22T06:15:19.046 INFO:tasks.workunit.client.1.smithi117.stdout:4/194: sync 2024-03-22T06:15:19.047 INFO:tasks.workunit.client.1.smithi117.stdout:4/195: write d0/d19/d28/f2e [494401,113210] 0 2024-03-22T06:15:19.049 INFO:tasks.workunit.client.0.smithi006.stdout:0/739: dwrite db/f80 [0,4194304] 0 2024-03-22T06:15:19.051 INFO:tasks.workunit.client.1.smithi117.stdout:6/210: rmdir d3/d40/d24/d2d/d30 39 2024-03-22T06:15:19.055 INFO:tasks.workunit.client.1.smithi117.stdout:5/140: fsync dd/df/d1e/f2e 0 2024-03-22T06:15:19.055 INFO:tasks.workunit.client.1.smithi117.stdout:5/141: write dd/df/f2c [302062,48378] 0 2024-03-22T06:15:19.055 INFO:tasks.workunit.client.1.smithi117.stdout:3/124: write d9/f18 [765260,107126] 0 2024-03-22T06:15:19.055 INFO:tasks.workunit.client.1.smithi117.stdout:3/125: stat d9/f26 0 2024-03-22T06:15:19.055 INFO:tasks.workunit.client.1.smithi117.stdout:3/126: read d9/f18 [320177,45464] 0 2024-03-22T06:15:19.055 INFO:tasks.workunit.client.1.smithi117.stdout:3/127: fdatasync d9/f22 0 2024-03-22T06:15:19.055 INFO:tasks.workunit.client.1.smithi117.stdout:0/200: rmdir d1/d10 39 2024-03-22T06:15:19.055 INFO:tasks.workunit.client.1.smithi117.stdout:0/201: creat d1/d10/d18/d26/dc/f47 x:0 0 0 2024-03-22T06:15:19.056 INFO:tasks.workunit.client.0.smithi006.stdout:4/978: creat d6/d1b/d20/d6b/f158 x:0 0 0 2024-03-22T06:15:19.061 INFO:tasks.workunit.client.0.smithi006.stdout:3/987: creat d0/d4/dbb/df7/d5f/d83/d155/f159 x:0 0 0 2024-03-22T06:15:19.063 INFO:tasks.workunit.client.0.smithi006.stdout:6/872: truncate d6/d1b/f2a 3541592 0 2024-03-22T06:15:19.063 INFO:tasks.workunit.client.0.smithi006.stdout:6/873: write d6/d24/d44/f62 [585122,71996] 0 2024-03-22T06:15:19.066 INFO:tasks.workunit.client.1.smithi117.stdout:8/181: dwrite d1/d2/d9/f13 [4194304,4194304] 0 2024-03-22T06:15:19.066 INFO:tasks.workunit.client.0.smithi006.stdout:5/986: dwrite d0/db/d50/d58/dad/fcf [0,4194304] 0 2024-03-22T06:15:19.066 INFO:tasks.workunit.client.0.smithi006.stdout:5/987: creat d0/db/d16/d1c/d34/d35/d85/d5e/d66/f12e x:0 0 0 2024-03-22T06:15:19.067 INFO:tasks.workunit.client.1.smithi117.stdout:7/150: rename d8/d9/da/l17 to d8/d9/d1a/d26/l2e 0 2024-03-22T06:15:19.067 INFO:tasks.workunit.client.1.smithi117.stdout:7/151: getdents d8/d9/da/d10/d23/d27 0 2024-03-22T06:15:19.067 INFO:tasks.workunit.client.1.smithi117.stdout:7/152: creat d8/f2f x:0 0 0 2024-03-22T06:15:19.068 INFO:tasks.workunit.client.0.smithi006.stdout:9/860: dwrite d8/d10/d28/d39/f93 [0,4194304] 0 2024-03-22T06:15:19.068 INFO:tasks.workunit.client.0.smithi006.stdout:0/740: dread db/d3d/d4e/f7d [0,4194304] 0 2024-03-22T06:15:19.068 INFO:tasks.workunit.client.0.smithi006.stdout:9/861: dread d8/d10/d23/f40 [4194304,4194304] 0 2024-03-22T06:15:19.068 INFO:tasks.workunit.client.0.smithi006.stdout:9/862: creat d8/d10/d28/d39/f11b x:0 0 0 2024-03-22T06:15:19.070 INFO:tasks.workunit.client.1.smithi117.stdout:7/153: write d8/d9/fd [7424428,3916] 0 2024-03-22T06:15:19.072 INFO:tasks.workunit.client.1.smithi117.stdout:2/159: mknod d2/d5/d8/d10/d11/c2a 0 2024-03-22T06:15:19.072 INFO:tasks.workunit.client.1.smithi117.stdout:2/160: truncate d2/d5/d8/f22 627443 0 2024-03-22T06:15:19.073 INFO:tasks.workunit.client.0.smithi006.stdout:4/979: mknod d6/d1b/d20/d21/d2a/c159 0 2024-03-22T06:15:19.075 INFO:tasks.workunit.client.1.smithi117.stdout:1/166: mknod d1/d30/d35/d36/c37 0 2024-03-22T06:15:19.075 INFO:tasks.workunit.client.1.smithi117.stdout:1/167: dread - d1/d22/d27/f34 zero size 2024-03-22T06:15:19.076 INFO:tasks.workunit.client.0.smithi006.stdout:6/874: chown d6/d1b/f21 4 1 2024-03-22T06:15:19.077 INFO:tasks.workunit.client.0.smithi006.stdout:3/988: dwrite d0/d4/dbb/df7/d5f/d90/da7/d97/fb7 [0,4194304] 0 2024-03-22T06:15:19.077 INFO:tasks.workunit.client.0.smithi006.stdout:3/989: write d0/d4/d3a/fde [418162,36916] 0 2024-03-22T06:15:19.077 INFO:tasks.workunit.client.0.smithi006.stdout:3/990: truncate d0/d4/dbb/df7/d5f/da3/f128 953780 0 2024-03-22T06:15:19.078 INFO:tasks.workunit.client.1.smithi117.stdout:4/196: mkdir d0/d19/d34/d43 0 2024-03-22T06:15:19.082 INFO:tasks.workunit.client.1.smithi117.stdout:6/211: rename d3/d7/da/de/f23 to d3/d7/da/de/f47 0 2024-03-22T06:15:19.082 INFO:tasks.workunit.client.1.smithi117.stdout:6/212: chown d3/d7/da/de/d2c/f2f 783393169 1 2024-03-22T06:15:19.082 INFO:tasks.workunit.client.1.smithi117.stdout:6/213: chown d3/d40/d24/d2d/d30/c39 732768841 1 2024-03-22T06:15:19.087 INFO:tasks.workunit.client.1.smithi117.stdout:3/128: symlink d9/d1a/l2c 0 2024-03-22T06:15:19.089 INFO:tasks.workunit.client.0.smithi006.stdout:5/988: mkdir d0/db/d76/db4/dfe/d10b/d12f 0 2024-03-22T06:15:19.089 INFO:tasks.workunit.client.0.smithi006.stdout:5/989: creat d0/db/d16/d70/f130 x:0 0 0 2024-03-22T06:15:19.093 INFO:tasks.workunit.client.1.smithi117.stdout:5/142: rename f4 to dd/df/d10/f2f 0 2024-03-22T06:15:19.094 INFO:tasks.workunit.client.0.smithi006.stdout:0/741: mkdir db/d10/d35/d7b/da1/da6/dc4/dfc 0 2024-03-22T06:15:19.097 INFO:tasks.workunit.client.1.smithi117.stdout:7/154: write d8/d9/da/d10/f19 [1046015,123829] 0 2024-03-22T06:15:19.098 INFO:tasks.workunit.client.0.smithi006.stdout:9/863: mknod d8/d10/d23/c11c 0 2024-03-22T06:15:19.099 INFO:tasks.workunit.client.1.smithi117.stdout:2/161: link d2/d1b/l25 d2/d5/d20/l2b 0 2024-03-22T06:15:19.100 INFO:tasks.workunit.client.0.smithi006.stdout:4/980: rename d6/d13/fe7 to d6/d1b/d20/d63/d93/d9a/f15a 0 2024-03-22T06:15:19.100 INFO:tasks.workunit.client.0.smithi006.stdout:4/981: creat d6/d1b/d20/d63/d93/d9a/d130/f15b x:0 0 0 2024-03-22T06:15:19.101 INFO:tasks.workunit.client.0.smithi006.stdout:6/875: creat d6/d1d/f10c x:0 0 0 2024-03-22T06:15:19.107 INFO:tasks.workunit.client.1.smithi117.stdout:6/214: truncate d3/f3f 4155001 0 2024-03-22T06:15:19.107 INFO:tasks.workunit.client.1.smithi117.stdout:5/143: dwrite dd/df/d10/f25 [0,4194304] 0 2024-03-22T06:15:19.109 INFO:tasks.workunit.client.1.smithi117.stdout:3/129: creat d9/d1a/f2d x:0 0 0 2024-03-22T06:15:19.111 INFO:tasks.workunit.client.1.smithi117.stdout:0/202: rmdir d1/d3/d2b/d3e/d3f 39 2024-03-22T06:15:19.111 INFO:tasks.workunit.client.1.smithi117.stdout:0/203: write d1/d10/d18/d26/dc/f25 [1134589,106142] 0 2024-03-22T06:15:19.112 INFO:tasks.workunit.client.0.smithi006.stdout:3/991: mkdir d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d12c/d13b/d15a 0 2024-03-22T06:15:19.114 INFO:tasks.workunit.client.1.smithi117.stdout:8/182: rename d1/d2/d9/lc to d1/d19/d38/l3e 0 2024-03-22T06:15:19.114 INFO:tasks.workunit.client.1.smithi117.stdout:8/183: write d1/d2/d9/f13 [12782904,42785] 0 2024-03-22T06:15:19.114 INFO:tasks.workunit.client.1.smithi117.stdout:8/184: creat d1/d19/f3f x:0 0 0 2024-03-22T06:15:19.114 INFO:tasks.workunit.client.1.smithi117.stdout:8/185: truncate d1/d19/f3f 551167 0 2024-03-22T06:15:19.116 INFO:tasks.workunit.client.1.smithi117.stdout:7/155: mknod d8/d9/da/d10/d23/c30 0 2024-03-22T06:15:19.120 INFO:tasks.workunit.client.1.smithi117.stdout:2/162: mkdir d2/d5/db/d2c 0 2024-03-22T06:15:19.121 INFO:tasks.workunit.client.0.smithi006.stdout:3/992: write d0/d4/dbb/df7/d5f/d90/da7/de7/d33/f44 [3138206,30350] 0 2024-03-22T06:15:19.126 INFO:tasks.workunit.client.0.smithi006.stdout:6/876: dwrite d6/d24/f58 [0,4194304] 0 2024-03-22T06:15:19.128 INFO:tasks.workunit.client.0.smithi006.stdout:5/990: unlink d0/db/dc/l42 0 2024-03-22T06:15:19.128 INFO:tasks.workunit.client.0.smithi006.stdout:5/991: write d0/db/d50/d82/d103/fef [177669,95465] 0 2024-03-22T06:15:19.130 INFO:tasks.workunit.client.0.smithi006.stdout:0/742: creat db/d10/d2a/d41/ffd x:0 0 0 2024-03-22T06:15:19.131 INFO:tasks.workunit.client.1.smithi117.stdout:1/168: rename d1/d1f/l2b to d1/d14/l38 0 2024-03-22T06:15:19.131 INFO:tasks.workunit.client.1.smithi117.stdout:1/169: readlink d1/l4 0 2024-03-22T06:15:19.131 INFO:tasks.workunit.client.1.smithi117.stdout:1/170: readlink d1/d14/l1c 0 2024-03-22T06:15:19.132 INFO:tasks.workunit.client.0.smithi006.stdout:9/864: mkdir d8/d10/d20/d45/d4d/d11d 0 2024-03-22T06:15:19.137 INFO:tasks.workunit.client.0.smithi006.stdout:4/982: mknod d6/d1b/d20/d63/d93/d9a/d130/c15c 0 2024-03-22T06:15:19.138 INFO:tasks.workunit.client.1.smithi117.stdout:2/163: symlink d2/d5/d8/d10/d11/l2d 0 2024-03-22T06:15:19.138 INFO:tasks.workunit.client.1.smithi117.stdout:2/164: write d2/d5/d20/d1d/f27 [238841,5620] 0 2024-03-22T06:15:19.139 INFO:tasks.workunit.client.1.smithi117.stdout:3/130: dwrite d9/f19 [0,4194304] 0 2024-03-22T06:15:19.139 INFO:tasks.workunit.client.1.smithi117.stdout:3/131: fdatasync f7 0 2024-03-22T06:15:19.140 INFO:tasks.workunit.client.0.smithi006.stdout:3/993: dwrite d0/d8/d6a/f11f [0,4194304] 0 2024-03-22T06:15:19.140 INFO:tasks.workunit.client.0.smithi006.stdout:3/994: read d0/d4/dbb/df7/d68/f6b [647472,104265] 0 2024-03-22T06:15:19.140 INFO:tasks.workunit.client.1.smithi117.stdout:5/144: rmdir dd/df/d10 39 2024-03-22T06:15:19.144 INFO:tasks.workunit.client.1.smithi117.stdout:0/204: rename d1/c9 to d1/d10/d18/d26/dc/c48 0 2024-03-22T06:15:19.146 INFO:tasks.workunit.client.0.smithi006.stdout:1/863: sync 2024-03-22T06:15:19.147 INFO:tasks.workunit.client.1.smithi117.stdout:2/165: truncate f1 8870509 0 2024-03-22T06:15:19.149 INFO:tasks.workunit.client.0.smithi006.stdout:5/992: truncate d0/db/d16/d1c/d34/d35/d85/dda/d65/fb2 462163 0 2024-03-22T06:15:19.149 INFO:tasks.workunit.client.0.smithi006.stdout:5/993: fdatasync d0/db/d16/d1c/d34/d35/f12a 0 2024-03-22T06:15:19.153 INFO:tasks.workunit.client.0.smithi006.stdout:7/882: sync 2024-03-22T06:15:19.155 INFO:tasks.workunit.client.0.smithi006.stdout:9/865: creat d8/d10/d20/d45/d4d/d11d/f11e x:0 0 0 2024-03-22T06:15:19.155 INFO:tasks.workunit.client.0.smithi006.stdout:9/866: chown d8/d10/d28/d39 5819495 1 2024-03-22T06:15:19.157 INFO:tasks.workunit.client.1.smithi117.stdout:7/156: dwrite d8/d9/da/d10/f25 [0,4194304] 0 2024-03-22T06:15:19.157 INFO:tasks.workunit.client.1.smithi117.stdout:7/157: write d8/d9/f1e [411989,127235] 0 2024-03-22T06:15:19.157 INFO:tasks.workunit.client.1.smithi117.stdout:7/158: dread - d8/d9/da/f28 zero size 2024-03-22T06:15:19.157 INFO:tasks.workunit.client.1.smithi117.stdout:7/159: fsync d8/f1f 0 2024-03-22T06:15:19.158 INFO:tasks.workunit.client.0.smithi006.stdout:4/983: symlink d6/d1b/d20/d21/d11d/l15d 0 2024-03-22T06:15:19.159 INFO:tasks.workunit.client.1.smithi117.stdout:6/215: dread d3/f3f [0,4194304] 0 2024-03-22T06:15:19.159 INFO:tasks.workunit.client.1.smithi117.stdout:3/132: mkdir d9/d2e 0 2024-03-22T06:15:19.161 INFO:tasks.workunit.client.1.smithi117.stdout:5/145: mkdir dd/df/d30 0 2024-03-22T06:15:19.161 INFO:tasks.workunit.client.1.smithi117.stdout:5/146: write dd/df/f1b [462577,58674] 0 2024-03-22T06:15:19.161 INFO:tasks.workunit.client.1.smithi117.stdout:5/147: truncate dd/f29 160888 0 2024-03-22T06:15:19.165 INFO:tasks.workunit.client.0.smithi006.stdout:3/995: mknod d0/d8/d88/dfa/c15b 0 2024-03-22T06:15:19.168 INFO:tasks.workunit.client.1.smithi117.stdout:7/160: write d8/d9/da/d10/f25 [1080138,5510] 0 2024-03-22T06:15:19.171 INFO:tasks.workunit.client.1.smithi117.stdout:6/216: dread d3/d7/fd [0,4194304] 0 2024-03-22T06:15:19.171 INFO:tasks.workunit.client.0.smithi006.stdout:6/877: rmdir d6/d1b/dae 39 2024-03-22T06:15:19.174 INFO:tasks.workunit.client.1.smithi117.stdout:0/205: dwrite d1/d3/d2b/d3e/d3f/f45 [0,4194304] 0 2024-03-22T06:15:19.175 INFO:tasks.workunit.client.0.smithi006.stdout:5/994: dwrite d0/db/d16/d1c/d34/d35/d85/dda/d65/d77/fba [0,4194304] 0 2024-03-22T06:15:19.176 INFO:tasks.workunit.client.0.smithi006.stdout:0/743: truncate db/f45 928767 0 2024-03-22T06:15:19.178 INFO:tasks.workunit.client.0.smithi006.stdout:0/744: dread db/dc/d38/fc1 [4194304,4194304] 0 2024-03-22T06:15:19.180 INFO:tasks.workunit.client.1.smithi117.stdout:8/186: rename d1/d19/d38/l3e to d1/d2/d9/l40 0 2024-03-22T06:15:19.182 INFO:tasks.workunit.client.0.smithi006.stdout:7/883: unlink d5/f56 0 2024-03-22T06:15:19.185 INFO:tasks.workunit.client.0.smithi006.stdout:9/867: rename d8/d10/d47/f87 to d8/d10/d20/d103/f11f 0 2024-03-22T06:15:19.186 INFO:tasks.workunit.client.0.smithi006.stdout:4/984: mknod d6/d1b/d20/c15e 0 2024-03-22T06:15:19.188 INFO:tasks.workunit.client.1.smithi117.stdout:5/148: creat dd/df/d10/f31 x:0 0 0 2024-03-22T06:15:19.189 INFO:tasks.workunit.client.0.smithi006.stdout:5/995: dread d0/db/d50/d82/d103/fe2 [0,4194304] 0 2024-03-22T06:15:19.191 INFO:tasks.workunit.client.1.smithi117.stdout:6/217: mknod d3/d40/c48 0 2024-03-22T06:15:19.191 INFO:tasks.workunit.client.1.smithi117.stdout:6/218: readlink d3/l9 0 2024-03-22T06:15:19.192 INFO:tasks.workunit.client.1.smithi117.stdout:0/206: mkdir d1/d10/d18/d26/d49 0 2024-03-22T06:15:19.193 INFO:tasks.workunit.client.1.smithi117.stdout:0/207: fsync d1/d10/f27 0 2024-03-22T06:15:19.193 INFO:tasks.workunit.client.1.smithi117.stdout:8/187: symlink d1/l41 0 2024-03-22T06:15:19.194 INFO:tasks.workunit.client.0.smithi006.stdout:0/745: rmdir db/d87/ddd/df8 0 2024-03-22T06:15:19.195 INFO:tasks.workunit.client.1.smithi117.stdout:3/133: unlink c0 0 2024-03-22T06:15:19.196 INFO:tasks.workunit.client.0.smithi006.stdout:7/884: mkdir d5/d1e/d29/d66/d91/ddc/dfa/d121 0 2024-03-22T06:15:19.196 INFO:tasks.workunit.client.0.smithi006.stdout:7/885: fdatasync d5/d1e/d63/f110 0 2024-03-22T06:15:19.197 INFO:tasks.workunit.client.1.smithi117.stdout:5/149: mkdir dd/d16/d1a/d32 0 2024-03-22T06:15:19.197 INFO:tasks.workunit.client.1.smithi117.stdout:5/150: chown dd/d16/f22 34666 1 2024-03-22T06:15:19.198 INFO:tasks.workunit.client.1.smithi117.stdout:0/208: mknod d1/d3/d2b/d3e/de/c4a 0 2024-03-22T06:15:19.198 INFO:tasks.workunit.client.1.smithi117.stdout:0/209: read - d1/f1c zero size 2024-03-22T06:15:19.199 INFO:tasks.workunit.client.1.smithi117.stdout:0/210: getdents d1/d10 0 2024-03-22T06:15:19.200 INFO:tasks.workunit.client.0.smithi006.stdout:9/868: mkdir d8/d10/d20/d103/d120 0 2024-03-22T06:15:19.201 INFO:tasks.workunit.client.1.smithi117.stdout:5/151: dread dd/d16/f22 [0,4194304] 0 2024-03-22T06:15:19.201 INFO:tasks.workunit.client.1.smithi117.stdout:5/152: write f2 [4789436,130244] 0 2024-03-22T06:15:19.201 INFO:tasks.workunit.client.1.smithi117.stdout:2/166: dwrite d2/d5/d8/d10/d11/f15 [4194304,4194304] 0 2024-03-22T06:15:19.202 INFO:tasks.workunit.client.1.smithi117.stdout:2/167: stat d2/d5/db/lf 0 2024-03-22T06:15:19.202 INFO:tasks.workunit.client.0.smithi006.stdout:3/996: dwrite d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d14c/f129 [0,4194304] 0 2024-03-22T06:15:19.202 INFO:tasks.workunit.client.1.smithi117.stdout:6/219: write d3/f20 [666886,84224] 0 2024-03-22T06:15:19.203 INFO:tasks.workunit.client.0.smithi006.stdout:4/985: link d6/d1b/d20/la3 d6/d1b/d20/d6b/d60/db1/l15f 0 2024-03-22T06:15:19.203 INFO:tasks.workunit.client.1.smithi117.stdout:7/161: dwrite d8/d9/da/d10/d23/f24 [0,4194304] 0 2024-03-22T06:15:19.204 INFO:tasks.workunit.client.1.smithi117.stdout:7/162: readlink d8/d9/l18 0 2024-03-22T06:15:19.205 INFO:tasks.workunit.client.1.smithi117.stdout:0/211: mkdir d1/d4b 0 2024-03-22T06:15:19.205 INFO:tasks.workunit.client.1.smithi117.stdout:0/212: write d1/d10/f32 [61608,13961] 0 2024-03-22T06:15:19.205 INFO:tasks.workunit.client.0.smithi006.stdout:5/996: truncate d0/db/d50/d82/d103/fa3 3143509 0 2024-03-22T06:15:19.207 INFO:tasks.workunit.client.1.smithi117.stdout:8/188: getdents d1/d28 0 2024-03-22T06:15:19.210 INFO:tasks.workunit.client.1.smithi117.stdout:5/153: mkdir dd/df/d30/d33 0 2024-03-22T06:15:19.211 INFO:tasks.workunit.client.0.smithi006.stdout:4/986: write d6/f141 [619206,130440] 0 2024-03-22T06:15:19.211 INFO:tasks.workunit.client.0.smithi006.stdout:4/987: dread - d6/d1b/fe9 zero size 2024-03-22T06:15:19.211 INFO:tasks.workunit.client.0.smithi006.stdout:4/988: write d6/d1b/d20/d6b/d95/fbb [690034,130154] 0 2024-03-22T06:15:19.211 INFO:tasks.workunit.client.0.smithi006.stdout:4/989: chown d6/d1b/d46/l6d 342856057 1 2024-03-22T06:15:19.211 INFO:tasks.workunit.client.0.smithi006.stdout:4/990: truncate d6/fbd 1377527 0 2024-03-22T06:15:19.211 INFO:tasks.workunit.client.0.smithi006.stdout:6/878: rmdir d6/d1d/d32 39 2024-03-22T06:15:19.213 INFO:tasks.workunit.client.0.smithi006.stdout:0/746: mkdir db/dc/d37/dfe 0 2024-03-22T06:15:19.221 INFO:tasks.workunit.client.1.smithi117.stdout:6/220: mkdir d3/d40/d24/d2d/d30/d3e/d49 0 2024-03-22T06:15:19.225 INFO:tasks.workunit.client.0.smithi006.stdout:3/997: mknod d0/d4/dbb/df7/d5f/d90/da7/de7/d33/d14c/c15c 0 2024-03-22T06:15:19.225 INFO:tasks.workunit.client.0.smithi006.stdout:3/998: read - d0/d8/d1c/d72/f144 zero size 2024-03-22T06:15:19.227 INFO:tasks.workunit.client.0.smithi006.stdout:9/869: dread d8/fb [4194304,4194304] 0 2024-03-22T06:15:19.231 INFO:tasks.workunit.client.1.smithi117.stdout:0/213: unlink d1/d3/d2b/f2f 0 2024-03-22T06:15:19.233 INFO:tasks.workunit.client.1.smithi117.stdout:8/189: creat d1/d19/d38/f42 x:0 0 0 2024-03-22T06:15:19.236 INFO:tasks.workunit.client.0.smithi006.stdout:5/997: symlink d0/db/d99/l131 0 2024-03-22T06:15:19.237 INFO:tasks.workunit.client.1.smithi117.stdout:5/154: truncate dd/d16/d24/f23 3444265 0 2024-03-22T06:15:19.238 INFO:tasks.workunit.client.0.smithi006.stdout:4/991: mkdir d6/d1b/d20/d6b/d95/d160 0 2024-03-22T06:15:19.240 INFO:tasks.workunit.client.0.smithi006.stdout:6/879: creat d6/d1d/d5c/d81/f10d x:0 0 0 2024-03-22T06:15:19.240 INFO:tasks.workunit.client.0.smithi006.stdout:6/880: dread - d6/d24/d44/fcb zero size 2024-03-22T06:15:19.242 INFO:tasks.workunit.client.0.smithi006.stdout:0/747: mkdir db/d10/d2a/d41/dff 0 2024-03-22T06:15:19.242 INFO:tasks.workunit.client.0.smithi006.stdout:0/748: creat db/dc/d38/d50/f100 x:0 0 0 2024-03-22T06:15:19.242 INFO:tasks.workunit.client.0.smithi006.stdout:0/749: readlink db/d10/d35/d7b/lf4 0 2024-03-22T06:15:19.243 INFO:tasks.workunit.client.1.smithi117.stdout:7/163: dwrite f7 [0,4194304] 0 2024-03-22T06:15:19.244 INFO:tasks.workunit.client.1.smithi117.stdout:2/168: dwrite d2/d5/d20/f1f [0,4194304] 0 2024-03-22T06:15:19.244 INFO:tasks.workunit.client.1.smithi117.stdout:6/221: dwrite d3/d7/da/d32/f38 [0,4194304] 0 2024-03-22T06:15:19.244 INFO:tasks.workunit.client.1.smithi117.stdout:6/222: truncate d3/d7/da/f29 684340 0 2024-03-22T06:15:19.244 INFO:tasks.workunit.client.1.smithi117.stdout:6/223: chown d3/d3d/f2e 7 1 2024-03-22T06:15:19.244 INFO:tasks.workunit.client.1.smithi117.stdout:6/224: creat d3/d7/da/d32/f4a x:0 0 0 2024-03-22T06:15:19.244 INFO:tasks.workunit.client.1.smithi117.stdout:6/225: fdatasync d3/f20 0 2024-03-22T06:15:19.244 INFO:tasks.workunit.client.1.smithi117.stdout:6/226: fdatasync d3/d7/da/d32/f38 0 2024-03-22T06:15:19.245 INFO:tasks.workunit.client.0.smithi006.stdout:1/864: dwrite d8/d71/fed [0,4194304] 0 2024-03-22T06:15:19.245 INFO:tasks.workunit.client.0.smithi006.stdout:3/999: symlink d0/d8/d88/ddf/d102/l15d 0 2024-03-22T06:15:19.247 INFO:tasks.workunit.client.1.smithi117.stdout:5/155: dread dd/df/f2b [0,4194304] 0 2024-03-22T06:15:19.247 INFO:tasks.workunit.client.1.smithi117.stdout:0/214: unlink d1/d3/d2b/d3e/l13 0 2024-03-22T06:15:19.247 INFO:tasks.workunit.client.1.smithi117.stdout:0/215: write d1/d10/d18/f38 [782831,73905] 0 2024-03-22T06:15:19.247 INFO:tasks.workunit.client.1.smithi117.stdout:0/216: chown d1/d3/d2b/d3e/de/d1f/f33 4096439 1 2024-03-22T06:15:19.247 INFO:tasks.workunit.client.0.smithi006.stdout:9/870: mknod d8/d10/d11/c121 0 2024-03-22T06:15:19.248 INFO:tasks.workunit.client.0.smithi006.stdout:9/871: read d8/d10/d28/f3d [2465275,54406] 0 2024-03-22T06:15:19.249 INFO:tasks.workunit.client.1.smithi117.stdout:6/227: dread d3/d7/f1b [0,4194304] 0 2024-03-22T06:15:19.249 INFO:tasks.workunit.client.1.smithi117.stdout:8/190: truncate d1/d2/fa 106248 0 2024-03-22T06:15:19.249 INFO:tasks.workunit.client.1.smithi117.stdout:8/191: write d1/d19/f2d [156489,16214] 0 2024-03-22T06:15:19.249 INFO:tasks.workunit.client.1.smithi117.stdout:8/192: readlink d1/d19/l32 0 2024-03-22T06:15:19.250 INFO:tasks.workunit.client.1.smithi117.stdout:7/164: fsync d8/d9/da/d10/f22 0 2024-03-22T06:15:19.250 INFO:tasks.workunit.client.1.smithi117.stdout:7/165: chown d8/f16 12627 1 2024-03-22T06:15:19.250 INFO:tasks.workunit.client.1.smithi117.stdout:7/166: fsync f5 0 2024-03-22T06:15:19.251 INFO:tasks.workunit.client.0.smithi006.stdout:0/750: write db/d10/d33/d9d/fa2 [2267208,10310] 0 2024-03-22T06:15:19.252 INFO:tasks.workunit.client.1.smithi117.stdout:9/218: sync 2024-03-22T06:15:19.253 INFO:tasks.workunit.client.0.smithi006.stdout:4/992: link d6/d1b/d20/d6b/fae d6/d13/d3b/f161 0 2024-03-22T06:15:19.253 INFO:tasks.workunit.client.0.smithi006.stdout:4/993: creat d6/d1b/d20/d21/d77/df1/f162 x:0 0 0 2024-03-22T06:15:19.255 INFO:tasks.workunit.client.0.smithi006.stdout:6/881: getdents d6/d1d/d99/de9 0 2024-03-22T06:15:19.260 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:19 smithi006 ceph-mon[29301]: pgmap v23: 65 pgs: 65 active+clean; 1.8 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 35 MiB/s rd, 46 MiB/s wr, 802 op/s 2024-03-22T06:15:19.262 INFO:tasks.workunit.client.1.smithi117.stdout:2/169: symlink d2/d1b/l2e 0 2024-03-22T06:15:19.262 INFO:tasks.workunit.client.1.smithi117.stdout:2/170: stat d2/d5/d8/d10/d11/c2a 0 2024-03-22T06:15:19.262 INFO:tasks.workunit.client.1.smithi117.stdout:2/171: chown d2/d5/d8/d10/d11/l2d 54322994 1 2024-03-22T06:15:19.262 INFO:tasks.workunit.client.1.smithi117.stdout:2/172: write d2/f26 [864367,15176] 0 2024-03-22T06:15:19.265 INFO:tasks.workunit.client.0.smithi006.stdout:1/865: mknod d8/d71/dcd/c123 0 2024-03-22T06:15:19.268 INFO:tasks.workunit.client.0.smithi006.stdout:9/872: symlink d8/l122 0 2024-03-22T06:15:19.270 INFO:tasks.workunit.client.1.smithi117.stdout:5/156: rename f7 to dd/d16/d24/f34 0 2024-03-22T06:15:19.271 INFO:tasks.workunit.client.0.smithi006.stdout:0/751: dwrite db/d10/d2a/f5e [0,4194304] 0 2024-03-22T06:15:19.271 INFO:tasks.workunit.client.0.smithi006.stdout:0/752: chown db/d10/d35/d7b/ld8 33 1 2024-03-22T06:15:19.272 INFO:tasks.workunit.client.0.smithi006.stdout:5/998: getdents d0/db/d16/d1c/d34/d35/d85/d5e/d66 0 2024-03-22T06:15:19.273 INFO:tasks.workunit.client.1.smithi117.stdout:0/217: mknod d1/d10/d18/d26/c4c 0 2024-03-22T06:15:19.277 INFO:tasks.workunit.client.1.smithi117.stdout:8/193: unlink d1/d2/d11/l3b 0 2024-03-22T06:15:19.277 INFO:tasks.workunit.client.1.smithi117.stdout:8/194: truncate d1/d19/f2d 726161 0 2024-03-22T06:15:19.279 INFO:tasks.workunit.client.1.smithi117.stdout:9/219: symlink d4/d7/d10/d3b/l43 0 2024-03-22T06:15:19.284 INFO:tasks.workunit.client.1.smithi117.stdout:6/228: dwrite d3/f20 [0,4194304] 0 2024-03-22T06:15:19.284 INFO:tasks.workunit.client.0.smithi006.stdout:6/882: dwrite d6/d1b/dae/dbc/ffd [0,4194304] 0 2024-03-22T06:15:19.284 INFO:tasks.workunit.client.0.smithi006.stdout:6/883: stat d6/d1d/d5c/d81/d8a/dac/cdb 0 2024-03-22T06:15:19.284 INFO:tasks.workunit.client.0.smithi006.stdout:6/884: write d6/d24/d44/fcb [628389,29157] 0 2024-03-22T06:15:19.285 INFO:tasks.workunit.client.0.smithi006.stdout:4/994: rename d6/d1b/d20/d21/f12b to d6/d1b/d20/d63/d152/f163 0 2024-03-22T06:15:19.287 INFO:tasks.workunit.client.1.smithi117.stdout:9/220: dread d4/d7/d10/f28 [0,4194304] 0 2024-03-22T06:15:19.287 INFO:tasks.workunit.client.0.smithi006.stdout:9/873: read d8/d10/d20/d103/f11f [2926280,12923] 0 2024-03-22T06:15:19.290 INFO:tasks.workunit.client.0.smithi006.stdout:9/874: dread d8/d10/d11/f3c [0,4194304] 0 2024-03-22T06:15:19.295 INFO:tasks.workunit.client.1.smithi117.stdout:2/173: mknod d2/d5/d20/c2f 0 2024-03-22T06:15:19.295 INFO:tasks.workunit.client.1.smithi117.stdout:2/174: creat d2/d5/d13/f30 x:0 0 0 2024-03-22T06:15:19.295 INFO:tasks.workunit.client.1.smithi117.stdout:4/197: sync 2024-03-22T06:15:19.295 INFO:tasks.workunit.client.1.smithi117.stdout:1/171: sync 2024-03-22T06:15:19.295 INFO:tasks.workunit.client.1.smithi117.stdout:1/172: write d1/d30/d35/f1a [1421406,71866] 0 2024-03-22T06:15:19.296 INFO:tasks.workunit.client.0.smithi006.stdout:0/753: creat db/d87/ddd/f101 x:0 0 0 2024-03-22T06:15:19.297 INFO:tasks.workunit.client.1.smithi117.stdout:2/175: write d2/d1b/f28 [1588253,22131] 0 2024-03-22T06:15:19.299 INFO:tasks.workunit.client.1.smithi117.stdout:0/218: dwrite d1/d3/d2b/d3e/de/d1f/d3b/f42 [0,4194304] 0 2024-03-22T06:15:19.300 INFO:tasks.workunit.client.0.smithi006.stdout:5/999: symlink d0/db/dc/l132 0 2024-03-22T06:15:19.302 INFO:tasks.workunit.client.1.smithi117.stdout:9/221: truncate d4/d8/f13 669302 0 2024-03-22T06:15:19.304 INFO:tasks.workunit.client.1.smithi117.stdout:1/173: dread d1/f13 [0,4194304] 0 2024-03-22T06:15:19.304 INFO:tasks.workunit.client.1.smithi117.stdout:8/195: dwrite d1/d2/d9/f18 [4194304,4194304] 0 2024-03-22T06:15:19.304 INFO:tasks.workunit.client.1.smithi117.stdout:1/174: chown d1/f25 89991 1 2024-03-22T06:15:19.307 INFO:tasks.workunit.client.1.smithi117.stdout:4/198: mkdir d0/d19/d34/d44 0 2024-03-22T06:15:19.310 INFO:tasks.workunit.client.1.smithi117.stdout:9/222: dread d4/d7/f19 [0,4194304] 0 2024-03-22T06:15:19.310 INFO:tasks.workunit.client.1.smithi117.stdout:9/223: write d4/d8/f1a [8316914,9043] 0 2024-03-22T06:15:19.311 INFO:tasks.workunit.client.1.smithi117.stdout:4/199: dread d0/f27 [0,4194304] 0 2024-03-22T06:15:19.311 INFO:tasks.workunit.client.1.smithi117.stdout:4/200: write d0/d4/f12 [7502808,26025] 0 2024-03-22T06:15:19.311 INFO:tasks.workunit.client.1.smithi117.stdout:4/201: creat d0/d19/d28/f45 x:0 0 0 2024-03-22T06:15:19.312 INFO:tasks.workunit.client.0.smithi006.stdout:4/995: mkdir d6/d13/d3b/d164 0 2024-03-22T06:15:19.315 INFO:tasks.workunit.client.0.smithi006.stdout:1/866: getdents d8/d9/d13/d39/d60/d8f/d10a/d59/db1 0 2024-03-22T06:15:19.316 INFO:tasks.workunit.client.0.smithi006.stdout:1/867: getdents d8/d9/d13/d39/d60/d8f/d10a/d59/db1 0 2024-03-22T06:15:19.319 INFO:tasks.workunit.client.0.smithi006.stdout:9/875: getdents d8/d10/d28/dc2 0 2024-03-22T06:15:19.323 INFO:tasks.workunit.client.0.smithi006.stdout:0/754: unlink db/d10/d2a/d41/l4c 0 2024-03-22T06:15:19.323 INFO:tasks.workunit.client.0.smithi006.stdout:0/755: chown db/d3d/d4e/f90 66575706 1 2024-03-22T06:15:19.323 INFO:tasks.workunit.client.1.smithi117.stdout:0/219: mknod d1/d3/d2b/c4d 0 2024-03-22T06:15:19.329 INFO:tasks.workunit.client.1.smithi117.stdout:8/196: dwrite d1/f1e [0,4194304] 0 2024-03-22T06:15:19.329 INFO:tasks.workunit.client.1.smithi117.stdout:8/197: creat d1/d19/f43 x:0 0 0 2024-03-22T06:15:19.329 INFO:tasks.workunit.client.1.smithi117.stdout:9/224: dwrite d4/d7/f1c [0,4194304] 0 2024-03-22T06:15:19.329 INFO:tasks.workunit.client.1.smithi117.stdout:9/225: chown d4/d8/c34 34 1 2024-03-22T06:15:19.334 INFO:tasks.workunit.client.0.smithi006.stdout:6/885: mknod d6/c10e 0 2024-03-22T06:15:19.337 INFO:tasks.workunit.client.1.smithi117.stdout:1/175: symlink d1/d1f/l39 0 2024-03-22T06:15:19.342 INFO:tasks.workunit.client.0.smithi006.stdout:0/756: dwrite db/f80 [0,4194304] 0 2024-03-22T06:15:19.342 INFO:tasks.workunit.client.0.smithi006.stdout:0/757: write db/dc/d38/f8e [1184744,54785] 0 2024-03-22T06:15:19.342 INFO:tasks.workunit.client.0.smithi006.stdout:1/868: unlink d8/d33/lc1 0 2024-03-22T06:15:19.343 INFO:tasks.workunit.client.1.smithi117.stdout:4/202: symlink d0/d3f/l46 0 2024-03-22T06:15:19.344 INFO:tasks.workunit.client.0.smithi006.stdout:4/996: rmdir d6/d1b/d20/d21/d77/da0 39 2024-03-22T06:15:19.344 INFO:tasks.workunit.client.0.smithi006.stdout:6/886: dread d6/d1d/d5c/f5e [0,4194304] 0 2024-03-22T06:15:19.346 INFO:tasks.workunit.client.1.smithi117.stdout:4/203: dread d0/f27 [0,4194304] 0 2024-03-22T06:15:19.346 INFO:tasks.workunit.client.1.smithi117.stdout:4/204: write d0/d32/f3d [468308,4952] 0 2024-03-22T06:15:19.347 INFO:tasks.workunit.client.1.smithi117.stdout:5/157: rename dd/df/d12 to dd/df/d30/d35 0 2024-03-22T06:15:19.347 INFO:tasks.workunit.client.1.smithi117.stdout:6/229: rename d3/d40 to d3/d40/d24/d2d/d42/d4b 22 2024-03-22T06:15:19.347 INFO:tasks.workunit.client.1.smithi117.stdout:6/230: readlink d3/d7/l25 0 2024-03-22T06:15:19.347 INFO:tasks.workunit.client.1.smithi117.stdout:5/158: fsync dd/fe 0 2024-03-22T06:15:19.347 INFO:tasks.workunit.client.1.smithi117.stdout:6/231: truncate d3/d40/d24/f44 60815 0 2024-03-22T06:15:19.348 INFO:tasks.workunit.client.0.smithi006.stdout:0/758: rmdir db/dc/dab 0 2024-03-22T06:15:19.350 INFO:tasks.workunit.client.1.smithi117.stdout:0/220: mkdir d1/d3/d2b/d3e/de/d14/d40/d4e 0 2024-03-22T06:15:19.350 INFO:tasks.workunit.client.1.smithi117.stdout:0/221: creat d1/f4f x:0 0 0 2024-03-22T06:15:19.350 INFO:tasks.workunit.client.1.smithi117.stdout:1/176: read d1/d22/d27/f28 [1968574,38660] 0 2024-03-22T06:15:19.352 INFO:tasks.workunit.client.0.smithi006.stdout:9/876: getdents d8/d10/da6 0 2024-03-22T06:15:19.352 INFO:tasks.workunit.client.0.smithi006.stdout:9/877: creat d8/d10/d11/f123 x:0 0 0 2024-03-22T06:15:19.354 INFO:tasks.workunit.client.0.smithi006.stdout:4/997: rename d6/d1b/d20/d21/d2a/f119 to d6/d1b/d20/d63/d93/d9a/d8f/da1/db5/f165 0 2024-03-22T06:15:19.355 INFO:tasks.workunit.client.0.smithi006.stdout:4/998: creat d6/d13/d3b/f166 x:0 0 0 2024-03-22T06:15:19.355 INFO:tasks.workunit.client.1.smithi117.stdout:5/159: write dd/d16/d24/f23 [1692666,114202] 0 2024-03-22T06:15:19.357 INFO:tasks.workunit.client.0.smithi006.stdout:1/869: dwrite d8/d9/d3a/d18/f56 [4194304,4194304] 0 2024-03-22T06:15:19.357 INFO:tasks.workunit.client.1.smithi117.stdout:8/198: truncate d1/d2/f1d 1822944 0 2024-03-22T06:15:19.358 INFO:tasks.workunit.client.1.smithi117.stdout:8/199: fdatasync d1/f34 0 2024-03-22T06:15:19.358 INFO:tasks.workunit.client.0.smithi006.stdout:0/759: creat db/d10/d33/dce/df9/f102 x:0 0 0 2024-03-22T06:15:19.361 INFO:tasks.workunit.client.0.smithi006.stdout:9/878: creat d8/d10/d82/df7/f124 x:0 0 0 2024-03-22T06:15:19.362 INFO:tasks.workunit.client.1.smithi117.stdout:9/226: link d4/d7/c20 d4/d7/c44 0 2024-03-22T06:15:19.365 INFO:tasks.workunit.client.0.smithi006.stdout:4/999: creat d6/d1b/d20/d21/d2a/dc9/f167 x:0 0 0 2024-03-22T06:15:19.367 INFO:tasks.workunit.client.1.smithi117.stdout:3/134: sync 2024-03-22T06:15:19.369 INFO:tasks.workunit.client.1.smithi117.stdout:4/205: getdents d0/d3f 0 2024-03-22T06:15:19.370 INFO:tasks.workunit.client.1.smithi117.stdout:6/232: mknod d3/d40/d24/d2d/d30/d3e/c4c 0 2024-03-22T06:15:19.374 INFO:tasks.workunit.client.1.smithi117.stdout:0/222: symlink d1/d10/d18/d26/dc/l50 0 2024-03-22T06:15:19.374 INFO:tasks.workunit.client.1.smithi117.stdout:0/223: write d1/d3/d2b/d3e/de/d1f/f3c [433006,32926] 0 2024-03-22T06:15:19.374 INFO:tasks.workunit.client.1.smithi117.stdout:0/224: write d1/d3/d2b/d3e/de/d1f/d3b/f46 [586793,63677] 0 2024-03-22T06:15:19.375 INFO:tasks.workunit.client.0.smithi006.stdout:6/887: rmdir d6/d1d 39 2024-03-22T06:15:19.375 INFO:tasks.workunit.client.0.smithi006.stdout:6/888: stat d6/d7c/dc7 0 2024-03-22T06:15:19.376 INFO:tasks.workunit.client.1.smithi117.stdout:1/177: truncate d1/f7 3359661 0 2024-03-22T06:15:19.381 INFO:tasks.workunit.client.0.smithi006.stdout:1/870: symlink d8/d9/d10/d7c/d96/l124 0 2024-03-22T06:15:19.382 INFO:tasks.workunit.client.1.smithi117.stdout:3/135: symlink d9/l2f 0 2024-03-22T06:15:19.385 INFO:tasks.workunit.client.0.smithi006.stdout:9/879: mknod d8/d10/d11/c125 0 2024-03-22T06:15:19.387 INFO:tasks.workunit.client.1.smithi117.stdout:7/167: sync 2024-03-22T06:15:19.393 INFO:tasks.workunit.client.1.smithi117.stdout:4/206: mkdir d0/d19/d28/d47 0 2024-03-22T06:15:19.394 INFO:tasks.workunit.client.1.smithi117.stdout:4/207: truncate d0/f27 1671948 0 2024-03-22T06:15:19.398 INFO:tasks.workunit.client.0.smithi006.stdout:6/889: dwrite d6/d6f/d94/fa9 [0,4194304] 0 2024-03-22T06:15:19.398 INFO:tasks.workunit.client.0.smithi006.stdout:6/890: fsync d6/d24/d44/f90 0 2024-03-22T06:15:19.398 INFO:tasks.workunit.client.0.smithi006.stdout:6/891: fsync d6/d1b/f97 0 2024-03-22T06:15:19.399 INFO:tasks.workunit.client.0.smithi006.stdout:0/760: mkdir db/dc/d103 0 2024-03-22T06:15:19.400 INFO:tasks.workunit.client.1.smithi117.stdout:0/225: creat d1/d3/d2b/d3e/de/f51 x:0 0 0 2024-03-22T06:15:19.401 INFO:tasks.workunit.client.0.smithi006.stdout:0/761: write db/d10/d33/f42 [1501806,42788] 0 2024-03-22T06:15:19.402 INFO:tasks.workunit.client.0.smithi006.stdout:9/880: symlink d8/d10/d47/dfc/d113/l126 0 2024-03-22T06:15:19.405 INFO:tasks.workunit.client.1.smithi117.stdout:6/233: link d3/d7/da/cb d3/c4d 0 2024-03-22T06:15:19.405 INFO:tasks.workunit.client.0.smithi006.stdout:0/762: symlink db/dc/d37/dfe/l104 0 2024-03-22T06:15:19.406 INFO:tasks.workunit.client.1.smithi117.stdout:5/160: rename dd/df/d10 to dd/d16/d24/d36 0 2024-03-22T06:15:19.407 INFO:tasks.workunit.client.0.smithi006.stdout:9/881: creat d8/d10/d28/d39/d112/f127 x:0 0 0 2024-03-22T06:15:19.408 INFO:tasks.workunit.client.1.smithi117.stdout:0/226: creat d1/d10/d18/d26/dc/d39/f52 x:0 0 0 2024-03-22T06:15:19.409 INFO:tasks.workunit.client.1.smithi117.stdout:0/227: read d1/d3/d2b/d3e/de/d1f/d3b/f46 [43861,102919] 0 2024-03-22T06:15:19.409 INFO:tasks.workunit.client.0.smithi006.stdout:0/763: symlink db/d3d/d4e/l105 0 2024-03-22T06:15:19.409 INFO:tasks.workunit.client.0.smithi006.stdout:0/764: truncate db/d3d/d4e/f90 2160300 0 2024-03-22T06:15:19.409 INFO:tasks.workunit.client.0.smithi006.stdout:0/765: truncate db/d10/f14 5473916 0 2024-03-22T06:15:19.409 INFO:tasks.workunit.client.0.smithi006.stdout:0/766: chown db/d3d/d4e/f90 396 1 2024-03-22T06:15:19.410 INFO:tasks.workunit.client.0.smithi006.stdout:7/886: sync 2024-03-22T06:15:19.411 INFO:tasks.workunit.client.1.smithi117.stdout:7/168: dwrite d8/d9/da/d10/d23/f24 [0,4194304] 0 2024-03-22T06:15:19.411 INFO:tasks.workunit.client.1.smithi117.stdout:1/178: dwrite d1/f7 [0,4194304] 0 2024-03-22T06:15:19.411 INFO:tasks.workunit.client.1.smithi117.stdout:7/169: write d8/f1f [430851,51535] 0 2024-03-22T06:15:19.411 INFO:tasks.workunit.client.1.smithi117.stdout:1/179: write d1/f25 [2011458,73066] 0 2024-03-22T06:15:19.411 INFO:tasks.workunit.client.1.smithi117.stdout:1/180: fdatasync d1/f8 0 2024-03-22T06:15:19.412 INFO:tasks.workunit.client.0.smithi006.stdout:9/882: symlink d8/l128 0 2024-03-22T06:15:19.416 INFO:tasks.workunit.client.0.smithi006.stdout:7/887: rename d5/d1e/d63/d85/fab to d5/d1e/d29/d1f/d2d/df7/f122 0 2024-03-22T06:15:19.417 INFO:tasks.workunit.client.1.smithi117.stdout:5/161: creat dd/d16/d1a/f37 x:0 0 0 2024-03-22T06:15:19.418 INFO:tasks.workunit.client.0.smithi006.stdout:9/883: mkdir d8/d10/da6/d129 0 2024-03-22T06:15:19.418 INFO:tasks.workunit.client.0.smithi006.stdout:9/884: dread - d8/d10/d47/ffb zero size 2024-03-22T06:15:19.420 INFO:tasks.workunit.client.1.smithi117.stdout:4/208: rename d0/d32 to d0/d48 0 2024-03-22T06:15:19.422 INFO:tasks.workunit.client.1.smithi117.stdout:0/228: dread d1/d10/f32 [0,4194304] 0 2024-03-22T06:15:19.424 INFO:tasks.workunit.client.1.smithi117.stdout:5/162: dread dd/d16/d24/f34 [0,4194304] 0 2024-03-22T06:15:19.424 INFO:tasks.workunit.client.1.smithi117.stdout:5/163: creat dd/df/f38 x:0 0 0 2024-03-22T06:15:19.424 INFO:tasks.workunit.client.1.smithi117.stdout:5/164: write dd/d16/d24/d36/f26 [397370,41046] 0 2024-03-22T06:15:19.424 INFO:tasks.workunit.client.1.smithi117.stdout:5/165: write dd/d16/d24/d36/f31 [60180,75683] 0 2024-03-22T06:15:19.424 INFO:tasks.workunit.client.1.smithi117.stdout:4/209: dread d0/d19/f22 [0,4194304] 0 2024-03-22T06:15:19.425 INFO:tasks.workunit.client.0.smithi006.stdout:0/767: write db/dc/d37/f63 [2966979,128470] 0 2024-03-22T06:15:19.425 INFO:tasks.workunit.client.0.smithi006.stdout:7/888: mknod d5/d1e/d63/d85/dac/db3/c123 0 2024-03-22T06:15:19.425 INFO:tasks.workunit.client.0.smithi006.stdout:7/889: fdatasync d5/d1e/f95 0 2024-03-22T06:15:19.426 INFO:tasks.workunit.client.1.smithi117.stdout:2/176: sync 2024-03-22T06:15:19.428 INFO:tasks.workunit.client.0.smithi006.stdout:9/885: rmdir d8/d1f/d34/dfa 39 2024-03-22T06:15:19.428 INFO:tasks.workunit.client.0.smithi006.stdout:9/886: write d8/d10/d28/f44 [1796279,22104] 0 2024-03-22T06:15:19.429 INFO:tasks.workunit.client.1.smithi117.stdout:7/170: mknod d8/d9/da/d2b/c31 0 2024-03-22T06:15:19.429 INFO:tasks.workunit.client.1.smithi117.stdout:7/171: write f5 [5731617,2400] 0 2024-03-22T06:15:19.430 INFO:tasks.workunit.client.0.smithi006.stdout:1/871: dwrite d8/d9/d10/d7c/d96/fba [0,4194304] 0 2024-03-22T06:15:19.431 INFO:tasks.workunit.client.1.smithi117.stdout:1/181: symlink d1/d22/d27/l3a 0 2024-03-22T06:15:19.431 INFO:tasks.workunit.client.1.smithi117.stdout:1/182: truncate d1/d22/d27/f34 20299 0 2024-03-22T06:15:19.433 INFO:tasks.workunit.client.0.smithi006.stdout:9/887: mknod d8/db3/d50/d91/c12a 0 2024-03-22T06:15:19.436 INFO:tasks.workunit.client.0.smithi006.stdout:0/768: getdents db/dc/d37 0 2024-03-22T06:15:19.443 INFO:tasks.workunit.client.1.smithi117.stdout:0/229: rmdir d1/d3/d2b/d3e/de/d14 39 2024-03-22T06:15:19.443 INFO:tasks.workunit.client.1.smithi117.stdout:0/230: write d1/d10/d18/d26/dc/f25 [1534931,13925] 0 2024-03-22T06:15:19.443 INFO:tasks.workunit.client.1.smithi117.stdout:0/231: chown d1/d10/f11 1671977 1 2024-03-22T06:15:19.445 INFO:tasks.workunit.client.0.smithi006.stdout:9/888: truncate d8/d10/d20/d45/d49/f8c 112101 0 2024-03-22T06:15:19.447 INFO:tasks.workunit.client.1.smithi117.stdout:5/166: rename dd/d16/d24/d36/f2f to dd/d16/d24/d36/f39 0 2024-03-22T06:15:19.447 INFO:tasks.workunit.client.1.smithi117.stdout:5/167: creat dd/df/f3a x:0 0 0 2024-03-22T06:15:19.449 INFO:tasks.workunit.client.1.smithi117.stdout:4/210: dwrite d0/d48/f3d [0,4194304] 0 2024-03-22T06:15:19.449 INFO:tasks.workunit.client.1.smithi117.stdout:4/211: chown d0/d4/f12 527968164 1 2024-03-22T06:15:19.449 INFO:tasks.workunit.client.0.smithi006.stdout:1/872: dwrite d8/d9/d13/d24/d32/d34/f77 [0,4194304] 0 2024-03-22T06:15:19.449 INFO:tasks.workunit.client.0.smithi006.stdout:1/873: dread - d8/d9/d13/d39/f121 zero size 2024-03-22T06:15:19.450 INFO:tasks.workunit.client.0.smithi006.stdout:0/769: mkdir db/d10/d33/d9d/dcd/d106 0 2024-03-22T06:15:19.452 INFO:tasks.workunit.client.1.smithi117.stdout:2/177: symlink d2/d5/d8/l31 0 2024-03-22T06:15:19.453 INFO:tasks.workunit.client.1.smithi117.stdout:2/178: stat d2/d5/d20/d1d/f27 0 2024-03-22T06:15:19.454 INFO:tasks.workunit.client.1.smithi117.stdout:7/172: mkdir d8/d9/da/d10/d23/d27/d32 0 2024-03-22T06:15:19.456 INFO:tasks.workunit.client.0.smithi006.stdout:1/874: rename d8/d9/d13/d39/d60/d8f/d10a/d59/le6 to d8/d9/d10/d7c/d96/l125 0 2024-03-22T06:15:19.456 INFO:tasks.workunit.client.0.smithi006.stdout:1/875: write d8/d9/d13/d24/fc8 [976941,73959] 0 2024-03-22T06:15:19.456 INFO:tasks.workunit.client.1.smithi117.stdout:1/183: symlink d1/d22/l3b 0 2024-03-22T06:15:19.457 INFO:tasks.workunit.client.0.smithi006.stdout:0/770: getdents db/d10/dc2 0 2024-03-22T06:15:19.459 INFO:tasks.workunit.client.1.smithi117.stdout:8/200: rename d1/d2 to d1/d3a/d44 0 2024-03-22T06:15:19.459 INFO:tasks.workunit.client.1.smithi117.stdout:8/201: chown d1 13233051 1 2024-03-22T06:15:19.460 INFO:tasks.workunit.client.1.smithi117.stdout:8/202: dread d1/d19/f3f [0,4194304] 0 2024-03-22T06:15:19.460 INFO:tasks.workunit.client.1.smithi117.stdout:8/203: write d1/d3a/d44/f1d [2679750,40440] 0 2024-03-22T06:15:19.461 INFO:tasks.workunit.client.1.smithi117.stdout:2/179: read d2/d5/d20/f1f [3405859,106399] 0 2024-03-22T06:15:19.461 INFO:tasks.workunit.client.1.smithi117.stdout:2/180: readlink d2/d1b/l2e 0 2024-03-22T06:15:19.461 INFO:tasks.workunit.client.1.smithi117.stdout:2/181: truncate d2/d5/d13/f19 4260833 0 2024-03-22T06:15:19.461 INFO:tasks.workunit.client.0.smithi006.stdout:9/889: dwrite d8/d10/d20/d45/d4d/fd0 [0,4194304] 0 2024-03-22T06:15:19.462 INFO:tasks.workunit.client.0.smithi006.stdout:1/876: rename d8/d9/d10/l98 to d8/d9/d13/d39/d94/db5/df1/l126 0 2024-03-22T06:15:19.466 INFO:tasks.workunit.client.0.smithi006.stdout:0/771: symlink db/d10/d33/l107 0 2024-03-22T06:15:19.468 INFO:tasks.workunit.client.1.smithi117.stdout:5/168: symlink dd/df/d1e/l3b 0 2024-03-22T06:15:19.471 INFO:tasks.workunit.client.1.smithi117.stdout:2/182: write d2/d5/d8/d10/d11/f15 [7506255,18171] 0 2024-03-22T06:15:19.471 INFO:tasks.workunit.client.1.smithi117.stdout:2/183: write d2/d1b/f28 [302851,60057] 0 2024-03-22T06:15:19.472 INFO:tasks.workunit.client.0.smithi006.stdout:1/877: write d8/d9/d13/d39/d60/d8f/d10a/d44/f110 [3580765,10869] 0 2024-03-22T06:15:19.472 INFO:tasks.workunit.client.1.smithi117.stdout:1/184: dwrite d1/f7 [0,4194304] 0 2024-03-22T06:15:19.474 INFO:tasks.workunit.client.0.smithi006.stdout:9/890: truncate d8/d10/d28/fd1 192416 0 2024-03-22T06:15:19.477 INFO:tasks.workunit.client.0.smithi006.stdout:0/772: creat db/dc/d37/dfe/f108 x:0 0 0 2024-03-22T06:15:19.481 INFO:tasks.workunit.client.1.smithi117.stdout:7/173: link c2 d8/d9/d1a/d26/c33 0 2024-03-22T06:15:19.485 INFO:tasks.workunit.client.1.smithi117.stdout:9/227: rename d4/d7/d10/f3a to d4/d8/d3e/f45 0 2024-03-22T06:15:19.486 INFO:tasks.workunit.client.0.smithi006.stdout:9/891: mknod d8/db3/d50/d91/dfe/d11a/c12b 0 2024-03-22T06:15:19.486 INFO:tasks.workunit.client.0.smithi006.stdout:9/892: dread - d8/d10/d28/f52 zero size 2024-03-22T06:15:19.486 INFO:tasks.workunit.client.0.smithi006.stdout:9/893: chown d8/db3/d10c 136 1 2024-03-22T06:15:19.489 INFO:tasks.workunit.client.1.smithi117.stdout:8/204: symlink d1/d19/d38/l45 0 2024-03-22T06:15:19.489 INFO:tasks.workunit.client.1.smithi117.stdout:8/205: fsync d1/d19/f2d 0 2024-03-22T06:15:19.491 INFO:tasks.workunit.client.0.smithi006.stdout:1/878: dread d8/d9/f35 [0,4194304] 0 2024-03-22T06:15:19.491 INFO:tasks.workunit.client.0.smithi006.stdout:1/879: fsync d8/d9/d10/f4e 0 2024-03-22T06:15:19.492 INFO:tasks.workunit.client.1.smithi117.stdout:5/169: symlink dd/df/d30/l3c 0 2024-03-22T06:15:19.496 INFO:tasks.workunit.client.1.smithi117.stdout:2/184: truncate d2/f4 2909712 0 2024-03-22T06:15:19.496 INFO:tasks.workunit.client.1.smithi117.stdout:2/185: chown d2/c16 12113656 1 2024-03-22T06:15:19.498 INFO:tasks.workunit.client.1.smithi117.stdout:9/228: dread d4/d7/f1c [0,4194304] 0 2024-03-22T06:15:19.498 INFO:tasks.workunit.client.1.smithi117.stdout:9/229: fsync d4/d8/f13 0 2024-03-22T06:15:19.498 INFO:tasks.workunit.client.1.smithi117.stdout:9/230: read - d4/d8/f41 zero size 2024-03-22T06:15:19.498 INFO:tasks.workunit.client.1.smithi117.stdout:1/185: mkdir d1/d3c 0 2024-03-22T06:15:19.499 INFO:tasks.workunit.client.0.smithi006.stdout:9/894: rename d8/d1f/d34 to d8/db3/d50/d91/dfe/d12c 0 2024-03-22T06:15:19.500 INFO:tasks.workunit.client.1.smithi117.stdout:4/212: truncate d0/f23 2596104 0 2024-03-22T06:15:19.500 INFO:tasks.workunit.client.1.smithi117.stdout:4/213: chown d0/f2d 3 1 2024-03-22T06:15:19.502 INFO:tasks.workunit.client.0.smithi006.stdout:0/773: dwrite db/d10/f1b [0,4194304] 0 2024-03-22T06:15:19.502 INFO:tasks.workunit.client.0.smithi006.stdout:0/774: fdatasync db/dc/d38/d50/fc3 0 2024-03-22T06:15:19.504 INFO:tasks.workunit.client.1.smithi117.stdout:3/136: rename f6 to d9/f30 0 2024-03-22T06:15:19.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:19 smithi117 ceph-mon[36514]: pgmap v23: 65 pgs: 65 active+clean; 1.8 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 35 MiB/s rd, 46 MiB/s wr, 802 op/s 2024-03-22T06:15:19.509 INFO:tasks.workunit.client.1.smithi117.stdout:3/137: write d9/fb [102053,74446] 0 2024-03-22T06:15:19.509 INFO:tasks.workunit.client.1.smithi117.stdout:1/186: dread d1/f7 [0,4194304] 0 2024-03-22T06:15:19.512 INFO:tasks.workunit.client.0.smithi006.stdout:0/775: mkdir db/d10/d2a/d41/d109 0 2024-03-22T06:15:19.513 INFO:tasks.workunit.client.1.smithi117.stdout:8/206: link d1/d19/d38/l45 d1/d3a/d44/d11/l46 0 2024-03-22T06:15:19.513 INFO:tasks.workunit.client.1.smithi117.stdout:8/207: chown d1/c5 34406970 1 2024-03-22T06:15:19.519 INFO:tasks.workunit.client.1.smithi117.stdout:5/170: link dd/df/d1e/l3b dd/df/l3d 0 2024-03-22T06:15:19.519 INFO:tasks.workunit.client.1.smithi117.stdout:5/171: fsync dd/df/f38 0 2024-03-22T06:15:19.519 INFO:tasks.workunit.client.1.smithi117.stdout:5/172: fsync dd/df/f2b 0 2024-03-22T06:15:19.519 INFO:tasks.workunit.client.1.smithi117.stdout:5/173: chown dd/df/d30 23748058 1 2024-03-22T06:15:19.521 INFO:tasks.workunit.client.1.smithi117.stdout:2/186: mknod d2/d5/d20/c32 0 2024-03-22T06:15:19.521 INFO:tasks.workunit.client.1.smithi117.stdout:2/187: truncate d2/d5/db/f24 723246 0 2024-03-22T06:15:19.521 INFO:tasks.workunit.client.1.smithi117.stdout:9/231: rmdir d4/d7 39 2024-03-22T06:15:19.521 INFO:tasks.workunit.client.1.smithi117.stdout:9/232: stat d4/d8/f22 0 2024-03-22T06:15:19.521 INFO:tasks.workunit.client.0.smithi006.stdout:1/880: dwrite d8/d9/d13/f28 [4194304,4194304] 0 2024-03-22T06:15:19.521 INFO:tasks.workunit.client.0.smithi006.stdout:1/881: read - d8/d9/d13/f10b zero size 2024-03-22T06:15:19.522 INFO:tasks.workunit.client.0.smithi006.stdout:1/882: chown d8/d9/d13/d39/d60/d8f/d10a/d44/d80/f88 602587 1 2024-03-22T06:15:19.522 INFO:tasks.workunit.client.0.smithi006.stdout:1/883: read d8/d9/d3a/d18/f1b [2822464,36105] 0 2024-03-22T06:15:19.522 INFO:tasks.workunit.client.0.smithi006.stdout:1/884: creat d8/d9/d10/f127 x:0 0 0 2024-03-22T06:15:19.522 INFO:tasks.workunit.client.0.smithi006.stdout:9/895: dwrite d8/d10/d20/d45/f66 [0,4194304] 0 2024-03-22T06:15:19.522 INFO:tasks.workunit.client.0.smithi006.stdout:9/896: chown d8/d1f/f68 373819229 1 2024-03-22T06:15:19.524 INFO:tasks.workunit.client.0.smithi006.stdout:1/885: symlink d8/d9/d13/d39/d60/d8f/d10a/d44/d80/l128 0 2024-03-22T06:15:19.525 INFO:tasks.workunit.client.1.smithi117.stdout:4/214: creat d0/d19/f49 x:0 0 0 2024-03-22T06:15:19.527 INFO:tasks.workunit.client.1.smithi117.stdout:6/234: rename d3/d7/da/d32/f4a to d3/d40/d24/f4e 0 2024-03-22T06:15:19.528 INFO:tasks.workunit.client.1.smithi117.stdout:6/235: chown d3/d40/f22 3 1 2024-03-22T06:15:19.530 INFO:tasks.workunit.client.1.smithi117.stdout:3/138: mknod d9/c31 0 2024-03-22T06:15:19.530 INFO:tasks.workunit.client.1.smithi117.stdout:3/139: chown d9/f18 11294 1 2024-03-22T06:15:19.533 INFO:tasks.workunit.client.0.smithi006.stdout:1/886: dread d8/d33/f9c [0,4194304] 0 2024-03-22T06:15:19.534 INFO:tasks.workunit.client.0.smithi006.stdout:9/897: mkdir d8/d10/d23/d12d 0 2024-03-22T06:15:19.534 INFO:tasks.workunit.client.0.smithi006.stdout:9/898: read d8/d10/d28/f25 [1154109,56977] 0 2024-03-22T06:15:19.535 INFO:tasks.workunit.client.0.smithi006.stdout:0/776: write db/d10/f23 [3664342,88362] 0 2024-03-22T06:15:19.538 INFO:tasks.workunit.client.1.smithi117.stdout:0/232: rename d1/d4b to d1/d10/d18/d26/d53 0 2024-03-22T06:15:19.539 INFO:tasks.workunit.client.1.smithi117.stdout:6/236: dread d3/d7/da/de/d2c/f2f [0,4194304] 0 2024-03-22T06:15:19.542 INFO:tasks.workunit.client.0.smithi006.stdout:9/899: truncate d8/d10/f56 552570 0 2024-03-22T06:15:19.542 INFO:tasks.workunit.client.0.smithi006.stdout:9/900: fsync d8/db3/d50/d91/f109 0 2024-03-22T06:15:19.542 INFO:tasks.workunit.client.0.smithi006.stdout:9/901: readlink d8/d10/l30 0 2024-03-22T06:15:19.545 INFO:tasks.workunit.client.1.smithi117.stdout:3/140: mknod d9/c32 0 2024-03-22T06:15:19.546 INFO:tasks.workunit.client.1.smithi117.stdout:3/141: readlink d9/d1a/l29 0 2024-03-22T06:15:19.546 INFO:tasks.workunit.client.1.smithi117.stdout:3/142: write d9/fb [1042350,44157] 0 2024-03-22T06:15:19.546 INFO:tasks.workunit.client.0.smithi006.stdout:1/887: dread d8/d9/d3a/d18/ffe [0,4194304] 0 2024-03-22T06:15:19.546 INFO:tasks.workunit.client.0.smithi006.stdout:0/777: mknod db/d10/d35/d7b/da1/da6/dc4/dfc/c10a 0 2024-03-22T06:15:19.547 INFO:tasks.workunit.client.1.smithi117.stdout:4/215: link d0/d19/d34/f37 d0/d19/f4a 0 2024-03-22T06:15:19.547 INFO:tasks.workunit.client.0.smithi006.stdout:9/902: symlink d8/db3/d10c/l12e 0 2024-03-22T06:15:19.547 INFO:tasks.workunit.client.0.smithi006.stdout:9/903: chown d8/db3/d50/d91/dfe/d11a/de0 49994 1 2024-03-22T06:15:19.548 INFO:tasks.workunit.client.0.smithi006.stdout:9/904: truncate d8/d10/d47/dbc/fcc 426904 0 2024-03-22T06:15:19.549 INFO:tasks.workunit.client.0.smithi006.stdout:1/888: creat d8/d9/d13/d39/d60/d8f/d10a/d44/d80/f129 x:0 0 0 2024-03-22T06:15:19.552 INFO:tasks.workunit.client.1.smithi117.stdout:1/187: rename d1/d1f/f31 to d1/d1f/d32/d33/f3d 0 2024-03-22T06:15:19.552 INFO:tasks.workunit.client.1.smithi117.stdout:1/188: creat d1/d1f/f3e x:0 0 0 2024-03-22T06:15:19.554 INFO:tasks.workunit.client.0.smithi006.stdout:0/778: unlink db/d10/d2a/d41/c73 0 2024-03-22T06:15:19.554 INFO:tasks.workunit.client.0.smithi006.stdout:0/779: creat db/d87/ddd/f10b x:0 0 0 2024-03-22T06:15:19.554 INFO:tasks.workunit.client.1.smithi117.stdout:0/233: mknod d1/d3/d2b/d3e/de/c54 0 2024-03-22T06:15:19.555 INFO:tasks.workunit.client.0.smithi006.stdout:9/905: creat d8/d10/d82/df7/d10b/f12f x:0 0 0 2024-03-22T06:15:19.557 INFO:tasks.workunit.client.1.smithi117.stdout:2/188: dwrite d2/fc [0,4194304] 0 2024-03-22T06:15:19.558 INFO:tasks.workunit.client.1.smithi117.stdout:5/174: dwrite dd/f29 [0,4194304] 0 2024-03-22T06:15:19.558 INFO:tasks.workunit.client.1.smithi117.stdout:9/233: dwrite d4/d7/f1c [0,4194304] 0 2024-03-22T06:15:19.558 INFO:tasks.workunit.client.1.smithi117.stdout:9/234: readlink d4/d7/l32 0 2024-03-22T06:15:19.558 INFO:tasks.workunit.client.1.smithi117.stdout:9/235: stat d4/d7/l3d 0 2024-03-22T06:15:19.559 INFO:tasks.workunit.client.1.smithi117.stdout:3/143: truncate f7 813226 0 2024-03-22T06:15:19.559 INFO:tasks.workunit.client.1.smithi117.stdout:3/144: getdents d9/d2e 0 2024-03-22T06:15:19.559 INFO:tasks.workunit.client.1.smithi117.stdout:3/145: dread - d9/d1a/f20 zero size 2024-03-22T06:15:19.559 INFO:tasks.workunit.client.1.smithi117.stdout:3/146: read - d9/f26 zero size 2024-03-22T06:15:19.561 INFO:tasks.workunit.client.0.smithi006.stdout:1/889: write d8/d71/fed [2922591,121527] 0 2024-03-22T06:15:19.561 INFO:tasks.workunit.client.0.smithi006.stdout:1/890: dread - d8/d9/d13/d39/d60/d8f/d10a/fd0 zero size 2024-03-22T06:15:19.562 INFO:tasks.workunit.client.0.smithi006.stdout:1/891: chown d8/d71/cde 522550 1 2024-03-22T06:15:19.564 INFO:tasks.workunit.client.0.smithi006.stdout:0/780: truncate db/dc/d38/f4b 1315 0 2024-03-22T06:15:19.570 INFO:tasks.workunit.client.0.smithi006.stdout:1/892: creat d8/d9/d13/d24/f12a x:0 0 0 2024-03-22T06:15:19.570 INFO:tasks.workunit.client.0.smithi006.stdout:1/893: chown d8/d9/d13/d39/d60/d8f/d10a/d44/d80 1047113 1 2024-03-22T06:15:19.571 INFO:tasks.workunit.client.0.smithi006.stdout:1/894: fsync d8/d9/d13/f28 0 2024-03-22T06:15:19.571 INFO:tasks.workunit.client.1.smithi117.stdout:9/236: dread f2 [0,4194304] 0 2024-03-22T06:15:19.572 INFO:tasks.workunit.client.1.smithi117.stdout:9/237: fsync d4/d8/f25 0 2024-03-22T06:15:19.573 INFO:tasks.workunit.client.0.smithi006.stdout:0/781: rename db/d87/c5b to db/d3d/d4e/c10c 0 2024-03-22T06:15:19.574 INFO:tasks.workunit.client.1.smithi117.stdout:1/189: rmdir d1/d22 39 2024-03-22T06:15:19.575 INFO:tasks.workunit.client.1.smithi117.stdout:5/175: dread fa [0,4194304] 0 2024-03-22T06:15:19.575 INFO:tasks.workunit.client.1.smithi117.stdout:5/176: readlink lc 0 2024-03-22T06:15:19.575 INFO:tasks.workunit.client.1.smithi117.stdout:5/177: read dd/d16/d24/d36/f39 [2393128,20845] 0 2024-03-22T06:15:19.575 INFO:tasks.workunit.client.1.smithi117.stdout:5/178: write dd/d16/f22 [3288364,126115] 0 2024-03-22T06:15:19.576 INFO:tasks.workunit.client.1.smithi117.stdout:2/189: dwrite d2/d1b/f28 [0,4194304] 0 2024-03-22T06:15:19.576 INFO:tasks.workunit.client.1.smithi117.stdout:0/234: mkdir d1/d3/d2b/d3e/de/d1f/d3b/d55 0 2024-03-22T06:15:19.579 INFO:tasks.workunit.client.0.smithi006.stdout:0/782: mknod db/d10/d2a/d41/d109/c10d 0 2024-03-22T06:15:19.580 INFO:tasks.workunit.client.0.smithi006.stdout:1/895: getdents d8/d71/dbb 0 2024-03-22T06:15:19.580 INFO:tasks.workunit.client.0.smithi006.stdout:1/896: write d8/d71/fed [4268868,88369] 0 2024-03-22T06:15:19.582 INFO:tasks.workunit.client.0.smithi006.stdout:9/906: dwrite d8/d10/d20/d45/d4d/f102 [0,4194304] 0 2024-03-22T06:15:19.585 INFO:tasks.workunit.client.1.smithi117.stdout:1/190: dread d1/d14/f19 [0,4194304] 0 2024-03-22T06:15:19.585 INFO:tasks.workunit.client.1.smithi117.stdout:5/179: dread dd/d16/d24/d36/f25 [0,4194304] 0 2024-03-22T06:15:19.586 INFO:tasks.workunit.client.1.smithi117.stdout:4/216: rmdir d0/d19/d28 39 2024-03-22T06:15:19.586 INFO:tasks.workunit.client.1.smithi117.stdout:4/217: write d0/d4/d2a/d2f/f31 [1525449,111666] 0 2024-03-22T06:15:19.588 INFO:tasks.workunit.client.1.smithi117.stdout:1/191: dread d1/f25 [0,4194304] 0 2024-03-22T06:15:19.588 INFO:tasks.workunit.client.0.smithi006.stdout:1/897: mknod d8/d9/d13/d24/d32/d34/daa/c12b 0 2024-03-22T06:15:19.588 INFO:tasks.workunit.client.0.smithi006.stdout:1/898: write d8/d9/d3a/f92 [4378369,46837] 0 2024-03-22T06:15:19.588 INFO:tasks.workunit.client.0.smithi006.stdout:1/899: chown d8/d71/dcd/l10d 170 1 2024-03-22T06:15:19.588 INFO:tasks.workunit.client.0.smithi006.stdout:1/900: chown d8/d9/d10/d7c/d96/la5 83 1 2024-03-22T06:15:19.590 INFO:tasks.workunit.client.0.smithi006.stdout:9/907: link d8/d10/d20/d45/l6d d8/d10/d28/dba/l130 0 2024-03-22T06:15:19.593 INFO:tasks.workunit.client.0.smithi006.stdout:0/783: mknod db/dc/c10e 0 2024-03-22T06:15:19.593 INFO:tasks.workunit.client.0.smithi006.stdout:0/784: dread - db/d10/d33/d9d/fe1 zero size 2024-03-22T06:15:19.593 INFO:tasks.workunit.client.0.smithi006.stdout:0/785: creat db/dc/d38/f10f x:0 0 0 2024-03-22T06:15:19.594 INFO:tasks.workunit.client.1.smithi117.stdout:5/180: read dd/df/d30/d35/f27 [4074795,50371] 0 2024-03-22T06:15:19.595 INFO:tasks.workunit.client.0.smithi006.stdout:1/901: creat d8/d9/d13/d39/d60/d8f/f12c x:0 0 0 2024-03-22T06:15:19.600 INFO:tasks.workunit.client.0.smithi006.stdout:9/908: mknod d8/d10/d82/c131 0 2024-03-22T06:15:19.608 INFO:tasks.workunit.client.1.smithi117.stdout:2/190: creat d2/d5/f33 x:0 0 0 2024-03-22T06:15:19.608 INFO:tasks.workunit.client.0.smithi006.stdout:9/909: dread d8/d10/da6/fab [0,4194304] 0 2024-03-22T06:15:19.609 INFO:tasks.workunit.client.0.smithi006.stdout:0/786: dwrite db/d10/f14 [0,4194304] 0 2024-03-22T06:15:19.609 INFO:tasks.workunit.client.0.smithi006.stdout:0/787: write db/d87/f9f [462604,81285] 0 2024-03-22T06:15:19.611 INFO:tasks.workunit.client.1.smithi117.stdout:0/235: symlink d1/d3/l56 0 2024-03-22T06:15:19.611 INFO:tasks.workunit.client.1.smithi117.stdout:0/236: chown d1/d3/d2b/d3e/de/d1f/f3c 4598924 1 2024-03-22T06:15:19.611 INFO:tasks.workunit.client.1.smithi117.stdout:0/237: write d1/d3/d2b/d3e/ff [3249971,65718] 0 2024-03-22T06:15:19.611 INFO:tasks.workunit.client.1.smithi117.stdout:0/238: write d1/d3/f12 [4106183,113475] 0 2024-03-22T06:15:19.611 INFO:tasks.workunit.client.1.smithi117.stdout:0/239: fsync d1/d10/d18/d26/dc/f25 0 2024-03-22T06:15:19.614 INFO:tasks.workunit.client.1.smithi117.stdout:1/192: mknod d1/d30/d35/d36/c3f 0 2024-03-22T06:15:19.614 INFO:tasks.workunit.client.1.smithi117.stdout:0/240: dread d1/d10/f27 [0,4194304] 0 2024-03-22T06:15:19.615 INFO:tasks.workunit.client.0.smithi006.stdout:9/910: creat d8/db3/d50/d91/dfe/d11a/f132 x:0 0 0 2024-03-22T06:15:19.615 INFO:tasks.workunit.client.0.smithi006.stdout:9/911: chown d8/db3/d10c/l12e 0 1 2024-03-22T06:15:19.616 INFO:tasks.workunit.client.0.smithi006.stdout:9/912: readlink d8/d1f/l94 0 2024-03-22T06:15:19.616 INFO:tasks.workunit.client.0.smithi006.stdout:9/913: chown d8/d10/da6/fed 2 1 2024-03-22T06:15:19.617 INFO:tasks.workunit.client.1.smithi117.stdout:6/237: rename d3/d7/da/de/d2c to d3/d40/d46/d4f 0 2024-03-22T06:15:19.619 INFO:tasks.workunit.client.0.smithi006.stdout:0/788: creat db/d10/d33/d9d/dcd/d106/f110 x:0 0 0 2024-03-22T06:15:19.619 INFO:tasks.workunit.client.0.smithi006.stdout:0/789: readlink db/d10/d2a/la4 0 2024-03-22T06:15:19.619 INFO:tasks.workunit.client.0.smithi006.stdout:0/790: chown db/d10/d35/d7b/fe8 23329235 1 2024-03-22T06:15:19.619 INFO:tasks.workunit.client.1.smithi117.stdout:9/238: mkdir d4/d46 0 2024-03-22T06:15:19.622 INFO:tasks.workunit.client.1.smithi117.stdout:5/181: truncate dd/df/d30/d35/f27 2046749 0 2024-03-22T06:15:19.622 INFO:tasks.workunit.client.1.smithi117.stdout:5/182: write dd/df/f2a [1113875,57905] 0 2024-03-22T06:15:19.622 INFO:tasks.workunit.client.1.smithi117.stdout:5/183: write dd/df/f3a [573188,90518] 0 2024-03-22T06:15:19.622 INFO:tasks.workunit.client.1.smithi117.stdout:5/184: write dd/fe [3648737,73628] 0 2024-03-22T06:15:19.622 INFO:tasks.workunit.client.1.smithi117.stdout:5/185: creat dd/d16/d1a/f3e x:0 0 0 2024-03-22T06:15:19.622 INFO:tasks.workunit.client.1.smithi117.stdout:5/186: truncate dd/d16/d24/d36/f31 414868 0 2024-03-22T06:15:19.623 INFO:tasks.workunit.client.1.smithi117.stdout:2/191: truncate d2/d5/d8/d10/d11/f15 5947763 0 2024-03-22T06:15:19.626 INFO:tasks.workunit.client.1.smithi117.stdout:3/147: rename d9/l13 to d9/d1a/l33 0 2024-03-22T06:15:19.626 INFO:tasks.workunit.client.1.smithi117.stdout:3/148: creat d9/d1a/f34 x:0 0 0 2024-03-22T06:15:19.626 INFO:tasks.workunit.client.1.smithi117.stdout:3/149: creat d9/f35 x:0 0 0 2024-03-22T06:15:19.627 INFO:tasks.workunit.client.0.smithi006.stdout:0/791: rmdir db/d10/d33/dce/df9 39 2024-03-22T06:15:19.627 INFO:tasks.workunit.client.0.smithi006.stdout:0/792: write db/dc/d38/fb9 [1362175,53253] 0 2024-03-22T06:15:19.628 INFO:tasks.workunit.client.1.smithi117.stdout:1/193: write d1/f8 [1619937,87966] 0 2024-03-22T06:15:19.630 INFO:tasks.workunit.client.1.smithi117.stdout:5/187: dread dd/d16/d24/f34 [0,4194304] 0 2024-03-22T06:15:19.630 INFO:tasks.workunit.client.0.smithi006.stdout:9/914: rename d8/db3/d50/d38 to d8/db3/d133 0 2024-03-22T06:15:19.631 INFO:tasks.workunit.client.1.smithi117.stdout:2/192: mknod d2/d5/d8/d10/c34 0 2024-03-22T06:15:19.632 INFO:tasks.workunit.client.1.smithi117.stdout:2/193: read d2/d5/d8/f22 [411239,79161] 0 2024-03-22T06:15:19.632 INFO:tasks.workunit.client.1.smithi117.stdout:2/194: dread - d2/d5/d13/f30 zero size 2024-03-22T06:15:19.633 INFO:tasks.workunit.client.1.smithi117.stdout:4/218: rename d0/d4/d2a/d2f/f33 to d0/d3f/f4b 0 2024-03-22T06:15:19.636 INFO:tasks.workunit.client.1.smithi117.stdout:9/239: dwrite d4/d7/d10/f42 [0,4194304] 0 2024-03-22T06:15:19.638 INFO:tasks.workunit.client.0.smithi006.stdout:0/793: dread db/f45 [0,4194304] 0 2024-03-22T06:15:19.639 INFO:tasks.workunit.client.1.smithi117.stdout:3/150: mkdir d9/d2e/d36 0 2024-03-22T06:15:19.640 INFO:tasks.workunit.client.1.smithi117.stdout:2/195: mkdir d2/d35 0 2024-03-22T06:15:19.640 INFO:tasks.workunit.client.1.smithi117.stdout:2/196: truncate d2/d5/d13/f30 777417 0 2024-03-22T06:15:19.640 INFO:tasks.workunit.client.1.smithi117.stdout:2/197: chown d2/c16 3814 1 2024-03-22T06:15:19.641 INFO:tasks.workunit.client.0.smithi006.stdout:0/794: symlink db/d10/d35/d7b/da1/l111 0 2024-03-22T06:15:19.642 INFO:tasks.workunit.client.1.smithi117.stdout:4/219: symlink d0/d19/d34/l4c 0 2024-03-22T06:15:19.644 INFO:tasks.workunit.client.1.smithi117.stdout:0/241: rename d1/d3/d2b/d3e/de/d14/l36 to d1/d3/d2b/d3e/de/l57 0 2024-03-22T06:15:19.645 INFO:tasks.workunit.client.0.smithi006.stdout:0/795: symlink db/d10/d33/dce/l112 0 2024-03-22T06:15:19.645 INFO:tasks.workunit.client.0.smithi006.stdout:0/796: fsync db/d10/d33/db4/fd7 0 2024-03-22T06:15:19.648 INFO:tasks.workunit.client.1.smithi117.stdout:9/240: rmdir d4/d7 39 2024-03-22T06:15:19.649 INFO:tasks.workunit.client.0.smithi006.stdout:0/797: creat db/dc/d103/f113 x:0 0 0 2024-03-22T06:15:19.649 INFO:tasks.workunit.client.0.smithi006.stdout:0/798: chown db/d10/d35/d7b/da1/da6/dc4/le6 924535 1 2024-03-22T06:15:19.650 INFO:tasks.workunit.client.1.smithi117.stdout:3/151: mkdir d9/d2e/d37 0 2024-03-22T06:15:19.651 INFO:tasks.workunit.client.0.smithi006.stdout:0/799: mkdir db/d3d/d4e/d114 0 2024-03-22T06:15:19.653 INFO:tasks.workunit.client.1.smithi117.stdout:6/238: rename d3/d7/da/de/c14 to d3/d40/d46/c50 0 2024-03-22T06:15:19.653 INFO:tasks.workunit.client.1.smithi117.stdout:6/239: truncate d3/d40/d24/f44 879544 0 2024-03-22T06:15:19.653 INFO:tasks.workunit.client.1.smithi117.stdout:6/240: rename d3 to d3/d7/d28/d51 22 2024-03-22T06:15:19.653 INFO:tasks.workunit.client.1.smithi117.stdout:0/242: write d1/d10/f11 [9387653,57270] 0 2024-03-22T06:15:19.653 INFO:tasks.workunit.client.0.smithi006.stdout:0/800: creat db/dc/f115 x:0 0 0 2024-03-22T06:15:19.655 INFO:tasks.workunit.client.1.smithi117.stdout:9/241: truncate d4/d8/f2c 247328 0 2024-03-22T06:15:19.657 INFO:tasks.workunit.client.0.smithi006.stdout:0/801: symlink db/d10/d35/d6c/dad/l116 0 2024-03-22T06:15:19.657 INFO:tasks.workunit.client.0.smithi006.stdout:0/802: fdatasync db/dc/d38/f8e 0 2024-03-22T06:15:19.657 INFO:tasks.workunit.client.1.smithi117.stdout:5/188: dwrite dd/d16/d1a/f3e [0,4194304] 0 2024-03-22T06:15:19.657 INFO:tasks.workunit.client.1.smithi117.stdout:1/194: dwrite d1/fd [0,4194304] 0 2024-03-22T06:15:19.658 INFO:tasks.workunit.client.1.smithi117.stdout:2/198: dwrite d2/d5/d20/d1d/f27 [0,4194304] 0 2024-03-22T06:15:19.659 INFO:tasks.workunit.client.1.smithi117.stdout:4/220: link d0/f11 d0/d19/d1c/f4d 0 2024-03-22T06:15:19.660 INFO:tasks.workunit.client.1.smithi117.stdout:6/241: creat d3/f52 x:0 0 0 2024-03-22T06:15:19.663 INFO:tasks.workunit.client.1.smithi117.stdout:4/221: dread d0/d4/f1b [0,4194304] 0 2024-03-22T06:15:19.664 INFO:tasks.workunit.client.0.smithi006.stdout:0/803: link db/d10/f2f db/f117 0 2024-03-22T06:15:19.664 INFO:tasks.workunit.client.0.smithi006.stdout:0/804: fdatasync db/d10/d35/d6c/f9c 0 2024-03-22T06:15:19.666 INFO:tasks.workunit.client.0.smithi006.stdout:0/805: readlink db/l24 0 2024-03-22T06:15:19.666 INFO:tasks.workunit.client.0.smithi006.stdout:0/806: read - db/d3d/fdb zero size 2024-03-22T06:15:19.668 INFO:tasks.workunit.client.0.smithi006.stdout:0/807: rmdir db/dc/d37 39 2024-03-22T06:15:19.668 INFO:tasks.workunit.client.0.smithi006.stdout:0/808: read - db/d10/d2a/d41/ffd zero size 2024-03-22T06:15:19.669 INFO:tasks.workunit.client.1.smithi117.stdout:5/189: dread dd/d16/d24/f23 [0,4194304] 0 2024-03-22T06:15:19.669 INFO:tasks.workunit.client.1.smithi117.stdout:5/190: chown dd/df 906 1 2024-03-22T06:15:19.671 INFO:tasks.workunit.client.0.smithi006.stdout:0/809: mkdir db/d10/d2a/d41/dff/d118 0 2024-03-22T06:15:19.671 INFO:tasks.workunit.client.0.smithi006.stdout:0/810: write db/dc/d38/fc0 [295428,24797] 0 2024-03-22T06:15:19.672 INFO:tasks.workunit.client.1.smithi117.stdout:7/174: sync 2024-03-22T06:15:19.672 INFO:tasks.workunit.client.1.smithi117.stdout:8/208: sync 2024-03-22T06:15:19.690 INFO:tasks.workunit.client.1.smithi117.stdout:8/209: dwrite d1/d3a/d44/f1d [0,4194304] 0 2024-03-22T06:15:19.690 INFO:tasks.workunit.client.1.smithi117.stdout:8/210: chown d1/d3a/d44/d9/c15 12834122 1 2024-03-22T06:15:19.692 INFO:tasks.workunit.client.1.smithi117.stdout:5/191: dwrite dd/d16/d1a/f37 [0,4194304] 0 2024-03-22T06:15:19.705 INFO:tasks.workunit.client.1.smithi117.stdout:8/211: dwrite d1/d3a/d44/d9/f18 [4194304,4194304] 0 2024-03-22T06:15:19.709 INFO:tasks.workunit.client.1.smithi117.stdout:1/195: mknod d1/d30/d35/c40 0 2024-03-22T06:15:19.718 INFO:tasks.workunit.client.1.smithi117.stdout:2/199: link d2/d5/d13/f19 d2/d5/d8/f36 0 2024-03-22T06:15:19.718 INFO:tasks.workunit.client.1.smithi117.stdout:2/200: chown d2/d5/d13 421023 1 2024-03-22T06:15:19.718 INFO:tasks.workunit.client.1.smithi117.stdout:3/152: rmdir d9/d1a 39 2024-03-22T06:15:19.719 INFO:tasks.workunit.client.1.smithi117.stdout:3/153: write d9/d1a/f2d [491307,91737] 0 2024-03-22T06:15:19.719 INFO:tasks.workunit.client.1.smithi117.stdout:3/154: dread f8 [0,4194304] 0 2024-03-22T06:15:19.720 INFO:tasks.workunit.client.1.smithi117.stdout:3/155: dread - d9/f21 zero size 2024-03-22T06:15:19.720 INFO:tasks.workunit.client.1.smithi117.stdout:3/156: fsync d9/f25 0 2024-03-22T06:15:19.720 INFO:tasks.workunit.client.1.smithi117.stdout:3/157: fsync d9/fa 0 2024-03-22T06:15:19.729 INFO:tasks.workunit.client.0.smithi006.stdout:0/811: getdents db/d3d/d4e 0 2024-03-22T06:15:19.732 INFO:tasks.workunit.client.0.smithi006.stdout:0/812: dread db/d10/d33/fb0 [0,4194304] 0 2024-03-22T06:15:19.732 INFO:tasks.workunit.client.0.smithi006.stdout:0/813: fdatasync db/f96 0 2024-03-22T06:15:19.738 INFO:tasks.workunit.client.0.smithi006.stdout:9/915: dwrite d8/d10/d28/d39/ff9 [0,4194304] 0 2024-03-22T06:15:19.739 INFO:tasks.workunit.client.1.smithi117.stdout:4/222: symlink d0/d19/d34/d43/l4e 0 2024-03-22T06:15:19.744 INFO:tasks.workunit.client.1.smithi117.stdout:7/175: creat d8/d9/d1a/d26/f34 x:0 0 0 2024-03-22T06:15:19.755 INFO:tasks.workunit.client.1.smithi117.stdout:5/192: symlink dd/d16/d1a/l3f 0 2024-03-22T06:15:19.773 INFO:tasks.workunit.client.1.smithi117.stdout:5/193: dwrite dd/d16/d1a/f37 [0,4194304] 0 2024-03-22T06:15:19.798 INFO:tasks.workunit.client.0.smithi006.stdout:0/814: truncate db/dc/d37/f77 5848063 0 2024-03-22T06:15:19.801 INFO:tasks.workunit.client.0.smithi006.stdout:9/916: mknod d8/d10/da6/c134 0 2024-03-22T06:15:19.811 INFO:tasks.workunit.client.1.smithi117.stdout:1/196: mknod d1/d14/c41 0 2024-03-22T06:15:19.811 INFO:tasks.workunit.client.1.smithi117.stdout:1/197: readlink l0 0 2024-03-22T06:15:19.812 INFO:tasks.workunit.client.0.smithi006.stdout:9/917: dread d8/db3/d50/d91/dfe/d12c/fbf [0,4194304] 0 2024-03-22T06:15:19.812 INFO:tasks.workunit.client.0.smithi006.stdout:9/918: chown d8/d1f/l10e 6129 1 2024-03-22T06:15:19.813 INFO:tasks.workunit.client.0.smithi006.stdout:9/919: chown d8/d10/d20/d45/d49/fd6 0 1 2024-03-22T06:15:19.813 INFO:tasks.workunit.client.1.smithi117.stdout:3/158: symlink d9/l38 0 2024-03-22T06:15:19.814 INFO:tasks.workunit.client.1.smithi117.stdout:4/223: rmdir d0/d19/d34/d44 0 2024-03-22T06:15:19.814 INFO:tasks.workunit.client.1.smithi117.stdout:4/224: fsync d0/d19/f1a 0 2024-03-22T06:15:19.816 INFO:tasks.workunit.client.1.smithi117.stdout:7/176: mknod d8/d9/da/c35 0 2024-03-22T06:15:19.818 INFO:tasks.workunit.client.0.smithi006.stdout:9/920: getdents d8/d10/d82/df7/d10b 0 2024-03-22T06:15:19.821 INFO:tasks.workunit.client.1.smithi117.stdout:5/194: creat dd/d16/d1a/d32/f40 x:0 0 0 2024-03-22T06:15:19.822 INFO:tasks.workunit.client.1.smithi117.stdout:1/198: mknod d1/d30/d35/d36/c42 0 2024-03-22T06:15:19.823 INFO:tasks.workunit.client.1.smithi117.stdout:2/201: rename d2/d5/d20/d1d/f27 to d2/d5/d8/d10/f37 0 2024-03-22T06:15:19.824 INFO:tasks.workunit.client.1.smithi117.stdout:7/177: mknod d8/d9/da/d10/d23/d27/d32/c36 0 2024-03-22T06:15:19.824 INFO:tasks.workunit.client.1.smithi117.stdout:5/195: symlink dd/d16/d24/l41 0 2024-03-22T06:15:19.825 INFO:tasks.workunit.client.0.smithi006.stdout:9/921: write d8/d10/d20/d45/d4d/f102 [112701,862] 0 2024-03-22T06:15:19.825 INFO:tasks.workunit.client.0.smithi006.stdout:9/922: truncate d8/d10/d11/f3c 2284102 0 2024-03-22T06:15:19.825 INFO:tasks.workunit.client.0.smithi006.stdout:9/923: fdatasync d8/d10/d23/f40 0 2024-03-22T06:15:19.825 INFO:tasks.workunit.client.0.smithi006.stdout:9/924: fsync d8/d1f/fe5 0 2024-03-22T06:15:19.826 INFO:tasks.workunit.client.0.smithi006.stdout:9/925: chown d8/d10/d47/dbc/fe6 1650224 1 2024-03-22T06:15:19.826 INFO:tasks.workunit.client.1.smithi117.stdout:2/202: mkdir d2/d35/d38 0 2024-03-22T06:15:19.827 INFO:tasks.workunit.client.0.smithi006.stdout:9/926: rmdir d8/db3/d133 39 2024-03-22T06:15:19.827 INFO:tasks.workunit.client.0.smithi006.stdout:9/927: chown d8/d10/d23/f7d 1161183960 1 2024-03-22T06:15:19.827 INFO:tasks.workunit.client.0.smithi006.stdout:9/928: fdatasync f3 0 2024-03-22T06:15:19.827 INFO:tasks.workunit.client.0.smithi006.stdout:9/929: readlink d8/d10/d20/d45/d4d/ld5 0 2024-03-22T06:15:19.828 INFO:tasks.workunit.client.1.smithi117.stdout:5/196: rmdir dd/df/d30 39 2024-03-22T06:15:19.828 INFO:tasks.workunit.client.1.smithi117.stdout:5/197: write dd/df/f38 [39215,33544] 0 2024-03-22T06:15:19.828 INFO:tasks.workunit.client.1.smithi117.stdout:5/198: fsync f2 0 2024-03-22T06:15:19.829 INFO:tasks.workunit.client.1.smithi117.stdout:2/203: mkdir d2/d1b/d39 0 2024-03-22T06:15:19.829 INFO:tasks.workunit.client.1.smithi117.stdout:2/204: chown d2/d5/d8/d10/c29 476844000 1 2024-03-22T06:15:19.830 INFO:tasks.workunit.client.1.smithi117.stdout:3/159: dwrite f8 [0,4194304] 0 2024-03-22T06:15:19.830 INFO:tasks.workunit.client.1.smithi117.stdout:5/199: symlink dd/l42 0 2024-03-22T06:15:19.831 INFO:tasks.workunit.client.1.smithi117.stdout:5/200: stat dd/l1c 0 2024-03-22T06:15:19.831 INFO:tasks.workunit.client.1.smithi117.stdout:7/178: getdents d8/d9/d1a/d26 0 2024-03-22T06:15:19.831 INFO:tasks.workunit.client.1.smithi117.stdout:7/179: read - d8/d9/da/d10/f22 zero size 2024-03-22T06:15:19.831 INFO:tasks.workunit.client.1.smithi117.stdout:7/180: truncate d8/d9/da/f28 220162 0 2024-03-22T06:15:19.831 INFO:tasks.workunit.client.1.smithi117.stdout:7/181: fdatasync d8/f1f 0 2024-03-22T06:15:19.834 INFO:tasks.workunit.client.1.smithi117.stdout:1/199: dread d1/d30/d35/f1a [0,4194304] 0 2024-03-22T06:15:19.836 INFO:tasks.workunit.client.1.smithi117.stdout:3/160: rmdir d9/d1a 39 2024-03-22T06:15:19.838 INFO:tasks.workunit.client.1.smithi117.stdout:7/182: unlink d8/d9/d1a/l21 0 2024-03-22T06:15:19.840 INFO:tasks.workunit.client.1.smithi117.stdout:1/200: truncate d1/d22/d27/f28 4083702 0 2024-03-22T06:15:19.840 INFO:tasks.workunit.client.1.smithi117.stdout:1/201: dread - d1/d1f/d32/d33/f3d zero size 2024-03-22T06:15:19.841 INFO:tasks.workunit.client.1.smithi117.stdout:5/201: dread dd/d16/d1a/f37 [0,4194304] 0 2024-03-22T06:15:19.841 INFO:tasks.workunit.client.1.smithi117.stdout:5/202: readlink dd/df/l3d 0 2024-03-22T06:15:19.841 INFO:tasks.workunit.client.1.smithi117.stdout:3/161: truncate d9/d1a/f1e 3270074 0 2024-03-22T06:15:19.842 INFO:tasks.workunit.client.1.smithi117.stdout:3/162: chown d9/f11 283097 1 2024-03-22T06:15:19.847 INFO:tasks.workunit.client.1.smithi117.stdout:7/183: dread d8/f16 [0,4194304] 0 2024-03-22T06:15:19.847 INFO:tasks.workunit.client.1.smithi117.stdout:7/184: write d8/d9/f1e [1143562,74901] 0 2024-03-22T06:15:19.849 INFO:tasks.workunit.client.1.smithi117.stdout:7/185: mkdir d8/d9/d37 0 2024-03-22T06:15:19.849 INFO:tasks.workunit.client.1.smithi117.stdout:7/186: truncate d8/d9/d1a/d26/f2c 8674613 0 2024-03-22T06:15:19.851 INFO:tasks.workunit.client.1.smithi117.stdout:1/202: write d1/fd [647691,56790] 0 2024-03-22T06:15:19.858 INFO:tasks.workunit.client.1.smithi117.stdout:7/187: write f7 [2462324,105306] 0 2024-03-22T06:15:19.858 INFO:tasks.workunit.client.1.smithi117.stdout:7/188: creat d8/d9/da/d10/d23/f38 x:0 0 0 2024-03-22T06:15:19.858 INFO:tasks.workunit.client.1.smithi117.stdout:7/189: chown d8 2 1 2024-03-22T06:15:19.858 INFO:tasks.workunit.client.1.smithi117.stdout:7/190: write d8/d9/da/d10/f22 [548876,99121] 0 2024-03-22T06:15:19.860 INFO:tasks.workunit.client.1.smithi117.stdout:5/203: dwrite dd/d16/d1a/d32/f40 [0,4194304] 0 2024-03-22T06:15:19.885 INFO:tasks.workunit.client.0.smithi006.stdout:0/815: rename db/dc/c27 to db/d10/d35/d7b/c119 0 2024-03-22T06:15:19.888 INFO:tasks.workunit.client.0.smithi006.stdout:9/930: rename d8/d10/d20/d45/d49/fd6 to d8/d10/d28/d39/f135 0 2024-03-22T06:15:19.888 INFO:tasks.workunit.client.0.smithi006.stdout:9/931: fdatasync d8/d10/d82/fc5 0 2024-03-22T06:15:19.888 INFO:tasks.workunit.client.0.smithi006.stdout:9/932: fdatasync d8/db3/d50/d91/dfe/d12c/f3a 0 2024-03-22T06:15:19.890 INFO:tasks.workunit.client.0.smithi006.stdout:9/933: unlink d8/d10/d20/d45/d49/fc7 0 2024-03-22T06:15:19.893 INFO:tasks.workunit.client.0.smithi006.stdout:9/934: link d8/db3/d50/d91/dfe/d12c/c98 d8/d10/d28/dc2/c136 0 2024-03-22T06:15:19.893 INFO:tasks.workunit.client.0.smithi006.stdout:7/890: sync 2024-03-22T06:15:19.893 INFO:tasks.workunit.client.0.smithi006.stdout:6/892: sync 2024-03-22T06:15:19.893 INFO:tasks.workunit.client.0.smithi006.stdout:1/902: sync 2024-03-22T06:15:19.894 INFO:tasks.workunit.client.0.smithi006.stdout:1/903: fsync d8/d9/d13/d39/fc9 0 2024-03-22T06:15:19.895 INFO:tasks.workunit.client.1.smithi117.stdout:2/205: rename d2/d5/d8/d10 to d2/d5/d20/d1d/d3a 0 2024-03-22T06:15:19.896 INFO:tasks.workunit.client.0.smithi006.stdout:9/935: symlink d8/d10/d28/d39/d112/l137 0 2024-03-22T06:15:19.896 INFO:tasks.workunit.client.0.smithi006.stdout:9/936: write d8/db3/d50/d91/dfe/d12c/ff8 [376450,88859] 0 2024-03-22T06:15:19.899 INFO:tasks.workunit.client.1.smithi117.stdout:5/204: rename dd/d16/d1a/c21 to dd/d16/d24/c43 0 2024-03-22T06:15:19.900 INFO:tasks.workunit.client.0.smithi006.stdout:7/891: mknod d5/d1e/d29/d1f/d2d/df7/c124 0 2024-03-22T06:15:19.901 INFO:tasks.workunit.client.0.smithi006.stdout:9/937: mknod d8/d10/d28/dba/c138 0 2024-03-22T06:15:19.902 INFO:tasks.workunit.client.1.smithi117.stdout:5/205: symlink dd/df/d1e/l44 0 2024-03-22T06:15:19.902 INFO:tasks.workunit.client.0.smithi006.stdout:6/893: mknod d6/d77/c10f 0 2024-03-22T06:15:19.904 INFO:tasks.workunit.client.0.smithi006.stdout:7/892: mkdir d5/d1e/d29/d66/d91/ddc/dfa/d125 0 2024-03-22T06:15:19.904 INFO:tasks.workunit.client.0.smithi006.stdout:7/893: write d5/d1e/d29/d59/f8e [465000,10372] 0 2024-03-22T06:15:19.904 INFO:tasks.workunit.client.1.smithi117.stdout:2/206: dread d2/d5/d13/f19 [0,4194304] 0 2024-03-22T06:15:19.904 INFO:tasks.workunit.client.1.smithi117.stdout:2/207: fsync d2/d5/d20/d1d/d3a/f37 0 2024-03-22T06:15:19.905 INFO:tasks.workunit.client.1.smithi117.stdout:5/206: creat dd/df/d30/f45 x:0 0 0 2024-03-22T06:15:19.905 INFO:tasks.workunit.client.1.smithi117.stdout:2/208: read d2/f26 [191337,20869] 0 2024-03-22T06:15:19.905 INFO:tasks.workunit.client.1.smithi117.stdout:2/209: dread - d2/d5/f33 zero size 2024-03-22T06:15:19.906 INFO:tasks.workunit.client.0.smithi006.stdout:9/938: creat d8/d1f/f139 x:0 0 0 2024-03-22T06:15:19.906 INFO:tasks.workunit.client.0.smithi006.stdout:9/939: creat d8/d10/d20/d45/d49/f13a x:0 0 0 2024-03-22T06:15:19.906 INFO:tasks.workunit.client.0.smithi006.stdout:6/894: dread - d6/d1d/d99/fe4 zero size 2024-03-22T06:15:19.908 INFO:tasks.workunit.client.1.smithi117.stdout:5/207: mknod dd/d16/d24/d36/c46 0 2024-03-22T06:15:19.908 INFO:tasks.workunit.client.1.smithi117.stdout:5/208: fdatasync dd/d16/d1a/f3e 0 2024-03-22T06:15:19.910 INFO:tasks.workunit.client.0.smithi006.stdout:7/894: link d5/d1e/d29/d66/d91/ffc d5/d1e/d29/f126 0 2024-03-22T06:15:19.910 INFO:tasks.workunit.client.0.smithi006.stdout:7/895: write d5/d1e/d29/d1f/d2d/d55/fa8 [591939,27443] 0 2024-03-22T06:15:19.914 INFO:tasks.workunit.client.1.smithi117.stdout:5/209: unlink f2 0 2024-03-22T06:15:19.916 INFO:tasks.workunit.client.0.smithi006.stdout:9/940: write d8/d10/d28/d39/f93 [2728793,80582] 0 2024-03-22T06:15:19.916 INFO:tasks.workunit.client.0.smithi006.stdout:9/941: truncate d8/d10/da6/fb4 875471 0 2024-03-22T06:15:19.916 INFO:tasks.workunit.client.0.smithi006.stdout:9/942: readlink d8/d1f/l10e 0 2024-03-22T06:15:19.916 INFO:tasks.workunit.client.0.smithi006.stdout:9/943: read - d8/d10/d82/df7/f124 zero size 2024-03-22T06:15:19.916 INFO:tasks.workunit.client.0.smithi006.stdout:0/816: dwrite db/d10/d33/f91 [0,4194304] 0 2024-03-22T06:15:19.916 INFO:tasks.workunit.client.0.smithi006.stdout:9/944: write d8/d10/d20/d45/d49/f8c [260351,72082] 0 2024-03-22T06:15:19.916 INFO:tasks.workunit.client.0.smithi006.stdout:9/945: stat d8/db3/d50/d91/dfe/d12c 0 2024-03-22T06:15:19.917 INFO:tasks.workunit.client.1.smithi117.stdout:1/203: link d1/d22/d27/f28 d1/f43 0 2024-03-22T06:15:19.918 INFO:tasks.workunit.client.0.smithi006.stdout:7/896: rename d5/d1e/d3f/f48 to d5/d1e/d63/d85/dac/db3/f127 0 2024-03-22T06:15:19.921 INFO:tasks.workunit.client.1.smithi117.stdout:1/204: mkdir d1/d30/d44 0 2024-03-22T06:15:19.922 INFO:tasks.workunit.client.0.smithi006.stdout:6/895: dread d6/f42 [0,4194304] 0 2024-03-22T06:15:19.922 INFO:tasks.workunit.client.0.smithi006.stdout:6/896: fsync d6/d24/f54 0 2024-03-22T06:15:19.922 INFO:tasks.workunit.client.0.smithi006.stdout:0/817: creat db/d10/f11a x:0 0 0 2024-03-22T06:15:19.922 INFO:tasks.workunit.client.0.smithi006.stdout:6/897: dread - d6/d1d/d99/ff1 zero size 2024-03-22T06:15:19.925 INFO:tasks.workunit.client.0.smithi006.stdout:7/897: link d5/d1e/d29/d1f/d2d/df7/f122 d5/d1e/d29/d1f/d2d/df7/ddd/f128 0 2024-03-22T06:15:19.925 INFO:tasks.workunit.client.0.smithi006.stdout:7/898: chown d5/d1e/d29/d59/d5c/c79 11401 1 2024-03-22T06:15:19.928 INFO:tasks.workunit.client.0.smithi006.stdout:1/904: dwrite d8/d9/d13/d39/d60/fb2 [0,4194304] 0 2024-03-22T06:15:19.928 INFO:tasks.workunit.client.0.smithi006.stdout:1/905: write d8/d71/dbb/f112 [956815,35356] 0 2024-03-22T06:15:19.928 INFO:tasks.workunit.client.0.smithi006.stdout:1/906: stat d8/d9/d10/d7c/ce5 0 2024-03-22T06:15:19.928 INFO:tasks.workunit.client.0.smithi006.stdout:1/907: truncate d8/d9/d3a/d18/f61 494277 0 2024-03-22T06:15:19.928 INFO:tasks.workunit.client.1.smithi117.stdout:2/210: dwrite d2/d5/d8/f36 [0,4194304] 0 2024-03-22T06:15:19.929 INFO:tasks.workunit.client.1.smithi117.stdout:2/211: write d2/d5/d20/f1f [2417154,42388] 0 2024-03-22T06:15:19.929 INFO:tasks.workunit.client.1.smithi117.stdout:2/212: write d2/d5/d20/f1f [3719994,37037] 0 2024-03-22T06:15:19.929 INFO:tasks.workunit.client.1.smithi117.stdout:1/205: unlink d1/d22/d27/l2c 0 2024-03-22T06:15:19.929 INFO:tasks.workunit.client.1.smithi117.stdout:2/213: dread d2/d5/d13/f30 [0,4194304] 0 2024-03-22T06:15:19.930 INFO:tasks.workunit.client.0.smithi006.stdout:0/818: creat db/d10/d35/d7b/f11b x:0 0 0 2024-03-22T06:15:19.931 INFO:tasks.workunit.client.0.smithi006.stdout:9/946: dread d8/db3/d50/f8e [0,4194304] 0 2024-03-22T06:15:19.931 INFO:tasks.workunit.client.0.smithi006.stdout:9/947: write d8/ff0 [618982,51352] 0 2024-03-22T06:15:19.931 INFO:tasks.workunit.client.0.smithi006.stdout:9/948: creat d8/db3/d50/f13b x:0 0 0 2024-03-22T06:15:19.933 INFO:tasks.workunit.client.0.smithi006.stdout:6/898: creat d6/d1b/dae/dbc/d105/f110 x:0 0 0 2024-03-22T06:15:19.937 INFO:tasks.workunit.client.0.smithi006.stdout:1/908: mknod d8/d9/d13/d39/d94/db5/df1/c12d 0 2024-03-22T06:15:19.940 INFO:tasks.workunit.client.0.smithi006.stdout:0/819: getdents db/dc/d38 0 2024-03-22T06:15:19.943 INFO:tasks.workunit.client.0.smithi006.stdout:0/820: getdents db/d10/d33/dce 0 2024-03-22T06:15:19.944 INFO:tasks.workunit.client.1.smithi117.stdout:2/214: dread d2/d1b/f28 [0,4194304] 0 2024-03-22T06:15:19.944 INFO:tasks.workunit.client.1.smithi117.stdout:2/215: rename d2/d35 to d2/d35/d3b 22 2024-03-22T06:15:19.945 INFO:tasks.workunit.client.0.smithi006.stdout:6/899: rename d6/d1d/dc6 to d6/d111 0 2024-03-22T06:15:19.945 INFO:tasks.workunit.client.0.smithi006.stdout:6/900: fdatasync d6/d1b/d2c/f72 0 2024-03-22T06:15:19.945 INFO:tasks.workunit.client.0.smithi006.stdout:6/901: chown d6/d1d/l7a 431 1 2024-03-22T06:15:19.945 INFO:tasks.workunit.client.1.smithi117.stdout:7/191: truncate d8/d9/da/d10/f22 409038 0 2024-03-22T06:15:19.945 INFO:tasks.workunit.client.1.smithi117.stdout:7/192: readlink d8/d9/l18 0 2024-03-22T06:15:19.946 INFO:tasks.workunit.client.1.smithi117.stdout:7/193: readlink d8/d9/l12 0 2024-03-22T06:15:19.946 INFO:tasks.workunit.client.0.smithi006.stdout:0/821: mkdir db/d10/d35/d7b/da1/da6/dc4/d11c 0 2024-03-22T06:15:19.948 INFO:tasks.workunit.client.1.smithi117.stdout:2/216: mknod d2/d5/d20/d1d/d3a/c3c 0 2024-03-22T06:15:19.948 INFO:tasks.workunit.client.1.smithi117.stdout:2/217: stat d2/d5/d20 0 2024-03-22T06:15:19.950 INFO:tasks.workunit.client.0.smithi006.stdout:9/949: read d8/d10/d20/d45/f66 [258239,94058] 0 2024-03-22T06:15:19.950 INFO:tasks.workunit.client.0.smithi006.stdout:0/822: truncate db/d10/d2a/f34 1327535 0 2024-03-22T06:15:19.951 INFO:tasks.workunit.client.0.smithi006.stdout:6/902: symlink d6/d1b/dae/l112 0 2024-03-22T06:15:19.951 INFO:tasks.workunit.client.0.smithi006.stdout:6/903: write d6/d1d/d5c/d81/f65 [1729774,8338] 0 2024-03-22T06:15:19.951 INFO:tasks.workunit.client.0.smithi006.stdout:6/904: chown d6/c10e 217965536 1 2024-03-22T06:15:19.954 INFO:tasks.workunit.client.0.smithi006.stdout:9/950: unlink d8/d10/d28/fa7 0 2024-03-22T06:15:19.958 INFO:tasks.workunit.client.0.smithi006.stdout:0/823: mkdir db/d10/d2a/d41/d11d 0 2024-03-22T06:15:19.958 INFO:tasks.workunit.client.0.smithi006.stdout:0/824: write db/dc/d38/f94 [978104,106440] 0 2024-03-22T06:15:19.958 INFO:tasks.workunit.client.0.smithi006.stdout:0/825: creat db/d10/d2a/f11e x:0 0 0 2024-03-22T06:15:19.958 INFO:tasks.workunit.client.0.smithi006.stdout:0/826: write db/dc/faa [525910,97708] 0 2024-03-22T06:15:19.966 INFO:tasks.workunit.client.1.smithi117.stdout:8/212: rmdir d1 39 2024-03-22T06:15:19.967 INFO:tasks.workunit.client.1.smithi117.stdout:9/242: mkdir d4/d47 0 2024-03-22T06:15:19.967 INFO:tasks.workunit.client.1.smithi117.stdout:9/243: fsync d4/d8/f13 0 2024-03-22T06:15:19.967 INFO:tasks.workunit.client.1.smithi117.stdout:9/244: fdatasync d4/d7/f2d 0 2024-03-22T06:15:19.969 INFO:tasks.workunit.client.1.smithi117.stdout:5/210: dwrite dd/df/d1e/f2e [0,4194304] 0 2024-03-22T06:15:19.969 INFO:tasks.workunit.client.1.smithi117.stdout:9/245: rename d4/d7/d27/c29 to d4/d7/d10/c48 0 2024-03-22T06:15:19.969 INFO:tasks.workunit.client.1.smithi117.stdout:9/246: fdatasync d4/d8/f41 0 2024-03-22T06:15:19.969 INFO:tasks.workunit.client.1.smithi117.stdout:9/247: chown d4/d7/d10/f42 125942 1 2024-03-22T06:15:19.969 INFO:tasks.workunit.client.1.smithi117.stdout:9/248: symlink d4/d8/l49 0 2024-03-22T06:15:19.970 INFO:tasks.workunit.client.1.smithi117.stdout:9/249: chown d4/d7 22931 1 2024-03-22T06:15:19.970 INFO:tasks.workunit.client.1.smithi117.stdout:9/250: write d4/d8/f13 [45572,81480] 0 2024-03-22T06:15:19.981 INFO:tasks.workunit.client.1.smithi117.stdout:8/213: write d1/d3a/d44/d9/f13 [11612849,65563] 0 2024-03-22T06:15:19.982 INFO:tasks.workunit.client.1.smithi117.stdout:5/211: dread dd/d16/d1a/f3e [0,4194304] 0 2024-03-22T06:15:19.982 INFO:tasks.workunit.client.1.smithi117.stdout:8/214: mknod d1/d1f/c47 0 2024-03-22T06:15:19.983 INFO:tasks.workunit.client.1.smithi117.stdout:8/215: truncate d1/d3a/d44/d11/f2a 3615480 0 2024-03-22T06:15:19.984 INFO:tasks.workunit.client.1.smithi117.stdout:1/206: dwrite d1/fc [0,4194304] 0 2024-03-22T06:15:19.985 INFO:tasks.workunit.client.1.smithi117.stdout:2/218: dwrite d2/f26 [0,4194304] 0 2024-03-22T06:15:19.985 INFO:tasks.workunit.client.1.smithi117.stdout:5/212: dread dd/d16/d24/f34 [0,4194304] 0 2024-03-22T06:15:19.985 INFO:tasks.workunit.client.1.smithi117.stdout:5/213: write dd/df/f3a [1179664,18820] 0 2024-03-22T06:15:19.985 INFO:tasks.workunit.client.1.smithi117.stdout:5/214: write dd/d16/d24/d36/f39 [3020212,76669] 0 2024-03-22T06:15:19.986 INFO:tasks.workunit.client.1.smithi117.stdout:1/207: symlink d1/d3c/l45 0 2024-03-22T06:15:19.986 INFO:tasks.workunit.client.1.smithi117.stdout:1/208: chown d1/d1f/f3e 10 1 2024-03-22T06:15:19.987 INFO:tasks.workunit.client.1.smithi117.stdout:2/219: rmdir d2/d1b 39 2024-03-22T06:15:19.990 INFO:tasks.workunit.client.1.smithi117.stdout:1/209: rename d1/d30/d35/l17 to d1/d1f/d32/l46 0 2024-03-22T06:15:19.990 INFO:tasks.workunit.client.1.smithi117.stdout:1/210: write d1/d30/d35/f1a [1547053,125525] 0 2024-03-22T06:15:19.990 INFO:tasks.workunit.client.1.smithi117.stdout:1/211: write d1/fd [4088512,26865] 0 2024-03-22T06:15:19.990 INFO:tasks.workunit.client.1.smithi117.stdout:1/212: stat d1/d1f/d32/d33/f3d 0 2024-03-22T06:15:19.990 INFO:tasks.workunit.client.1.smithi117.stdout:1/213: dread d1/d22/d27/f34 [0,4194304] 0 2024-03-22T06:15:19.992 INFO:tasks.workunit.client.1.smithi117.stdout:1/214: unlink d1/f25 0 2024-03-22T06:15:19.994 INFO:tasks.workunit.client.0.smithi006.stdout:6/905: dwrite d6/d1d/d5c/d81/f7e [0,4194304] 0 2024-03-22T06:15:19.994 INFO:tasks.workunit.client.0.smithi006.stdout:6/906: chown d6/d1b/d2c/f59 3 1 2024-03-22T06:15:19.994 INFO:tasks.workunit.client.0.smithi006.stdout:0/827: dwrite db/dc/d38/d50/f100 [0,4194304] 0 2024-03-22T06:15:19.994 INFO:tasks.workunit.client.0.smithi006.stdout:0/828: readlink db/l8f 0 2024-03-22T06:15:19.994 INFO:tasks.workunit.client.0.smithi006.stdout:0/829: write db/d10/d2a/d41/ffd [229730,57419] 0 2024-03-22T06:15:19.994 INFO:tasks.workunit.client.1.smithi117.stdout:9/251: dwrite d4/d7/d10/f28 [0,4194304] 0 2024-03-22T06:15:19.995 INFO:tasks.workunit.client.0.smithi006.stdout:1/909: dwrite d8/d71/dcd/ff6 [0,4194304] 0 2024-03-22T06:15:19.996 INFO:tasks.workunit.client.0.smithi006.stdout:6/907: mkdir d6/d1d/d5c/d81/d8a/dac/d113 0 2024-03-22T06:15:19.996 INFO:tasks.workunit.client.0.smithi006.stdout:7/899: dwrite d5/d1e/d29/d66/fdb [0,4194304] 0 2024-03-22T06:15:19.996 INFO:tasks.workunit.client.0.smithi006.stdout:6/908: getdents d6/d7c/d92 0 2024-03-22T06:15:19.999 INFO:tasks.workunit.client.1.smithi117.stdout:5/215: dread dd/f29 [0,4194304] 0 2024-03-22T06:15:19.999 INFO:tasks.workunit.client.1.smithi117.stdout:5/216: chown dd/df/d30/d35/c18 20 1 2024-03-22T06:15:19.999 INFO:tasks.workunit.client.1.smithi117.stdout:5/217: truncate dd/d16/d24/d36/f26 1301377 0 2024-03-22T06:15:19.999 INFO:tasks.workunit.client.1.smithi117.stdout:5/218: chown dd/df/d30/d35/f27 6759 1 2024-03-22T06:15:20.003 INFO:tasks.workunit.client.0.smithi006.stdout:0/830: dread db/dc/d38/fc1 [0,4194304] 0 2024-03-22T06:15:20.004 INFO:tasks.workunit.client.0.smithi006.stdout:1/910: unlink d8/d9/d13/d39/d60/d8f/d10a/d44/d80/f88 0 2024-03-22T06:15:20.004 INFO:tasks.workunit.client.0.smithi006.stdout:1/911: stat d8/cf 0 2024-03-22T06:15:20.004 INFO:tasks.workunit.client.0.smithi006.stdout:1/912: creat d8/d9/d13/d39/d60/d8f/dea/f12e x:0 0 0 2024-03-22T06:15:20.008 INFO:tasks.workunit.client.1.smithi117.stdout:9/252: dread d4/d7/f1b [0,4194304] 0 2024-03-22T06:15:20.008 INFO:tasks.workunit.client.1.smithi117.stdout:9/253: truncate d4/d8/f2c 518555 0 2024-03-22T06:15:20.008 INFO:tasks.workunit.client.0.smithi006.stdout:6/909: write d6/d1b/d2c/f74 [2628121,61205] 0 2024-03-22T06:15:20.010 INFO:tasks.workunit.client.0.smithi006.stdout:7/900: rename d5/d1e/d29/cce to d5/d1e/d29/d10/d6f/c129 0 2024-03-22T06:15:20.010 INFO:tasks.workunit.client.0.smithi006.stdout:7/901: readlink d5/d1e/d29/d1f/l27 0 2024-03-22T06:15:20.011 INFO:tasks.workunit.client.0.smithi006.stdout:0/831: mknod db/d10/d35/d7b/da1/c11f 0 2024-03-22T06:15:20.015 INFO:tasks.workunit.client.0.smithi006.stdout:1/913: symlink d8/d9/d13/l12f 0 2024-03-22T06:15:20.017 INFO:tasks.workunit.client.1.smithi117.stdout:2/220: dwrite d2/f26 [0,4194304] 0 2024-03-22T06:15:20.017 INFO:tasks.workunit.client.1.smithi117.stdout:1/215: dwrite d1/d1f/d32/d33/f3d [0,4194304] 0 2024-03-22T06:15:20.018 INFO:tasks.workunit.client.1.smithi117.stdout:5/219: dwrite dd/df/d30/f45 [0,4194304] 0 2024-03-22T06:15:20.019 INFO:tasks.workunit.client.1.smithi117.stdout:2/221: dread d2/d5/d13/f30 [0,4194304] 0 2024-03-22T06:15:20.019 INFO:tasks.workunit.client.0.smithi006.stdout:6/910: mknod d6/d1d/c114 0 2024-03-22T06:15:20.021 INFO:tasks.workunit.client.1.smithi117.stdout:5/220: read dd/df/f3a [788229,128162] 0 2024-03-22T06:15:20.022 INFO:tasks.workunit.client.0.smithi006.stdout:0/832: creat db/d10/df7/f120 x:0 0 0 2024-03-22T06:15:20.023 INFO:tasks.workunit.client.0.smithi006.stdout:7/902: dread d5/d1e/d63/d85/dca/f101 [0,4194304] 0 2024-03-22T06:15:20.024 INFO:tasks.workunit.client.0.smithi006.stdout:6/911: mknod d6/d24/d44/c115 0 2024-03-22T06:15:20.027 INFO:tasks.workunit.client.0.smithi006.stdout:0/833: dread db/d10/d33/f42 [0,4194304] 0 2024-03-22T06:15:20.028 INFO:tasks.workunit.client.0.smithi006.stdout:7/903: rename d5/d1e/d3f/fb6 to d5/d1e/d63/f12a 0 2024-03-22T06:15:20.031 INFO:tasks.workunit.client.0.smithi006.stdout:6/912: creat d6/d1d/d99/f116 x:0 0 0 2024-03-22T06:15:20.032 INFO:tasks.workunit.client.1.smithi117.stdout:1/216: mknod d1/d14/c47 0 2024-03-22T06:15:20.033 INFO:tasks.workunit.client.0.smithi006.stdout:0/834: link db/fe9 db/d87/ddd/f121 0 2024-03-22T06:15:20.034 INFO:tasks.workunit.client.0.smithi006.stdout:0/835: getdents db/d10/d33/dce 0 2024-03-22T06:15:20.038 INFO:tasks.workunit.client.0.smithi006.stdout:7/904: mkdir d5/d1e/d29/d66/d91/ddc/dfa/d121/d12b 0 2024-03-22T06:15:20.038 INFO:tasks.workunit.client.0.smithi006.stdout:7/905: chown d5/d1e/d29/d1f/d2d/c35 779143 1 2024-03-22T06:15:20.041 INFO:tasks.workunit.client.1.smithi117.stdout:9/254: dwrite d4/d7/f24 [0,4194304] 0 2024-03-22T06:15:20.041 INFO:tasks.workunit.client.0.smithi006.stdout:6/913: creat d6/d1d/d5c/d81/dab/f117 x:0 0 0 2024-03-22T06:15:20.041 INFO:tasks.workunit.client.0.smithi006.stdout:1/914: dwrite d8/d9/d3a/d18/f1b [4194304,4194304] 0 2024-03-22T06:15:20.041 INFO:tasks.workunit.client.0.smithi006.stdout:1/915: write d8/d9/d13/d39/d60/d8f/dea/f12e [766945,61018] 0 2024-03-22T06:15:20.042 INFO:tasks.workunit.client.0.smithi006.stdout:0/836: mkdir db/d10/d35/d7b/da1/da6/d122 0 2024-03-22T06:15:20.046 INFO:tasks.workunit.client.0.smithi006.stdout:6/914: getdents d6/d1d/d5c/d81/d8a 0 2024-03-22T06:15:20.050 INFO:tasks.workunit.client.0.smithi006.stdout:6/915: rename d6/d1d/d5c/dc4 to d6/d1d/d32/d118 0 2024-03-22T06:15:20.050 INFO:tasks.workunit.client.0.smithi006.stdout:6/916: stat d6/d77/c10f 0 2024-03-22T06:15:20.052 INFO:tasks.workunit.client.0.smithi006.stdout:1/916: dread d8/fac [0,4194304] 0 2024-03-22T06:15:20.053 INFO:tasks.workunit.client.1.smithi117.stdout:2/222: mknod d2/d5/d8/c3d 0 2024-03-22T06:15:20.056 INFO:tasks.workunit.client.1.smithi117.stdout:5/221: creat dd/d16/d24/d36/f47 x:0 0 0 2024-03-22T06:15:20.057 INFO:tasks.workunit.client.1.smithi117.stdout:5/222: readlink dd/df/d30/l3c 0 2024-03-22T06:15:20.057 INFO:tasks.workunit.client.1.smithi117.stdout:5/223: write dd/d16/d24/d36/f47 [603516,107080] 0 2024-03-22T06:15:20.057 INFO:tasks.workunit.client.1.smithi117.stdout:5/224: chown dd/df/f2c 1283099 1 2024-03-22T06:15:20.057 INFO:tasks.workunit.client.0.smithi006.stdout:7/906: dwrite d5/d1e/d29/d1f/d2d/f51 [0,4194304] 0 2024-03-22T06:15:20.057 INFO:tasks.workunit.client.0.smithi006.stdout:6/917: symlink d6/d1d/d99/dcd/l119 0 2024-03-22T06:15:20.058 INFO:tasks.workunit.client.1.smithi117.stdout:1/217: symlink d1/d30/d35/l48 0 2024-03-22T06:15:20.058 INFO:tasks.workunit.client.1.smithi117.stdout:1/218: read d1/fd [2053799,86581] 0 2024-03-22T06:15:20.063 INFO:tasks.workunit.client.0.smithi006.stdout:9/951: truncate d8/ff0 295440 0 2024-03-22T06:15:20.064 INFO:tasks.workunit.client.1.smithi117.stdout:9/255: symlink d4/d8/d3f/l4a 0 2024-03-22T06:15:20.065 INFO:tasks.workunit.client.1.smithi117.stdout:6/242: sync 2024-03-22T06:15:20.066 INFO:tasks.workunit.client.1.smithi117.stdout:0/243: sync 2024-03-22T06:15:20.066 INFO:tasks.workunit.client.1.smithi117.stdout:6/243: truncate d3/d7/da/de/f47 529730 0 2024-03-22T06:15:20.066 INFO:tasks.workunit.client.1.smithi117.stdout:2/223: creat d2/d5/d20/d1d/d3a/f3e x:0 0 0 2024-03-22T06:15:20.069 INFO:tasks.workunit.client.1.smithi117.stdout:5/225: creat dd/d16/f48 x:0 0 0 2024-03-22T06:15:20.072 INFO:tasks.workunit.client.1.smithi117.stdout:1/219: link d1/d30/d35/l48 d1/d1f/l49 0 2024-03-22T06:15:20.072 INFO:tasks.workunit.client.1.smithi117.stdout:1/220: chown d1/d3c 2527 1 2024-03-22T06:15:20.074 INFO:tasks.workunit.client.1.smithi117.stdout:2/224: write d2/d5/d20/d1d/d3a/f37 [970533,7067] 0 2024-03-22T06:15:20.075 INFO:tasks.workunit.client.1.smithi117.stdout:2/225: write d2/d5/f33 [57446,809] 0 2024-03-22T06:15:20.075 INFO:tasks.workunit.client.1.smithi117.stdout:0/244: creat d1/d10/d18/d26/f58 x:0 0 0 2024-03-22T06:15:20.075 INFO:tasks.workunit.client.1.smithi117.stdout:4/225: sync 2024-03-22T06:15:20.077 INFO:tasks.workunit.client.1.smithi117.stdout:6/244: mkdir d3/d40/d24/d2d/d53 0 2024-03-22T06:15:20.080 INFO:tasks.workunit.client.0.smithi006.stdout:1/917: mkdir d8/d9/d3a/d18/dd2/d130 0 2024-03-22T06:15:20.080 INFO:tasks.workunit.client.0.smithi006.stdout:1/918: read - d8/d9/d13/d39/d60/d8f/d10a/d59/db1/ff8 zero size 2024-03-22T06:15:20.080 INFO:tasks.workunit.client.0.smithi006.stdout:1/919: fsync d8/d9/d13/d24/d32/dc7/fe8 0 2024-03-22T06:15:20.081 INFO:tasks.workunit.client.1.smithi117.stdout:5/226: getdents dd/df/d30/d35 0 2024-03-22T06:15:20.081 INFO:tasks.workunit.client.1.smithi117.stdout:8/216: dread d1/d3a/d44/d11/f2a [0,4194304] 0 2024-03-22T06:15:20.089 INFO:tasks.workunit.client.0.smithi006.stdout:1/920: dread d8/d9/d3a/f92 [0,4194304] 0 2024-03-22T06:15:20.089 INFO:tasks.workunit.client.0.smithi006.stdout:1/921: chown d8/d9/d13/d39/d60/d8f/dea/f12e 0 1 2024-03-22T06:15:20.089 INFO:tasks.workunit.client.0.smithi006.stdout:1/922: dread - d8/d9/d13/f10b zero size 2024-03-22T06:15:20.090 INFO:tasks.workunit.client.1.smithi117.stdout:1/221: rename d1/l10 to d1/d22/d27/l4a 0 2024-03-22T06:15:20.091 INFO:tasks.workunit.client.0.smithi006.stdout:7/907: creat d5/d1e/d29/d66/d91/f12c x:0 0 0 2024-03-22T06:15:20.092 INFO:tasks.workunit.client.1.smithi117.stdout:0/245: mkdir d1/d3/d2b/d3e/de/d1f/d59 0 2024-03-22T06:15:20.093 INFO:tasks.workunit.client.0.smithi006.stdout:6/918: symlink d6/d1d/d5c/d81/d8a/dac/d113/l11a 0 2024-03-22T06:15:20.095 INFO:tasks.workunit.client.1.smithi117.stdout:4/226: truncate d0/f11 2432727 0 2024-03-22T06:15:20.099 INFO:tasks.workunit.client.1.smithi117.stdout:6/245: link d3/d7/l25 d3/d40/d24/l54 0 2024-03-22T06:15:20.099 INFO:tasks.workunit.client.1.smithi117.stdout:6/246: fsync d3/d7/da/de/f47 0 2024-03-22T06:15:20.102 INFO:tasks.workunit.client.1.smithi117.stdout:5/227: symlink dd/l49 0 2024-03-22T06:15:20.102 INFO:tasks.workunit.client.1.smithi117.stdout:5/228: chown dd/df/f2c 966973391 1 2024-03-22T06:15:20.104 INFO:tasks.workunit.client.1.smithi117.stdout:8/217: mkdir d1/d48 0 2024-03-22T06:15:20.104 INFO:tasks.workunit.client.1.smithi117.stdout:8/218: chown d1/d3a/d44/d11/c35 560 1 2024-03-22T06:15:20.105 INFO:tasks.workunit.client.0.smithi006.stdout:7/908: mkdir d5/d1e/d29/d66/d91/d12d 0 2024-03-22T06:15:20.108 INFO:tasks.workunit.client.0.smithi006.stdout:1/923: dwrite d8/d9/d13/d39/d60/d8f/d10a/d44/f110 [0,4194304] 0 2024-03-22T06:15:20.109 INFO:tasks.workunit.client.0.smithi006.stdout:9/952: getdents d8/d10/d20/d45/d4d/d11d 0 2024-03-22T06:15:20.109 INFO:tasks.workunit.client.0.smithi006.stdout:6/919: dwrite d6/d24/d44/f90 [0,4194304] 0 2024-03-22T06:15:20.112 INFO:tasks.workunit.client.1.smithi117.stdout:1/222: creat d1/d22/f4b x:0 0 0 2024-03-22T06:15:20.112 INFO:tasks.workunit.client.1.smithi117.stdout:1/223: stat d1/d30 0 2024-03-22T06:15:20.113 INFO:tasks.workunit.client.0.smithi006.stdout:7/909: mknod d5/d1e/d3f/c12e 0 2024-03-22T06:15:20.115 INFO:tasks.workunit.client.1.smithi117.stdout:0/246: mknod d1/d10/d18/c5a 0 2024-03-22T06:15:20.115 INFO:tasks.workunit.client.1.smithi117.stdout:0/247: fsync d1/d3/d2b/d3e/d3f/f45 0 2024-03-22T06:15:20.116 INFO:tasks.workunit.client.0.smithi006.stdout:1/924: creat d8/d9/d13/d24/d32/dc0/f131 x:0 0 0 2024-03-22T06:15:20.118 INFO:tasks.workunit.client.1.smithi117.stdout:6/247: truncate d3/d7/da/f29 629715 0 2024-03-22T06:15:20.120 INFO:tasks.workunit.client.1.smithi117.stdout:1/224: write d1/fc [8372068,127903] 0 2024-03-22T06:15:20.122 INFO:tasks.workunit.client.0.smithi006.stdout:1/925: mkdir d8/d9/d13/d24/d32/d34/d132 0 2024-03-22T06:15:20.122 INFO:tasks.workunit.client.0.smithi006.stdout:1/926: truncate d8/f11d 952527 0 2024-03-22T06:15:20.123 INFO:tasks.workunit.client.1.smithi117.stdout:0/248: stat d1/d10/c21 0 2024-03-22T06:15:20.124 INFO:tasks.workunit.client.1.smithi117.stdout:6/248: symlink d3/d40/d46/d4f/d3a/l55 0 2024-03-22T06:15:20.124 INFO:tasks.workunit.client.1.smithi117.stdout:6/249: truncate d3/d3d/f43 1087873 0 2024-03-22T06:15:20.124 INFO:tasks.workunit.client.1.smithi117.stdout:6/250: stat d3/d3d/f43 0 2024-03-22T06:15:20.124 INFO:tasks.workunit.client.1.smithi117.stdout:6/251: dread - d3/d40/d24/f4e zero size 2024-03-22T06:15:20.124 INFO:tasks.workunit.client.1.smithi117.stdout:6/252: truncate d3/f52 637398 0 2024-03-22T06:15:20.124 INFO:tasks.workunit.client.1.smithi117.stdout:3/163: sync 2024-03-22T06:15:20.126 INFO:tasks.workunit.client.1.smithi117.stdout:5/229: rmdir dd/d16/d1a/d32 39 2024-03-22T06:15:20.126 INFO:tasks.workunit.client.1.smithi117.stdout:5/230: chown dd/d16/d24/f23 13 1 2024-03-22T06:15:20.126 INFO:tasks.workunit.client.1.smithi117.stdout:5/231: write dd/df/f1b [2306306,31301] 0 2024-03-22T06:15:20.128 INFO:tasks.workunit.client.1.smithi117.stdout:1/225: stat d1/d22/d27/l4a 0 2024-03-22T06:15:20.129 INFO:tasks.workunit.client.1.smithi117.stdout:3/164: mkdir d9/d1a/d39 0 2024-03-22T06:15:20.129 INFO:tasks.workunit.client.1.smithi117.stdout:3/165: chown d9/d1a/l29 19 1 2024-03-22T06:15:20.130 INFO:tasks.workunit.client.0.smithi006.stdout:9/953: write f6 [3456804,10907] 0 2024-03-22T06:15:20.132 INFO:tasks.workunit.client.1.smithi117.stdout:1/226: mknod d1/d30/c4c 0 2024-03-22T06:15:20.132 INFO:tasks.workunit.client.1.smithi117.stdout:1/227: write d1/d22/d27/f34 [455120,104008] 0 2024-03-22T06:15:20.133 INFO:tasks.workunit.client.1.smithi117.stdout:3/166: mkdir d9/d1a/d3a 0 2024-03-22T06:15:20.133 INFO:tasks.workunit.client.1.smithi117.stdout:3/167: fdatasync d9/d1a/f1e 0 2024-03-22T06:15:20.134 INFO:tasks.workunit.client.1.smithi117.stdout:1/228: unlink d1/d14/l38 0 2024-03-22T06:15:20.135 INFO:tasks.workunit.client.1.smithi117.stdout:1/229: mkdir d1/d3c/d4d 0 2024-03-22T06:15:20.138 INFO:tasks.workunit.client.1.smithi117.stdout:5/232: write dd/d16/d24/d36/f26 [484214,35425] 0 2024-03-22T06:15:20.138 INFO:tasks.workunit.client.1.smithi117.stdout:5/233: chown dd/d16/d1a/f37 61742715 1 2024-03-22T06:15:20.140 INFO:tasks.workunit.client.1.smithi117.stdout:7/194: sync 2024-03-22T06:15:20.140 INFO:tasks.workunit.client.0.smithi006.stdout:1/927: dread d8/d9/d13/d24/d32/d34/f36 [0,4194304] 0 2024-03-22T06:15:20.141 INFO:tasks.workunit.client.0.smithi006.stdout:7/910: dwrite d5/d1e/d3f/f5a [0,4194304] 0 2024-03-22T06:15:20.141 INFO:tasks.workunit.client.0.smithi006.stdout:7/911: stat d5/d1e/d29/d59/f7a 0 2024-03-22T06:15:20.141 INFO:tasks.workunit.client.1.smithi117.stdout:8/219: dwrite d1/f34 [0,4194304] 0 2024-03-22T06:15:20.142 INFO:tasks.workunit.client.1.smithi117.stdout:5/234: link dd/df/f2b dd/f4a 0 2024-03-22T06:15:20.142 INFO:tasks.workunit.client.1.smithi117.stdout:8/220: write d1/d28/f31 [2625523,96924] 0 2024-03-22T06:15:20.142 INFO:tasks.workunit.client.0.smithi006.stdout:1/928: rmdir d8/d9/d13/d39/d94/db5 39 2024-03-22T06:15:20.142 INFO:tasks.workunit.client.0.smithi006.stdout:1/929: write d8/d9/d13/d39/d60/d8f/dea/f12e [988120,4568] 0 2024-03-22T06:15:20.142 INFO:tasks.workunit.client.0.smithi006.stdout:1/930: write d8/d9/d3a/f111 [790702,47807] 0 2024-03-22T06:15:20.143 INFO:tasks.workunit.client.0.smithi006.stdout:9/954: dread d8/d10/f2e [0,4194304] 0 2024-03-22T06:15:20.143 INFO:tasks.workunit.client.0.smithi006.stdout:9/955: chown d8/db3/d50/d91/dfe/fff 69700 1 2024-03-22T06:15:20.143 INFO:tasks.workunit.client.0.smithi006.stdout:9/956: read d8/d10/d23/f40 [4605032,36566] 0 2024-03-22T06:15:20.146 INFO:tasks.workunit.client.0.smithi006.stdout:1/931: truncate d8/d9/d13/d39/d60/d8f/d10a/d44/f66 2793299 0 2024-03-22T06:15:20.146 INFO:tasks.workunit.client.0.smithi006.stdout:1/932: write d8/d9/d13/d39/d60/d8f/d10a/fd0 [898577,45040] 0 2024-03-22T06:15:20.147 INFO:tasks.workunit.client.1.smithi117.stdout:1/230: dread d1/d14/f19 [0,4194304] 0 2024-03-22T06:15:20.148 INFO:tasks.workunit.client.1.smithi117.stdout:2/226: rename d2/d5/d20 to d2/d5/d13/d3f 0 2024-03-22T06:15:20.148 INFO:tasks.workunit.client.0.smithi006.stdout:9/957: rmdir d8/d10/d20 39 2024-03-22T06:15:20.148 INFO:tasks.workunit.client.0.smithi006.stdout:9/958: dread - d8/d10/d28/d39/d112/f127 zero size 2024-03-22T06:15:20.149 INFO:tasks.workunit.client.0.smithi006.stdout:7/912: dread d5/d1e/d3f/f6c [0,4194304] 0 2024-03-22T06:15:20.152 INFO:tasks.workunit.client.0.smithi006.stdout:7/913: unlink d5/f49 0 2024-03-22T06:15:20.152 INFO:tasks.workunit.client.0.smithi006.stdout:7/914: readlink d5/d1e/d29/d1f/l24 0 2024-03-22T06:15:20.153 INFO:tasks.workunit.client.0.smithi006.stdout:1/933: write d8/d9/d13/d24/fa3 [1423180,8127] 0 2024-03-22T06:15:20.153 INFO:tasks.workunit.client.0.smithi006.stdout:1/934: fdatasync d8/d9/d3a/d18/f56 0 2024-03-22T06:15:20.156 INFO:tasks.workunit.client.0.smithi006.stdout:9/959: dread d8/db3/d50/d91/dfe/d11a/de0/f5a [0,4194304] 0 2024-03-22T06:15:20.157 INFO:tasks.workunit.client.0.smithi006.stdout:9/960: readlink d8/d10/d20/d45/l6d 0 2024-03-22T06:15:20.161 INFO:tasks.workunit.client.0.smithi006.stdout:1/935: creat d8/f133 x:0 0 0 2024-03-22T06:15:20.162 INFO:tasks.workunit.client.0.smithi006.stdout:1/936: readlink d8/d9/d13/d39/d60/d8f/d10a/d44/d80/lc2 0 2024-03-22T06:15:20.162 INFO:tasks.workunit.client.0.smithi006.stdout:7/915: rmdir d5/d1e/d29/d1f/d2d/d41/db7 39 2024-03-22T06:15:20.163 INFO:tasks.workunit.client.0.smithi006.stdout:9/961: dread d8/fd [0,4194304] 0 2024-03-22T06:15:20.163 INFO:tasks.workunit.client.0.smithi006.stdout:9/962: dread - d8/d1f/f139 zero size 2024-03-22T06:15:20.164 INFO:tasks.workunit.client.0.smithi006.stdout:9/963: write d8/db3/d50/d91/dfe/d11a/fc6 [650525,114875] 0 2024-03-22T06:15:20.164 INFO:tasks.workunit.client.1.smithi117.stdout:5/235: creat dd/f4b x:0 0 0 2024-03-22T06:15:20.166 INFO:tasks.workunit.client.1.smithi117.stdout:8/221: symlink d1/d3a/d44/d11/l49 0 2024-03-22T06:15:20.169 INFO:tasks.workunit.client.0.smithi006.stdout:7/916: dread d5/d1e/d29/d66/ff6 [0,4194304] 0 2024-03-22T06:15:20.170 INFO:tasks.workunit.client.1.smithi117.stdout:1/231: mkdir d1/d22/d27/d4e 0 2024-03-22T06:15:20.170 INFO:tasks.workunit.client.1.smithi117.stdout:1/232: truncate d1/d22/d27/f28 5037500 0 2024-03-22T06:15:20.170 INFO:tasks.workunit.client.1.smithi117.stdout:1/233: read d1/f7 [848223,45910] 0 2024-03-22T06:15:20.174 INFO:tasks.workunit.client.1.smithi117.stdout:4/227: rename d0/d19/d28/f2b to d0/d19/d34/f4f 0 2024-03-22T06:15:20.176 INFO:tasks.workunit.client.0.smithi006.stdout:7/917: mkdir d5/d1e/d63/d85/d12f 0 2024-03-22T06:15:20.177 INFO:tasks.workunit.client.1.smithi117.stdout:2/227: mknod d2/d35/d38/c40 0 2024-03-22T06:15:20.178 INFO:tasks.workunit.client.1.smithi117.stdout:5/236: link dd/d16/d24/d36/f31 dd/df/d1e/f4c 0 2024-03-22T06:15:20.178 INFO:tasks.workunit.client.1.smithi117.stdout:5/237: write dd/df/f2c [1317381,34788] 0 2024-03-22T06:15:20.179 INFO:tasks.workunit.client.0.smithi006.stdout:7/918: symlink d5/d1e/d29/d1f/d2d/d41/db7/l130 0 2024-03-22T06:15:20.180 INFO:tasks.workunit.client.1.smithi117.stdout:1/234: mkdir d1/d1f/d32/d4f 0 2024-03-22T06:15:20.182 INFO:tasks.workunit.client.1.smithi117.stdout:0/249: rename d1/d10/d18/d26/l3a to d1/d10/d18/d26/d53/l5b 0 2024-03-22T06:15:20.182 INFO:tasks.workunit.client.1.smithi117.stdout:0/250: write d1/f4f [937742,11157] 0 2024-03-22T06:15:20.182 INFO:tasks.workunit.client.1.smithi117.stdout:0/251: write d1/d10/d18/d26/dc/d39/f52 [594050,79296] 0 2024-03-22T06:15:20.184 INFO:tasks.workunit.client.1.smithi117.stdout:4/228: mknod d0/d48/c50 0 2024-03-22T06:15:20.186 INFO:tasks.workunit.client.1.smithi117.stdout:2/228: unlink d2/l17 0 2024-03-22T06:15:20.188 INFO:tasks.workunit.client.1.smithi117.stdout:1/235: mkdir d1/d30/d50 0 2024-03-22T06:15:20.189 INFO:tasks.workunit.client.0.smithi006.stdout:7/919: dread d5/d1e/d29/d1f/d2d/d55/f113 [0,4194304] 0 2024-03-22T06:15:20.190 INFO:tasks.workunit.client.0.smithi006.stdout:7/920: dread - d5/d1e/d29/f118 zero size 2024-03-22T06:15:20.190 INFO:tasks.workunit.client.0.smithi006.stdout:7/921: stat d5/d1e/d29/d66/d91/ddc/dfa/d121/d12b 0 2024-03-22T06:15:20.191 INFO:tasks.workunit.client.1.smithi117.stdout:4/229: mkdir d0/d51 0 2024-03-22T06:15:20.191 INFO:tasks.workunit.client.1.smithi117.stdout:4/230: stat d0/d19/d34 0 2024-03-22T06:15:20.191 INFO:tasks.workunit.client.1.smithi117.stdout:4/231: fsync d0/d48/f3d 0 2024-03-22T06:15:20.191 INFO:tasks.workunit.client.1.smithi117.stdout:5/238: dread dd/df/f1b [0,4194304] 0 2024-03-22T06:15:20.193 INFO:tasks.workunit.client.1.smithi117.stdout:6/253: rename d3/d40/d46/d4f to d3/d7/da/d56 0 2024-03-22T06:15:20.193 INFO:tasks.workunit.client.1.smithi117.stdout:6/254: readlink d3/d7/l18 0 2024-03-22T06:15:20.194 INFO:tasks.workunit.client.1.smithi117.stdout:2/229: mkdir d2/d5/d41 0 2024-03-22T06:15:20.195 INFO:tasks.workunit.client.0.smithi006.stdout:9/964: dwrite f3 [0,4194304] 0 2024-03-22T06:15:20.195 INFO:tasks.workunit.client.1.smithi117.stdout:8/222: dwrite d1/d19/f43 [0,4194304] 0 2024-03-22T06:15:20.195 INFO:tasks.workunit.client.1.smithi117.stdout:8/223: fsync d1/d19/d38/f42 0 2024-03-22T06:15:20.195 INFO:tasks.workunit.client.1.smithi117.stdout:8/224: truncate d1/d28/f31 2842165 0 2024-03-22T06:15:20.196 INFO:tasks.workunit.client.0.smithi006.stdout:7/922: creat d5/d1e/d63/d85/dac/f131 x:0 0 0 2024-03-22T06:15:20.196 INFO:tasks.workunit.client.0.smithi006.stdout:7/923: fdatasync d5/d1e/d29/d59/f77 0 2024-03-22T06:15:20.198 INFO:tasks.workunit.client.1.smithi117.stdout:3/168: rename d9/c24 to d9/d2e/d37/c3b 0 2024-03-22T06:15:20.201 INFO:tasks.workunit.client.0.smithi006.stdout:7/924: dread d5/d1e/d63/d85/dac/db3/f127 [0,4194304] 0 2024-03-22T06:15:20.202 INFO:tasks.workunit.client.1.smithi117.stdout:6/255: rename d3/d7/fd to d3/d7/f57 0 2024-03-22T06:15:20.203 INFO:tasks.workunit.client.1.smithi117.stdout:6/256: dread d3/d3d/f2e [0,4194304] 0 2024-03-22T06:15:20.204 INFO:tasks.workunit.client.1.smithi117.stdout:2/230: write d2/f26 [2766801,96214] 0 2024-03-22T06:15:20.204 INFO:tasks.workunit.client.1.smithi117.stdout:2/231: read d2/d5/d13/f30 [123569,112469] 0 2024-03-22T06:15:20.206 INFO:tasks.workunit.client.1.smithi117.stdout:5/239: dwrite dd/df/f3a [0,4194304] 0 2024-03-22T06:15:20.209 INFO:tasks.workunit.client.0.smithi006.stdout:9/965: dread d8/db3/d50/d91/dfe/d11a/de0/fae [0,4194304] 0 2024-03-22T06:15:20.210 INFO:tasks.workunit.client.1.smithi117.stdout:8/225: link d1/c26 d1/d19/c4a 0 2024-03-22T06:15:20.218 INFO:tasks.workunit.client.1.smithi117.stdout:7/195: rename d8/d9/d1a to d8/d9/da/d10/d23/d27/d39 0 2024-03-22T06:15:20.218 INFO:tasks.workunit.client.1.smithi117.stdout:5/240: dread dd/df/d30/f45 [0,4194304] 0 2024-03-22T06:15:20.219 INFO:tasks.workunit.client.1.smithi117.stdout:6/257: symlink d3/d40/d24/d2d/d42/l58 0 2024-03-22T06:15:20.221 INFO:tasks.workunit.client.1.smithi117.stdout:9/256: sync 2024-03-22T06:15:20.221 INFO:tasks.workunit.client.1.smithi117.stdout:8/226: dread f0 [0,4194304] 0 2024-03-22T06:15:20.221 INFO:tasks.workunit.client.1.smithi117.stdout:7/196: rmdir d8 39 2024-03-22T06:15:20.224 INFO:tasks.workunit.client.1.smithi117.stdout:1/236: rename d1/d14/l1c to d1/d1f/d32/d33/l51 0 2024-03-22T06:15:20.224 INFO:tasks.workunit.client.1.smithi117.stdout:1/237: read d1/d22/d27/f28 [3977376,79776] 0 2024-03-22T06:15:20.225 INFO:tasks.workunit.client.1.smithi117.stdout:5/241: mknod dd/d16/c4d 0 2024-03-22T06:15:20.226 INFO:tasks.workunit.client.1.smithi117.stdout:6/258: mkdir d3/d40/d24/d2d/d53/d59 0 2024-03-22T06:15:20.230 INFO:tasks.workunit.client.1.smithi117.stdout:9/257: creat d4/d8/d37/f4b x:0 0 0 2024-03-22T06:15:20.230 INFO:tasks.workunit.client.1.smithi117.stdout:5/242: dread dd/d16/d24/d36/f26 [0,4194304] 0 2024-03-22T06:15:20.231 INFO:tasks.workunit.client.0.smithi006.stdout:9/966: dwrite d8/db3/d50/d91/dfe/d11a/fdd [0,4194304] 0 2024-03-22T06:15:20.231 INFO:tasks.workunit.client.0.smithi006.stdout:9/967: stat d8/fd 0 2024-03-22T06:15:20.232 INFO:tasks.workunit.client.1.smithi117.stdout:7/197: creat d8/f3a x:0 0 0 2024-03-22T06:15:20.234 INFO:tasks.workunit.client.1.smithi117.stdout:4/232: rename d0/d19/d34/l4c to d0/d19/d28/d47/l52 0 2024-03-22T06:15:20.234 INFO:tasks.workunit.client.0.smithi006.stdout:1/937: truncate d8/d9/d3a/f111 324821 0 2024-03-22T06:15:20.234 INFO:tasks.workunit.client.0.smithi006.stdout:1/938: chown d8/d9/d13 51 1 2024-03-22T06:15:20.234 INFO:tasks.workunit.client.0.smithi006.stdout:1/939: write d8/d9/d13/d39/d60/d8f/f12c [218617,119406] 0 2024-03-22T06:15:20.236 INFO:tasks.workunit.client.1.smithi117.stdout:1/238: mknod d1/d30/d50/c52 0 2024-03-22T06:15:20.236 INFO:tasks.workunit.client.1.smithi117.stdout:1/239: fsync d1/d1f/f3e 0 2024-03-22T06:15:20.238 INFO:tasks.workunit.client.1.smithi117.stdout:6/259: mkdir d3/d7/da/d5a 0 2024-03-22T06:15:20.239 INFO:tasks.workunit.client.1.smithi117.stdout:6/260: fdatasync d3/d3d/f43 0 2024-03-22T06:15:20.239 INFO:tasks.workunit.client.1.smithi117.stdout:5/243: dread dd/d16/d24/f23 [0,4194304] 0 2024-03-22T06:15:20.240 INFO:tasks.workunit.client.1.smithi117.stdout:9/258: rename d4/d7/d27/f36 to d4/d7/d27/f4c 0 2024-03-22T06:15:20.242 INFO:tasks.workunit.client.0.smithi006.stdout:9/968: dread d8/db3/d133/fa9 [0,4194304] 0 2024-03-22T06:15:20.243 INFO:tasks.workunit.client.1.smithi117.stdout:4/233: link d0/d4/d2a/f30 d0/d4/d2a/d2f/f53 0 2024-03-22T06:15:20.244 INFO:tasks.workunit.client.1.smithi117.stdout:2/232: rename f1 to d2/d5/db/d2c/f42 0 2024-03-22T06:15:20.244 INFO:tasks.workunit.client.1.smithi117.stdout:2/233: chown d2/d5/db/d2c/f42 189 1 2024-03-22T06:15:20.247 INFO:tasks.workunit.client.1.smithi117.stdout:1/240: symlink d1/d1f/d32/l53 0 2024-03-22T06:15:20.247 INFO:tasks.workunit.client.1.smithi117.stdout:1/241: chown d1/d30/d35/l48 0 1 2024-03-22T06:15:20.248 INFO:tasks.workunit.client.1.smithi117.stdout:6/261: rmdir d3/d7/da/d5a 0 2024-03-22T06:15:20.248 INFO:tasks.workunit.client.1.smithi117.stdout:6/262: dread - d3/d40/d24/f4e zero size 2024-03-22T06:15:20.249 INFO:tasks.workunit.client.1.smithi117.stdout:5/244: write dd/df/d30/f45 [3027680,82939] 0 2024-03-22T06:15:20.251 INFO:tasks.workunit.client.1.smithi117.stdout:4/234: creat d0/d3f/f54 x:0 0 0 2024-03-22T06:15:20.253 INFO:tasks.workunit.client.1.smithi117.stdout:2/234: write d2/f4 [2376394,31089] 0 2024-03-22T06:15:20.255 INFO:tasks.workunit.client.1.smithi117.stdout:6/263: rename d3/d7/da/d32/f38 to d3/d40/d24/d2d/d30/d3e/d49/f5b 0 2024-03-22T06:15:20.258 INFO:tasks.workunit.client.1.smithi117.stdout:4/235: creat d0/d19/d28/f55 x:0 0 0 2024-03-22T06:15:20.259 INFO:tasks.workunit.client.1.smithi117.stdout:4/236: write d0/d3f/f54 [545593,7265] 0 2024-03-22T06:15:20.260 INFO:tasks.workunit.client.1.smithi117.stdout:2/235: mknod d2/d5/d13/d3f/c43 0 2024-03-22T06:15:20.263 INFO:tasks.workunit.client.1.smithi117.stdout:4/237: rename d0/d4/f1b to d0/d4/d2a/d2f/f56 0 2024-03-22T06:15:20.263 INFO:tasks.workunit.client.1.smithi117.stdout:4/238: write d0/d19/d28/f2e [1335200,106045] 0 2024-03-22T06:15:20.264 INFO:tasks.workunit.client.1.smithi117.stdout:4/239: creat d0/d19/d34/d43/f57 x:0 0 0 2024-03-22T06:15:20.271 INFO:tasks.workunit.client.1.smithi117.stdout:1/242: dread d1/f13 [0,4194304] 0 2024-03-22T06:15:20.274 INFO:tasks.workunit.client.1.smithi117.stdout:6/264: dread d3/d40/d24/d2d/d30/d3e/d49/f5b [0,4194304] 0 2024-03-22T06:15:20.274 INFO:tasks.workunit.client.1.smithi117.stdout:6/265: chown d3/d7/da/cb 997765 1 2024-03-22T06:15:20.274 INFO:tasks.workunit.client.1.smithi117.stdout:6/266: chown d3/c13 2 1 2024-03-22T06:15:20.274 INFO:tasks.workunit.client.1.smithi117.stdout:6/267: write d3/d40/d24/f4e [46509,79708] 0 2024-03-22T06:15:20.274 INFO:tasks.workunit.client.1.smithi117.stdout:1/243: creat d1/d1f/d32/f54 x:0 0 0 2024-03-22T06:15:20.276 INFO:tasks.workunit.client.1.smithi117.stdout:6/268: mknod d3/d40/d24/d2d/d42/c5c 0 2024-03-22T06:15:20.276 INFO:tasks.workunit.client.1.smithi117.stdout:6/269: chown d3/d7/l21 56952 1 2024-03-22T06:15:20.276 INFO:tasks.workunit.client.1.smithi117.stdout:2/236: dread d2/fc [4194304,4194304] 0 2024-03-22T06:15:20.276 INFO:tasks.workunit.client.1.smithi117.stdout:4/240: dread d0/d4/d2a/d2f/f53 [0,4194304] 0 2024-03-22T06:15:20.278 INFO:tasks.workunit.client.1.smithi117.stdout:2/237: dread d2/d5/db/f24 [0,4194304] 0 2024-03-22T06:15:20.278 INFO:tasks.workunit.client.1.smithi117.stdout:2/238: readlink d2/d1b/l2e 0 2024-03-22T06:15:20.279 INFO:tasks.workunit.client.1.smithi117.stdout:2/239: mknod d2/d5/db/d2c/c44 0 2024-03-22T06:15:20.281 INFO:tasks.workunit.client.1.smithi117.stdout:2/240: truncate d2/d5/db/d2c/f42 5567725 0 2024-03-22T06:15:20.281 INFO:tasks.workunit.client.1.smithi117.stdout:2/241: chown d2/d5/d13/d3f/d1d/d3a/c34 8004 1 2024-03-22T06:15:20.282 INFO:tasks.workunit.client.1.smithi117.stdout:8/227: dwrite d1/f34 [0,4194304] 0 2024-03-22T06:15:20.283 INFO:tasks.workunit.client.1.smithi117.stdout:1/244: write d1/f13 [2414416,37714] 0 2024-03-22T06:15:20.283 INFO:tasks.workunit.client.1.smithi117.stdout:1/245: fdatasync d1/d22/d27/f34 0 2024-03-22T06:15:20.287 INFO:tasks.workunit.client.1.smithi117.stdout:7/198: dwrite d8/d9/fd [0,4194304] 0 2024-03-22T06:15:20.288 INFO:tasks.workunit.client.1.smithi117.stdout:9/259: dwrite d4/d7/f26 [0,4194304] 0 2024-03-22T06:15:20.288 INFO:tasks.workunit.client.0.smithi006.stdout:1/940: dwrite d8/d9/d3a/d18/f78 [0,4194304] 0 2024-03-22T06:15:20.288 INFO:tasks.workunit.client.1.smithi117.stdout:8/228: rename d1/d1f to d1/d3a/d44/d4b 0 2024-03-22T06:15:20.288 INFO:tasks.workunit.client.1.smithi117.stdout:8/229: stat d1/d3a/d44/d16/c21 0 2024-03-22T06:15:20.290 INFO:tasks.workunit.client.0.smithi006.stdout:7/925: dwrite d5/d1e/d29/d1f/d2d/d61/daa/fc0 [0,4194304] 0 2024-03-22T06:15:20.293 INFO:tasks.workunit.client.1.smithi117.stdout:5/245: dwrite fa [0,4194304] 0 2024-03-22T06:15:20.294 INFO:tasks.workunit.client.1.smithi117.stdout:5/246: write dd/df/f2a [1578611,46314] 0 2024-03-22T06:15:20.298 INFO:tasks.workunit.client.0.smithi006.stdout:7/926: creat d5/d1e/d63/d85/da1/f132 x:0 0 0 2024-03-22T06:15:20.310 INFO:tasks.workunit.client.1.smithi117.stdout:9/260: unlink d4/f1e 0 2024-03-22T06:15:20.310 INFO:tasks.workunit.client.1.smithi117.stdout:9/261: dread - d4/d7/d10/d2f/f39 zero size 2024-03-22T06:15:20.311 INFO:tasks.workunit.client.1.smithi117.stdout:8/230: getdents d1/d3a/d44/d4b 0 2024-03-22T06:15:20.315 INFO:tasks.workunit.client.1.smithi117.stdout:9/262: creat d4/d47/f4d x:0 0 0 2024-03-22T06:15:20.316 INFO:tasks.workunit.client.1.smithi117.stdout:9/263: write d4/d7/f2a [713971,20189] 0 2024-03-22T06:15:20.316 INFO:tasks.workunit.client.1.smithi117.stdout:9/264: dread - d4/d8/d3e/f45 zero size 2024-03-22T06:15:20.316 INFO:tasks.workunit.client.1.smithi117.stdout:9/265: readlink d4/d7/l32 0 2024-03-22T06:15:20.317 INFO:tasks.workunit.client.1.smithi117.stdout:9/266: creat d4/d7/d10/d2f/f4e x:0 0 0 2024-03-22T06:15:20.320 INFO:tasks.workunit.client.1.smithi117.stdout:9/267: mknod d4/d7/d10/d3b/c4f 0 2024-03-22T06:15:20.326 INFO:tasks.workunit.client.1.smithi117.stdout:4/241: dwrite d0/d19/f22 [0,4194304] 0 2024-03-22T06:15:20.328 INFO:tasks.workunit.client.1.smithi117.stdout:4/242: truncate d0/d4/f17 2905677 0 2024-03-22T06:15:20.328 INFO:tasks.workunit.client.1.smithi117.stdout:4/243: truncate d0/d19/f49 488816 0 2024-03-22T06:15:20.328 INFO:tasks.workunit.client.1.smithi117.stdout:4/244: chown d0/d4/d2a/d2f/f56 4560 1 2024-03-22T06:15:20.329 INFO:tasks.workunit.client.1.smithi117.stdout:7/199: dwrite d8/d9/da/d10/f25 [0,4194304] 0 2024-03-22T06:15:20.329 INFO:tasks.workunit.client.1.smithi117.stdout:7/200: stat d8/d9/da/d10/d23/d27/d32/c36 0 2024-03-22T06:15:20.329 INFO:tasks.workunit.client.1.smithi117.stdout:7/201: fsync d8/d9/fd 0 2024-03-22T06:15:20.331 INFO:tasks.workunit.client.1.smithi117.stdout:9/268: write d4/d7/f1c [2814016,17557] 0 2024-03-22T06:15:20.332 INFO:tasks.workunit.client.1.smithi117.stdout:6/270: dwrite d3/d7/da/f29 [0,4194304] 0 2024-03-22T06:15:20.333 INFO:tasks.workunit.client.1.smithi117.stdout:6/271: stat d3/d7/da/f29 0 2024-03-22T06:15:20.333 INFO:tasks.workunit.client.1.smithi117.stdout:6/272: chown d3/d40/d46/c50 1766 1 2024-03-22T06:15:20.334 INFO:tasks.workunit.client.1.smithi117.stdout:0/252: sync 2024-03-22T06:15:20.334 INFO:tasks.workunit.client.1.smithi117.stdout:9/269: mkdir d4/d50 0 2024-03-22T06:15:20.337 INFO:tasks.workunit.client.1.smithi117.stdout:4/245: write d0/d19/d1c/f4d [1079075,80155] 0 2024-03-22T06:15:20.337 INFO:tasks.workunit.client.1.smithi117.stdout:3/169: sync 2024-03-22T06:15:20.337 INFO:tasks.workunit.client.1.smithi117.stdout:3/170: write d9/f18 [2300012,71253] 0 2024-03-22T06:15:20.340 INFO:tasks.workunit.client.0.smithi006.stdout:1/941: dwrite d8/d9/d13/d39/f107 [0,4194304] 0 2024-03-22T06:15:20.340 INFO:tasks.workunit.client.0.smithi006.stdout:1/942: fdatasync d8/d33/f11e 0 2024-03-22T06:15:20.341 INFO:tasks.workunit.client.1.smithi117.stdout:5/247: dwrite dd/d16/f48 [0,4194304] 0 2024-03-22T06:15:20.341 INFO:tasks.workunit.client.1.smithi117.stdout:5/248: read dd/f4a [369895,29209] 0 2024-03-22T06:15:20.341 INFO:tasks.workunit.client.1.smithi117.stdout:5/249: write dd/df/d30/d35/f27 [1373512,79970] 0 2024-03-22T06:15:20.341 INFO:tasks.workunit.client.0.smithi006.stdout:7/927: dwrite d5/d1e/d29/d66/fdb [0,4194304] 0 2024-03-22T06:15:20.342 INFO:tasks.workunit.client.1.smithi117.stdout:8/231: dwrite d1/f1e [0,4194304] 0 2024-03-22T06:15:20.342 INFO:tasks.workunit.client.1.smithi117.stdout:3/171: symlink d9/d1a/l3c 0 2024-03-22T06:15:20.342 INFO:tasks.workunit.client.1.smithi117.stdout:9/270: dread d4/d7/f2d [0,4194304] 0 2024-03-22T06:15:20.342 INFO:tasks.workunit.client.1.smithi117.stdout:9/271: chown d4/d7/d10 7498002 1 2024-03-22T06:15:20.342 INFO:tasks.workunit.client.1.smithi117.stdout:9/272: write d4/d7/f2a [1713214,44359] 0 2024-03-22T06:15:20.343 INFO:tasks.workunit.client.1.smithi117.stdout:5/250: dread dd/d16/d24/d36/f26 [0,4194304] 0 2024-03-22T06:15:20.345 INFO:tasks.workunit.client.1.smithi117.stdout:0/253: rename d1/d3/d2b/d3e/de/d14 to d1/d10/d5c 0 2024-03-22T06:15:20.345 INFO:tasks.workunit.client.1.smithi117.stdout:3/172: rename d9/d2e to d9/d2e/d37/d3d 22 2024-03-22T06:15:20.345 INFO:tasks.workunit.client.1.smithi117.stdout:0/254: fsync d1/d10/f27 0 2024-03-22T06:15:20.346 INFO:tasks.workunit.client.1.smithi117.stdout:7/202: dwrite f5 [4194304,4194304] 0 2024-03-22T06:15:20.346 INFO:tasks.workunit.client.1.smithi117.stdout:7/203: chown d8/d9/da/d10/d23/d27/d39/d26/l2e 126 1 2024-03-22T06:15:20.346 INFO:tasks.workunit.client.1.smithi117.stdout:7/204: fsync d8/d9/da/d10/d23/f24 0 2024-03-22T06:15:20.350 INFO:tasks.workunit.client.1.smithi117.stdout:6/273: dread d3/f5 [0,4194304] 0 2024-03-22T06:15:20.350 INFO:tasks.workunit.client.1.smithi117.stdout:6/274: chown d3/d7/da/d56/d3a/c41 613 1 2024-03-22T06:15:20.350 INFO:tasks.workunit.client.1.smithi117.stdout:6/275: chown d3/l9 53240 1 2024-03-22T06:15:20.350 INFO:tasks.workunit.client.1.smithi117.stdout:6/276: write d3/d3d/f2e [467417,47469] 0 2024-03-22T06:15:20.352 INFO:tasks.workunit.client.1.smithi117.stdout:6/277: dread d3/d7/f1b [0,4194304] 0 2024-03-22T06:15:20.354 INFO:tasks.workunit.client.0.smithi006.stdout:7/928: mkdir d5/d1e/d63/d85/dac/d133 0 2024-03-22T06:15:20.355 INFO:tasks.workunit.client.1.smithi117.stdout:8/232: truncate d1/d3a/d44/d9/f13 10528540 0 2024-03-22T06:15:20.356 INFO:tasks.workunit.client.0.smithi006.stdout:1/943: rename d8/d9/d13/d24/l4c to d8/d9/d13/d39/d60/d8f/d10a/da1/l134 0 2024-03-22T06:15:20.358 INFO:tasks.workunit.client.1.smithi117.stdout:4/246: dwrite d0/d19/f22 [0,4194304] 0 2024-03-22T06:15:20.358 INFO:tasks.workunit.client.1.smithi117.stdout:9/273: link d4/d8/lb d4/d8/d3e/l51 0 2024-03-22T06:15:20.360 INFO:tasks.workunit.client.0.smithi006.stdout:7/929: mknod d5/d1e/d63/d85/d80/c134 0 2024-03-22T06:15:20.360 INFO:tasks.workunit.client.0.smithi006.stdout:7/930: dread - d5/d1e/d29/db9/ff9 zero size 2024-03-22T06:15:20.362 INFO:tasks.workunit.client.1.smithi117.stdout:5/251: mknod dd/df/d30/d33/c4e 0 2024-03-22T06:15:20.363 INFO:tasks.workunit.client.1.smithi117.stdout:3/173: mknod d9/d1a/d3a/c3e 0 2024-03-22T06:15:20.363 INFO:tasks.workunit.client.1.smithi117.stdout:3/174: chown d9/d1a/l28 7119 1 2024-03-22T06:15:20.365 INFO:tasks.workunit.client.0.smithi006.stdout:7/931: unlink d5/d1e/d29/d10/f25 0 2024-03-22T06:15:20.365 INFO:tasks.workunit.client.0.smithi006.stdout:7/932: dread - d5/d1e/d63/f11c zero size 2024-03-22T06:15:20.365 INFO:tasks.workunit.client.0.smithi006.stdout:7/933: dread - d5/d1e/d29/d1f/d2d/d55/fe8 zero size 2024-03-22T06:15:20.365 INFO:tasks.workunit.client.0.smithi006.stdout:7/934: write d5/d1e/d63/f12a [703444,33318] 0 2024-03-22T06:15:20.366 INFO:tasks.workunit.client.1.smithi117.stdout:0/255: symlink d1/d10/d18/d26/d49/l5d 0 2024-03-22T06:15:20.366 INFO:tasks.workunit.client.1.smithi117.stdout:0/256: chown d1/d10/d18/d26/l44 95 1 2024-03-22T06:15:20.366 INFO:tasks.workunit.client.1.smithi117.stdout:0/257: write d1/d10/d18/d26/dc/f47 [393728,129143] 0 2024-03-22T06:15:20.367 INFO:tasks.workunit.client.1.smithi117.stdout:7/205: creat d8/d9/da/d10/d23/d27/d39/f3b x:0 0 0 2024-03-22T06:15:20.371 INFO:tasks.workunit.client.1.smithi117.stdout:6/278: link d3/d7/da/cb d3/d40/d24/d2d/d53/c5d 0 2024-03-22T06:15:20.371 INFO:tasks.workunit.client.1.smithi117.stdout:6/279: dread - d3/d3d/d17/d31/f34 zero size 2024-03-22T06:15:20.371 INFO:tasks.workunit.client.1.smithi117.stdout:6/280: chown d3/d40/d24/d2d/d53/c5d 0 1 2024-03-22T06:15:20.371 INFO:tasks.workunit.client.1.smithi117.stdout:6/281: readlink d3/d7/l25 0 2024-03-22T06:15:20.371 INFO:tasks.workunit.client.1.smithi117.stdout:6/282: fsync d3/d7/da/d56/f2f 0 2024-03-22T06:15:20.372 INFO:tasks.workunit.client.1.smithi117.stdout:5/252: dread dd/df/f3a [0,4194304] 0 2024-03-22T06:15:20.373 INFO:tasks.workunit.client.0.smithi006.stdout:7/935: rename d5/d1e/d29/d1f/d2d/d55/d84/ce5 to d5/d1e/c135 0 2024-03-22T06:15:20.373 INFO:tasks.workunit.client.0.smithi006.stdout:7/936: write d5/d1e/d3f/f45 [1596891,51160] 0 2024-03-22T06:15:20.374 INFO:tasks.workunit.client.1.smithi117.stdout:8/233: rename d1/c2e to d1/d3a/d44/d4b/c4c 0 2024-03-22T06:15:20.378 INFO:tasks.workunit.client.0.smithi006.stdout:7/937: creat d5/d1e/d3f/f136 x:0 0 0 2024-03-22T06:15:20.378 INFO:tasks.workunit.client.0.smithi006.stdout:7/938: chown d5/d1e/d63/d85/dac/db3/f127 27896 1 2024-03-22T06:15:20.381 INFO:tasks.workunit.client.1.smithi117.stdout:3/175: dwrite f8 [0,4194304] 0 2024-03-22T06:15:20.381 INFO:tasks.workunit.client.0.smithi006.stdout:1/944: dwrite d8/d9/d13/f28 [4194304,4194304] 0 2024-03-22T06:15:20.381 INFO:tasks.workunit.client.0.smithi006.stdout:1/945: write d8/d9/d13/d24/d32/d34/f99 [869788,17107] 0 2024-03-22T06:15:20.387 INFO:tasks.workunit.client.1.smithi117.stdout:4/247: mknod d0/d19/d28/c58 0 2024-03-22T06:15:20.388 INFO:tasks.workunit.client.1.smithi117.stdout:9/274: link d4/d7/d10/d2f/f4e d4/d8/d3f/f52 0 2024-03-22T06:15:20.389 INFO:tasks.workunit.client.1.smithi117.stdout:5/253: creat dd/d16/d24/f4f x:0 0 0 2024-03-22T06:15:20.390 INFO:tasks.workunit.client.1.smithi117.stdout:3/176: symlink d9/d1a/d39/l3f 0 2024-03-22T06:15:20.393 INFO:tasks.workunit.client.1.smithi117.stdout:5/254: symlink dd/df/d30/d35/l50 0 2024-03-22T06:15:20.393 INFO:tasks.workunit.client.1.smithi117.stdout:8/234: getdents d1/d19 0 2024-03-22T06:15:20.393 INFO:tasks.workunit.client.1.smithi117.stdout:8/235: stat d1/d3a/d44/d9/ld 0 2024-03-22T06:15:20.394 INFO:tasks.workunit.client.1.smithi117.stdout:3/177: creat d9/d1a/d3a/f40 x:0 0 0 2024-03-22T06:15:20.394 INFO:tasks.workunit.client.1.smithi117.stdout:3/178: dread - d9/d1a/f20 zero size 2024-03-22T06:15:20.394 INFO:tasks.workunit.client.1.smithi117.stdout:5/255: creat dd/d16/d1a/d32/f51 x:0 0 0 2024-03-22T06:15:20.396 INFO:tasks.workunit.client.1.smithi117.stdout:3/179: mkdir d9/d2e/d36/d41 0 2024-03-22T06:15:20.396 INFO:tasks.workunit.client.1.smithi117.stdout:3/180: truncate d9/f26 942030 0 2024-03-22T06:15:20.396 INFO:tasks.workunit.client.1.smithi117.stdout:3/181: truncate d9/f22 481401 0 2024-03-22T06:15:20.396 INFO:tasks.workunit.client.1.smithi117.stdout:3/182: stat d9/c15 0 2024-03-22T06:15:20.397 INFO:tasks.workunit.client.1.smithi117.stdout:3/183: creat d9/d2e/f42 x:0 0 0 2024-03-22T06:15:20.399 INFO:tasks.workunit.client.1.smithi117.stdout:3/184: creat d9/d2e/d36/f43 x:0 0 0 2024-03-22T06:15:20.399 INFO:tasks.workunit.client.1.smithi117.stdout:3/185: fdatasync d9/d1a/f1e 0 2024-03-22T06:15:20.399 INFO:tasks.workunit.client.1.smithi117.stdout:3/186: creat d9/d2e/f44 x:0 0 0 2024-03-22T06:15:20.399 INFO:tasks.workunit.client.1.smithi117.stdout:3/187: fdatasync f2 0 2024-03-22T06:15:20.399 INFO:tasks.workunit.client.1.smithi117.stdout:9/275: read d4/d7/d10/f28 [1342743,107879] 0 2024-03-22T06:15:20.400 INFO:tasks.workunit.client.1.smithi117.stdout:9/276: chown d4/d7/d10/d2f 9082278 1 2024-03-22T06:15:20.400 INFO:tasks.workunit.client.1.smithi117.stdout:9/277: write d4/d8/f1a [786,122746] 0 2024-03-22T06:15:20.402 INFO:tasks.workunit.client.1.smithi117.stdout:3/188: creat d9/d1a/d39/f45 x:0 0 0 2024-03-22T06:15:20.402 INFO:tasks.workunit.client.1.smithi117.stdout:3/189: creat d9/d2e/f46 x:0 0 0 2024-03-22T06:15:20.402 INFO:tasks.workunit.client.1.smithi117.stdout:3/190: stat d9/c2b 0 2024-03-22T06:15:20.402 INFO:tasks.workunit.client.1.smithi117.stdout:3/191: creat d9/d1a/d3a/f47 x:0 0 0 2024-03-22T06:15:20.402 INFO:tasks.workunit.client.1.smithi117.stdout:3/192: creat d9/d1a/d39/f48 x:0 0 0 2024-03-22T06:15:20.402 INFO:tasks.workunit.client.1.smithi117.stdout:3/193: creat d9/d2e/f49 x:0 0 0 2024-03-22T06:15:20.409 INFO:tasks.workunit.client.1.smithi117.stdout:3/194: rename d9/d1a/l28 to d9/d1a/l4a 0 2024-03-22T06:15:20.411 INFO:tasks.workunit.client.1.smithi117.stdout:7/206: rmdir d8/d9 39 2024-03-22T06:15:20.413 INFO:tasks.workunit.client.1.smithi117.stdout:7/207: symlink d8/d9/d37/l3c 0 2024-03-22T06:15:20.414 INFO:tasks.workunit.client.1.smithi117.stdout:3/195: dread f2 [0,4194304] 0 2024-03-22T06:15:20.415 INFO:tasks.workunit.client.1.smithi117.stdout:3/196: truncate d9/f18 4865949 0 2024-03-22T06:15:20.415 INFO:tasks.workunit.client.1.smithi117.stdout:7/208: rename d8/d9/fd to d8/d9/da/d2b/f3d 0 2024-03-22T06:15:20.417 INFO:tasks.workunit.client.0.smithi006.stdout:1/946: dwrite d8/d9/d13/d24/d32/dc7/fc6 [0,4194304] 0 2024-03-22T06:15:20.417 INFO:tasks.workunit.client.0.smithi006.stdout:1/947: fsync d8/d9/f8d 0 2024-03-22T06:15:20.417 INFO:tasks.workunit.client.0.smithi006.stdout:1/948: dread - d8/d9/d13/d39/f121 zero size 2024-03-22T06:15:20.417 INFO:tasks.workunit.client.0.smithi006.stdout:7/939: dwrite d5/d1e/d3f/f4b [0,4194304] 0 2024-03-22T06:15:20.418 INFO:tasks.workunit.client.1.smithi117.stdout:7/209: unlink d8/d9/da/d10/d23/d27/d39/d26/l2e 0 2024-03-22T06:15:20.419 INFO:tasks.workunit.client.0.smithi006.stdout:1/949: mknod d8/d71/c135 0 2024-03-22T06:15:20.420 INFO:tasks.workunit.client.1.smithi117.stdout:5/256: dwrite dd/df/f1b [0,4194304] 0 2024-03-22T06:15:20.421 INFO:tasks.workunit.client.1.smithi117.stdout:7/210: unlink d8/d9/d37/l3c 0 2024-03-22T06:15:20.423 INFO:tasks.workunit.client.0.smithi006.stdout:7/940: creat d5/f137 x:0 0 0 2024-03-22T06:15:20.424 INFO:tasks.workunit.client.1.smithi117.stdout:9/278: dwrite d4/d8/f2c [0,4194304] 0 2024-03-22T06:15:20.425 INFO:tasks.workunit.client.1.smithi117.stdout:7/211: creat d8/d9/da/d10/d23/d27/f3e x:0 0 0 2024-03-22T06:15:20.427 INFO:tasks.workunit.client.0.smithi006.stdout:7/941: rename d5/d1e/d3f/fc8 to d5/d1e/d29/d1f/de4/f138 0 2024-03-22T06:15:20.428 INFO:tasks.workunit.client.1.smithi117.stdout:1/246: sync 2024-03-22T06:15:20.428 INFO:tasks.workunit.client.1.smithi117.stdout:2/242: sync 2024-03-22T06:15:20.430 INFO:tasks.workunit.client.0.smithi006.stdout:1/950: read d8/d9/d3a/f6b [1255407,129177] 0 2024-03-22T06:15:20.431 INFO:tasks.workunit.client.0.smithi006.stdout:7/942: mknod d5/d1e/d29/d66/d91/ddc/dfa/c139 0 2024-03-22T06:15:20.433 INFO:tasks.workunit.client.1.smithi117.stdout:9/279: unlink f2 0 2024-03-22T06:15:20.434 INFO:tasks.workunit.client.1.smithi117.stdout:7/212: symlink d8/d9/da/d10/d23/d27/d32/l3f 0 2024-03-22T06:15:20.434 INFO:tasks.workunit.client.1.smithi117.stdout:7/213: write d8/d9/da/d10/d23/d27/d39/d26/f34 [1044724,41206] 0 2024-03-22T06:15:20.436 INFO:tasks.workunit.client.0.smithi006.stdout:7/943: creat d5/d1e/d29/d66/d91/ddc/f13a x:0 0 0 2024-03-22T06:15:20.436 INFO:tasks.workunit.client.0.smithi006.stdout:7/944: write d5/d1e/d29/d59/f77 [1412511,80637] 0 2024-03-22T06:15:20.437 INFO:tasks.workunit.client.1.smithi117.stdout:2/243: dread d2/d5/d13/f19 [0,4194304] 0 2024-03-22T06:15:20.438 INFO:tasks.workunit.client.1.smithi117.stdout:1/247: mknod d1/d22/d27/d4e/c55 0 2024-03-22T06:15:20.441 INFO:tasks.workunit.client.1.smithi117.stdout:2/244: write d2/d1b/f28 [2587573,92221] 0 2024-03-22T06:15:20.441 INFO:tasks.workunit.client.1.smithi117.stdout:2/245: read d2/d5/d13/d3f/d1d/d3a/f37 [2069975,37472] 0 2024-03-22T06:15:20.441 INFO:tasks.workunit.client.1.smithi117.stdout:2/246: chown d2/fc 8455 1 2024-03-22T06:15:20.443 INFO:tasks.workunit.client.1.smithi117.stdout:9/280: dread d4/d7/d27/f2e [0,4194304] 0 2024-03-22T06:15:20.443 INFO:tasks.workunit.client.1.smithi117.stdout:9/281: write d4/d8/f41 [427891,472] 0 2024-03-22T06:15:20.443 INFO:tasks.workunit.client.1.smithi117.stdout:9/282: getdents d4/d46 0 2024-03-22T06:15:20.443 INFO:tasks.workunit.client.1.smithi117.stdout:9/283: write d4/d7/d10/d2f/f4e [616519,60736] 0 2024-03-22T06:15:20.443 INFO:tasks.workunit.client.1.smithi117.stdout:9/284: fsync d4/d8/d37/f4b 0 2024-03-22T06:15:20.443 INFO:tasks.workunit.client.1.smithi117.stdout:9/285: write d4/d7/d10/f28 [2336990,91746] 0 2024-03-22T06:15:20.446 INFO:tasks.workunit.client.1.smithi117.stdout:7/214: symlink d8/d9/d37/l40 0 2024-03-22T06:15:20.446 INFO:tasks.workunit.client.0.smithi006.stdout:1/951: dwrite d8/d9/d3a/f11b [0,4194304] 0 2024-03-22T06:15:20.446 INFO:tasks.workunit.client.0.smithi006.stdout:1/952: dread - d8/d9/d13/d39/d60/d8f/d10a/d59/f118 zero size 2024-03-22T06:15:20.449 INFO:tasks.workunit.client.1.smithi117.stdout:1/248: mknod d1/c56 0 2024-03-22T06:15:20.450 INFO:tasks.workunit.client.1.smithi117.stdout:1/249: fdatasync d1/fc 0 2024-03-22T06:15:20.450 INFO:tasks.workunit.client.1.smithi117.stdout:3/197: getdents d9/d2e/d36 0 2024-03-22T06:15:20.450 INFO:tasks.workunit.client.1.smithi117.stdout:3/198: stat d9/c14 0 2024-03-22T06:15:20.459 INFO:tasks.workunit.client.1.smithi117.stdout:1/250: symlink d1/d22/l57 0 2024-03-22T06:15:20.459 INFO:tasks.workunit.client.1.smithi117.stdout:1/251: write d1/f7 [3612722,40180] 0 2024-03-22T06:15:20.459 INFO:tasks.workunit.client.1.smithi117.stdout:1/252: chown d1/fd 14 1 2024-03-22T06:15:20.459 INFO:tasks.workunit.client.1.smithi117.stdout:1/253: write d1/f43 [2927282,1590] 0 2024-03-22T06:15:20.459 INFO:tasks.workunit.client.1.smithi117.stdout:1/254: write d1/fd [387476,13100] 0 2024-03-22T06:15:20.459 INFO:tasks.workunit.client.1.smithi117.stdout:3/199: dread d9/d1a/f2d [0,4194304] 0 2024-03-22T06:15:20.464 INFO:tasks.workunit.client.1.smithi117.stdout:1/255: mkdir d1/d30/d35/d58 0 2024-03-22T06:15:20.467 INFO:tasks.workunit.client.1.smithi117.stdout:3/200: rename f8 to d9/d1a/f4b 0 2024-03-22T06:15:20.467 INFO:tasks.workunit.client.1.smithi117.stdout:3/201: read - d9/f35 zero size 2024-03-22T06:15:20.469 INFO:tasks.workunit.client.1.smithi117.stdout:1/256: unlink d1/d22/d27/f28 0 2024-03-22T06:15:20.470 INFO:tasks.workunit.client.0.smithi006.stdout:7/945: dwrite d5/d1e/d29/d59/d5c/fb1 [0,4194304] 0 2024-03-22T06:15:20.470 INFO:tasks.workunit.client.0.smithi006.stdout:7/946: truncate d5/d1e/d29/d59/d5c/fc7 24907 0 2024-03-22T06:15:20.470 INFO:tasks.workunit.client.0.smithi006.stdout:7/947: stat d5/d1e/d29/d1f/d2d 0 2024-03-22T06:15:20.471 INFO:tasks.workunit.client.1.smithi117.stdout:3/202: unlink d9/d2e/f49 0 2024-03-22T06:15:20.472 INFO:tasks.workunit.client.1.smithi117.stdout:7/215: dwrite d8/f16 [4194304,4194304] 0 2024-03-22T06:15:20.474 INFO:tasks.workunit.client.1.smithi117.stdout:9/286: dwrite d4/d8/f25 [0,4194304] 0 2024-03-22T06:15:20.474 INFO:tasks.workunit.client.1.smithi117.stdout:9/287: readlink d4/d7/l32 0 2024-03-22T06:15:20.474 INFO:tasks.workunit.client.1.smithi117.stdout:9/288: write d4/d7/f2d [2213203,77674] 0 2024-03-22T06:15:20.475 INFO:tasks.workunit.client.1.smithi117.stdout:2/247: dwrite d2/f4 [0,4194304] 0 2024-03-22T06:15:20.476 INFO:tasks.workunit.client.1.smithi117.stdout:1/257: dread d1/f13 [0,4194304] 0 2024-03-22T06:15:20.476 INFO:tasks.workunit.client.1.smithi117.stdout:2/248: dread d2/d5/f33 [0,4194304] 0 2024-03-22T06:15:20.477 INFO:tasks.workunit.client.1.smithi117.stdout:2/249: dread d2/d5/d8/f22 [0,4194304] 0 2024-03-22T06:15:20.483 INFO:tasks.workunit.client.0.smithi006.stdout:1/953: mknod d8/d9/d13/d39/d60/d8f/c136 0 2024-03-22T06:15:20.484 INFO:tasks.workunit.client.1.smithi117.stdout:2/250: dread d2/d5/d13/d3f/f1f [0,4194304] 0 2024-03-22T06:15:20.487 INFO:tasks.workunit.client.1.smithi117.stdout:9/289: mkdir d4/d8/d53 0 2024-03-22T06:15:20.487 INFO:tasks.workunit.client.1.smithi117.stdout:9/290: readlink d4/l40 0 2024-03-22T06:15:20.487 INFO:tasks.workunit.client.1.smithi117.stdout:9/291: truncate d4/d47/f4d 391130 0 2024-03-22T06:15:20.487 INFO:tasks.workunit.client.1.smithi117.stdout:9/292: getdents d4/d46 0 2024-03-22T06:15:20.487 INFO:tasks.workunit.client.1.smithi117.stdout:9/293: creat d4/d7/d10/d2f/f54 x:0 0 0 2024-03-22T06:15:20.488 INFO:tasks.workunit.client.0.smithi006.stdout:1/954: mknod d8/d9/d13/d24/d32/d34/d132/c137 0 2024-03-22T06:15:20.489 INFO:tasks.workunit.client.0.smithi006.stdout:7/948: rmdir d5/d1e/d29/d1f/d2d/d41 39 2024-03-22T06:15:20.491 INFO:tasks.workunit.client.0.smithi006.stdout:1/955: mknod d8/d9/d13/d24/d32/dc0/c138 0 2024-03-22T06:15:20.494 INFO:tasks.workunit.client.0.smithi006.stdout:7/949: link d5/d1e/d63/d85/fad d5/d1e/d29/d1f/d2d/d61/daa/f13b 0 2024-03-22T06:15:20.494 INFO:tasks.workunit.client.0.smithi006.stdout:7/950: truncate d5/d1e/d63/f11c 31054 0 2024-03-22T06:15:20.494 INFO:tasks.workunit.client.0.smithi006.stdout:7/951: fsync d5/d1e/d63/d85/d80/ff8 0 2024-03-22T06:15:20.495 INFO:tasks.workunit.client.0.smithi006.stdout:1/956: getdents d8/d9/d13/d39/d60/d8f/d10a/d59 0 2024-03-22T06:15:20.496 INFO:tasks.workunit.client.0.smithi006.stdout:1/957: write d8/d9/d13/d39/d60/d8f/d10a/d44/f66 [3722562,10535] 0 2024-03-22T06:15:20.496 INFO:tasks.workunit.client.0.smithi006.stdout:1/958: chown d8/d9/d13/lf9 1121 1 2024-03-22T06:15:20.496 INFO:tasks.workunit.client.0.smithi006.stdout:1/959: chown d8/d9/d13/d24/d32/d34/f77 6647963 1 2024-03-22T06:15:20.500 INFO:tasks.workunit.client.0.smithi006.stdout:7/952: symlink d5/l13c 0 2024-03-22T06:15:20.504 INFO:tasks.workunit.client.0.smithi006.stdout:1/960: creat d8/d9/d3a/d18/dd2/f139 x:0 0 0 2024-03-22T06:15:20.505 INFO:tasks.workunit.client.0.smithi006.stdout:7/953: mknod d5/d1e/d29/d10/d6f/c13d 0 2024-03-22T06:15:20.506 INFO:tasks.workunit.client.1.smithi117.stdout:2/251: mkdir d2/d1b/d39/d45 0 2024-03-22T06:15:20.508 INFO:tasks.workunit.client.1.smithi117.stdout:1/258: dread d1/f43 [0,4194304] 0 2024-03-22T06:15:20.508 INFO:tasks.workunit.client.0.smithi006.stdout:7/954: link d5/d1e/d29/d1f/d2d/f4c d5/d1e/d29/d66/d91/ddc/dfa/d121/d12b/f13e 0 2024-03-22T06:15:20.510 INFO:tasks.workunit.client.1.smithi117.stdout:9/294: creat d4/d46/f55 x:0 0 0 2024-03-22T06:15:20.510 INFO:tasks.workunit.client.1.smithi117.stdout:9/295: write d4/d47/f4d [1053380,107961] 0 2024-03-22T06:15:20.512 INFO:tasks.workunit.client.0.smithi006.stdout:1/961: write d8/d9/d3a/d18/ffe [2109978,12145] 0 2024-03-22T06:15:20.513 INFO:tasks.workunit.client.1.smithi117.stdout:3/203: rename d9/d2e to d9/d1a/d39/d4c 0 2024-03-22T06:15:20.513 INFO:tasks.workunit.client.1.smithi117.stdout:3/204: chown d9/f35 281294 1 2024-03-22T06:15:20.515 INFO:tasks.workunit.client.1.smithi117.stdout:2/252: rmdir d2/d1b/d39 39 2024-03-22T06:15:20.516 INFO:tasks.workunit.client.1.smithi117.stdout:1/259: dread - d1/d1f/f3e zero size 2024-03-22T06:15:20.518 INFO:tasks.workunit.client.1.smithi117.stdout:9/296: creat d4/d7/d27/f56 x:0 0 0 2024-03-22T06:15:20.521 INFO:tasks.workunit.client.1.smithi117.stdout:9/297: dread d4/d8/f13 [0,4194304] 0 2024-03-22T06:15:20.521 INFO:tasks.workunit.client.1.smithi117.stdout:3/205: unlink d9/c14 0 2024-03-22T06:15:20.521 INFO:tasks.workunit.client.1.smithi117.stdout:3/206: fsync d9/d1a/f1e 0 2024-03-22T06:15:20.521 INFO:tasks.workunit.client.1.smithi117.stdout:3/207: write d9/f30 [3437017,30987] 0 2024-03-22T06:15:20.522 INFO:tasks.workunit.client.1.smithi117.stdout:3/208: write d9/d1a/f2d [1001339,75461] 0 2024-03-22T06:15:20.524 INFO:tasks.workunit.client.1.smithi117.stdout:2/253: creat d2/d5/d13/d3f/d1d/f46 x:0 0 0 2024-03-22T06:15:20.524 INFO:tasks.workunit.client.1.smithi117.stdout:2/254: creat d2/d5/d8/f47 x:0 0 0 2024-03-22T06:15:20.524 INFO:tasks.workunit.client.1.smithi117.stdout:2/255: write d2/d5/d8/f47 [454213,99505] 0 2024-03-22T06:15:20.524 INFO:tasks.workunit.client.1.smithi117.stdout:2/256: stat d2/d5/d13/d3f/d1d/d3a/c29 0 2024-03-22T06:15:20.524 INFO:tasks.workunit.client.1.smithi117.stdout:2/257: chown d2/d5/db 306 1 2024-03-22T06:15:20.525 INFO:tasks.workunit.client.0.smithi006.stdout:7/955: unlink d5/l64 0 2024-03-22T06:15:20.525 INFO:tasks.workunit.client.0.smithi006.stdout:7/956: fsync d5/d1e/d63/d85/d80/ff8 0 2024-03-22T06:15:20.526 INFO:tasks.workunit.client.1.smithi117.stdout:2/258: dread d2/d5/d13/f30 [0,4194304] 0 2024-03-22T06:15:20.526 INFO:tasks.workunit.client.1.smithi117.stdout:2/259: write d2/d5/d8/f47 [544692,122711] 0 2024-03-22T06:15:20.526 INFO:tasks.workunit.client.1.smithi117.stdout:0/258: sync 2024-03-22T06:15:20.526 INFO:tasks.workunit.client.1.smithi117.stdout:6/283: sync 2024-03-22T06:15:20.526 INFO:tasks.workunit.client.1.smithi117.stdout:6/284: creat d3/d40/d24/f5e x:0 0 0 2024-03-22T06:15:20.526 INFO:tasks.workunit.client.1.smithi117.stdout:6/285: write d3/d7/f1b [1060859,124461] 0 2024-03-22T06:15:20.527 INFO:tasks.workunit.client.1.smithi117.stdout:1/260: mknod d1/d1f/d32/c59 0 2024-03-22T06:15:20.528 INFO:tasks.workunit.client.0.smithi006.stdout:1/962: rename d8/d9/d13/d39/l43 to d8/d9/d10/d7c/l13a 0 2024-03-22T06:15:20.528 INFO:tasks.workunit.client.0.smithi006.stdout:1/963: dread - d8/d9/d13/d39/d60/d8f/d10a/d59/f118 zero size 2024-03-22T06:15:20.530 INFO:tasks.workunit.client.0.smithi006.stdout:1/964: dread d8/d9/d13/d24/f104 [0,4194304] 0 2024-03-22T06:15:20.538 INFO:tasks.workunit.client.0.smithi006.stdout:7/957: unlink d5/d1e/d29/d1f/l27 0 2024-03-22T06:15:20.541 INFO:tasks.workunit.client.1.smithi117.stdout:2/260: mknod d2/d5/d13/d3f/d1d/d3a/c48 0 2024-03-22T06:15:20.543 INFO:tasks.workunit.client.0.smithi006.stdout:6/920: sync 2024-03-22T06:15:20.543 INFO:tasks.workunit.client.0.smithi006.stdout:7/958: creat d5/d1e/d29/d66/d91/ddc/dfa/d121/f13f x:0 0 0 2024-03-22T06:15:20.543 INFO:tasks.workunit.client.0.smithi006.stdout:6/921: truncate d6/d1b/f3c 4594279 0 2024-03-22T06:15:20.543 INFO:tasks.workunit.client.0.smithi006.stdout:6/922: truncate d6/d1d/d32/f45 1558435 0 2024-03-22T06:15:20.545 INFO:tasks.workunit.client.0.smithi006.stdout:1/965: mkdir d8/d9/d13/d24/d13b 0 2024-03-22T06:15:20.545 INFO:tasks.workunit.client.0.smithi006.stdout:1/966: read - d8/d9/d13/d39/d60/d8f/d10a/d44/d80/f129 zero size 2024-03-22T06:15:20.547 INFO:tasks.workunit.client.0.smithi006.stdout:7/959: truncate d5/d1e/d63/d85/dca/f101 1488094 0 2024-03-22T06:15:20.548 INFO:tasks.workunit.client.1.smithi117.stdout:6/286: unlink d3/d7/f57 0 2024-03-22T06:15:20.552 INFO:tasks.workunit.client.0.smithi006.stdout:7/960: creat d5/d1e/d29/d59/d5c/f140 x:0 0 0 2024-03-22T06:15:20.553 INFO:tasks.workunit.client.0.smithi006.stdout:9/969: sync 2024-03-22T06:15:20.553 INFO:tasks.workunit.client.0.smithi006.stdout:0/837: sync 2024-03-22T06:15:20.554 INFO:tasks.workunit.client.0.smithi006.stdout:0/838: chown db/d10/d35/d6c 1105 1 2024-03-22T06:15:20.554 INFO:tasks.workunit.client.0.smithi006.stdout:9/970: read - d8/d10/d28/fdb zero size 2024-03-22T06:15:20.554 INFO:tasks.workunit.client.0.smithi006.stdout:9/971: chown d8/d10/d47/dfc/f116 97775 1 2024-03-22T06:15:20.554 INFO:tasks.workunit.client.0.smithi006.stdout:9/972: chown d8/d1f/f139 4755 1 2024-03-22T06:15:20.554 INFO:tasks.workunit.client.0.smithi006.stdout:9/973: fsync d8/db3/d50/f61 0 2024-03-22T06:15:20.555 INFO:tasks.workunit.client.1.smithi117.stdout:2/261: mknod d2/d5/d13/c49 0 2024-03-22T06:15:20.555 INFO:tasks.workunit.client.1.smithi117.stdout:9/298: dwrite d4/d7/f19 [0,4194304] 0 2024-03-22T06:15:20.555 INFO:tasks.workunit.client.1.smithi117.stdout:9/299: chown d4/f21 248 1 2024-03-22T06:15:20.555 INFO:tasks.workunit.client.1.smithi117.stdout:1/261: dwrite d1/d1f/f3e [0,4194304] 0 2024-03-22T06:15:20.555 INFO:tasks.workunit.client.1.smithi117.stdout:1/262: truncate d1/d1f/f20 4522053 0 2024-03-22T06:15:20.555 INFO:tasks.workunit.client.1.smithi117.stdout:1/263: creat d1/d1f/f5a x:0 0 0 2024-03-22T06:15:20.556 INFO:tasks.workunit.client.0.smithi006.stdout:1/967: rmdir d8/d9/d13/d39/d60/d8f/dea 39 2024-03-22T06:15:20.556 INFO:tasks.workunit.client.1.smithi117.stdout:3/209: dwrite d9/d1a/d3a/f40 [0,4194304] 0 2024-03-22T06:15:20.556 INFO:tasks.workunit.client.1.smithi117.stdout:3/210: stat d9/d1a 0 2024-03-22T06:15:20.558 INFO:tasks.workunit.client.1.smithi117.stdout:0/259: dwrite d1/d10/d5c/d40/f41 [0,4194304] 0 2024-03-22T06:15:20.558 INFO:tasks.workunit.client.1.smithi117.stdout:6/287: symlink d3/d40/d24/d2d/d30/d3e/l5f 0 2024-03-22T06:15:20.560 INFO:tasks.workunit.client.0.smithi006.stdout:6/923: write d6/d24/f38 [4065903,61463] 0 2024-03-22T06:15:20.565 INFO:tasks.workunit.client.1.smithi117.stdout:9/300: creat d4/d7/d10/d3b/f57 x:0 0 0 2024-03-22T06:15:20.570 INFO:tasks.workunit.client.0.smithi006.stdout:9/974: getdents d8/db3/d50 0 2024-03-22T06:15:20.570 INFO:tasks.workunit.client.0.smithi006.stdout:1/968: dread d8/d71/f8e [0,4194304] 0 2024-03-22T06:15:20.570 INFO:tasks.workunit.client.0.smithi006.stdout:1/969: fsync d8/d9/d3a/f111 0 2024-03-22T06:15:20.570 INFO:tasks.workunit.client.0.smithi006.stdout:7/961: dwrite d5/d1e/d29/d1f/d2d/d61/f6d [0,4194304] 0 2024-03-22T06:15:20.570 INFO:tasks.workunit.client.0.smithi006.stdout:7/962: stat d5/d1e/fcc 0 2024-03-22T06:15:20.578 INFO:tasks.workunit.client.0.smithi006.stdout:0/839: dwrite db/d10/d35/d7b/da1/da6/fd2 [0,4194304] 0 2024-03-22T06:15:20.578 INFO:tasks.workunit.client.0.smithi006.stdout:9/975: mkdir d8/db3/d50/d91/dfe/d12c/d13c 0 2024-03-22T06:15:20.579 INFO:tasks.workunit.client.1.smithi117.stdout:1/264: fsync d1/f43 0 2024-03-22T06:15:20.584 INFO:tasks.workunit.client.0.smithi006.stdout:7/963: mkdir d5/d1e/d29/d141 0 2024-03-22T06:15:20.585 INFO:tasks.workunit.client.0.smithi006.stdout:7/964: read - d5/d1e/d3f/f136 zero size 2024-03-22T06:15:20.585 INFO:tasks.workunit.client.1.smithi117.stdout:3/211: rmdir d9/d1a/d39/d4c/d37 39 2024-03-22T06:15:20.593 INFO:tasks.workunit.client.0.smithi006.stdout:1/970: dwrite d8/d9/d13/d39/d60/d8f/d10a/d44/f66 [0,4194304] 0 2024-03-22T06:15:20.593 INFO:tasks.workunit.client.0.smithi006.stdout:1/971: read - d8/d9/d13/d24/f122 zero size 2024-03-22T06:15:20.596 INFO:tasks.workunit.client.0.smithi006.stdout:7/965: read d5/f9 [1308220,68912] 0 2024-03-22T06:15:20.596 INFO:tasks.workunit.client.0.smithi006.stdout:7/966: dread - d5/d1e/d29/d1f/d2d/df7/f11a zero size 2024-03-22T06:15:20.596 INFO:tasks.workunit.client.0.smithi006.stdout:7/967: truncate d5/d1e/d29/d66/d91/ddc/dfa/d121/f13f 879254 0 2024-03-22T06:15:20.596 INFO:tasks.workunit.client.0.smithi006.stdout:7/968: dread - d5/d1e/fba zero size 2024-03-22T06:15:20.596 INFO:tasks.workunit.client.0.smithi006.stdout:7/969: truncate d5/d1e/d29/d1f/d2d/d55/f108 887087 0 2024-03-22T06:15:20.596 INFO:tasks.workunit.client.0.smithi006.stdout:7/970: readlink d5/la 0 2024-03-22T06:15:20.597 INFO:tasks.workunit.client.0.smithi006.stdout:0/840: link db/d3d/cb8 db/d10/d33/dce/df9/c123 0 2024-03-22T06:15:20.597 INFO:tasks.workunit.client.0.smithi006.stdout:6/924: rmdir d6/d24/df0 0 2024-03-22T06:15:20.598 INFO:tasks.workunit.client.0.smithi006.stdout:1/972: creat d8/d9/d13/d24/d32/f13c x:0 0 0 2024-03-22T06:15:20.598 INFO:tasks.workunit.client.0.smithi006.stdout:1/973: truncate d8/d9/d3a/f111 1022286 0 2024-03-22T06:15:20.599 INFO:tasks.workunit.client.0.smithi006.stdout:9/976: dwrite d8/db3/d50/d91/dfe/d11a/fc6 [0,4194304] 0 2024-03-22T06:15:20.599 INFO:tasks.workunit.client.0.smithi006.stdout:6/925: truncate d6/d24/d44/f46 3094351 0 2024-03-22T06:15:20.601 INFO:tasks.workunit.client.0.smithi006.stdout:7/971: rename d5/d1e/d29/d1f/d2d/d55/d84/lef to d5/d1e/d29/d1f/de4/l142 0 2024-03-22T06:15:20.601 INFO:tasks.workunit.client.1.smithi117.stdout:3/212: dwrite d9/d1a/f34 [0,4194304] 0 2024-03-22T06:15:20.605 INFO:tasks.workunit.client.1.smithi117.stdout:0/260: creat d1/d3/d2b/d3e/de/d1f/d3b/d55/f5e x:0 0 0 2024-03-22T06:15:20.605 INFO:tasks.workunit.client.1.smithi117.stdout:0/261: dread - d1/d3/d2b/d3e/de/d1f/d3b/d55/f5e zero size 2024-03-22T06:15:20.605 INFO:tasks.workunit.client.1.smithi117.stdout:0/262: fsync d1/f3d 0 2024-03-22T06:15:20.606 INFO:tasks.workunit.client.1.smithi117.stdout:0/263: read d1/d10/f11 [9260108,110580] 0 2024-03-22T06:15:20.606 INFO:tasks.workunit.client.1.smithi117.stdout:0/264: chown d1/d10/d18/d26/dc/cd 12906 1 2024-03-22T06:15:20.606 INFO:tasks.workunit.client.1.smithi117.stdout:6/288: truncate d3/d40/d24/d2d/d30/d3e/d49/f5b 3879040 0 2024-03-22T06:15:20.606 INFO:tasks.workunit.client.0.smithi006.stdout:6/926: dread d6/d7c/d83/fa2 [0,4194304] 0 2024-03-22T06:15:20.607 INFO:tasks.workunit.client.1.smithi117.stdout:0/265: dread d1/d10/d18/d26/dc/d39/f52 [0,4194304] 0 2024-03-22T06:15:20.607 INFO:tasks.workunit.client.1.smithi117.stdout:0/266: chown d1/d3/l20 2262476 1 2024-03-22T06:15:20.607 INFO:tasks.workunit.client.1.smithi117.stdout:0/267: write d1/d3/d2b/f2e [290054,95189] 0 2024-03-22T06:15:20.608 INFO:tasks.workunit.client.0.smithi006.stdout:9/977: creat d8/d10/f13d x:0 0 0 2024-03-22T06:15:20.608 INFO:tasks.workunit.client.0.smithi006.stdout:9/978: stat d8/db3/d50/f61 0 2024-03-22T06:15:20.608 INFO:tasks.workunit.client.1.smithi117.stdout:2/262: getdents d2/d5/d13/d3f/d1d 0 2024-03-22T06:15:20.610 INFO:tasks.workunit.client.1.smithi117.stdout:4/248: sync 2024-03-22T06:15:20.610 INFO:tasks.workunit.client.1.smithi117.stdout:4/249: truncate d0/d19/f4a 53937 0 2024-03-22T06:15:20.610 INFO:tasks.workunit.client.1.smithi117.stdout:7/216: sync 2024-03-22T06:15:20.610 INFO:tasks.workunit.client.1.smithi117.stdout:8/236: sync 2024-03-22T06:15:20.610 INFO:tasks.workunit.client.1.smithi117.stdout:5/257: sync 2024-03-22T06:15:20.610 INFO:tasks.workunit.client.1.smithi117.stdout:5/258: chown dd/d16/d24/d36/f25 473221 1 2024-03-22T06:15:20.611 INFO:tasks.workunit.client.1.smithi117.stdout:7/217: dread - d8/d9/da/d10/d23/f38 zero size 2024-03-22T06:15:20.611 INFO:tasks.workunit.client.1.smithi117.stdout:4/250: write d0/d4/d2a/f30 [82506,109471] 0 2024-03-22T06:15:20.611 INFO:tasks.workunit.client.1.smithi117.stdout:4/251: fdatasync d0/d19/f4a 0 2024-03-22T06:15:20.612 INFO:tasks.workunit.client.1.smithi117.stdout:9/301: mkdir d4/d8/d37/d58 0 2024-03-22T06:15:20.612 INFO:tasks.workunit.client.1.smithi117.stdout:9/302: fsync d4/d7/d10/f28 0 2024-03-22T06:15:20.612 INFO:tasks.workunit.client.1.smithi117.stdout:9/303: fdatasync d4/d7/d10/d2f/f39 0 2024-03-22T06:15:20.613 INFO:tasks.workunit.client.0.smithi006.stdout:7/972: unlink d5/d1e/d3f/f90 0 2024-03-22T06:15:20.616 INFO:tasks.workunit.client.0.smithi006.stdout:6/927: creat d6/d1d/d99/dcd/f11b x:0 0 0 2024-03-22T06:15:20.616 INFO:tasks.workunit.client.1.smithi117.stdout:6/289: read d3/d7/da/f29 [1487074,53594] 0 2024-03-22T06:15:20.621 INFO:tasks.workunit.client.1.smithi117.stdout:5/259: dread dd/d16/d1a/d32/f40 [0,4194304] 0 2024-03-22T06:15:20.623 INFO:tasks.workunit.client.1.smithi117.stdout:3/213: symlink d9/d1a/d39/d4c/d36/l4d 0 2024-03-22T06:15:20.626 INFO:tasks.workunit.client.0.smithi006.stdout:7/973: symlink d5/d1e/d29/d1f/de4/l143 0 2024-03-22T06:15:20.626 INFO:tasks.workunit.client.0.smithi006.stdout:7/974: chown d5/d1e/d29/d1f/d2d/fda 0 1 2024-03-22T06:15:20.626 INFO:tasks.workunit.client.0.smithi006.stdout:7/975: stat d5/d1e/d29/d1f/d2d/c82 0 2024-03-22T06:15:20.626 INFO:tasks.workunit.client.0.smithi006.stdout:7/976: fdatasync d5/d1e/d63/d85/d80/ff8 0 2024-03-22T06:15:20.628 INFO:tasks.workunit.client.0.smithi006.stdout:6/928: truncate d6/d24/d44/f46 2056034 0 2024-03-22T06:15:20.629 INFO:tasks.workunit.client.0.smithi006.stdout:6/929: getdents d6/d1d/d5c/d81/d8a 0 2024-03-22T06:15:20.630 INFO:tasks.workunit.client.1.smithi117.stdout:2/263: creat d2/f4a x:0 0 0 2024-03-22T06:15:20.633 INFO:tasks.workunit.client.1.smithi117.stdout:5/260: dread dd/d16/d1a/f37 [0,4194304] 0 2024-03-22T06:15:20.633 INFO:tasks.workunit.client.0.smithi006.stdout:0/841: rename db/dc/d37/dfe to db/d3d/d4e/d114/d124 0 2024-03-22T06:15:20.634 INFO:tasks.workunit.client.1.smithi117.stdout:8/237: creat d1/d48/f4d x:0 0 0 2024-03-22T06:15:20.636 INFO:tasks.workunit.client.0.smithi006.stdout:7/977: mkdir d5/d1e/d29/d1f/d144 0 2024-03-22T06:15:20.637 INFO:tasks.workunit.client.1.smithi117.stdout:7/218: mkdir d8/d9/da/d10/d23/d41 0 2024-03-22T06:15:20.639 INFO:tasks.workunit.client.0.smithi006.stdout:1/974: rename d8/d9/d13/lb8 to d8/d9/d3a/d18/l13d 0 2024-03-22T06:15:20.641 INFO:tasks.workunit.client.0.smithi006.stdout:6/930: dread d6/fc1 [0,4194304] 0 2024-03-22T06:15:20.641 INFO:tasks.workunit.client.0.smithi006.stdout:0/842: mknod db/d10/d35/d7b/c125 0 2024-03-22T06:15:20.641 INFO:tasks.workunit.client.0.smithi006.stdout:0/843: chown db/d3d 14905973 1 2024-03-22T06:15:20.642 INFO:tasks.workunit.client.0.smithi006.stdout:0/844: chown db/d10/d2a/d41/cef 37828 1 2024-03-22T06:15:20.642 INFO:tasks.workunit.client.1.smithi117.stdout:6/290: mkdir d3/d3d/d17/d60 0 2024-03-22T06:15:20.643 INFO:tasks.workunit.client.0.smithi006.stdout:7/978: creat d5/d1e/d29/d66/d91/d12d/f145 x:0 0 0 2024-03-22T06:15:20.644 INFO:tasks.workunit.client.1.smithi117.stdout:3/214: rename d9/d1a/d3a/f40 to d9/d1a/d39/d4c/f4e 0 2024-03-22T06:15:20.644 INFO:tasks.workunit.client.0.smithi006.stdout:1/975: mknod d8/d9/d13/d39/d60/d8f/d10a/d59/c13e 0 2024-03-22T06:15:20.645 INFO:tasks.workunit.client.1.smithi117.stdout:0/268: rmdir d1/d10/d18/d26/dc/d39 39 2024-03-22T06:15:20.645 INFO:tasks.workunit.client.0.smithi006.stdout:6/931: rmdir d6/d1d/d5c 39 2024-03-22T06:15:20.646 INFO:tasks.workunit.client.1.smithi117.stdout:2/264: creat d2/d1b/d39/f4b x:0 0 0 2024-03-22T06:15:20.647 INFO:tasks.workunit.client.0.smithi006.stdout:0/845: mknod db/d10/d35/c126 0 2024-03-22T06:15:20.648 INFO:tasks.workunit.client.1.smithi117.stdout:5/261: symlink dd/df/d30/d33/l52 0 2024-03-22T06:15:20.648 INFO:tasks.workunit.client.1.smithi117.stdout:5/262: write dd/d16/d24/d36/f26 [1923638,127186] 0 2024-03-22T06:15:20.649 INFO:tasks.workunit.client.0.smithi006.stdout:6/932: mknod d6/d1b/dae/dbc/d105/c11c 0 2024-03-22T06:15:20.649 INFO:tasks.workunit.client.1.smithi117.stdout:1/265: rmdir d1/d22/d27 39 2024-03-22T06:15:20.650 INFO:tasks.workunit.client.0.smithi006.stdout:0/846: mkdir db/d10/d2a/d127 0 2024-03-22T06:15:20.650 INFO:tasks.workunit.client.0.smithi006.stdout:0/847: creat db/d10/d35/d7b/da1/da6/dc4/f128 x:0 0 0 2024-03-22T06:15:20.651 INFO:tasks.workunit.client.1.smithi117.stdout:6/291: rmdir d3/d40/d46 39 2024-03-22T06:15:20.652 INFO:tasks.workunit.client.1.smithi117.stdout:4/252: dwrite d0/d4/d2a/d2f/f31 [0,4194304] 0 2024-03-22T06:15:20.652 INFO:tasks.workunit.client.1.smithi117.stdout:4/253: write d0/d19/d34/f37 [292779,85188] 0 2024-03-22T06:15:20.652 INFO:tasks.workunit.client.1.smithi117.stdout:4/254: chown d0/d19 423085 1 2024-03-22T06:15:20.652 INFO:tasks.workunit.client.1.smithi117.stdout:4/255: creat d0/d19/d28/f59 x:0 0 0 2024-03-22T06:15:20.653 INFO:tasks.workunit.client.0.smithi006.stdout:9/979: dwrite d8/d10/d28/fb6 [0,4194304] 0 2024-03-22T06:15:20.653 INFO:tasks.workunit.client.0.smithi006.stdout:7/979: rename d5/d1e/d3f/c11b to d5/d1e/d29/d66/c146 0 2024-03-22T06:15:20.655 INFO:tasks.workunit.client.1.smithi117.stdout:7/219: rename d8/d9/l12 to d8/d9/da/d10/d23/d27/d39/l42 0 2024-03-22T06:15:20.657 INFO:tasks.workunit.client.1.smithi117.stdout:3/215: dread d9/fa [0,4194304] 0 2024-03-22T06:15:20.657 INFO:tasks.workunit.client.0.smithi006.stdout:6/933: creat d6/d1d/d32/d118/f11d x:0 0 0 2024-03-22T06:15:20.662 INFO:tasks.workunit.client.1.smithi117.stdout:5/263: mknod dd/d16/c53 0 2024-03-22T06:15:20.662 INFO:tasks.workunit.client.1.smithi117.stdout:5/264: stat dd/c14 0 2024-03-22T06:15:20.664 INFO:tasks.workunit.client.0.smithi006.stdout:9/980: read f3 [3548277,39913] 0 2024-03-22T06:15:20.664 INFO:tasks.workunit.client.1.smithi117.stdout:9/304: dwrite d4/d7/f1b [4194304,4194304] 0 2024-03-22T06:15:20.665 INFO:tasks.workunit.client.0.smithi006.stdout:0/848: mknod db/dc/d103/c129 0 2024-03-22T06:15:20.671 INFO:tasks.workunit.client.0.smithi006.stdout:1/976: rename d8/d9/c7e to d8/d9/d13/d24/d32/d34/d132/c13f 0 2024-03-22T06:15:20.672 INFO:tasks.workunit.client.0.smithi006.stdout:7/980: unlink d5/d1e/l97 0 2024-03-22T06:15:20.674 INFO:tasks.workunit.client.0.smithi006.stdout:9/981: mknod d8/d10/d28/c13e 0 2024-03-22T06:15:20.675 INFO:tasks.workunit.client.0.smithi006.stdout:9/982: write d8/d10/d11/fee [1262474,87389] 0 2024-03-22T06:15:20.675 INFO:tasks.workunit.client.0.smithi006.stdout:0/849: read db/d10/f1a [672035,44665] 0 2024-03-22T06:15:20.675 INFO:tasks.workunit.client.1.smithi117.stdout:1/266: dread d1/d30/d35/f1a [0,4194304] 0 2024-03-22T06:15:20.675 INFO:tasks.workunit.client.1.smithi117.stdout:1/267: stat d1/d1f/d32/c59 0 2024-03-22T06:15:20.677 INFO:tasks.workunit.client.1.smithi117.stdout:2/265: dwrite d2/d5/d13/d3f/d1d/f46 [0,4194304] 0 2024-03-22T06:15:20.679 INFO:tasks.workunit.client.1.smithi117.stdout:0/269: dwrite d1/d3/d2b/d3e/de/d1f/f3c [0,4194304] 0 2024-03-22T06:15:20.679 INFO:tasks.workunit.client.0.smithi006.stdout:9/983: symlink d8/d10/d47/l13f 0 2024-03-22T06:15:20.680 INFO:tasks.workunit.client.1.smithi117.stdout:4/256: link d0/d4/c29 d0/d3f/c5a 0 2024-03-22T06:15:20.684 INFO:tasks.workunit.client.1.smithi117.stdout:5/265: dwrite dd/d16/d24/d36/f25 [4194304,4194304] 0 2024-03-22T06:15:20.685 INFO:tasks.workunit.client.1.smithi117.stdout:7/220: mknod d8/d9/c43 0 2024-03-22T06:15:20.691 INFO:tasks.workunit.client.1.smithi117.stdout:9/305: creat d4/f59 x:0 0 0 2024-03-22T06:15:20.691 INFO:tasks.workunit.client.1.smithi117.stdout:9/306: creat d4/d7/d27/f5a x:0 0 0 2024-03-22T06:15:20.694 INFO:tasks.workunit.client.1.smithi117.stdout:6/292: mkdir d3/d61 0 2024-03-22T06:15:20.694 INFO:tasks.workunit.client.1.smithi117.stdout:6/293: readlink d3/d40/d24/d2d/d42/l58 0 2024-03-22T06:15:20.696 INFO:tasks.workunit.client.1.smithi117.stdout:5/266: dread dd/df/f2b [0,4194304] 0 2024-03-22T06:15:20.696 INFO:tasks.workunit.client.1.smithi117.stdout:5/267: stat dd/d16/d24/d36/f47 0 2024-03-22T06:15:20.697 INFO:tasks.workunit.client.0.smithi006.stdout:0/850: dwrite db/dc/faa [0,4194304] 0 2024-03-22T06:15:20.697 INFO:tasks.workunit.client.1.smithi117.stdout:6/294: dread d3/d3d/d17/f33 [0,4194304] 0 2024-03-22T06:15:20.701 INFO:tasks.workunit.client.1.smithi117.stdout:0/270: creat d1/d3/d2b/d3e/de/d1f/d59/f5f x:0 0 0 2024-03-22T06:15:20.702 INFO:tasks.workunit.client.1.smithi117.stdout:0/271: chown d1/d10/d18/d26 372 1 2024-03-22T06:15:20.704 INFO:tasks.workunit.client.1.smithi117.stdout:4/257: mkdir d0/d4/d2a/d5b 0 2024-03-22T06:15:20.704 INFO:tasks.workunit.client.1.smithi117.stdout:4/258: fdatasync d0/d3f/f4b 0 2024-03-22T06:15:20.707 INFO:tasks.workunit.client.0.smithi006.stdout:0/851: rename db/d10/d33/dce/ld0 to db/d10/d35/d7b/da1/l12a 0 2024-03-22T06:15:20.709 INFO:tasks.workunit.client.1.smithi117.stdout:3/216: rename c4 to d9/d1a/d39/d4c/c4f 0 2024-03-22T06:15:20.710 INFO:tasks.workunit.client.1.smithi117.stdout:9/307: mkdir d4/d7/d27/d5b 0 2024-03-22T06:15:20.711 INFO:tasks.workunit.client.1.smithi117.stdout:9/308: write d4/d8/d3f/f52 [872663,69390] 0 2024-03-22T06:15:20.713 INFO:tasks.workunit.client.1.smithi117.stdout:5/268: mknod dd/df/d30/c54 0 2024-03-22T06:15:20.713 INFO:tasks.workunit.client.1.smithi117.stdout:5/269: rename dd to dd/df/d30/d55 22 2024-03-22T06:15:20.715 INFO:tasks.workunit.client.1.smithi117.stdout:6/295: link d3/d40/d24/l54 d3/d7/da/d56/l62 0 2024-03-22T06:15:20.716 INFO:tasks.workunit.client.1.smithi117.stdout:0/272: truncate d1/f4f 237939 0 2024-03-22T06:15:20.718 INFO:tasks.workunit.client.0.smithi006.stdout:7/981: dwrite d5/d1e/d29/d1f/d2d/d41/db7/dcb/fd4 [0,4194304] 0 2024-03-22T06:15:20.718 INFO:tasks.workunit.client.0.smithi006.stdout:7/982: stat d5/d1e/d29/d10/d6f/c13d 0 2024-03-22T06:15:20.718 INFO:tasks.workunit.client.0.smithi006.stdout:7/983: fdatasync d5/d1e/d3f/f76 0 2024-03-22T06:15:20.718 INFO:tasks.workunit.client.1.smithi117.stdout:4/259: mkdir d0/d5c 0 2024-03-22T06:15:20.719 INFO:tasks.workunit.client.1.smithi117.stdout:7/221: dwrite d8/d9/da/d10/f25 [0,4194304] 0 2024-03-22T06:15:20.719 INFO:tasks.workunit.client.0.smithi006.stdout:9/984: dwrite d8/d10/d28/dba/f115 [0,4194304] 0 2024-03-22T06:15:20.720 INFO:tasks.workunit.client.0.smithi006.stdout:0/852: dread db/d10/f1b [0,4194304] 0 2024-03-22T06:15:20.720 INFO:tasks.workunit.client.1.smithi117.stdout:9/309: write d4/d8/f25 [3179688,106421] 0 2024-03-22T06:15:20.722 INFO:tasks.workunit.client.1.smithi117.stdout:3/217: creat d9/d1a/d39/d4c/d36/f50 x:0 0 0 2024-03-22T06:15:20.723 INFO:tasks.workunit.client.1.smithi117.stdout:4/260: dread d0/d19/d28/f2e [0,4194304] 0 2024-03-22T06:15:20.724 INFO:tasks.workunit.client.1.smithi117.stdout:1/268: getdents d1/d14 0 2024-03-22T06:15:20.726 INFO:tasks.workunit.client.1.smithi117.stdout:4/261: dread d0/d19/d28/f2e [0,4194304] 0 2024-03-22T06:15:20.726 INFO:tasks.workunit.client.1.smithi117.stdout:4/262: readlink d0/d19/d34/d43/l4e 0 2024-03-22T06:15:20.726 INFO:tasks.workunit.client.1.smithi117.stdout:4/263: write d0/f1d [1017636,53483] 0 2024-03-22T06:15:20.726 INFO:tasks.workunit.client.1.smithi117.stdout:4/264: creat d0/d4/d2a/f5d x:0 0 0 2024-03-22T06:15:20.726 INFO:tasks.workunit.client.1.smithi117.stdout:4/265: dread - d0/d4/d2a/f5d zero size 2024-03-22T06:15:20.726 INFO:tasks.workunit.client.1.smithi117.stdout:4/266: fsync d0/f25 0 2024-03-22T06:15:20.726 INFO:tasks.workunit.client.1.smithi117.stdout:4/267: readlink d0/l1f 0 2024-03-22T06:15:20.726 INFO:tasks.workunit.client.1.smithi117.stdout:4/268: write d0/f1d [761505,28061] 0 2024-03-22T06:15:20.727 INFO:tasks.workunit.client.1.smithi117.stdout:1/269: dread d1/d22/d27/f34 [0,4194304] 0 2024-03-22T06:15:20.728 INFO:tasks.workunit.client.1.smithi117.stdout:5/270: unlink dd/df/f1b 0 2024-03-22T06:15:20.732 INFO:tasks.workunit.client.0.smithi006.stdout:7/984: dread d5/d1e/d3f/f7d [0,4194304] 0 2024-03-22T06:15:20.732 INFO:tasks.workunit.client.1.smithi117.stdout:6/296: mknod d3/d7/da/d56/d3a/c63 0 2024-03-22T06:15:20.734 INFO:tasks.workunit.client.1.smithi117.stdout:0/273: creat d1/d3/d2b/d3e/f60 x:0 0 0 2024-03-22T06:15:20.736 INFO:tasks.workunit.client.1.smithi117.stdout:0/274: dread d1/d10/f27 [0,4194304] 0 2024-03-22T06:15:20.736 INFO:tasks.workunit.client.1.smithi117.stdout:0/275: creat d1/d3/d2b/d3e/de/d1f/d3b/f61 x:0 0 0 2024-03-22T06:15:20.736 INFO:tasks.workunit.client.1.smithi117.stdout:0/276: write d1/d3/d2b/d3e/de/d1f/f33 [246173,31120] 0 2024-03-22T06:15:20.736 INFO:tasks.workunit.client.0.smithi006.stdout:9/985: symlink d8/d10/d47/dfc/l140 0 2024-03-22T06:15:20.739 INFO:tasks.workunit.client.0.smithi006.stdout:0/853: creat db/d10/d33/f12b x:0 0 0 2024-03-22T06:15:20.739 INFO:tasks.workunit.client.0.smithi006.stdout:0/854: readlink db/dc/d38/lbe 0 2024-03-22T06:15:20.739 INFO:tasks.workunit.client.0.smithi006.stdout:0/855: fdatasync db/d87/ddd/f101 0 2024-03-22T06:15:20.739 INFO:tasks.workunit.client.0.smithi006.stdout:0/856: fdatasync db/d3d/d4e/f7d 0 2024-03-22T06:15:20.740 INFO:tasks.workunit.client.1.smithi117.stdout:7/222: rmdir d8/d9/da/d10/d23 39 2024-03-22T06:15:20.741 INFO:tasks.workunit.client.0.smithi006.stdout:7/985: mknod d5/d1e/d29/d10/c147 0 2024-03-22T06:15:20.742 INFO:tasks.workunit.client.1.smithi117.stdout:3/218: dwrite d9/f30 [0,4194304] 0 2024-03-22T06:15:20.743 INFO:tasks.workunit.client.0.smithi006.stdout:9/986: symlink d8/d10/d82/l141 0 2024-03-22T06:15:20.746 INFO:tasks.workunit.client.0.smithi006.stdout:0/857: rmdir db/dc 39 2024-03-22T06:15:20.749 INFO:tasks.workunit.client.0.smithi006.stdout:9/987: creat d8/d10/d28/dc2/f142 x:0 0 0 2024-03-22T06:15:20.752 INFO:tasks.workunit.client.1.smithi117.stdout:4/269: mknod d0/d48/c5e 0 2024-03-22T06:15:20.753 INFO:tasks.workunit.client.0.smithi006.stdout:0/858: mkdir db/d10/d35/d7b/da1/da6/dc4/dfc/d12c 0 2024-03-22T06:15:20.759 INFO:tasks.workunit.client.0.smithi006.stdout:9/988: rename d8/db3/d50/d91/dfe/d12c/f88 to d8/d10/d20/d45/d4d/d11d/f143 0 2024-03-22T06:15:20.759 INFO:tasks.workunit.client.0.smithi006.stdout:9/989: dread - d8/d10/d47/fb2 zero size 2024-03-22T06:15:20.760 INFO:tasks.workunit.client.0.smithi006.stdout:0/859: mknod db/d10/d2a/d41/c12d 0 2024-03-22T06:15:20.761 INFO:tasks.workunit.client.1.smithi117.stdout:6/297: truncate d3/d7/f1b 877527 0 2024-03-22T06:15:20.765 INFO:tasks.workunit.client.0.smithi006.stdout:9/990: rename d8/d10/d20/d103/d120 to d8/d10/d28/dba/d144 0 2024-03-22T06:15:20.766 INFO:tasks.workunit.client.0.smithi006.stdout:7/986: dwrite d5/d1e/d63/f12a [0,4194304] 0 2024-03-22T06:15:20.768 INFO:tasks.workunit.client.1.smithi117.stdout:0/277: link d1/d10/d18/d26/c4c d1/d10/d18/c62 0 2024-03-22T06:15:20.768 INFO:tasks.workunit.client.1.smithi117.stdout:7/223: dwrite d8/d9/da/d10/f19 [0,4194304] 0 2024-03-22T06:15:20.770 INFO:tasks.workunit.client.1.smithi117.stdout:4/270: dwrite d0/d4/f17 [0,4194304] 0 2024-03-22T06:15:20.773 INFO:tasks.workunit.client.1.smithi117.stdout:3/219: mknod d9/d1a/d39/d4c/d37/c51 0 2024-03-22T06:15:20.774 INFO:tasks.workunit.client.0.smithi006.stdout:9/991: write f1 [1693188,14838] 0 2024-03-22T06:15:20.774 INFO:tasks.workunit.client.0.smithi006.stdout:9/992: chown d8/d10/d82/fc5 3840 1 2024-03-22T06:15:20.774 INFO:tasks.workunit.client.0.smithi006.stdout:7/987: creat d5/d1e/d29/d1f/d2d/d41/db7/f148 x:0 0 0 2024-03-22T06:15:20.775 INFO:tasks.workunit.client.1.smithi117.stdout:9/310: getdents d4/d7/d10 0 2024-03-22T06:15:20.775 INFO:tasks.workunit.client.1.smithi117.stdout:9/311: readlink d4/d7/d10/d3b/l43 0 2024-03-22T06:15:20.779 INFO:tasks.workunit.client.1.smithi117.stdout:7/224: dread d8/d9/da/d10/d23/f24 [0,4194304] 0 2024-03-22T06:15:20.782 INFO:tasks.workunit.client.1.smithi117.stdout:4/271: dread d0/d19/d28/f2c [0,4194304] 0 2024-03-22T06:15:20.783 INFO:tasks.workunit.client.1.smithi117.stdout:1/270: mkdir d1/d3c/d4d/d5b 0 2024-03-22T06:15:20.783 INFO:tasks.workunit.client.1.smithi117.stdout:1/271: fsync d1/d30/d35/f1a 0 2024-03-22T06:15:20.783 INFO:tasks.workunit.client.1.smithi117.stdout:1/272: write d1/d1f/f20 [5020207,121934] 0 2024-03-22T06:15:20.785 INFO:tasks.workunit.client.1.smithi117.stdout:6/298: truncate d3/d7/da/f29 1594229 0 2024-03-22T06:15:20.785 INFO:tasks.workunit.client.0.smithi006.stdout:7/988: mkdir d5/d1e/d29/d10/d149 0 2024-03-22T06:15:20.786 INFO:tasks.workunit.client.0.smithi006.stdout:7/989: write d5/d1e/d29/d1f/d2d/d41/fd2 [591669,76983] 0 2024-03-22T06:15:20.787 INFO:tasks.workunit.client.1.smithi117.stdout:0/278: mknod d1/d3/d2b/d3e/de/d1f/d59/c63 0 2024-03-22T06:15:20.789 INFO:tasks.workunit.client.0.smithi006.stdout:9/993: dwrite d8/fb8 [0,4194304] 0 2024-03-22T06:15:20.796 INFO:tasks.workunit.client.1.smithi117.stdout:3/220: symlink d9/d1a/d39/d4c/d37/l52 0 2024-03-22T06:15:20.797 INFO:tasks.workunit.client.1.smithi117.stdout:3/221: creat d9/d1a/d3a/f53 x:0 0 0 2024-03-22T06:15:20.797 INFO:tasks.workunit.client.1.smithi117.stdout:3/222: write d9/f25 [949284,41157] 0 2024-03-22T06:15:20.798 INFO:tasks.workunit.client.1.smithi117.stdout:8/238: sync 2024-03-22T06:15:20.803 INFO:tasks.workunit.client.0.smithi006.stdout:7/990: creat d5/d1e/d29/d66/d91/ddc/dfa/d121/d12b/f14a x:0 0 0 2024-03-22T06:15:20.805 INFO:tasks.workunit.client.1.smithi117.stdout:4/272: mkdir d0/d5c/d5f 0 2024-03-22T06:15:20.806 INFO:tasks.workunit.client.1.smithi117.stdout:4/273: write d0/f42 [848342,18412] 0 2024-03-22T06:15:20.808 INFO:tasks.workunit.client.1.smithi117.stdout:1/273: rename d1/d30/c4c to d1/d30/d44/c5c 0 2024-03-22T06:15:20.808 INFO:tasks.workunit.client.1.smithi117.stdout:1/274: truncate d1/d1f/f20 5645414 0 2024-03-22T06:15:20.810 INFO:tasks.workunit.client.0.smithi006.stdout:9/994: creat d8/d10/d20/d45/d4d/f145 x:0 0 0 2024-03-22T06:15:20.811 INFO:tasks.workunit.client.1.smithi117.stdout:6/299: symlink d3/d3d/d17/d27/l64 0 2024-03-22T06:15:20.814 INFO:tasks.workunit.client.0.smithi006.stdout:7/991: getdents d5/d1e/d29/d10 0 2024-03-22T06:15:20.817 INFO:tasks.workunit.client.0.smithi006.stdout:7/992: creat d5/d1e/d29/d1f/d2d/d41/f14b x:0 0 0 2024-03-22T06:15:20.818 INFO:tasks.workunit.client.1.smithi117.stdout:3/223: creat d9/d1a/d39/d4c/d36/d41/f54 x:0 0 0 2024-03-22T06:15:20.822 INFO:tasks.workunit.client.1.smithi117.stdout:8/239: symlink d1/d3a/d44/d11/l4e 0 2024-03-22T06:15:20.825 INFO:tasks.workunit.client.0.smithi006.stdout:7/993: creat d5/d1e/d29/d1f/d144/f14c x:0 0 0 2024-03-22T06:15:20.827 INFO:tasks.workunit.client.1.smithi117.stdout:9/312: mknod d4/c5c 0 2024-03-22T06:15:20.827 INFO:tasks.workunit.client.1.smithi117.stdout:9/313: truncate d4/d8/d37/f4b 204963 0 2024-03-22T06:15:20.827 INFO:tasks.workunit.client.0.smithi006.stdout:9/995: dread d8/d10/d11/f16 [0,4194304] 0 2024-03-22T06:15:20.827 INFO:tasks.workunit.client.0.smithi006.stdout:0/860: dwrite db/dc/d37/f77 [4194304,4194304] 0 2024-03-22T06:15:20.828 INFO:tasks.workunit.client.0.smithi006.stdout:7/994: mknod d5/d1e/d29/d66/d91/ddc/dfa/c14d 0 2024-03-22T06:15:20.829 INFO:tasks.workunit.client.1.smithi117.stdout:6/300: dread d3/d40/d24/d2d/d30/d3e/d49/f5b [0,4194304] 0 2024-03-22T06:15:20.829 INFO:tasks.workunit.client.1.smithi117.stdout:6/301: write d3/d3d/d17/f33 [610415,26958] 0 2024-03-22T06:15:20.831 INFO:tasks.workunit.client.0.smithi006.stdout:9/996: truncate d8/db3/d50/d91/dfe/d12c/ff2 357856 0 2024-03-22T06:15:20.831 INFO:tasks.workunit.client.0.smithi006.stdout:9/997: read - d8/db3/d50/d91/dfe/d11a/f132 zero size 2024-03-22T06:15:20.832 INFO:tasks.workunit.client.1.smithi117.stdout:7/225: getdents d8/d9/d37 0 2024-03-22T06:15:20.832 INFO:tasks.workunit.client.1.smithi117.stdout:4/274: symlink d0/d4/d2a/d2f/l60 0 2024-03-22T06:15:20.832 INFO:tasks.workunit.client.1.smithi117.stdout:8/240: dread d1/d3a/d44/d9/f18 [4194304,4194304] 0 2024-03-22T06:15:20.833 INFO:tasks.workunit.client.1.smithi117.stdout:2/266: sync 2024-03-22T06:15:20.833 INFO:tasks.workunit.client.1.smithi117.stdout:2/267: chown d2/d5/d13/d3f/c2f 202 1 2024-03-22T06:15:20.833 INFO:tasks.workunit.client.0.smithi006.stdout:7/995: symlink d5/d1e/d29/d66/d91/d12d/l14e 0 2024-03-22T06:15:20.835 INFO:tasks.workunit.client.1.smithi117.stdout:4/275: dread d0/d3f/f54 [0,4194304] 0 2024-03-22T06:15:20.837 INFO:tasks.workunit.client.1.smithi117.stdout:1/275: mkdir d1/d30/d5d 0 2024-03-22T06:15:20.837 INFO:tasks.workunit.client.1.smithi117.stdout:1/276: chown d1/d30/d35/l2d 41103299 1 2024-03-22T06:15:20.845 INFO:tasks.workunit.client.1.smithi117.stdout:6/302: rename d3/d40/d24/d2d/c36 to d3/d40/d24/d2d/d53/c65 0 2024-03-22T06:15:20.845 INFO:tasks.workunit.client.1.smithi117.stdout:6/303: chown d3/d3d 566759 1 2024-03-22T06:15:20.847 INFO:tasks.workunit.client.1.smithi117.stdout:4/276: dread d0/d4/d2a/d2f/f31 [0,4194304] 0 2024-03-22T06:15:20.847 INFO:tasks.workunit.client.1.smithi117.stdout:4/277: stat d0/d3f/l46 0 2024-03-22T06:15:20.847 INFO:tasks.workunit.client.1.smithi117.stdout:4/278: readlink d0/d19/l3e 0 2024-03-22T06:15:20.847 INFO:tasks.workunit.client.1.smithi117.stdout:7/226: creat d8/d9/da/d10/d23/d27/d32/f44 x:0 0 0 2024-03-22T06:15:20.849 INFO:tasks.workunit.client.1.smithi117.stdout:8/241: link d1/d19/l32 d1/d3a/d44/d16/l4f 0 2024-03-22T06:15:20.850 INFO:tasks.workunit.client.1.smithi117.stdout:1/277: mknod d1/d30/c5e 0 2024-03-22T06:15:20.852 INFO:tasks.workunit.client.1.smithi117.stdout:6/304: creat d3/d61/f66 x:0 0 0 2024-03-22T06:15:20.854 INFO:tasks.workunit.client.1.smithi117.stdout:7/227: getdents d8/d9/da/d2b 0 2024-03-22T06:15:20.855 INFO:tasks.workunit.client.1.smithi117.stdout:8/242: dread d1/d3a/d44/fa [0,4194304] 0 2024-03-22T06:15:20.855 INFO:tasks.workunit.client.1.smithi117.stdout:8/243: stat d1/l41 0 2024-03-22T06:15:20.855 INFO:tasks.workunit.client.1.smithi117.stdout:1/278: rmdir d1/d30/d44 39 2024-03-22T06:15:20.856 INFO:tasks.workunit.client.1.smithi117.stdout:2/268: dwrite d2/d1b/f28 [0,4194304] 0 2024-03-22T06:15:20.856 INFO:tasks.workunit.client.1.smithi117.stdout:2/269: stat d2/d5/d13/d3f/d1d/d3a/d11/c2a 0 2024-03-22T06:15:20.859 INFO:tasks.workunit.client.1.smithi117.stdout:6/305: mkdir d3/d7/d67 0 2024-03-22T06:15:20.859 INFO:tasks.workunit.client.1.smithi117.stdout:7/228: mkdir d8/d9/d37/d45 0 2024-03-22T06:15:20.861 INFO:tasks.workunit.client.0.smithi006.stdout:7/996: dwrite d5/d1e/d63/d85/dac/db3/f127 [0,4194304] 0 2024-03-22T06:15:20.861 INFO:tasks.workunit.client.0.smithi006.stdout:7/997: dread - d5/d1e/d29/d1f/de4/f138 zero size 2024-03-22T06:15:20.861 INFO:tasks.workunit.client.1.smithi117.stdout:8/244: truncate d1/f1e 4120656 0 2024-03-22T06:15:20.864 INFO:tasks.workunit.client.1.smithi117.stdout:1/279: rename d1/d14/l21 to d1/d22/d27/d4e/l5f 0 2024-03-22T06:15:20.869 INFO:tasks.workunit.client.1.smithi117.stdout:2/270: unlink d2/d5/d13/l21 0 2024-03-22T06:15:20.871 INFO:tasks.workunit.client.1.smithi117.stdout:7/229: mknod d8/d9/da/d2b/c46 0 2024-03-22T06:15:20.871 INFO:tasks.workunit.client.1.smithi117.stdout:5/271: sync 2024-03-22T06:15:20.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/280: creat d1/d3c/d4d/d5b/f60 x:0 0 0 2024-03-22T06:15:20.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/281: creat d1/d3c/d4d/d5b/f61 x:0 0 0 2024-03-22T06:15:20.872 INFO:tasks.workunit.client.1.smithi117.stdout:8/245: dread d1/f34 [0,4194304] 0 2024-03-22T06:15:20.873 INFO:tasks.workunit.client.1.smithi117.stdout:2/271: mkdir d2/d35/d4c 0 2024-03-22T06:15:20.873 INFO:tasks.workunit.client.1.smithi117.stdout:2/272: fdatasync d2/d1b/d39/f4b 0 2024-03-22T06:15:20.873 INFO:tasks.workunit.client.1.smithi117.stdout:7/230: mknod d8/c47 0 2024-03-22T06:15:20.874 INFO:tasks.workunit.client.1.smithi117.stdout:4/279: dwrite d0/d19/d1c/f36 [0,4194304] 0 2024-03-22T06:15:20.877 INFO:tasks.workunit.client.0.smithi006.stdout:7/998: write d5/d1e/d63/f12a [1311302,47476] 0 2024-03-22T06:15:20.877 INFO:tasks.workunit.client.0.smithi006.stdout:7/999: write d5/d1e/d29/f126 [323800,4331] 0 2024-03-22T06:15:20.879 INFO:tasks.workunit.client.1.smithi117.stdout:8/246: rename d1/l8 to d1/d3a/d44/d16/l50 0 2024-03-22T06:15:20.879 INFO:tasks.workunit.client.1.smithi117.stdout:8/247: chown d1/d3a/d44/l10 8179503 1 2024-03-22T06:15:20.880 INFO:tasks.workunit.client.1.smithi117.stdout:7/231: mkdir d8/d9/da/d10/d23/d48 0 2024-03-22T06:15:20.882 INFO:tasks.workunit.client.1.smithi117.stdout:6/306: dwrite d3/d61/f66 [0,4194304] 0 2024-03-22T06:15:20.884 INFO:tasks.workunit.client.1.smithi117.stdout:1/282: mkdir d1/d62 0 2024-03-22T06:15:20.886 INFO:tasks.workunit.client.1.smithi117.stdout:5/272: rename dd/df/d30/d35/c18 to dd/df/d30/c56 0 2024-03-22T06:15:20.887 INFO:tasks.workunit.client.1.smithi117.stdout:5/273: creat dd/d16/d24/f57 x:0 0 0 2024-03-22T06:15:20.892 INFO:tasks.workunit.client.1.smithi117.stdout:1/283: creat d1/d30/d50/f63 x:0 0 0 2024-03-22T06:15:20.895 INFO:tasks.workunit.client.1.smithi117.stdout:8/248: rename d1/d3a/d44/d9/f18 to d1/d19/f51 0 2024-03-22T06:15:20.898 INFO:tasks.workunit.client.1.smithi117.stdout:5/274: mknod dd/df/c58 0 2024-03-22T06:15:20.898 INFO:tasks.workunit.client.1.smithi117.stdout:6/307: dread d3/d7/da/f29 [0,4194304] 0 2024-03-22T06:15:20.898 INFO:tasks.workunit.client.1.smithi117.stdout:6/308: stat d3/d40/d24/d2d 0 2024-03-22T06:15:20.899 INFO:tasks.workunit.client.1.smithi117.stdout:8/249: mknod d1/d3a/d44/d4b/c52 0 2024-03-22T06:15:20.900 INFO:tasks.workunit.client.1.smithi117.stdout:1/284: link d1/f7 d1/d30/d35/f64 0 2024-03-22T06:15:20.901 INFO:tasks.workunit.client.1.smithi117.stdout:6/309: getdents d3/d40/d24/d2d/d30/d3e/d49 0 2024-03-22T06:15:20.902 INFO:tasks.workunit.client.1.smithi117.stdout:8/250: symlink d1/d3a/d44/d11/l53 0 2024-03-22T06:15:20.904 INFO:tasks.workunit.client.1.smithi117.stdout:1/285: creat d1/d30/d35/d36/f65 x:0 0 0 2024-03-22T06:15:20.904 INFO:tasks.workunit.client.1.smithi117.stdout:1/286: readlink d1/d1f/d32/l53 0 2024-03-22T06:15:20.904 INFO:tasks.workunit.client.1.smithi117.stdout:2/273: dwrite d2/d5/d8/f36 [0,4194304] 0 2024-03-22T06:15:20.904 INFO:tasks.workunit.client.1.smithi117.stdout:4/280: dwrite d0/d19/d1c/f4d [0,4194304] 0 2024-03-22T06:15:20.905 INFO:tasks.workunit.client.1.smithi117.stdout:6/310: mkdir d3/d40/d24/d2d/d42/d68 0 2024-03-22T06:15:20.905 INFO:tasks.workunit.client.1.smithi117.stdout:7/232: dwrite d8/d9/da/d10/d23/d27/d39/f3b [0,4194304] 0 2024-03-22T06:15:20.907 INFO:tasks.workunit.client.1.smithi117.stdout:8/251: rename d1/d3a/d44/d4b/c4c to d1/d28/c54 0 2024-03-22T06:15:20.907 INFO:tasks.workunit.client.1.smithi117.stdout:8/252: fsync d1/d19/f2d 0 2024-03-22T06:15:20.907 INFO:tasks.workunit.client.1.smithi117.stdout:5/275: dread dd/fe [0,4194304] 0 2024-03-22T06:15:20.907 INFO:tasks.workunit.client.1.smithi117.stdout:5/276: creat dd/d16/d1a/f59 x:0 0 0 2024-03-22T06:15:20.913 INFO:tasks.workunit.client.1.smithi117.stdout:1/287: dread d1/d14/f19 [0,4194304] 0 2024-03-22T06:15:20.913 INFO:tasks.workunit.client.1.smithi117.stdout:5/277: chown dd/df/l3d 47644 1 2024-03-22T06:15:20.914 INFO:tasks.workunit.client.1.smithi117.stdout:5/278: mkdir dd/d16/d5a 0 2024-03-22T06:15:20.914 INFO:tasks.workunit.client.1.smithi117.stdout:1/288: dread d1/d22/d27/f34 [0,4194304] 0 2024-03-22T06:15:20.914 INFO:tasks.workunit.client.1.smithi117.stdout:7/233: write d8/d9/f14 [2676704,123651] 0 2024-03-22T06:15:20.915 INFO:tasks.workunit.client.1.smithi117.stdout:5/279: mknod dd/d16/c5b 0 2024-03-22T06:15:20.916 INFO:tasks.workunit.client.1.smithi117.stdout:1/289: chown d1/f13 14580 1 2024-03-22T06:15:20.916 INFO:tasks.workunit.client.1.smithi117.stdout:1/290: fdatasync d1/d30/d35/f1a 0 2024-03-22T06:15:20.919 INFO:tasks.workunit.client.1.smithi117.stdout:7/234: rename d8/d9/da/d10/d23/f24 to d8/d9/da/d10/d23/d48/f49 0 2024-03-22T06:15:20.920 INFO:tasks.workunit.client.1.smithi117.stdout:7/235: read d8/d9/da/d10/d23/d27/d39/d26/f34 [614427,56175] 0 2024-03-22T06:15:20.922 INFO:tasks.workunit.client.1.smithi117.stdout:1/291: mknod d1/c66 0 2024-03-22T06:15:20.923 INFO:tasks.workunit.client.1.smithi117.stdout:7/236: truncate d8/d9/da/d10/f13 882479 0 2024-03-22T06:15:20.924 INFO:tasks.workunit.client.1.smithi117.stdout:1/292: write d1/fd [495453,3076] 0 2024-03-22T06:15:20.925 INFO:tasks.workunit.client.1.smithi117.stdout:7/237: rmdir d8/d9/da/d2b 39 2024-03-22T06:15:20.927 INFO:tasks.workunit.client.1.smithi117.stdout:8/253: getdents d1/d3a/d44/d4b 0 2024-03-22T06:15:20.927 INFO:tasks.workunit.client.1.smithi117.stdout:8/254: fdatasync d1/d19/d38/f42 0 2024-03-22T06:15:20.927 INFO:tasks.workunit.client.1.smithi117.stdout:8/255: write d1/d48/f4d [242640,40864] 0 2024-03-22T06:15:20.928 INFO:tasks.workunit.client.1.smithi117.stdout:1/293: creat d1/d62/f67 x:0 0 0 2024-03-22T06:15:20.929 INFO:tasks.workunit.client.1.smithi117.stdout:8/256: truncate d1/f34 4125351 0 2024-03-22T06:15:20.930 INFO:tasks.workunit.client.1.smithi117.stdout:1/294: creat d1/d22/d27/f68 x:0 0 0 2024-03-22T06:15:20.930 INFO:tasks.workunit.client.1.smithi117.stdout:1/295: chown d1/d30/d35/d36 483221 1 2024-03-22T06:15:20.930 INFO:tasks.workunit.client.1.smithi117.stdout:9/314: sync 2024-03-22T06:15:20.930 INFO:tasks.workunit.client.1.smithi117.stdout:0/279: sync 2024-03-22T06:15:20.930 INFO:tasks.workunit.client.1.smithi117.stdout:3/224: sync 2024-03-22T06:15:20.930 INFO:tasks.workunit.client.1.smithi117.stdout:3/225: dread - d9/d1a/d39/d4c/d36/f43 zero size 2024-03-22T06:15:20.931 INFO:tasks.workunit.client.1.smithi117.stdout:8/257: truncate d1/d3a/d44/d11/f2a 772921 0 2024-03-22T06:15:20.931 INFO:tasks.workunit.client.1.smithi117.stdout:6/311: dwrite d3/d3d/f43 [0,4194304] 0 2024-03-22T06:15:20.933 INFO:tasks.workunit.client.1.smithi117.stdout:9/315: rmdir d4/d8/d3f 39 2024-03-22T06:15:20.933 INFO:tasks.workunit.client.1.smithi117.stdout:5/280: dwrite dd/d16/d24/d36/f25 [0,4194304] 0 2024-03-22T06:15:20.933 INFO:tasks.workunit.client.1.smithi117.stdout:8/258: mknod d1/d3a/d44/d9/c55 0 2024-03-22T06:15:20.933 INFO:tasks.workunit.client.1.smithi117.stdout:3/226: mkdir d9/d55 0 2024-03-22T06:15:20.933 INFO:tasks.workunit.client.1.smithi117.stdout:3/227: creat d9/d1a/d3a/f56 x:0 0 0 2024-03-22T06:15:20.934 INFO:tasks.workunit.client.1.smithi117.stdout:3/228: truncate d9/d1a/d39/d4c/d36/f43 186239 0 2024-03-22T06:15:20.934 INFO:tasks.workunit.client.1.smithi117.stdout:1/296: rename d1/c1e to d1/d1f/d32/c69 0 2024-03-22T06:15:20.936 INFO:tasks.workunit.client.1.smithi117.stdout:7/238: write d8/d9/da/d10/d23/d48/f49 [2157646,128472] 0 2024-03-22T06:15:20.937 INFO:tasks.workunit.client.1.smithi117.stdout:9/316: rmdir d4/d7/d10 39 2024-03-22T06:15:20.938 INFO:tasks.workunit.client.1.smithi117.stdout:5/281: truncate dd/d16/f22 2787577 0 2024-03-22T06:15:20.941 INFO:tasks.workunit.client.1.smithi117.stdout:3/229: creat d9/d55/f57 x:0 0 0 2024-03-22T06:15:20.945 INFO:tasks.workunit.client.1.smithi117.stdout:1/297: creat d1/d22/d27/d4e/f6a x:0 0 0 2024-03-22T06:15:20.947 INFO:tasks.workunit.client.1.smithi117.stdout:7/239: link d8/d9/da/d10/f22 d8/d9/da/d10/d23/d27/d39/f4a 0 2024-03-22T06:15:20.949 INFO:tasks.workunit.client.1.smithi117.stdout:9/317: mknod d4/d7/d10/d2f/c5d 0 2024-03-22T06:15:20.949 INFO:tasks.workunit.client.1.smithi117.stdout:9/318: truncate d4/d8/d3e/f45 929550 0 2024-03-22T06:15:20.953 INFO:tasks.workunit.client.1.smithi117.stdout:1/298: rename d1/d22/c23 to d1/d30/d44/c6b 0 2024-03-22T06:15:20.955 INFO:tasks.workunit.client.1.smithi117.stdout:9/319: unlink d4/c5c 0 2024-03-22T06:15:20.955 INFO:tasks.workunit.client.1.smithi117.stdout:9/320: truncate d4/f59 526018 0 2024-03-22T06:15:20.957 INFO:tasks.workunit.client.1.smithi117.stdout:5/282: mknod dd/df/c5c 0 2024-03-22T06:15:20.961 INFO:tasks.workunit.client.1.smithi117.stdout:3/230: rmdir d9/d1a/d39 39 2024-03-22T06:15:20.964 INFO:tasks.workunit.client.1.smithi117.stdout:3/231: mknod d9/d55/c58 0 2024-03-22T06:15:20.964 INFO:tasks.workunit.client.1.smithi117.stdout:3/232: fsync d9/f35 0 2024-03-22T06:15:20.966 INFO:tasks.workunit.client.1.smithi117.stdout:3/233: write d9/f25 [452589,91578] 0 2024-03-22T06:15:20.966 INFO:tasks.workunit.client.1.smithi117.stdout:3/234: creat d9/d55/f59 x:0 0 0 2024-03-22T06:15:20.966 INFO:tasks.workunit.client.1.smithi117.stdout:1/299: unlink d1/la 0 2024-03-22T06:15:20.967 INFO:tasks.workunit.client.1.smithi117.stdout:5/283: getdents dd/df/d1e 0 2024-03-22T06:15:20.967 INFO:tasks.workunit.client.1.smithi117.stdout:5/284: creat dd/d16/d24/f5d x:0 0 0 2024-03-22T06:15:20.970 INFO:tasks.workunit.client.1.smithi117.stdout:5/285: rmdir dd/df/d1e 39 2024-03-22T06:15:20.971 INFO:tasks.workunit.client.1.smithi117.stdout:5/286: chown dd/df/d30/d35/l50 7817970 1 2024-03-22T06:15:20.971 INFO:tasks.workunit.client.1.smithi117.stdout:5/287: stat dd/d16/d24/d36/f25 0 2024-03-22T06:15:20.972 INFO:tasks.workunit.client.1.smithi117.stdout:5/288: mkdir dd/df/d5e 0 2024-03-22T06:15:20.973 INFO:tasks.workunit.client.1.smithi117.stdout:5/289: stat dd/df/f38 0 2024-03-22T06:15:20.973 INFO:tasks.workunit.client.1.smithi117.stdout:5/290: chown dd/d16/d1a/f59 2 1 2024-03-22T06:15:20.983 INFO:tasks.workunit.client.1.smithi117.stdout:1/300: write d1/d14/f19 [7369361,107429] 0 2024-03-22T06:15:20.983 INFO:tasks.workunit.client.1.smithi117.stdout:1/301: getdents d1/d30/d5d 0 2024-03-22T06:15:20.983 INFO:tasks.workunit.client.1.smithi117.stdout:8/259: dwrite d1/d3a/d44/fa [0,4194304] 0 2024-03-22T06:15:20.983 INFO:tasks.workunit.client.1.smithi117.stdout:0/280: dwrite d1/d3/f12 [0,4194304] 0 2024-03-22T06:15:20.983 INFO:tasks.workunit.client.1.smithi117.stdout:8/260: chown d1/d3a/d44/d4b 218912 1 2024-03-22T06:15:20.983 INFO:tasks.workunit.client.1.smithi117.stdout:6/312: dwrite d3/d7/da/de/f19 [0,4194304] 0 2024-03-22T06:15:20.983 INFO:tasks.workunit.client.1.smithi117.stdout:6/313: write d3/d40/d24/f5e [982685,73140] 0 2024-03-22T06:15:20.983 INFO:tasks.workunit.client.1.smithi117.stdout:7/240: dwrite d8/d9/da/d2b/f3d [4194304,4194304] 0 2024-03-22T06:15:20.984 INFO:tasks.workunit.client.1.smithi117.stdout:7/241: write f7 [3677266,93167] 0 2024-03-22T06:15:20.984 INFO:tasks.workunit.client.1.smithi117.stdout:7/242: fdatasync d8/d9/da/d10/f19 0 2024-03-22T06:15:20.984 INFO:tasks.workunit.client.1.smithi117.stdout:9/321: dwrite d4/d7/f26 [0,4194304] 0 2024-03-22T06:15:20.984 INFO:tasks.workunit.client.1.smithi117.stdout:9/322: chown d4/c18 433766 1 2024-03-22T06:15:20.984 INFO:tasks.workunit.client.1.smithi117.stdout:9/323: fsync d4/d7/d10/f42 0 2024-03-22T06:15:20.986 INFO:tasks.workunit.client.1.smithi117.stdout:1/302: truncate d1/d30/d35/f64 3402259 0 2024-03-22T06:15:20.988 INFO:tasks.workunit.client.1.smithi117.stdout:8/261: rename d1/c26 to d1/d3a/d44/c56 0 2024-03-22T06:15:20.988 INFO:tasks.workunit.client.1.smithi117.stdout:8/262: truncate d1/d3a/d44/f3c 292193 0 2024-03-22T06:15:20.989 INFO:tasks.workunit.client.1.smithi117.stdout:0/281: symlink d1/d3/d2b/d3e/l64 0 2024-03-22T06:15:20.991 INFO:tasks.workunit.client.1.smithi117.stdout:7/243: rmdir d8/d9/da/d10/d23 39 2024-03-22T06:15:20.993 INFO:tasks.workunit.client.1.smithi117.stdout:6/314: dread d3/f20 [0,4194304] 0 2024-03-22T06:15:21.002 INFO:tasks.workunit.client.1.smithi117.stdout:3/235: dwrite d9/d1a/d39/f48 [0,4194304] 0 2024-03-22T06:15:21.002 INFO:tasks.workunit.client.1.smithi117.stdout:3/236: chown d9/d55/f57 6366 1 2024-03-22T06:15:21.006 INFO:tasks.workunit.client.1.smithi117.stdout:9/324: dwrite d4/d7/d10/d2f/f4e [0,4194304] 0 2024-03-22T06:15:21.006 INFO:tasks.workunit.client.1.smithi117.stdout:9/325: chown d4/d8/c34 1516846 1 2024-03-22T06:15:21.006 INFO:tasks.workunit.client.1.smithi117.stdout:8/263: dread d1/d48/f4d [0,4194304] 0 2024-03-22T06:15:21.006 INFO:tasks.workunit.client.1.smithi117.stdout:8/264: chown d1/d3a/d44/d16/l50 54 1 2024-03-22T06:15:21.007 INFO:tasks.workunit.client.1.smithi117.stdout:5/291: getdents dd/d16/d24 0 2024-03-22T06:15:21.011 INFO:tasks.workunit.client.1.smithi117.stdout:1/303: creat d1/d14/f6c x:0 0 0 2024-03-22T06:15:21.013 INFO:tasks.workunit.client.1.smithi117.stdout:0/282: creat d1/d10/d5c/d40/d4e/f65 x:0 0 0 2024-03-22T06:15:21.013 INFO:tasks.workunit.client.1.smithi117.stdout:0/283: stat d1 0 2024-03-22T06:15:21.013 INFO:tasks.workunit.client.1.smithi117.stdout:0/284: dread - d1/d10/d18/d26/f58 zero size 2024-03-22T06:15:21.013 INFO:tasks.workunit.client.1.smithi117.stdout:0/285: write d1/d3/d2b/d3e/de/d1f/f33 [646210,22824] 0 2024-03-22T06:15:21.013 INFO:tasks.workunit.client.1.smithi117.stdout:0/286: stat d1/f4f 0 2024-03-22T06:15:21.013 INFO:tasks.workunit.client.1.smithi117.stdout:0/287: chown d1/d10/d18 43991 1 2024-03-22T06:15:21.014 INFO:tasks.workunit.client.1.smithi117.stdout:6/315: creat d3/d7/da/d56/d3a/f69 x:0 0 0 2024-03-22T06:15:21.019 INFO:tasks.workunit.client.1.smithi117.stdout:9/326: mknod d4/d7/c5e 0 2024-03-22T06:15:21.021 INFO:tasks.workunit.client.1.smithi117.stdout:8/265: rename d1/d3a/d44/d11/l49 to d1/d19/d38/l57 0 2024-03-22T06:15:21.021 INFO:tasks.workunit.client.1.smithi117.stdout:7/244: dwrite d8/d9/da/d10/f13 [0,4194304] 0 2024-03-22T06:15:21.023 INFO:tasks.workunit.client.1.smithi117.stdout:5/292: truncate dd/df/d30/d35/f27 1580927 0 2024-03-22T06:15:21.024 INFO:tasks.workunit.client.1.smithi117.stdout:5/293: write dd/d16/d24/d36/f47 [406123,99229] 0 2024-03-22T06:15:21.024 INFO:tasks.workunit.client.1.smithi117.stdout:9/327: getdents d4/d8/d3f 0 2024-03-22T06:15:21.025 INFO:tasks.workunit.client.1.smithi117.stdout:8/266: truncate d1/d3a/d44/d11/f2a 472429 0 2024-03-22T06:15:21.026 INFO:tasks.workunit.client.1.smithi117.stdout:6/316: rename d3/d3d/d17/d27 to d3/d40/d24/d2d/d30/d3e/d6a 0 2024-03-22T06:15:21.026 INFO:tasks.workunit.client.0.smithi006.stdout:6/934: sync 2024-03-22T06:15:21.027 INFO:tasks.workunit.client.0.smithi006.stdout:6/935: chown d6/d24/le6 183940520 1 2024-03-22T06:15:21.028 INFO:tasks.workunit.client.1.smithi117.stdout:7/245: link d8/d9/da/d10/f22 d8/d9/da/f4b 0 2024-03-22T06:15:21.031 INFO:tasks.workunit.client.0.smithi006.stdout:6/936: rename d6/d1b/f2d to d6/d1b/dae/dbc/d105/f11e 0 2024-03-22T06:15:21.032 INFO:tasks.workunit.client.1.smithi117.stdout:5/294: symlink dd/d16/d24/l5f 0 2024-03-22T06:15:21.032 INFO:tasks.workunit.client.1.smithi117.stdout:8/267: getdents d1/d19 0 2024-03-22T06:15:21.033 INFO:tasks.workunit.client.1.smithi117.stdout:8/268: dread d1/d19/f3f [0,4194304] 0 2024-03-22T06:15:21.034 INFO:tasks.workunit.client.1.smithi117.stdout:9/328: dread d4/d8/f2c [0,4194304] 0 2024-03-22T06:15:21.035 INFO:tasks.workunit.client.1.smithi117.stdout:6/317: creat d3/d40/d46/f6b x:0 0 0 2024-03-22T06:15:21.035 INFO:tasks.workunit.client.1.smithi117.stdout:5/295: mkdir dd/df/d30/d60 0 2024-03-22T06:15:21.037 INFO:tasks.workunit.client.1.smithi117.stdout:3/237: dwrite d9/f17 [0,4194304] 0 2024-03-22T06:15:21.037 INFO:tasks.workunit.client.1.smithi117.stdout:3/238: chown d9/d1a/d39/d4c/f4e 2 1 2024-03-22T06:15:21.037 INFO:tasks.workunit.client.1.smithi117.stdout:6/318: mknod d3/d7/c6c 0 2024-03-22T06:15:21.038 INFO:tasks.workunit.client.1.smithi117.stdout:6/319: write d3/d7/da/f29 [1811161,53102] 0 2024-03-22T06:15:21.038 INFO:tasks.workunit.client.1.smithi117.stdout:6/320: write d3/d40/d24/f5e [89189,15242] 0 2024-03-22T06:15:21.039 INFO:tasks.workunit.client.1.smithi117.stdout:5/296: mknod dd/d16/d24/c61 0 2024-03-22T06:15:21.039 INFO:tasks.workunit.client.1.smithi117.stdout:5/297: creat dd/d16/d1a/f62 x:0 0 0 2024-03-22T06:15:21.039 INFO:tasks.workunit.client.1.smithi117.stdout:5/298: stat dd/d16/c53 0 2024-03-22T06:15:21.039 INFO:tasks.workunit.client.1.smithi117.stdout:2/274: sync 2024-03-22T06:15:21.040 INFO:tasks.workunit.client.1.smithi117.stdout:4/281: sync 2024-03-22T06:15:21.040 INFO:tasks.workunit.client.1.smithi117.stdout:4/282: chown d0/f27 315972509 1 2024-03-22T06:15:21.041 INFO:tasks.workunit.client.1.smithi117.stdout:9/329: symlink d4/l5f 0 2024-03-22T06:15:21.042 INFO:tasks.workunit.client.1.smithi117.stdout:9/330: readlink d4/l40 0 2024-03-22T06:15:21.042 INFO:tasks.workunit.client.1.smithi117.stdout:9/331: truncate d4/d8/d37/f4b 980819 0 2024-03-22T06:15:21.043 INFO:tasks.workunit.client.1.smithi117.stdout:0/288: dwrite d1/d10/d18/f38 [0,4194304] 0 2024-03-22T06:15:21.043 INFO:tasks.workunit.client.1.smithi117.stdout:8/269: write d1/d19/f43 [3657223,124284] 0 2024-03-22T06:15:21.047 INFO:tasks.workunit.client.1.smithi117.stdout:3/239: creat d9/d1a/f5a x:0 0 0 2024-03-22T06:15:21.050 INFO:tasks.workunit.client.1.smithi117.stdout:7/246: dread d8/d9/da/d10/d23/d48/f49 [0,4194304] 0 2024-03-22T06:15:21.051 INFO:tasks.workunit.client.1.smithi117.stdout:1/304: link d1/d22/l2e d1/d22/d27/d4e/l6d 0 2024-03-22T06:15:21.051 INFO:tasks.workunit.client.1.smithi117.stdout:2/275: dread d2/f26 [0,4194304] 0 2024-03-22T06:15:21.056 INFO:tasks.workunit.client.1.smithi117.stdout:6/321: rename d3/f20 to d3/d40/d24/d2d/d30/f6d 0 2024-03-22T06:15:21.056 INFO:tasks.workunit.client.1.smithi117.stdout:6/322: dread - d3/d40/d46/f6b zero size 2024-03-22T06:15:21.056 INFO:tasks.workunit.client.1.smithi117.stdout:6/323: fdatasync d3/d7/da/de/f47 0 2024-03-22T06:15:21.056 INFO:tasks.workunit.client.1.smithi117.stdout:6/324: fsync d3/d7/da/d56/f2f 0 2024-03-22T06:15:21.056 INFO:tasks.workunit.client.1.smithi117.stdout:6/325: fdatasync d3/d40/d24/f44 0 2024-03-22T06:15:21.063 INFO:tasks.workunit.client.1.smithi117.stdout:8/270: mknod d1/d3a/d44/c58 0 2024-03-22T06:15:21.064 INFO:tasks.workunit.client.1.smithi117.stdout:8/271: dread d1/d19/f3f [0,4194304] 0 2024-03-22T06:15:21.069 INFO:tasks.workunit.client.0.smithi006.stdout:6/937: getdents d6/d1b/dae/dbc/d105 0 2024-03-22T06:15:21.073 INFO:tasks.workunit.client.1.smithi117.stdout:7/247: mknod d8/d9/da/d10/d23/d27/d39/c4c 0 2024-03-22T06:15:21.073 INFO:tasks.workunit.client.1.smithi117.stdout:7/248: truncate d8/d9/da/d10/d23/d27/f3e 261278 0 2024-03-22T06:15:21.077 INFO:tasks.workunit.client.1.smithi117.stdout:1/305: mknod d1/d62/c6e 0 2024-03-22T06:15:21.079 INFO:tasks.workunit.client.0.smithi006.stdout:1/977: sync 2024-03-22T06:15:21.079 INFO:tasks.workunit.client.0.smithi006.stdout:0/861: sync 2024-03-22T06:15:21.079 INFO:tasks.workunit.client.0.smithi006.stdout:0/862: creat db/d10/d2a/d41/f12e x:0 0 0 2024-03-22T06:15:21.081 INFO:tasks.workunit.client.0.smithi006.stdout:1/978: mkdir d8/d9/d140 0 2024-03-22T06:15:21.084 INFO:tasks.workunit.client.1.smithi117.stdout:5/299: rename dd/df/l15 to dd/d16/d1a/l63 0 2024-03-22T06:15:21.084 INFO:tasks.workunit.client.1.smithi117.stdout:5/300: fdatasync dd/d16/d1a/f37 0 2024-03-22T06:15:21.084 INFO:tasks.workunit.client.1.smithi117.stdout:5/301: creat dd/d16/f64 x:0 0 0 2024-03-22T06:15:21.085 INFO:tasks.workunit.client.1.smithi117.stdout:5/302: dread - dd/d16/d1a/d32/f51 zero size 2024-03-22T06:15:21.085 INFO:tasks.workunit.client.1.smithi117.stdout:5/303: chown dd/df/f38 252735307 1 2024-03-22T06:15:21.085 INFO:tasks.workunit.client.1.smithi117.stdout:7/249: rename d8/f1f to d8/d9/da/d10/d23/d27/d32/f4d 0 2024-03-22T06:15:21.086 INFO:tasks.workunit.client.1.smithi117.stdout:5/304: read dd/d16/d24/d36/f26 [1804579,90043] 0 2024-03-22T06:15:21.086 INFO:tasks.workunit.client.1.smithi117.stdout:5/305: chown dd/df/d1e 453 1 2024-03-22T06:15:21.089 INFO:tasks.workunit.client.1.smithi117.stdout:3/240: dwrite d9/fb [0,4194304] 0 2024-03-22T06:15:21.094 INFO:tasks.workunit.client.1.smithi117.stdout:0/289: dwrite d1/d10/d5c/d40/f41 [0,4194304] 0 2024-03-22T06:15:21.094 INFO:tasks.workunit.client.1.smithi117.stdout:0/290: stat d1/d10/c34 0 2024-03-22T06:15:21.096 INFO:tasks.workunit.client.1.smithi117.stdout:8/272: dwrite d1/d19/f3f [0,4194304] 0 2024-03-22T06:15:21.096 INFO:tasks.workunit.client.1.smithi117.stdout:3/241: rmdir d9/d1a/d39/d4c/d37 39 2024-03-22T06:15:21.097 INFO:tasks.workunit.client.0.smithi006.stdout:6/938: dwrite d6/d1b/d2c/f43 [0,4194304] 0 2024-03-22T06:15:21.097 INFO:tasks.workunit.client.0.smithi006.stdout:6/939: creat d6/d1d/d99/dcd/f11f x:0 0 0 2024-03-22T06:15:21.097 INFO:tasks.workunit.client.0.smithi006.stdout:1/979: write d8/d71/f8e [279224,130389] 0 2024-03-22T06:15:21.100 INFO:tasks.workunit.client.1.smithi117.stdout:6/326: dwrite d3/d40/d24/f5e [0,4194304] 0 2024-03-22T06:15:21.100 INFO:tasks.workunit.client.1.smithi117.stdout:8/273: rename d1/d3a/d44/l10 to d1/d3a/d44/d4b/l59 0 2024-03-22T06:15:21.103 INFO:tasks.workunit.client.1.smithi117.stdout:3/242: link d9/d1a/d39/f45 d9/d1a/d3a/f5b 0 2024-03-22T06:15:21.106 INFO:tasks.workunit.client.0.smithi006.stdout:6/940: getdents d6/d77/db7 0 2024-03-22T06:15:21.106 INFO:tasks.workunit.client.0.smithi006.stdout:6/941: fsync d6/d24/f58 0 2024-03-22T06:15:21.108 INFO:tasks.workunit.client.0.smithi006.stdout:1/980: creat d8/d9/d10/d7c/f141 x:0 0 0 2024-03-22T06:15:21.108 INFO:tasks.workunit.client.1.smithi117.stdout:0/291: dread d1/d3/f12 [0,4194304] 0 2024-03-22T06:15:21.108 INFO:tasks.workunit.client.1.smithi117.stdout:0/292: stat d1/d3/d2b/d3e/l64 0 2024-03-22T06:15:21.108 INFO:tasks.workunit.client.1.smithi117.stdout:0/293: readlink d1/d3/d2b/d3e/l64 0 2024-03-22T06:15:21.110 INFO:tasks.workunit.client.0.smithi006.stdout:6/942: creat d6/d7c/d92/f120 x:0 0 0 2024-03-22T06:15:21.113 INFO:tasks.workunit.client.0.smithi006.stdout:1/981: symlink d8/d9/d13/d39/d60/d8f/d10a/d59/db1/l142 0 2024-03-22T06:15:21.114 INFO:tasks.workunit.client.0.smithi006.stdout:1/982: fsync d8/d9/f1e 0 2024-03-22T06:15:21.114 INFO:tasks.workunit.client.1.smithi117.stdout:5/306: dwrite dd/d16/d24/d36/f39 [0,4194304] 0 2024-03-22T06:15:21.116 INFO:tasks.workunit.client.1.smithi117.stdout:7/250: dwrite d8/d9/da/d10/f22 [0,4194304] 0 2024-03-22T06:15:21.116 INFO:tasks.workunit.client.1.smithi117.stdout:8/274: dread d1/d3a/d44/fa [0,4194304] 0 2024-03-22T06:15:21.116 INFO:tasks.workunit.client.1.smithi117.stdout:8/275: chown d1 1940877397 1 2024-03-22T06:15:21.116 INFO:tasks.workunit.client.1.smithi117.stdout:8/276: fsync d1/d3a/d44/d9/f13 0 2024-03-22T06:15:21.116 INFO:tasks.workunit.client.1.smithi117.stdout:6/327: symlink d3/d40/d46/l6e 0 2024-03-22T06:15:21.116 INFO:tasks.workunit.client.1.smithi117.stdout:6/328: chown d3/d40/d24/d2d/d42 29 1 2024-03-22T06:15:21.119 INFO:tasks.workunit.client.1.smithi117.stdout:3/243: dwrite d9/f11 [0,4194304] 0 2024-03-22T06:15:21.120 INFO:tasks.workunit.client.0.smithi006.stdout:6/943: mknod d6/c121 0 2024-03-22T06:15:21.122 INFO:tasks.workunit.client.1.smithi117.stdout:7/251: creat d8/d9/d37/f4e x:0 0 0 2024-03-22T06:15:21.124 INFO:tasks.workunit.client.1.smithi117.stdout:8/277: rename d1/d48/f4d to d1/d3a/d44/d11/f5a 0 2024-03-22T06:15:21.124 INFO:tasks.workunit.client.1.smithi117.stdout:6/329: unlink d3/d7/d28/c3b 0 2024-03-22T06:15:21.125 INFO:tasks.workunit.client.1.smithi117.stdout:0/294: write d1/d10/d5c/d40/f41 [1544485,86022] 0 2024-03-22T06:15:21.125 INFO:tasks.workunit.client.1.smithi117.stdout:0/295: creat d1/d10/d18/d26/dc/f66 x:0 0 0 2024-03-22T06:15:21.125 INFO:tasks.workunit.client.1.smithi117.stdout:0/296: truncate d1/f3d 186048 0 2024-03-22T06:15:21.126 INFO:tasks.workunit.client.1.smithi117.stdout:7/252: rmdir d8/d9/da/d10/d23/d27/d39 39 2024-03-22T06:15:21.126 INFO:tasks.workunit.client.1.smithi117.stdout:8/278: symlink d1/d28/l5b 0 2024-03-22T06:15:21.127 INFO:tasks.workunit.client.1.smithi117.stdout:6/330: truncate d3/d7/da/d56/f2f 1291696 0 2024-03-22T06:15:21.129 INFO:tasks.workunit.client.1.smithi117.stdout:0/297: symlink d1/d10/l67 0 2024-03-22T06:15:21.129 INFO:tasks.workunit.client.1.smithi117.stdout:8/279: rmdir d1/d19 39 2024-03-22T06:15:21.131 INFO:tasks.workunit.client.1.smithi117.stdout:3/244: dread d9/f11 [0,4194304] 0 2024-03-22T06:15:21.133 INFO:tasks.workunit.client.1.smithi117.stdout:0/298: link d1/d3/l56 d1/d3/d2b/d3e/de/d1f/d3b/d55/l68 0 2024-03-22T06:15:21.133 INFO:tasks.workunit.client.1.smithi117.stdout:0/299: truncate d1/d3/d2b/d3e/f60 720258 0 2024-03-22T06:15:21.133 INFO:tasks.workunit.client.1.smithi117.stdout:0/300: chown d1/d10/d18/c5a 34496 1 2024-03-22T06:15:21.138 INFO:tasks.workunit.client.1.smithi117.stdout:3/245: dread f7 [0,4194304] 0 2024-03-22T06:15:21.138 INFO:tasks.workunit.client.1.smithi117.stdout:0/301: mkdir d1/d10/d5c/d40/d69 0 2024-03-22T06:15:21.138 INFO:tasks.workunit.client.1.smithi117.stdout:0/302: write d1/f1c [509084,117024] 0 2024-03-22T06:15:21.138 INFO:tasks.workunit.client.1.smithi117.stdout:0/303: dread d1/d3/d2b/d3e/f60 [0,4194304] 0 2024-03-22T06:15:21.140 INFO:tasks.workunit.client.1.smithi117.stdout:3/246: symlink d9/d1a/d39/d4c/d37/l5c 0 2024-03-22T06:15:21.140 INFO:tasks.workunit.client.1.smithi117.stdout:3/247: read - d9/d1a/d39/d4c/d36/f50 zero size 2024-03-22T06:15:21.142 INFO:tasks.workunit.client.1.smithi117.stdout:0/304: mkdir d1/d10/d18/d26/dc/d39/d6a 0 2024-03-22T06:15:21.142 INFO:tasks.workunit.client.1.smithi117.stdout:0/305: creat d1/d3/d2b/d3e/de/d1f/d3b/f6b x:0 0 0 2024-03-22T06:15:21.142 INFO:tasks.workunit.client.1.smithi117.stdout:5/307: dwrite dd/d16/d24/d36/f39 [0,4194304] 0 2024-03-22T06:15:21.144 INFO:tasks.workunit.client.1.smithi117.stdout:6/331: dread d3/f4 [0,4194304] 0 2024-03-22T06:15:21.144 INFO:tasks.workunit.client.1.smithi117.stdout:6/332: creat d3/d7/da/de/f6f x:0 0 0 2024-03-22T06:15:21.145 INFO:tasks.workunit.client.1.smithi117.stdout:6/333: read d3/f52 [390579,54014] 0 2024-03-22T06:15:21.145 INFO:tasks.workunit.client.1.smithi117.stdout:6/334: chown d3/d40/d24/f5e 16308879 1 2024-03-22T06:15:21.145 INFO:tasks.workunit.client.1.smithi117.stdout:6/335: readlink d3/d7/da/d56/d3a/l55 0 2024-03-22T06:15:21.147 INFO:tasks.workunit.client.1.smithi117.stdout:6/336: creat d3/d40/d24/d2d/d53/f70 x:0 0 0 2024-03-22T06:15:21.147 INFO:tasks.workunit.client.1.smithi117.stdout:6/337: fsync d3/d40/d24/f4e 0 2024-03-22T06:15:21.149 INFO:tasks.workunit.client.1.smithi117.stdout:6/338: rmdir d3/d40/d24/d2d/d42/d68 0 2024-03-22T06:15:21.154 INFO:tasks.workunit.client.1.smithi117.stdout:0/306: write d1/d3/d2b/d3e/de/d1f/d3b/f42 [531124,52689] 0 2024-03-22T06:15:21.155 INFO:tasks.workunit.client.0.smithi006.stdout:1/983: dwrite d8/d33/f11e [0,4194304] 0 2024-03-22T06:15:21.155 INFO:tasks.workunit.client.1.smithi117.stdout:3/248: dread d9/f30 [0,4194304] 0 2024-03-22T06:15:21.155 INFO:tasks.workunit.client.1.smithi117.stdout:3/249: creat d9/d55/f5d x:0 0 0 2024-03-22T06:15:21.156 INFO:tasks.workunit.client.1.smithi117.stdout:3/250: dread - d9/f35 zero size 2024-03-22T06:15:21.156 INFO:tasks.workunit.client.0.smithi006.stdout:6/944: dwrite d6/d24/f38 [4194304,4194304] 0 2024-03-22T06:15:21.160 INFO:tasks.workunit.client.1.smithi117.stdout:0/307: rename d1/d10/d18/d26/dc/f66 to d1/d10/f6c 0 2024-03-22T06:15:21.164 INFO:tasks.workunit.client.1.smithi117.stdout:7/253: dwrite d8/d9/da/d10/d23/d27/d32/f44 [0,4194304] 0 2024-03-22T06:15:21.164 INFO:tasks.workunit.client.1.smithi117.stdout:8/280: dwrite d1/d28/f31 [0,4194304] 0 2024-03-22T06:15:21.164 INFO:tasks.workunit.client.1.smithi117.stdout:8/281: read d1/d3a/d44/d9/f13 [8440006,129313] 0 2024-03-22T06:15:21.165 INFO:tasks.workunit.client.0.smithi006.stdout:6/945: creat d6/d1d/d64/f122 x:0 0 0 2024-03-22T06:15:21.165 INFO:tasks.workunit.client.0.smithi006.stdout:6/946: write d6/d24/f38 [3619551,22680] 0 2024-03-22T06:15:21.165 INFO:tasks.workunit.client.0.smithi006.stdout:6/947: fsync d6/d77/fec 0 2024-03-22T06:15:21.167 INFO:tasks.workunit.client.0.smithi006.stdout:6/948: mknod d6/d24/c123 0 2024-03-22T06:15:21.169 INFO:tasks.workunit.client.0.smithi006.stdout:6/949: rename d6/d24/d44/f46 to d6/d1d/d99/dcd/f124 0 2024-03-22T06:15:21.172 INFO:tasks.workunit.client.1.smithi117.stdout:7/254: link c2 d8/d9/d37/c4f 0 2024-03-22T06:15:21.173 INFO:tasks.workunit.client.1.smithi117.stdout:8/282: rmdir d1 39 2024-03-22T06:15:21.174 INFO:tasks.workunit.client.1.smithi117.stdout:3/251: rename d9/d55/f5d to d9/d1a/d39/d4c/f5e 0 2024-03-22T06:15:21.174 INFO:tasks.workunit.client.1.smithi117.stdout:3/252: dread - d9/d1a/d39/f45 zero size 2024-03-22T06:15:21.174 INFO:tasks.workunit.client.1.smithi117.stdout:3/253: readlink d9/l2f 0 2024-03-22T06:15:21.174 INFO:tasks.workunit.client.1.smithi117.stdout:3/254: truncate d9/d1a/d3a/f47 877925 0 2024-03-22T06:15:21.174 INFO:tasks.workunit.client.1.smithi117.stdout:3/255: chown d9/d1a/d3a 492275 1 2024-03-22T06:15:21.175 INFO:tasks.workunit.client.1.smithi117.stdout:7/255: rmdir d8/d9/da/d10/d23/d27 39 2024-03-22T06:15:21.177 INFO:tasks.workunit.client.1.smithi117.stdout:7/256: unlink d8/d9/da/d2b/f3d 0 2024-03-22T06:15:21.178 INFO:tasks.workunit.client.1.smithi117.stdout:7/257: mkdir d8/d50 0 2024-03-22T06:15:21.181 INFO:tasks.workunit.client.1.smithi117.stdout:7/258: mknod d8/d9/da/d10/d23/c51 0 2024-03-22T06:15:21.182 INFO:tasks.workunit.client.1.smithi117.stdout:7/259: symlink d8/l52 0 2024-03-22T06:15:21.182 INFO:tasks.workunit.client.1.smithi117.stdout:7/260: fdatasync f5 0 2024-03-22T06:15:21.182 INFO:tasks.workunit.client.1.smithi117.stdout:7/261: rmdir d8/d9/da 39 2024-03-22T06:15:21.183 INFO:tasks.workunit.client.1.smithi117.stdout:7/262: mknod d8/d9/d37/c53 0 2024-03-22T06:15:21.183 INFO:tasks.workunit.client.1.smithi117.stdout:7/263: read d8/d9/da/d10/d23/d48/f49 [2417527,46662] 0 2024-03-22T06:15:21.183 INFO:tasks.workunit.client.0.smithi006.stdout:6/950: dread d6/d1b/f97 [0,4194304] 0 2024-03-22T06:15:21.187 INFO:tasks.workunit.client.1.smithi117.stdout:3/256: write d9/d1a/d39/d4c/f4e [3108390,67724] 0 2024-03-22T06:15:21.188 INFO:tasks.workunit.client.1.smithi117.stdout:3/257: mknod d9/c5f 0 2024-03-22T06:15:21.190 INFO:tasks.workunit.client.1.smithi117.stdout:3/258: symlink d9/d1a/d3a/l60 0 2024-03-22T06:15:21.190 INFO:tasks.workunit.client.1.smithi117.stdout:3/259: fsync d9/d1a/f1f 0 2024-03-22T06:15:21.190 INFO:tasks.workunit.client.0.smithi006.stdout:1/984: dwrite d8/d9/d13/d24/f7f [0,4194304] 0 2024-03-22T06:15:21.191 INFO:tasks.workunit.client.1.smithi117.stdout:8/283: dwrite d1/d19/d38/f42 [0,4194304] 0 2024-03-22T06:15:21.192 INFO:tasks.workunit.client.1.smithi117.stdout:3/260: rename d9/fb to d9/d1a/d39/d4c/d37/f61 0 2024-03-22T06:15:21.192 INFO:tasks.workunit.client.1.smithi117.stdout:3/261: write d9/d1a/d3a/f56 [190766,52004] 0 2024-03-22T06:15:21.193 INFO:tasks.workunit.client.1.smithi117.stdout:3/262: write d9/d1a/d39/d4c/d36/d41/f54 [141863,69361] 0 2024-03-22T06:15:21.193 INFO:tasks.workunit.client.1.smithi117.stdout:6/339: dwrite d3/d7/da/d56/d3a/f69 [0,4194304] 0 2024-03-22T06:15:21.193 INFO:tasks.workunit.client.1.smithi117.stdout:7/264: dread d8/d9/da/f4b [0,4194304] 0 2024-03-22T06:15:21.193 INFO:tasks.workunit.client.1.smithi117.stdout:7/265: truncate d8/d9/da/d10/d23/d27/f3e 522519 0 2024-03-22T06:15:21.196 INFO:tasks.workunit.client.0.smithi006.stdout:9/998: sync 2024-03-22T06:15:21.196 INFO:tasks.workunit.client.0.smithi006.stdout:9/999: readlink d8/d10/d20/la3 0 2024-03-22T06:15:21.197 INFO:tasks.workunit.client.0.smithi006.stdout:1/985: truncate d8/d9/d13/d24/d32/f6f 4480735 0 2024-03-22T06:15:21.197 INFO:tasks.workunit.client.0.smithi006.stdout:1/986: creat d8/d9/d13/d39/d60/d8f/d10a/d44/f143 x:0 0 0 2024-03-22T06:15:21.200 INFO:tasks.workunit.client.1.smithi117.stdout:8/284: symlink d1/d3a/l5c 0 2024-03-22T06:15:21.201 INFO:tasks.workunit.client.1.smithi117.stdout:8/285: dread d1/d3a/d44/f3c [0,4194304] 0 2024-03-22T06:15:21.208 INFO:tasks.workunit.client.1.smithi117.stdout:6/340: symlink d3/d3d/d17/l71 0 2024-03-22T06:15:21.209 INFO:tasks.workunit.client.1.smithi117.stdout:0/308: getdents d1/d10 0 2024-03-22T06:15:21.210 INFO:tasks.workunit.client.1.smithi117.stdout:0/309: dread d1/f3d [0,4194304] 0 2024-03-22T06:15:21.210 INFO:tasks.workunit.client.1.smithi117.stdout:0/310: dread - d1/d10/f6c zero size 2024-03-22T06:15:21.217 INFO:tasks.workunit.client.0.smithi006.stdout:6/951: rename d6/d1d/d99 to d6/d1b/d2c/d4a/d125 0 2024-03-22T06:15:21.218 INFO:tasks.workunit.client.1.smithi117.stdout:7/266: dwrite d8/d9/da/d10/d23/d27/d32/f44 [0,4194304] 0 2024-03-22T06:15:21.220 INFO:tasks.workunit.client.0.smithi006.stdout:6/952: link d6/d1d/d5c/d89/cbf d6/d1b/dae/dbc/c126 0 2024-03-22T06:15:21.220 INFO:tasks.workunit.client.0.smithi006.stdout:6/953: write d6/d7c/d92/f120 [594622,70812] 0 2024-03-22T06:15:21.220 INFO:tasks.workunit.client.0.smithi006.stdout:6/954: readlink d6/d1b/d2c/lff 0 2024-03-22T06:15:21.227 INFO:tasks.workunit.client.1.smithi117.stdout:3/263: getdents d9/d55 0 2024-03-22T06:15:21.230 INFO:tasks.workunit.client.1.smithi117.stdout:9/332: sync 2024-03-22T06:15:21.230 INFO:tasks.workunit.client.1.smithi117.stdout:2/276: sync 2024-03-22T06:15:21.230 INFO:tasks.workunit.client.1.smithi117.stdout:1/306: sync 2024-03-22T06:15:21.230 INFO:tasks.workunit.client.1.smithi117.stdout:4/283: sync 2024-03-22T06:15:21.230 INFO:tasks.workunit.client.1.smithi117.stdout:9/333: creat d4/d7/d27/f60 x:0 0 0 2024-03-22T06:15:21.233 INFO:tasks.workunit.client.1.smithi117.stdout:7/267: link d8/d9/da/f28 d8/d9/da/d10/d23/d27/d32/f54 0 2024-03-22T06:15:21.234 INFO:tasks.workunit.client.1.smithi117.stdout:8/286: dwrite d1/f34 [0,4194304] 0 2024-03-22T06:15:21.236 INFO:tasks.workunit.client.1.smithi117.stdout:1/307: mkdir d1/d30/d5d/d6f 0 2024-03-22T06:15:21.241 INFO:tasks.workunit.client.1.smithi117.stdout:9/334: write d4/d7/f2d [1304501,19063] 0 2024-03-22T06:15:21.242 INFO:tasks.workunit.client.1.smithi117.stdout:7/268: dread d8/d9/f1e [0,4194304] 0 2024-03-22T06:15:21.246 INFO:tasks.workunit.client.0.smithi006.stdout:1/987: dwrite d8/d9/d13/d24/f122 [0,4194304] 0 2024-03-22T06:15:21.248 INFO:tasks.workunit.client.1.smithi117.stdout:9/335: rename d4/d7/c15 to d4/d7/d10/d2f/c61 0 2024-03-22T06:15:21.249 INFO:tasks.workunit.client.1.smithi117.stdout:9/336: chown d4/l5f 6 1 2024-03-22T06:15:21.249 INFO:tasks.workunit.client.1.smithi117.stdout:6/341: dwrite d3/d40/d24/f44 [0,4194304] 0 2024-03-22T06:15:21.249 INFO:tasks.workunit.client.1.smithi117.stdout:6/342: chown d3/d40/d24/d2d/d30/d3e/d49 1828 1 2024-03-22T06:15:21.252 INFO:tasks.workunit.client.1.smithi117.stdout:9/337: rename d4/l5f to d4/d7/l62 0 2024-03-22T06:15:21.254 INFO:tasks.workunit.client.1.smithi117.stdout:6/343: symlink d3/d40/d24/d2d/l72 0 2024-03-22T06:15:21.254 INFO:tasks.workunit.client.1.smithi117.stdout:7/269: mknod d8/d9/da/d10/c55 0 2024-03-22T06:15:21.256 INFO:tasks.workunit.client.1.smithi117.stdout:9/338: creat d4/d7/d10/f63 x:0 0 0 2024-03-22T06:15:21.258 INFO:tasks.workunit.client.1.smithi117.stdout:7/270: mkdir d8/d9/da/d10/d23/d27/d56 0 2024-03-22T06:15:21.260 INFO:tasks.workunit.client.1.smithi117.stdout:9/339: unlink d4/d8/f13 0 2024-03-22T06:15:21.260 INFO:tasks.workunit.client.1.smithi117.stdout:9/340: dread - d4/d7/d27/f60 zero size 2024-03-22T06:15:21.260 INFO:tasks.workunit.client.1.smithi117.stdout:9/341: dread - d4/d46/f55 zero size 2024-03-22T06:15:21.260 INFO:tasks.workunit.client.1.smithi117.stdout:7/271: readlink d8/d9/da/d10/d23/d27/d39/d26/l2d 0 2024-03-22T06:15:21.262 INFO:tasks.workunit.client.1.smithi117.stdout:6/344: rmdir d3/d7/da/d32 0 2024-03-22T06:15:21.264 INFO:tasks.workunit.client.1.smithi117.stdout:6/345: rename d3/d7/d67 to d3/d61/d73 0 2024-03-22T06:15:21.273 INFO:tasks.workunit.client.0.smithi006.stdout:1/988: write d8/d9/d13/f28 [1896359,118979] 0 2024-03-22T06:15:21.274 INFO:tasks.workunit.client.0.smithi006.stdout:1/989: creat d8/d71/dcd/f144 x:0 0 0 2024-03-22T06:15:21.276 INFO:tasks.workunit.client.0.smithi006.stdout:1/990: link d8/cf d8/d9/d10/d7c/c145 0 2024-03-22T06:15:21.279 INFO:tasks.workunit.client.1.smithi117.stdout:7/272: write d8/d9/da/f4b [2990187,110092] 0 2024-03-22T06:15:21.285 INFO:tasks.workunit.client.1.smithi117.stdout:2/277: dwrite d2/f4a [0,4194304] 0 2024-03-22T06:15:21.285 INFO:tasks.workunit.client.1.smithi117.stdout:0/311: dwrite d1/d3/d2b/d3e/de/f51 [0,4194304] 0 2024-03-22T06:15:21.285 INFO:tasks.workunit.client.1.smithi117.stdout:1/308: dwrite d1/d22/d27/f34 [0,4194304] 0 2024-03-22T06:15:21.285 INFO:tasks.workunit.client.0.smithi006.stdout:6/955: dwrite d6/d1d/d5c/d89/f8b [0,4194304] 0 2024-03-22T06:15:21.286 INFO:tasks.workunit.client.1.smithi117.stdout:9/342: dwrite d4/d8/f22 [0,4194304] 0 2024-03-22T06:15:21.287 INFO:tasks.workunit.client.0.smithi006.stdout:6/956: write d6/d77/fec [156860,57750] 0 2024-03-22T06:15:21.287 INFO:tasks.workunit.client.1.smithi117.stdout:4/284: dwrite d0/d3f/f54 [0,4194304] 0 2024-03-22T06:15:21.287 INFO:tasks.workunit.client.1.smithi117.stdout:4/285: chown d0/d4/f17 0 1 2024-03-22T06:15:21.288 INFO:tasks.workunit.client.1.smithi117.stdout:4/286: fdatasync d0/d4/fe 0 2024-03-22T06:15:21.288 INFO:tasks.workunit.client.1.smithi117.stdout:4/287: write d0/d19/d28/f45 [898902,28446] 0 2024-03-22T06:15:21.289 INFO:tasks.workunit.client.1.smithi117.stdout:3/264: dwrite d9/d1a/f4b [0,4194304] 0 2024-03-22T06:15:21.289 INFO:tasks.workunit.client.1.smithi117.stdout:7/273: dread f7 [0,4194304] 0 2024-03-22T06:15:21.291 INFO:tasks.workunit.client.1.smithi117.stdout:2/278: symlink d2/l4d 0 2024-03-22T06:15:21.292 INFO:tasks.workunit.client.1.smithi117.stdout:5/308: sync 2024-03-22T06:15:21.292 INFO:tasks.workunit.client.1.smithi117.stdout:5/309: readlink dd/df/d30/d35/l50 0 2024-03-22T06:15:21.298 INFO:tasks.workunit.client.1.smithi117.stdout:0/312: creat d1/d3/d2b/d3e/d3f/f6d x:0 0 0 2024-03-22T06:15:21.299 INFO:tasks.workunit.client.1.smithi117.stdout:7/274: write d8/d9/da/d10/d23/d27/d32/f44 [720626,41021] 0 2024-03-22T06:15:21.299 INFO:tasks.workunit.client.1.smithi117.stdout:7/275: truncate d8/f2f 367597 0 2024-03-22T06:15:21.304 INFO:tasks.workunit.client.1.smithi117.stdout:4/288: read d0/d19/d1c/f36 [1189889,30330] 0 2024-03-22T06:15:21.307 INFO:tasks.workunit.client.0.smithi006.stdout:6/957: truncate d6/d1b/d2c/f43 502924 0 2024-03-22T06:15:21.307 INFO:tasks.workunit.client.0.smithi006.stdout:6/958: chown d6/d24 0 1 2024-03-22T06:15:21.316 INFO:tasks.workunit.client.1.smithi117.stdout:4/289: dread d0/d19/f1a [0,4194304] 0 2024-03-22T06:15:21.318 INFO:tasks.workunit.client.1.smithi117.stdout:3/265: creat d9/d1a/d39/d4c/d36/f62 x:0 0 0 2024-03-22T06:15:21.319 INFO:tasks.workunit.client.1.smithi117.stdout:9/343: dwrite d4/d7/d10/f42 [0,4194304] 0 2024-03-22T06:15:21.319 INFO:tasks.workunit.client.1.smithi117.stdout:9/344: chown d4/d7/l32 174093046 1 2024-03-22T06:15:21.320 INFO:tasks.workunit.client.1.smithi117.stdout:2/279: creat d2/d5/d13/d3f/f4e x:0 0 0 2024-03-22T06:15:21.320 INFO:tasks.workunit.client.1.smithi117.stdout:7/276: truncate d8/d9/da/d10/d23/d27/d39/f3b 3234532 0 2024-03-22T06:15:21.320 INFO:tasks.workunit.client.1.smithi117.stdout:7/277: fdatasync d8/d9/da/d10/d23/d27/f3e 0 2024-03-22T06:15:21.322 INFO:tasks.workunit.client.1.smithi117.stdout:1/309: symlink d1/l70 0 2024-03-22T06:15:21.322 INFO:tasks.workunit.client.1.smithi117.stdout:3/266: dread d9/d1a/f1b [0,4194304] 0 2024-03-22T06:15:21.322 INFO:tasks.workunit.client.1.smithi117.stdout:3/267: chown d9/d1a/d39/d4c/f46 11 1 2024-03-22T06:15:21.323 INFO:tasks.workunit.client.1.smithi117.stdout:5/310: dwrite dd/df/d1e/f20 [0,4194304] 0 2024-03-22T06:15:21.323 INFO:tasks.workunit.client.1.smithi117.stdout:5/311: dread - dd/d16/d1a/f59 zero size 2024-03-22T06:15:21.323 INFO:tasks.workunit.client.1.smithi117.stdout:5/312: creat dd/d16/f65 x:0 0 0 2024-03-22T06:15:21.324 INFO:tasks.workunit.client.1.smithi117.stdout:9/345: mkdir d4/d47/d64 0 2024-03-22T06:15:21.325 INFO:tasks.workunit.client.1.smithi117.stdout:2/280: mkdir d2/d5/d13/d3f/d1d/d3a/d11/d4f 0 2024-03-22T06:15:21.328 INFO:tasks.workunit.client.1.smithi117.stdout:1/310: mknod d1/d1f/d32/c71 0 2024-03-22T06:15:21.330 INFO:tasks.workunit.client.1.smithi117.stdout:5/313: dread dd/df/d1e/f20 [0,4194304] 0 2024-03-22T06:15:21.334 INFO:tasks.workunit.client.1.smithi117.stdout:4/290: mknod d0/d19/d1c/c61 0 2024-03-22T06:15:21.335 INFO:tasks.workunit.client.1.smithi117.stdout:9/346: link d4/d7/c5e d4/d7/c65 0 2024-03-22T06:15:21.335 INFO:tasks.workunit.client.1.smithi117.stdout:0/313: dwrite d1/f3d [0,4194304] 0 2024-03-22T06:15:21.335 INFO:tasks.workunit.client.1.smithi117.stdout:0/314: chown d1/d10/f32 106700 1 2024-03-22T06:15:21.336 INFO:tasks.workunit.client.1.smithi117.stdout:7/278: dread d8/d9/da/d10/d23/d48/f49 [0,4194304] 0 2024-03-22T06:15:21.337 INFO:tasks.workunit.client.1.smithi117.stdout:1/311: rename d1/d22/d27/l4a to d1/d30/d35/d36/l72 0 2024-03-22T06:15:21.339 INFO:tasks.workunit.client.1.smithi117.stdout:5/314: mkdir dd/d16/d5a/d66 0 2024-03-22T06:15:21.339 INFO:tasks.workunit.client.1.smithi117.stdout:5/315: chown l5 8886204 1 2024-03-22T06:15:21.339 INFO:tasks.workunit.client.1.smithi117.stdout:5/316: getdents dd/df/d30/d60 0 2024-03-22T06:15:21.340 INFO:tasks.workunit.client.0.smithi006.stdout:6/959: dwrite d6/d1d/d32/f41 [0,4194304] 0 2024-03-22T06:15:21.341 INFO:tasks.workunit.client.1.smithi117.stdout:4/291: truncate d0/d19/d34/f37 272225 0 2024-03-22T06:15:21.344 INFO:tasks.workunit.client.0.smithi006.stdout:6/960: rename d6/d6f/d94/ff8 to d6/d1b/dae/dbc/d105/f127 0 2024-03-22T06:15:21.346 INFO:tasks.workunit.client.1.smithi117.stdout:3/268: dwrite d9/f25 [0,4194304] 0 2024-03-22T06:15:21.346 INFO:tasks.workunit.client.1.smithi117.stdout:3/269: write d9/d1a/d3a/f5b [786356,97612] 0 2024-03-22T06:15:21.346 INFO:tasks.workunit.client.1.smithi117.stdout:0/315: read d1/d3/d2b/d3e/fb [1140768,22840] 0 2024-03-22T06:15:21.347 INFO:tasks.workunit.client.1.smithi117.stdout:7/279: creat d8/d9/da/d10/d23/d48/f57 x:0 0 0 2024-03-22T06:15:21.349 INFO:tasks.workunit.client.1.smithi117.stdout:0/316: dread d1/d3/d2b/d3e/de/d1f/d3b/f46 [0,4194304] 0 2024-03-22T06:15:21.349 INFO:tasks.workunit.client.1.smithi117.stdout:0/317: chown d1 1782567017 1 2024-03-22T06:15:21.350 INFO:tasks.workunit.client.1.smithi117.stdout:9/347: rename d4/d7/d10/c1f to d4/d7/d10/d3b/c66 0 2024-03-22T06:15:21.351 INFO:tasks.workunit.client.1.smithi117.stdout:2/281: dwrite d2/d5/d13/d3f/d1d/f46 [0,4194304] 0 2024-03-22T06:15:21.351 INFO:tasks.workunit.client.1.smithi117.stdout:2/282: write d2/f26 [1712223,36714] 0 2024-03-22T06:15:21.352 INFO:tasks.workunit.client.1.smithi117.stdout:5/317: link dd/d16/c4d dd/d16/c67 0 2024-03-22T06:15:21.359 INFO:tasks.workunit.client.1.smithi117.stdout:3/270: unlink d9/d1a/d3a/f5b 0 2024-03-22T06:15:21.360 INFO:tasks.workunit.client.1.smithi117.stdout:7/280: rmdir d8/d9/da/d10/d23/d27/d39 39 2024-03-22T06:15:21.362 INFO:tasks.workunit.client.1.smithi117.stdout:9/348: dread d4/d7/d10/f42 [0,4194304] 0 2024-03-22T06:15:21.362 INFO:tasks.workunit.client.1.smithi117.stdout:9/349: fsync d4/d7/d27/f2e 0 2024-03-22T06:15:21.362 INFO:tasks.workunit.client.1.smithi117.stdout:0/318: mknod d1/d10/d18/c6e 0 2024-03-22T06:15:21.362 INFO:tasks.workunit.client.1.smithi117.stdout:0/319: read d1/d10/f32 [683111,70693] 0 2024-03-22T06:15:21.362 INFO:tasks.workunit.client.1.smithi117.stdout:0/320: write d1/d3/d2b/d3e/d3f/f6d [749447,51694] 0 2024-03-22T06:15:21.363 INFO:tasks.workunit.client.0.smithi006.stdout:6/961: dread d6/d1b/dae/fb4 [0,4194304] 0 2024-03-22T06:15:21.365 INFO:tasks.workunit.client.1.smithi117.stdout:7/281: mknod d8/d9/c58 0 2024-03-22T06:15:21.365 INFO:tasks.workunit.client.1.smithi117.stdout:7/282: truncate d8/d9/da/f28 880661 0 2024-03-22T06:15:21.368 INFO:tasks.workunit.client.0.smithi006.stdout:6/962: rename d6/d111 to d6/d1d/d32/d118/d128 0 2024-03-22T06:15:21.368 INFO:tasks.workunit.client.0.smithi006.stdout:6/963: write d6/d1b/d2c/f72 [3454976,103648] 0 2024-03-22T06:15:21.370 INFO:tasks.workunit.client.1.smithi117.stdout:9/350: rename d4/d47/f4d to d4/d8/f67 0 2024-03-22T06:15:21.370 INFO:tasks.workunit.client.1.smithi117.stdout:9/351: dread - d4/d7/d10/d3b/f57 zero size 2024-03-22T06:15:21.370 INFO:tasks.workunit.client.1.smithi117.stdout:9/352: dread - d4/d7/d10/f63 zero size 2024-03-22T06:15:21.370 INFO:tasks.workunit.client.1.smithi117.stdout:3/271: dread d9/d1a/f2d [0,4194304] 0 2024-03-22T06:15:21.370 INFO:tasks.workunit.client.1.smithi117.stdout:7/283: symlink d8/d9/da/d10/d23/d48/l59 0 2024-03-22T06:15:21.373 INFO:tasks.workunit.client.0.smithi006.stdout:6/964: creat d6/d1b/d2c/d4a/d125/de9/f129 x:0 0 0 2024-03-22T06:15:21.373 INFO:tasks.workunit.client.1.smithi117.stdout:4/292: dwrite d0/d4/d2a/d2f/f53 [0,4194304] 0 2024-03-22T06:15:21.373 INFO:tasks.workunit.client.1.smithi117.stdout:1/312: dwrite d1/d3c/d4d/d5b/f60 [0,4194304] 0 2024-03-22T06:15:21.375 INFO:tasks.workunit.client.1.smithi117.stdout:3/272: rename d9/d1a/f1b to d9/f63 0 2024-03-22T06:15:21.378 INFO:tasks.workunit.client.0.smithi006.stdout:6/965: unlink d6/d7c/d92/fad 0 2024-03-22T06:15:21.389 INFO:tasks.workunit.client.1.smithi117.stdout:7/284: read d8/d9/da/d10/f13 [2970808,469] 0 2024-03-22T06:15:21.389 INFO:tasks.workunit.client.1.smithi117.stdout:7/285: write d8/d9/f1e [2011810,71639] 0 2024-03-22T06:15:21.389 INFO:tasks.workunit.client.1.smithi117.stdout:7/286: write f7 [6494786,104307] 0 2024-03-22T06:15:21.392 INFO:tasks.workunit.client.0.smithi006.stdout:6/966: write d6/d1d/d5c/d81/f7e [488408,42028] 0 2024-03-22T06:15:21.393 INFO:tasks.workunit.client.1.smithi117.stdout:4/293: rename d0/d4/d2a/d2f/f31 to d0/d19/d34/d43/f62 0 2024-03-22T06:15:21.394 INFO:tasks.workunit.client.1.smithi117.stdout:1/313: symlink d1/l73 0 2024-03-22T06:15:21.394 INFO:tasks.workunit.client.1.smithi117.stdout:8/287: sync 2024-03-22T06:15:21.396 INFO:tasks.workunit.client.1.smithi117.stdout:2/283: dwrite d2/d5/d13/d3f/d1d/d3a/d11/f15 [4194304,4194304] 0 2024-03-22T06:15:21.397 INFO:tasks.workunit.client.0.smithi006.stdout:6/967: symlink d6/d1b/dae/dbc/d105/l12a 0 2024-03-22T06:15:21.400 INFO:tasks.workunit.client.1.smithi117.stdout:2/284: truncate d2/d1b/f28 492975 0 2024-03-22T06:15:21.400 INFO:tasks.workunit.client.1.smithi117.stdout:1/314: write d1/fc [9248804,74785] 0 2024-03-22T06:15:21.400 INFO:tasks.workunit.client.1.smithi117.stdout:1/315: chown d1/d30/d50 1 1 2024-03-22T06:15:21.404 INFO:tasks.workunit.client.1.smithi117.stdout:0/321: dwrite d1/d10/f11 [4194304,4194304] 0 2024-03-22T06:15:21.408 INFO:tasks.workunit.client.0.smithi006.stdout:6/968: dread d6/f51 [0,4194304] 0 2024-03-22T06:15:21.408 INFO:tasks.workunit.client.0.smithi006.stdout:6/969: truncate d6/f18 4845239 0 2024-03-22T06:15:21.413 INFO:tasks.workunit.client.1.smithi117.stdout:5/318: dwrite dd/d16/d24/f57 [0,4194304] 0 2024-03-22T06:15:21.413 INFO:tasks.workunit.client.1.smithi117.stdout:5/319: fsync dd/d16/d1a/d32/f51 0 2024-03-22T06:15:21.413 INFO:tasks.workunit.client.1.smithi117.stdout:5/320: write dd/fe [2512238,5094] 0 2024-03-22T06:15:21.413 INFO:tasks.workunit.client.1.smithi117.stdout:7/287: dwrite d8/f16 [4194304,4194304] 0 2024-03-22T06:15:21.414 INFO:tasks.workunit.client.1.smithi117.stdout:7/288: chown f7 70987 1 2024-03-22T06:15:21.414 INFO:tasks.workunit.client.0.smithi006.stdout:6/970: read d6/d1b/d2c/f4b [900818,31703] 0 2024-03-22T06:15:21.414 INFO:tasks.workunit.client.1.smithi117.stdout:9/353: dwrite d4/d7/d10/d2f/f4e [4194304,4194304] 0 2024-03-22T06:15:21.414 INFO:tasks.workunit.client.1.smithi117.stdout:7/289: write d8/d9/da/d10/d23/d27/f3e [460116,95248] 0 2024-03-22T06:15:21.415 INFO:tasks.workunit.client.1.smithi117.stdout:7/290: read d8/f2f [45895,28020] 0 2024-03-22T06:15:21.418 INFO:tasks.workunit.client.1.smithi117.stdout:0/322: write d1/d3/d2b/d3e/de/d1f/f3c [2106480,64198] 0 2024-03-22T06:15:21.418 INFO:tasks.workunit.client.0.smithi006.stdout:6/971: unlink d6/d1b/d2c/d4a/d125/de9/f129 0 2024-03-22T06:15:21.421 INFO:tasks.workunit.client.1.smithi117.stdout:5/321: rename dd/f4a to dd/df/d30/d33/f68 0 2024-03-22T06:15:21.424 INFO:tasks.workunit.client.0.smithi006.stdout:6/972: symlink d6/d1d/d32/d118/d128/l12b 0 2024-03-22T06:15:21.427 INFO:tasks.workunit.client.1.smithi117.stdout:8/288: dwrite d1/d3a/d44/d9/f13 [4194304,4194304] 0 2024-03-22T06:15:21.430 INFO:tasks.workunit.client.1.smithi117.stdout:1/316: dwrite d1/d3c/d4d/d5b/f61 [0,4194304] 0 2024-03-22T06:15:21.430 INFO:tasks.workunit.client.1.smithi117.stdout:8/289: dread d1/d19/f2d [0,4194304] 0 2024-03-22T06:15:21.430 INFO:tasks.workunit.client.1.smithi117.stdout:8/290: truncate d1/d3a/d44/f3c 812269 0 2024-03-22T06:15:21.430 INFO:tasks.workunit.client.1.smithi117.stdout:8/291: write d1/d3a/d44/d11/f2a [1375484,110002] 0 2024-03-22T06:15:21.430 INFO:tasks.workunit.client.1.smithi117.stdout:9/354: rmdir d4/d8/d53 0 2024-03-22T06:15:21.433 INFO:tasks.workunit.client.1.smithi117.stdout:7/291: symlink d8/d9/d37/l5a 0 2024-03-22T06:15:21.435 INFO:tasks.workunit.client.1.smithi117.stdout:2/285: dwrite d2/d5/d13/d3f/d1d/f46 [4194304,4194304] 0 2024-03-22T06:15:21.436 INFO:tasks.workunit.client.1.smithi117.stdout:0/323: dwrite d1/d3/f12 [0,4194304] 0 2024-03-22T06:15:21.441 INFO:tasks.workunit.client.1.smithi117.stdout:8/292: dread f0 [0,4194304] 0 2024-03-22T06:15:21.441 INFO:tasks.workunit.client.1.smithi117.stdout:5/322: unlink dd/d16/c5b 0 2024-03-22T06:15:21.441 INFO:tasks.workunit.client.1.smithi117.stdout:5/323: chown dd/df/c58 0 1 2024-03-22T06:15:21.441 INFO:tasks.workunit.client.1.smithi117.stdout:5/324: dread - dd/d16/d1a/d32/f51 zero size 2024-03-22T06:15:21.441 INFO:tasks.workunit.client.1.smithi117.stdout:8/293: write d1/d19/f43 [3316658,64449] 0 2024-03-22T06:15:21.441 INFO:tasks.workunit.client.1.smithi117.stdout:5/325: chown dd/df/d1e/f20 1266906802 1 2024-03-22T06:15:21.441 INFO:tasks.workunit.client.1.smithi117.stdout:8/294: write d1/d3a/d44/f1d [4375097,41672] 0 2024-03-22T06:15:21.442 INFO:tasks.workunit.client.1.smithi117.stdout:6/346: sync 2024-03-22T06:15:21.444 INFO:tasks.workunit.client.1.smithi117.stdout:1/317: symlink d1/d3c/d4d/l74 0 2024-03-22T06:15:21.445 INFO:tasks.workunit.client.1.smithi117.stdout:1/318: creat d1/d1f/f75 x:0 0 0 2024-03-22T06:15:21.445 INFO:tasks.workunit.client.1.smithi117.stdout:7/292: stat d8/d9/d37/c4f 0 2024-03-22T06:15:21.448 INFO:tasks.workunit.client.1.smithi117.stdout:9/355: dwrite d4/d7/d10/d3b/f57 [0,4194304] 0 2024-03-22T06:15:21.448 INFO:tasks.workunit.client.1.smithi117.stdout:9/356: write d4/f21 [1388549,129664] 0 2024-03-22T06:15:21.449 INFO:tasks.workunit.client.0.smithi006.stdout:6/973: rename d6/d1b/dae/dbc to d6/d1d/d12c 0 2024-03-22T06:15:21.449 INFO:tasks.workunit.client.0.smithi006.stdout:6/974: write d6/d1b/d2c/d4a/d125/dcd/f11f [498918,29674] 0 2024-03-22T06:15:21.452 INFO:tasks.workunit.client.1.smithi117.stdout:2/286: symlink d2/d35/l50 0 2024-03-22T06:15:21.453 INFO:tasks.workunit.client.1.smithi117.stdout:2/287: read d2/d5/db/f24 [709730,114294] 0 2024-03-22T06:15:21.453 INFO:tasks.workunit.client.1.smithi117.stdout:2/288: creat d2/d5/d13/d3f/f51 x:0 0 0 2024-03-22T06:15:21.454 INFO:tasks.workunit.client.1.smithi117.stdout:0/324: creat d1/d10/d18/d26/dc/d39/f6f x:0 0 0 2024-03-22T06:15:21.459 INFO:tasks.workunit.client.1.smithi117.stdout:5/326: symlink dd/df/d30/l69 0 2024-03-22T06:15:21.460 INFO:tasks.workunit.client.1.smithi117.stdout:9/357: dread d4/f21 [0,4194304] 0 2024-03-22T06:15:21.461 INFO:tasks.workunit.client.1.smithi117.stdout:8/295: mknod d1/d3a/c5d 0 2024-03-22T06:15:21.465 INFO:tasks.workunit.client.1.smithi117.stdout:1/319: creat d1/f76 x:0 0 0 2024-03-22T06:15:21.466 INFO:tasks.workunit.client.1.smithi117.stdout:6/347: dwrite d3/f5 [0,4194304] 0 2024-03-22T06:15:21.466 INFO:tasks.workunit.client.1.smithi117.stdout:6/348: dread - d3/d40/d46/f6b zero size 2024-03-22T06:15:21.466 INFO:tasks.workunit.client.1.smithi117.stdout:6/349: fdatasync d3/d3d/f2e 0 2024-03-22T06:15:21.466 INFO:tasks.workunit.client.0.smithi006.stdout:6/975: dwrite d6/d1b/dae/fb6 [0,4194304] 0 2024-03-22T06:15:21.467 INFO:tasks.workunit.client.1.smithi117.stdout:7/293: link d8/d9/da/d10/d23/c51 d8/d9/da/c5b 0 2024-03-22T06:15:21.468 INFO:tasks.workunit.client.1.smithi117.stdout:7/294: dread d8/d9/da/d10/d23/d27/d32/f54 [0,4194304] 0 2024-03-22T06:15:21.468 INFO:tasks.workunit.client.1.smithi117.stdout:7/295: chown d8/d9/da/d2b/c31 47177 1 2024-03-22T06:15:21.468 INFO:tasks.workunit.client.1.smithi117.stdout:7/296: read - d8/f3a zero size 2024-03-22T06:15:21.473 INFO:tasks.workunit.client.1.smithi117.stdout:2/289: mknod d2/d1b/d39/c52 0 2024-03-22T06:15:21.475 INFO:tasks.workunit.client.1.smithi117.stdout:0/325: mknod d1/d3/d2b/d3e/c70 0 2024-03-22T06:15:21.476 INFO:tasks.workunit.client.0.smithi006.stdout:6/976: symlink d6/d24/d44/l12d 0 2024-03-22T06:15:21.477 INFO:tasks.workunit.client.0.smithi006.stdout:6/977: truncate d6/d1d/d5c/d89/f8b 4224868 0 2024-03-22T06:15:21.478 INFO:tasks.workunit.client.1.smithi117.stdout:5/327: mkdir dd/df/d30/d35/d6a 0 2024-03-22T06:15:21.478 INFO:tasks.workunit.client.1.smithi117.stdout:8/296: dwrite d1/f1e [0,4194304] 0 2024-03-22T06:15:21.482 INFO:tasks.workunit.client.1.smithi117.stdout:1/320: mkdir d1/d30/d35/d36/d77 0 2024-03-22T06:15:21.484 INFO:tasks.workunit.client.0.smithi006.stdout:6/978: getdents d6/d77 0 2024-03-22T06:15:21.484 INFO:tasks.workunit.client.0.smithi006.stdout:6/979: readlink d6/d1d/d5c/d81/d8a/dac/ldf 0 2024-03-22T06:15:21.487 INFO:tasks.workunit.client.1.smithi117.stdout:5/328: dread dd/fe [0,4194304] 0 2024-03-22T06:15:21.489 INFO:tasks.workunit.client.0.smithi006.stdout:6/980: getdents d6/d24 0 2024-03-22T06:15:21.489 INFO:tasks.workunit.client.0.smithi006.stdout:6/981: write d6/d7c/d92/f120 [870404,17854] 0 2024-03-22T06:15:21.489 INFO:tasks.workunit.client.0.smithi006.stdout:6/982: readlink d6/d1b/d2c/l47 0 2024-03-22T06:15:21.489 INFO:tasks.workunit.client.1.smithi117.stdout:8/297: dread d1/d19/f3f [0,4194304] 0 2024-03-22T06:15:21.492 INFO:tasks.workunit.client.1.smithi117.stdout:0/326: dwrite d1/d3/d2b/d3e/de/d1f/d3b/f42 [0,4194304] 0 2024-03-22T06:15:21.497 INFO:tasks.workunit.client.1.smithi117.stdout:7/297: unlink d8/d9/da/c35 0 2024-03-22T06:15:21.497 INFO:tasks.workunit.client.1.smithi117.stdout:7/298: fsync d8/d9/da/d10/d23/f38 0 2024-03-22T06:15:21.498 INFO:tasks.workunit.client.0.smithi006.stdout:0/863: sync 2024-03-22T06:15:21.498 INFO:tasks.workunit.client.0.smithi006.stdout:0/864: write db/dc/d38/f94 [1882507,82170] 0 2024-03-22T06:15:21.498 INFO:tasks.workunit.client.1.smithi117.stdout:2/290: truncate d2/f26 2212514 0 2024-03-22T06:15:21.498 INFO:tasks.workunit.client.1.smithi117.stdout:2/291: dread - d2/d5/d13/d3f/f51 zero size 2024-03-22T06:15:21.498 INFO:tasks.workunit.client.1.smithi117.stdout:2/292: write d2/d5/d13/f19 [4621449,64742] 0 2024-03-22T06:15:21.500 INFO:tasks.workunit.client.1.smithi117.stdout:9/358: symlink d4/l68 0 2024-03-22T06:15:21.500 INFO:tasks.workunit.client.1.smithi117.stdout:9/359: chown d4/d7/d27/d5b 779 1 2024-03-22T06:15:21.502 INFO:tasks.workunit.client.0.smithi006.stdout:0/865: symlink db/d3d/d4e/d114/l12f 0 2024-03-22T06:15:21.502 INFO:tasks.workunit.client.0.smithi006.stdout:1/991: sync 2024-03-22T06:15:21.503 INFO:tasks.workunit.client.1.smithi117.stdout:1/321: mknod d1/d30/d5d/c78 0 2024-03-22T06:15:21.503 INFO:tasks.workunit.client.1.smithi117.stdout:1/322: creat d1/d1f/f79 x:0 0 0 2024-03-22T06:15:21.504 INFO:tasks.workunit.client.1.smithi117.stdout:5/329: creat dd/df/d30/d60/f6b x:0 0 0 2024-03-22T06:15:21.504 INFO:tasks.workunit.client.1.smithi117.stdout:5/330: creat dd/d16/f6c x:0 0 0 2024-03-22T06:15:21.504 INFO:tasks.workunit.client.1.smithi117.stdout:5/331: write dd/d16/f64 [892355,127352] 0 2024-03-22T06:15:21.504 INFO:tasks.workunit.client.1.smithi117.stdout:5/332: chown dd/d16/c53 560865251 1 2024-03-22T06:15:21.505 INFO:tasks.workunit.client.1.smithi117.stdout:5/333: fdatasync dd/d16/d24/d36/f25 0 2024-03-22T06:15:21.505 INFO:tasks.workunit.client.0.smithi006.stdout:0/866: creat db/d3d/d4e/d114/d124/f130 x:0 0 0 2024-03-22T06:15:21.507 INFO:tasks.workunit.client.1.smithi117.stdout:8/298: creat d1/d3a/d44/d9/f5e x:0 0 0 2024-03-22T06:15:21.507 INFO:tasks.workunit.client.1.smithi117.stdout:8/299: truncate d1/f1e 4577306 0 2024-03-22T06:15:21.511 INFO:tasks.workunit.client.0.smithi006.stdout:6/983: dwrite d6/d1b/d2c/d4a/d125/dcd/f124 [0,4194304] 0 2024-03-22T06:15:21.519 INFO:tasks.workunit.client.0.smithi006.stdout:0/867: write db/d10/f14 [3667263,72903] 0 2024-03-22T06:15:21.520 INFO:tasks.workunit.client.1.smithi117.stdout:1/323: dread d1/d14/f19 [4194304,4194304] 0 2024-03-22T06:15:21.521 INFO:tasks.workunit.client.1.smithi117.stdout:4/294: sync 2024-03-22T06:15:21.521 INFO:tasks.workunit.client.1.smithi117.stdout:3/273: sync 2024-03-22T06:15:21.521 INFO:tasks.workunit.client.1.smithi117.stdout:7/299: mkdir d8/d9/da/d10/d23/d27/d39/d5c 0 2024-03-22T06:15:21.521 INFO:tasks.workunit.client.1.smithi117.stdout:7/300: fdatasync d8/d9/d37/f4e 0 2024-03-22T06:15:21.523 INFO:tasks.workunit.client.1.smithi117.stdout:5/334: dwrite dd/d16/f22 [0,4194304] 0 2024-03-22T06:15:21.523 INFO:tasks.workunit.client.1.smithi117.stdout:5/335: creat dd/f6d x:0 0 0 2024-03-22T06:15:21.526 INFO:tasks.workunit.client.1.smithi117.stdout:2/293: mknod d2/c53 0 2024-03-22T06:15:21.526 INFO:tasks.workunit.client.1.smithi117.stdout:2/294: read d2/d5/db/f24 [624182,103193] 0 2024-03-22T06:15:21.534 INFO:tasks.workunit.client.0.smithi006.stdout:1/992: write d8/d9/d13/d39/d60/fd4 [75053,51961] 0 2024-03-22T06:15:21.535 INFO:tasks.workunit.client.0.smithi006.stdout:1/993: stat d8/d9/d13/d24/d32/d34/fb3 0 2024-03-22T06:15:21.535 INFO:tasks.workunit.client.0.smithi006.stdout:1/994: creat d8/d71/f146 x:0 0 0 2024-03-22T06:15:21.535 INFO:tasks.workunit.client.0.smithi006.stdout:1/995: dread - d8/d9/d13/d24/d32/dc0/f131 zero size 2024-03-22T06:15:21.535 INFO:tasks.workunit.client.0.smithi006.stdout:1/996: truncate d8/d9/d13/d39/d60/d8f/d10a/d59/db1/ff8 391411 0 2024-03-22T06:15:21.535 INFO:tasks.workunit.client.1.smithi117.stdout:0/327: rmdir d1/d3/d2b/d3e 39 2024-03-22T06:15:21.536 INFO:tasks.workunit.client.1.smithi117.stdout:8/300: dwrite d1/d3a/d44/d11/f2a [0,4194304] 0 2024-03-22T06:15:21.536 INFO:tasks.workunit.client.0.smithi006.stdout:0/868: dread db/d10/f2f [4194304,4194304] 0 2024-03-22T06:15:21.536 INFO:tasks.workunit.client.0.smithi006.stdout:0/869: getdents db/d10/d2a/d41/dff/d118 0 2024-03-22T06:15:21.537 INFO:tasks.workunit.client.0.smithi006.stdout:6/984: dwrite d6/d1d/d12c/d105/f110 [0,4194304] 0 2024-03-22T06:15:21.537 INFO:tasks.workunit.client.0.smithi006.stdout:6/985: dread - d6/f9c zero size 2024-03-22T06:15:21.537 INFO:tasks.workunit.client.1.smithi117.stdout:6/350: rename d3/d7 to d3/d40/d24/d2d/d53/d59/d74 0 2024-03-22T06:15:21.540 INFO:tasks.workunit.client.1.smithi117.stdout:1/324: creat d1/d22/d27/d4e/f7a x:0 0 0 2024-03-22T06:15:21.541 INFO:tasks.workunit.client.0.smithi006.stdout:1/997: mkdir d8/d9/d3a/d18/d147 0 2024-03-22T06:15:21.543 INFO:tasks.workunit.client.1.smithi117.stdout:4/295: mkdir d0/d19/d34/d43/d63 0 2024-03-22T06:15:21.547 INFO:tasks.workunit.client.1.smithi117.stdout:5/336: mkdir dd/df/d30/d60/d6e 0 2024-03-22T06:15:21.547 INFO:tasks.workunit.client.1.smithi117.stdout:5/337: read dd/df/d1e/f4c [110727,6714] 0 2024-03-22T06:15:21.548 INFO:tasks.workunit.client.1.smithi117.stdout:2/295: dwrite d2/d5/db/d2c/f42 [0,4194304] 0 2024-03-22T06:15:21.549 INFO:tasks.workunit.client.1.smithi117.stdout:0/328: unlink d1/d10/l67 0 2024-03-22T06:15:21.552 INFO:tasks.workunit.client.0.smithi006.stdout:6/986: unlink d6/d1b/d2c/d4a/d125/de9/c106 0 2024-03-22T06:15:21.553 INFO:tasks.workunit.client.0.smithi006.stdout:1/998: write d8/d9/d13/f26 [3721522,18863] 0 2024-03-22T06:15:21.553 INFO:tasks.workunit.client.1.smithi117.stdout:8/301: symlink d1/d28/l5f 0 2024-03-22T06:15:21.554 INFO:tasks.workunit.client.1.smithi117.stdout:8/302: stat d1/d19/f43 0 2024-03-22T06:15:21.554 INFO:tasks.workunit.client.1.smithi117.stdout:8/303: read d1/d19/f2d [564388,71388] 0 2024-03-22T06:15:21.554 INFO:tasks.workunit.client.1.smithi117.stdout:8/304: truncate d1/f34 4279191 0 2024-03-22T06:15:21.555 INFO:tasks.workunit.client.0.smithi006.stdout:0/870: mknod db/dc/c131 0 2024-03-22T06:15:21.555 INFO:tasks.workunit.client.0.smithi006.stdout:0/871: write db/d10/d33/d9d/dcd/fd4 [765813,4522] 0 2024-03-22T06:15:21.556 INFO:tasks.workunit.client.0.smithi006.stdout:6/987: dread d6/d24/f54 [0,4194304] 0 2024-03-22T06:15:21.558 INFO:tasks.workunit.client.1.smithi117.stdout:9/360: rename d4/d8/c3c to d4/d8/d37/c69 0 2024-03-22T06:15:21.559 INFO:tasks.workunit.client.0.smithi006.stdout:1/999: getdents d8/d9/d3a/d18 0 2024-03-22T06:15:21.561 INFO:tasks.workunit.client.1.smithi117.stdout:9/361: dread d4/d7/f2a [0,4194304] 0 2024-03-22T06:15:21.562 INFO:tasks.workunit.client.0.smithi006.stdout:0/872: dread db/dc/d38/f94 [0,4194304] 0 2024-03-22T06:15:21.563 INFO:tasks.workunit.client.0.smithi006.stdout:6/988: read d6/d1b/f82 [440511,48783] 0 2024-03-22T06:15:21.565 INFO:tasks.workunit.client.0.smithi006.stdout:0/873: dread db/dc/d38/f94 [0,4194304] 0 2024-03-22T06:15:21.566 INFO:tasks.workunit.client.1.smithi117.stdout:4/296: unlink d0/d19/d1c/c61 0 2024-03-22T06:15:21.570 INFO:tasks.workunit.client.1.smithi117.stdout:5/338: link dd/f29 dd/df/d30/d35/f6f 0 2024-03-22T06:15:21.571 INFO:tasks.workunit.client.0.smithi006.stdout:6/989: dread d6/f51 [0,4194304] 0 2024-03-22T06:15:21.571 INFO:tasks.workunit.client.0.smithi006.stdout:6/990: dread - d6/d1d/fdd zero size 2024-03-22T06:15:21.572 INFO:tasks.workunit.client.0.smithi006.stdout:6/991: readlink d6/d24/d44/l12d 0 2024-03-22T06:15:21.572 INFO:tasks.workunit.client.1.smithi117.stdout:2/296: mknod d2/d5/d41/c54 0 2024-03-22T06:15:21.574 INFO:tasks.workunit.client.1.smithi117.stdout:0/329: creat d1/d3/d2b/d3e/d3f/f71 x:0 0 0 2024-03-22T06:15:21.574 INFO:tasks.workunit.client.1.smithi117.stdout:0/330: readlink d1/d3/l56 0 2024-03-22T06:15:21.574 INFO:tasks.workunit.client.0.smithi006.stdout:6/992: getdents d6/d7c/dc7 0 2024-03-22T06:15:21.575 INFO:tasks.workunit.client.1.smithi117.stdout:0/331: write d1/d3/d2b/d3e/d3f/f6d [451639,25584] 0 2024-03-22T06:15:21.575 INFO:tasks.workunit.client.1.smithi117.stdout:0/332: write d1/d3/d2b/d3e/de/d1f/d59/f5f [209058,71382] 0 2024-03-22T06:15:21.575 INFO:tasks.workunit.client.1.smithi117.stdout:4/297: dread d0/f3 [4194304,4194304] 0 2024-03-22T06:15:21.575 INFO:tasks.workunit.client.1.smithi117.stdout:8/305: mknod d1/d28/c60 0 2024-03-22T06:15:21.578 INFO:tasks.workunit.client.1.smithi117.stdout:6/351: rename d3/d40/d24/d2d/d30/d3e/d6a to d3/d40/d24/d2d/d53/d75 0 2024-03-22T06:15:21.579 INFO:tasks.workunit.client.0.smithi006.stdout:6/993: mkdir d6/d1b/d12e 0 2024-03-22T06:15:21.580 INFO:tasks.workunit.client.1.smithi117.stdout:9/362: creat d4/d7/d27/d5b/f6a x:0 0 0 2024-03-22T06:15:21.581 INFO:tasks.workunit.client.0.smithi006.stdout:0/874: dwrite db/d10/d35/d7b/da1/da6/dc4/f128 [0,4194304] 0 2024-03-22T06:15:21.581 INFO:tasks.workunit.client.1.smithi117.stdout:5/339: mkdir dd/d16/d24/d36/d70 0 2024-03-22T06:15:21.583 INFO:tasks.workunit.client.0.smithi006.stdout:6/994: rename d6/d1b/d2c/d4a/d125/ca3 to d6/d7c/c12f 0 2024-03-22T06:15:21.585 INFO:tasks.workunit.client.1.smithi117.stdout:0/333: mkdir d1/d10/d5c/d40/d69/d72 0 2024-03-22T06:15:21.587 INFO:tasks.workunit.client.1.smithi117.stdout:4/298: mknod d0/d48/c64 0 2024-03-22T06:15:21.587 INFO:tasks.workunit.client.1.smithi117.stdout:4/299: write d0/d4/f17 [106184,22695] 0 2024-03-22T06:15:21.587 INFO:tasks.workunit.client.1.smithi117.stdout:9/363: dread d4/d7/f1b [0,4194304] 0 2024-03-22T06:15:21.588 INFO:tasks.workunit.client.1.smithi117.stdout:8/306: symlink d1/l61 0 2024-03-22T06:15:21.589 INFO:tasks.workunit.client.1.smithi117.stdout:6/352: creat d3/d40/d24/d2d/d42/f76 x:0 0 0 2024-03-22T06:15:21.594 INFO:tasks.workunit.client.0.smithi006.stdout:0/875: dread db/d87/f93 [0,4194304] 0 2024-03-22T06:15:21.594 INFO:tasks.workunit.client.1.smithi117.stdout:1/325: rename d1/d1f/d32/f54 to d1/d30/d44/f7b 0 2024-03-22T06:15:21.597 INFO:tasks.workunit.client.0.smithi006.stdout:6/995: read d6/f42 [1891454,51686] 0 2024-03-22T06:15:21.598 INFO:tasks.workunit.client.1.smithi117.stdout:0/334: creat d1/d10/d5c/d40/f73 x:0 0 0 2024-03-22T06:15:21.601 INFO:tasks.workunit.client.0.smithi006.stdout:6/996: mknod d6/d1d/d5c/d81/d8a/c130 0 2024-03-22T06:15:21.601 INFO:tasks.workunit.client.0.smithi006.stdout:6/997: chown d6/d1d/d5c/d89 30 1 2024-03-22T06:15:21.601 INFO:tasks.workunit.client.0.smithi006.stdout:0/876: dread db/d3d/d4e/f54 [0,4194304] 0 2024-03-22T06:15:21.602 INFO:tasks.workunit.client.1.smithi117.stdout:4/300: symlink d0/d5c/l65 0 2024-03-22T06:15:21.603 INFO:tasks.workunit.client.1.smithi117.stdout:4/301: write d0/d3f/f4b [374009,4074] 0 2024-03-22T06:15:21.603 INFO:tasks.workunit.client.1.smithi117.stdout:9/364: creat d4/d50/f6b x:0 0 0 2024-03-22T06:15:21.603 INFO:tasks.workunit.client.1.smithi117.stdout:9/365: chown d4/d47 426 1 2024-03-22T06:15:21.604 INFO:tasks.workunit.client.0.smithi006.stdout:6/998: rename d6/d1d/l6d to d6/d7c/d92/l131 0 2024-03-22T06:15:21.606 INFO:tasks.workunit.client.1.smithi117.stdout:2/297: rename d2/d5/d13/d3f/d1d/d3a/d11/c2a to d2/d5/db/c55 0 2024-03-22T06:15:21.609 INFO:tasks.workunit.client.1.smithi117.stdout:1/326: mkdir d1/d62/d7c 0 2024-03-22T06:15:21.611 INFO:tasks.workunit.client.0.smithi006.stdout:0/877: rename db/d10/d35/d7b/l95 to db/d10/dc2/l132 0 2024-03-22T06:15:21.611 INFO:tasks.workunit.client.0.smithi006.stdout:0/878: chown db/d87/ddd/f10b 33237 1 2024-03-22T06:15:21.611 INFO:tasks.workunit.client.0.smithi006.stdout:0/879: dread - db/d10/d33/d9d/dcd/d106/f110 zero size 2024-03-22T06:15:21.611 INFO:tasks.workunit.client.0.smithi006.stdout:0/880: write db/d10/f23 [4590610,12191] 0 2024-03-22T06:15:21.612 INFO:tasks.workunit.client.1.smithi117.stdout:5/340: symlink dd/df/l71 0 2024-03-22T06:15:21.613 INFO:tasks.workunit.client.1.smithi117.stdout:5/341: write dd/d16/f64 [42226,39511] 0 2024-03-22T06:15:21.614 INFO:tasks.workunit.client.1.smithi117.stdout:0/335: symlink d1/d10/d5c/d40/l74 0 2024-03-22T06:15:21.614 INFO:tasks.workunit.client.1.smithi117.stdout:0/336: truncate d1/d3/d2b/d3e/de/d1f/d3b/d55/f5e 310338 0 2024-03-22T06:15:21.614 INFO:tasks.workunit.client.1.smithi117.stdout:0/337: creat d1/d10/d18/d26/f75 x:0 0 0 2024-03-22T06:15:21.614 INFO:tasks.workunit.client.1.smithi117.stdout:0/338: write d1/d10/d18/d26/dc/f47 [988510,101632] 0 2024-03-22T06:15:21.614 INFO:tasks.workunit.client.0.smithi006.stdout:0/881: getdents db/d10/d33/db4 0 2024-03-22T06:15:21.614 INFO:tasks.workunit.client.0.smithi006.stdout:0/882: write db/d3d/d4e/d114/d124/f108 [213408,115334] 0 2024-03-22T06:15:21.617 INFO:tasks.workunit.client.1.smithi117.stdout:8/307: rename d1/d3a/d44/d9/l17 to d1/d19/l62 0 2024-03-22T06:15:21.617 INFO:tasks.workunit.client.1.smithi117.stdout:2/298: mknod d2/d5/d13/d3f/c56 0 2024-03-22T06:15:21.617 INFO:tasks.workunit.client.1.smithi117.stdout:1/327: stat d1/d22/d27/d4e/l6d 0 2024-03-22T06:15:21.618 INFO:tasks.workunit.client.1.smithi117.stdout:1/328: write d1/d30/d35/d36/f65 [427326,94803] 0 2024-03-22T06:15:21.618 INFO:tasks.workunit.client.0.smithi006.stdout:0/883: dread db/f96 [0,4194304] 0 2024-03-22T06:15:21.618 INFO:tasks.workunit.client.0.smithi006.stdout:0/884: write db/d10/d2a/ff6 [681755,107023] 0 2024-03-22T06:15:21.619 INFO:tasks.workunit.client.0.smithi006.stdout:0/885: chown db/fe9 58 1 2024-03-22T06:15:21.619 INFO:tasks.workunit.client.0.smithi006.stdout:0/886: chown db/d3d/d4e/f9b 12249 1 2024-03-22T06:15:21.619 INFO:tasks.workunit.client.0.smithi006.stdout:0/887: write db/dc/d103/f113 [289737,40914] 0 2024-03-22T06:15:21.620 INFO:tasks.workunit.client.1.smithi117.stdout:6/353: rename d3/d40/d24/d2d/d53/d59/d74/d28/c35 to d3/d3d/d17/d31/c77 0 2024-03-22T06:15:21.622 INFO:tasks.workunit.client.1.smithi117.stdout:3/274: sync 2024-03-22T06:15:21.622 INFO:tasks.workunit.client.1.smithi117.stdout:7/301: sync 2024-03-22T06:15:21.622 INFO:tasks.workunit.client.1.smithi117.stdout:3/275: read - d9/d1a/d39/d4c/d36/f62 zero size 2024-03-22T06:15:21.623 INFO:tasks.workunit.client.1.smithi117.stdout:8/308: mkdir d1/d19/d38/d63 0 2024-03-22T06:15:21.623 INFO:tasks.workunit.client.1.smithi117.stdout:7/302: write d8/d9/f1e [313538,54733] 0 2024-03-22T06:15:21.623 INFO:tasks.workunit.client.1.smithi117.stdout:7/303: write d8/d9/da/d10/f22 [3569624,44691] 0 2024-03-22T06:15:21.623 INFO:tasks.workunit.client.1.smithi117.stdout:7/304: write d8/d9/da/d10/d23/d48/f57 [11105,75543] 0 2024-03-22T06:15:21.624 INFO:tasks.workunit.client.1.smithi117.stdout:4/302: rename d0/d4/c3c to d0/d5c/d5f/c66 0 2024-03-22T06:15:21.624 INFO:tasks.workunit.client.1.smithi117.stdout:7/305: dread d8/d9/da/f28 [0,4194304] 0 2024-03-22T06:15:21.624 INFO:tasks.workunit.client.1.smithi117.stdout:7/306: write d8/d9/da/d10/d23/d48/f49 [1982243,26813] 0 2024-03-22T06:15:21.625 INFO:tasks.workunit.client.1.smithi117.stdout:3/276: link d9/d1a/d39/d4c/d37/c51 d9/d1a/c64 0 2024-03-22T06:15:21.628 INFO:tasks.workunit.client.1.smithi117.stdout:8/309: creat d1/d19/f64 x:0 0 0 2024-03-22T06:15:21.629 INFO:tasks.workunit.client.1.smithi117.stdout:6/354: dread d3/f3f [0,4194304] 0 2024-03-22T06:15:21.629 INFO:tasks.workunit.client.1.smithi117.stdout:6/355: fdatasync d3/d40/d24/d2d/d53/d59/d74/da/d56/d3a/f69 0 2024-03-22T06:15:21.630 INFO:tasks.workunit.client.1.smithi117.stdout:9/366: rename d4/d7/f19 to d4/d7/d10/f6c 0 2024-03-22T06:15:21.630 INFO:tasks.workunit.client.1.smithi117.stdout:9/367: creat d4/d46/f6d x:0 0 0 2024-03-22T06:15:21.631 INFO:tasks.workunit.client.1.smithi117.stdout:4/303: creat d0/d19/d28/d47/f67 x:0 0 0 2024-03-22T06:15:21.631 INFO:tasks.workunit.client.1.smithi117.stdout:4/304: readlink d0/d3f/l46 0 2024-03-22T06:15:21.632 INFO:tasks.workunit.client.1.smithi117.stdout:3/277: creat d9/d1a/d39/d4c/d36/d41/f65 x:0 0 0 2024-03-22T06:15:21.632 INFO:tasks.workunit.client.1.smithi117.stdout:3/278: stat d9/d1a/d39/d4c/d36/f43 0 2024-03-22T06:15:21.634 INFO:tasks.workunit.client.1.smithi117.stdout:9/368: link d4/c18 d4/d8/d3f/c6e 0 2024-03-22T06:15:21.635 INFO:tasks.workunit.client.1.smithi117.stdout:4/305: symlink d0/d19/d34/d43/l68 0 2024-03-22T06:15:21.638 INFO:tasks.workunit.client.1.smithi117.stdout:4/306: mknod d0/c69 0 2024-03-22T06:15:21.638 INFO:tasks.workunit.client.0.smithi006.stdout:6/999: dwrite d6/d7c/d83/fa2 [0,4194304] 0 2024-03-22T06:15:21.640 INFO:tasks.workunit.client.1.smithi117.stdout:8/310: write d1/d19/f3f [453868,64086] 0 2024-03-22T06:15:21.641 INFO:tasks.workunit.client.1.smithi117.stdout:5/342: rename dd/df/d30 to dd/d16/d24/d72 0 2024-03-22T06:15:21.641 INFO:tasks.workunit.client.1.smithi117.stdout:5/343: dread - dd/d16/d1a/d32/f51 zero size 2024-03-22T06:15:21.643 INFO:tasks.workunit.client.1.smithi117.stdout:6/356: write d3/d40/d24/d2d/d53/d59/d74/da/de/f19 [4192446,27004] 0 2024-03-22T06:15:21.644 INFO:tasks.workunit.client.1.smithi117.stdout:1/329: dwrite d1/d22/d27/f34 [0,4194304] 0 2024-03-22T06:15:21.647 INFO:tasks.workunit.client.1.smithi117.stdout:0/339: rename d1/d3/d2b/d3e/de/d1f/d3b/f61 to d1/d10/d5c/d40/d69/d72/f76 0 2024-03-22T06:15:21.647 INFO:tasks.workunit.client.1.smithi117.stdout:0/340: read d1/d10/d5c/d40/f41 [2437315,960] 0 2024-03-22T06:15:21.647 INFO:tasks.workunit.client.1.smithi117.stdout:0/341: stat d1/d3/d2b/d3e/de/d1f 0 2024-03-22T06:15:21.647 INFO:tasks.workunit.client.1.smithi117.stdout:8/311: creat d1/d19/d38/d63/f65 x:0 0 0 2024-03-22T06:15:21.651 INFO:tasks.workunit.client.1.smithi117.stdout:5/344: mkdir dd/d16/d1a/d32/d73 0 2024-03-22T06:15:21.657 INFO:tasks.workunit.client.1.smithi117.stdout:0/342: rmdir d1/d3/d2b/d3e/de/d1f/d3b/d55 39 2024-03-22T06:15:21.657 INFO:tasks.workunit.client.1.smithi117.stdout:0/343: write d1/d10/f27 [1155519,17454] 0 2024-03-22T06:15:21.658 INFO:tasks.workunit.client.1.smithi117.stdout:6/357: dread d3/d40/f22 [0,4194304] 0 2024-03-22T06:15:21.659 INFO:tasks.workunit.client.1.smithi117.stdout:6/358: dread d3/d3d/d17/f33 [0,4194304] 0 2024-03-22T06:15:21.660 INFO:tasks.workunit.client.1.smithi117.stdout:5/345: link dd/l1c dd/l74 0 2024-03-22T06:15:21.663 INFO:tasks.workunit.client.1.smithi117.stdout:9/369: dwrite d4/d50/f6b [0,4194304] 0 2024-03-22T06:15:21.663 INFO:tasks.workunit.client.1.smithi117.stdout:8/312: dread d1/d3a/d44/fa [0,4194304] 0 2024-03-22T06:15:21.663 INFO:tasks.workunit.client.1.smithi117.stdout:9/370: dread - d4/d7/d27/f4c zero size 2024-03-22T06:15:21.663 INFO:tasks.workunit.client.1.smithi117.stdout:8/313: fdatasync f0 0 2024-03-22T06:15:21.669 INFO:tasks.workunit.client.1.smithi117.stdout:3/279: dwrite d9/d1a/d3a/f56 [0,4194304] 0 2024-03-22T06:15:21.670 INFO:tasks.workunit.client.1.smithi117.stdout:4/307: dwrite d0/d19/d28/d47/f67 [0,4194304] 0 2024-03-22T06:15:21.670 INFO:tasks.workunit.client.1.smithi117.stdout:4/308: truncate d0/d19/d28/f55 253798 0 2024-03-22T06:15:21.670 INFO:tasks.workunit.client.1.smithi117.stdout:6/359: truncate d3/d40/d24/f44 4009451 0 2024-03-22T06:15:21.671 INFO:tasks.workunit.client.1.smithi117.stdout:9/371: dread d4/f21 [0,4194304] 0 2024-03-22T06:15:21.672 INFO:tasks.workunit.client.1.smithi117.stdout:5/346: mknod dd/df/d1e/c75 0 2024-03-22T06:15:21.675 INFO:tasks.workunit.client.1.smithi117.stdout:2/299: rename d2/d5/d8 to d2/d5/d13/d57 0 2024-03-22T06:15:21.686 INFO:tasks.workunit.client.1.smithi117.stdout:3/280: mkdir d9/d66 0 2024-03-22T06:15:21.687 INFO:tasks.workunit.client.1.smithi117.stdout:8/314: dwrite d1/d3a/d44/d11/f5a [0,4194304] 0 2024-03-22T06:15:21.687 INFO:tasks.workunit.client.0.smithi006.stdout:0/888: dwrite db/d87/f93 [0,4194304] 0 2024-03-22T06:15:21.687 INFO:tasks.workunit.client.0.smithi006.stdout:0/889: truncate db/d10/d33/f67 839771 0 2024-03-22T06:15:21.692 INFO:tasks.workunit.client.1.smithi117.stdout:4/309: mkdir d0/d5c/d6a 0 2024-03-22T06:15:21.697 INFO:tasks.workunit.client.1.smithi117.stdout:6/360: creat d3/d3d/d17/d31/f78 x:0 0 0 2024-03-22T06:15:21.697 INFO:tasks.workunit.client.1.smithi117.stdout:6/361: truncate d3/d3d/d17/d31/f78 636174 0 2024-03-22T06:15:21.699 INFO:tasks.workunit.client.1.smithi117.stdout:5/347: rmdir dd/d16/d24/d72/d35/d6a 0 2024-03-22T06:15:21.701 INFO:tasks.workunit.client.0.smithi006.stdout:0/890: dwrite db/dc/d38/f4b [0,4194304] 0 2024-03-22T06:15:21.701 INFO:tasks.workunit.client.0.smithi006.stdout:0/891: creat db/d87/f133 x:0 0 0 2024-03-22T06:15:21.703 INFO:tasks.workunit.client.1.smithi117.stdout:7/307: rename d8/d9/f14 to d8/d9/da/d10/d23/d27/d39/f5d 0 2024-03-22T06:15:21.704 INFO:tasks.workunit.client.1.smithi117.stdout:2/300: symlink d2/d5/d13/d3f/l58 0 2024-03-22T06:15:21.706 INFO:tasks.workunit.client.0.smithi006.stdout:0/892: chown db/c29 12199535 1 2024-03-22T06:15:21.710 INFO:tasks.workunit.client.1.smithi117.stdout:8/315: mkdir d1/d28/d66 0 2024-03-22T06:15:21.710 INFO:tasks.workunit.client.1.smithi117.stdout:8/316: write d1/d19/f43 [2051187,72252] 0 2024-03-22T06:15:21.712 INFO:tasks.workunit.client.1.smithi117.stdout:2/301: write d2/d5/d13/d3f/d1d/d3a/d11/f15 [6077941,91544] 0 2024-03-22T06:15:21.714 INFO:tasks.workunit.client.1.smithi117.stdout:4/310: truncate d0/d4/f12 2642142 0 2024-03-22T06:15:21.714 INFO:tasks.workunit.client.1.smithi117.stdout:6/362: mkdir d3/d40/d24/d2d/d30/d3e/d49/d79 0 2024-03-22T06:15:21.716 INFO:tasks.workunit.client.1.smithi117.stdout:5/348: symlink dd/d16/d24/d36/d70/l76 0 2024-03-22T06:15:21.716 INFO:tasks.workunit.client.1.smithi117.stdout:7/308: mkdir d8/d50/d5e 0 2024-03-22T06:15:21.717 INFO:tasks.workunit.client.1.smithi117.stdout:8/317: dread d1/d3a/d44/f1d [0,4194304] 0 2024-03-22T06:15:21.718 INFO:tasks.workunit.client.1.smithi117.stdout:8/318: fsync d1/d28/f31 0 2024-03-22T06:15:21.720 INFO:tasks.workunit.client.1.smithi117.stdout:6/363: getdents d3/d40/d24/d2d/d53/d59/d74/da/d56 0 2024-03-22T06:15:21.725 INFO:tasks.workunit.client.1.smithi117.stdout:5/349: mknod dd/df/d5e/c77 0 2024-03-22T06:15:21.727 INFO:tasks.workunit.client.1.smithi117.stdout:7/309: mkdir d8/d9/d37/d5f 0 2024-03-22T06:15:21.729 INFO:tasks.workunit.client.1.smithi117.stdout:4/311: getdents d0/d19/d28/d47 0 2024-03-22T06:15:21.729 INFO:tasks.workunit.client.1.smithi117.stdout:4/312: rename d0/d19 to d0/d19/d28/d47/d6b 22 2024-03-22T06:15:21.730 INFO:tasks.workunit.client.1.smithi117.stdout:4/313: chown d0/d19/d34/d43/f62 393497 1 2024-03-22T06:15:21.730 INFO:tasks.workunit.client.1.smithi117.stdout:4/314: chown d0/d4/d2a/d2f/f56 166764 1 2024-03-22T06:15:21.732 INFO:tasks.workunit.client.1.smithi117.stdout:7/310: mknod d8/d9/da/d10/d23/d48/c60 0 2024-03-22T06:15:21.734 INFO:tasks.workunit.client.1.smithi117.stdout:4/315: dread d0/f42 [0,4194304] 0 2024-03-22T06:15:21.742 INFO:tasks.workunit.client.0.smithi006.stdout:0/893: write db/d3d/d4e/f9b [1591277,35186] 0 2024-03-22T06:15:21.748 INFO:tasks.workunit.client.1.smithi117.stdout:5/350: dwrite dd/d16/d1a/f62 [0,4194304] 0 2024-03-22T06:15:21.748 INFO:tasks.workunit.client.1.smithi117.stdout:5/351: fsync dd/f29 0 2024-03-22T06:15:21.751 INFO:tasks.workunit.client.1.smithi117.stdout:8/319: dwrite d1/f34 [0,4194304] 0 2024-03-22T06:15:21.752 INFO:tasks.workunit.client.1.smithi117.stdout:6/364: dwrite d3/d61/f66 [0,4194304] 0 2024-03-22T06:15:21.752 INFO:tasks.workunit.client.1.smithi117.stdout:8/320: mknod d1/c67 0 2024-03-22T06:15:21.757 INFO:tasks.workunit.client.1.smithi117.stdout:6/365: creat d3/d40/d24/d2d/d53/d59/d74/da/d56/f7a x:0 0 0 2024-03-22T06:15:21.758 INFO:tasks.workunit.client.1.smithi117.stdout:5/352: dread dd/d16/d24/d72/d35/f6f [0,4194304] 0 2024-03-22T06:15:21.758 INFO:tasks.workunit.client.1.smithi117.stdout:5/353: fsync fa 0 2024-03-22T06:15:21.758 INFO:tasks.workunit.client.1.smithi117.stdout:5/354: readlink dd/l49 0 2024-03-22T06:15:21.758 INFO:tasks.workunit.client.1.smithi117.stdout:8/321: mkdir d1/d3a/d44/d16/d68 0 2024-03-22T06:15:21.758 INFO:tasks.workunit.client.1.smithi117.stdout:8/322: truncate d1/d19/f64 960975 0 2024-03-22T06:15:21.760 INFO:tasks.workunit.client.1.smithi117.stdout:6/366: creat d3/d40/d24/d2d/d30/d3e/f7b x:0 0 0 2024-03-22T06:15:21.760 INFO:tasks.workunit.client.1.smithi117.stdout:6/367: chown d3/d40/c48 34545730 1 2024-03-22T06:15:21.760 INFO:tasks.workunit.client.1.smithi117.stdout:8/323: truncate d1/d3a/d44/d9/f13 9433664 0 2024-03-22T06:15:21.760 INFO:tasks.workunit.client.1.smithi117.stdout:8/324: chown d1/d3a/d44/d11/l53 417 1 2024-03-22T06:15:21.762 INFO:tasks.workunit.client.1.smithi117.stdout:6/368: mknod d3/d40/d24/d2d/d53/d59/d74/c7c 0 2024-03-22T06:15:21.763 INFO:tasks.workunit.client.0.smithi006.stdout:0/894: dwrite db/d87/ddd/f121 [0,4194304] 0 2024-03-22T06:15:21.763 INFO:tasks.workunit.client.0.smithi006.stdout:0/895: chown db/d10/d33/d9d 6961 1 2024-03-22T06:15:21.763 INFO:tasks.workunit.client.1.smithi117.stdout:6/369: link d3/d40/d24/d2d/d53/d59/d74/da/de/f19 d3/d40/d24/d2d/d30/d3e/d49/d79/f7d 0 2024-03-22T06:15:21.763 INFO:tasks.workunit.client.1.smithi117.stdout:6/370: fsync d3/f5 0 2024-03-22T06:15:21.765 INFO:tasks.workunit.client.1.smithi117.stdout:6/371: symlink d3/d3d/l7e 0 2024-03-22T06:15:21.765 INFO:tasks.workunit.client.1.smithi117.stdout:6/372: readlink d3/d40/d24/d2d/d53/d59/d74/da/d56/d3a/l55 0 2024-03-22T06:15:21.766 INFO:tasks.workunit.client.1.smithi117.stdout:6/373: read - d3/d40/d24/d2d/d30/d3e/f7b zero size 2024-03-22T06:15:21.766 INFO:tasks.workunit.client.1.smithi117.stdout:5/355: write dd/fe [105256,115238] 0 2024-03-22T06:15:21.766 INFO:tasks.workunit.client.1.smithi117.stdout:5/356: write dd/df/d1e/f4c [913495,20716] 0 2024-03-22T06:15:21.766 INFO:tasks.workunit.client.1.smithi117.stdout:5/357: fdatasync dd/d16/f6c 0 2024-03-22T06:15:21.768 INFO:tasks.workunit.client.0.smithi006.stdout:0/896: rename db/d10/d35/d6c/fb2 to db/d10/d33/dce/f134 0 2024-03-22T06:15:21.769 INFO:tasks.workunit.client.1.smithi117.stdout:6/374: mkdir d3/d40/d24/d2d/d53/d59/d74/da/d7f 0 2024-03-22T06:15:21.769 INFO:tasks.workunit.client.1.smithi117.stdout:6/375: write d3/d3d/d17/d31/f78 [918002,13466] 0 2024-03-22T06:15:21.771 INFO:tasks.workunit.client.0.smithi006.stdout:0/897: mknod db/d10/d35/d7b/da1/da6/dc4/c135 0 2024-03-22T06:15:21.772 INFO:tasks.workunit.client.1.smithi117.stdout:8/325: dread d1/d19/f51 [4194304,4194304] 0 2024-03-22T06:15:21.773 INFO:tasks.workunit.client.1.smithi117.stdout:8/326: fsync d1/f1e 0 2024-03-22T06:15:21.773 INFO:tasks.workunit.client.1.smithi117.stdout:8/327: readlink d1/d3a/l5c 0 2024-03-22T06:15:21.773 INFO:tasks.workunit.client.1.smithi117.stdout:5/358: dread dd/df/d1e/f4c [0,4194304] 0 2024-03-22T06:15:21.773 INFO:tasks.workunit.client.0.smithi006.stdout:0/898: dread db/d10/d33/fb0 [0,4194304] 0 2024-03-22T06:15:21.775 INFO:tasks.workunit.client.1.smithi117.stdout:5/359: mknod dd/d16/d5a/c78 0 2024-03-22T06:15:21.776 INFO:tasks.workunit.client.1.smithi117.stdout:5/360: mknod dd/d16/d1a/d32/d73/c79 0 2024-03-22T06:15:21.777 INFO:tasks.workunit.client.0.smithi006.stdout:0/899: dread db/d10/d35/d7b/da1/da6/dc4/ff2 [0,4194304] 0 2024-03-22T06:15:21.778 INFO:tasks.workunit.client.1.smithi117.stdout:5/361: creat dd/d16/d24/d72/d60/d6e/f7a x:0 0 0 2024-03-22T06:15:21.779 INFO:tasks.workunit.client.0.smithi006.stdout:0/900: rename db/dc/d38/d50/ca8 to db/d3d/d4e/c136 0 2024-03-22T06:15:21.779 INFO:tasks.workunit.client.0.smithi006.stdout:0/901: chown db/dc/c22 326 1 2024-03-22T06:15:21.781 INFO:tasks.workunit.client.0.smithi006.stdout:0/902: mkdir db/dc/d37/d137 0 2024-03-22T06:15:21.782 INFO:tasks.workunit.client.0.smithi006.stdout:0/903: truncate db/d10/d33/d9d/fa2 3855290 0 2024-03-22T06:15:21.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:21 smithi006 ceph-mon[29301]: pgmap v24: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 134 MiB/s rd, 163 MiB/s wr, 979 op/s 2024-03-22T06:15:21.787 INFO:tasks.workunit.client.0.smithi006.stdout:0/904: rename db/d10/df7 to db/d10/d35/d7b/da1/d138 0 2024-03-22T06:15:21.788 INFO:tasks.workunit.client.1.smithi117.stdout:5/362: fsync dd/d16/d24/d72/d60/d6e/f7a 0 2024-03-22T06:15:21.791 INFO:tasks.workunit.client.1.smithi117.stdout:5/363: symlink dd/d16/d24/d72/l7b 0 2024-03-22T06:15:21.791 INFO:tasks.workunit.client.1.smithi117.stdout:8/328: dwrite d1/d19/f2d [0,4194304] 0 2024-03-22T06:15:21.793 INFO:tasks.workunit.client.1.smithi117.stdout:0/344: sync 2024-03-22T06:15:21.793 INFO:tasks.workunit.client.1.smithi117.stdout:1/330: sync 2024-03-22T06:15:21.796 INFO:tasks.workunit.client.1.smithi117.stdout:8/329: mknod d1/d3a/c69 0 2024-03-22T06:15:21.796 INFO:tasks.workunit.client.1.smithi117.stdout:8/330: chown d1/d3a/d44/d4b/c52 1 1 2024-03-22T06:15:21.797 INFO:tasks.workunit.client.1.smithi117.stdout:1/331: creat d1/d22/d27/d4e/f7d x:0 0 0 2024-03-22T06:15:21.797 INFO:tasks.workunit.client.1.smithi117.stdout:6/376: dwrite d3/d40/d24/d2d/d53/d59/d74/da/d56/d3a/f69 [0,4194304] 0 2024-03-22T06:15:21.797 INFO:tasks.workunit.client.1.smithi117.stdout:6/377: write d3/d40/d24/d2d/d53/d59/d74/da/de/f6f [631620,53009] 0 2024-03-22T06:15:21.797 INFO:tasks.workunit.client.1.smithi117.stdout:6/378: fdatasync d3/d3d/d17/d31/f34 0 2024-03-22T06:15:21.799 INFO:tasks.workunit.client.1.smithi117.stdout:8/331: getdents d1 0 2024-03-22T06:15:21.801 INFO:tasks.workunit.client.1.smithi117.stdout:5/364: read dd/d16/f48 [2229388,82100] 0 2024-03-22T06:15:21.807 INFO:tasks.workunit.client.1.smithi117.stdout:6/379: creat d3/d3d/d17/d60/f80 x:0 0 0 2024-03-22T06:15:21.807 INFO:tasks.workunit.client.0.smithi006.stdout:0/905: dwrite db/d10/d33/f42 [0,4194304] 0 2024-03-22T06:15:21.808 INFO:tasks.workunit.client.0.smithi006.stdout:0/906: write db/dc/d38/fc0 [728453,20447] 0 2024-03-22T06:15:21.808 INFO:tasks.workunit.client.1.smithi117.stdout:8/332: mkdir d1/d3a/d44/d6a 0 2024-03-22T06:15:21.809 INFO:tasks.workunit.client.1.smithi117.stdout:5/365: link fa dd/d16/d24/d72/d33/f7c 0 2024-03-22T06:15:21.809 INFO:tasks.workunit.client.1.smithi117.stdout:5/366: creat dd/f7d x:0 0 0 2024-03-22T06:15:21.810 INFO:tasks.workunit.client.0.smithi006.stdout:0/907: mknod db/d10/d35/d7b/da1/da6/d122/c139 0 2024-03-22T06:15:21.811 INFO:tasks.workunit.client.1.smithi117.stdout:6/380: rmdir d3/d40/d24/d2d/d53/d59/d74/da/d7f 0 2024-03-22T06:15:21.811 INFO:tasks.workunit.client.1.smithi117.stdout:6/381: fsync d3/d40/d24/d2d/d30/d3e/d49/f5b 0 2024-03-22T06:15:21.814 INFO:tasks.workunit.client.0.smithi006.stdout:0/908: rmdir db/d3d/d4e/d76 39 2024-03-22T06:15:21.814 INFO:tasks.workunit.client.0.smithi006.stdout:0/909: readlink db/d3d/d4e/d114/l12f 0 2024-03-22T06:15:21.815 INFO:tasks.workunit.client.1.smithi117.stdout:6/382: mknod d3/d40/c81 0 2024-03-22T06:15:21.816 INFO:tasks.workunit.client.0.smithi006.stdout:0/910: creat db/d10/dc2/f13a x:0 0 0 2024-03-22T06:15:21.816 INFO:tasks.workunit.client.0.smithi006.stdout:0/911: truncate db/d10/d35/d7b/da1/da6/dc4/ff2 1404605 0 2024-03-22T06:15:21.818 INFO:tasks.workunit.client.1.smithi117.stdout:6/383: symlink d3/d40/d24/d2d/d53/d59/d74/d28/l82 0 2024-03-22T06:15:21.820 INFO:tasks.workunit.client.0.smithi006.stdout:0/912: mknod db/c13b 0 2024-03-22T06:15:21.820 INFO:tasks.workunit.client.0.smithi006.stdout:0/913: readlink db/dc/d38/lbe 0 2024-03-22T06:15:21.821 INFO:tasks.workunit.client.1.smithi117.stdout:5/367: dread fa [0,4194304] 0 2024-03-22T06:15:21.827 INFO:tasks.workunit.client.1.smithi117.stdout:1/332: dwrite d1/d62/f67 [0,4194304] 0 2024-03-22T06:15:21.836 INFO:tasks.workunit.client.1.smithi117.stdout:6/384: dread d3/f3f [0,4194304] 0 2024-03-22T06:15:21.844 INFO:tasks.workunit.client.1.smithi117.stdout:1/333: dread d1/f43 [0,4194304] 0 2024-03-22T06:15:21.846 INFO:tasks.workunit.client.1.smithi117.stdout:1/334: symlink d1/l7e 0 2024-03-22T06:15:21.846 INFO:tasks.workunit.client.1.smithi117.stdout:1/335: write d1/d14/f6c [844022,58452] 0 2024-03-22T06:15:21.865 INFO:tasks.workunit.client.0.smithi006.stdout:0/914: dwrite db/d10/d2a/d41/f82 [4194304,4194304] 0 2024-03-22T06:15:21.874 INFO:tasks.workunit.client.1.smithi117.stdout:6/385: dwrite d3/d3d/d17/f33 [0,4194304] 0 2024-03-22T06:15:21.874 INFO:tasks.workunit.client.1.smithi117.stdout:6/386: fsync d3/d40/d24/f44 0 2024-03-22T06:15:21.874 INFO:tasks.workunit.client.1.smithi117.stdout:5/368: dwrite dd/d16/d24/d72/d33/f68 [4194304,4194304] 0 2024-03-22T06:15:21.874 INFO:tasks.workunit.client.1.smithi117.stdout:1/336: dwrite d1/d14/f19 [4194304,4194304] 0 2024-03-22T06:15:21.874 INFO:tasks.workunit.client.1.smithi117.stdout:1/337: dread - d1/d30/d50/f63 zero size 2024-03-22T06:15:21.874 INFO:tasks.workunit.client.1.smithi117.stdout:1/338: creat d1/d1f/f7f x:0 0 0 2024-03-22T06:15:21.874 INFO:tasks.workunit.client.1.smithi117.stdout:1/339: stat d1/d1f/d32/d33 0 2024-03-22T06:15:21.874 INFO:tasks.workunit.client.1.smithi117.stdout:1/340: chown d1/d22/d27/d4e/f7d 0 1 2024-03-22T06:15:21.876 INFO:tasks.workunit.client.1.smithi117.stdout:6/387: mkdir d3/d40/d83 0 2024-03-22T06:15:21.879 INFO:tasks.workunit.client.1.smithi117.stdout:5/369: truncate dd/d16/d1a/f62 834158 0 2024-03-22T06:15:21.879 INFO:tasks.workunit.client.1.smithi117.stdout:5/370: fdatasync dd/df/d1e/f4c 0 2024-03-22T06:15:21.879 INFO:tasks.workunit.client.0.smithi006.stdout:0/915: dread db/d10/d35/d7b/da1/da6/dc4/f128 [0,4194304] 0 2024-03-22T06:15:21.880 INFO:tasks.workunit.client.0.smithi006.stdout:0/916: write db/d10/dc2/fd6 [844978,48404] 0 2024-03-22T06:15:21.880 INFO:tasks.workunit.client.1.smithi117.stdout:1/341: mkdir d1/d30/d35/d58/d80 0 2024-03-22T06:15:21.882 INFO:tasks.workunit.client.1.smithi117.stdout:6/388: dread d3/d40/d24/d2d/d30/d3e/d49/f5b [0,4194304] 0 2024-03-22T06:15:21.883 INFO:tasks.workunit.client.0.smithi006.stdout:0/917: mkdir db/d10/d35/d7b/da1/d138/d13c 0 2024-03-22T06:15:21.886 INFO:tasks.workunit.client.1.smithi117.stdout:5/371: rename dd/df/d1e/f4c to dd/d16/d24/f7e 0 2024-03-22T06:15:21.888 INFO:tasks.workunit.client.0.smithi006.stdout:0/918: symlink db/dc/d37/d137/l13d 0 2024-03-22T06:15:21.889 INFO:tasks.workunit.client.0.smithi006.stdout:0/919: read db/d10/d33/fb0 [1104596,1438] 0 2024-03-22T06:15:21.890 INFO:tasks.workunit.client.0.smithi006.stdout:0/920: dread db/d10/fb5 [0,4194304] 0 2024-03-22T06:15:21.890 INFO:tasks.workunit.client.0.smithi006.stdout:0/921: chown db/dc/c22 88908 1 2024-03-22T06:15:21.890 INFO:tasks.workunit.client.1.smithi117.stdout:6/389: unlink d3/d3d/l7e 0 2024-03-22T06:15:21.890 INFO:tasks.workunit.client.1.smithi117.stdout:1/342: dread d1/d62/f67 [0,4194304] 0 2024-03-22T06:15:21.891 INFO:tasks.workunit.client.1.smithi117.stdout:1/343: truncate d1/d1f/f7f 30446 0 2024-03-22T06:15:21.891 INFO:tasks.workunit.client.0.smithi006.stdout:0/922: mknod db/d3d/d4e/d76/c13e 0 2024-03-22T06:15:21.895 INFO:tasks.workunit.client.1.smithi117.stdout:1/344: symlink d1/d1f/d32/l81 0 2024-03-22T06:15:21.895 INFO:tasks.workunit.client.1.smithi117.stdout:1/345: chown d1/d1f/d32/l81 3360 1 2024-03-22T06:15:21.897 INFO:tasks.workunit.client.1.smithi117.stdout:1/346: symlink d1/d30/d35/l82 0 2024-03-22T06:15:21.898 INFO:tasks.workunit.client.1.smithi117.stdout:5/372: write dd/d16/d24/d72/d33/f68 [4528126,125606] 0 2024-03-22T06:15:21.898 INFO:tasks.workunit.client.1.smithi117.stdout:5/373: fdatasync dd/df/f38 0 2024-03-22T06:15:21.899 INFO:tasks.workunit.client.0.smithi006.stdout:0/923: dread db/d10/d2a/d41/f7f [0,4194304] 0 2024-03-22T06:15:21.900 INFO:tasks.workunit.client.1.smithi117.stdout:6/390: rename d3/d40/d24/d2d/d30/d3e to d3/d40/d46/d84 0 2024-03-22T06:15:21.901 INFO:tasks.workunit.client.1.smithi117.stdout:5/374: mknod dd/d16/d24/d72/c7f 0 2024-03-22T06:15:21.901 INFO:tasks.workunit.client.1.smithi117.stdout:9/372: sync 2024-03-22T06:15:21.902 INFO:tasks.workunit.client.1.smithi117.stdout:9/373: read - d4/d7/d27/f4c zero size 2024-03-22T06:15:21.902 INFO:tasks.workunit.client.1.smithi117.stdout:9/374: fsync d4/d7/d10/d3b/f57 0 2024-03-22T06:15:21.902 INFO:tasks.workunit.client.1.smithi117.stdout:9/375: stat d4/d8/f41 0 2024-03-22T06:15:21.902 INFO:tasks.workunit.client.1.smithi117.stdout:9/376: fsync d4/d7/d27/f2e 0 2024-03-22T06:15:21.902 INFO:tasks.workunit.client.1.smithi117.stdout:9/377: fdatasync d4/d7/d27/f2e 0 2024-03-22T06:15:21.902 INFO:tasks.workunit.client.1.smithi117.stdout:9/378: stat d4/d8/f41 0 2024-03-22T06:15:21.902 INFO:tasks.workunit.client.0.smithi006.stdout:0/924: mkdir db/d10/d35/d7b/da1/da6/d122/d13f 0 2024-03-22T06:15:21.903 INFO:tasks.workunit.client.0.smithi006.stdout:0/925: fsync db/d10/d2a/d41/f12e 0 2024-03-22T06:15:21.905 INFO:tasks.workunit.client.1.smithi117.stdout:1/347: rename d1/d30/d35/d58/d80 to d1/d30/d44/d83 0 2024-03-22T06:15:21.905 INFO:tasks.workunit.client.1.smithi117.stdout:1/348: readlink d1/d3c/d4d/l74 0 2024-03-22T06:15:21.905 INFO:tasks.workunit.client.1.smithi117.stdout:1/349: chown d1/d30/d35 1 1 2024-03-22T06:15:21.908 INFO:tasks.workunit.client.0.smithi006.stdout:0/926: rename db/d10/d35/d7b/c119 to db/dc/c140 0 2024-03-22T06:15:21.909 INFO:tasks.workunit.client.1.smithi117.stdout:9/379: write d4/d7/d27/f2e [3119483,15805] 0 2024-03-22T06:15:21.909 INFO:tasks.workunit.client.1.smithi117.stdout:9/380: creat d4/d7/d10/d2f/f6f x:0 0 0 2024-03-22T06:15:21.911 INFO:tasks.workunit.client.1.smithi117.stdout:5/375: rename dd/d16/d24/d36/c1d to dd/d16/d1a/d32/d73/c80 0 2024-03-22T06:15:21.911 INFO:tasks.workunit.client.1.smithi117.stdout:5/376: read - dd/d16/d24/f4f zero size 2024-03-22T06:15:21.913 INFO:tasks.workunit.client.1.smithi117.stdout:5/377: mkdir dd/d16/d24/d72/d35/d81 0 2024-03-22T06:15:21.917 INFO:tasks.workunit.client.0.smithi006.stdout:0/927: dread db/f117 [4194304,4194304] 0 2024-03-22T06:15:21.918 INFO:tasks.workunit.client.0.smithi006.stdout:0/928: dread - db/d87/ddd/f10b zero size 2024-03-22T06:15:21.919 INFO:tasks.workunit.client.0.smithi006.stdout:0/929: link db/dc/f115 db/d10/d35/d7b/da1/da6/dc4/f141 0 2024-03-22T06:15:21.920 INFO:tasks.workunit.client.1.smithi117.stdout:5/378: dread dd/df/f2c [0,4194304] 0 2024-03-22T06:15:21.920 INFO:tasks.workunit.client.1.smithi117.stdout:5/379: dread - dd/d16/d24/d72/d60/f6b zero size 2024-03-22T06:15:21.921 INFO:tasks.workunit.client.1.smithi117.stdout:5/380: dread dd/df/f2c [0,4194304] 0 2024-03-22T06:15:21.922 INFO:tasks.workunit.client.1.smithi117.stdout:5/381: stat dd/df/c5c 0 2024-03-22T06:15:21.922 INFO:tasks.workunit.client.1.smithi117.stdout:5/382: dread - dd/d16/d24/f4f zero size 2024-03-22T06:15:21.922 INFO:tasks.workunit.client.1.smithi117.stdout:1/350: dwrite d1/d14/f19 [0,4194304] 0 2024-03-22T06:15:21.923 INFO:tasks.workunit.client.1.smithi117.stdout:1/351: chown d1/l7e 3993456 1 2024-03-22T06:15:21.923 INFO:tasks.workunit.client.1.smithi117.stdout:1/352: chown d1/d1f/d32/d4f 450101094 1 2024-03-22T06:15:21.927 INFO:tasks.workunit.client.1.smithi117.stdout:1/353: rename d1/d30/d35/d36 to d1/d84 0 2024-03-22T06:15:21.928 INFO:tasks.workunit.client.0.smithi006.stdout:0/930: write db/d10/d2a/d41/f7f [3928704,96298] 0 2024-03-22T06:15:21.928 INFO:tasks.workunit.client.0.smithi006.stdout:0/931: read - db/d10/d35/d7b/fe8 zero size 2024-03-22T06:15:21.931 INFO:tasks.workunit.client.1.smithi117.stdout:1/354: unlink d1/d30/d35/l82 0 2024-03-22T06:15:21.933 INFO:tasks.workunit.client.0.smithi006.stdout:0/932: getdents db/d10/d33/d9d/dcd/d106 0 2024-03-22T06:15:21.934 INFO:tasks.workunit.client.0.smithi006.stdout:0/933: creat db/d10/d35/d6c/dad/f142 x:0 0 0 2024-03-22T06:15:21.935 INFO:tasks.workunit.client.0.smithi006.stdout:0/934: creat db/d10/d35/d7b/da1/f143 x:0 0 0 2024-03-22T06:15:21.936 INFO:tasks.workunit.client.0.smithi006.stdout:0/935: creat db/d3d/d4e/d114/f144 x:0 0 0 2024-03-22T06:15:21.937 INFO:tasks.workunit.client.0.smithi006.stdout:0/936: mkdir db/d10/d2a/d127/d145 0 2024-03-22T06:15:21.948 INFO:tasks.workunit.client.0.smithi006.stdout:0/937: dread db/d87/ddd/f121 [0,4194304] 0 2024-03-22T06:15:21.948 INFO:tasks.workunit.client.0.smithi006.stdout:0/938: chown db/d10/d35/f69 5203 1 2024-03-22T06:15:21.949 INFO:tasks.workunit.client.1.smithi117.stdout:5/383: dwrite dd/df/f2a [0,4194304] 0 2024-03-22T06:15:21.949 INFO:tasks.workunit.client.1.smithi117.stdout:5/384: fsync dd/d16/d1a/f59 0 2024-03-22T06:15:21.949 INFO:tasks.workunit.client.1.smithi117.stdout:1/355: dwrite d1/d22/f4b [0,4194304] 0 2024-03-22T06:15:21.949 INFO:tasks.workunit.client.0.smithi006.stdout:0/939: mknod db/d3d/d4e/d114/c146 0 2024-03-22T06:15:21.950 INFO:tasks.workunit.client.1.smithi117.stdout:3/281: sync 2024-03-22T06:15:21.951 INFO:tasks.workunit.client.1.smithi117.stdout:2/302: sync 2024-03-22T06:15:21.951 INFO:tasks.workunit.client.1.smithi117.stdout:2/303: fsync d2/d5/f33 0 2024-03-22T06:15:21.951 INFO:tasks.workunit.client.0.smithi006.stdout:0/940: symlink db/d10/d2a/d41/d109/l147 0 2024-03-22T06:15:21.955 INFO:tasks.workunit.client.1.smithi117.stdout:1/356: truncate d1/d30/d35/f1a 2207750 0 2024-03-22T06:15:21.956 INFO:tasks.workunit.client.0.smithi006.stdout:0/941: rmdir db/d87/ddd 39 2024-03-22T06:15:21.957 INFO:tasks.workunit.client.1.smithi117.stdout:3/282: creat d9/d55/f67 x:0 0 0 2024-03-22T06:15:21.959 INFO:tasks.workunit.client.0.smithi006.stdout:0/942: mknod db/d3d/d4e/c148 0 2024-03-22T06:15:21.959 INFO:tasks.workunit.client.0.smithi006.stdout:0/943: write db/d10/d33/fb0 [355637,29051] 0 2024-03-22T06:15:21.960 INFO:tasks.workunit.client.1.smithi117.stdout:5/385: dread dd/df/f2b [4194304,4194304] 0 2024-03-22T06:15:21.960 INFO:tasks.workunit.client.1.smithi117.stdout:2/304: rename d2/d5/d13/d3f/d1d/d3a/c34 to d2/d5/d41/c59 0 2024-03-22T06:15:21.962 INFO:tasks.workunit.client.1.smithi117.stdout:1/357: dread d1/fd [0,4194304] 0 2024-03-22T06:15:21.963 INFO:tasks.workunit.client.0.smithi006.stdout:0/944: rename db/d10/d35/d7b/da1/fd5 to db/d10/d35/d7b/da1/da6/d122/d13f/f149 0 2024-03-22T06:15:21.964 INFO:tasks.workunit.client.1.smithi117.stdout:3/283: truncate d9/d1a/d39/d4c/d37/f61 2491525 0 2024-03-22T06:15:21.966 INFO:tasks.workunit.client.1.smithi117.stdout:2/305: symlink d2/d5/d13/d3f/d1d/d3a/l5a 0 2024-03-22T06:15:21.966 INFO:tasks.workunit.client.1.smithi117.stdout:2/306: read - d2/d5/d13/d3f/d1d/d3a/f3e zero size 2024-03-22T06:15:21.967 INFO:tasks.workunit.client.0.smithi006.stdout:0/945: mknod db/d10/d35/d7b/da1/da6/d122/d13f/c14a 0 2024-03-22T06:15:21.968 INFO:tasks.workunit.client.1.smithi117.stdout:1/358: link d1/d30/d50/c52 d1/d30/d44/c85 0 2024-03-22T06:15:21.968 INFO:tasks.workunit.client.1.smithi117.stdout:1/359: dread - d1/d30/d44/f7b zero size 2024-03-22T06:15:21.970 INFO:tasks.workunit.client.1.smithi117.stdout:3/284: symlink d9/d1a/d3a/l68 0 2024-03-22T06:15:21.970 INFO:tasks.workunit.client.1.smithi117.stdout:3/285: dread - d9/d1a/d39/d4c/d36/d41/f65 zero size 2024-03-22T06:15:21.971 INFO:tasks.workunit.client.0.smithi006.stdout:0/946: creat db/d3d/d4e/d76/f14b x:0 0 0 2024-03-22T06:15:21.973 INFO:tasks.workunit.client.1.smithi117.stdout:2/307: creat d2/d5/d41/f5b x:0 0 0 2024-03-22T06:15:21.974 INFO:tasks.workunit.client.1.smithi117.stdout:1/360: rename l0 to d1/d84/l86 0 2024-03-22T06:15:21.975 INFO:tasks.workunit.client.0.smithi006.stdout:0/947: mkdir db/d10/d35/d6c/d14c 0 2024-03-22T06:15:21.977 INFO:tasks.workunit.client.1.smithi117.stdout:3/286: mkdir d9/d1a/d39/d4c/d37/d69 0 2024-03-22T06:15:21.977 INFO:tasks.workunit.client.1.smithi117.stdout:3/287: write d9/d55/f67 [935447,24294] 0 2024-03-22T06:15:21.978 INFO:tasks.workunit.client.1.smithi117.stdout:2/308: truncate d2/d5/db/d2c/f42 3507237 0 2024-03-22T06:15:21.978 INFO:tasks.workunit.client.1.smithi117.stdout:2/309: chown d2/d35/d38/c40 3899 1 2024-03-22T06:15:21.981 INFO:tasks.workunit.client.1.smithi117.stdout:1/361: rmdir d1/d1f/d32/d4f 0 2024-03-22T06:15:21.981 INFO:tasks.workunit.client.1.smithi117.stdout:1/362: chown d1/d1f/f75 2039655130 1 2024-03-22T06:15:21.983 INFO:tasks.workunit.client.1.smithi117.stdout:3/288: symlink d9/d1a/d39/d4c/l6a 0 2024-03-22T06:15:21.983 INFO:tasks.workunit.client.1.smithi117.stdout:3/289: fsync d9/d1a/f1e 0 2024-03-22T06:15:21.983 INFO:tasks.workunit.client.1.smithi117.stdout:3/290: write d9/d1a/d39/d4c/f4e [1864976,104264] 0 2024-03-22T06:15:21.983 INFO:tasks.workunit.client.0.smithi006.stdout:0/948: rmdir db/dc/d38/d50 39 2024-03-22T06:15:21.986 INFO:tasks.workunit.client.1.smithi117.stdout:2/310: rename d2/d5/d13/d3f/c43 to d2/d5/d13/d3f/d1d/d3a/c5c 0 2024-03-22T06:15:21.986 INFO:tasks.workunit.client.1.smithi117.stdout:2/311: chown d2/f26 3809 1 2024-03-22T06:15:21.987 INFO:tasks.workunit.client.1.smithi117.stdout:2/312: creat d2/d5/d13/f5d x:0 0 0 2024-03-22T06:15:21.988 INFO:tasks.workunit.client.1.smithi117.stdout:1/363: symlink d1/d84/l87 0 2024-03-22T06:15:21.993 INFO:tasks.workunit.client.1.smithi117.stdout:3/291: rename d9/d1a/d3a/l68 to d9/d1a/d39/d4c/d36/l6b 0 2024-03-22T06:15:21.996 INFO:tasks.workunit.client.1.smithi117.stdout:1/364: truncate d1/f13 3311261 0 2024-03-22T06:15:21.998 INFO:tasks.workunit.client.1.smithi117.stdout:2/313: rename d2/d1b/f28 to d2/d35/d38/f5e 0 2024-03-22T06:15:22.002 INFO:tasks.workunit.client.0.smithi006.stdout:0/949: dwrite db/d10/d35/d7b/da1/da6/dc4/f128 [4194304,4194304] 0 2024-03-22T06:15:22.003 INFO:tasks.workunit.client.1.smithi117.stdout:3/292: symlink d9/d55/l6c 0 2024-03-22T06:15:22.003 INFO:tasks.workunit.client.1.smithi117.stdout:3/293: read - d9/d1a/d39/d4c/d36/f50 zero size 2024-03-22T06:15:22.003 INFO:tasks.workunit.client.1.smithi117.stdout:3/294: write d9/d55/f67 [1435247,69745] 0 2024-03-22T06:15:22.003 INFO:tasks.workunit.client.1.smithi117.stdout:3/295: write d9/d55/f59 [335178,80372] 0 2024-03-22T06:15:22.003 INFO:tasks.workunit.client.1.smithi117.stdout:3/296: write d9/f18 [1790257,95145] 0 2024-03-22T06:15:22.003 INFO:tasks.workunit.client.1.smithi117.stdout:1/365: creat d1/d3c/f88 x:0 0 0 2024-03-22T06:15:22.003 INFO:tasks.workunit.client.1.smithi117.stdout:7/311: sync 2024-03-22T06:15:22.003 INFO:tasks.workunit.client.1.smithi117.stdout:4/316: sync 2024-03-22T06:15:22.003 INFO:tasks.workunit.client.1.smithi117.stdout:4/317: fdatasync d0/f3 0 2024-03-22T06:15:22.003 INFO:tasks.workunit.client.1.smithi117.stdout:0/345: sync 2024-03-22T06:15:22.004 INFO:tasks.workunit.client.1.smithi117.stdout:8/333: sync 2024-03-22T06:15:22.004 INFO:tasks.workunit.client.1.smithi117.stdout:8/334: chown d1/d28/c33 111453929 1 2024-03-22T06:15:22.004 INFO:tasks.workunit.client.1.smithi117.stdout:2/314: symlink d2/d5/d13/d3f/d1d/d3a/d11/d4f/l5f 0 2024-03-22T06:15:22.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:21 smithi117 ceph-mon[36514]: pgmap v24: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 134 MiB/s rd, 163 MiB/s wr, 979 op/s 2024-03-22T06:15:22.006 INFO:tasks.workunit.client.0.smithi006.stdout:0/950: creat db/dc/f14d x:0 0 0 2024-03-22T06:15:22.006 INFO:tasks.workunit.client.0.smithi006.stdout:0/951: write db/d10/d33/d9d/dcd/d106/f110 [132598,69948] 0 2024-03-22T06:15:22.008 INFO:tasks.workunit.client.1.smithi117.stdout:3/297: creat d9/d66/f6d x:0 0 0 2024-03-22T06:15:22.011 INFO:tasks.workunit.client.1.smithi117.stdout:0/346: dread d1/d3/d2b/d3e/de/d1f/d3b/f42 [0,4194304] 0 2024-03-22T06:15:22.013 INFO:tasks.workunit.client.0.smithi006.stdout:0/952: link db/d10/f1b db/d10/d35/d7b/da1/da6/dc4/f14e 0 2024-03-22T06:15:22.014 INFO:tasks.workunit.client.1.smithi117.stdout:7/312: dwrite d8/d9/da/d10/d23/d27/d32/f54 [0,4194304] 0 2024-03-22T06:15:22.016 INFO:tasks.workunit.client.1.smithi117.stdout:4/318: truncate d0/d4/fe 125169 0 2024-03-22T06:15:22.016 INFO:tasks.workunit.client.1.smithi117.stdout:7/313: dread d8/f2f [0,4194304] 0 2024-03-22T06:15:22.018 INFO:tasks.workunit.client.1.smithi117.stdout:8/335: rmdir d1/d3a/d44/d11 39 2024-03-22T06:15:22.018 INFO:tasks.workunit.client.0.smithi006.stdout:0/953: creat db/d10/d35/f14f x:0 0 0 2024-03-22T06:15:22.021 INFO:tasks.workunit.client.1.smithi117.stdout:2/315: link d2/d1b/l2e d2/d1b/d39/d45/l60 0 2024-03-22T06:15:22.022 INFO:tasks.workunit.client.1.smithi117.stdout:1/366: rmdir d1/d30/d35 39 2024-03-22T06:15:22.023 INFO:tasks.workunit.client.1.smithi117.stdout:2/316: dread d2/d5/d13/d57/f22 [0,4194304] 0 2024-03-22T06:15:22.024 INFO:tasks.workunit.client.0.smithi006.stdout:0/954: rename db/d3d/d4e/d76/f14b to db/d10/d35/d7b/f150 0 2024-03-22T06:15:22.024 INFO:tasks.workunit.client.0.smithi006.stdout:0/955: chown db/d10/dc2/f13a 8818 1 2024-03-22T06:15:22.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/347: creat d1/d10/d18/d26/d49/f77 x:0 0 0 2024-03-22T06:15:22.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/348: creat d1/d3/d2b/d3e/d3f/f78 x:0 0 0 2024-03-22T06:15:22.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/349: stat d1/d3/d2b/d3e/de/d1f/d3b/f46 0 2024-03-22T06:15:22.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/350: truncate d1/f4f 713533 0 2024-03-22T06:15:22.029 INFO:tasks.workunit.client.1.smithi117.stdout:3/298: dwrite d9/f21 [0,4194304] 0 2024-03-22T06:15:22.029 INFO:tasks.workunit.client.1.smithi117.stdout:3/299: dread - d9/d1a/d39/d4c/f42 zero size 2024-03-22T06:15:22.030 INFO:tasks.workunit.client.0.smithi006.stdout:0/956: mkdir db/d10/d35/d6c/dad/d151 0 2024-03-22T06:15:22.035 INFO:tasks.workunit.client.1.smithi117.stdout:4/319: creat d0/f6c x:0 0 0 2024-03-22T06:15:22.039 INFO:tasks.workunit.client.0.smithi006.stdout:0/957: creat db/dc/d37/f152 x:0 0 0 2024-03-22T06:15:22.039 INFO:tasks.workunit.client.0.smithi006.stdout:0/958: truncate db/dc/d38/f10f 619140 0 2024-03-22T06:15:22.039 INFO:tasks.workunit.client.1.smithi117.stdout:0/351: write d1/d10/d18/f38 [3181087,93415] 0 2024-03-22T06:15:22.039 INFO:tasks.workunit.client.1.smithi117.stdout:0/352: creat d1/d10/d5c/d40/d4e/f79 x:0 0 0 2024-03-22T06:15:22.040 INFO:tasks.workunit.client.1.smithi117.stdout:7/314: truncate f5 2319347 0 2024-03-22T06:15:22.040 INFO:tasks.workunit.client.1.smithi117.stdout:7/315: stat d8/d9/d37 0 2024-03-22T06:15:22.044 INFO:tasks.workunit.client.1.smithi117.stdout:2/317: getdents d2/d35/d38 0 2024-03-22T06:15:22.050 INFO:tasks.workunit.client.0.smithi006.stdout:0/959: read db/d87/f93 [234944,82111] 0 2024-03-22T06:15:22.050 INFO:tasks.workunit.client.0.smithi006.stdout:0/960: write db/d10/d2a/d41/f7f [5239589,53651] 0 2024-03-22T06:15:22.050 INFO:tasks.workunit.client.0.smithi006.stdout:0/961: creat db/d10/d35/d7b/da1/da6/dc4/f153 x:0 0 0 2024-03-22T06:15:22.052 INFO:tasks.workunit.client.1.smithi117.stdout:8/336: dwrite d1/d3a/d44/d11/f5a [0,4194304] 0 2024-03-22T06:15:22.052 INFO:tasks.workunit.client.1.smithi117.stdout:8/337: write d1/d3a/d44/d9/f5e [839911,99664] 0 2024-03-22T06:15:22.052 INFO:tasks.workunit.client.1.smithi117.stdout:0/353: mknod d1/d3/d2b/d3e/de/d1f/d59/c7a 0 2024-03-22T06:15:22.052 INFO:tasks.workunit.client.1.smithi117.stdout:0/354: fsync d1/d3/d2b/d3e/de/d1f/f3c 0 2024-03-22T06:15:22.052 INFO:tasks.workunit.client.1.smithi117.stdout:0/355: creat d1/d10/d18/f7b x:0 0 0 2024-03-22T06:15:22.052 INFO:tasks.workunit.client.1.smithi117.stdout:2/318: truncate d2/d5/d13/d3f/d1d/d3a/d11/f15 486165 0 2024-03-22T06:15:22.053 INFO:tasks.workunit.client.1.smithi117.stdout:8/338: mkdir d1/d28/d6b 0 2024-03-22T06:15:22.055 INFO:tasks.workunit.client.1.smithi117.stdout:0/356: mknod d1/d10/d18/d26/d53/c7c 0 2024-03-22T06:15:22.055 INFO:tasks.workunit.client.1.smithi117.stdout:0/357: dread - d1/d3/d2b/d3e/d3f/f71 zero size 2024-03-22T06:15:22.055 INFO:tasks.workunit.client.1.smithi117.stdout:0/358: chown d1/d10/d18/d26/f58 7731 1 2024-03-22T06:15:22.056 INFO:tasks.workunit.client.1.smithi117.stdout:2/319: mknod d2/d35/c61 0 2024-03-22T06:15:22.057 INFO:tasks.workunit.client.1.smithi117.stdout:8/339: rename d1/d3a/d44/d9/f5e to d1/d3a/d44/d9/f6c 0 2024-03-22T06:15:22.058 INFO:tasks.workunit.client.1.smithi117.stdout:1/367: dwrite d1/d22/d27/f68 [0,4194304] 0 2024-03-22T06:15:22.060 INFO:tasks.workunit.client.1.smithi117.stdout:2/320: rename d2/d5/d13/d57/f22 to d2/d5/d13/d3f/d1d/d3a/d11/d4f/f62 0 2024-03-22T06:15:22.060 INFO:tasks.workunit.client.1.smithi117.stdout:2/321: fsync d2/d5/d13/d3f/d1d/d3a/f37 0 2024-03-22T06:15:22.060 INFO:tasks.workunit.client.1.smithi117.stdout:2/322: dread - d2/d5/d13/d3f/f51 zero size 2024-03-22T06:15:22.060 INFO:tasks.workunit.client.1.smithi117.stdout:2/323: fdatasync d2/f4a 0 2024-03-22T06:15:22.060 INFO:tasks.workunit.client.1.smithi117.stdout:2/324: write d2/d5/d13/f30 [1630800,22534] 0 2024-03-22T06:15:22.060 INFO:tasks.workunit.client.1.smithi117.stdout:3/300: dwrite d9/d1a/d39/d4c/d36/f43 [0,4194304] 0 2024-03-22T06:15:22.061 INFO:tasks.workunit.client.1.smithi117.stdout:3/301: read d9/f18 [4370556,127122] 0 2024-03-22T06:15:22.061 INFO:tasks.workunit.client.1.smithi117.stdout:4/320: dwrite d0/d19/d28/f45 [0,4194304] 0 2024-03-22T06:15:22.061 INFO:tasks.workunit.client.1.smithi117.stdout:4/321: creat d0/f6d x:0 0 0 2024-03-22T06:15:22.064 INFO:tasks.workunit.client.1.smithi117.stdout:0/359: write d1/d10/f32 [1458273,34036] 0 2024-03-22T06:15:22.067 INFO:tasks.workunit.client.1.smithi117.stdout:1/368: dread d1/d3c/d4d/d5b/f60 [0,4194304] 0 2024-03-22T06:15:22.068 INFO:tasks.workunit.client.0.smithi006.stdout:0/962: dwrite db/d10/d2a/f5e [0,4194304] 0 2024-03-22T06:15:22.068 INFO:tasks.workunit.client.0.smithi006.stdout:0/963: truncate db/dc/d103/f113 1078437 0 2024-03-22T06:15:22.068 INFO:tasks.workunit.client.0.smithi006.stdout:0/964: truncate db/d10/d33/d9d/dcd/fd4 971773 0 2024-03-22T06:15:22.069 INFO:tasks.workunit.client.1.smithi117.stdout:2/325: creat d2/d5/d13/d57/f63 x:0 0 0 2024-03-22T06:15:22.075 INFO:tasks.workunit.client.1.smithi117.stdout:3/302: rename d9/d1a/d3a/l60 to d9/d1a/d39/d4c/d36/d41/l6e 0 2024-03-22T06:15:22.080 INFO:tasks.workunit.client.1.smithi117.stdout:1/369: write d1/d62/f67 [938509,106300] 0 2024-03-22T06:15:22.080 INFO:tasks.workunit.client.1.smithi117.stdout:1/370: write d1/d22/d27/d4e/f7d [284752,95755] 0 2024-03-22T06:15:22.081 INFO:tasks.workunit.client.1.smithi117.stdout:1/371: fdatasync d1/f13 0 2024-03-22T06:15:22.081 INFO:tasks.workunit.client.1.smithi117.stdout:1/372: chown d1/d30/d44/f7b 58 1 2024-03-22T06:15:22.081 INFO:tasks.workunit.client.1.smithi117.stdout:4/322: link d0/d4/d2a/f5d d0/d4/d2a/d5b/f6e 0 2024-03-22T06:15:22.081 INFO:tasks.workunit.client.1.smithi117.stdout:4/323: chown d0/d19/d28/d47/f67 134002945 1 2024-03-22T06:15:22.081 INFO:tasks.workunit.client.1.smithi117.stdout:4/324: chown d0/c69 387 1 2024-03-22T06:15:22.081 INFO:tasks.workunit.client.1.smithi117.stdout:4/325: chown d0/d48/c50 33304844 1 2024-03-22T06:15:22.085 INFO:tasks.workunit.client.1.smithi117.stdout:8/340: rmdir d1/d19/d38 39 2024-03-22T06:15:22.085 INFO:tasks.workunit.client.1.smithi117.stdout:8/341: stat d1/d3a/d44/d9 0 2024-03-22T06:15:22.088 INFO:tasks.workunit.client.1.smithi117.stdout:2/326: dwrite d2/d35/d38/f5e [0,4194304] 0 2024-03-22T06:15:22.089 INFO:tasks.workunit.client.1.smithi117.stdout:3/303: symlink d9/d1a/l6f 0 2024-03-22T06:15:22.091 INFO:tasks.workunit.client.0.smithi006.stdout:0/965: dread db/d10/f23 [8388608,4194304] 0 2024-03-22T06:15:22.091 INFO:tasks.workunit.client.0.smithi006.stdout:0/966: truncate db/d10/d35/f69 2383065 0 2024-03-22T06:15:22.091 INFO:tasks.workunit.client.0.smithi006.stdout:0/967: chown db/dc/faa 1233 1 2024-03-22T06:15:22.094 INFO:tasks.workunit.client.1.smithi117.stdout:8/342: symlink d1/d28/d66/l6d 0 2024-03-22T06:15:22.094 INFO:tasks.workunit.client.1.smithi117.stdout:9/381: sync 2024-03-22T06:15:22.094 INFO:tasks.workunit.client.1.smithi117.stdout:6/391: sync 2024-03-22T06:15:22.094 INFO:tasks.workunit.client.1.smithi117.stdout:6/392: fdatasync d3/d40/d24/d2d/d53/d59/d74/da/de/f19 0 2024-03-22T06:15:22.095 INFO:tasks.workunit.client.0.smithi006.stdout:0/968: truncate db/dc/d38/d50/f79 3403137 0 2024-03-22T06:15:22.095 INFO:tasks.workunit.client.0.smithi006.stdout:0/969: fsync db/d3d/fdb 0 2024-03-22T06:15:22.096 INFO:tasks.workunit.client.1.smithi117.stdout:0/360: getdents d1/d3/d2b/d3e/d3f 0 2024-03-22T06:15:22.099 INFO:tasks.workunit.client.0.smithi006.stdout:0/970: link db/dc/d38/f3a db/d10/d2a/f154 0 2024-03-22T06:15:22.101 INFO:tasks.workunit.client.1.smithi117.stdout:2/327: creat d2/d5/d13/d3f/d1d/d3a/f64 x:0 0 0 2024-03-22T06:15:22.101 INFO:tasks.workunit.client.1.smithi117.stdout:2/328: fsync d2/d5/d13/d3f/d1d/d3a/d11/d4f/f62 0 2024-03-22T06:15:22.101 INFO:tasks.workunit.client.1.smithi117.stdout:2/329: write d2/d5/db/f24 [1007938,74436] 0 2024-03-22T06:15:22.101 INFO:tasks.workunit.client.1.smithi117.stdout:6/393: dread d3/d3d/f43 [0,4194304] 0 2024-03-22T06:15:22.102 INFO:tasks.workunit.client.1.smithi117.stdout:1/373: link d1/f43 d1/d22/d27/d4e/f89 0 2024-03-22T06:15:22.103 INFO:tasks.workunit.client.1.smithi117.stdout:1/374: creat d1/d62/f8a x:0 0 0 2024-03-22T06:15:22.104 INFO:tasks.workunit.client.1.smithi117.stdout:4/326: dwrite d0/f6c [0,4194304] 0 2024-03-22T06:15:22.107 INFO:tasks.workunit.client.1.smithi117.stdout:9/382: rename d4/d7/d10/d2f/f4e to d4/d7/d10/f70 0 2024-03-22T06:15:22.110 INFO:tasks.workunit.client.1.smithi117.stdout:0/361: truncate d1/d3/d2b/d3e/de/f51 3653317 0 2024-03-22T06:15:22.110 INFO:tasks.workunit.client.1.smithi117.stdout:0/362: write d1/d3/d2b/d3e/de/d1f/f3c [4283776,17590] 0 2024-03-22T06:15:22.113 INFO:tasks.workunit.client.1.smithi117.stdout:0/363: write d1/f19 [215996,74400] 0 2024-03-22T06:15:22.114 INFO:tasks.workunit.client.1.smithi117.stdout:0/364: write d1/f4f [1264618,21507] 0 2024-03-22T06:15:22.114 INFO:tasks.workunit.client.1.smithi117.stdout:0/365: fsync d1/d3/d2b/d3e/de/d1f/d3b/f42 0 2024-03-22T06:15:22.114 INFO:tasks.workunit.client.1.smithi117.stdout:0/366: chown d1/d10/d18/d26/d49/l5d 0 1 2024-03-22T06:15:22.114 INFO:tasks.workunit.client.1.smithi117.stdout:0/367: write d1/d3/d2b/d3e/d3f/f78 [723405,47690] 0 2024-03-22T06:15:22.115 INFO:tasks.workunit.client.1.smithi117.stdout:1/375: mknod d1/d62/c8b 0 2024-03-22T06:15:22.115 INFO:tasks.workunit.client.1.smithi117.stdout:1/376: write d1/d62/f67 [2240256,40446] 0 2024-03-22T06:15:22.116 INFO:tasks.workunit.client.1.smithi117.stdout:5/386: sync 2024-03-22T06:15:22.123 INFO:tasks.workunit.client.1.smithi117.stdout:2/330: dwrite d2/d5/d13/d3f/d1d/d3a/f64 [0,4194304] 0 2024-03-22T06:15:22.123 INFO:tasks.workunit.client.1.smithi117.stdout:2/331: write d2/fc [9392643,98044] 0 2024-03-22T06:15:22.123 INFO:tasks.workunit.client.1.smithi117.stdout:2/332: chown d2/d35/l50 1941 1 2024-03-22T06:15:22.123 INFO:tasks.workunit.client.1.smithi117.stdout:2/333: chown d2 5792 1 2024-03-22T06:15:22.128 INFO:tasks.workunit.client.1.smithi117.stdout:1/377: dread d1/d1f/f3e [0,4194304] 0 2024-03-22T06:15:22.129 INFO:tasks.workunit.client.1.smithi117.stdout:4/327: rename d0/d5c to d0/d19/d34/d43/d63/d6f 0 2024-03-22T06:15:22.134 INFO:tasks.workunit.client.1.smithi117.stdout:0/368: creat d1/d10/d5c/d40/d69/f7d x:0 0 0 2024-03-22T06:15:22.134 INFO:tasks.workunit.client.1.smithi117.stdout:0/369: dread - d1/d10/d5c/d40/d4e/f79 zero size 2024-03-22T06:15:22.137 INFO:tasks.workunit.client.1.smithi117.stdout:5/387: dwrite dd/d16/d24/d72/d33/f7c [0,4194304] 0 2024-03-22T06:15:22.137 INFO:tasks.workunit.client.1.smithi117.stdout:2/334: dread d2/d5/d13/d3f/d1d/d3a/f37 [0,4194304] 0 2024-03-22T06:15:22.137 INFO:tasks.workunit.client.1.smithi117.stdout:6/394: dwrite d3/d40/d24/d2d/d53/d59/d74/da/d56/f7a [0,4194304] 0 2024-03-22T06:15:22.137 INFO:tasks.workunit.client.1.smithi117.stdout:6/395: fdatasync d3/d40/d46/d84/f7b 0 2024-03-22T06:15:22.137 INFO:tasks.workunit.client.1.smithi117.stdout:2/335: creat d2/d1b/d39/f65 x:0 0 0 2024-03-22T06:15:22.137 INFO:tasks.workunit.client.1.smithi117.stdout:2/336: chown d2/d5/d13/d3f/d1d/d3a/d11/f15 22 1 2024-03-22T06:15:22.139 INFO:tasks.workunit.client.1.smithi117.stdout:5/388: dread dd/d16/d1a/f62 [0,4194304] 0 2024-03-22T06:15:22.143 INFO:tasks.workunit.client.1.smithi117.stdout:1/378: mknod d1/d84/d77/c8c 0 2024-03-22T06:15:22.143 INFO:tasks.workunit.client.1.smithi117.stdout:1/379: dread - d1/d1f/f75 zero size 2024-03-22T06:15:22.146 INFO:tasks.workunit.client.1.smithi117.stdout:9/383: rename d4/d7/d10/d2f/c5d to d4/d7/d27/c71 0 2024-03-22T06:15:22.149 INFO:tasks.workunit.client.1.smithi117.stdout:4/328: unlink d0/c69 0 2024-03-22T06:15:22.158 INFO:tasks.workunit.client.0.smithi006.stdout:0/971: dwrite db/f45 [0,4194304] 0 2024-03-22T06:15:22.158 INFO:tasks.workunit.client.0.smithi006.stdout:0/972: chown db/d3d/d4e/d114/d124/l104 537361 1 2024-03-22T06:15:22.159 INFO:tasks.workunit.client.1.smithi117.stdout:2/337: rmdir d2/d5/d13/d3f/d1d 39 2024-03-22T06:15:22.160 INFO:tasks.workunit.client.1.smithi117.stdout:1/380: getdents d1/d30/d35 0 2024-03-22T06:15:22.160 INFO:tasks.workunit.client.1.smithi117.stdout:1/381: getdents d1/d30/d5d/d6f 0 2024-03-22T06:15:22.160 INFO:tasks.workunit.client.1.smithi117.stdout:5/389: dwrite dd/d16/d24/d72/d60/f6b [0,4194304] 0 2024-03-22T06:15:22.164 INFO:tasks.workunit.client.1.smithi117.stdout:9/384: mkdir d4/d47/d64/d72 0 2024-03-22T06:15:22.164 INFO:tasks.workunit.client.1.smithi117.stdout:9/385: write d4/d7/d10/f28 [4737353,15727] 0 2024-03-22T06:15:22.166 INFO:tasks.workunit.client.1.smithi117.stdout:4/329: mkdir d0/d19/d34/d43/d63/d6f/d70 0 2024-03-22T06:15:22.174 INFO:tasks.workunit.client.1.smithi117.stdout:5/390: mknod dd/d16/d1a/d32/d73/c82 0 2024-03-22T06:15:22.181 INFO:tasks.workunit.client.1.smithi117.stdout:4/330: mkdir d0/d19/d34/d71 0 2024-03-22T06:15:22.182 INFO:tasks.workunit.client.1.smithi117.stdout:5/391: creat dd/d16/d5a/f83 x:0 0 0 2024-03-22T06:15:22.184 INFO:tasks.workunit.client.1.smithi117.stdout:0/370: rename d1/d10/d18/d26/dc to d1/d10/d18/d26/d49/d7e 0 2024-03-22T06:15:22.184 INFO:tasks.workunit.client.1.smithi117.stdout:0/371: creat d1/d10/d5c/d40/f7f x:0 0 0 2024-03-22T06:15:22.185 INFO:tasks.workunit.client.1.smithi117.stdout:5/392: truncate dd/d16/d1a/f3e 735196 0 2024-03-22T06:15:22.185 INFO:tasks.workunit.client.1.smithi117.stdout:7/316: sync 2024-03-22T06:15:22.186 INFO:tasks.workunit.client.1.smithi117.stdout:7/317: fsync d8/d9/da/d10/f13 0 2024-03-22T06:15:22.188 INFO:tasks.workunit.client.1.smithi117.stdout:6/396: rename d3/d40/d24/d2d/d53/d59/d74/da/d56/d3a to d3/d40/d24/d2d/d53/d59/d74/d28/d85 0 2024-03-22T06:15:22.193 INFO:tasks.workunit.client.1.smithi117.stdout:5/393: unlink dd/f4b 0 2024-03-22T06:15:22.195 INFO:tasks.workunit.client.1.smithi117.stdout:7/318: mkdir d8/d9/d37/d45/d61 0 2024-03-22T06:15:22.195 INFO:tasks.workunit.client.1.smithi117.stdout:4/331: dread d0/f2d [0,4194304] 0 2024-03-22T06:15:22.195 INFO:tasks.workunit.client.1.smithi117.stdout:4/332: stat d0/d3f 0 2024-03-22T06:15:22.195 INFO:tasks.workunit.client.1.smithi117.stdout:1/382: dwrite d1/d22/d27/d4e/f89 [0,4194304] 0 2024-03-22T06:15:22.195 INFO:tasks.workunit.client.1.smithi117.stdout:4/333: write d0/d19/d28/f2e [2222777,106930] 0 2024-03-22T06:15:22.196 INFO:tasks.workunit.client.1.smithi117.stdout:6/397: creat d3/d40/d46/d84/d49/f86 x:0 0 0 2024-03-22T06:15:22.198 INFO:tasks.workunit.client.1.smithi117.stdout:7/319: rename d8/d9/da/c5b to d8/d9/d37/d45/d61/c62 0 2024-03-22T06:15:22.200 INFO:tasks.workunit.client.1.smithi117.stdout:6/398: link d3/f3f d3/d3d/d17/d31/f87 0 2024-03-22T06:15:22.201 INFO:tasks.workunit.client.1.smithi117.stdout:5/394: symlink dd/df/l84 0 2024-03-22T06:15:22.201 INFO:tasks.workunit.client.1.smithi117.stdout:5/395: truncate dd/f7d 825647 0 2024-03-22T06:15:22.201 INFO:tasks.workunit.client.1.smithi117.stdout:5/396: write dd/d16/d1a/f62 [1724397,21968] 0 2024-03-22T06:15:22.201 INFO:tasks.workunit.client.1.smithi117.stdout:5/397: creat dd/d16/d24/d72/d60/d6e/f85 x:0 0 0 2024-03-22T06:15:22.201 INFO:tasks.workunit.client.1.smithi117.stdout:4/334: dread d0/d4/f17 [0,4194304] 0 2024-03-22T06:15:22.202 INFO:tasks.workunit.client.1.smithi117.stdout:7/320: unlink d8/d9/le 0 2024-03-22T06:15:22.203 INFO:tasks.workunit.client.1.smithi117.stdout:6/399: creat d3/d40/d24/d2d/d53/d59/f88 x:0 0 0 2024-03-22T06:15:22.204 INFO:tasks.workunit.client.1.smithi117.stdout:9/386: dwrite d4/d7/d27/f56 [0,4194304] 0 2024-03-22T06:15:22.205 INFO:tasks.workunit.client.1.smithi117.stdout:1/383: read d1/d3c/d4d/d5b/f60 [3804505,97483] 0 2024-03-22T06:15:22.205 INFO:tasks.workunit.client.1.smithi117.stdout:2/338: dwrite d2/d5/d13/d57/f47 [0,4194304] 0 2024-03-22T06:15:22.205 INFO:tasks.workunit.client.1.smithi117.stdout:2/339: write d2/d5/db/f24 [1553604,13477] 0 2024-03-22T06:15:22.205 INFO:tasks.workunit.client.1.smithi117.stdout:0/372: dwrite d1/d10/f6c [0,4194304] 0 2024-03-22T06:15:22.206 INFO:tasks.workunit.client.1.smithi117.stdout:4/335: rename d0/d19/d34/l40 to d0/d51/l72 0 2024-03-22T06:15:22.207 INFO:tasks.workunit.client.1.smithi117.stdout:6/400: mknod d3/d40/d24/d2d/c89 0 2024-03-22T06:15:22.207 INFO:tasks.workunit.client.0.smithi006.stdout:0/973: dwrite db/d87/faf [0,4194304] 0 2024-03-22T06:15:22.208 INFO:tasks.workunit.client.1.smithi117.stdout:5/398: fsync dd/d16/d1a/f62 0 2024-03-22T06:15:22.213 INFO:tasks.workunit.client.1.smithi117.stdout:2/340: write d2/d5/d13/d57/f36 [506337,115911] 0 2024-03-22T06:15:22.213 INFO:tasks.workunit.client.1.smithi117.stdout:2/341: write d2/d5/d13/d3f/f51 [971632,32890] 0 2024-03-22T06:15:22.215 INFO:tasks.workunit.client.1.smithi117.stdout:9/387: unlink d4/d7/f1c 0 2024-03-22T06:15:22.216 INFO:tasks.workunit.client.1.smithi117.stdout:6/401: dread d3/d40/d24/d2d/d53/d59/d74/f11 [0,4194304] 0 2024-03-22T06:15:22.220 INFO:tasks.workunit.client.1.smithi117.stdout:4/336: dread d0/d19/f22 [0,4194304] 0 2024-03-22T06:15:22.221 INFO:tasks.workunit.client.0.smithi006.stdout:0/974: creat db/d10/d33/db4/f155 x:0 0 0 2024-03-22T06:15:22.221 INFO:tasks.workunit.client.0.smithi006.stdout:0/975: stat db/d10/d35/d7b/da1/da6/fd2 0 2024-03-22T06:15:22.221 INFO:tasks.workunit.client.0.smithi006.stdout:0/976: fsync db/d3d/fdb 0 2024-03-22T06:15:22.224 INFO:tasks.workunit.client.0.smithi006.stdout:0/977: mknod db/d10/d35/d6c/c156 0 2024-03-22T06:15:22.227 INFO:tasks.workunit.client.1.smithi117.stdout:6/402: dread d3/d40/d24/d2d/d53/d59/d74/da/de/f19 [0,4194304] 0 2024-03-22T06:15:22.227 INFO:tasks.workunit.client.1.smithi117.stdout:6/403: fsync d3/d3d/f43 0 2024-03-22T06:15:22.229 INFO:tasks.workunit.client.1.smithi117.stdout:7/321: dwrite d8/d9/f1e [0,4194304] 0 2024-03-22T06:15:22.229 INFO:tasks.workunit.client.1.smithi117.stdout:7/322: fsync d8/d9/da/d10/f19 0 2024-03-22T06:15:22.239 INFO:tasks.workunit.client.1.smithi117.stdout:0/373: link d1/d10/d18/d26/l44 d1/d10/d18/d26/d49/d7e/d39/d6a/l80 0 2024-03-22T06:15:22.239 INFO:tasks.workunit.client.1.smithi117.stdout:0/374: fsync d1/f19 0 2024-03-22T06:15:22.241 INFO:tasks.workunit.client.1.smithi117.stdout:5/399: mkdir dd/d16/d86 0 2024-03-22T06:15:22.242 INFO:tasks.workunit.client.0.smithi006.stdout:0/978: dwrite db/f96 [0,4194304] 0 2024-03-22T06:15:22.242 INFO:tasks.workunit.client.0.smithi006.stdout:0/979: chown db/d10/d35/d7b/da1/lca 137129991 1 2024-03-22T06:15:22.244 INFO:tasks.workunit.client.1.smithi117.stdout:2/342: creat d2/d5/db/f66 x:0 0 0 2024-03-22T06:15:22.245 INFO:tasks.workunit.client.1.smithi117.stdout:6/404: dread d3/f3c [0,4194304] 0 2024-03-22T06:15:22.245 INFO:tasks.workunit.client.1.smithi117.stdout:1/384: dwrite d1/d3c/d4d/d5b/f61 [0,4194304] 0 2024-03-22T06:15:22.246 INFO:tasks.workunit.client.1.smithi117.stdout:9/388: symlink d4/d8/d3f/l73 0 2024-03-22T06:15:22.246 INFO:tasks.workunit.client.1.smithi117.stdout:9/389: chown d4/d8/f1a 484 1 2024-03-22T06:15:22.248 INFO:tasks.workunit.client.1.smithi117.stdout:4/337: truncate d0/f2d 226153 0 2024-03-22T06:15:22.252 INFO:tasks.workunit.client.0.smithi006.stdout:0/980: mkdir db/d10/d35/d7b/da1/d138/d157 0 2024-03-22T06:15:22.255 INFO:tasks.workunit.client.1.smithi117.stdout:3/304: sync 2024-03-22T06:15:22.255 INFO:tasks.workunit.client.1.smithi117.stdout:8/343: sync 2024-03-22T06:15:22.255 INFO:tasks.workunit.client.1.smithi117.stdout:8/344: chown d1/l61 14964 1 2024-03-22T06:15:22.255 INFO:tasks.workunit.client.1.smithi117.stdout:7/323: symlink d8/d9/d37/d5f/l63 0 2024-03-22T06:15:22.255 INFO:tasks.workunit.client.1.smithi117.stdout:3/305: creat d9/f70 x:0 0 0 2024-03-22T06:15:22.255 INFO:tasks.workunit.client.1.smithi117.stdout:8/345: write d1/d19/f2d [4225574,70681] 0 2024-03-22T06:15:22.255 INFO:tasks.workunit.client.1.smithi117.stdout:8/346: dread d1/f1e [4194304,4194304] 0 2024-03-22T06:15:22.255 INFO:tasks.workunit.client.0.smithi006.stdout:0/981: symlink db/d10/d35/l158 0 2024-03-22T06:15:22.259 INFO:tasks.workunit.client.0.smithi006.stdout:0/982: mkdir db/d10/d2a/d41/dff/d118/d159 0 2024-03-22T06:15:22.261 INFO:tasks.workunit.client.1.smithi117.stdout:2/343: rmdir d2/d35/d38 39 2024-03-22T06:15:22.263 INFO:tasks.workunit.client.1.smithi117.stdout:1/385: unlink d1/d14/l26 0 2024-03-22T06:15:22.263 INFO:tasks.workunit.client.1.smithi117.stdout:1/386: fdatasync d1/f43 0 2024-03-22T06:15:22.264 INFO:tasks.workunit.client.1.smithi117.stdout:0/375: dwrite d1/f3d [0,4194304] 0 2024-03-22T06:15:22.265 INFO:tasks.workunit.client.1.smithi117.stdout:6/405: creat d3/d40/d24/d2d/d53/d59/d74/d28/d85/f8a x:0 0 0 2024-03-22T06:15:22.270 INFO:tasks.workunit.client.0.smithi006.stdout:0/983: read db/dc/d38/d50/f100 [2745417,11082] 0 2024-03-22T06:15:22.272 INFO:tasks.workunit.client.1.smithi117.stdout:4/338: dwrite d0/d19/d28/f55 [0,4194304] 0 2024-03-22T06:15:22.272 INFO:tasks.workunit.client.1.smithi117.stdout:4/339: readlink d0/d51/l72 0 2024-03-22T06:15:22.272 INFO:tasks.workunit.client.1.smithi117.stdout:8/347: dwrite d1/d3a/d44/f1d [0,4194304] 0 2024-03-22T06:15:22.273 INFO:tasks.workunit.client.0.smithi006.stdout:0/984: symlink db/d3d/d4e/d114/d124/l15a 0 2024-03-22T06:15:22.273 INFO:tasks.workunit.client.0.smithi006.stdout:0/985: creat db/d3d/d4e/d76/f15b x:0 0 0 2024-03-22T06:15:22.274 INFO:tasks.workunit.client.1.smithi117.stdout:3/306: creat d9/d1a/d39/f71 x:0 0 0 2024-03-22T06:15:22.277 INFO:tasks.workunit.client.1.smithi117.stdout:2/344: mknod d2/d5/db/d2c/c67 0 2024-03-22T06:15:22.278 INFO:tasks.workunit.client.0.smithi006.stdout:0/986: mkdir db/d10/d35/d6c/d15c 0 2024-03-22T06:15:22.278 INFO:tasks.workunit.client.0.smithi006.stdout:0/987: dread - db/d10/d35/f14f zero size 2024-03-22T06:15:22.282 INFO:tasks.workunit.client.1.smithi117.stdout:1/387: rmdir d1/d30/d35 39 2024-03-22T06:15:22.282 INFO:tasks.workunit.client.1.smithi117.stdout:1/388: write d1/d22/d27/d4e/f7a [1015887,6481] 0 2024-03-22T06:15:22.282 INFO:tasks.workunit.client.1.smithi117.stdout:1/389: chown d1/d84/l87 15 1 2024-03-22T06:15:22.282 INFO:tasks.workunit.client.1.smithi117.stdout:1/390: fsync d1/d30/d50/f63 0 2024-03-22T06:15:22.282 INFO:tasks.workunit.client.1.smithi117.stdout:8/348: write d1/d19/f51 [5604677,130112] 0 2024-03-22T06:15:22.285 INFO:tasks.workunit.client.1.smithi117.stdout:8/349: fdatasync d1/d19/f51 0 2024-03-22T06:15:22.286 INFO:tasks.workunit.client.1.smithi117.stdout:0/376: symlink d1/d10/d18/l81 0 2024-03-22T06:15:22.286 INFO:tasks.workunit.client.1.smithi117.stdout:0/377: dread - d1/d3/d2b/d3e/d3f/f71 zero size 2024-03-22T06:15:22.295 INFO:tasks.workunit.client.0.smithi006.stdout:0/988: dwrite db/dc/f44 [0,4194304] 0 2024-03-22T06:15:22.295 INFO:tasks.workunit.client.0.smithi006.stdout:0/989: chown db/d10/d33/d9d 1641 1 2024-03-22T06:15:22.297 INFO:tasks.workunit.client.0.smithi006.stdout:0/990: mknod db/d3d/d4e/d114/d124/c15d 0 2024-03-22T06:15:22.297 INFO:tasks.workunit.client.0.smithi006.stdout:0/991: readlink db/dc/d38/l56 0 2024-03-22T06:15:22.297 INFO:tasks.workunit.client.1.smithi117.stdout:8/350: dread d1/d3a/d44/fa [0,4194304] 0 2024-03-22T06:15:22.297 INFO:tasks.workunit.client.1.smithi117.stdout:9/390: rmdir d4 39 2024-03-22T06:15:22.298 INFO:tasks.workunit.client.0.smithi006.stdout:0/992: creat db/d10/d33/dce/f15e x:0 0 0 2024-03-22T06:15:22.298 INFO:tasks.workunit.client.0.smithi006.stdout:0/993: dread - db/d10/d33/dce/f15e zero size 2024-03-22T06:15:22.300 INFO:tasks.workunit.client.1.smithi117.stdout:4/340: symlink d0/d4/d2a/d5b/l73 0 2024-03-22T06:15:22.302 INFO:tasks.workunit.client.1.smithi117.stdout:3/307: symlink d9/d55/l72 0 2024-03-22T06:15:22.304 INFO:tasks.workunit.client.1.smithi117.stdout:2/345: symlink d2/d5/d41/l68 0 2024-03-22T06:15:22.304 INFO:tasks.workunit.client.1.smithi117.stdout:2/346: dread - d2/d5/d13/f5d zero size 2024-03-22T06:15:22.304 INFO:tasks.workunit.client.1.smithi117.stdout:2/347: fdatasync d2/d5/d13/d3f/d1d/d3a/d11/f15 0 2024-03-22T06:15:22.304 INFO:tasks.workunit.client.1.smithi117.stdout:1/391: mknod d1/d62/c8d 0 2024-03-22T06:15:22.304 INFO:tasks.workunit.client.1.smithi117.stdout:1/392: creat d1/d22/d27/f8e x:0 0 0 2024-03-22T06:15:22.306 INFO:tasks.workunit.client.1.smithi117.stdout:0/378: dread d1/d10/f6c [0,4194304] 0 2024-03-22T06:15:22.306 INFO:tasks.workunit.client.1.smithi117.stdout:0/379: write d1/d10/f27 [434578,60952] 0 2024-03-22T06:15:22.309 INFO:tasks.workunit.client.1.smithi117.stdout:8/351: link d1/d19/l62 d1/d3a/l6e 0 2024-03-22T06:15:22.312 INFO:tasks.workunit.client.1.smithi117.stdout:7/324: unlink d8/d9/da/d10/d23/d27/d39/d26/f2c 0 2024-03-22T06:15:22.312 INFO:tasks.workunit.client.1.smithi117.stdout:7/325: chown d8/d9/d37 0 1 2024-03-22T06:15:22.317 INFO:tasks.workunit.client.1.smithi117.stdout:3/308: mknod d9/d1a/d3a/c73 0 2024-03-22T06:15:22.317 INFO:tasks.workunit.client.1.smithi117.stdout:3/309: chown d9/d1a/d39/f45 186554 1 2024-03-22T06:15:22.317 INFO:tasks.workunit.client.1.smithi117.stdout:3/310: creat d9/d66/f74 x:0 0 0 2024-03-22T06:15:22.317 INFO:tasks.workunit.client.1.smithi117.stdout:1/393: dread d1/d22/d27/f68 [0,4194304] 0 2024-03-22T06:15:22.318 INFO:tasks.workunit.client.1.smithi117.stdout:6/406: rename d3/d40/d24/d2d/d53/d59/d74/da/d56 to d3/d40/d24/d2d/d53/d59/d74/d8b 0 2024-03-22T06:15:22.318 INFO:tasks.workunit.client.1.smithi117.stdout:0/380: creat d1/d10/d18/d26/d53/f82 x:0 0 0 2024-03-22T06:15:22.319 INFO:tasks.workunit.client.1.smithi117.stdout:8/352: symlink d1/d3a/d44/d9/l6f 0 2024-03-22T06:15:22.321 INFO:tasks.workunit.client.1.smithi117.stdout:7/326: link d8/d9/da/d10/c55 d8/d9/da/d10/d23/c64 0 2024-03-22T06:15:22.325 INFO:tasks.workunit.client.1.smithi117.stdout:1/394: rename d1/d84/f65 to d1/d1f/f8f 0 2024-03-22T06:15:22.325 INFO:tasks.workunit.client.1.smithi117.stdout:1/395: read d1/d22/d27/d4e/f7a [887767,96251] 0 2024-03-22T06:15:22.325 INFO:tasks.workunit.client.1.smithi117.stdout:1/396: creat d1/d22/d27/f90 x:0 0 0 2024-03-22T06:15:22.326 INFO:tasks.workunit.client.1.smithi117.stdout:1/397: stat d1/d22/f4b 0 2024-03-22T06:15:22.326 INFO:tasks.workunit.client.1.smithi117.stdout:1/398: creat d1/d3c/d4d/d5b/f91 x:0 0 0 2024-03-22T06:15:22.326 INFO:tasks.workunit.client.1.smithi117.stdout:6/407: symlink d3/d40/d24/d2d/d53/d75/l8c 0 2024-03-22T06:15:22.326 INFO:tasks.workunit.client.1.smithi117.stdout:6/408: write d3/d40/d24/d2d/d53/d59/d74/da/f29 [218024,99995] 0 2024-03-22T06:15:22.326 INFO:tasks.workunit.client.1.smithi117.stdout:6/409: read - d3/d40/d24/d2d/d42/f76 zero size 2024-03-22T06:15:22.327 INFO:tasks.workunit.client.1.smithi117.stdout:6/410: creat d3/d3d/f8d x:0 0 0 2024-03-22T06:15:22.327 INFO:tasks.workunit.client.1.smithi117.stdout:0/381: creat d1/d3/d2b/f83 x:0 0 0 2024-03-22T06:15:22.328 INFO:tasks.workunit.client.1.smithi117.stdout:9/391: dwrite d4/d7/d10/f6c [0,4194304] 0 2024-03-22T06:15:22.328 INFO:tasks.workunit.client.1.smithi117.stdout:8/353: creat d1/d19/d38/f70 x:0 0 0 2024-03-22T06:15:22.329 INFO:tasks.workunit.client.1.smithi117.stdout:7/327: mknod d8/d9/da/d2b/c65 0 2024-03-22T06:15:22.329 INFO:tasks.workunit.client.1.smithi117.stdout:7/328: fdatasync d8/d9/da/d10/d23/d27/d32/f4d 0 2024-03-22T06:15:22.332 INFO:tasks.workunit.client.1.smithi117.stdout:1/399: unlink d1/d1f/l39 0 2024-03-22T06:15:22.332 INFO:tasks.workunit.client.1.smithi117.stdout:0/382: dread d1/d10/d18/d26/d49/d7e/f25 [0,4194304] 0 2024-03-22T06:15:22.333 INFO:tasks.workunit.client.1.smithi117.stdout:6/411: unlink d3/d40/d24/d2d/d53/d59/d74/d28/d85/c41 0 2024-03-22T06:15:22.334 INFO:tasks.workunit.client.1.smithi117.stdout:9/392: truncate d4/d7/f1b 1244839 0 2024-03-22T06:15:22.335 INFO:tasks.workunit.client.1.smithi117.stdout:0/383: mknod d1/d3/d2b/d3e/de/d1f/d3b/c84 0 2024-03-22T06:15:22.336 INFO:tasks.workunit.client.1.smithi117.stdout:0/384: fsync d1/d10/d18/d26/f75 0 2024-03-22T06:15:22.337 INFO:tasks.workunit.client.1.smithi117.stdout:1/400: mkdir d1/d22/d27/d4e/d92 0 2024-03-22T06:15:22.338 INFO:tasks.workunit.client.1.smithi117.stdout:0/385: symlink d1/l85 0 2024-03-22T06:15:22.340 INFO:tasks.workunit.client.1.smithi117.stdout:1/401: creat d1/d62/d7c/f93 x:0 0 0 2024-03-22T06:15:22.340 INFO:tasks.workunit.client.1.smithi117.stdout:8/354: write d1/d3a/d44/fa [3410552,29913] 0 2024-03-22T06:15:22.341 INFO:tasks.workunit.client.1.smithi117.stdout:4/341: dwrite d0/f11 [4194304,4194304] 0 2024-03-22T06:15:22.341 INFO:tasks.workunit.client.1.smithi117.stdout:1/402: mknod d1/d30/d5d/c94 0 2024-03-22T06:15:22.342 INFO:tasks.workunit.client.1.smithi117.stdout:1/403: write d1/f7 [4015497,119729] 0 2024-03-22T06:15:22.342 INFO:tasks.workunit.client.0.smithi006.stdout:0/994: dwrite db/d87/ddd/f121 [0,4194304] 0 2024-03-22T06:15:22.342 INFO:tasks.workunit.client.1.smithi117.stdout:2/348: dwrite d2/d5/db/d2c/f42 [0,4194304] 0 2024-03-22T06:15:22.343 INFO:tasks.workunit.client.1.smithi117.stdout:7/329: write d8/d9/da/f28 [3743179,52338] 0 2024-03-22T06:15:22.344 INFO:tasks.workunit.client.1.smithi117.stdout:8/355: rename d1/d3a/d44/c1b to d1/d3a/d44/d6a/c71 0 2024-03-22T06:15:22.345 INFO:tasks.workunit.client.1.smithi117.stdout:8/356: write d1/d19/f3f [263730,82565] 0 2024-03-22T06:15:22.349 INFO:tasks.workunit.client.0.smithi006.stdout:0/995: rename db/dc/d37/d137 to db/d3d/d4e/d15f 0 2024-03-22T06:15:22.351 INFO:tasks.workunit.client.1.smithi117.stdout:4/342: truncate d0/d19/f22 3281369 0 2024-03-22T06:15:22.351 INFO:tasks.workunit.client.1.smithi117.stdout:4/343: fdatasync d0/d19/d34/d43/f62 0 2024-03-22T06:15:22.352 INFO:tasks.workunit.client.1.smithi117.stdout:0/386: dread d1/f3d [0,4194304] 0 2024-03-22T06:15:22.354 INFO:tasks.workunit.client.0.smithi006.stdout:0/996: mkdir db/d10/d35/d7b/da1/da6/dc4/d11c/d160 0 2024-03-22T06:15:22.354 INFO:tasks.workunit.client.0.smithi006.stdout:0/997: creat db/d10/d35/d7b/f161 x:0 0 0 2024-03-22T06:15:22.355 INFO:tasks.workunit.client.1.smithi117.stdout:2/349: truncate d2/f26 1512451 0 2024-03-22T06:15:22.355 INFO:tasks.workunit.client.1.smithi117.stdout:2/350: truncate d2/f4a 4235668 0 2024-03-22T06:15:22.360 INFO:tasks.workunit.client.1.smithi117.stdout:0/387: dread d1/f3d [0,4194304] 0 2024-03-22T06:15:22.360 INFO:tasks.workunit.client.1.smithi117.stdout:0/388: stat d1/d10/d5c/d40/d4e/f65 0 2024-03-22T06:15:22.360 INFO:tasks.workunit.client.1.smithi117.stdout:3/311: dwrite d9/f22 [0,4194304] 0 2024-03-22T06:15:22.360 INFO:tasks.workunit.client.1.smithi117.stdout:0/389: chown d1/d3/d2b/d3e/de/d1f/f3c 2017 1 2024-03-22T06:15:22.360 INFO:tasks.workunit.client.1.smithi117.stdout:3/312: creat d9/d1a/d39/d4c/d36/f75 x:0 0 0 2024-03-22T06:15:22.362 INFO:tasks.workunit.client.1.smithi117.stdout:5/400: sync 2024-03-22T06:15:22.365 INFO:tasks.workunit.client.1.smithi117.stdout:6/412: dwrite d3/d3d/f43 [0,4194304] 0 2024-03-22T06:15:22.366 INFO:tasks.workunit.client.1.smithi117.stdout:8/357: unlink d1/d19/d38/d63/f65 0 2024-03-22T06:15:22.367 INFO:tasks.workunit.client.1.smithi117.stdout:4/344: creat d0/d19/d34/d43/d63/f74 x:0 0 0 2024-03-22T06:15:22.368 INFO:tasks.workunit.client.1.smithi117.stdout:7/330: rmdir d8/d9/da/d10/d23/d27/d39/d26 39 2024-03-22T06:15:22.368 INFO:tasks.workunit.client.1.smithi117.stdout:7/331: write d8/d9/da/d10/d23/d27/d32/f4d [437945,46067] 0 2024-03-22T06:15:22.368 INFO:tasks.workunit.client.1.smithi117.stdout:9/393: dwrite d4/d8/f1a [0,4194304] 0 2024-03-22T06:15:22.369 INFO:tasks.workunit.client.1.smithi117.stdout:1/404: dwrite d1/d1f/d32/d33/f3d [0,4194304] 0 2024-03-22T06:15:22.369 INFO:tasks.workunit.client.1.smithi117.stdout:1/405: write d1/d22/d27/f8e [702586,5819] 0 2024-03-22T06:15:22.371 INFO:tasks.workunit.client.0.smithi006.stdout:0/998: dread db/d10/f14 [4194304,4194304] 0 2024-03-22T06:15:22.376 INFO:tasks.workunit.client.0.smithi006.stdout:0/999: symlink db/dc/d37/l162 0 2024-03-22T06:15:22.377 INFO:tasks.workunit.client.1.smithi117.stdout:4/345: write d0/d19/d28/f2c [2963420,111109] 0 2024-03-22T06:15:22.377 INFO:tasks.workunit.client.1.smithi117.stdout:4/346: fdatasync d0/d4/d2a/d5b/f6e 0 2024-03-22T06:15:22.379 INFO:tasks.workunit.client.1.smithi117.stdout:6/413: link d3/d40/d24/d2d/d53/d59/d74/da/f29 d3/d40/d24/d2d/d53/d75/f8e 0 2024-03-22T06:15:22.379 INFO:tasks.workunit.client.1.smithi117.stdout:6/414: dread - d3/d40/d24/d2d/d53/d59/d74/d28/d85/f8a zero size 2024-03-22T06:15:22.379 INFO:tasks.workunit.client.1.smithi117.stdout:6/415: dread - d3/d40/d46/d84/d49/f86 zero size 2024-03-22T06:15:22.380 INFO:tasks.workunit.client.1.smithi117.stdout:7/332: dread d8/d9/da/d10/d23/d27/d32/f44 [0,4194304] 0 2024-03-22T06:15:22.383 INFO:tasks.workunit.client.1.smithi117.stdout:2/351: dwrite d2/d1b/d39/f65 [0,4194304] 0 2024-03-22T06:15:22.383 INFO:tasks.workunit.client.1.smithi117.stdout:2/352: readlink d2/d35/l50 0 2024-03-22T06:15:22.383 INFO:tasks.workunit.client.1.smithi117.stdout:2/353: write d2/d5/d13/f19 [2941994,33268] 0 2024-03-22T06:15:22.383 INFO:tasks.workunit.client.1.smithi117.stdout:2/354: creat d2/f69 x:0 0 0 2024-03-22T06:15:22.385 INFO:tasks.workunit.client.1.smithi117.stdout:5/401: dwrite dd/d16/d1a/f59 [0,4194304] 0 2024-03-22T06:15:22.385 INFO:tasks.workunit.client.1.smithi117.stdout:5/402: readlink dd/df/d1e/l44 0 2024-03-22T06:15:22.386 INFO:tasks.workunit.client.0.smithi006.stderr:+ rm -rf -- ./tmp.Oar7AqeBar 2024-03-22T06:15:22.390 INFO:tasks.workunit.client.1.smithi117.stdout:1/406: dread - d1/d22/d27/d4e/f6a zero size 2024-03-22T06:15:22.411 INFO:tasks.workunit.client.1.smithi117.stdout:3/313: getdents d9/d1a/d39/d4c/d36/d41 0 2024-03-22T06:15:22.411 INFO:tasks.workunit.client.1.smithi117.stdout:3/314: fdatasync d9/d66/f6d 0 2024-03-22T06:15:22.411 INFO:tasks.workunit.client.1.smithi117.stdout:3/315: chown d9/d1a/d3a/c3e 996591 1 2024-03-22T06:15:22.417 INFO:tasks.workunit.client.1.smithi117.stdout:5/403: dwrite dd/d16/d24/d72/d60/d6e/f7a [0,4194304] 0 2024-03-22T06:15:22.417 INFO:tasks.workunit.client.1.smithi117.stdout:5/404: write dd/d16/d5a/f83 [61145,52176] 0 2024-03-22T06:15:22.417 INFO:tasks.workunit.client.1.smithi117.stdout:5/405: truncate dd/df/d1e/f20 4404624 0 2024-03-22T06:15:22.417 INFO:tasks.workunit.client.1.smithi117.stdout:5/406: read - dd/d16/f65 zero size 2024-03-22T06:15:22.417 INFO:tasks.workunit.client.1.smithi117.stdout:4/347: creat d0/d19/d34/d43/d63/d6f/d5f/f75 x:0 0 0 2024-03-22T06:15:22.419 INFO:tasks.workunit.client.1.smithi117.stdout:2/355: dwrite d2/d5/d13/d57/f36 [0,4194304] 0 2024-03-22T06:15:22.419 INFO:tasks.workunit.client.1.smithi117.stdout:2/356: truncate d2/d5/d13/d57/f63 7809 0 2024-03-22T06:15:22.419 INFO:tasks.workunit.client.1.smithi117.stdout:2/357: fsync d2/d5/d13/d57/f47 0 2024-03-22T06:15:22.419 INFO:tasks.workunit.client.1.smithi117.stdout:1/407: dwrite d1/fd [4194304,4194304] 0 2024-03-22T06:15:22.419 INFO:tasks.workunit.client.1.smithi117.stdout:1/408: write d1/d62/f8a [138342,102599] 0 2024-03-22T06:15:22.419 INFO:tasks.workunit.client.1.smithi117.stdout:6/416: rename d3/d40/d46/d84/d49/f86 to d3/d3d/d17/d60/f8f 0 2024-03-22T06:15:22.421 INFO:tasks.workunit.client.1.smithi117.stdout:8/358: getdents d1/d19 0 2024-03-22T06:15:22.425 INFO:tasks.workunit.client.1.smithi117.stdout:7/333: mkdir d8/d50/d66 0 2024-03-22T06:15:22.428 INFO:tasks.workunit.client.1.smithi117.stdout:2/358: dread d2/f4 [0,4194304] 0 2024-03-22T06:15:22.428 INFO:tasks.workunit.client.1.smithi117.stdout:2/359: chown d2/d5/d13/d3f/d1d/d3a/c48 159631 1 2024-03-22T06:15:22.434 INFO:tasks.workunit.client.1.smithi117.stdout:6/417: dwrite d3/d40/d24/d2d/d53/d59/d74/f11 [0,4194304] 0 2024-03-22T06:15:22.434 INFO:tasks.workunit.client.1.smithi117.stdout:6/418: chown d3/d40/d24/d2d/d53/d59/d74/d8b 157592100 1 2024-03-22T06:15:22.434 INFO:tasks.workunit.client.1.smithi117.stdout:6/419: dread - d3/d40/d24/d2d/d53/d59/f88 zero size 2024-03-22T06:15:22.438 INFO:tasks.workunit.client.1.smithi117.stdout:3/316: symlink d9/d1a/d39/d4c/d36/d41/l76 0 2024-03-22T06:15:22.438 INFO:tasks.workunit.client.1.smithi117.stdout:3/317: write d9/d1a/f20 [574628,104955] 0 2024-03-22T06:15:22.438 INFO:tasks.workunit.client.1.smithi117.stdout:2/360: dread d2/d5/d13/d3f/d1d/f46 [4194304,4194304] 0 2024-03-22T06:15:22.445 INFO:tasks.workunit.client.1.smithi117.stdout:5/407: mknod dd/d16/d1a/d32/c87 0 2024-03-22T06:15:22.445 INFO:tasks.workunit.client.1.smithi117.stdout:5/408: chown dd/d16/f6c 37024 1 2024-03-22T06:15:22.448 INFO:tasks.workunit.client.1.smithi117.stdout:4/348: mknod d0/d48/c76 0 2024-03-22T06:15:22.449 INFO:tasks.workunit.client.1.smithi117.stdout:4/349: creat d0/d19/d34/f77 x:0 0 0 2024-03-22T06:15:22.449 INFO:tasks.workunit.client.1.smithi117.stdout:4/350: write d0/d4/d2a/f5d [842184,65575] 0 2024-03-22T06:15:22.463 INFO:tasks.workunit.client.1.smithi117.stdout:1/409: creat d1/d30/d5d/d6f/f95 x:0 0 0 2024-03-22T06:15:22.464 INFO:tasks.workunit.client.1.smithi117.stdout:1/410: creat d1/d3c/d4d/d5b/f96 x:0 0 0 2024-03-22T06:15:22.464 INFO:tasks.workunit.client.1.smithi117.stdout:1/411: fdatasync d1/fd 0 2024-03-22T06:15:22.466 INFO:tasks.workunit.client.1.smithi117.stdout:8/359: link d1/d3a/d44/c58 d1/d28/d6b/c72 0 2024-03-22T06:15:22.467 INFO:tasks.workunit.client.1.smithi117.stdout:8/360: dread d1/d19/f64 [0,4194304] 0 2024-03-22T06:15:22.470 INFO:tasks.workunit.client.1.smithi117.stdout:7/334: fdatasync d8/d9/da/d10/d23/d27/d39/f5d 0 2024-03-22T06:15:22.473 INFO:tasks.workunit.client.1.smithi117.stdout:0/390: sync 2024-03-22T06:15:22.473 INFO:tasks.workunit.client.1.smithi117.stdout:6/420: creat d3/d40/d24/d2d/d30/f90 x:0 0 0 2024-03-22T06:15:22.482 INFO:tasks.workunit.client.1.smithi117.stdout:3/318: symlink d9/d1a/d39/d4c/l77 0 2024-03-22T06:15:22.482 INFO:tasks.workunit.client.1.smithi117.stdout:6/421: read d3/d40/d24/d2d/d53/d59/d74/d8b/f7a [1103914,14709] 0 2024-03-22T06:15:22.483 INFO:tasks.workunit.client.1.smithi117.stdout:2/361: symlink d2/d5/d13/d3f/d1d/d3a/d11/d4f/l6a 0 2024-03-22T06:15:22.485 INFO:tasks.workunit.client.1.smithi117.stdout:6/422: dread d3/d40/d46/d84/d49/d79/f7d [0,4194304] 0 2024-03-22T06:15:22.487 INFO:tasks.workunit.client.1.smithi117.stdout:9/394: sync 2024-03-22T06:15:22.488 INFO:tasks.workunit.client.1.smithi117.stdout:9/395: dread d4/d8/d3e/f45 [0,4194304] 0 2024-03-22T06:15:22.489 INFO:tasks.workunit.client.1.smithi117.stdout:4/351: rename d0/d3f/l46 to d0/d19/d34/d43/d63/l78 0 2024-03-22T06:15:22.489 INFO:tasks.workunit.client.1.smithi117.stdout:4/352: chown d0/d4/d2a/d5b/f6e 1008613 1 2024-03-22T06:15:22.489 INFO:tasks.workunit.client.1.smithi117.stdout:3/319: read d9/f11 [674454,56211] 0 2024-03-22T06:15:22.493 INFO:tasks.workunit.client.1.smithi117.stdout:7/335: truncate d8/d9/da/d10/d23/d27/d39/f3b 3622860 0 2024-03-22T06:15:22.493 INFO:tasks.workunit.client.1.smithi117.stdout:7/336: chown d8/d9/da/f28 2246097 1 2024-03-22T06:15:22.493 INFO:tasks.workunit.client.1.smithi117.stdout:7/337: stat d8/f3a 0 2024-03-22T06:15:22.495 INFO:tasks.workunit.client.1.smithi117.stdout:0/391: unlink d1/d3/d2b/f83 0 2024-03-22T06:15:22.495 INFO:tasks.workunit.client.1.smithi117.stdout:0/392: write d1/d3/d2b/d3e/fb [770694,91831] 0 2024-03-22T06:15:22.495 INFO:tasks.workunit.client.1.smithi117.stdout:0/393: truncate d1/d10/d5c/d40/d69/d72/f76 582871 0 2024-03-22T06:15:22.495 INFO:tasks.workunit.client.1.smithi117.stdout:0/394: fdatasync d1/d3/d2b/d3e/de/f23 0 2024-03-22T06:15:22.495 INFO:tasks.workunit.client.1.smithi117.stdout:0/395: truncate d1/d10/d5c/d40/f73 293708 0 2024-03-22T06:15:22.495 INFO:tasks.workunit.client.1.smithi117.stdout:0/396: readlink d1/d3/d2b/d3e/de/l57 0 2024-03-22T06:15:22.498 INFO:tasks.workunit.client.1.smithi117.stdout:2/362: truncate d2/d5/d13/d3f/f51 473869 0 2024-03-22T06:15:22.498 INFO:tasks.workunit.client.1.smithi117.stdout:2/363: write d2/d5/f33 [390792,45348] 0 2024-03-22T06:15:22.498 INFO:tasks.workunit.client.1.smithi117.stdout:2/364: creat d2/d5/d13/d3f/f6b x:0 0 0 2024-03-22T06:15:22.499 INFO:tasks.workunit.client.1.smithi117.stdout:5/409: mkdir dd/df/d88 0 2024-03-22T06:15:22.503 INFO:tasks.workunit.client.1.smithi117.stdout:9/396: mknod d4/d7/d10/d2f/c74 0 2024-03-22T06:15:22.505 INFO:tasks.workunit.client.1.smithi117.stdout:1/412: rename d1/d62/c6e to d1/d1f/c97 0 2024-03-22T06:15:22.508 INFO:tasks.workunit.client.1.smithi117.stdout:4/353: rmdir d0/d19/d34/d71 0 2024-03-22T06:15:22.508 INFO:tasks.workunit.client.1.smithi117.stdout:5/410: write dd/d16/d24/d72/d60/f6b [2211290,105931] 0 2024-03-22T06:15:22.511 INFO:tasks.workunit.client.1.smithi117.stdout:3/320: link d9/d1a/d39/d4c/d36/d41/l76 d9/d1a/d39/d4c/d37/l78 0 2024-03-22T06:15:22.512 INFO:tasks.workunit.client.1.smithi117.stdout:1/413: write d1/d22/f4b [3579001,20346] 0 2024-03-22T06:15:22.512 INFO:tasks.workunit.client.1.smithi117.stdout:1/414: fdatasync d1/d30/d35/f64 0 2024-03-22T06:15:22.512 INFO:tasks.workunit.client.1.smithi117.stdout:1/415: readlink d1/d30/d35/l2d 0 2024-03-22T06:15:22.514 INFO:tasks.workunit.client.1.smithi117.stdout:7/338: getdents d8/d9/d37/d45/d61 0 2024-03-22T06:15:22.515 INFO:tasks.workunit.client.1.smithi117.stdout:0/397: creat d1/d10/f86 x:0 0 0 2024-03-22T06:15:22.515 INFO:tasks.workunit.client.1.smithi117.stdout:0/398: creat d1/f87 x:0 0 0 2024-03-22T06:15:22.519 INFO:tasks.workunit.client.1.smithi117.stdout:2/365: rmdir d2/d35/d38 39 2024-03-22T06:15:22.519 INFO:tasks.workunit.client.1.smithi117.stdout:2/366: truncate d2/d5/f33 1269843 0 2024-03-22T06:15:22.519 INFO:tasks.workunit.client.1.smithi117.stdout:2/367: dread - d2/d5/d41/f5b zero size 2024-03-22T06:15:22.520 INFO:tasks.workunit.client.1.smithi117.stdout:6/423: rename d3/d3d/d17/d31/c77 to d3/d3d/d17/d60/c91 0 2024-03-22T06:15:22.523 INFO:tasks.workunit.client.1.smithi117.stdout:5/411: symlink dd/d16/d1a/d32/d73/l89 0 2024-03-22T06:15:22.524 INFO:tasks.workunit.client.1.smithi117.stdout:5/412: creat dd/f8a x:0 0 0 2024-03-22T06:15:22.524 INFO:tasks.workunit.client.1.smithi117.stdout:5/413: truncate dd/d16/d24/f4f 370911 0 2024-03-22T06:15:22.524 INFO:tasks.workunit.client.1.smithi117.stdout:5/414: creat dd/d16/d1a/d32/f8b x:0 0 0 2024-03-22T06:15:22.524 INFO:tasks.workunit.client.1.smithi117.stdout:5/415: stat dd/d16/d24/d36/f39 0 2024-03-22T06:15:22.524 INFO:tasks.workunit.client.1.smithi117.stdout:5/416: chown dd/d16/d24/d36/f39 405427 1 2024-03-22T06:15:22.525 INFO:tasks.workunit.client.1.smithi117.stdout:3/321: symlink d9/d1a/d39/d4c/d37/l79 0 2024-03-22T06:15:22.525 INFO:tasks.workunit.client.1.smithi117.stdout:3/322: stat d9/d1a/d3a/f53 0 2024-03-22T06:15:22.525 INFO:tasks.workunit.client.1.smithi117.stdout:3/323: chown d9/d1a/d39/d4c/d37/f61 517350149 1 2024-03-22T06:15:22.529 INFO:tasks.workunit.client.1.smithi117.stdout:7/339: mknod d8/d9/da/d10/d23/c67 0 2024-03-22T06:15:22.531 INFO:tasks.workunit.client.1.smithi117.stdout:5/417: dread dd/d16/d24/d72/d35/f6f [0,4194304] 0 2024-03-22T06:15:22.531 INFO:tasks.workunit.client.1.smithi117.stdout:5/418: read - dd/d16/d1a/d32/f8b zero size 2024-03-22T06:15:22.531 INFO:tasks.workunit.client.1.smithi117.stdout:0/399: creat d1/d10/d18/d26/d49/d7e/d39/f88 x:0 0 0 2024-03-22T06:15:22.534 INFO:tasks.workunit.client.1.smithi117.stdout:2/368: creat d2/d5/d13/d3f/d1d/f6c x:0 0 0 2024-03-22T06:15:22.534 INFO:tasks.workunit.client.1.smithi117.stdout:3/324: write d9/d1a/f1f [3857211,45441] 0 2024-03-22T06:15:22.534 INFO:tasks.workunit.client.1.smithi117.stdout:3/325: creat d9/d1a/d3a/f7a x:0 0 0 2024-03-22T06:15:22.535 INFO:tasks.workunit.client.1.smithi117.stdout:3/326: stat d9/f30 0 2024-03-22T06:15:22.536 INFO:tasks.workunit.client.1.smithi117.stdout:9/397: rename d4/d8/l49 to d4/d8/d3e/l75 0 2024-03-22T06:15:22.536 INFO:tasks.workunit.client.1.smithi117.stdout:9/398: write d4/d7/f16 [2754357,20256] 0 2024-03-22T06:15:22.539 INFO:tasks.workunit.client.1.smithi117.stdout:6/424: mknod d3/d40/d24/d2d/d53/d59/d74/c92 0 2024-03-22T06:15:22.539 INFO:tasks.workunit.client.1.smithi117.stdout:6/425: write d3/d40/d24/d2d/d53/d59/d74/f1b [1700775,114869] 0 2024-03-22T06:15:22.539 INFO:tasks.workunit.client.1.smithi117.stdout:6/426: dread - d3/d3d/f8d zero size 2024-03-22T06:15:22.539 INFO:tasks.workunit.client.1.smithi117.stdout:6/427: chown d3/d40/d24/d2d/d53/d59/d74/d28/d85/c63 45837 1 2024-03-22T06:15:22.541 INFO:tasks.workunit.client.1.smithi117.stdout:4/354: getdents d0/d19/d34/d43/d63/d6f/d5f 0 2024-03-22T06:15:22.545 INFO:tasks.workunit.client.1.smithi117.stdout:6/428: dread d3/d40/d46/d84/d49/f5b [0,4194304] 0 2024-03-22T06:15:22.545 INFO:tasks.workunit.client.1.smithi117.stdout:0/400: dread d1/d10/d18/f38 [0,4194304] 0 2024-03-22T06:15:22.545 INFO:tasks.workunit.client.1.smithi117.stdout:0/401: readlink d1/d10/d18/d26/d49/d7e/d39/d6a/l80 0 2024-03-22T06:15:22.545 INFO:tasks.workunit.client.1.smithi117.stdout:0/402: creat d1/d10/f89 x:0 0 0 2024-03-22T06:15:22.545 INFO:tasks.workunit.client.1.smithi117.stdout:0/403: chown c0 0 1 2024-03-22T06:15:22.550 INFO:tasks.workunit.client.1.smithi117.stdout:8/361: rmdir d1/d48 0 2024-03-22T06:15:22.551 INFO:tasks.workunit.client.1.smithi117.stdout:5/419: rmdir dd/df/d5e 39 2024-03-22T06:15:22.558 INFO:tasks.workunit.client.1.smithi117.stdout:1/416: rename d1/d22/d27/f90 to d1/d84/d77/f98 0 2024-03-22T06:15:22.559 INFO:tasks.workunit.client.1.smithi117.stdout:1/417: stat d1/d62/d7c 0 2024-03-22T06:15:22.560 INFO:tasks.workunit.client.1.smithi117.stdout:8/362: write f0 [832519,105406] 0 2024-03-22T06:15:22.561 INFO:tasks.workunit.client.1.smithi117.stdout:4/355: mknod d0/d4/d2a/d2f/c79 0 2024-03-22T06:15:22.561 INFO:tasks.workunit.client.1.smithi117.stdout:4/356: chown d0/d19/d34/d43/l68 31 1 2024-03-22T06:15:22.561 INFO:tasks.workunit.client.1.smithi117.stdout:4/357: write d0/d4/d2a/d5b/f6e [861701,87928] 0 2024-03-22T06:15:22.561 INFO:tasks.workunit.client.1.smithi117.stdout:6/429: stat d3/d40/d24/d2d/d53/c65 0 2024-03-22T06:15:22.562 INFO:tasks.workunit.client.1.smithi117.stdout:7/340: dwrite d8/d9/da/d10/d23/f38 [0,4194304] 0 2024-03-22T06:15:22.564 INFO:tasks.workunit.client.1.smithi117.stdout:1/418: mkdir d1/d30/d44/d99 0 2024-03-22T06:15:22.564 INFO:tasks.workunit.client.1.smithi117.stdout:1/419: stat d1/f43 0 2024-03-22T06:15:22.566 INFO:tasks.workunit.client.1.smithi117.stdout:8/363: rename d1/d3a/d44/d9/l6f to d1/d3a/d44/d6a/l73 0 2024-03-22T06:15:22.567 INFO:tasks.workunit.client.1.smithi117.stdout:3/327: dwrite f7 [0,4194304] 0 2024-03-22T06:15:22.567 INFO:tasks.workunit.client.1.smithi117.stdout:8/364: write d1/d19/f3f [2126769,74980] 0 2024-03-22T06:15:22.567 INFO:tasks.workunit.client.1.smithi117.stdout:3/328: creat d9/d55/f7b x:0 0 0 2024-03-22T06:15:22.567 INFO:tasks.workunit.client.1.smithi117.stdout:3/329: write d9/d1a/d39/d4c/d37/f61 [3098292,38098] 0 2024-03-22T06:15:22.567 INFO:tasks.workunit.client.1.smithi117.stdout:4/358: truncate d0/d19/f22 1350642 0 2024-03-22T06:15:22.567 INFO:tasks.workunit.client.1.smithi117.stdout:4/359: chown d0/d48 1050911827 1 2024-03-22T06:15:22.568 INFO:tasks.workunit.client.1.smithi117.stdout:4/360: readlink d0/d19/l3e 0 2024-03-22T06:15:22.569 INFO:tasks.workunit.client.1.smithi117.stdout:9/399: dwrite d4/d7/d27/f56 [0,4194304] 0 2024-03-22T06:15:22.569 INFO:tasks.workunit.client.1.smithi117.stdout:2/369: dwrite d2/d5/d13/f5d [0,4194304] 0 2024-03-22T06:15:22.576 INFO:tasks.workunit.client.1.smithi117.stdout:4/361: write d0/d48/f3d [2105119,55845] 0 2024-03-22T06:15:22.576 INFO:tasks.workunit.client.1.smithi117.stdout:6/430: rename d3/d40/d24/d2d/d53/d59/d74/c92 to d3/d40/d24/d2d/d53/d75/c93 0 2024-03-22T06:15:22.581 INFO:tasks.workunit.client.1.smithi117.stdout:5/420: dread dd/d16/d24/d72/d60/f6b [0,4194304] 0 2024-03-22T06:15:22.582 INFO:tasks.workunit.client.1.smithi117.stdout:8/365: creat d1/d28/d66/f74 x:0 0 0 2024-03-22T06:15:22.582 INFO:tasks.workunit.client.1.smithi117.stdout:8/366: creat d1/d19/d38/f75 x:0 0 0 2024-03-22T06:15:22.587 INFO:tasks.workunit.client.1.smithi117.stdout:9/400: mkdir d4/d46/d76 0 2024-03-22T06:15:22.588 INFO:tasks.workunit.client.1.smithi117.stdout:1/420: dwrite d1/d1f/f79 [0,4194304] 0 2024-03-22T06:15:22.588 INFO:tasks.workunit.client.1.smithi117.stdout:1/421: write d1/d3c/f88 [448496,26284] 0 2024-03-22T06:15:22.589 INFO:tasks.workunit.client.1.smithi117.stdout:2/370: fdatasync d2/d5/f33 0 2024-03-22T06:15:22.589 INFO:tasks.workunit.client.1.smithi117.stdout:2/371: truncate d2/f26 2044212 0 2024-03-22T06:15:22.589 INFO:tasks.workunit.client.1.smithi117.stdout:2/372: creat d2/d5/d13/d3f/d1d/f6d x:0 0 0 2024-03-22T06:15:22.589 INFO:tasks.workunit.client.1.smithi117.stdout:4/362: write d0/d19/d34/d43/f62 [1708679,47486] 0 2024-03-22T06:15:22.592 INFO:tasks.workunit.client.1.smithi117.stdout:0/404: dwrite d1/d3/d2b/d3e/de/f51 [0,4194304] 0 2024-03-22T06:15:22.593 INFO:tasks.workunit.client.1.smithi117.stdout:5/421: dread dd/d16/f48 [0,4194304] 0 2024-03-22T06:15:22.593 INFO:tasks.workunit.client.1.smithi117.stdout:7/341: symlink d8/d9/da/d10/d23/d27/d39/d5c/l68 0 2024-03-22T06:15:22.593 INFO:tasks.workunit.client.1.smithi117.stdout:7/342: write d8/d9/da/f28 [2321454,91818] 0 2024-03-22T06:15:22.593 INFO:tasks.workunit.client.1.smithi117.stdout:7/343: chown d8/d9/c2a 2512791 1 2024-03-22T06:15:22.596 INFO:tasks.workunit.client.1.smithi117.stdout:6/431: truncate d3/d40/d24/d2d/d53/d59/d74/f11 3174992 0 2024-03-22T06:15:22.596 INFO:tasks.workunit.client.1.smithi117.stdout:6/432: write d3/f3f [2473206,49986] 0 2024-03-22T06:15:22.599 INFO:tasks.workunit.client.1.smithi117.stdout:1/422: write d1/d22/d27/f34 [1976882,100039] 0 2024-03-22T06:15:22.599 INFO:tasks.workunit.client.1.smithi117.stdout:1/423: truncate d1/d3c/d4d/d5b/f91 594280 0 2024-03-22T06:15:22.599 INFO:tasks.workunit.client.1.smithi117.stdout:1/424: creat d1/d3c/f9a x:0 0 0 2024-03-22T06:15:22.603 INFO:tasks.workunit.client.1.smithi117.stdout:8/367: symlink d1/d3a/l76 0 2024-03-22T06:15:22.607 INFO:tasks.workunit.client.1.smithi117.stdout:3/330: link d9/d1a/d39/d4c/d36/d41/f54 d9/d55/f7c 0 2024-03-22T06:15:22.607 INFO:tasks.workunit.client.1.smithi117.stdout:3/331: write d9/d55/f7b [667443,46273] 0 2024-03-22T06:15:22.610 INFO:tasks.workunit.client.1.smithi117.stdout:9/401: mknod d4/d8/d37/c77 0 2024-03-22T06:15:22.616 INFO:tasks.workunit.client.1.smithi117.stdout:4/363: dwrite d0/d19/d28/f55 [4194304,4194304] 0 2024-03-22T06:15:22.617 INFO:tasks.workunit.client.1.smithi117.stdout:2/373: dwrite d2/d5/db/f24 [0,4194304] 0 2024-03-22T06:15:22.617 INFO:tasks.workunit.client.1.smithi117.stdout:2/374: getdents d2/d35/d38 0 2024-03-22T06:15:22.619 INFO:tasks.workunit.client.1.smithi117.stdout:0/405: symlink d1/d10/d18/d26/d53/l8a 0 2024-03-22T06:15:22.622 INFO:tasks.workunit.client.1.smithi117.stdout:5/422: mknod dd/df/d5e/c8c 0 2024-03-22T06:15:22.624 INFO:tasks.workunit.client.1.smithi117.stdout:6/433: dwrite d3/d40/d24/d2d/d53/d59/d74/da/de/f19 [4194304,4194304] 0 2024-03-22T06:15:22.624 INFO:tasks.workunit.client.1.smithi117.stdout:6/434: write d3/d3d/d17/d31/f34 [218930,34792] 0 2024-03-22T06:15:22.624 INFO:tasks.workunit.client.1.smithi117.stdout:3/332: dwrite d9/d1a/d39/d4c/f42 [0,4194304] 0 2024-03-22T06:15:22.625 INFO:tasks.workunit.client.1.smithi117.stdout:7/344: rmdir d8/d9/da/d10/d23/d27/d39 39 2024-03-22T06:15:22.627 INFO:tasks.workunit.client.1.smithi117.stdout:2/375: dread d2/d5/db/f24 [0,4194304] 0 2024-03-22T06:15:22.628 INFO:tasks.workunit.client.1.smithi117.stdout:6/435: fdatasync d3/d3d/d17/d31/f87 0 2024-03-22T06:15:22.628 INFO:tasks.workunit.client.1.smithi117.stdout:6/436: chown d3/d40/d24/d2d/d42/c45 1328272 1 2024-03-22T06:15:22.630 INFO:tasks.workunit.client.1.smithi117.stdout:1/425: creat d1/d22/d27/d4e/f9b x:0 0 0 2024-03-22T06:15:22.630 INFO:tasks.workunit.client.1.smithi117.stdout:1/426: write d1/d22/d27/d4e/f9b [38734,38833] 0 2024-03-22T06:15:22.632 INFO:tasks.workunit.client.1.smithi117.stdout:8/368: link d1/d19/d38/f70 d1/f77 0 2024-03-22T06:15:22.632 INFO:tasks.workunit.client.1.smithi117.stdout:2/376: dread d2/f26 [0,4194304] 0 2024-03-22T06:15:22.633 INFO:tasks.workunit.client.1.smithi117.stdout:9/402: creat d4/d7/d27/d5b/f78 x:0 0 0 2024-03-22T06:15:22.636 INFO:tasks.workunit.client.1.smithi117.stdout:2/377: dread d2/f26 [0,4194304] 0 2024-03-22T06:15:22.642 INFO:tasks.workunit.client.1.smithi117.stdout:0/406: dwrite d1/d3/d2b/d3e/de/d1f/d3b/f42 [0,4194304] 0 2024-03-22T06:15:22.642 INFO:tasks.workunit.client.1.smithi117.stdout:4/364: rename d0/d4/d2a/d2f/c79 to d0/d4/c7a 0 2024-03-22T06:15:22.642 INFO:tasks.workunit.client.1.smithi117.stdout:0/407: chown c0 9517980 1 2024-03-22T06:15:22.648 INFO:tasks.workunit.client.1.smithi117.stdout:4/365: dread d0/d48/f3d [0,4194304] 0 2024-03-22T06:15:22.648 INFO:tasks.workunit.client.1.smithi117.stdout:4/366: write d0/d19/d34/d43/f57 [873595,51530] 0 2024-03-22T06:15:22.653 INFO:tasks.workunit.client.1.smithi117.stdout:5/423: link dd/df/d1e/l44 dd/d16/d24/d72/d60/l8d 0 2024-03-22T06:15:22.656 INFO:tasks.workunit.client.1.smithi117.stdout:3/333: mkdir d9/d1a/d39/d7d 0 2024-03-22T06:15:22.657 INFO:tasks.workunit.client.1.smithi117.stdout:7/345: creat d8/d9/da/f69 x:0 0 0 2024-03-22T06:15:22.658 INFO:tasks.workunit.client.1.smithi117.stdout:6/437: getdents d3/d40 0 2024-03-22T06:15:22.659 INFO:tasks.workunit.client.1.smithi117.stdout:1/427: mkdir d1/d30/d44/d9c 0 2024-03-22T06:15:22.662 INFO:tasks.workunit.client.1.smithi117.stdout:2/378: mkdir d2/d6e 0 2024-03-22T06:15:22.663 INFO:tasks.workunit.client.1.smithi117.stdout:8/369: dwrite d1/f34 [0,4194304] 0 2024-03-22T06:15:22.663 INFO:tasks.workunit.client.1.smithi117.stdout:8/370: truncate d1/d19/d38/f70 685354 0 2024-03-22T06:15:22.664 INFO:tasks.workunit.client.1.smithi117.stdout:8/371: readlink d1/l41 0 2024-03-22T06:15:22.664 INFO:tasks.workunit.client.1.smithi117.stdout:0/408: dwrite d1/d10/d18/d26/f75 [0,4194304] 0 2024-03-22T06:15:22.680 INFO:tasks.workunit.client.1.smithi117.stdout:4/367: symlink d0/d19/d34/l7b 0 2024-03-22T06:15:22.701 INFO:tasks.workunit.client.1.smithi117.stdout:5/424: symlink dd/d16/d1a/d32/d73/l8e 0 2024-03-22T06:15:22.702 INFO:tasks.workunit.client.1.smithi117.stdout:3/334: creat d9/d1a/d39/d7d/f7e x:0 0 0 2024-03-22T06:15:22.703 INFO:tasks.workunit.client.1.smithi117.stdout:3/335: chown d9/d1a/d39/d4c/f5e 7 1 2024-03-22T06:15:22.703 INFO:tasks.workunit.client.1.smithi117.stdout:7/346: symlink d8/d9/da/d10/d23/d41/l6a 0 2024-03-22T06:15:22.705 INFO:tasks.workunit.client.1.smithi117.stdout:6/438: symlink d3/d40/d46/d84/l94 0 2024-03-22T06:15:22.705 INFO:tasks.workunit.client.1.smithi117.stdout:6/439: stat d3/d40/d24/d2d/d53/d59/d74/d8b 0 2024-03-22T06:15:22.708 INFO:tasks.workunit.client.1.smithi117.stdout:2/379: unlink d2/d5/d13/d3f/d1d/d3a/f37 0 2024-03-22T06:15:22.708 INFO:tasks.workunit.client.1.smithi117.stdout:2/380: creat d2/d5/d41/f6f x:0 0 0 2024-03-22T06:15:22.712 INFO:tasks.workunit.client.1.smithi117.stdout:0/409: stat d1/c2 0 2024-03-22T06:15:22.720 INFO:tasks.workunit.client.1.smithi117.stdout:7/347: dwrite d8/d9/da/d10/f22 [0,4194304] 0 2024-03-22T06:15:22.725 INFO:tasks.workunit.client.1.smithi117.stdout:3/336: rename d9/f22 to d9/d1a/d39/f7f 0 2024-03-22T06:15:22.726 INFO:tasks.workunit.client.1.smithi117.stdout:6/440: truncate d3/d40/d46/d84/d49/d79/f7d 6426870 0 2024-03-22T06:15:22.727 INFO:tasks.workunit.client.1.smithi117.stdout:6/441: write d3/d3d/d17/d31/f78 [378080,104188] 0 2024-03-22T06:15:22.732 INFO:tasks.workunit.client.1.smithi117.stdout:2/381: link d2/d5/d13/d3f/l58 d2/d5/d13/d3f/d1d/l70 0 2024-03-22T06:15:22.733 INFO:tasks.workunit.client.1.smithi117.stdout:8/372: truncate d1/f34 743759 0 2024-03-22T06:15:22.733 INFO:tasks.workunit.client.1.smithi117.stdout:8/373: fdatasync d1/d19/f64 0 2024-03-22T06:15:22.735 INFO:tasks.workunit.client.1.smithi117.stdout:0/410: rmdir d1/d10/d5c 39 2024-03-22T06:15:22.737 INFO:tasks.workunit.client.1.smithi117.stdout:4/368: getdents d0/d19/d34/d43/d63 0 2024-03-22T06:15:22.737 INFO:tasks.workunit.client.1.smithi117.stdout:4/369: fdatasync d0/d3f/f54 0 2024-03-22T06:15:22.737 INFO:tasks.workunit.client.1.smithi117.stdout:4/370: write d0/d3f/f4b [359124,71067] 0 2024-03-22T06:15:22.738 INFO:tasks.workunit.client.1.smithi117.stdout:7/348: dwrite d8/d9/d37/f4e [0,4194304] 0 2024-03-22T06:15:22.739 INFO:tasks.workunit.client.1.smithi117.stdout:5/425: mknod dd/df/c8f 0 2024-03-22T06:15:22.739 INFO:tasks.workunit.client.1.smithi117.stdout:5/426: chown l5 86965 1 2024-03-22T06:15:22.741 INFO:tasks.workunit.client.1.smithi117.stdout:3/337: symlink d9/d1a/d39/d4c/d37/d69/l80 0 2024-03-22T06:15:22.741 INFO:tasks.workunit.client.1.smithi117.stdout:3/338: stat d9/d55/l72 0 2024-03-22T06:15:22.741 INFO:tasks.workunit.client.1.smithi117.stdout:3/339: fsync d9/d1a/d39/f48 0 2024-03-22T06:15:22.741 INFO:tasks.workunit.client.1.smithi117.stdout:3/340: creat d9/d55/f81 x:0 0 0 2024-03-22T06:15:22.742 INFO:tasks.workunit.client.1.smithi117.stdout:6/442: creat d3/d40/f95 x:0 0 0 2024-03-22T06:15:22.745 INFO:tasks.workunit.client.1.smithi117.stdout:2/382: mkdir d2/d5/d13/d3f/d1d/d3a/d71 0 2024-03-22T06:15:22.746 INFO:tasks.workunit.client.1.smithi117.stdout:9/403: sync 2024-03-22T06:15:22.748 INFO:tasks.workunit.client.1.smithi117.stdout:4/371: rename d0/f6c to d0/d19/d34/d43/f7c 0 2024-03-22T06:15:22.750 INFO:tasks.workunit.client.1.smithi117.stdout:7/349: dread d8/d9/da/d10/f22 [0,4194304] 0 2024-03-22T06:15:22.750 INFO:tasks.workunit.client.1.smithi117.stdout:1/428: sync 2024-03-22T06:15:22.752 INFO:tasks.workunit.client.1.smithi117.stdout:5/427: mkdir dd/d16/d24/d90 0 2024-03-22T06:15:22.754 INFO:tasks.workunit.client.1.smithi117.stdout:8/374: dwrite d1/d19/f64 [0,4194304] 0 2024-03-22T06:15:22.760 INFO:tasks.workunit.client.1.smithi117.stdout:3/341: mkdir d9/d1a/d39/d82 0 2024-03-22T06:15:22.762 INFO:tasks.workunit.client.1.smithi117.stdout:6/443: mknod d3/d40/d24/d2d/d53/d59/c96 0 2024-03-22T06:15:22.767 INFO:tasks.workunit.client.1.smithi117.stdout:1/429: dwrite d1/d1f/f75 [0,4194304] 0 2024-03-22T06:15:22.767 INFO:tasks.workunit.client.1.smithi117.stdout:1/430: write d1/d3c/d4d/d5b/f96 [521596,27230] 0 2024-03-22T06:15:22.767 INFO:tasks.workunit.client.1.smithi117.stdout:8/375: write d1/d3a/d44/d11/f2a [123838,69913] 0 2024-03-22T06:15:22.767 INFO:tasks.workunit.client.1.smithi117.stdout:8/376: stat d1/d3a/c69 0 2024-03-22T06:15:22.774 INFO:tasks.workunit.client.1.smithi117.stdout:9/404: creat d4/d46/f79 x:0 0 0 2024-03-22T06:15:22.784 INFO:tasks.workunit.client.1.smithi117.stdout:2/383: getdents d2/d5/d13/d57 0 2024-03-22T06:15:22.784 INFO:tasks.workunit.client.1.smithi117.stdout:2/384: chown d2/d5/d13/d3f/f51 3 1 2024-03-22T06:15:22.786 INFO:tasks.workunit.client.1.smithi117.stdout:4/372: rmdir d0/d19/d34 39 2024-03-22T06:15:22.789 INFO:tasks.workunit.client.1.smithi117.stdout:5/428: rename dd/d16/d24 to dd/df/d5e/d91 0 2024-03-22T06:15:22.790 INFO:tasks.workunit.client.1.smithi117.stdout:2/385: link d2/d5/d13/d3f/d1d/d3a/f64 d2/d5/db/f72 0 2024-03-22T06:15:22.790 INFO:tasks.workunit.client.1.smithi117.stdout:2/386: stat d2/d5/d41/f5b 0 2024-03-22T06:15:22.790 INFO:tasks.workunit.client.1.smithi117.stdout:5/429: creat dd/df/d5e/f92 x:0 0 0 2024-03-22T06:15:22.791 INFO:tasks.workunit.client.1.smithi117.stdout:5/430: write dd/df/d5e/d91/f4f [654008,98187] 0 2024-03-22T06:15:22.791 INFO:tasks.workunit.client.1.smithi117.stdout:5/431: mkdir dd/d16/d86/d93 0 2024-03-22T06:15:22.792 INFO:tasks.workunit.client.1.smithi117.stdout:5/432: symlink dd/df/d5e/d91/d36/d70/l94 0 2024-03-22T06:15:22.792 INFO:tasks.workunit.client.1.smithi117.stdout:5/433: chown dd/df/d5e/c77 13639 1 2024-03-22T06:15:22.793 INFO:tasks.workunit.client.1.smithi117.stdout:5/434: mkdir dd/df/d95 0 2024-03-22T06:15:22.793 INFO:tasks.workunit.client.1.smithi117.stdout:7/350: dread d8/d9/da/d10/d23/d27/d39/d26/f34 [0,4194304] 0 2024-03-22T06:15:22.794 INFO:tasks.workunit.client.1.smithi117.stdout:5/435: write dd/df/f3a [4914573,107388] 0 2024-03-22T06:15:22.795 INFO:tasks.workunit.client.1.smithi117.stdout:7/351: symlink d8/d9/da/d10/d23/l6b 0 2024-03-22T06:15:22.797 INFO:tasks.workunit.client.1.smithi117.stdout:4/373: dread d0/f23 [0,4194304] 0 2024-03-22T06:15:22.797 INFO:tasks.workunit.client.1.smithi117.stdout:4/374: fdatasync d0/d19/f22 0 2024-03-22T06:15:22.797 INFO:tasks.workunit.client.1.smithi117.stdout:4/375: chown d0/d4/d2a/d2f/f56 15369 1 2024-03-22T06:15:22.797 INFO:tasks.workunit.client.1.smithi117.stdout:5/436: rename dd/df/d5e/d91/d72/l3c to dd/df/d5e/d91/l96 0 2024-03-22T06:15:22.797 INFO:tasks.workunit.client.1.smithi117.stdout:5/437: creat dd/d16/d5a/f97 x:0 0 0 2024-03-22T06:15:22.797 INFO:tasks.workunit.client.1.smithi117.stdout:7/352: mkdir d8/d6c 0 2024-03-22T06:15:22.798 INFO:tasks.workunit.client.1.smithi117.stdout:4/376: creat d0/d19/d34/d43/d63/d6f/d70/f7d x:0 0 0 2024-03-22T06:15:22.799 INFO:tasks.workunit.client.1.smithi117.stdout:5/438: mkdir dd/df/d5e/d91/d36/d98 0 2024-03-22T06:15:22.799 INFO:tasks.workunit.client.1.smithi117.stdout:5/439: stat dd/df/d5e/d91/d72/d60/d6e 0 2024-03-22T06:15:22.800 INFO:tasks.workunit.client.1.smithi117.stdout:5/440: fsync dd/df/d5e/d91/d72/f45 0 2024-03-22T06:15:22.800 INFO:tasks.workunit.client.1.smithi117.stdout:7/353: rename d8/d9/da/d2b/c46 to d8/c6d 0 2024-03-22T06:15:22.803 INFO:tasks.workunit.client.1.smithi117.stdout:5/441: creat dd/df/d5e/d91/d36/f99 x:0 0 0 2024-03-22T06:15:22.804 INFO:tasks.workunit.client.1.smithi117.stdout:7/354: creat d8/d9/da/d2b/f6e x:0 0 0 2024-03-22T06:15:22.804 INFO:tasks.workunit.client.1.smithi117.stdout:7/355: read d8/d9/da/f28 [491898,89880] 0 2024-03-22T06:15:22.805 INFO:tasks.workunit.client.1.smithi117.stdout:1/431: dwrite d1/d30/d5d/d6f/f95 [0,4194304] 0 2024-03-22T06:15:22.806 INFO:tasks.workunit.client.1.smithi117.stdout:4/377: getdents d0/d4/d2a 0 2024-03-22T06:15:22.808 INFO:tasks.workunit.client.1.smithi117.stdout:5/442: mkdir dd/df/d5e/d91/d72/d33/d9a 0 2024-03-22T06:15:22.810 INFO:tasks.workunit.client.1.smithi117.stdout:7/356: creat d8/d9/d37/d45/d61/f6f x:0 0 0 2024-03-22T06:15:22.815 INFO:tasks.workunit.client.1.smithi117.stdout:4/378: creat d0/d51/f7e x:0 0 0 2024-03-22T06:15:22.816 INFO:tasks.workunit.client.1.smithi117.stdout:5/443: getdents dd/df/d1e 0 2024-03-22T06:15:22.821 INFO:tasks.workunit.client.1.smithi117.stdout:9/405: dwrite d4/d46/f55 [0,4194304] 0 2024-03-22T06:15:22.821 INFO:tasks.workunit.client.1.smithi117.stdout:9/406: dread - d4/d7/d10/d2f/f39 zero size 2024-03-22T06:15:22.821 INFO:tasks.workunit.client.1.smithi117.stdout:2/387: dwrite d2/d5/d13/d3f/d1d/d3a/d11/d4f/f62 [0,4194304] 0 2024-03-22T06:15:22.821 INFO:tasks.workunit.client.1.smithi117.stdout:4/379: creat d0/d19/f7f x:0 0 0 2024-03-22T06:15:22.821 INFO:tasks.workunit.client.1.smithi117.stdout:8/377: dwrite d1/d3a/d44/d9/f13 [4194304,4194304] 0 2024-03-22T06:15:22.822 INFO:tasks.workunit.client.1.smithi117.stdout:3/342: dwrite d9/d1a/d39/d4c/f44 [0,4194304] 0 2024-03-22T06:15:22.822 INFO:tasks.workunit.client.1.smithi117.stdout:3/343: creat d9/d1a/f83 x:0 0 0 2024-03-22T06:15:22.824 INFO:tasks.workunit.client.1.smithi117.stdout:1/432: dwrite d1/d22/d27/d4e/f9b [0,4194304] 0 2024-03-22T06:15:22.826 INFO:tasks.workunit.client.1.smithi117.stdout:6/444: dwrite d3/d40/d24/d2d/d30/f6d [0,4194304] 0 2024-03-22T06:15:22.826 INFO:tasks.workunit.client.1.smithi117.stdout:6/445: fdatasync d3/d40/d24/d2d/d53/d59/d74/d8b/f2f 0 2024-03-22T06:15:22.826 INFO:tasks.workunit.client.1.smithi117.stdout:6/446: stat d3/f4 0 2024-03-22T06:15:22.827 INFO:tasks.workunit.client.1.smithi117.stdout:9/407: rename d4/d7/d10 to d4/d8/d3e/d7a 0 2024-03-22T06:15:22.832 INFO:tasks.workunit.client.1.smithi117.stdout:3/344: dread d9/d1a/f34 [0,4194304] 0 2024-03-22T06:15:22.834 INFO:tasks.workunit.client.1.smithi117.stdout:2/388: mknod d2/d35/d4c/c73 0 2024-03-22T06:15:22.834 INFO:tasks.workunit.client.1.smithi117.stdout:2/389: chown d2/d5/d13/d3f/l58 169693487 1 2024-03-22T06:15:22.836 INFO:tasks.workunit.client.1.smithi117.stdout:6/447: read d3/d3d/d17/f33 [1202782,118907] 0 2024-03-22T06:15:22.837 INFO:tasks.workunit.client.1.smithi117.stdout:8/378: mknod d1/d3a/d44/d6a/c78 0 2024-03-22T06:15:22.838 INFO:tasks.workunit.client.1.smithi117.stdout:8/379: dread d1/d3a/d44/f3c [0,4194304] 0 2024-03-22T06:15:22.838 INFO:tasks.workunit.client.1.smithi117.stdout:8/380: fsync d1/d28/d66/f74 0 2024-03-22T06:15:22.847 INFO:tasks.workunit.client.1.smithi117.stdout:1/433: unlink d1/d22/d27/d4e/f9b 0 2024-03-22T06:15:22.848 INFO:tasks.workunit.client.1.smithi117.stdout:1/434: creat d1/d3c/d4d/d5b/f9d x:0 0 0 2024-03-22T06:15:22.848 INFO:tasks.workunit.client.1.smithi117.stdout:1/435: truncate d1/d1f/f8f 819076 0 2024-03-22T06:15:22.848 INFO:tasks.workunit.client.1.smithi117.stdout:7/357: dwrite d8/f16 [0,4194304] 0 2024-03-22T06:15:22.848 INFO:tasks.workunit.client.1.smithi117.stdout:6/448: dread d3/d3d/f43 [0,4194304] 0 2024-03-22T06:15:22.854 INFO:tasks.workunit.client.1.smithi117.stdout:9/408: creat d4/d8/d3e/d7a/d2f/f7b x:0 0 0 2024-03-22T06:15:22.855 INFO:tasks.workunit.client.1.smithi117.stdout:9/409: truncate d4/d7/d27/d5b/f6a 188571 0 2024-03-22T06:15:22.855 INFO:tasks.workunit.client.1.smithi117.stdout:9/410: fsync d4/d50/f6b 0 2024-03-22T06:15:22.855 INFO:tasks.workunit.client.1.smithi117.stdout:5/444: dwrite dd/d16/d1a/f59 [0,4194304] 0 2024-03-22T06:15:22.855 INFO:tasks.workunit.client.1.smithi117.stdout:5/445: truncate dd/df/d5e/d91/d72/d60/d6e/f85 813227 0 2024-03-22T06:15:22.855 INFO:tasks.workunit.client.1.smithi117.stdout:5/446: truncate dd/d16/f65 388246 0 2024-03-22T06:15:22.855 INFO:tasks.workunit.client.1.smithi117.stdout:3/345: creat d9/d1a/d39/d82/f84 x:0 0 0 2024-03-22T06:15:22.857 INFO:tasks.workunit.client.1.smithi117.stdout:2/390: symlink d2/d1b/l74 0 2024-03-22T06:15:22.859 INFO:tasks.workunit.client.1.smithi117.stdout:8/381: mknod d1/d28/d6b/c79 0 2024-03-22T06:15:22.860 INFO:tasks.workunit.client.1.smithi117.stdout:1/436: write d1/d1f/f75 [17121,86494] 0 2024-03-22T06:15:22.860 INFO:tasks.workunit.client.1.smithi117.stdout:4/380: dwrite d0/f3 [0,4194304] 0 2024-03-22T06:15:22.863 INFO:tasks.workunit.client.1.smithi117.stdout:7/358: mknod d8/d9/da/d10/d23/d48/c70 0 2024-03-22T06:15:22.866 INFO:tasks.workunit.client.1.smithi117.stdout:6/449: mknod d3/d40/d24/d2d/d53/c97 0 2024-03-22T06:15:22.866 INFO:tasks.workunit.client.1.smithi117.stdout:6/450: chown d3/d40/d24/d2d/d53/d59/d74 26626214 1 2024-03-22T06:15:22.866 INFO:tasks.workunit.client.1.smithi117.stdout:6/451: creat d3/d40/d24/d2d/d30/f98 x:0 0 0 2024-03-22T06:15:22.868 INFO:tasks.workunit.client.1.smithi117.stdout:5/447: dwrite dd/df/d5e/d91/d36/f25 [0,4194304] 0 2024-03-22T06:15:22.869 INFO:tasks.workunit.client.1.smithi117.stdout:9/411: mknod d4/d8/d3e/c7c 0 2024-03-22T06:15:22.869 INFO:tasks.workunit.client.1.smithi117.stdout:9/412: stat d4/d8/f67 0 2024-03-22T06:15:22.871 INFO:tasks.workunit.client.1.smithi117.stdout:3/346: unlink d9/d1a/f5a 0 2024-03-22T06:15:22.871 INFO:tasks.workunit.client.1.smithi117.stdout:3/347: write d9/f35 [1047079,36554] 0 2024-03-22T06:15:22.871 INFO:tasks.workunit.client.1.smithi117.stdout:3/348: write d9/d55/f59 [1270346,6232] 0 2024-03-22T06:15:22.872 INFO:tasks.workunit.client.1.smithi117.stdout:0/411: sync 2024-03-22T06:15:22.872 INFO:tasks.workunit.client.1.smithi117.stdout:0/412: chown d1/d10/l31 376 1 2024-03-22T06:15:22.873 INFO:tasks.workunit.client.1.smithi117.stdout:2/391: symlink d2/d5/l75 0 2024-03-22T06:15:22.873 INFO:tasks.workunit.client.1.smithi117.stdout:2/392: dread - d2/d5/d41/f6f zero size 2024-03-22T06:15:22.874 INFO:tasks.workunit.client.1.smithi117.stdout:8/382: link d1/d28/d66/f74 d1/d3a/d44/d16/f7a 0 2024-03-22T06:15:22.876 INFO:tasks.workunit.client.1.smithi117.stdout:1/437: creat d1/d30/d35/d58/f9e x:0 0 0 2024-03-22T06:15:22.876 INFO:tasks.workunit.client.1.smithi117.stdout:1/438: creat d1/d30/d5d/d6f/f9f x:0 0 0 2024-03-22T06:15:22.876 INFO:tasks.workunit.client.1.smithi117.stdout:1/439: write d1/d14/f6c [1902032,4257] 0 2024-03-22T06:15:22.876 INFO:tasks.workunit.client.1.smithi117.stdout:1/440: chown d1/d30/d35/d58/f9e 72 1 2024-03-22T06:15:22.876 INFO:tasks.workunit.client.1.smithi117.stdout:1/441: stat d1/d22/l3b 0 2024-03-22T06:15:22.876 INFO:tasks.workunit.client.1.smithi117.stdout:1/442: write d1/d3c/d4d/d5b/f91 [126426,84915] 0 2024-03-22T06:15:22.877 INFO:tasks.workunit.client.1.smithi117.stdout:4/381: chown d0/d19/d1c/l3a 14 1 2024-03-22T06:15:22.880 INFO:tasks.workunit.client.1.smithi117.stdout:7/359: unlink f5 0 2024-03-22T06:15:22.880 INFO:tasks.workunit.client.1.smithi117.stdout:7/360: stat d8/d9/d37/d45 0 2024-03-22T06:15:22.880 INFO:tasks.workunit.client.1.smithi117.stdout:7/361: getdents d8/d6c 0 2024-03-22T06:15:22.880 INFO:tasks.workunit.client.1.smithi117.stdout:7/362: write f7 [1171191,103388] 0 2024-03-22T06:15:22.880 INFO:tasks.workunit.client.1.smithi117.stdout:7/363: creat d8/d9/d37/d45/d61/f71 x:0 0 0 2024-03-22T06:15:22.880 INFO:tasks.workunit.client.1.smithi117.stdout:7/364: creat d8/d9/da/f72 x:0 0 0 2024-03-22T06:15:22.883 INFO:tasks.workunit.client.1.smithi117.stdout:5/448: mknod dd/df/d5e/d91/d72/d33/c9b 0 2024-03-22T06:15:22.883 INFO:tasks.workunit.client.1.smithi117.stdout:5/449: creat dd/df/d5e/f9c x:0 0 0 2024-03-22T06:15:22.883 INFO:tasks.workunit.client.1.smithi117.stdout:5/450: stat dd/d16/f48 0 2024-03-22T06:15:22.883 INFO:tasks.workunit.client.1.smithi117.stdout:5/451: chown dd/d16/d1a/d32/d73/c82 55636205 1 2024-03-22T06:15:22.886 INFO:tasks.workunit.client.1.smithi117.stdout:9/413: truncate d4/d8/f2c 1089753 0 2024-03-22T06:15:22.889 INFO:tasks.workunit.client.1.smithi117.stdout:0/413: mkdir d1/d3/d2b/d3e/d3f/d8b 0 2024-03-22T06:15:22.896 INFO:tasks.workunit.client.1.smithi117.stdout:7/365: dwrite d8/d9/da/d10/d23/d27/f3e [0,4194304] 0 2024-03-22T06:15:22.896 INFO:tasks.workunit.client.1.smithi117.stdout:7/366: chown d8/d9/c43 208682 1 2024-03-22T06:15:22.898 INFO:tasks.workunit.client.1.smithi117.stdout:2/393: mkdir d2/d5/d13/d3f/d1d/d3a/d71/d76 0 2024-03-22T06:15:22.900 INFO:tasks.workunit.client.1.smithi117.stdout:8/383: mknod d1/d28/d6b/c7b 0 2024-03-22T06:15:22.900 INFO:tasks.workunit.client.1.smithi117.stdout:8/384: write d1/d28/d66/f74 [307971,74568] 0 2024-03-22T06:15:22.900 INFO:tasks.workunit.client.1.smithi117.stdout:9/414: write d4/d50/f6b [684503,3432] 0 2024-03-22T06:15:22.900 INFO:tasks.workunit.client.1.smithi117.stdout:8/385: chown d1/d3a/d44/d16 95 1 2024-03-22T06:15:22.900 INFO:tasks.workunit.client.1.smithi117.stdout:8/386: chown d1/d3a 0 1 2024-03-22T06:15:22.900 INFO:tasks.workunit.client.1.smithi117.stdout:1/443: dwrite d1/d62/f67 [0,4194304] 0 2024-03-22T06:15:22.901 INFO:tasks.workunit.client.1.smithi117.stdout:1/444: chown d1/d1f/f8f 48778 1 2024-03-22T06:15:22.902 INFO:tasks.workunit.client.1.smithi117.stdout:4/382: symlink d0/d4/d2a/d2f/l80 0 2024-03-22T06:15:22.906 INFO:tasks.workunit.client.1.smithi117.stdout:5/452: getdents dd/df/d5e/d91/d72/d60 0 2024-03-22T06:15:22.908 INFO:tasks.workunit.client.1.smithi117.stdout:3/349: mknod d9/d1a/d39/d4c/d36/d41/c85 0 2024-03-22T06:15:22.908 INFO:tasks.workunit.client.1.smithi117.stdout:3/350: write d9/d1a/d39/d4c/f4e [3581137,109968] 0 2024-03-22T06:15:22.909 INFO:tasks.workunit.client.1.smithi117.stdout:0/414: mkdir d1/d10/d18/d8c 0 2024-03-22T06:15:22.916 INFO:tasks.workunit.client.1.smithi117.stdout:2/394: mkdir d2/d1b/d39/d77 0 2024-03-22T06:15:22.916 INFO:tasks.workunit.client.1.smithi117.stdout:2/395: dread - d2/d5/d41/f6f zero size 2024-03-22T06:15:22.916 INFO:tasks.workunit.client.1.smithi117.stdout:2/396: chown d2/f4a 42088994 1 2024-03-22T06:15:22.916 INFO:tasks.workunit.client.1.smithi117.stdout:2/397: creat d2/d5/d13/d3f/d1d/d3a/f78 x:0 0 0 2024-03-22T06:15:22.922 INFO:tasks.workunit.client.1.smithi117.stdout:4/383: symlink d0/d3f/l81 0 2024-03-22T06:15:22.922 INFO:tasks.workunit.client.1.smithi117.stdout:4/384: rename d0/d19/d34 to d0/d19/d34/d43/d82 22 2024-03-22T06:15:22.922 INFO:tasks.workunit.client.1.smithi117.stdout:4/385: chown d0/d19/d34/l7b 2583813 1 2024-03-22T06:15:22.924 INFO:tasks.workunit.client.1.smithi117.stdout:5/453: creat dd/d16/d86/f9d x:0 0 0 2024-03-22T06:15:22.927 INFO:tasks.workunit.client.1.smithi117.stdout:3/351: unlink d9/d1a/l3c 0 2024-03-22T06:15:22.927 INFO:tasks.workunit.client.1.smithi117.stdout:3/352: chown d9/f35 846 1 2024-03-22T06:15:22.927 INFO:tasks.workunit.client.1.smithi117.stdout:3/353: write d9/d1a/d3a/f7a [962417,123987] 0 2024-03-22T06:15:22.928 INFO:tasks.workunit.client.1.smithi117.stdout:0/415: creat d1/d3/d2b/d3e/de/d1f/f8d x:0 0 0 2024-03-22T06:15:22.929 INFO:tasks.workunit.client.1.smithi117.stdout:7/367: symlink d8/d9/da/d10/d23/d27/d39/l73 0 2024-03-22T06:15:22.929 INFO:tasks.workunit.client.1.smithi117.stdout:0/416: dread d1/d10/d5c/d40/f73 [0,4194304] 0 2024-03-22T06:15:22.934 INFO:tasks.workunit.client.1.smithi117.stdout:2/398: unlink d2/d5/d13/d3f/f4e 0 2024-03-22T06:15:22.935 INFO:tasks.workunit.client.1.smithi117.stdout:3/354: write d9/d1a/d39/d4c/d36/f43 [105230,24958] 0 2024-03-22T06:15:22.937 INFO:tasks.workunit.client.1.smithi117.stdout:1/445: rmdir d1/d62 39 2024-03-22T06:15:22.940 INFO:tasks.workunit.client.1.smithi117.stdout:0/417: creat d1/d3/d2b/d3e/de/d1f/d3b/d55/f8e x:0 0 0 2024-03-22T06:15:22.943 INFO:tasks.workunit.client.1.smithi117.stdout:2/399: dread d2/d5/d13/d57/f36 [0,4194304] 0 2024-03-22T06:15:22.945 INFO:tasks.workunit.client.1.smithi117.stdout:1/446: mknod d1/d62/ca0 0 2024-03-22T06:15:22.945 INFO:tasks.workunit.client.1.smithi117.stdout:1/447: readlink d1/l7e 0 2024-03-22T06:15:22.945 INFO:tasks.workunit.client.1.smithi117.stdout:1/448: fsync d1/d1f/f8f 0 2024-03-22T06:15:22.945 INFO:tasks.workunit.client.1.smithi117.stdout:1/449: creat d1/d30/d35/d58/fa1 x:0 0 0 2024-03-22T06:15:22.945 INFO:tasks.workunit.client.1.smithi117.stdout:1/450: fsync d1/d30/d44/f7b 0 2024-03-22T06:15:22.945 INFO:tasks.workunit.client.1.smithi117.stdout:1/451: truncate d1/d84/d77/f98 370862 0 2024-03-22T06:15:22.945 INFO:tasks.workunit.client.1.smithi117.stdout:0/418: mkdir d1/d10/d5c/d40/d69/d8f 0 2024-03-22T06:15:22.948 INFO:tasks.workunit.client.1.smithi117.stdout:2/400: rename d2/d5/d13/d3f/l58 to d2/d5/d13/d3f/l79 0 2024-03-22T06:15:22.950 INFO:tasks.workunit.client.1.smithi117.stdout:5/454: dwrite dd/df/d5e/d91/d36/f31 [0,4194304] 0 2024-03-22T06:15:22.950 INFO:tasks.workunit.client.1.smithi117.stdout:7/368: dwrite d8/d9/da/d10/d23/d27/d39/f3b [0,4194304] 0 2024-03-22T06:15:22.950 INFO:tasks.workunit.client.1.smithi117.stdout:5/455: fsync dd/df/d5e/d91/d36/f39 0 2024-03-22T06:15:22.950 INFO:tasks.workunit.client.1.smithi117.stdout:1/452: dread d1/d22/d27/f34 [0,4194304] 0 2024-03-22T06:15:22.950 INFO:tasks.workunit.client.1.smithi117.stdout:7/369: write d8/d9/da/d10/f22 [4747941,109853] 0 2024-03-22T06:15:22.951 INFO:tasks.workunit.client.1.smithi117.stdout:3/355: rmdir d9/d1a/d39 39 2024-03-22T06:15:22.951 INFO:tasks.workunit.client.1.smithi117.stdout:2/401: write d2/d5/f33 [94532,78845] 0 2024-03-22T06:15:22.951 INFO:tasks.workunit.client.1.smithi117.stdout:2/402: chown d2/d6e 11 1 2024-03-22T06:15:22.951 INFO:tasks.workunit.client.1.smithi117.stdout:2/403: chown d2/d5/d13/d3f/d1d/d3a/f78 32502335 1 2024-03-22T06:15:22.951 INFO:tasks.workunit.client.1.smithi117.stdout:2/404: readlink d2/d35/l50 0 2024-03-22T06:15:22.952 INFO:tasks.workunit.client.1.smithi117.stdout:2/405: chown d2/f69 736 1 2024-03-22T06:15:22.952 INFO:tasks.workunit.client.1.smithi117.stdout:2/406: creat d2/d5/d13/f7a x:0 0 0 2024-03-22T06:15:22.952 INFO:tasks.workunit.client.1.smithi117.stdout:2/407: fdatasync d2/d5/db/f72 0 2024-03-22T06:15:22.960 INFO:tasks.workunit.client.1.smithi117.stdout:0/419: stat d1/d10/d18/d26/d49/d7e/d39/c43 0 2024-03-22T06:15:22.961 INFO:tasks.workunit.client.1.smithi117.stdout:0/420: read d1/d10/d18/d26/d49/d7e/f47 [990467,19041] 0 2024-03-22T06:15:22.961 INFO:tasks.workunit.client.1.smithi117.stdout:5/456: creat dd/df/d95/f9e x:0 0 0 2024-03-22T06:15:22.970 INFO:tasks.workunit.client.1.smithi117.stdout:7/370: mkdir d8/d9/d37/d5f/d74 0 2024-03-22T06:15:22.970 INFO:tasks.workunit.client.1.smithi117.stdout:6/452: sync 2024-03-22T06:15:22.972 INFO:tasks.workunit.client.1.smithi117.stdout:3/356: mknod d9/d1a/d39/d82/c86 0 2024-03-22T06:15:22.974 INFO:tasks.workunit.client.1.smithi117.stdout:2/408: symlink d2/d5/d13/d3f/d1d/d3a/d71/d76/l7b 0 2024-03-22T06:15:22.976 INFO:tasks.workunit.client.1.smithi117.stdout:5/457: creat dd/df/d5e/d91/d72/d35/f9f x:0 0 0 2024-03-22T06:15:22.976 INFO:tasks.workunit.client.1.smithi117.stdout:5/458: read - dd/f8a zero size 2024-03-22T06:15:22.976 INFO:tasks.workunit.client.1.smithi117.stdout:5/459: write dd/d16/d5a/f97 [409377,69155] 0 2024-03-22T06:15:22.977 INFO:tasks.workunit.client.1.smithi117.stdout:0/421: dwrite d1/d3/d2b/d3e/de/f23 [0,4194304] 0 2024-03-22T06:15:22.977 INFO:tasks.workunit.client.1.smithi117.stdout:0/422: creat d1/d10/d18/d26/d49/d7e/d39/f90 x:0 0 0 2024-03-22T06:15:22.978 INFO:tasks.workunit.client.1.smithi117.stdout:6/453: mknod d3/d3d/c99 0 2024-03-22T06:15:22.978 INFO:tasks.workunit.client.1.smithi117.stdout:6/454: write d3/d40/d24/d2d/d53/d59/d74/d8b/f2f [1733930,33318] 0 2024-03-22T06:15:22.983 INFO:tasks.workunit.client.1.smithi117.stdout:2/409: mkdir d2/d1b/d39/d45/d7c 0 2024-03-22T06:15:22.983 INFO:tasks.workunit.client.1.smithi117.stdout:2/410: write d2/d5/f33 [2036792,118379] 0 2024-03-22T06:15:22.983 INFO:tasks.workunit.client.1.smithi117.stdout:2/411: truncate d2/d5/d13/d3f/f6b 767450 0 2024-03-22T06:15:22.983 INFO:tasks.workunit.client.1.smithi117.stdout:2/412: fsync d2/d5/db/f66 0 2024-03-22T06:15:22.983 INFO:tasks.workunit.client.1.smithi117.stdout:1/453: dread d1/f8 [0,4194304] 0 2024-03-22T06:15:22.984 INFO:tasks.workunit.client.1.smithi117.stdout:1/454: chown d1/d1f/f20 0 1 2024-03-22T06:15:22.985 INFO:tasks.workunit.client.1.smithi117.stdout:5/460: mkdir dd/df/d95/da0 0 2024-03-22T06:15:22.985 INFO:tasks.workunit.client.1.smithi117.stdout:5/461: chown dd/df/d5e/d91/l41 1 1 2024-03-22T06:15:22.997 INFO:tasks.workunit.client.1.smithi117.stdout:6/455: creat d3/d3d/d17/f9a x:0 0 0 2024-03-22T06:15:23.000 INFO:tasks.workunit.client.1.smithi117.stdout:1/455: rename d1/d1f/d32/l46 to d1/d62/d7c/la2 0 2024-03-22T06:15:23.001 INFO:tasks.workunit.client.1.smithi117.stdout:0/423: dwrite d1/f1c [0,4194304] 0 2024-03-22T06:15:23.001 INFO:tasks.workunit.client.1.smithi117.stdout:0/424: creat d1/d3/d2b/d3e/de/d1f/d3b/d55/f91 x:0 0 0 2024-03-22T06:15:23.001 INFO:tasks.workunit.client.1.smithi117.stdout:0/425: creat d1/d3/d2b/d3e/de/d1f/d3b/d55/f92 x:0 0 0 2024-03-22T06:15:23.002 INFO:tasks.workunit.client.1.smithi117.stdout:1/456: dread d1/d22/d27/f8e [0,4194304] 0 2024-03-22T06:15:23.002 INFO:tasks.workunit.client.1.smithi117.stdout:1/457: fsync d1/d30/d35/d58/f9e 0 2024-03-22T06:15:23.003 INFO:tasks.workunit.client.1.smithi117.stdout:3/357: dwrite d9/d1a/d39/f45 [0,4194304] 0 2024-03-22T06:15:23.003 INFO:tasks.workunit.client.1.smithi117.stdout:3/358: write d9/d1a/d39/d7d/f7e [18010,98229] 0 2024-03-22T06:15:23.010 INFO:tasks.workunit.client.1.smithi117.stdout:3/359: read f2 [248797,37549] 0 2024-03-22T06:15:23.011 INFO:tasks.workunit.client.1.smithi117.stdout:2/413: dwrite d2/d5/d13/d3f/d1d/f46 [4194304,4194304] 0 2024-03-22T06:15:23.014 INFO:tasks.workunit.client.1.smithi117.stdout:2/414: mkdir d2/d5/db/d2c/d7d 0 2024-03-22T06:15:23.014 INFO:tasks.workunit.client.1.smithi117.stdout:2/415: write d2/d5/d13/d3f/d1d/d3a/d11/f15 [580085,114906] 0 2024-03-22T06:15:23.020 INFO:tasks.workunit.client.1.smithi117.stdout:3/360: rmdir d9 39 2024-03-22T06:15:23.020 INFO:tasks.workunit.client.1.smithi117.stdout:3/361: fdatasync d9/d1a/d39/d4c/d36/f43 0 2024-03-22T06:15:23.020 INFO:tasks.workunit.client.1.smithi117.stdout:3/362: chown d9/d1a/l29 2 1 2024-03-22T06:15:23.021 INFO:tasks.workunit.client.1.smithi117.stdout:3/363: creat d9/d1a/d39/d4c/d37/d69/f87 x:0 0 0 2024-03-22T06:15:23.026 INFO:tasks.workunit.client.1.smithi117.stdout:3/364: creat d9/d1a/d39/d7d/f88 x:0 0 0 2024-03-22T06:15:23.027 INFO:tasks.workunit.client.1.smithi117.stdout:3/365: write d9/f63 [5332892,17463] 0 2024-03-22T06:15:23.027 INFO:tasks.workunit.client.1.smithi117.stdout:3/366: write d9/d55/f59 [1379080,68182] 0 2024-03-22T06:15:23.027 INFO:tasks.workunit.client.1.smithi117.stdout:3/367: write f2 [6605,5018] 0 2024-03-22T06:15:23.027 INFO:tasks.workunit.client.1.smithi117.stdout:3/368: creat d9/d1a/d39/d4c/d37/f89 x:0 0 0 2024-03-22T06:15:23.027 INFO:tasks.workunit.client.1.smithi117.stdout:8/387: sync 2024-03-22T06:15:23.027 INFO:tasks.workunit.client.1.smithi117.stdout:9/415: sync 2024-03-22T06:15:23.027 INFO:tasks.workunit.client.1.smithi117.stdout:9/416: truncate d4/d8/d3e/d7a/d3b/f57 4588421 0 2024-03-22T06:15:23.027 INFO:tasks.workunit.client.1.smithi117.stdout:9/417: fdatasync d4/d8/d3e/d7a/d2f/f39 0 2024-03-22T06:15:23.027 INFO:tasks.workunit.client.1.smithi117.stdout:9/418: stat d4/f21 0 2024-03-22T06:15:23.028 INFO:tasks.workunit.client.1.smithi117.stdout:2/416: read d2/d5/db/d2c/f42 [131340,9661] 0 2024-03-22T06:15:23.028 INFO:tasks.workunit.client.1.smithi117.stdout:2/417: fsync d2/f4a 0 2024-03-22T06:15:23.028 INFO:tasks.workunit.client.1.smithi117.stdout:2/418: getdents d2/d1b/d39/d77 0 2024-03-22T06:15:23.028 INFO:tasks.workunit.client.1.smithi117.stdout:0/426: dwrite d1/d10/d18/d26/d49/d7e/d39/f52 [0,4194304] 0 2024-03-22T06:15:23.028 INFO:tasks.workunit.client.1.smithi117.stdout:0/427: write d1/d3/d2b/d3e/ff [828467,8175] 0 2024-03-22T06:15:23.029 INFO:tasks.workunit.client.1.smithi117.stdout:1/458: dwrite d1/d30/d35/f1a [0,4194304] 0 2024-03-22T06:15:23.030 INFO:tasks.workunit.client.1.smithi117.stdout:9/419: creat d4/d7/d27/f7d x:0 0 0 2024-03-22T06:15:23.030 INFO:tasks.workunit.client.1.smithi117.stdout:9/420: read d4/d8/f1a [7191585,58541] 0 2024-03-22T06:15:23.035 INFO:tasks.workunit.client.1.smithi117.stdout:2/419: symlink d2/d1b/l7e 0 2024-03-22T06:15:23.035 INFO:tasks.workunit.client.1.smithi117.stdout:2/420: chown d2/l12 26600 1 2024-03-22T06:15:23.036 INFO:tasks.workunit.client.1.smithi117.stdout:0/428: creat d1/d10/d18/d26/d53/f93 x:0 0 0 2024-03-22T06:15:23.036 INFO:tasks.workunit.client.1.smithi117.stdout:6/456: dwrite d3/d40/d24/f4e [0,4194304] 0 2024-03-22T06:15:23.036 INFO:tasks.workunit.client.1.smithi117.stdout:0/429: write d1/d10/d18/d26/d49/d7e/d39/f88 [917926,62940] 0 2024-03-22T06:15:23.036 INFO:tasks.workunit.client.1.smithi117.stdout:6/457: creat d3/d61/f9b x:0 0 0 2024-03-22T06:15:23.037 INFO:tasks.workunit.client.1.smithi117.stdout:8/388: dread d1/d19/f2d [0,4194304] 0 2024-03-22T06:15:23.038 INFO:tasks.workunit.client.1.smithi117.stdout:1/459: creat d1/d30/fa3 x:0 0 0 2024-03-22T06:15:23.038 INFO:tasks.workunit.client.1.smithi117.stdout:1/460: write d1/f7 [931890,84907] 0 2024-03-22T06:15:23.038 INFO:tasks.workunit.client.1.smithi117.stdout:1/461: write d1/d30/fa3 [857133,13490] 0 2024-03-22T06:15:23.040 INFO:tasks.workunit.client.1.smithi117.stdout:4/386: sync 2024-03-22T06:15:23.040 INFO:tasks.workunit.client.1.smithi117.stdout:4/387: write d0/d19/f4a [368749,11218] 0 2024-03-22T06:15:23.040 INFO:tasks.workunit.client.1.smithi117.stdout:4/388: creat d0/f83 x:0 0 0 2024-03-22T06:15:23.041 INFO:tasks.workunit.client.1.smithi117.stdout:1/462: write d1/f8 [3338011,118408] 0 2024-03-22T06:15:23.041 INFO:tasks.workunit.client.1.smithi117.stdout:1/463: readlink d1/d62/d7c/la2 0 2024-03-22T06:15:23.041 INFO:tasks.workunit.client.1.smithi117.stdout:1/464: fsync d1/d30/d50/f63 0 2024-03-22T06:15:23.041 INFO:tasks.workunit.client.1.smithi117.stdout:9/421: dread d4/d8/d3e/d7a/d3b/f57 [0,4194304] 0 2024-03-22T06:15:23.041 INFO:tasks.workunit.client.1.smithi117.stdout:2/421: rename d2/d5/d13/d3f/f1f to d2/d5/f7f 0 2024-03-22T06:15:23.041 INFO:tasks.workunit.client.1.smithi117.stdout:2/422: stat d2/d5/d13/d3f/d1d/d3a/d71/d76/l7b 0 2024-03-22T06:15:23.052 INFO:tasks.workunit.client.1.smithi117.stdout:1/465: mkdir d1/d1f/d32/d33/da4 0 2024-03-22T06:15:23.053 INFO:tasks.workunit.client.1.smithi117.stdout:9/422: creat d4/d8/d3e/f7e x:0 0 0 2024-03-22T06:15:23.054 INFO:tasks.workunit.client.1.smithi117.stdout:3/369: dwrite d9/d55/f81 [0,4194304] 0 2024-03-22T06:15:23.057 INFO:tasks.workunit.client.1.smithi117.stdout:9/423: creat d4/d7/f7f x:0 0 0 2024-03-22T06:15:23.062 INFO:tasks.workunit.client.1.smithi117.stdout:1/466: write d1/d1f/f3e [4003048,35105] 0 2024-03-22T06:15:23.062 INFO:tasks.workunit.client.1.smithi117.stdout:1/467: chown d1/d22/d27 7601512 1 2024-03-22T06:15:23.070 INFO:tasks.workunit.client.1.smithi117.stdout:8/389: dread d1/f1e [0,4194304] 0 2024-03-22T06:15:23.071 INFO:tasks.workunit.client.1.smithi117.stdout:8/390: chown d1/d3a/d44/d11/c35 4012028 1 2024-03-22T06:15:23.071 INFO:tasks.workunit.client.1.smithi117.stdout:8/391: dread d1/d3a/d44/f3c [0,4194304] 0 2024-03-22T06:15:23.085 INFO:tasks.workunit.client.1.smithi117.stdout:1/468: mknod d1/d14/ca5 0 2024-03-22T06:15:23.085 INFO:tasks.workunit.client.1.smithi117.stdout:4/389: dwrite d0/d19/d34/f4f [0,4194304] 0 2024-03-22T06:15:23.085 INFO:tasks.workunit.client.1.smithi117.stdout:8/392: mknod d1/d3a/d44/d9/c7c 0 2024-03-22T06:15:23.086 INFO:tasks.workunit.client.1.smithi117.stdout:2/423: dwrite d2/d35/d38/f5e [4194304,4194304] 0 2024-03-22T06:15:23.087 INFO:tasks.workunit.client.1.smithi117.stdout:6/458: dwrite d3/f3f [0,4194304] 0 2024-03-22T06:15:23.087 INFO:tasks.workunit.client.1.smithi117.stdout:9/424: dwrite d4/d7/d27/f60 [0,4194304] 0 2024-03-22T06:15:23.088 INFO:tasks.workunit.client.1.smithi117.stdout:1/469: symlink d1/d22/d27/d4e/d92/la6 0 2024-03-22T06:15:23.088 INFO:tasks.workunit.client.1.smithi117.stdout:0/430: dwrite d1/d3/d2b/d3e/d3f/f45 [0,4194304] 0 2024-03-22T06:15:23.088 INFO:tasks.workunit.client.1.smithi117.stdout:0/431: fsync d1/d3/d2b/d3e/f60 0 2024-03-22T06:15:23.092 INFO:tasks.workunit.client.1.smithi117.stdout:8/393: unlink d1/f34 0 2024-03-22T06:15:23.101 INFO:tasks.workunit.client.1.smithi117.stdout:9/425: dread d4/d8/f22 [0,4194304] 0 2024-03-22T06:15:23.101 INFO:tasks.workunit.client.1.smithi117.stdout:9/426: fdatasync d4/d50/f6b 0 2024-03-22T06:15:23.101 INFO:tasks.workunit.client.1.smithi117.stdout:9/427: write d4/d8/f67 [1281946,89297] 0 2024-03-22T06:15:23.101 INFO:tasks.workunit.client.1.smithi117.stdout:9/428: dread - d4/d8/d3e/d7a/d2f/f39 zero size 2024-03-22T06:15:23.104 INFO:tasks.workunit.client.1.smithi117.stdout:0/432: dread d1/d3/d2b/d3e/de/f51 [0,4194304] 0 2024-03-22T06:15:23.104 INFO:tasks.workunit.client.1.smithi117.stdout:6/459: creat d3/d40/d46/d84/f9c x:0 0 0 2024-03-22T06:15:23.104 INFO:tasks.workunit.client.1.smithi117.stdout:6/460: chown d3/d40/d24/d2d/d53/c65 74807 1 2024-03-22T06:15:23.104 INFO:tasks.workunit.client.1.smithi117.stdout:6/461: fsync d3/d3d/d17/d60/f8f 0 2024-03-22T06:15:23.112 INFO:tasks.workunit.client.1.smithi117.stdout:4/390: rename d0/d19/d34/d43/f57 to d0/d19/d1c/f84 0 2024-03-22T06:15:23.113 INFO:tasks.workunit.client.1.smithi117.stdout:9/429: symlink d4/d8/d3e/d7a/d3b/l80 0 2024-03-22T06:15:23.113 INFO:tasks.workunit.client.1.smithi117.stdout:9/430: write d4/d8/d3e/d7a/d2f/f39 [936356,64817] 0 2024-03-22T06:15:23.115 INFO:tasks.workunit.client.1.smithi117.stdout:6/462: mkdir d3/d40/d46/d84/d49/d79/d9d 0 2024-03-22T06:15:23.115 INFO:tasks.workunit.client.1.smithi117.stdout:6/463: readlink d3/d40/d24/d2d/d53/d75/l8c 0 2024-03-22T06:15:23.116 INFO:tasks.workunit.client.1.smithi117.stdout:4/391: creat d0/d4/d2a/d5b/f85 x:0 0 0 2024-03-22T06:15:23.117 INFO:tasks.workunit.client.1.smithi117.stdout:6/464: mknod d3/d40/d46/d84/d49/d79/d9d/c9e 0 2024-03-22T06:15:23.118 INFO:tasks.workunit.client.1.smithi117.stdout:0/433: write d1/d3/d2b/d3e/d3f/f45 [478514,23313] 0 2024-03-22T06:15:23.119 INFO:tasks.workunit.client.1.smithi117.stdout:4/392: symlink d0/d51/l86 0 2024-03-22T06:15:23.121 INFO:tasks.workunit.client.1.smithi117.stdout:2/424: dwrite d2/f26 [0,4194304] 0 2024-03-22T06:15:23.121 INFO:tasks.workunit.client.1.smithi117.stdout:1/470: dwrite d1/d22/d27/f34 [0,4194304] 0 2024-03-22T06:15:23.121 INFO:tasks.workunit.client.1.smithi117.stdout:8/394: dwrite f0 [0,4194304] 0 2024-03-22T06:15:23.122 INFO:tasks.workunit.client.1.smithi117.stdout:6/465: rmdir d3/d40/d83 0 2024-03-22T06:15:23.122 INFO:tasks.workunit.client.1.smithi117.stdout:0/434: creat d1/d10/d18/d26/d49/d7e/d39/d6a/f94 x:0 0 0 2024-03-22T06:15:23.122 INFO:tasks.workunit.client.1.smithi117.stdout:0/435: write d1/d10/d18/d26/d49/d7e/d39/d6a/f94 [168619,14946] 0 2024-03-22T06:15:23.125 INFO:tasks.workunit.client.1.smithi117.stdout:4/393: link d0/d4/c29 d0/d19/d34/d43/d63/d6f/d5f/c87 0 2024-03-22T06:15:23.128 INFO:tasks.workunit.client.1.smithi117.stdout:1/471: mknod d1/d62/ca7 0 2024-03-22T06:15:23.129 INFO:tasks.workunit.client.1.smithi117.stdout:9/431: dwrite d4/d8/d3e/d7a/d3b/f57 [4194304,4194304] 0 2024-03-22T06:15:23.131 INFO:tasks.workunit.client.1.smithi117.stdout:7/371: sync 2024-03-22T06:15:23.132 INFO:tasks.workunit.client.1.smithi117.stdout:5/462: sync 2024-03-22T06:15:23.132 INFO:tasks.workunit.client.1.smithi117.stdout:5/463: chown dd/d16/d86/f9d 16739889 1 2024-03-22T06:15:23.137 INFO:tasks.workunit.client.1.smithi117.stdout:8/395: creat d1/d28/f7d x:0 0 0 2024-03-22T06:15:23.138 INFO:tasks.workunit.client.1.smithi117.stdout:6/466: getdents d3/d40/d24/d2d/d53/d59/d74/d28 0 2024-03-22T06:15:23.138 INFO:tasks.workunit.client.1.smithi117.stdout:6/467: chown d3/d3d/f43 23328778 1 2024-03-22T06:15:23.139 INFO:tasks.workunit.client.1.smithi117.stdout:1/472: dread d1/d62/f67 [0,4194304] 0 2024-03-22T06:15:23.141 INFO:tasks.workunit.client.1.smithi117.stdout:0/436: unlink d1/d3/d2b/d3e/de/c54 0 2024-03-22T06:15:23.141 INFO:tasks.workunit.client.1.smithi117.stdout:0/437: chown d1/d3/d2b/d3e/f60 20970 1 2024-03-22T06:15:23.141 INFO:tasks.workunit.client.1.smithi117.stdout:0/438: read d1/d3/d2b/d3e/ff [566372,95917] 0 2024-03-22T06:15:23.141 INFO:tasks.workunit.client.1.smithi117.stdout:6/468: dread d3/d40/d24/d2d/d53/d59/d74/d8b/f2f [0,4194304] 0 2024-03-22T06:15:23.144 INFO:tasks.workunit.client.1.smithi117.stdout:2/425: dwrite d2/d5/d13/d57/f47 [0,4194304] 0 2024-03-22T06:15:23.147 INFO:tasks.workunit.client.1.smithi117.stdout:9/432: mknod d4/d8/d3e/d7a/c81 0 2024-03-22T06:15:23.147 INFO:tasks.workunit.client.1.smithi117.stdout:9/433: stat d4/d8/d3e/d7a/d3b/l80 0 2024-03-22T06:15:23.150 INFO:tasks.workunit.client.1.smithi117.stdout:7/372: link d8/d9/da/d10/d23/f38 d8/d9/da/d10/d23/d41/f75 0 2024-03-22T06:15:23.151 INFO:tasks.workunit.client.1.smithi117.stdout:1/473: write d1/d22/d27/d4e/f89 [1647319,89463] 0 2024-03-22T06:15:23.155 INFO:tasks.workunit.client.1.smithi117.stdout:8/396: rename d1/d19/d38/l45 to d1/d3a/d44/d6a/l7e 0 2024-03-22T06:15:23.155 INFO:tasks.workunit.client.1.smithi117.stdout:2/426: dread d2/f4a [0,4194304] 0 2024-03-22T06:15:23.156 INFO:tasks.workunit.client.1.smithi117.stdout:0/439: dwrite d1/d10/f11 [8388608,4194304] 0 2024-03-22T06:15:23.157 INFO:tasks.workunit.client.1.smithi117.stdout:4/394: link d0/d4/d2a/d2f/l80 d0/d19/l88 0 2024-03-22T06:15:23.158 INFO:tasks.workunit.client.1.smithi117.stdout:9/434: mkdir d4/d8/d3e/d82 0 2024-03-22T06:15:23.159 INFO:tasks.workunit.client.1.smithi117.stdout:7/373: mknod d8/d9/da/d10/d23/d27/d56/c76 0 2024-03-22T06:15:23.163 INFO:tasks.workunit.client.1.smithi117.stdout:5/464: getdents dd/d16/d86 0 2024-03-22T06:15:23.165 INFO:tasks.workunit.client.1.smithi117.stdout:0/440: dread d1/d10/d18/d26/d49/d7e/d39/f52 [0,4194304] 0 2024-03-22T06:15:23.165 INFO:tasks.workunit.client.1.smithi117.stdout:0/441: creat d1/d10/d18/f95 x:0 0 0 2024-03-22T06:15:23.167 INFO:tasks.workunit.client.1.smithi117.stdout:4/395: dread d0/f25 [0,4194304] 0 2024-03-22T06:15:23.168 INFO:tasks.workunit.client.1.smithi117.stdout:6/469: rename d3/d40/d24/d2d/d53/c65 to d3/d40/d24/d2d/d53/d59/d74/da/de/c9f 0 2024-03-22T06:15:23.168 INFO:tasks.workunit.client.1.smithi117.stdout:6/470: read - d3/d40/d46/f6b zero size 2024-03-22T06:15:23.168 INFO:tasks.workunit.client.1.smithi117.stdout:1/474: dwrite d1/f13 [0,4194304] 0 2024-03-22T06:15:23.170 INFO:tasks.workunit.client.1.smithi117.stdout:8/397: unlink d1/d28/d6b/c79 0 2024-03-22T06:15:23.172 INFO:tasks.workunit.client.1.smithi117.stdout:2/427: creat d2/d1b/d39/d77/f80 x:0 0 0 2024-03-22T06:15:23.172 INFO:tasks.workunit.client.1.smithi117.stdout:2/428: dread - d2/d5/d13/d3f/d1d/f6d zero size 2024-03-22T06:15:23.172 INFO:tasks.workunit.client.1.smithi117.stdout:2/429: read - d2/d1b/d39/d77/f80 zero size 2024-03-22T06:15:23.173 INFO:tasks.workunit.client.1.smithi117.stdout:5/465: mknod dd/df/d5e/d91/d72/d33/ca1 0 2024-03-22T06:15:23.177 INFO:tasks.workunit.client.1.smithi117.stdout:0/442: symlink d1/d3/d2b/d3e/de/d1f/d3b/l96 0 2024-03-22T06:15:23.177 INFO:tasks.workunit.client.1.smithi117.stdout:0/443: fdatasync d1/d10/d18/f95 0 2024-03-22T06:15:23.180 INFO:tasks.workunit.client.1.smithi117.stdout:4/396: creat d0/d19/d34/d43/d63/f89 x:0 0 0 2024-03-22T06:15:23.180 INFO:tasks.workunit.client.1.smithi117.stdout:6/471: dread d3/d40/d24/d2d/d53/d59/d74/da/f29 [0,4194304] 0 2024-03-22T06:15:23.180 INFO:tasks.workunit.client.1.smithi117.stdout:0/444: dread d1/d3/d2b/d3e/de/d1f/d3b/f46 [0,4194304] 0 2024-03-22T06:15:23.180 INFO:tasks.workunit.client.1.smithi117.stdout:0/445: chown d1/d3/l20 11 1 2024-03-22T06:15:23.180 INFO:tasks.workunit.client.1.smithi117.stdout:0/446: truncate d1/d3/d2b/d3e/f60 1450766 0 2024-03-22T06:15:23.181 INFO:tasks.workunit.client.1.smithi117.stdout:3/370: sync 2024-03-22T06:15:23.181 INFO:tasks.workunit.client.1.smithi117.stdout:9/435: dwrite d4/d46/f79 [0,4194304] 0 2024-03-22T06:15:23.182 INFO:tasks.workunit.client.1.smithi117.stdout:7/374: rename d8/d9/da/d10/d23/d48/l59 to d8/d50/d66/l77 0 2024-03-22T06:15:23.187 INFO:tasks.workunit.client.1.smithi117.stdout:2/430: mknod d2/d5/db/c81 0 2024-03-22T06:15:23.189 INFO:tasks.workunit.client.1.smithi117.stdout:5/466: mkdir dd/df/d5e/d91/d72/d35/da2 0 2024-03-22T06:15:23.189 INFO:tasks.workunit.client.1.smithi117.stdout:5/467: fsync dd/df/d5e/d91/d72/f45 0 2024-03-22T06:15:23.189 INFO:tasks.workunit.client.1.smithi117.stdout:1/475: dread d1/d30/d35/f64 [0,4194304] 0 2024-03-22T06:15:23.189 INFO:tasks.workunit.client.1.smithi117.stdout:1/476: creat d1/d30/fa8 x:0 0 0 2024-03-22T06:15:23.191 INFO:tasks.workunit.client.1.smithi117.stdout:3/371: mkdir d9/d1a/d39/d8a 0 2024-03-22T06:15:23.191 INFO:tasks.workunit.client.1.smithi117.stdout:3/372: chown d9/f11 27779218 1 2024-03-22T06:15:23.199 INFO:tasks.workunit.client.1.smithi117.stdout:9/436: rmdir d4/d46/d76 0 2024-03-22T06:15:23.199 INFO:tasks.workunit.client.1.smithi117.stdout:9/437: chown d4/d7/c20 20694 1 2024-03-22T06:15:23.199 INFO:tasks.workunit.client.1.smithi117.stdout:9/438: write d4/d8/f22 [4681328,60617] 0 2024-03-22T06:15:23.203 INFO:tasks.workunit.client.1.smithi117.stdout:7/375: getdents d8/d9/da/d10/d23/d27/d39 0 2024-03-22T06:15:23.203 INFO:tasks.workunit.client.1.smithi117.stdout:4/397: dwrite d0/d4/d2a/d2f/f56 [0,4194304] 0 2024-03-22T06:15:23.204 INFO:tasks.workunit.client.1.smithi117.stdout:3/373: dread d9/f18 [0,4194304] 0 2024-03-22T06:15:23.205 INFO:tasks.workunit.client.1.smithi117.stdout:8/398: dwrite d1/d19/f43 [0,4194304] 0 2024-03-22T06:15:23.206 INFO:tasks.workunit.client.1.smithi117.stdout:6/472: rename d3/d40/d24/d2d/d53/d59/d74/da/de/f6f to d3/d40/d24/d2d/d53/d75/fa0 0 2024-03-22T06:15:23.206 INFO:tasks.workunit.client.1.smithi117.stdout:5/468: rename dd/df/d5e/d91/d72/d35/d81 to dd/df/d5e/d91/d72/d35/d81/da3 22 2024-03-22T06:15:23.206 INFO:tasks.workunit.client.1.smithi117.stdout:3/374: write d9/d1a/d3a/f7a [627396,59712] 0 2024-03-22T06:15:23.214 INFO:tasks.workunit.client.1.smithi117.stdout:1/477: mkdir d1/d30/d44/d9c/da9 0 2024-03-22T06:15:23.214 INFO:tasks.workunit.client.1.smithi117.stdout:1/478: dread - d1/d22/d27/d4e/f6a zero size 2024-03-22T06:15:23.220 INFO:tasks.workunit.client.1.smithi117.stdout:8/399: getdents d1/d3a/d44/d16 0 2024-03-22T06:15:23.226 INFO:tasks.workunit.client.1.smithi117.stdout:9/439: dwrite d4/d7/f2a [0,4194304] 0 2024-03-22T06:15:23.227 INFO:tasks.workunit.client.1.smithi117.stdout:9/440: readlink d4/d8/lb 0 2024-03-22T06:15:23.227 INFO:tasks.workunit.client.1.smithi117.stdout:9/441: write d4/d8/d3e/d7a/d2f/f6f [754038,48307] 0 2024-03-22T06:15:23.227 INFO:tasks.workunit.client.1.smithi117.stdout:6/473: dwrite d3/d3d/f2e [0,4194304] 0 2024-03-22T06:15:23.227 INFO:tasks.workunit.client.1.smithi117.stdout:6/474: write d3/d40/d24/d2d/d53/d59/d74/d8b/f7a [5019814,98802] 0 2024-03-22T06:15:23.234 INFO:tasks.workunit.client.1.smithi117.stdout:8/400: write d1/d3a/d44/f1d [3065046,19431] 0 2024-03-22T06:15:23.234 INFO:tasks.workunit.client.1.smithi117.stdout:8/401: read f0 [3488198,23688] 0 2024-03-22T06:15:23.236 INFO:tasks.workunit.client.1.smithi117.stdout:7/376: rename d8/d9/da/d10/d23/d27/d56/c76 to d8/d6c/c78 0 2024-03-22T06:15:23.238 INFO:tasks.workunit.client.1.smithi117.stdout:5/469: creat dd/df/d5e/d91/d90/fa4 x:0 0 0 2024-03-22T06:15:23.239 INFO:tasks.workunit.client.1.smithi117.stdout:3/375: symlink d9/d1a/d39/d4c/l8b 0 2024-03-22T06:15:23.240 INFO:tasks.workunit.client.1.smithi117.stdout:9/442: dwrite d4/d8/d3e/f7e [0,4194304] 0 2024-03-22T06:15:23.248 INFO:tasks.workunit.client.1.smithi117.stdout:6/475: truncate d3/d40/d46/d84/d49/f5b 3581095 0 2024-03-22T06:15:23.249 INFO:tasks.workunit.client.1.smithi117.stdout:6/476: creat d3/d40/d46/fa1 x:0 0 0 2024-03-22T06:15:23.249 INFO:tasks.workunit.client.1.smithi117.stdout:3/376: write d9/d1a/d39/d4c/f42 [4180096,74267] 0 2024-03-22T06:15:23.249 INFO:tasks.workunit.client.1.smithi117.stdout:3/377: creat d9/d1a/d39/d4c/d37/f8c x:0 0 0 2024-03-22T06:15:23.251 INFO:tasks.workunit.client.1.smithi117.stdout:4/398: rename d0/d4/d2a/d2f/l80 to d0/d19/d28/d47/l8a 0 2024-03-22T06:15:23.252 INFO:tasks.workunit.client.1.smithi117.stdout:5/470: dwrite dd/df/d5e/d91/d36/f47 [0,4194304] 0 2024-03-22T06:15:23.252 INFO:tasks.workunit.client.1.smithi117.stdout:5/471: creat dd/df/d5e/d91/d72/d35/fa5 x:0 0 0 2024-03-22T06:15:23.252 INFO:tasks.workunit.client.1.smithi117.stdout:7/377: creat d8/d9/d37/d5f/f79 x:0 0 0 2024-03-22T06:15:23.252 INFO:tasks.workunit.client.1.smithi117.stdout:7/378: chown d8/d9/da/f28 887 1 2024-03-22T06:15:23.266 INFO:tasks.workunit.client.1.smithi117.stdout:7/379: dread d8/d9/da/d10/d23/d27/d39/f4a [0,4194304] 0 2024-03-22T06:15:23.266 INFO:tasks.workunit.client.1.smithi117.stdout:7/380: write d8/d9/da/d10/d23/d27/d39/f5d [4256248,93852] 0 2024-03-22T06:15:23.266 INFO:tasks.workunit.client.1.smithi117.stdout:7/381: write d8/d9/da/d2b/f6e [956605,17345] 0 2024-03-22T06:15:23.266 INFO:tasks.workunit.client.1.smithi117.stdout:7/382: write d8/d9/d37/d45/d61/f6f [372850,22038] 0 2024-03-22T06:15:23.266 INFO:tasks.workunit.client.1.smithi117.stdout:7/383: chown l1 12095235 1 2024-03-22T06:15:23.267 INFO:tasks.workunit.client.1.smithi117.stdout:7/384: fsync d8/d9/da/d10/d23/d27/d39/f4a 0 2024-03-22T06:15:23.268 INFO:tasks.workunit.client.1.smithi117.stdout:9/443: mknod d4/d8/d3e/d7a/c83 0 2024-03-22T06:15:23.269 INFO:tasks.workunit.client.1.smithi117.stdout:1/479: rmdir d1/d1f/d32 39 2024-03-22T06:15:23.269 INFO:tasks.workunit.client.1.smithi117.stdout:1/480: stat d1/d3c/d4d/d5b/f96 0 2024-03-22T06:15:23.269 INFO:tasks.workunit.client.1.smithi117.stdout:1/481: write d1/d1f/f3e [1477268,61093] 0 2024-03-22T06:15:23.269 INFO:tasks.workunit.client.1.smithi117.stdout:1/482: chown d1/d1f/f3e 0 1 2024-03-22T06:15:23.273 INFO:tasks.workunit.client.1.smithi117.stdout:0/447: sync 2024-03-22T06:15:23.273 INFO:tasks.workunit.client.1.smithi117.stdout:2/431: sync 2024-03-22T06:15:23.277 INFO:tasks.workunit.client.1.smithi117.stdout:0/448: dread d1/d3/d2b/d3e/d3f/f78 [0,4194304] 0 2024-03-22T06:15:23.277 INFO:tasks.workunit.client.1.smithi117.stdout:3/378: symlink d9/l8d 0 2024-03-22T06:15:23.288 INFO:tasks.workunit.client.1.smithi117.stdout:4/399: dwrite d0/f1d [0,4194304] 0 2024-03-22T06:15:23.288 INFO:tasks.workunit.client.1.smithi117.stdout:4/400: fdatasync d0/d19/f4a 0 2024-03-22T06:15:23.288 INFO:tasks.workunit.client.1.smithi117.stdout:9/444: dwrite d4/d7/f26 [0,4194304] 0 2024-03-22T06:15:23.290 INFO:tasks.workunit.client.1.smithi117.stdout:2/432: read d2/d5/d13/f5d [2510220,64960] 0 2024-03-22T06:15:23.292 INFO:tasks.workunit.client.1.smithi117.stdout:5/472: truncate dd/d16/d1a/f37 1168449 0 2024-03-22T06:15:23.294 INFO:tasks.workunit.client.1.smithi117.stdout:7/385: truncate d8/d9/da/d10/d23/d27/d39/f3b 1600724 0 2024-03-22T06:15:23.298 INFO:tasks.workunit.client.1.smithi117.stdout:1/483: symlink d1/d62/d7c/laa 0 2024-03-22T06:15:23.298 INFO:tasks.workunit.client.1.smithi117.stdout:1/484: read - d1/d30/d5d/d6f/f9f zero size 2024-03-22T06:15:23.300 INFO:tasks.workunit.client.1.smithi117.stdout:0/449: mknod d1/d3/d2b/d3e/c97 0 2024-03-22T06:15:23.300 INFO:tasks.workunit.client.1.smithi117.stdout:3/379: dwrite d9/d1a/d39/d7d/f88 [0,4194304] 0 2024-03-22T06:15:23.301 INFO:tasks.workunit.client.1.smithi117.stdout:0/450: dread d1/d10/d18/d26/d49/d7e/d39/d6a/f94 [0,4194304] 0 2024-03-22T06:15:23.305 INFO:tasks.workunit.client.1.smithi117.stdout:3/380: dread d9/f11 [0,4194304] 0 2024-03-22T06:15:23.306 INFO:tasks.workunit.client.1.smithi117.stdout:3/381: chown d9/d1a/d39/d4c/d36/d41 510780837 1 2024-03-22T06:15:23.306 INFO:tasks.workunit.client.1.smithi117.stdout:3/382: write d9/d66/f6d [595641,125619] 0 2024-03-22T06:15:23.306 INFO:tasks.workunit.client.1.smithi117.stdout:3/383: dread - d9/d1a/d39/d4c/d37/f89 zero size 2024-03-22T06:15:23.306 INFO:tasks.workunit.client.1.smithi117.stdout:3/384: stat d9/d1a/d39/d4c/d36/f50 0 2024-03-22T06:15:23.306 INFO:tasks.workunit.client.1.smithi117.stdout:3/385: stat d9/d1a/d39/d4c/d36/f43 0 2024-03-22T06:15:23.308 INFO:tasks.workunit.client.1.smithi117.stdout:4/401: mknod d0/c8b 0 2024-03-22T06:15:23.312 INFO:tasks.workunit.client.1.smithi117.stdout:9/445: dwrite d4/f21 [0,4194304] 0 2024-03-22T06:15:23.313 INFO:tasks.workunit.client.1.smithi117.stdout:2/433: link d2/d5/d13/d3f/d1d/d3a/d11/d4f/l5f d2/d1b/d39/d77/l82 0 2024-03-22T06:15:23.313 INFO:tasks.workunit.client.1.smithi117.stdout:2/434: dread - d2/f69 zero size 2024-03-22T06:15:23.313 INFO:tasks.workunit.client.1.smithi117.stdout:2/435: write d2/d5/db/f66 [657604,129263] 0 2024-03-22T06:15:23.319 INFO:tasks.workunit.client.1.smithi117.stdout:1/485: creat d1/d22/d27/d4e/d92/fab x:0 0 0 2024-03-22T06:15:23.323 INFO:tasks.workunit.client.1.smithi117.stdout:0/451: creat d1/d3/d2b/d3e/f98 x:0 0 0 2024-03-22T06:15:23.325 INFO:tasks.workunit.client.1.smithi117.stdout:8/402: rename d1/d19/d38/d63 to d1/d28/d66/d7f 0 2024-03-22T06:15:23.326 INFO:tasks.workunit.client.1.smithi117.stdout:3/386: truncate d9/d55/f7b 574717 0 2024-03-22T06:15:23.328 INFO:tasks.workunit.client.1.smithi117.stdout:1/486: dread d1/d22/d27/f68 [0,4194304] 0 2024-03-22T06:15:23.329 INFO:tasks.workunit.client.1.smithi117.stdout:1/487: write d1/d3c/d4d/d5b/f91 [575996,32654] 0 2024-03-22T06:15:23.329 INFO:tasks.workunit.client.1.smithi117.stdout:4/402: creat d0/d19/d34/d43/d63/d6f/d6a/f8c x:0 0 0 2024-03-22T06:15:23.337 INFO:tasks.workunit.client.1.smithi117.stdout:2/436: mkdir d2/d5/d41/d83 0 2024-03-22T06:15:23.337 INFO:tasks.workunit.client.1.smithi117.stdout:2/437: readlink d2/d1b/l7e 0 2024-03-22T06:15:23.337 INFO:tasks.workunit.client.1.smithi117.stdout:2/438: chown d2/d5/db/f72 6 1 2024-03-22T06:15:23.337 INFO:tasks.workunit.client.1.smithi117.stdout:2/439: write d2/d5/d13/f7a [437851,1996] 0 2024-03-22T06:15:23.337 INFO:tasks.workunit.client.1.smithi117.stdout:7/386: mknod d8/d9/da/d10/c7a 0 2024-03-22T06:15:23.350 INFO:tasks.workunit.client.1.smithi117.stdout:7/387: dread d8/d9/da/d10/d23/d27/d32/f54 [0,4194304] 0 2024-03-22T06:15:23.350 INFO:tasks.workunit.client.1.smithi117.stdout:7/388: read - d8/f3a zero size 2024-03-22T06:15:23.350 INFO:tasks.workunit.client.1.smithi117.stdout:3/387: dwrite d9/d1a/d39/d82/f84 [0,4194304] 0 2024-03-22T06:15:23.351 INFO:tasks.workunit.client.1.smithi117.stdout:3/388: chown d9/d1a/l33 0 1 2024-03-22T06:15:23.355 INFO:tasks.workunit.client.1.smithi117.stdout:0/452: creat d1/d10/d18/d26/d49/d7e/f99 x:0 0 0 2024-03-22T06:15:23.356 INFO:tasks.workunit.client.1.smithi117.stdout:6/477: rename d3/d40/d24/d2d/d53/d59/d74/d8b to d3/d40/d24/d2d/d42/da2 0 2024-03-22T06:15:23.357 INFO:tasks.workunit.client.1.smithi117.stdout:1/488: dwrite d1/fc [8388608,4194304] 0 2024-03-22T06:15:23.357 INFO:tasks.workunit.client.1.smithi117.stdout:1/489: write d1/d1f/f20 [5718729,98225] 0 2024-03-22T06:15:23.358 INFO:tasks.workunit.client.1.smithi117.stdout:8/403: symlink d1/d3a/d44/d4b/l80 0 2024-03-22T06:15:23.358 INFO:tasks.workunit.client.1.smithi117.stdout:8/404: creat d1/d3a/d44/d9/f81 x:0 0 0 2024-03-22T06:15:23.366 INFO:tasks.workunit.client.1.smithi117.stdout:0/453: mknod d1/d10/d18/c9a 0 2024-03-22T06:15:23.366 INFO:tasks.workunit.client.1.smithi117.stdout:0/454: dread - d1/d10/d18/d26/d53/f93 zero size 2024-03-22T06:15:23.366 INFO:tasks.workunit.client.1.smithi117.stdout:0/455: creat d1/d3/d2b/d3e/de/d1f/f9b x:0 0 0 2024-03-22T06:15:23.368 INFO:tasks.workunit.client.1.smithi117.stdout:5/473: rename dd/df/d5e/d91/d72/d60/d6e/f7a to dd/df/d5e/d91/d90/fa6 0 2024-03-22T06:15:23.370 INFO:tasks.workunit.client.1.smithi117.stdout:6/478: link d3/d3d/f2e d3/d40/d24/d2d/d53/d59/d74/da/de/fa3 0 2024-03-22T06:15:23.370 INFO:tasks.workunit.client.1.smithi117.stdout:6/479: fdatasync d3/d40/d24/d2d/d53/d59/d74/da/de/f19 0 2024-03-22T06:15:23.371 INFO:tasks.workunit.client.1.smithi117.stdout:2/440: dwrite d2/d1b/d39/f65 [0,4194304] 0 2024-03-22T06:15:23.371 INFO:tasks.workunit.client.1.smithi117.stdout:2/441: fsync d2/d5/d41/f5b 0 2024-03-22T06:15:23.371 INFO:tasks.workunit.client.1.smithi117.stdout:7/389: dwrite d8/d9/d37/f4e [0,4194304] 0 2024-03-22T06:15:23.372 INFO:tasks.workunit.client.1.smithi117.stdout:7/390: fsync d8/d9/da/d2b/f6e 0 2024-03-22T06:15:23.372 INFO:tasks.workunit.client.1.smithi117.stdout:1/490: symlink d1/d30/d5d/lac 0 2024-03-22T06:15:23.373 INFO:tasks.workunit.client.1.smithi117.stdout:7/391: dread d8/d9/da/d10/d23/d27/d39/d26/f34 [0,4194304] 0 2024-03-22T06:15:23.376 INFO:tasks.workunit.client.1.smithi117.stdout:8/405: rmdir d1/d3a 39 2024-03-22T06:15:23.378 INFO:tasks.workunit.client.1.smithi117.stdout:4/403: rmdir d0/d19/d34/d43/d63/d6f/d6a 39 2024-03-22T06:15:23.378 INFO:tasks.workunit.client.1.smithi117.stdout:4/404: dread - d0/d51/f7e zero size 2024-03-22T06:15:23.380 INFO:tasks.workunit.client.1.smithi117.stdout:2/442: dread d2/d5/d13/d57/f47 [0,4194304] 0 2024-03-22T06:15:23.385 INFO:tasks.workunit.client.1.smithi117.stdout:3/389: rename d9/d55/l6c to d9/d1a/d3a/l8e 0 2024-03-22T06:15:23.385 INFO:tasks.workunit.client.1.smithi117.stdout:3/390: fsync d9/f63 0 2024-03-22T06:15:23.386 INFO:tasks.workunit.client.1.smithi117.stdout:5/474: getdents dd/df/d5e/d91/d72 0 2024-03-22T06:15:23.388 INFO:tasks.workunit.client.1.smithi117.stdout:6/480: rename d3/d40/d24/d2d/d30/f90 to d3/d40/d24/d2d/d30/fa4 0 2024-03-22T06:15:23.388 INFO:tasks.workunit.client.1.smithi117.stdout:6/481: fsync d3/d40/d24/d2d/d53/d59/d74/d28/d85/f8a 0 2024-03-22T06:15:23.390 INFO:tasks.workunit.client.1.smithi117.stdout:4/405: write d0/d19/d34/d43/f7c [31258,99171] 0 2024-03-22T06:15:23.390 INFO:tasks.workunit.client.1.smithi117.stdout:1/491: mknod d1/d84/cad 0 2024-03-22T06:15:23.394 INFO:tasks.workunit.client.1.smithi117.stdout:7/392: unlink d8/f3a 0 2024-03-22T06:15:23.394 INFO:tasks.workunit.client.1.smithi117.stdout:8/406: dwrite d1/d3a/d44/fa [4194304,4194304] 0 2024-03-22T06:15:23.396 INFO:tasks.workunit.client.1.smithi117.stdout:2/443: unlink d2/d5/d13/f5d 0 2024-03-22T06:15:23.396 INFO:tasks.workunit.client.1.smithi117.stdout:2/444: chown d2/d5/d13/d57 3891 1 2024-03-22T06:15:23.397 INFO:tasks.workunit.client.1.smithi117.stdout:3/391: write d9/d1a/d39/d82/f84 [2532302,41326] 0 2024-03-22T06:15:23.397 INFO:tasks.workunit.client.1.smithi117.stdout:0/456: rename d1/d3/d2b/d3e/ff to d1/d3/d2b/d3e/d3f/f9c 0 2024-03-22T06:15:23.401 INFO:tasks.workunit.client.1.smithi117.stdout:1/492: dread d1/d1f/d32/d33/f3d [0,4194304] 0 2024-03-22T06:15:23.401 INFO:tasks.workunit.client.1.smithi117.stdout:1/493: write d1/f7 [2026821,53622] 0 2024-03-22T06:15:23.401 INFO:tasks.workunit.client.1.smithi117.stdout:1/494: chown d1/d22/d27/f34 10 1 2024-03-22T06:15:23.401 INFO:tasks.workunit.client.1.smithi117.stdout:1/495: creat d1/d1f/fae x:0 0 0 2024-03-22T06:15:23.401 INFO:tasks.workunit.client.1.smithi117.stdout:1/496: creat d1/d22/d27/faf x:0 0 0 2024-03-22T06:15:23.402 INFO:tasks.workunit.client.1.smithi117.stdout:4/406: truncate d0/d19/d34/d43/f62 1841359 0 2024-03-22T06:15:23.403 INFO:tasks.workunit.client.1.smithi117.stdout:1/497: dread d1/d22/d27/d4e/f7d [0,4194304] 0 2024-03-22T06:15:23.404 INFO:tasks.workunit.client.1.smithi117.stdout:1/498: chown d1/d22 129005723 1 2024-03-22T06:15:23.404 INFO:tasks.workunit.client.1.smithi117.stdout:1/499: fsync d1/d22/d27/d4e/f7d 0 2024-03-22T06:15:23.404 INFO:tasks.workunit.client.1.smithi117.stdout:1/500: chown d1/d22/d27/f34 36908237 1 2024-03-22T06:15:23.404 INFO:tasks.workunit.client.1.smithi117.stdout:1/501: creat d1/d30/d35/d58/fb0 x:0 0 0 2024-03-22T06:15:23.409 INFO:tasks.workunit.client.1.smithi117.stdout:2/445: creat d2/d6e/f84 x:0 0 0 2024-03-22T06:15:23.409 INFO:tasks.workunit.client.1.smithi117.stdout:2/446: write d2/d1b/d39/f4b [354437,19291] 0 2024-03-22T06:15:23.409 INFO:tasks.workunit.client.1.smithi117.stdout:3/392: symlink d9/d1a/d3a/l8f 0 2024-03-22T06:15:23.410 INFO:tasks.workunit.client.1.smithi117.stdout:3/393: readlink d9/l23 0 2024-03-22T06:15:23.410 INFO:tasks.workunit.client.1.smithi117.stdout:0/457: mkdir d1/d9d 0 2024-03-22T06:15:23.412 INFO:tasks.workunit.client.1.smithi117.stdout:4/407: mkdir d0/d19/d34/d43/d63/d8d 0 2024-03-22T06:15:23.412 INFO:tasks.workunit.client.1.smithi117.stdout:4/408: chown d0/f6d 19 1 2024-03-22T06:15:23.415 INFO:tasks.workunit.client.1.smithi117.stdout:8/407: getdents d1/d28/d66 0 2024-03-22T06:15:23.415 INFO:tasks.workunit.client.1.smithi117.stdout:8/408: stat d1/l2c 0 2024-03-22T06:15:23.415 INFO:tasks.workunit.client.1.smithi117.stdout:5/475: dwrite dd/df/d5e/d91/f23 [0,4194304] 0 2024-03-22T06:15:23.417 INFO:tasks.workunit.client.1.smithi117.stdout:2/447: creat d2/d5/db/d2c/f85 x:0 0 0 2024-03-22T06:15:23.418 INFO:tasks.workunit.client.1.smithi117.stdout:7/393: dwrite d8/d9/da/d10/d23/d48/f57 [0,4194304] 0 2024-03-22T06:15:23.418 INFO:tasks.workunit.client.1.smithi117.stdout:7/394: chown d8/d9/da/d10/d23/d27/d39/d26 11616566 1 2024-03-22T06:15:23.418 INFO:tasks.workunit.client.1.smithi117.stdout:7/395: fsync d8/d9/da/d10/f25 0 2024-03-22T06:15:23.419 INFO:tasks.workunit.client.1.smithi117.stdout:6/482: dwrite d3/d3d/d17/d31/f87 [0,4194304] 0 2024-03-22T06:15:23.419 INFO:tasks.workunit.client.1.smithi117.stdout:1/502: rmdir d1/d30/d5d/d6f 39 2024-03-22T06:15:23.419 INFO:tasks.workunit.client.1.smithi117.stdout:1/503: creat d1/d1f/fb1 x:0 0 0 2024-03-22T06:15:23.423 INFO:tasks.workunit.client.1.smithi117.stdout:5/476: link dd/d16/d1a/d32/c87 dd/df/d5e/d91/d72/ca7 0 2024-03-22T06:15:23.423 INFO:tasks.workunit.client.1.smithi117.stdout:5/477: readlink dd/df/d5e/d91/d72/d33/l52 0 2024-03-22T06:15:23.423 INFO:tasks.workunit.client.1.smithi117.stdout:2/448: rmdir d2/d5/d13/d3f/d1d/d3a/d71 39 2024-03-22T06:15:23.428 INFO:tasks.workunit.client.1.smithi117.stdout:7/396: truncate d8/d9/da/f28 2844770 0 2024-03-22T06:15:23.429 INFO:tasks.workunit.client.1.smithi117.stdout:7/397: fdatasync d8/d9/da/f28 0 2024-03-22T06:15:23.429 INFO:tasks.workunit.client.1.smithi117.stdout:7/398: fsync d8/d9/da/d10/f22 0 2024-03-22T06:15:23.434 INFO:tasks.workunit.client.1.smithi117.stdout:1/504: mkdir d1/d1f/db2 0 2024-03-22T06:15:23.437 INFO:tasks.workunit.client.1.smithi117.stdout:2/449: truncate d2/d5/d13/d3f/d1d/d3a/f64 2252439 0 2024-03-22T06:15:23.438 INFO:tasks.workunit.client.1.smithi117.stdout:3/394: dwrite d9/f30 [0,4194304] 0 2024-03-22T06:15:23.438 INFO:tasks.workunit.client.1.smithi117.stdout:3/395: chown d9/d66/f74 482256 1 2024-03-22T06:15:23.438 INFO:tasks.workunit.client.1.smithi117.stdout:3/396: dread - d9/f70 zero size 2024-03-22T06:15:23.439 INFO:tasks.workunit.client.1.smithi117.stdout:7/399: creat d8/d9/da/d10/d23/d27/d39/f7b x:0 0 0 2024-03-22T06:15:23.441 INFO:tasks.workunit.client.1.smithi117.stdout:6/483: rmdir d3/d40/d24/d2d/d53/d59/d74/d28/d85 39 2024-03-22T06:15:23.443 INFO:tasks.workunit.client.1.smithi117.stdout:4/409: dwrite d0/d51/f7e [0,4194304] 0 2024-03-22T06:15:23.443 INFO:tasks.workunit.client.1.smithi117.stdout:0/458: dwrite d1/d10/d18/f38 [0,4194304] 0 2024-03-22T06:15:23.443 INFO:tasks.workunit.client.1.smithi117.stdout:1/505: link d1/d1f/d32/l53 d1/d30/d44/d99/lb3 0 2024-03-22T06:15:23.443 INFO:tasks.workunit.client.1.smithi117.stdout:4/410: creat d0/d4/d2a/d5b/f8e x:0 0 0 2024-03-22T06:15:23.443 INFO:tasks.workunit.client.1.smithi117.stdout:4/411: fsync d0/d4/f17 0 2024-03-22T06:15:23.444 INFO:tasks.workunit.client.1.smithi117.stdout:2/450: mkdir d2/d5/d13/d3f/d1d/d86 0 2024-03-22T06:15:23.444 INFO:tasks.workunit.client.1.smithi117.stdout:2/451: getdents d2/d1b/d39/d45/d7c 0 2024-03-22T06:15:23.448 INFO:tasks.workunit.client.1.smithi117.stdout:3/397: rename d9/f11 to d9/d1a/f90 0 2024-03-22T06:15:23.451 INFO:tasks.workunit.client.1.smithi117.stdout:9/446: sync 2024-03-22T06:15:23.451 INFO:tasks.workunit.client.1.smithi117.stdout:1/506: dread d1/d14/f19 [4194304,4194304] 0 2024-03-22T06:15:23.452 INFO:tasks.workunit.client.1.smithi117.stdout:1/507: read d1/f8 [3629497,127443] 0 2024-03-22T06:15:23.453 INFO:tasks.workunit.client.1.smithi117.stdout:1/508: creat d1/d30/fb4 x:0 0 0 2024-03-22T06:15:23.455 INFO:tasks.workunit.client.1.smithi117.stdout:7/400: link d8/d9/da/d10/d23/d48/f57 d8/d9/f7c 0 2024-03-22T06:15:23.455 INFO:tasks.workunit.client.1.smithi117.stdout:7/401: fsync d8/d9/da/d10/d23/d27/d39/f3b 0 2024-03-22T06:15:23.456 INFO:tasks.workunit.client.1.smithi117.stdout:6/484: mknod d3/d40/d24/d2d/d53/d59/d74/d28/d85/ca5 0 2024-03-22T06:15:23.465 INFO:tasks.workunit.client.1.smithi117.stdout:0/459: symlink d1/d3/d2b/d3e/de/l9e 0 2024-03-22T06:15:23.465 INFO:tasks.workunit.client.1.smithi117.stdout:0/460: write d1/d10/d18/d26/d49/d7e/d39/f6f [667821,110697] 0 2024-03-22T06:15:23.465 INFO:tasks.workunit.client.1.smithi117.stdout:0/461: fdatasync d1/f87 0 2024-03-22T06:15:23.468 INFO:tasks.workunit.client.1.smithi117.stdout:4/412: symlink d0/d19/d34/d43/d63/d6f/d6a/l8f 0 2024-03-22T06:15:23.470 INFO:tasks.workunit.client.1.smithi117.stdout:2/452: creat d2/d5/d41/d83/f87 x:0 0 0 2024-03-22T06:15:23.472 INFO:tasks.workunit.client.1.smithi117.stdout:3/398: link d9/d55/l72 d9/d1a/d39/d7d/l91 0 2024-03-22T06:15:23.472 INFO:tasks.workunit.client.1.smithi117.stdout:3/399: fsync d9/d1a/d39/d4c/d36/f62 0 2024-03-22T06:15:23.473 INFO:tasks.workunit.client.1.smithi117.stdout:9/447: creat d4/d47/d64/f84 x:0 0 0 2024-03-22T06:15:23.475 INFO:tasks.workunit.client.1.smithi117.stdout:1/509: mknod d1/d3c/cb5 0 2024-03-22T06:15:23.478 INFO:tasks.workunit.client.1.smithi117.stdout:6/485: mknod d3/d40/ca6 0 2024-03-22T06:15:23.479 INFO:tasks.workunit.client.1.smithi117.stdout:0/462: symlink d1/d3/d2b/d3e/d3f/l9f 0 2024-03-22T06:15:23.479 INFO:tasks.workunit.client.1.smithi117.stdout:0/463: readlink d1/d10/l31 0 2024-03-22T06:15:23.480 INFO:tasks.workunit.client.1.smithi117.stdout:1/510: dread d1/d1f/f75 [0,4194304] 0 2024-03-22T06:15:23.481 INFO:tasks.workunit.client.1.smithi117.stdout:4/413: creat d0/d19/d34/d43/d63/d8d/f90 x:0 0 0 2024-03-22T06:15:23.482 INFO:tasks.workunit.client.1.smithi117.stdout:2/453: mkdir d2/d1b/d39/d45/d88 0 2024-03-22T06:15:23.484 INFO:tasks.workunit.client.1.smithi117.stdout:9/448: mkdir d4/d47/d85 0 2024-03-22T06:15:23.484 INFO:tasks.workunit.client.1.smithi117.stdout:7/402: symlink d8/d9/da/d10/l7d 0 2024-03-22T06:15:23.485 INFO:tasks.workunit.client.1.smithi117.stdout:6/486: symlink d3/d40/d24/d2d/d42/la7 0 2024-03-22T06:15:23.489 INFO:tasks.workunit.client.1.smithi117.stdout:9/449: dread d4/d8/f41 [0,4194304] 0 2024-03-22T06:15:23.489 INFO:tasks.workunit.client.1.smithi117.stdout:0/464: mknod d1/d10/d18/ca0 0 2024-03-22T06:15:23.490 INFO:tasks.workunit.client.1.smithi117.stdout:3/400: dwrite d9/d1a/f2d [0,4194304] 0 2024-03-22T06:15:23.492 INFO:tasks.workunit.client.1.smithi117.stdout:1/511: truncate d1/d14/f19 358158 0 2024-03-22T06:15:23.495 INFO:tasks.workunit.client.1.smithi117.stdout:2/454: mkdir d2/d5/d13/d3f/d1d/d3a/d89 0 2024-03-22T06:15:23.497 INFO:tasks.workunit.client.1.smithi117.stdout:6/487: truncate d3/d40/d24/d2d/d53/d59/d74/f11 865340 0 2024-03-22T06:15:23.503 INFO:tasks.workunit.client.1.smithi117.stdout:9/450: write d4/d8/d3e/d7a/f42 [540325,62231] 0 2024-03-22T06:15:23.503 INFO:tasks.workunit.client.1.smithi117.stdout:9/451: stat d4/d46/f79 0 2024-03-22T06:15:23.504 INFO:tasks.workunit.client.1.smithi117.stdout:9/452: chown d4/d8/d3e/d7a/d2f 57 1 2024-03-22T06:15:23.504 INFO:tasks.workunit.client.1.smithi117.stdout:9/453: dread - d4/d7/d27/f5a zero size 2024-03-22T06:15:23.504 INFO:tasks.workunit.client.1.smithi117.stdout:9/454: fdatasync d4/d7/d27/f5a 0 2024-03-22T06:15:23.506 INFO:tasks.workunit.client.1.smithi117.stdout:1/512: mkdir d1/d3c/db6 0 2024-03-22T06:15:23.506 INFO:tasks.workunit.client.1.smithi117.stdout:1/513: write d1/f7 [434931,45518] 0 2024-03-22T06:15:23.509 INFO:tasks.workunit.client.1.smithi117.stdout:6/488: mkdir d3/d40/d46/d84/d49/da8 0 2024-03-22T06:15:23.514 INFO:tasks.workunit.client.1.smithi117.stdout:1/514: creat d1/d30/d44/d99/fb7 x:0 0 0 2024-03-22T06:15:23.514 INFO:tasks.workunit.client.1.smithi117.stdout:1/515: fdatasync d1/d3c/f88 0 2024-03-22T06:15:23.515 INFO:tasks.workunit.client.1.smithi117.stdout:7/403: dwrite d8/d9/d37/d45/d61/f71 [0,4194304] 0 2024-03-22T06:15:23.516 INFO:tasks.workunit.client.1.smithi117.stdout:6/489: link d3/d40/d46/d84/f7b d3/d40/fa9 0 2024-03-22T06:15:23.518 INFO:tasks.workunit.client.1.smithi117.stdout:1/516: mkdir d1/d3c/d4d/d5b/db8 0 2024-03-22T06:15:23.527 INFO:tasks.workunit.client.1.smithi117.stdout:6/490: mknod d3/d40/d24/d2d/d53/d59/caa 0 2024-03-22T06:15:23.527 INFO:tasks.workunit.client.1.smithi117.stdout:6/491: fdatasync d3/d40/d24/d2d/d42/da2/f2f 0 2024-03-22T06:15:23.528 INFO:tasks.workunit.client.1.smithi117.stdout:4/414: dwrite d0/d19/d34/d43/f7c [4194304,4194304] 0 2024-03-22T06:15:23.528 INFO:tasks.workunit.client.1.smithi117.stdout:0/465: dwrite d1/d3/d2b/d3e/de/d1f/d3b/d55/f8e [0,4194304] 0 2024-03-22T06:15:23.528 INFO:tasks.workunit.client.1.smithi117.stdout:3/401: dwrite d9/d55/f59 [0,4194304] 0 2024-03-22T06:15:23.530 INFO:tasks.workunit.client.1.smithi117.stdout:2/455: dwrite d2/d5/d13/f30 [0,4194304] 0 2024-03-22T06:15:23.530 INFO:tasks.workunit.client.1.smithi117.stdout:9/455: dwrite d4/d8/d3f/f52 [4194304,4194304] 0 2024-03-22T06:15:23.530 INFO:tasks.workunit.client.1.smithi117.stdout:9/456: chown d4/d7/f24 1791 1 2024-03-22T06:15:23.530 INFO:tasks.workunit.client.1.smithi117.stdout:7/404: rmdir d8/d9/d37/d5f 39 2024-03-22T06:15:23.532 INFO:tasks.workunit.client.1.smithi117.stdout:6/492: getdents d3/d40/d24/d2d/d42/da2 0 2024-03-22T06:15:23.532 INFO:tasks.workunit.client.1.smithi117.stdout:6/493: fdatasync d3/d40/d24/f4e 0 2024-03-22T06:15:23.546 INFO:tasks.workunit.client.1.smithi117.stdout:4/415: link d0/d19/d34/d43/d63/d6f/d5f/f75 d0/d19/d34/d43/d63/d6f/f91 0 2024-03-22T06:15:23.546 INFO:tasks.workunit.client.1.smithi117.stdout:4/416: chown d0/d4/d2a/d5b/l73 989 1 2024-03-22T06:15:23.549 INFO:tasks.workunit.client.1.smithi117.stdout:3/402: symlink d9/d1a/d39/d8a/l92 0 2024-03-22T06:15:23.550 INFO:tasks.workunit.client.1.smithi117.stdout:0/466: dwrite d1/d10/d18/d26/d49/d7e/d39/d6a/f94 [0,4194304] 0 2024-03-22T06:15:23.550 INFO:tasks.workunit.client.1.smithi117.stdout:0/467: chown d1/c2 39405133 1 2024-03-22T06:15:23.550 INFO:tasks.workunit.client.1.smithi117.stdout:0/468: chown c0 1869873 1 2024-03-22T06:15:23.552 INFO:tasks.workunit.client.1.smithi117.stdout:2/456: rmdir d2/d5/d13/d3f/d1d/d3a/d11/d4f 39 2024-03-22T06:15:23.553 INFO:tasks.workunit.client.1.smithi117.stdout:2/457: read - d2/d1b/d39/d77/f80 zero size 2024-03-22T06:15:23.553 INFO:tasks.workunit.client.1.smithi117.stdout:2/458: creat d2/d5/d41/f8a x:0 0 0 2024-03-22T06:15:23.553 INFO:tasks.workunit.client.1.smithi117.stdout:9/457: dwrite d4/d7/d27/f7d [0,4194304] 0 2024-03-22T06:15:23.554 INFO:tasks.workunit.client.1.smithi117.stdout:7/405: creat d8/d9/d37/d45/f7e x:0 0 0 2024-03-22T06:15:23.555 INFO:tasks.workunit.client.1.smithi117.stdout:2/459: dread d2/d1b/d39/f4b [0,4194304] 0 2024-03-22T06:15:23.557 INFO:tasks.workunit.client.1.smithi117.stdout:6/494: symlink d3/d40/d24/d2d/d30/lab 0 2024-03-22T06:15:23.559 INFO:tasks.workunit.client.1.smithi117.stdout:4/417: mkdir d0/d51/d92 0 2024-03-22T06:15:23.559 INFO:tasks.workunit.client.1.smithi117.stdout:3/403: truncate d9/d1a/d39/d4c/f4e 2477833 0 2024-03-22T06:15:23.567 INFO:tasks.workunit.client.1.smithi117.stdout:0/469: rmdir d1/d10/d5c 39 2024-03-22T06:15:23.573 INFO:tasks.workunit.client.1.smithi117.stdout:9/458: mknod d4/d8/d3e/c86 0 2024-03-22T06:15:23.573 INFO:tasks.workunit.client.1.smithi117.stdout:8/409: sync 2024-03-22T06:15:23.577 INFO:tasks.workunit.client.1.smithi117.stdout:6/495: link d3/d40/d24/d2d/d42/la7 d3/d40/d46/d84/d49/d79/d9d/lac 0 2024-03-22T06:15:23.577 INFO:tasks.workunit.client.1.smithi117.stdout:6/496: fdatasync d3/f52 0 2024-03-22T06:15:23.577 INFO:tasks.workunit.client.1.smithi117.stdout:5/478: sync 2024-03-22T06:15:23.578 INFO:tasks.workunit.client.1.smithi117.stdout:4/418: getdents d0/d19/d34 0 2024-03-22T06:15:23.581 INFO:tasks.workunit.client.1.smithi117.stdout:7/406: dwrite d8/d9/d37/d45/d61/f6f [0,4194304] 0 2024-03-22T06:15:23.581 INFO:tasks.workunit.client.1.smithi117.stdout:2/460: dwrite d2/d1b/d39/d77/f80 [0,4194304] 0 2024-03-22T06:15:23.582 INFO:tasks.workunit.client.1.smithi117.stdout:3/404: mkdir d9/d93 0 2024-03-22T06:15:23.584 INFO:tasks.workunit.client.1.smithi117.stdout:0/470: link d1/d3/d2b/d3e/de/l9e d1/d9d/la1 0 2024-03-22T06:15:23.584 INFO:tasks.workunit.client.1.smithi117.stdout:0/471: creat d1/d10/d5c/d40/d69/fa2 x:0 0 0 2024-03-22T06:15:23.587 INFO:tasks.workunit.client.1.smithi117.stdout:9/459: symlink d4/d8/d3e/d7a/d3b/l87 0 2024-03-22T06:15:23.587 INFO:tasks.workunit.client.1.smithi117.stdout:9/460: stat d4/d8/d3e/f45 0 2024-03-22T06:15:23.587 INFO:tasks.workunit.client.1.smithi117.stdout:9/461: fsync d4/d47/d64/f84 0 2024-03-22T06:15:23.587 INFO:tasks.workunit.client.1.smithi117.stdout:4/419: dread d0/d19/d34/f4f [0,4194304] 0 2024-03-22T06:15:23.589 INFO:tasks.workunit.client.1.smithi117.stdout:6/497: mknod d3/d40/d24/d2d/d53/d59/d74/d28/cad 0 2024-03-22T06:15:23.590 INFO:tasks.workunit.client.1.smithi117.stdout:6/498: fdatasync d3/d40/d24/d2d/d53/d59/f88 0 2024-03-22T06:15:23.590 INFO:tasks.workunit.client.1.smithi117.stdout:5/479: mkdir dd/df/d95/da0/da8 0 2024-03-22T06:15:23.593 INFO:tasks.workunit.client.1.smithi117.stdout:7/407: write d8/d9/d37/f4e [2137877,33591] 0 2024-03-22T06:15:23.593 INFO:tasks.workunit.client.1.smithi117.stdout:7/408: fsync d8/d9/da/d10/d23/d27/f3e 0 2024-03-22T06:15:23.595 INFO:tasks.workunit.client.1.smithi117.stdout:8/410: dwrite d1/d19/f64 [0,4194304] 0 2024-03-22T06:15:23.605 INFO:tasks.workunit.client.1.smithi117.stdout:2/461: link d2/d35/c61 d2/d5/d13/d3f/c8b 0 2024-03-22T06:15:23.610 INFO:tasks.workunit.client.1.smithi117.stdout:7/409: dwrite d8/d9/da/d2b/f6e [0,4194304] 0 2024-03-22T06:15:23.610 INFO:tasks.workunit.client.1.smithi117.stdout:9/462: dwrite d4/d8/f1a [0,4194304] 0 2024-03-22T06:15:23.611 INFO:tasks.workunit.client.1.smithi117.stdout:0/472: mknod d1/d3/d2b/d3e/de/d1f/d59/ca3 0 2024-03-22T06:15:23.615 INFO:tasks.workunit.client.1.smithi117.stdout:4/420: symlink d0/d51/l93 0 2024-03-22T06:15:23.617 INFO:tasks.workunit.client.1.smithi117.stdout:6/499: symlink d3/d40/d46/d84/d49/d79/d9d/lae 0 2024-03-22T06:15:23.617 INFO:tasks.workunit.client.1.smithi117.stdout:6/500: read - d3/d40/d24/d2d/d30/fa4 zero size 2024-03-22T06:15:23.619 INFO:tasks.workunit.client.1.smithi117.stdout:7/410: read d8/d9/f1e [3951532,78433] 0 2024-03-22T06:15:23.620 INFO:tasks.workunit.client.1.smithi117.stdout:7/411: chown d8/d9/da/d10/d23/d27/d39/d26 16481948 1 2024-03-22T06:15:23.621 INFO:tasks.workunit.client.1.smithi117.stdout:5/480: creat dd/df/d88/fa9 x:0 0 0 2024-03-22T06:15:23.624 INFO:tasks.workunit.client.1.smithi117.stdout:9/463: dread d4/d7/f24 [0,4194304] 0 2024-03-22T06:15:23.625 INFO:tasks.workunit.client.1.smithi117.stdout:4/421: dread d0/d19/d34/f4f [0,4194304] 0 2024-03-22T06:15:23.625 INFO:tasks.workunit.client.1.smithi117.stdout:4/422: fdatasync d0/d19/d34/f4f 0 2024-03-22T06:15:23.626 INFO:tasks.workunit.client.1.smithi117.stdout:8/411: symlink d1/d3a/d44/d16/l82 0 2024-03-22T06:15:23.626 INFO:tasks.workunit.client.1.smithi117.stdout:8/412: readlink d1/d3a/d44/d11/l53 0 2024-03-22T06:15:23.628 INFO:tasks.workunit.client.1.smithi117.stdout:2/462: creat d2/d5/d13/d3f/d1d/d3a/d11/d4f/f8c x:0 0 0 2024-03-22T06:15:23.631 INFO:tasks.workunit.client.1.smithi117.stdout:3/405: link d9/f35 d9/d1a/d39/d4c/d36/d41/f94 0 2024-03-22T06:15:23.637 INFO:tasks.workunit.client.1.smithi117.stdout:7/412: dwrite d8/fb [0,4194304] 0 2024-03-22T06:15:23.643 INFO:tasks.workunit.client.1.smithi117.stdout:0/473: link d1/d10/d18/d26/f75 d1/d3/d2b/d3e/de/fa4 0 2024-03-22T06:15:23.643 INFO:tasks.workunit.client.1.smithi117.stdout:6/501: mknod d3/d40/d46/d84/caf 0 2024-03-22T06:15:23.643 INFO:tasks.workunit.client.1.smithi117.stdout:6/502: fsync d3/d40/d24/d2d/d30/f98 0 2024-03-22T06:15:23.643 INFO:tasks.workunit.client.1.smithi117.stdout:6/503: truncate d3/d40/d46/d84/f7b 9989 0 2024-03-22T06:15:23.643 INFO:tasks.workunit.client.1.smithi117.stdout:5/481: symlink dd/df/d5e/d91/d72/d60/laa 0 2024-03-22T06:15:23.643 INFO:tasks.workunit.client.1.smithi117.stdout:9/464: creat d4/d50/f88 x:0 0 0 2024-03-22T06:15:23.643 INFO:tasks.workunit.client.1.smithi117.stdout:9/465: chown d4/d47/d64 5 1 2024-03-22T06:15:23.644 INFO:tasks.workunit.client.1.smithi117.stdout:4/423: rmdir d0/d19/d34/d43/d63/d6f 39 2024-03-22T06:15:23.644 INFO:tasks.workunit.client.1.smithi117.stdout:4/424: chown d0/d3f/f54 0 1 2024-03-22T06:15:23.644 INFO:tasks.workunit.client.1.smithi117.stdout:4/425: read - d0/d4/d2a/d5b/f8e zero size 2024-03-22T06:15:23.644 INFO:tasks.workunit.client.1.smithi117.stdout:2/463: dread d2/d35/d38/f5e [4194304,4194304] 0 2024-03-22T06:15:23.644 INFO:tasks.workunit.client.1.smithi117.stdout:4/426: write d0/d3f/f54 [4765891,114664] 0 2024-03-22T06:15:23.644 INFO:tasks.workunit.client.1.smithi117.stdout:4/427: fsync d0/d19/d34/f4f 0 2024-03-22T06:15:23.645 INFO:tasks.workunit.client.1.smithi117.stdout:4/428: write d0/d19/d28/f59 [641308,111805] 0 2024-03-22T06:15:23.645 INFO:tasks.workunit.client.1.smithi117.stdout:4/429: write d0/d19/d28/f59 [812341,2742] 0 2024-03-22T06:15:23.645 INFO:tasks.workunit.client.1.smithi117.stdout:8/413: symlink d1/d3a/d44/d6a/l83 0 2024-03-22T06:15:23.646 INFO:tasks.workunit.client.1.smithi117.stdout:3/406: link d9/d1a/d39/f7f d9/d1a/d39/d4c/f95 0 2024-03-22T06:15:23.646 INFO:tasks.workunit.client.1.smithi117.stdout:3/407: chown d9/d1a/d39/f7f 347 1 2024-03-22T06:15:23.646 INFO:tasks.workunit.client.1.smithi117.stdout:3/408: chown d9/d1a/d3a/f7a 71523 1 2024-03-22T06:15:23.646 INFO:tasks.workunit.client.1.smithi117.stdout:7/413: read d8/fb [735078,94191] 0 2024-03-22T06:15:23.647 INFO:tasks.workunit.client.1.smithi117.stdout:7/414: readlink d8/d9/da/d10/d23/d27/d39/l42 0 2024-03-22T06:15:23.647 INFO:tasks.workunit.client.1.smithi117.stdout:7/415: fdatasync d8/d9/da/d10/d23/d48/f57 0 2024-03-22T06:15:23.651 INFO:tasks.workunit.client.1.smithi117.stdout:6/504: unlink d3/d3d/d17/d31/f34 0 2024-03-22T06:15:23.653 INFO:tasks.workunit.client.1.smithi117.stdout:1/517: sync 2024-03-22T06:15:23.654 INFO:tasks.workunit.client.1.smithi117.stdout:0/474: dwrite d1/d3/d2b/d3e/f60 [0,4194304] 0 2024-03-22T06:15:23.654 INFO:tasks.workunit.client.1.smithi117.stdout:5/482: rename dd/df/d1e/f20 to dd/df/d95/fab 0 2024-03-22T06:15:23.655 INFO:tasks.workunit.client.1.smithi117.stdout:9/466: creat d4/d8/d3f/f89 x:0 0 0 2024-03-22T06:15:23.656 INFO:tasks.workunit.client.1.smithi117.stdout:3/409: dread d9/f19 [0,4194304] 0 2024-03-22T06:15:23.656 INFO:tasks.workunit.client.1.smithi117.stdout:3/410: read d9/d1a/d39/d4c/d36/f43 [936258,123394] 0 2024-03-22T06:15:23.657 INFO:tasks.workunit.client.1.smithi117.stdout:2/464: mkdir d2/d5/d13/d3f/d1d/d3a/d89/d8d 0 2024-03-22T06:15:23.660 INFO:tasks.workunit.client.1.smithi117.stdout:4/430: mknod d0/d19/d34/c94 0 2024-03-22T06:15:23.661 INFO:tasks.workunit.client.1.smithi117.stdout:8/414: mknod d1/d28/d6b/c84 0 2024-03-22T06:15:23.662 INFO:tasks.workunit.client.1.smithi117.stdout:8/415: write d1/d3a/d44/f1d [4157715,63405] 0 2024-03-22T06:15:23.662 INFO:tasks.workunit.client.1.smithi117.stdout:8/416: truncate d1/d19/f43 4235557 0 2024-03-22T06:15:23.664 INFO:tasks.workunit.client.1.smithi117.stdout:5/483: dread dd/df/d5e/d91/d72/d60/f6b [0,4194304] 0 2024-03-22T06:15:23.664 INFO:tasks.workunit.client.1.smithi117.stdout:5/484: readlink dd/l42 0 2024-03-22T06:15:23.664 INFO:tasks.workunit.client.1.smithi117.stdout:5/485: dread - dd/d16/d1a/d32/f51 zero size 2024-03-22T06:15:23.664 INFO:tasks.workunit.client.1.smithi117.stdout:5/486: write dd/df/d5e/d91/d72/d60/f6b [2048170,56136] 0 2024-03-22T06:15:23.664 INFO:tasks.workunit.client.1.smithi117.stdout:5/487: read dd/d16/f64 [856953,63186] 0 2024-03-22T06:15:23.665 INFO:tasks.workunit.client.1.smithi117.stdout:6/505: creat d3/d40/d24/d2d/d53/d75/fb0 x:0 0 0 2024-03-22T06:15:23.667 INFO:tasks.workunit.client.1.smithi117.stdout:1/518: creat d1/d84/d77/fb9 x:0 0 0 2024-03-22T06:15:23.671 INFO:tasks.workunit.client.1.smithi117.stdout:9/467: dwrite d4/d7/d27/f2e [0,4194304] 0 2024-03-22T06:15:23.671 INFO:tasks.workunit.client.1.smithi117.stdout:0/475: symlink d1/d10/d18/la5 0 2024-03-22T06:15:23.672 INFO:tasks.workunit.client.1.smithi117.stdout:3/411: truncate f2 403213 0 2024-03-22T06:15:23.674 INFO:tasks.workunit.client.1.smithi117.stdout:2/465: rename d2/d5/db/f66 to d2/d35/d4c/f8e 0 2024-03-22T06:15:23.674 INFO:tasks.workunit.client.1.smithi117.stdout:2/466: creat d2/d5/d13/d3f/d1d/f8f x:0 0 0 2024-03-22T06:15:23.676 INFO:tasks.workunit.client.1.smithi117.stdout:7/416: link d8/d9/d37/d45/f7e d8/d9/da/d10/d23/d27/d39/d26/f7f 0 2024-03-22T06:15:23.678 INFO:tasks.workunit.client.1.smithi117.stdout:5/488: mknod dd/df/d5e/d91/d36/d70/cac 0 2024-03-22T06:15:23.678 INFO:tasks.workunit.client.1.smithi117.stdout:5/489: fdatasync dd/df/d5e/d91/f23 0 2024-03-22T06:15:23.679 INFO:tasks.workunit.client.1.smithi117.stdout:1/519: mknod d1/d30/d44/d9c/da9/cba 0 2024-03-22T06:15:23.682 INFO:tasks.workunit.client.1.smithi117.stdout:9/468: truncate d4/d50/f6b 814212 0 2024-03-22T06:15:23.683 INFO:tasks.workunit.client.1.smithi117.stdout:6/506: dwrite d3/f3f [0,4194304] 0 2024-03-22T06:15:23.684 INFO:tasks.workunit.client.1.smithi117.stdout:0/476: creat d1/d3/d2b/d3e/de/d1f/d3b/fa6 x:0 0 0 2024-03-22T06:15:23.685 INFO:tasks.workunit.client.1.smithi117.stdout:0/477: fsync d1/d10/d5c/d40/f73 0 2024-03-22T06:15:23.685 INFO:tasks.workunit.client.1.smithi117.stdout:0/478: truncate d1/d3/d2b/d3e/de/d1f/d3b/d55/f5e 1016674 0 2024-03-22T06:15:23.691 INFO:tasks.workunit.client.1.smithi117.stdout:3/412: creat d9/d1a/d39/f96 x:0 0 0 2024-03-22T06:15:23.692 INFO:tasks.workunit.client.1.smithi117.stdout:4/431: rename d0/d19/d28/d47/f67 to d0/d3f/f95 0 2024-03-22T06:15:23.692 INFO:tasks.workunit.client.1.smithi117.stdout:6/507: dread d3/f3f [0,4194304] 0 2024-03-22T06:15:23.693 INFO:tasks.workunit.client.1.smithi117.stdout:4/432: write d0/d19/f4a [27066,82059] 0 2024-03-22T06:15:23.693 INFO:tasks.workunit.client.1.smithi117.stdout:4/433: chown d0/d3f 59010 1 2024-03-22T06:15:23.693 INFO:tasks.workunit.client.1.smithi117.stdout:4/434: readlink d0/d4/d2a/d5b/l73 0 2024-03-22T06:15:23.701 INFO:tasks.workunit.client.1.smithi117.stdout:5/490: mknod dd/df/d5e/d91/d72/cad 0 2024-03-22T06:15:23.705 INFO:tasks.workunit.client.1.smithi117.stdout:1/520: unlink d1/d84/c3f 0 2024-03-22T06:15:23.706 INFO:tasks.workunit.client.1.smithi117.stdout:9/469: creat d4/d8/d37/d58/f8a x:0 0 0 2024-03-22T06:15:23.707 INFO:tasks.workunit.client.1.smithi117.stdout:9/470: fdatasync d4/d8/d37/f4b 0 2024-03-22T06:15:23.708 INFO:tasks.workunit.client.1.smithi117.stdout:0/479: mkdir d1/da7 0 2024-03-22T06:15:23.709 INFO:tasks.workunit.client.1.smithi117.stdout:3/413: stat d9/d1a/d39/d4c/d36/d41/l76 0 2024-03-22T06:15:23.710 INFO:tasks.workunit.client.1.smithi117.stdout:3/414: read d9/d1a/d3a/f47 [821383,20538] 0 2024-03-22T06:15:23.710 INFO:tasks.workunit.client.1.smithi117.stdout:3/415: dread - d9/f2a zero size 2024-03-22T06:15:23.710 INFO:tasks.workunit.client.1.smithi117.stdout:6/508: symlink d3/d40/d46/d84/d49/da8/lb1 0 2024-03-22T06:15:23.710 INFO:tasks.workunit.client.1.smithi117.stdout:5/491: write dd/df/d5e/d91/f57 [1844091,5548] 0 2024-03-22T06:15:23.712 INFO:tasks.workunit.client.1.smithi117.stdout:5/492: dread dd/d16/d5a/f83 [0,4194304] 0 2024-03-22T06:15:23.713 INFO:tasks.workunit.client.1.smithi117.stdout:4/435: mknod d0/d19/d34/d43/d63/d6f/d5f/c96 0 2024-03-22T06:15:23.713 INFO:tasks.workunit.client.1.smithi117.stdout:4/436: creat d0/d4/d2a/d5b/f97 x:0 0 0 2024-03-22T06:15:23.713 INFO:tasks.workunit.client.1.smithi117.stdout:4/437: dread - d0/d19/d34/d43/d63/f89 zero size 2024-03-22T06:15:23.715 INFO:tasks.workunit.client.1.smithi117.stdout:7/417: symlink d8/d9/da/d10/d23/d27/d39/d26/l80 0 2024-03-22T06:15:23.716 INFO:tasks.workunit.client.1.smithi117.stdout:7/418: chown d8/d50 323141 1 2024-03-22T06:15:23.718 INFO:tasks.workunit.client.1.smithi117.stdout:0/480: dread d1/d3/d2b/d3e/de/d1f/d3b/f42 [0,4194304] 0 2024-03-22T06:15:23.721 INFO:tasks.workunit.client.1.smithi117.stdout:2/467: getdents d2/d5/d13/d3f/d1d 0 2024-03-22T06:15:23.722 INFO:tasks.workunit.client.1.smithi117.stdout:2/468: write d2/d5/d13/f19 [4738687,88008] 0 2024-03-22T06:15:23.722 INFO:tasks.workunit.client.1.smithi117.stdout:2/469: chown d2/d1b/l25 106523 1 2024-03-22T06:15:23.722 INFO:tasks.workunit.client.1.smithi117.stdout:2/470: write d2/d5/d13/d3f/d1d/d3a/d11/f15 [1166246,20939] 0 2024-03-22T06:15:23.724 INFO:tasks.workunit.client.1.smithi117.stdout:1/521: truncate d1/d1f/d32/d33/f3d 4000935 0 2024-03-22T06:15:23.726 INFO:tasks.workunit.client.1.smithi117.stdout:7/419: dread d8/d9/da/d10/d23/d27/d39/f4a [0,4194304] 0 2024-03-22T06:15:23.727 INFO:tasks.workunit.client.1.smithi117.stdout:9/471: link d4/d8/d3e/d7a/d2f/c74 d4/d8/d3e/d7a/d3b/c8b 0 2024-03-22T06:15:23.729 INFO:tasks.workunit.client.1.smithi117.stdout:3/416: getdents d9/d1a/d39/d4c/d36 0 2024-03-22T06:15:23.729 INFO:tasks.workunit.client.1.smithi117.stdout:3/417: readlink d9/d1a/d39/d4c/d37/d69/l80 0 2024-03-22T06:15:23.731 INFO:tasks.workunit.client.1.smithi117.stdout:6/509: symlink d3/d40/d24/d2d/d42/da2/lb2 0 2024-03-22T06:15:23.732 INFO:tasks.workunit.client.1.smithi117.stdout:7/420: write d8/d9/da/d10/d23/d27/d39/f3b [1246354,42407] 0 2024-03-22T06:15:23.740 INFO:tasks.workunit.client.1.smithi117.stdout:1/522: mknod d1/d22/cbb 0 2024-03-22T06:15:23.741 INFO:tasks.workunit.client.1.smithi117.stdout:7/421: read d8/d9/f7c [4102743,11966] 0 2024-03-22T06:15:23.741 INFO:tasks.workunit.client.1.smithi117.stdout:7/422: fsync d8/d9/d37/f4e 0 2024-03-22T06:15:23.742 INFO:tasks.workunit.client.1.smithi117.stdout:9/472: mkdir d4/d46/d8c 0 2024-03-22T06:15:23.743 INFO:tasks.workunit.client.1.smithi117.stdout:8/417: sync 2024-03-22T06:15:23.744 INFO:tasks.workunit.client.1.smithi117.stdout:0/481: dwrite d1/d10/d18/d26/d49/d7e/d39/f88 [0,4194304] 0 2024-03-22T06:15:23.745 INFO:tasks.workunit.client.1.smithi117.stdout:3/418: truncate d9/d1a/f1f 1996827 0 2024-03-22T06:15:23.745 INFO:tasks.workunit.client.1.smithi117.stdout:3/419: fsync d9/d1a/d3a/f53 0 2024-03-22T06:15:23.745 INFO:tasks.workunit.client.1.smithi117.stdout:3/420: truncate d9/d55/f57 88810 0 2024-03-22T06:15:23.746 INFO:tasks.workunit.client.1.smithi117.stdout:2/471: dwrite d2/d35/d38/f5e [0,4194304] 0 2024-03-22T06:15:23.746 INFO:tasks.workunit.client.1.smithi117.stdout:2/472: chown d2/d5/db/c55 465635294 1 2024-03-22T06:15:23.746 INFO:tasks.workunit.client.1.smithi117.stdout:4/438: dwrite d0/f83 [0,4194304] 0 2024-03-22T06:15:23.746 INFO:tasks.workunit.client.1.smithi117.stdout:4/439: fdatasync d0/d4/f17 0 2024-03-22T06:15:23.750 INFO:tasks.workunit.client.1.smithi117.stdout:5/493: mkdir dd/df/dae 0 2024-03-22T06:15:23.759 INFO:tasks.workunit.client.1.smithi117.stdout:9/473: dwrite d4/d8/d37/d58/f8a [0,4194304] 0 2024-03-22T06:15:23.762 INFO:tasks.workunit.client.1.smithi117.stdout:3/421: mkdir d9/d1a/d39/d7d/d97 0 2024-03-22T06:15:23.762 INFO:tasks.workunit.client.1.smithi117.stdout:3/422: creat d9/d1a/d39/d7d/f98 x:0 0 0 2024-03-22T06:15:23.763 INFO:tasks.workunit.client.1.smithi117.stdout:2/473: creat d2/d5/d13/d57/f90 x:0 0 0 2024-03-22T06:15:23.763 INFO:tasks.workunit.client.1.smithi117.stdout:2/474: fdatasync d2/d35/d4c/f8e 0 2024-03-22T06:15:23.772 INFO:tasks.workunit.client.1.smithi117.stdout:4/440: creat d0/d19/f98 x:0 0 0 2024-03-22T06:15:23.779 INFO:tasks.workunit.client.1.smithi117.stdout:5/494: link dd/df/l3d dd/d16/d5a/d66/laf 0 2024-03-22T06:15:23.779 INFO:tasks.workunit.client.1.smithi117.stdout:5/495: fdatasync dd/d16/d1a/d32/f40 0 2024-03-22T06:15:23.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:23 smithi006 ceph-mon[29301]: pgmap v25: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 125 MiB/s rd, 146 MiB/s wr, 349 op/s 2024-03-22T06:15:23.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:23 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:23.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:23 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:23.796 INFO:tasks.workunit.client.1.smithi117.stdout:7/423: rmdir d8/d9 39 2024-03-22T06:15:23.796 INFO:tasks.workunit.client.1.smithi117.stdout:7/424: chown d8/d9/da/d10/d23/c64 39 1 2024-03-22T06:15:23.801 INFO:tasks.workunit.client.1.smithi117.stdout:9/474: mkdir d4/d47/d64/d72/d8d 0 2024-03-22T06:15:23.801 INFO:tasks.workunit.client.1.smithi117.stdout:9/475: chown d4/d47/d64/d72 17445 1 2024-03-22T06:15:23.806 INFO:tasks.workunit.client.1.smithi117.stdout:3/423: mkdir d9/d1a/d39/d4c/d99 0 2024-03-22T06:15:23.807 INFO:tasks.workunit.client.1.smithi117.stdout:3/424: write d9/d1a/d39/d4c/f42 [1255935,2320] 0 2024-03-22T06:15:23.811 INFO:tasks.workunit.client.1.smithi117.stdout:6/510: rename d3/d40/d24/d2d/d30 to d3/d40/d24/d2d/d53/d59/d74/da/db3 0 2024-03-22T06:15:23.811 INFO:tasks.workunit.client.1.smithi117.stdout:6/511: dread - d3/d40/d24/d2d/d53/d59/f88 zero size 2024-03-22T06:15:23.811 INFO:tasks.workunit.client.1.smithi117.stdout:6/512: dread - d3/d40/d24/d2d/d53/d59/d74/da/db3/f98 zero size 2024-03-22T06:15:23.811 INFO:tasks.workunit.client.1.smithi117.stdout:6/513: write d3/d40/d24/d2d/d53/d59/d74/da/f29 [2870120,129996] 0 2024-03-22T06:15:23.811 INFO:tasks.workunit.client.1.smithi117.stdout:6/514: write d3/d40/d24/d2d/d53/d59/d74/f11 [1174664,55075] 0 2024-03-22T06:15:23.817 INFO:tasks.workunit.client.1.smithi117.stdout:9/476: dwrite d4/d8/d3e/d7a/f28 [0,4194304] 0 2024-03-22T06:15:23.817 INFO:tasks.workunit.client.1.smithi117.stdout:5/496: mknod dd/df/d5e/d91/d36/d98/cb0 0 2024-03-22T06:15:23.817 INFO:tasks.workunit.client.1.smithi117.stdout:5/497: readlink lc 0 2024-03-22T06:15:23.818 INFO:tasks.workunit.client.1.smithi117.stdout:7/425: getdents d8/d9/da/d10/d23/d27 0 2024-03-22T06:15:23.818 INFO:tasks.workunit.client.1.smithi117.stdout:7/426: read - d8/d9/da/f69 zero size 2024-03-22T06:15:23.818 INFO:tasks.workunit.client.1.smithi117.stdout:7/427: fdatasync d8/d9/da/d10/d23/d27/d39/f3b 0 2024-03-22T06:15:23.823 INFO:tasks.workunit.client.1.smithi117.stdout:3/425: creat d9/d1a/d39/d4c/d99/f9a x:0 0 0 2024-03-22T06:15:23.823 INFO:tasks.workunit.client.1.smithi117.stdout:3/426: creat d9/d1a/d39/d7d/f9b x:0 0 0 2024-03-22T06:15:23.823 INFO:tasks.workunit.client.1.smithi117.stdout:3/427: read - d9/d1a/d39/f71 zero size 2024-03-22T06:15:23.823 INFO:tasks.workunit.client.1.smithi117.stdout:3/428: dread - d9/d1a/d39/d7d/f9b zero size 2024-03-22T06:15:23.823 INFO:tasks.workunit.client.1.smithi117.stdout:3/429: readlink d9/d1a/d39/d4c/d36/l6b 0 2024-03-22T06:15:23.823 INFO:tasks.workunit.client.1.smithi117.stdout:3/430: chown d9/d1a/d3a/l8e 20776389 1 2024-03-22T06:15:23.823 INFO:tasks.workunit.client.1.smithi117.stdout:3/431: stat d9/d1a/d39/d7d/f88 0 2024-03-22T06:15:23.825 INFO:tasks.workunit.client.1.smithi117.stdout:0/482: rename d1/d3/d2b/d3e/d3f/f9c to d1/d10/d5c/d40/d69/fa8 0 2024-03-22T06:15:23.825 INFO:tasks.workunit.client.1.smithi117.stdout:0/483: stat d1/d10/d18/d26/f75 0 2024-03-22T06:15:23.826 INFO:tasks.workunit.client.1.smithi117.stdout:9/477: mknod d4/d47/c8e 0 2024-03-22T06:15:23.828 INFO:tasks.workunit.client.1.smithi117.stdout:7/428: creat d8/d9/da/d10/d23/d27/d32/f81 x:0 0 0 2024-03-22T06:15:23.828 INFO:tasks.workunit.client.1.smithi117.stdout:7/429: stat d8/d9/d37/d5f/d74 0 2024-03-22T06:15:23.834 INFO:tasks.workunit.client.1.smithi117.stdout:3/432: mknod d9/d1a/d39/d8a/c9c 0 2024-03-22T06:15:23.834 INFO:tasks.workunit.client.1.smithi117.stdout:3/433: dread - d9/d1a/d39/d4c/d36/f75 zero size 2024-03-22T06:15:23.834 INFO:tasks.workunit.client.1.smithi117.stdout:3/434: write d9/d1a/d39/d7d/f98 [86989,12745] 0 2024-03-22T06:15:23.834 INFO:tasks.workunit.client.1.smithi117.stdout:3/435: chown d9/d1a/l33 3 1 2024-03-22T06:15:23.834 INFO:tasks.workunit.client.1.smithi117.stdout:9/478: dread d4/d7/f2a [0,4194304] 0 2024-03-22T06:15:23.834 INFO:tasks.workunit.client.1.smithi117.stdout:9/479: readlink d4/l68 0 2024-03-22T06:15:23.834 INFO:tasks.workunit.client.1.smithi117.stdout:9/480: dread - d4/d46/f6d zero size 2024-03-22T06:15:23.834 INFO:tasks.workunit.client.1.smithi117.stdout:9/481: creat d4/d8/d3e/f8f x:0 0 0 2024-03-22T06:15:23.835 INFO:tasks.workunit.client.1.smithi117.stdout:2/475: rename d2/c16 to d2/d5/d13/d3f/d1d/d3a/d71/d76/c91 0 2024-03-22T06:15:23.837 INFO:tasks.workunit.client.1.smithi117.stdout:0/484: mknod d1/d3/d2b/d3e/d3f/ca9 0 2024-03-22T06:15:23.837 INFO:tasks.workunit.client.1.smithi117.stdout:0/485: stat d1/d3/d2b/d3e/de/d1f 0 2024-03-22T06:15:23.837 INFO:tasks.workunit.client.1.smithi117.stdout:7/430: mkdir d8/d9/d37/d5f/d82 0 2024-03-22T06:15:23.839 INFO:tasks.workunit.client.1.smithi117.stdout:3/436: symlink d9/d1a/l9d 0 2024-03-22T06:15:23.842 INFO:tasks.workunit.client.1.smithi117.stdout:3/437: dread d9/d1a/d39/d4c/d36/d41/f94 [0,4194304] 0 2024-03-22T06:15:23.842 INFO:tasks.workunit.client.1.smithi117.stdout:3/438: chown d9/d1a/l1c 124557 1 2024-03-22T06:15:23.843 INFO:tasks.workunit.client.1.smithi117.stdout:4/441: rename d0/d19/d34/d43/d63/d6f/d6a/l8f to d0/d19/d34/d43/d63/d8d/l99 0 2024-03-22T06:15:23.843 INFO:tasks.workunit.client.1.smithi117.stdout:4/442: truncate d0/d19/d34/f4f 4326510 0 2024-03-22T06:15:23.843 INFO:tasks.workunit.client.1.smithi117.stdout:4/443: chown d0/f2d 1272 1 2024-03-22T06:15:23.845 INFO:tasks.workunit.client.1.smithi117.stdout:2/476: unlink d2/d5/d13/f19 0 2024-03-22T06:15:23.846 INFO:tasks.workunit.client.1.smithi117.stdout:0/486: write d1/d10/d18/d26/d49/d7e/d39/f52 [358248,107565] 0 2024-03-22T06:15:23.851 INFO:tasks.workunit.client.1.smithi117.stdout:8/418: sync 2024-03-22T06:15:23.851 INFO:tasks.workunit.client.1.smithi117.stdout:1/523: sync 2024-03-22T06:15:23.851 INFO:tasks.workunit.client.1.smithi117.stdout:1/524: write d1/d30/d35/d58/f9e [141192,10489] 0 2024-03-22T06:15:23.851 INFO:tasks.workunit.client.1.smithi117.stdout:9/482: dwrite d4/d46/f6d [0,4194304] 0 2024-03-22T06:15:23.853 INFO:tasks.workunit.client.1.smithi117.stdout:3/439: write d9/d1a/f2d [3210700,75004] 0 2024-03-22T06:15:23.858 INFO:tasks.workunit.client.1.smithi117.stdout:0/487: write d1/d10/d18/f38 [3933073,86580] 0 2024-03-22T06:15:23.859 INFO:tasks.workunit.client.1.smithi117.stdout:9/483: read d4/d8/f2c [294741,37091] 0 2024-03-22T06:15:23.859 INFO:tasks.workunit.client.1.smithi117.stdout:4/444: dwrite d0/d19/f1a [0,4194304] 0 2024-03-22T06:15:23.859 INFO:tasks.workunit.client.1.smithi117.stdout:2/477: write d2/d5/d13/f30 [263468,75109] 0 2024-03-22T06:15:23.859 INFO:tasks.workunit.client.1.smithi117.stdout:2/478: truncate d2/d5/d13/d3f/d1d/d3a/d11/d4f/f8c 239911 0 2024-03-22T06:15:23.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/525: dwrite d1/d1f/f8f [0,4194304] 0 2024-03-22T06:15:23.875 INFO:tasks.workunit.client.1.smithi117.stdout:0/488: link d1/d3/d2b/d3e/de/d1f/d3b/fa6 d1/d3/d2b/d3e/d3f/faa 0 2024-03-22T06:15:23.875 INFO:tasks.workunit.client.1.smithi117.stdout:0/489: creat d1/d10/d18/d26/d49/fab x:0 0 0 2024-03-22T06:15:23.877 INFO:tasks.workunit.client.1.smithi117.stdout:9/484: symlink d4/d7/d27/l90 0 2024-03-22T06:15:23.878 INFO:tasks.workunit.client.1.smithi117.stdout:3/440: dwrite d9/d1a/d3a/f56 [4194304,4194304] 0 2024-03-22T06:15:23.879 INFO:tasks.workunit.client.1.smithi117.stdout:4/445: creat d0/d19/d34/d43/d63/d6f/d5f/f9a x:0 0 0 2024-03-22T06:15:23.882 INFO:tasks.workunit.client.1.smithi117.stdout:2/479: symlink d2/d5/l92 0 2024-03-22T06:15:23.891 INFO:tasks.workunit.client.1.smithi117.stdout:6/515: rename d3/d40/d24 to d3/db4 0 2024-03-22T06:15:23.891 INFO:tasks.workunit.client.1.smithi117.stdout:6/516: readlink d3/db4/d2d/d42/la7 0 2024-03-22T06:15:23.899 INFO:tasks.workunit.client.1.smithi117.stdout:1/526: rmdir d1/d3c/d4d 39 2024-03-22T06:15:23.903 INFO:tasks.workunit.client.1.smithi117.stdout:9/485: symlink d4/d47/d85/l91 0 2024-03-22T06:15:23.903 INFO:tasks.workunit.client.1.smithi117.stdout:9/486: creat d4/d8/d37/d58/f92 x:0 0 0 2024-03-22T06:15:23.907 INFO:tasks.workunit.client.1.smithi117.stdout:3/441: mknod d9/d1a/d39/d4c/d36/c9e 0 2024-03-22T06:15:23.912 INFO:tasks.workunit.client.1.smithi117.stdout:2/480: creat d2/d5/d13/d3f/d1d/d3a/d89/f93 x:0 0 0 2024-03-22T06:15:23.914 INFO:tasks.workunit.client.1.smithi117.stdout:9/487: dread d4/d8/f1a [4194304,4194304] 0 2024-03-22T06:15:23.914 INFO:tasks.workunit.client.1.smithi117.stdout:5/498: rename dd/df/l84 to dd/df/d5e/d91/d72/d35/da2/lb1 0 2024-03-22T06:15:23.918 INFO:tasks.workunit.client.1.smithi117.stdout:6/517: mknod d3/db4/d2d/cb5 0 2024-03-22T06:15:23.919 INFO:tasks.workunit.client.1.smithi117.stdout:1/527: symlink d1/d30/d44/lbc 0 2024-03-22T06:15:23.926 INFO:tasks.workunit.client.1.smithi117.stdout:4/446: mkdir d0/d19/d9b 0 2024-03-22T06:15:23.928 INFO:tasks.workunit.client.1.smithi117.stdout:9/488: dread d4/d7/d27/f7d [0,4194304] 0 2024-03-22T06:15:23.929 INFO:tasks.workunit.client.1.smithi117.stdout:6/518: truncate d3/d3d/f2e 67865 0 2024-03-22T06:15:23.930 INFO:tasks.workunit.client.1.smithi117.stdout:4/447: symlink d0/d19/d28/l9c 0 2024-03-22T06:15:23.933 INFO:tasks.workunit.client.1.smithi117.stdout:3/442: dwrite d9/d1a/d39/d4c/f44 [0,4194304] 0 2024-03-22T06:15:23.935 INFO:tasks.workunit.client.1.smithi117.stdout:6/519: truncate d3/d40/f22 3760641 0 2024-03-22T06:15:23.935 INFO:tasks.workunit.client.1.smithi117.stdout:6/520: fdatasync d3/db4/d2d/d42/f76 0 2024-03-22T06:15:23.935 INFO:tasks.workunit.client.1.smithi117.stdout:9/489: getdents d4/d8/d3f 0 2024-03-22T06:15:23.935 INFO:tasks.workunit.client.1.smithi117.stdout:9/490: write d4/d7/f2d [2462067,104396] 0 2024-03-22T06:15:23.936 INFO:tasks.workunit.client.1.smithi117.stdout:5/499: dwrite dd/df/d5e/d91/f4f [0,4194304] 0 2024-03-22T06:15:23.938 INFO:tasks.workunit.client.1.smithi117.stdout:3/443: link d9/d1a/d39/d7d/f98 d9/d1a/d39/d4c/d36/f9f 0 2024-03-22T06:15:23.941 INFO:tasks.workunit.client.1.smithi117.stdout:6/521: creat d3/db4/d2d/d53/d59/fb6 x:0 0 0 2024-03-22T06:15:23.947 INFO:tasks.workunit.client.1.smithi117.stdout:5/500: dread dd/df/f2b [0,4194304] 0 2024-03-22T06:15:23.950 INFO:tasks.workunit.client.1.smithi117.stdout:6/522: write d3/db4/d2d/d53/d59/d74/da/de/f19 [5761145,71111] 0 2024-03-22T06:15:23.953 INFO:tasks.workunit.client.1.smithi117.stdout:1/528: dwrite d1/d14/f6c [0,4194304] 0 2024-03-22T06:15:23.953 INFO:tasks.workunit.client.1.smithi117.stdout:6/523: dread d3/db4/d2d/d53/d75/fa0 [0,4194304] 0 2024-03-22T06:15:23.953 INFO:tasks.workunit.client.1.smithi117.stdout:6/524: chown d3 482 1 2024-03-22T06:15:23.953 INFO:tasks.workunit.client.1.smithi117.stdout:6/525: chown d3/db4/d2d/d53/d75 0 1 2024-03-22T06:15:23.953 INFO:tasks.workunit.client.1.smithi117.stdout:3/444: creat d9/d55/fa0 x:0 0 0 2024-03-22T06:15:23.953 INFO:tasks.workunit.client.1.smithi117.stdout:3/445: readlink d9/l8d 0 2024-03-22T06:15:23.957 INFO:tasks.workunit.client.1.smithi117.stdout:9/491: getdents d4/d47/d64 0 2024-03-22T06:15:23.961 INFO:tasks.workunit.client.1.smithi117.stdout:4/448: dwrite d0/f6d [0,4194304] 0 2024-03-22T06:15:23.965 INFO:tasks.workunit.client.1.smithi117.stdout:1/529: dread d1/d22/d27/f34 [0,4194304] 0 2024-03-22T06:15:23.966 INFO:tasks.workunit.client.1.smithi117.stdout:3/446: mkdir d9/d1a/d39/d8a/da1 0 2024-03-22T06:15:23.968 INFO:tasks.workunit.client.1.smithi117.stdout:9/492: truncate d4/d8/f67 704434 0 2024-03-22T06:15:23.976 INFO:tasks.workunit.client.1.smithi117.stdout:6/526: dwrite d3/db4/d2d/d53/d59/d74/da/db3/fa4 [0,4194304] 0 2024-03-22T06:15:23.979 INFO:tasks.workunit.client.1.smithi117.stdout:4/449: dwrite d0/d4/d2a/d5b/f6e [0,4194304] 0 2024-03-22T06:15:23.980 INFO:tasks.workunit.client.1.smithi117.stdout:1/530: mknod d1/d3c/db6/cbd 0 2024-03-22T06:15:23.980 INFO:tasks.workunit.client.1.smithi117.stdout:1/531: chown d1/fd 159942 1 2024-03-22T06:15:23.980 INFO:tasks.workunit.client.1.smithi117.stdout:1/532: write d1/d30/fa8 [728586,67282] 0 2024-03-22T06:15:23.981 INFO:tasks.workunit.client.1.smithi117.stdout:3/447: mknod d9/d1a/d39/d4c/d99/ca2 0 2024-03-22T06:15:23.981 INFO:tasks.workunit.client.1.smithi117.stdout:3/448: chown d9/f30 206580 1 2024-03-22T06:15:23.984 INFO:tasks.workunit.client.1.smithi117.stdout:9/493: mkdir d4/d8/d93 0 2024-03-22T06:15:23.990 INFO:tasks.workunit.client.1.smithi117.stdout:6/527: dread d3/db4/d2d/d53/d59/d74/da/db3/fa4 [0,4194304] 0 2024-03-22T06:15:23.992 INFO:tasks.workunit.client.1.smithi117.stdout:7/431: sync 2024-03-22T06:15:23.994 INFO:tasks.workunit.client.1.smithi117.stdout:1/533: stat d1/d1f/d32/l53 0 2024-03-22T06:15:23.999 INFO:tasks.workunit.client.1.smithi117.stdout:9/494: symlink d4/d7/d27/d5b/l94 0 2024-03-22T06:15:23.999 INFO:tasks.workunit.client.1.smithi117.stdout:9/495: fdatasync d4/d8/d3e/d7a/f70 0 2024-03-22T06:15:24.000 INFO:tasks.workunit.client.1.smithi117.stdout:9/496: write d4/d8/d3e/d7a/f42 [4372358,49138] 0 2024-03-22T06:15:24.000 INFO:tasks.workunit.client.1.smithi117.stdout:3/449: dwrite d9/d1a/f20 [0,4194304] 0 2024-03-22T06:15:24.001 INFO:tasks.workunit.client.1.smithi117.stdout:9/497: truncate d4/d8/d3e/d7a/d2f/f54 179180 0 2024-03-22T06:15:24.002 INFO:tasks.workunit.client.1.smithi117.stdout:4/450: getdents d0/d4/d2a/d2f 0 2024-03-22T06:15:24.003 INFO:tasks.workunit.client.1.smithi117.stdout:6/528: getdents d3/d40/d46/d84 0 2024-03-22T06:15:24.003 INFO:tasks.workunit.client.1.smithi117.stdout:6/529: creat d3/db4/d2d/d53/d59/fb7 x:0 0 0 2024-03-22T06:15:24.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:23 smithi117 ceph-mon[36514]: pgmap v25: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 125 MiB/s rd, 146 MiB/s wr, 349 op/s 2024-03-22T06:15:24.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:23 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:24.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:23 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:24.009 INFO:tasks.workunit.client.1.smithi117.stdout:8/419: sync 2024-03-22T06:15:24.010 INFO:tasks.workunit.client.1.smithi117.stdout:0/490: sync 2024-03-22T06:15:24.010 INFO:tasks.workunit.client.1.smithi117.stdout:0/491: dread - d1/d3/d2b/d3e/de/d1f/d3b/d55/f91 zero size 2024-03-22T06:15:24.011 INFO:tasks.workunit.client.1.smithi117.stdout:9/498: write d4/d8/d3e/d7a/f70 [7657232,105702] 0 2024-03-22T06:15:24.011 INFO:tasks.workunit.client.1.smithi117.stdout:9/499: creat d4/d46/f95 x:0 0 0 2024-03-22T06:15:24.015 INFO:tasks.workunit.client.1.smithi117.stdout:3/450: mknod d9/d1a/d39/d8a/ca3 0 2024-03-22T06:15:24.015 INFO:tasks.workunit.client.1.smithi117.stdout:3/451: chown d9/d1a/d3a/l8f 1241944478 1 2024-03-22T06:15:24.016 INFO:tasks.workunit.client.1.smithi117.stdout:1/534: dwrite d1/d30/d50/f63 [0,4194304] 0 2024-03-22T06:15:24.017 INFO:tasks.workunit.client.1.smithi117.stdout:7/432: dwrite d8/d9/f1e [0,4194304] 0 2024-03-22T06:15:24.017 INFO:tasks.workunit.client.1.smithi117.stdout:7/433: chown d8/d50/d66 12 1 2024-03-22T06:15:24.023 INFO:tasks.workunit.client.1.smithi117.stdout:4/451: creat d0/d19/d34/d43/d63/d6f/d6a/f9d x:0 0 0 2024-03-22T06:15:24.023 INFO:tasks.workunit.client.1.smithi117.stdout:4/452: readlink d0/d19/l88 0 2024-03-22T06:15:24.027 INFO:tasks.workunit.client.1.smithi117.stdout:8/420: rmdir d1/d3a/d44/d16 39 2024-03-22T06:15:24.027 INFO:tasks.workunit.client.1.smithi117.stdout:8/421: write d1/d3a/d44/d11/f2a [617806,14473] 0 2024-03-22T06:15:24.027 INFO:tasks.workunit.client.1.smithi117.stdout:8/422: chown d1/d3a/d44/d16/l4f 23 1 2024-03-22T06:15:24.030 INFO:tasks.workunit.client.1.smithi117.stdout:6/530: dwrite d3/d40/d46/fa1 [0,4194304] 0 2024-03-22T06:15:24.030 INFO:tasks.workunit.client.1.smithi117.stdout:6/531: truncate d3/d40/d46/d84/f9c 1009514 0 2024-03-22T06:15:24.033 INFO:tasks.workunit.client.1.smithi117.stdout:1/535: creat d1/d1f/db2/fbe x:0 0 0 2024-03-22T06:15:24.033 INFO:tasks.workunit.client.1.smithi117.stdout:1/536: creat d1/d14/fbf x:0 0 0 2024-03-22T06:15:24.033 INFO:tasks.workunit.client.1.smithi117.stdout:1/537: chown d1/d1f/d32/c71 227384638 1 2024-03-22T06:15:24.037 INFO:tasks.workunit.client.1.smithi117.stdout:7/434: creat d8/d9/da/d10/d23/d27/d39/f83 x:0 0 0 2024-03-22T06:15:24.038 INFO:tasks.workunit.client.1.smithi117.stdout:7/435: truncate d8/d9/f1e 5226124 0 2024-03-22T06:15:24.038 INFO:tasks.workunit.client.1.smithi117.stdout:4/453: creat d0/d4/d2a/f9e x:0 0 0 2024-03-22T06:15:24.041 INFO:tasks.workunit.client.1.smithi117.stdout:8/423: mknod d1/d19/d38/c85 0 2024-03-22T06:15:24.041 INFO:tasks.workunit.client.1.smithi117.stdout:8/424: read d1/d19/f3f [3342579,126518] 0 2024-03-22T06:15:24.041 INFO:tasks.workunit.client.1.smithi117.stdout:8/425: stat d1 0 2024-03-22T06:15:24.041 INFO:tasks.workunit.client.1.smithi117.stdout:8/426: truncate d1/d3a/d44/f3c 1210488 0 2024-03-22T06:15:24.041 INFO:tasks.workunit.client.1.smithi117.stdout:8/427: chown d1/d19/d38/f42 11932 1 2024-03-22T06:15:24.043 INFO:tasks.workunit.client.1.smithi117.stdout:6/532: symlink d3/db4/d2d/d53/d59/d74/lb8 0 2024-03-22T06:15:24.043 INFO:tasks.workunit.client.1.smithi117.stdout:6/533: fdatasync d3/f52 0 2024-03-22T06:15:24.044 INFO:tasks.workunit.client.1.smithi117.stdout:9/500: dwrite d4/d8/d3e/d7a/d2f/f7b [0,4194304] 0 2024-03-22T06:15:24.045 INFO:tasks.workunit.client.1.smithi117.stdout:0/492: dwrite d1/d10/d18/d26/d49/d7e/d39/f52 [0,4194304] 0 2024-03-22T06:15:24.045 INFO:tasks.workunit.client.1.smithi117.stdout:7/436: rmdir d8/d9/d37/d5f/d74 0 2024-03-22T06:15:24.048 INFO:tasks.workunit.client.1.smithi117.stdout:8/428: creat d1/d19/f86 x:0 0 0 2024-03-22T06:15:24.049 INFO:tasks.workunit.client.1.smithi117.stdout:0/493: dread d1/d10/d18/d26/d49/d7e/f47 [0,4194304] 0 2024-03-22T06:15:24.050 INFO:tasks.workunit.client.1.smithi117.stdout:6/534: mknod d3/db4/d2d/d42/da2/cb9 0 2024-03-22T06:15:24.056 INFO:tasks.workunit.client.1.smithi117.stdout:8/429: creat d1/f87 x:0 0 0 2024-03-22T06:15:24.058 INFO:tasks.workunit.client.1.smithi117.stdout:0/494: mkdir d1/d10/d5c/d40/dac 0 2024-03-22T06:15:24.061 INFO:tasks.workunit.client.1.smithi117.stdout:6/535: creat d3/db4/d2d/d53/d59/d74/d28/d85/fba x:0 0 0 2024-03-22T06:15:24.061 INFO:tasks.workunit.client.1.smithi117.stdout:6/536: truncate d3/d40/fa9 287695 0 2024-03-22T06:15:24.063 INFO:tasks.workunit.client.1.smithi117.stdout:9/501: rename d4/d8/d3f/l73 to d4/l96 0 2024-03-22T06:15:24.063 INFO:tasks.workunit.client.1.smithi117.stdout:9/502: creat d4/d7/d27/f97 x:0 0 0 2024-03-22T06:15:24.066 INFO:tasks.workunit.client.1.smithi117.stdout:8/430: mknod d1/d28/d66/c88 0 2024-03-22T06:15:24.066 INFO:tasks.workunit.client.1.smithi117.stdout:2/481: sync 2024-03-22T06:15:24.066 INFO:tasks.workunit.client.1.smithi117.stdout:5/501: sync 2024-03-22T06:15:24.073 INFO:tasks.workunit.client.1.smithi117.stdout:0/495: write d1/d3/d2b/d3e/de/f23 [3323852,76311] 0 2024-03-22T06:15:24.073 INFO:tasks.workunit.client.1.smithi117.stdout:4/454: dwrite d0/d19/d34/d43/d63/d8d/f90 [0,4194304] 0 2024-03-22T06:15:24.073 INFO:tasks.workunit.client.1.smithi117.stdout:1/538: dwrite d1/d1f/f3e [0,4194304] 0 2024-03-22T06:15:24.074 INFO:tasks.workunit.client.1.smithi117.stdout:7/437: dwrite d8/d9/da/d10/d23/d27/d39/f7b [0,4194304] 0 2024-03-22T06:15:24.074 INFO:tasks.workunit.client.1.smithi117.stdout:6/537: mknod d3/db4/d2d/d42/da2/cbb 0 2024-03-22T06:15:24.074 INFO:tasks.workunit.client.1.smithi117.stdout:3/452: dwrite d9/d1a/d39/d4c/f44 [0,4194304] 0 2024-03-22T06:15:24.074 INFO:tasks.workunit.client.1.smithi117.stdout:3/453: chown d9/d1a/d3a 4 1 2024-03-22T06:15:24.078 INFO:tasks.workunit.client.1.smithi117.stdout:8/431: rename d1/d3a/d44/d4b/c52 to d1/d3a/d44/d16/c89 0 2024-03-22T06:15:24.078 INFO:tasks.workunit.client.1.smithi117.stdout:8/432: write d1/d3a/d44/d16/f7a [552294,46691] 0 2024-03-22T06:15:24.078 INFO:tasks.workunit.client.1.smithi117.stdout:8/433: fdatasync d1/d19/f43 0 2024-03-22T06:15:24.078 INFO:tasks.workunit.client.1.smithi117.stdout:8/434: fdatasync d1/d3a/d44/d9/f13 0 2024-03-22T06:15:24.079 INFO:tasks.workunit.client.1.smithi117.stdout:0/496: mkdir d1/d3/d2b/d3e/dad 0 2024-03-22T06:15:24.080 INFO:tasks.workunit.client.1.smithi117.stdout:0/497: dread - d1/d10/d5c/d40/d69/fa2 zero size 2024-03-22T06:15:24.083 INFO:tasks.workunit.client.1.smithi117.stdout:4/455: truncate d0/f6d 604035 0 2024-03-22T06:15:24.085 INFO:tasks.workunit.client.1.smithi117.stdout:6/538: symlink d3/db4/d2d/d42/da2/lbc 0 2024-03-22T06:15:24.086 INFO:tasks.workunit.client.1.smithi117.stdout:3/454: symlink d9/la4 0 2024-03-22T06:15:24.087 INFO:tasks.workunit.client.1.smithi117.stdout:6/539: dread d3/db4/d2d/d53/d59/d74/da/de/f47 [0,4194304] 0 2024-03-22T06:15:24.087 INFO:tasks.workunit.client.1.smithi117.stdout:6/540: write d3/d3d/f2e [273298,111659] 0 2024-03-22T06:15:24.089 INFO:tasks.workunit.client.1.smithi117.stdout:8/435: dread d1/d19/f2d [0,4194304] 0 2024-03-22T06:15:24.089 INFO:tasks.workunit.client.1.smithi117.stdout:8/436: chown d1/d3a/d44/d11/f5a 0 1 2024-03-22T06:15:24.089 INFO:tasks.workunit.client.1.smithi117.stdout:6/541: dread d3/db4/d2d/d42/da2/f7a [4194304,4194304] 0 2024-03-22T06:15:24.094 INFO:tasks.workunit.client.1.smithi117.stdout:5/502: dwrite dd/f29 [0,4194304] 0 2024-03-22T06:15:24.095 INFO:tasks.workunit.client.1.smithi117.stdout:9/503: dwrite d4/d7/f24 [0,4194304] 0 2024-03-22T06:15:24.095 INFO:tasks.workunit.client.1.smithi117.stdout:9/504: truncate d4/d7/d27/f5a 916245 0 2024-03-22T06:15:24.098 INFO:tasks.workunit.client.1.smithi117.stdout:1/539: rename d1/d62/ca0 to d1/d62/d7c/cc0 0 2024-03-22T06:15:24.098 INFO:tasks.workunit.client.1.smithi117.stdout:1/540: fdatasync d1/fd 0 2024-03-22T06:15:24.098 INFO:tasks.workunit.client.1.smithi117.stdout:1/541: readlink d1/d62/d7c/la2 0 2024-03-22T06:15:24.102 INFO:tasks.workunit.client.1.smithi117.stdout:0/498: mkdir d1/d3/dae 0 2024-03-22T06:15:24.103 INFO:tasks.workunit.client.1.smithi117.stdout:6/542: creat d3/db4/d2d/d53/d59/d74/da/fbd x:0 0 0 2024-03-22T06:15:24.106 INFO:tasks.workunit.client.1.smithi117.stdout:5/503: creat dd/df/d1e/fb2 x:0 0 0 2024-03-22T06:15:24.108 INFO:tasks.workunit.client.1.smithi117.stdout:3/455: dwrite d9/d1a/d39/d4c/d99/f9a [0,4194304] 0 2024-03-22T06:15:24.111 INFO:tasks.workunit.client.1.smithi117.stdout:1/542: mkdir d1/d22/d27/d4e/d92/dc1 0 2024-03-22T06:15:24.111 INFO:tasks.workunit.client.1.smithi117.stdout:0/499: symlink d1/d3/d2b/d3e/de/d1f/laf 0 2024-03-22T06:15:24.114 INFO:tasks.workunit.client.1.smithi117.stdout:6/543: mkdir d3/d40/d46/d84/dbe 0 2024-03-22T06:15:24.114 INFO:tasks.workunit.client.1.smithi117.stdout:6/544: dread - d3/db4/d2d/d53/d59/fb7 zero size 2024-03-22T06:15:24.119 INFO:tasks.workunit.client.1.smithi117.stdout:3/456: write d9/fa [481805,114068] 0 2024-03-22T06:15:24.119 INFO:tasks.workunit.client.1.smithi117.stdout:3/457: readlink d9/d1a/l4a 0 2024-03-22T06:15:24.120 INFO:tasks.workunit.client.1.smithi117.stdout:3/458: truncate d9/d1a/d39/d4c/d36/f75 825183 0 2024-03-22T06:15:24.120 INFO:tasks.workunit.client.1.smithi117.stdout:3/459: write d9/d1a/d39/d7d/f98 [983285,105430] 0 2024-03-22T06:15:24.126 INFO:tasks.workunit.client.1.smithi117.stdout:8/437: dwrite d1/d19/f51 [8388608,4194304] 0 2024-03-22T06:15:24.126 INFO:tasks.workunit.client.1.smithi117.stdout:8/438: chown d1/d3a/d44/f1d 57513715 1 2024-03-22T06:15:24.127 INFO:tasks.workunit.client.1.smithi117.stdout:7/438: rename d8/d9/d37/c53 to d8/d9/da/d10/c84 0 2024-03-22T06:15:24.127 INFO:tasks.workunit.client.1.smithi117.stdout:4/456: rename d0/d19/d28 to d0/d19/d28/d47/d9f 22 2024-03-22T06:15:24.127 INFO:tasks.workunit.client.1.smithi117.stdout:4/457: truncate d0/d19/d34/f77 406061 0 2024-03-22T06:15:24.128 INFO:tasks.workunit.client.1.smithi117.stdout:1/543: dwrite d1/d3c/d4d/d5b/f91 [0,4194304] 0 2024-03-22T06:15:24.134 INFO:tasks.workunit.client.1.smithi117.stdout:5/504: getdents dd 0 2024-03-22T06:15:24.138 INFO:tasks.workunit.client.1.smithi117.stdout:3/460: mknod d9/d1a/d39/d82/ca5 0 2024-03-22T06:15:24.138 INFO:tasks.workunit.client.1.smithi117.stdout:3/461: write d9/d66/f74 [653672,83698] 0 2024-03-22T06:15:24.139 INFO:tasks.workunit.client.1.smithi117.stdout:6/545: dwrite d3/db4/d2d/d53/d59/d74/da/db3/fa4 [4194304,4194304] 0 2024-03-22T06:15:24.139 INFO:tasks.workunit.client.1.smithi117.stdout:8/439: getdents d1/d3a/d44/d4b 0 2024-03-22T06:15:24.139 INFO:tasks.workunit.client.1.smithi117.stdout:8/440: readlink d1/d3a/d44/d11/l4e 0 2024-03-22T06:15:24.139 INFO:tasks.workunit.client.1.smithi117.stdout:6/546: write d3/d40/d46/d84/f9c [332875,91853] 0 2024-03-22T06:15:24.141 INFO:tasks.workunit.client.1.smithi117.stdout:7/439: dwrite d8/fb [4194304,4194304] 0 2024-03-22T06:15:24.143 INFO:tasks.workunit.client.1.smithi117.stdout:0/500: rename d1/d10/d5c/d40/d69/d8f to d1/d10/d18/db0 0 2024-03-22T06:15:24.147 INFO:tasks.workunit.client.1.smithi117.stdout:4/458: mkdir d0/d48/da0 0 2024-03-22T06:15:24.147 INFO:tasks.workunit.client.1.smithi117.stdout:4/459: chown d0/d19/d34/d43/d63/d6f/d6a/f8c 333034 1 2024-03-22T06:15:24.148 INFO:tasks.workunit.client.1.smithi117.stdout:4/460: getdents d0/d48/da0 0 2024-03-22T06:15:24.148 INFO:tasks.workunit.client.1.smithi117.stdout:4/461: write d0/f83 [4402982,27756] 0 2024-03-22T06:15:24.148 INFO:tasks.workunit.client.1.smithi117.stdout:4/462: chown d0/d19/d34/d43/f7c 15254 1 2024-03-22T06:15:24.148 INFO:tasks.workunit.client.1.smithi117.stdout:6/547: dread d3/f3f [0,4194304] 0 2024-03-22T06:15:24.152 INFO:tasks.workunit.client.1.smithi117.stdout:7/440: dread d8/d9/da/d10/f19 [0,4194304] 0 2024-03-22T06:15:24.152 INFO:tasks.workunit.client.1.smithi117.stdout:7/441: write d8/d9/d37/d5f/f79 [1012636,43400] 0 2024-03-22T06:15:24.153 INFO:tasks.workunit.client.1.smithi117.stdout:1/544: readlink d1/l4 0 2024-03-22T06:15:24.153 INFO:tasks.workunit.client.1.smithi117.stdout:1/545: chown d1/d84/d77/c8c 9080 1 2024-03-22T06:15:24.154 INFO:tasks.workunit.client.1.smithi117.stdout:5/505: getdents dd/df/d5e/d91/d72/d60 0 2024-03-22T06:15:24.154 INFO:tasks.workunit.client.1.smithi117.stdout:5/506: dread - dd/df/d1e/fb2 zero size 2024-03-22T06:15:24.158 INFO:tasks.workunit.client.1.smithi117.stdout:8/441: unlink d1/d3a/d44/c6 0 2024-03-22T06:15:24.158 INFO:tasks.workunit.client.1.smithi117.stdout:8/442: write f0 [71741,73603] 0 2024-03-22T06:15:24.160 INFO:tasks.workunit.client.1.smithi117.stdout:7/442: dread d8/d9/da/d10/d23/d27/d39/f5d [0,4194304] 0 2024-03-22T06:15:24.162 INFO:tasks.workunit.client.1.smithi117.stdout:4/463: dwrite d0/f11 [0,4194304] 0 2024-03-22T06:15:24.162 INFO:tasks.workunit.client.1.smithi117.stdout:4/464: fdatasync d0/d19/f22 0 2024-03-22T06:15:24.188 INFO:tasks.workunit.client.1.smithi117.stdout:3/462: rename d9/d1a/f2d to d9/fa6 0 2024-03-22T06:15:24.188 INFO:tasks.workunit.client.1.smithi117.stdout:3/463: write d9/d66/f74 [46181,114252] 0 2024-03-22T06:15:24.189 INFO:tasks.workunit.client.1.smithi117.stdout:0/501: mknod d1/d10/d18/d26/d49/cb1 0 2024-03-22T06:15:24.191 INFO:tasks.workunit.client.1.smithi117.stdout:2/482: sync 2024-03-22T06:15:24.191 INFO:tasks.workunit.client.1.smithi117.stdout:2/483: write d2/d5/db/f24 [5050181,26926] 0 2024-03-22T06:15:24.191 INFO:tasks.workunit.client.1.smithi117.stdout:6/548: mkdir d3/d40/d46/d84/d49/d79/d9d/dbf 0 2024-03-22T06:15:24.192 INFO:tasks.workunit.client.1.smithi117.stdout:1/546: symlink d1/d3c/d4d/d5b/lc2 0 2024-03-22T06:15:24.194 INFO:tasks.workunit.client.1.smithi117.stdout:8/443: unlink d1/d28/d6b/c72 0 2024-03-22T06:15:24.195 INFO:tasks.workunit.client.1.smithi117.stdout:8/444: chown d1/d3a 426 1 2024-03-22T06:15:24.195 INFO:tasks.workunit.client.1.smithi117.stdout:8/445: dread d1/f77 [0,4194304] 0 2024-03-22T06:15:24.202 INFO:tasks.workunit.client.1.smithi117.stdout:3/464: creat d9/d1a/fa7 x:0 0 0 2024-03-22T06:15:24.205 INFO:tasks.workunit.client.1.smithi117.stdout:0/502: link d1/d10/d5c/d40/d4e/f79 d1/d3/d2b/fb2 0 2024-03-22T06:15:24.205 INFO:tasks.workunit.client.1.smithi117.stdout:0/503: creat d1/d10/d18/d26/d49/d7e/d39/d6a/fb3 x:0 0 0 2024-03-22T06:15:24.205 INFO:tasks.workunit.client.1.smithi117.stdout:0/504: truncate d1/d10/d18/d26/d49/d7e/d39/f90 331230 0 2024-03-22T06:15:24.208 INFO:tasks.workunit.client.1.smithi117.stdout:2/484: symlink d2/d5/l94 0 2024-03-22T06:15:24.208 INFO:tasks.workunit.client.1.smithi117.stdout:2/485: creat d2/d1b/d39/d77/f95 x:0 0 0 2024-03-22T06:15:24.208 INFO:tasks.workunit.client.1.smithi117.stdout:8/446: dwrite d1/d28/d66/f74 [0,4194304] 0 2024-03-22T06:15:24.209 INFO:tasks.workunit.client.1.smithi117.stdout:6/549: getdents d3/db4/d2d 0 2024-03-22T06:15:24.212 INFO:tasks.workunit.client.1.smithi117.stdout:1/547: unlink d1/d3c/d4d/d5b/f61 0 2024-03-22T06:15:24.212 INFO:tasks.workunit.client.1.smithi117.stdout:1/548: chown d1/d22/d27/faf 0 1 2024-03-22T06:15:24.212 INFO:tasks.workunit.client.1.smithi117.stdout:9/505: sync 2024-03-22T06:15:24.212 INFO:tasks.workunit.client.1.smithi117.stdout:1/549: truncate d1/d22/d27/d4e/f6a 416956 0 2024-03-22T06:15:24.214 INFO:tasks.workunit.client.1.smithi117.stdout:5/507: rename dd/d16/d5a/d66/laf to dd/df/lb3 0 2024-03-22T06:15:24.215 INFO:tasks.workunit.client.1.smithi117.stdout:5/508: chown dd/d16/d1a/d32/d73/c80 2 1 2024-03-22T06:15:24.216 INFO:tasks.workunit.client.1.smithi117.stdout:5/509: read dd/d16/d5a/f97 [350362,52285] 0 2024-03-22T06:15:24.216 INFO:tasks.workunit.client.1.smithi117.stdout:3/465: link d9/fa6 d9/d1a/d39/d4c/d37/d69/fa8 0 2024-03-22T06:15:24.220 INFO:tasks.workunit.client.1.smithi117.stdout:0/505: mknod d1/d10/d5c/d40/d69/d72/cb4 0 2024-03-22T06:15:24.220 INFO:tasks.workunit.client.1.smithi117.stdout:0/506: chown d1/d10/d18/d26/d49/d7e/d39/d6a/l80 112792 1 2024-03-22T06:15:24.222 INFO:tasks.workunit.client.1.smithi117.stdout:2/486: mkdir d2/d5/d13/d3f/d1d/d3a/d89/d8d/d96 0 2024-03-22T06:15:24.226 INFO:tasks.workunit.client.1.smithi117.stdout:6/550: mknod d3/d40/d46/d84/d49/d79/d9d/cc0 0 2024-03-22T06:15:24.227 INFO:tasks.workunit.client.1.smithi117.stdout:7/443: rename d8/d9/da/d10/d23/c30 to d8/d9/da/d10/d23/d27/d39/d5c/c85 0 2024-03-22T06:15:24.227 INFO:tasks.workunit.client.1.smithi117.stdout:7/444: write d8/d9/da/d2b/f6e [5221165,72443] 0 2024-03-22T06:15:24.229 INFO:tasks.workunit.client.1.smithi117.stdout:3/466: creat d9/d66/fa9 x:0 0 0 2024-03-22T06:15:24.234 INFO:tasks.workunit.client.1.smithi117.stdout:7/445: dread d8/d9/da/f4b [0,4194304] 0 2024-03-22T06:15:24.237 INFO:tasks.workunit.client.1.smithi117.stdout:3/467: write d9/d1a/d39/d4c/d99/f9a [2484508,118050] 0 2024-03-22T06:15:24.237 INFO:tasks.workunit.client.1.smithi117.stdout:3/468: write d9/d55/f7c [476672,17293] 0 2024-03-22T06:15:24.241 INFO:tasks.workunit.client.1.smithi117.stdout:6/551: creat d3/db4/d2d/d53/d75/fc1 x:0 0 0 2024-03-22T06:15:24.241 INFO:tasks.workunit.client.1.smithi117.stdout:6/552: chown d3/db4/d2d/d53/d75/l64 2373 1 2024-03-22T06:15:24.244 INFO:tasks.workunit.client.1.smithi117.stdout:4/465: rename d0/d19/d34/d43/d63/d6f/l65 to d0/d4/d2a/d2f/la1 0 2024-03-22T06:15:24.246 INFO:tasks.workunit.client.1.smithi117.stdout:0/507: getdents d1/d10/d5c/d40/d69 0 2024-03-22T06:15:24.246 INFO:tasks.workunit.client.1.smithi117.stdout:0/508: stat d1/d10/d18/d26/d49/f77 0 2024-03-22T06:15:24.247 INFO:tasks.workunit.client.1.smithi117.stdout:7/446: mknod d8/d9/da/d10/d23/d27/d32/c86 0 2024-03-22T06:15:24.252 INFO:tasks.workunit.client.1.smithi117.stdout:3/469: creat d9/faa x:0 0 0 2024-03-22T06:15:24.254 INFO:tasks.workunit.client.1.smithi117.stdout:8/447: rename d1/d3a/d44/c56 to d1/d3a/d44/d11/c8a 0 2024-03-22T06:15:24.254 INFO:tasks.workunit.client.1.smithi117.stdout:8/448: write d1/d28/f7d [813454,45802] 0 2024-03-22T06:15:24.258 INFO:tasks.workunit.client.1.smithi117.stdout:8/449: dread d1/d19/d38/f70 [0,4194304] 0 2024-03-22T06:15:24.258 INFO:tasks.workunit.client.1.smithi117.stdout:0/509: creat d1/d10/d18/d8c/fb5 x:0 0 0 2024-03-22T06:15:24.258 INFO:tasks.workunit.client.1.smithi117.stdout:0/510: chown d1/d10/d18/d26/d53/l5b 6515381 1 2024-03-22T06:15:24.258 INFO:tasks.workunit.client.1.smithi117.stdout:7/447: symlink d8/d50/l87 0 2024-03-22T06:15:24.258 INFO:tasks.workunit.client.1.smithi117.stdout:4/466: dread d0/f1d [0,4194304] 0 2024-03-22T06:15:24.258 INFO:tasks.workunit.client.1.smithi117.stdout:7/448: truncate d8/d9/da/d10/d23/d27/d39/f83 698283 0 2024-03-22T06:15:24.258 INFO:tasks.workunit.client.1.smithi117.stdout:4/467: dread - d0/d19/d34/d43/d63/d6f/d6a/f8c zero size 2024-03-22T06:15:24.259 INFO:tasks.workunit.client.1.smithi117.stdout:1/550: rename d1/d22/l3b to d1/d3c/d4d/lc3 0 2024-03-22T06:15:24.259 INFO:tasks.workunit.client.1.smithi117.stdout:8/450: mknod d1/d28/d66/d7f/c8b 0 2024-03-22T06:15:24.259 INFO:tasks.workunit.client.1.smithi117.stdout:1/551: write d1/d30/fa3 [200582,24751] 0 2024-03-22T06:15:24.260 INFO:tasks.workunit.client.1.smithi117.stdout:8/451: creat d1/d19/d38/f8c x:0 0 0 2024-03-22T06:15:24.260 INFO:tasks.workunit.client.1.smithi117.stdout:1/552: fdatasync d1/f7 0 2024-03-22T06:15:24.260 INFO:tasks.workunit.client.1.smithi117.stdout:1/553: write d1/d30/d44/f7b [44494,75977] 0 2024-03-22T06:15:24.260 INFO:tasks.workunit.client.1.smithi117.stdout:1/554: creat d1/d30/fc4 x:0 0 0 2024-03-22T06:15:24.260 INFO:tasks.workunit.client.1.smithi117.stdout:0/511: mkdir d1/d10/d18/d26/db6 0 2024-03-22T06:15:24.263 INFO:tasks.workunit.client.1.smithi117.stdout:7/449: write d8/d9/d37/d45/d61/f71 [27176,106884] 0 2024-03-22T06:15:24.263 INFO:tasks.workunit.client.1.smithi117.stdout:9/506: rename d4/d8/d3e/d7a/d3b/c66 to d4/d7/c98 0 2024-03-22T06:15:24.267 INFO:tasks.workunit.client.1.smithi117.stdout:6/553: dwrite d3/db4/f5e [0,4194304] 0 2024-03-22T06:15:24.273 INFO:tasks.workunit.client.1.smithi117.stdout:3/470: dwrite d9/f18 [4194304,4194304] 0 2024-03-22T06:15:24.273 INFO:tasks.workunit.client.1.smithi117.stdout:1/555: creat d1/d30/d35/fc5 x:0 0 0 2024-03-22T06:15:24.273 INFO:tasks.workunit.client.1.smithi117.stdout:1/556: fsync d1/d30/fb4 0 2024-03-22T06:15:24.275 INFO:tasks.workunit.client.1.smithi117.stdout:0/512: mknod d1/d10/d18/d26/db6/cb7 0 2024-03-22T06:15:24.275 INFO:tasks.workunit.client.1.smithi117.stdout:0/513: readlink d1/d10/d18/l1e 0 2024-03-22T06:15:24.275 INFO:tasks.workunit.client.1.smithi117.stdout:0/514: fdatasync d1/d10/d18/d26/d49/fab 0 2024-03-22T06:15:24.279 INFO:tasks.workunit.client.1.smithi117.stdout:4/468: getdents d0/d19/d1c 0 2024-03-22T06:15:24.283 INFO:tasks.workunit.client.1.smithi117.stdout:7/450: mkdir d8/d9/d37/d45/d88 0 2024-03-22T06:15:24.285 INFO:tasks.workunit.client.1.smithi117.stdout:2/487: rename d2/f69 to d2/d5/d13/d3f/f97 0 2024-03-22T06:15:24.287 INFO:tasks.workunit.client.1.smithi117.stdout:6/554: truncate d3/db4/d2d/d53/d59/d74/f11 381517 0 2024-03-22T06:15:24.292 INFO:tasks.workunit.client.1.smithi117.stdout:9/507: dwrite d4/d8/f41 [0,4194304] 0 2024-03-22T06:15:24.293 INFO:tasks.workunit.client.1.smithi117.stdout:3/471: dwrite d9/fa6 [4194304,4194304] 0 2024-03-22T06:15:24.297 INFO:tasks.workunit.client.1.smithi117.stdout:7/451: dread d8/d9/da/d10/f25 [4194304,4194304] 0 2024-03-22T06:15:24.297 INFO:tasks.workunit.client.1.smithi117.stdout:7/452: write d8/d9/da/d10/d23/d27/d39/f5d [3768935,49957] 0 2024-03-22T06:15:24.298 INFO:tasks.workunit.client.1.smithi117.stdout:1/557: mkdir d1/d1f/d32/dc6 0 2024-03-22T06:15:24.301 INFO:tasks.workunit.client.1.smithi117.stdout:0/515: creat d1/d3/d2b/d3e/de/fb8 x:0 0 0 2024-03-22T06:15:24.305 INFO:tasks.workunit.client.1.smithi117.stdout:8/452: rename d1/d3a/d44/d11/f5a to d1/d3a/d44/f8d 0 2024-03-22T06:15:24.305 INFO:tasks.workunit.client.1.smithi117.stdout:8/453: fdatasync d1/d3a/d44/d9/f81 0 2024-03-22T06:15:24.306 INFO:tasks.workunit.client.1.smithi117.stdout:2/488: mkdir d2/d35/d38/d98 0 2024-03-22T06:15:24.306 INFO:tasks.workunit.client.1.smithi117.stdout:2/489: creat d2/d1b/d39/d77/f99 x:0 0 0 2024-03-22T06:15:24.308 INFO:tasks.workunit.client.1.smithi117.stdout:6/555: rmdir d3/d40/d46/d84/d49/d79/d9d 39 2024-03-22T06:15:24.314 INFO:tasks.workunit.client.1.smithi117.stdout:5/510: sync 2024-03-22T06:15:24.314 INFO:tasks.workunit.client.1.smithi117.stdout:5/511: chown dd/df/d5e/d91/d72/d35/da2 1700 1 2024-03-22T06:15:24.317 INFO:tasks.workunit.client.1.smithi117.stdout:0/516: dread d1/d3/d2b/d3e/de/fa4 [0,4194304] 0 2024-03-22T06:15:24.317 INFO:tasks.workunit.client.1.smithi117.stdout:3/472: dwrite d9/d1a/d39/d4c/d37/d69/f87 [0,4194304] 0 2024-03-22T06:15:24.317 INFO:tasks.workunit.client.1.smithi117.stdout:0/517: fsync d1/d10/f32 0 2024-03-22T06:15:24.317 INFO:tasks.workunit.client.1.smithi117.stdout:3/473: write d9/d1a/d39/d7d/f9b [726852,34734] 0 2024-03-22T06:15:24.321 INFO:tasks.workunit.client.1.smithi117.stdout:8/454: dwrite d1/d19/d38/f70 [0,4194304] 0 2024-03-22T06:15:24.321 INFO:tasks.workunit.client.1.smithi117.stdout:7/453: truncate d8/d9/d37/d45/d61/f6f 510805 0 2024-03-22T06:15:24.327 INFO:tasks.workunit.client.1.smithi117.stdout:1/558: rename d1/d62/d7c/laa to d1/d1f/db2/lc7 0 2024-03-22T06:15:24.332 INFO:tasks.workunit.client.1.smithi117.stdout:2/490: mkdir d2/d9a 0 2024-03-22T06:15:24.334 INFO:tasks.workunit.client.1.smithi117.stdout:9/508: rmdir d4/d8/d3e/d7a/d2f 39 2024-03-22T06:15:24.341 INFO:tasks.workunit.client.1.smithi117.stdout:0/518: unlink d1/d3/d2b/d3e/de/l9e 0 2024-03-22T06:15:24.341 INFO:tasks.workunit.client.1.smithi117.stdout:0/519: dread - d1/d10/d18/d26/d53/f82 zero size 2024-03-22T06:15:24.345 INFO:tasks.workunit.client.1.smithi117.stdout:5/512: dwrite dd/df/d5e/d91/f23 [0,4194304] 0 2024-03-22T06:15:24.348 INFO:tasks.workunit.client.1.smithi117.stdout:8/455: symlink d1/d28/l8e 0 2024-03-22T06:15:24.351 INFO:tasks.workunit.client.1.smithi117.stdout:6/556: rename d3/db4/d2d/d53/d59/d74/da/db3/c37 to d3/d40/d46/d84/d49/d79/cc2 0 2024-03-22T06:15:24.351 INFO:tasks.workunit.client.1.smithi117.stdout:6/557: write d3/db4/d2d/d42/f76 [233612,21892] 0 2024-03-22T06:15:24.352 INFO:tasks.workunit.client.1.smithi117.stdout:2/491: mknod d2/d35/d38/c9b 0 2024-03-22T06:15:24.355 INFO:tasks.workunit.client.1.smithi117.stdout:0/520: creat d1/d10/d18/d26/db6/fb9 x:0 0 0 2024-03-22T06:15:24.357 INFO:tasks.workunit.client.1.smithi117.stdout:3/474: dwrite d9/d1a/d3a/f47 [0,4194304] 0 2024-03-22T06:15:24.362 INFO:tasks.workunit.client.1.smithi117.stdout:8/456: rmdir d1/d3a/d44/d16/d68 0 2024-03-22T06:15:24.362 INFO:tasks.workunit.client.1.smithi117.stdout:8/457: fsync d1/d19/f2d 0 2024-03-22T06:15:24.363 INFO:tasks.workunit.client.1.smithi117.stdout:2/492: dread d2/d5/d13/d3f/d1d/d3a/d11/d4f/f62 [0,4194304] 0 2024-03-22T06:15:24.363 INFO:tasks.workunit.client.1.smithi117.stdout:0/521: truncate d1/d10/d18/f38 859421 0 2024-03-22T06:15:24.364 INFO:tasks.workunit.client.1.smithi117.stdout:9/509: dwrite d4/d8/d3e/d7a/f63 [0,4194304] 0 2024-03-22T06:15:24.364 INFO:tasks.workunit.client.1.smithi117.stdout:0/522: read d1/d10/d18/d26/d49/d7e/f25 [1357323,71180] 0 2024-03-22T06:15:24.364 INFO:tasks.workunit.client.1.smithi117.stdout:7/454: dwrite d8/d9/da/d10/d23/d27/d32/f4d [0,4194304] 0 2024-03-22T06:15:24.369 INFO:tasks.workunit.client.1.smithi117.stdout:3/475: mknod d9/d1a/d39/d8a/cab 0 2024-03-22T06:15:24.373 INFO:tasks.workunit.client.1.smithi117.stdout:2/493: unlink d2/d5/d41/f6f 0 2024-03-22T06:15:24.374 INFO:tasks.workunit.client.1.smithi117.stdout:6/558: dwrite d3/f4 [0,4194304] 0 2024-03-22T06:15:24.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/458: write d1/d19/d38/f70 [827061,113974] 0 2024-03-22T06:15:24.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/459: dread - d1/d3a/d44/d9/f81 zero size 2024-03-22T06:15:24.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/460: creat d1/d3a/d44/d9/f8f x:0 0 0 2024-03-22T06:15:24.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/461: fsync d1/d19/f3f 0 2024-03-22T06:15:24.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/462: read - d1/d19/f86 zero size 2024-03-22T06:15:24.377 INFO:tasks.workunit.client.1.smithi117.stdout:9/510: write d4/d7/f26 [2776831,104161] 0 2024-03-22T06:15:24.377 INFO:tasks.workunit.client.1.smithi117.stdout:9/511: readlink d4/d8/d3e/d7a/d3b/l43 0 2024-03-22T06:15:24.377 INFO:tasks.workunit.client.1.smithi117.stdout:5/513: dwrite dd/f6d [0,4194304] 0 2024-03-22T06:15:24.377 INFO:tasks.workunit.client.1.smithi117.stdout:5/514: write dd/df/d5e/d91/d72/d60/f6b [876775,51023] 0 2024-03-22T06:15:24.383 INFO:tasks.workunit.client.1.smithi117.stdout:8/463: dread d1/d19/f64 [0,4194304] 0 2024-03-22T06:15:24.383 INFO:tasks.workunit.client.1.smithi117.stdout:8/464: truncate d1/d19/f86 288536 0 2024-03-22T06:15:24.390 INFO:tasks.workunit.client.1.smithi117.stdout:0/523: unlink d1/d3/d2b/d3e/de/d1f/d59/c63 0 2024-03-22T06:15:24.393 INFO:tasks.workunit.client.1.smithi117.stdout:7/455: symlink d8/d9/da/d10/d23/d48/l89 0 2024-03-22T06:15:24.393 INFO:tasks.workunit.client.1.smithi117.stdout:7/456: dread - d8/d9/da/f72 zero size 2024-03-22T06:15:24.393 INFO:tasks.workunit.client.1.smithi117.stdout:4/469: sync 2024-03-22T06:15:24.395 INFO:tasks.workunit.client.1.smithi117.stdout:3/476: symlink d9/d93/lac 0 2024-03-22T06:15:24.396 INFO:tasks.workunit.client.1.smithi117.stdout:3/477: chown d9/d1a/f1e 15248 1 2024-03-22T06:15:24.396 INFO:tasks.workunit.client.1.smithi117.stdout:2/494: symlink d2/d1b/d39/d45/d7c/l9c 0 2024-03-22T06:15:24.396 INFO:tasks.workunit.client.1.smithi117.stdout:2/495: write d2/d5/db/f72 [2699658,8631] 0 2024-03-22T06:15:24.400 INFO:tasks.workunit.client.1.smithi117.stdout:8/465: dwrite d1/d3a/d44/d11/f2a [0,4194304] 0 2024-03-22T06:15:24.400 INFO:tasks.workunit.client.1.smithi117.stdout:8/466: write d1/d19/d38/f8c [525102,54360] 0 2024-03-22T06:15:24.400 INFO:tasks.workunit.client.1.smithi117.stdout:8/467: chown d1/d3a/d44/d9/f8f 0 1 2024-03-22T06:15:24.400 INFO:tasks.workunit.client.1.smithi117.stdout:6/559: mknod d3/d61/cc3 0 2024-03-22T06:15:24.400 INFO:tasks.workunit.client.1.smithi117.stdout:6/560: creat d3/d3d/d17/fc4 x:0 0 0 2024-03-22T06:15:24.405 INFO:tasks.workunit.client.1.smithi117.stdout:4/470: write d0/d19/d34/d43/f7c [6741682,61408] 0 2024-03-22T06:15:24.405 INFO:tasks.workunit.client.1.smithi117.stdout:4/471: creat d0/d19/d34/d43/d63/d6f/d70/fa2 x:0 0 0 2024-03-22T06:15:24.409 INFO:tasks.workunit.client.1.smithi117.stdout:6/561: write d3/f3f [3128903,105520] 0 2024-03-22T06:15:24.413 INFO:tasks.workunit.client.1.smithi117.stdout:9/512: creat d4/d47/d64/d72/f99 x:0 0 0 2024-03-22T06:15:24.413 INFO:tasks.workunit.client.1.smithi117.stdout:9/513: read - d4/d8/d37/d58/f92 zero size 2024-03-22T06:15:24.415 INFO:tasks.workunit.client.1.smithi117.stdout:5/515: link dd/df/d5e/d91/d72/d35/f6f dd/df/d5e/d91/d72/d60/fb4 0 2024-03-22T06:15:24.415 INFO:tasks.workunit.client.1.smithi117.stdout:5/516: chown dd/df/f2c 0 1 2024-03-22T06:15:24.416 INFO:tasks.workunit.client.1.smithi117.stdout:1/559: sync 2024-03-22T06:15:24.416 INFO:tasks.workunit.client.1.smithi117.stdout:1/560: read - d1/d30/d44/d99/fb7 zero size 2024-03-22T06:15:24.417 INFO:tasks.workunit.client.1.smithi117.stdout:0/524: mknod d1/d10/d18/d8c/cba 0 2024-03-22T06:15:24.417 INFO:tasks.workunit.client.1.smithi117.stdout:0/525: chown d1/d10/d18/d26 6025 1 2024-03-22T06:15:24.419 INFO:tasks.workunit.client.1.smithi117.stdout:0/526: dread d1/d3/d2b/d3e/d3f/f6d [0,4194304] 0 2024-03-22T06:15:24.420 INFO:tasks.workunit.client.1.smithi117.stdout:7/457: symlink d8/d9/da/d10/d23/d27/d39/d5c/l8a 0 2024-03-22T06:15:24.421 INFO:tasks.workunit.client.1.smithi117.stdout:3/478: unlink d9/c31 0 2024-03-22T06:15:24.424 INFO:tasks.workunit.client.1.smithi117.stdout:5/517: dread dd/df/d5e/d91/f7e [0,4194304] 0 2024-03-22T06:15:24.424 INFO:tasks.workunit.client.1.smithi117.stdout:2/496: link d2/d5/d13/d3f/d1d/d3a/d11/f15 d2/d5/d13/d3f/d1d/d3a/d71/f9d 0 2024-03-22T06:15:24.426 INFO:tasks.workunit.client.1.smithi117.stdout:8/468: symlink d1/d28/d6b/l90 0 2024-03-22T06:15:24.429 INFO:tasks.workunit.client.1.smithi117.stdout:4/472: mkdir d0/d19/d28/da3 0 2024-03-22T06:15:24.430 INFO:tasks.workunit.client.1.smithi117.stdout:4/473: chown d0/d19/f4a 105304887 1 2024-03-22T06:15:24.431 INFO:tasks.workunit.client.1.smithi117.stdout:6/562: creat d3/db4/fc5 x:0 0 0 2024-03-22T06:15:24.431 INFO:tasks.workunit.client.1.smithi117.stdout:6/563: read - d3/db4/d2d/d53/f70 zero size 2024-03-22T06:15:24.431 INFO:tasks.workunit.client.1.smithi117.stdout:6/564: dread - d3/d40/d46/f6b zero size 2024-03-22T06:15:24.431 INFO:tasks.workunit.client.1.smithi117.stdout:6/565: chown d3/d40/d46/d84/d49/d79/d9d/cc0 233468648 1 2024-03-22T06:15:24.432 INFO:tasks.workunit.client.1.smithi117.stdout:9/514: rmdir d4/d47 39 2024-03-22T06:15:24.432 INFO:tasks.workunit.client.1.smithi117.stdout:9/515: chown d4/d8/d37/c69 16 1 2024-03-22T06:15:24.433 INFO:tasks.workunit.client.1.smithi117.stdout:1/561: getdents d1/d30/d44/d9c/da9 0 2024-03-22T06:15:24.433 INFO:tasks.workunit.client.1.smithi117.stdout:5/518: dread fa [0,4194304] 0 2024-03-22T06:15:24.434 INFO:tasks.workunit.client.1.smithi117.stdout:0/527: mkdir d1/d10/d5c/d40/dbb 0 2024-03-22T06:15:24.435 INFO:tasks.workunit.client.1.smithi117.stdout:3/479: mknod d9/d93/cad 0 2024-03-22T06:15:24.436 INFO:tasks.workunit.client.1.smithi117.stdout:2/497: rmdir d2/d5/d13/d3f/d1d 39 2024-03-22T06:15:24.436 INFO:tasks.workunit.client.1.smithi117.stdout:2/498: chown d2/c53 72 1 2024-03-22T06:15:24.437 INFO:tasks.workunit.client.1.smithi117.stdout:8/469: mkdir d1/d28/d6b/d91 0 2024-03-22T06:15:24.438 INFO:tasks.workunit.client.1.smithi117.stdout:4/474: mkdir d0/d51/da4 0 2024-03-22T06:15:24.439 INFO:tasks.workunit.client.1.smithi117.stdout:9/516: link d4/d8/d3e/l75 d4/d47/d64/d72/l9a 0 2024-03-22T06:15:24.439 INFO:tasks.workunit.client.1.smithi117.stdout:9/517: chown d4/d8/d3e/f8f 1371918691 1 2024-03-22T06:15:24.440 INFO:tasks.workunit.client.1.smithi117.stdout:9/518: write d4/d8/d37/d58/f92 [286700,38736] 0 2024-03-22T06:15:24.440 INFO:tasks.workunit.client.1.smithi117.stdout:5/519: rmdir dd 39 2024-03-22T06:15:24.440 INFO:tasks.workunit.client.1.smithi117.stdout:5/520: getdents dd/df/d5e/d91/d72/d35/d81 0 2024-03-22T06:15:24.440 INFO:tasks.workunit.client.1.smithi117.stdout:6/566: write d3/db4/f5e [1483574,63506] 0 2024-03-22T06:15:24.440 INFO:tasks.workunit.client.1.smithi117.stdout:6/567: creat d3/db4/d2d/d53/d59/d74/da/fc6 x:0 0 0 2024-03-22T06:15:24.441 INFO:tasks.workunit.client.1.smithi117.stdout:1/562: dread d1/d22/d27/f34 [0,4194304] 0 2024-03-22T06:15:24.441 INFO:tasks.workunit.client.1.smithi117.stdout:7/458: mknod d8/d9/da/d10/c8b 0 2024-03-22T06:15:24.442 INFO:tasks.workunit.client.1.smithi117.stdout:2/499: symlink d2/d5/d13/d3f/l9e 0 2024-03-22T06:15:24.443 INFO:tasks.workunit.client.1.smithi117.stdout:8/470: creat d1/d28/d66/d7f/f92 x:0 0 0 2024-03-22T06:15:24.443 INFO:tasks.workunit.client.1.smithi117.stdout:8/471: fsync d1/d3a/d44/f8d 0 2024-03-22T06:15:24.444 INFO:tasks.workunit.client.1.smithi117.stdout:4/475: mknod d0/d19/d34/d43/ca5 0 2024-03-22T06:15:24.444 INFO:tasks.workunit.client.1.smithi117.stdout:4/476: chown d0/f83 20413097 1 2024-03-22T06:15:24.444 INFO:tasks.workunit.client.1.smithi117.stdout:4/477: dread - d0/d19/d34/d43/d63/d6f/d5f/f9a zero size 2024-03-22T06:15:24.444 INFO:tasks.workunit.client.1.smithi117.stdout:4/478: creat d0/d4/d2a/d5b/fa6 x:0 0 0 2024-03-22T06:15:24.444 INFO:tasks.workunit.client.1.smithi117.stdout:4/479: write d0/d19/d34/d43/f7c [6200685,38761] 0 2024-03-22T06:15:24.444 INFO:tasks.workunit.client.1.smithi117.stdout:4/480: chown d0/d3f/f4b 82073633 1 2024-03-22T06:15:24.445 INFO:tasks.workunit.client.1.smithi117.stdout:9/519: mkdir d4/d8/d37/d9b 0 2024-03-22T06:15:24.446 INFO:tasks.workunit.client.1.smithi117.stdout:5/521: symlink dd/d16/d5a/d66/lb5 0 2024-03-22T06:15:24.448 INFO:tasks.workunit.client.1.smithi117.stdout:6/568: mkdir d3/db4/d2d/d53/dc7 0 2024-03-22T06:15:24.450 INFO:tasks.workunit.client.1.smithi117.stdout:1/563: rename d1/f8 to d1/d1f/db2/fc8 0 2024-03-22T06:15:24.453 INFO:tasks.workunit.client.1.smithi117.stdout:7/459: mknod d8/d9/d37/d45/c8c 0 2024-03-22T06:15:24.454 INFO:tasks.workunit.client.1.smithi117.stdout:8/472: symlink d1/l93 0 2024-03-22T06:15:24.455 INFO:tasks.workunit.client.1.smithi117.stdout:3/480: dwrite d9/d1a/d39/f45 [0,4194304] 0 2024-03-22T06:15:24.455 INFO:tasks.workunit.client.1.smithi117.stdout:3/481: chown d9/d1a/d39/d4c/f44 39156922 1 2024-03-22T06:15:24.457 INFO:tasks.workunit.client.1.smithi117.stdout:4/481: symlink d0/d19/d34/d43/d63/d6f/la7 0 2024-03-22T06:15:24.458 INFO:tasks.workunit.client.1.smithi117.stdout:9/520: creat d4/d8/d37/d9b/f9c x:0 0 0 2024-03-22T06:15:24.461 INFO:tasks.workunit.client.1.smithi117.stdout:5/522: rename dd/d16/c67 to dd/df/d5e/d91/d72/d33/d9a/cb6 0 2024-03-22T06:15:24.466 INFO:tasks.workunit.client.1.smithi117.stdout:1/564: mknod d1/d3c/cc9 0 2024-03-22T06:15:24.466 INFO:tasks.workunit.client.1.smithi117.stdout:1/565: write d1/d22/d27/f8e [1595943,63520] 0 2024-03-22T06:15:24.466 INFO:tasks.workunit.client.1.smithi117.stdout:8/473: truncate f0 2860762 0 2024-03-22T06:15:24.466 INFO:tasks.workunit.client.1.smithi117.stdout:8/474: readlink d1/d3a/l76 0 2024-03-22T06:15:24.466 INFO:tasks.workunit.client.1.smithi117.stdout:8/475: dread - d1/d3a/d44/d9/f81 zero size 2024-03-22T06:15:24.467 INFO:tasks.workunit.client.1.smithi117.stdout:8/476: dread - d1/d3a/d44/d9/f81 zero size 2024-03-22T06:15:24.467 INFO:tasks.workunit.client.1.smithi117.stdout:8/477: creat d1/d28/d66/d7f/f94 x:0 0 0 2024-03-22T06:15:24.467 INFO:tasks.workunit.client.1.smithi117.stdout:3/482: symlink d9/d1a/d39/d8a/da1/lae 0 2024-03-22T06:15:24.468 INFO:tasks.workunit.client.1.smithi117.stdout:6/569: dwrite d3/db4/d2d/d53/d75/fc1 [0,4194304] 0 2024-03-22T06:15:24.468 INFO:tasks.workunit.client.1.smithi117.stdout:6/570: truncate d3/d40/d46/d84/f7b 764065 0 2024-03-22T06:15:24.468 INFO:tasks.workunit.client.1.smithi117.stdout:6/571: dread - d3/db4/d2d/d53/d75/fb0 zero size 2024-03-22T06:15:24.470 INFO:tasks.workunit.client.1.smithi117.stdout:7/460: read d8/d9/da/d2b/f6e [213501,4556] 0 2024-03-22T06:15:24.470 INFO:tasks.workunit.client.1.smithi117.stdout:7/461: chown d8/l52 73764397 1 2024-03-22T06:15:24.470 INFO:tasks.workunit.client.1.smithi117.stdout:0/528: rmdir d1/d10/d5c/d40 39 2024-03-22T06:15:24.470 INFO:tasks.workunit.client.1.smithi117.stdout:0/529: readlink d1/d3/d2b/d3e/de/d1f/laf 0 2024-03-22T06:15:24.472 INFO:tasks.workunit.client.1.smithi117.stdout:1/566: creat d1/d3c/db6/fca x:0 0 0 2024-03-22T06:15:24.472 INFO:tasks.workunit.client.1.smithi117.stdout:8/478: dread d1/d19/f2d [0,4194304] 0 2024-03-22T06:15:24.473 INFO:tasks.workunit.client.1.smithi117.stdout:2/500: dwrite d2/d5/d13/d3f/f51 [0,4194304] 0 2024-03-22T06:15:24.474 INFO:tasks.workunit.client.1.smithi117.stdout:5/523: mknod dd/df/cb7 0 2024-03-22T06:15:24.479 INFO:tasks.workunit.client.1.smithi117.stdout:7/462: creat d8/d9/da/d10/f8d x:0 0 0 2024-03-22T06:15:24.481 INFO:tasks.workunit.client.1.smithi117.stdout:0/530: mkdir d1/d10/d5c/d40/d4e/dbc 0 2024-03-22T06:15:24.481 INFO:tasks.workunit.client.1.smithi117.stdout:0/531: fdatasync d1/d10/d18/d26/d53/f82 0 2024-03-22T06:15:24.481 INFO:tasks.workunit.client.1.smithi117.stdout:0/532: chown d1/d3/d2b/d3e/d3f/f45 75 1 2024-03-22T06:15:24.482 INFO:tasks.workunit.client.1.smithi117.stdout:1/567: mkdir d1/d14/dcb 0 2024-03-22T06:15:24.482 INFO:tasks.workunit.client.1.smithi117.stdout:1/568: dread - d1/d1f/fae zero size 2024-03-22T06:15:24.482 INFO:tasks.workunit.client.1.smithi117.stdout:1/569: stat d1/d1f/db2/fc8 0 2024-03-22T06:15:24.482 INFO:tasks.workunit.client.1.smithi117.stdout:2/501: dread d2/d5/d13/f7a [0,4194304] 0 2024-03-22T06:15:24.487 INFO:tasks.workunit.client.1.smithi117.stdout:5/524: unlink dd/df/d1e/f2e 0 2024-03-22T06:15:24.487 INFO:tasks.workunit.client.1.smithi117.stdout:7/463: creat d8/d50/d5e/f8e x:0 0 0 2024-03-22T06:15:24.489 INFO:tasks.workunit.client.1.smithi117.stdout:0/533: symlink d1/d10/d18/d26/lbd 0 2024-03-22T06:15:24.489 INFO:tasks.workunit.client.1.smithi117.stdout:0/534: fdatasync d1/d10/d18/d26/d49/fab 0 2024-03-22T06:15:24.490 INFO:tasks.workunit.client.1.smithi117.stdout:8/479: dread d1/d19/d38/f70 [0,4194304] 0 2024-03-22T06:15:24.491 INFO:tasks.workunit.client.1.smithi117.stdout:1/570: unlink d1/d30/fa3 0 2024-03-22T06:15:24.491 INFO:tasks.workunit.client.1.smithi117.stdout:1/571: readlink d1/d3c/d4d/lc3 0 2024-03-22T06:15:24.493 INFO:tasks.workunit.client.1.smithi117.stdout:2/502: link d2/d5/d13/d3f/d1d/f46 d2/d35/d4c/f9f 0 2024-03-22T06:15:24.498 INFO:tasks.workunit.client.1.smithi117.stdout:7/464: symlink d8/d9/da/d10/d23/d41/l8f 0 2024-03-22T06:15:24.499 INFO:tasks.workunit.client.1.smithi117.stdout:8/480: creat d1/d3a/d44/d11/f95 x:0 0 0 2024-03-22T06:15:24.499 INFO:tasks.workunit.client.1.smithi117.stdout:8/481: chown d1/d28/d6b/c7b 141537 1 2024-03-22T06:15:24.502 INFO:tasks.workunit.client.1.smithi117.stdout:7/465: getdents d8/d6c 0 2024-03-22T06:15:24.504 INFO:tasks.workunit.client.1.smithi117.stdout:8/482: rename d1/d19/l1c to d1/d3a/d44/d16/l96 0 2024-03-22T06:15:24.505 INFO:tasks.workunit.client.1.smithi117.stdout:2/503: getdents d2/d5/d13/d3f 0 2024-03-22T06:15:24.506 INFO:tasks.workunit.client.1.smithi117.stdout:7/466: symlink d8/d9/da/d10/d23/d27/l90 0 2024-03-22T06:15:24.508 INFO:tasks.workunit.client.1.smithi117.stdout:8/483: mkdir d1/d3a/d44/d11/d97 0 2024-03-22T06:15:24.508 INFO:tasks.workunit.client.1.smithi117.stdout:8/484: dread - d1/d28/d66/d7f/f94 zero size 2024-03-22T06:15:24.508 INFO:tasks.workunit.client.1.smithi117.stdout:2/504: truncate d2/d5/db/f24 2254587 0 2024-03-22T06:15:24.509 INFO:tasks.workunit.client.1.smithi117.stdout:2/505: fsync d2/d5/d13/d3f/f51 0 2024-03-22T06:15:24.509 INFO:tasks.workunit.client.1.smithi117.stdout:2/506: fsync d2/d5/d13/d3f/d1d/d3a/f78 0 2024-03-22T06:15:24.510 INFO:tasks.workunit.client.1.smithi117.stdout:2/507: symlink d2/d5/d13/d57/la0 0 2024-03-22T06:15:24.511 INFO:tasks.workunit.client.1.smithi117.stdout:2/508: symlink d2/d5/d13/d3f/d1d/d3a/d89/la1 0 2024-03-22T06:15:24.513 INFO:tasks.workunit.client.1.smithi117.stdout:2/509: unlink d2/d5/d13/d57/la0 0 2024-03-22T06:15:24.515 INFO:tasks.workunit.client.1.smithi117.stdout:2/510: symlink d2/d5/d13/d3f/d1d/d3a/d11/d4f/la2 0 2024-03-22T06:15:24.516 INFO:tasks.workunit.client.1.smithi117.stdout:2/511: symlink d2/d5/db/d2c/d7d/la3 0 2024-03-22T06:15:24.516 INFO:tasks.workunit.client.1.smithi117.stdout:3/483: dwrite d9/f63 [0,4194304] 0 2024-03-22T06:15:24.516 INFO:tasks.workunit.client.1.smithi117.stdout:3/484: creat d9/d1a/faf x:0 0 0 2024-03-22T06:15:24.516 INFO:tasks.workunit.client.1.smithi117.stdout:3/485: chown d9/d1a/d39/f71 204411 1 2024-03-22T06:15:24.517 INFO:tasks.workunit.client.1.smithi117.stdout:4/482: dwrite d0/f1d [0,4194304] 0 2024-03-22T06:15:24.517 INFO:tasks.workunit.client.1.smithi117.stdout:9/521: dwrite d4/d8/f41 [4194304,4194304] 0 2024-03-22T06:15:24.517 INFO:tasks.workunit.client.1.smithi117.stdout:2/512: rename d2/d5/f7f to d2/d5/d13/d3f/d1d/fa4 0 2024-03-22T06:15:24.517 INFO:tasks.workunit.client.1.smithi117.stdout:7/467: read d8/f16 [5369056,23370] 0 2024-03-22T06:15:24.518 INFO:tasks.workunit.client.1.smithi117.stdout:7/468: chown d8/d9/d37/d5f/f79 61 1 2024-03-22T06:15:24.518 INFO:tasks.workunit.client.1.smithi117.stdout:0/535: dwrite d1/d10/d5c/d40/d69/f7d [0,4194304] 0 2024-03-22T06:15:24.518 INFO:tasks.workunit.client.1.smithi117.stdout:0/536: readlink d1/d3/d2b/d3e/d3f/l9f 0 2024-03-22T06:15:24.518 INFO:tasks.workunit.client.1.smithi117.stdout:5/525: dwrite dd/df/f2c [0,4194304] 0 2024-03-22T06:15:24.520 INFO:tasks.workunit.client.1.smithi117.stdout:4/483: creat d0/d19/d9b/fa8 x:0 0 0 2024-03-22T06:15:24.520 INFO:tasks.workunit.client.1.smithi117.stdout:4/484: fdatasync d0/d19/d28/f2e 0 2024-03-22T06:15:24.522 INFO:tasks.workunit.client.1.smithi117.stdout:9/522: mknod d4/d8/d3e/d7a/c9d 0 2024-03-22T06:15:24.524 INFO:tasks.workunit.client.1.smithi117.stdout:3/486: rename d9/d1a/d39/d82/c86 to d9/d93/cb0 0 2024-03-22T06:15:24.526 INFO:tasks.workunit.client.1.smithi117.stdout:7/469: mkdir d8/d9/da/d10/d23/d48/d91 0 2024-03-22T06:15:24.526 INFO:tasks.workunit.client.1.smithi117.stdout:7/470: stat d8/d9/f7c 0 2024-03-22T06:15:24.536 INFO:tasks.workunit.client.1.smithi117.stdout:9/523: dwrite d4/d7/f2d [0,4194304] 0 2024-03-22T06:15:24.536 INFO:tasks.workunit.client.1.smithi117.stdout:5/526: mkdir dd/d16/d1a/d32/d73/db8 0 2024-03-22T06:15:24.537 INFO:tasks.workunit.client.1.smithi117.stdout:4/485: truncate d0/d19/f22 648500 0 2024-03-22T06:15:24.538 INFO:tasks.workunit.client.1.smithi117.stdout:4/486: creat d0/d51/fa9 x:0 0 0 2024-03-22T06:15:24.539 INFO:tasks.workunit.client.1.smithi117.stdout:0/537: rename d1/da7 to d1/d3/dae/dbe 0 2024-03-22T06:15:24.541 INFO:tasks.workunit.client.1.smithi117.stdout:3/487: unlink d9/d55/c58 0 2024-03-22T06:15:24.542 INFO:tasks.workunit.client.1.smithi117.stdout:2/513: getdents d2/d5/d41/d83 0 2024-03-22T06:15:24.542 INFO:tasks.workunit.client.1.smithi117.stdout:2/514: fdatasync d2/fc 0 2024-03-22T06:15:24.542 INFO:tasks.workunit.client.1.smithi117.stdout:2/515: dread - d2/d5/d13/d3f/d1d/d3a/f78 zero size 2024-03-22T06:15:24.543 INFO:tasks.workunit.client.1.smithi117.stdout:7/471: creat d8/d9/da/d10/d23/f92 x:0 0 0 2024-03-22T06:15:24.546 INFO:tasks.workunit.client.1.smithi117.stdout:5/527: dread dd/df/d95/fab [0,4194304] 0 2024-03-22T06:15:24.546 INFO:tasks.workunit.client.1.smithi117.stdout:5/528: fdatasync dd/d16/d1a/f59 0 2024-03-22T06:15:24.546 INFO:tasks.workunit.client.1.smithi117.stdout:5/529: readlink dd/df/d5e/d91/d72/l69 0 2024-03-22T06:15:24.549 INFO:tasks.workunit.client.1.smithi117.stdout:9/524: rename d4/d7/d27/f4c to d4/d8/d3e/d7a/f9e 0 2024-03-22T06:15:24.551 INFO:tasks.workunit.client.1.smithi117.stdout:7/472: read d8/d9/da/d10/f25 [2404973,95271] 0 2024-03-22T06:15:24.551 INFO:tasks.workunit.client.1.smithi117.stdout:0/538: mkdir d1/d10/d5c/d40/dbb/dbf 0 2024-03-22T06:15:24.551 INFO:tasks.workunit.client.1.smithi117.stdout:0/539: truncate d1/d3/d2b/d3e/d3f/f6d 1011862 0 2024-03-22T06:15:24.554 INFO:tasks.workunit.client.1.smithi117.stdout:3/488: symlink d9/d1a/d39/d7d/d97/lb1 0 2024-03-22T06:15:24.554 INFO:tasks.workunit.client.1.smithi117.stdout:3/489: chown d9/d1a/d39 7334 1 2024-03-22T06:15:24.554 INFO:tasks.workunit.client.1.smithi117.stdout:3/490: fdatasync d9/d1a/d39/d4c/d37/f61 0 2024-03-22T06:15:24.555 INFO:tasks.workunit.client.1.smithi117.stdout:5/530: write dd/df/f2b [642800,2390] 0 2024-03-22T06:15:24.555 INFO:tasks.workunit.client.1.smithi117.stdout:5/531: fdatasync dd/df/d5e/d91/d36/f25 0 2024-03-22T06:15:24.555 INFO:tasks.workunit.client.1.smithi117.stdout:5/532: chown dd/df/d1e 2082494 1 2024-03-22T06:15:24.556 INFO:tasks.workunit.client.1.smithi117.stdout:5/533: chown dd/d16/c53 323 1 2024-03-22T06:15:24.556 INFO:tasks.workunit.client.1.smithi117.stdout:5/534: dread - dd/df/d5e/f92 zero size 2024-03-22T06:15:24.556 INFO:tasks.workunit.client.1.smithi117.stdout:5/535: dread dd/df/d5e/d91/d72/d60/d6e/f85 [0,4194304] 0 2024-03-22T06:15:24.556 INFO:tasks.workunit.client.1.smithi117.stdout:5/536: stat dd/df/l71 0 2024-03-22T06:15:24.556 INFO:tasks.workunit.client.1.smithi117.stdout:5/537: readlink dd/df/d5e/d91/l96 0 2024-03-22T06:15:24.557 INFO:tasks.workunit.client.1.smithi117.stdout:4/487: rename d0/d19/d34/d43/ca5 to d0/d19/d34/d43/d63/d6f/d70/caa 0 2024-03-22T06:15:24.557 INFO:tasks.workunit.client.1.smithi117.stdout:4/488: write d0/d19/d9b/fa8 [459651,40644] 0 2024-03-22T06:15:24.557 INFO:tasks.workunit.client.1.smithi117.stdout:4/489: chown d0/d4/d2a/d5b 11051 1 2024-03-22T06:15:24.558 INFO:tasks.workunit.client.1.smithi117.stdout:9/525: creat d4/d8/d3e/d7a/d2f/f9f x:0 0 0 2024-03-22T06:15:24.560 INFO:tasks.workunit.client.1.smithi117.stdout:2/516: dwrite d2/d5/d13/d3f/d1d/f6c [0,4194304] 0 2024-03-22T06:15:24.565 INFO:tasks.workunit.client.1.smithi117.stdout:7/473: write d8/fb [5021149,95306] 0 2024-03-22T06:15:24.572 INFO:tasks.workunit.client.1.smithi117.stdout:5/538: dwrite dd/d16/d86/f9d [0,4194304] 0 2024-03-22T06:15:24.572 INFO:tasks.workunit.client.1.smithi117.stdout:5/539: chown dd/d16/d5a/c78 3314116 1 2024-03-22T06:15:24.572 INFO:tasks.workunit.client.1.smithi117.stdout:5/540: fsync dd/d16/d5a/f97 0 2024-03-22T06:15:24.572 INFO:tasks.workunit.client.1.smithi117.stdout:5/541: readlink dd/df/d5e/d91/d72/d33/l52 0 2024-03-22T06:15:24.576 INFO:tasks.workunit.client.1.smithi117.stdout:0/540: creat d1/d3/d2b/fc0 x:0 0 0 2024-03-22T06:15:24.576 INFO:tasks.workunit.client.1.smithi117.stdout:0/541: fsync d1/d3/d2b/d3e/de/d1f/d3b/f6b 0 2024-03-22T06:15:24.576 INFO:tasks.workunit.client.1.smithi117.stdout:5/542: dread dd/df/d95/fab [0,4194304] 0 2024-03-22T06:15:24.576 INFO:tasks.workunit.client.1.smithi117.stdout:5/543: write dd/df/d5e/d91/d72/d35/f6f [4297682,33948] 0 2024-03-22T06:15:24.577 INFO:tasks.workunit.client.1.smithi117.stdout:3/491: creat d9/d1a/d39/d4c/d36/fb2 x:0 0 0 2024-03-22T06:15:24.577 INFO:tasks.workunit.client.1.smithi117.stdout:3/492: dread - d9/d1a/d39/d4c/d37/f8c zero size 2024-03-22T06:15:24.580 INFO:tasks.workunit.client.1.smithi117.stdout:4/490: mknod d0/d19/d34/d43/d63/cab 0 2024-03-22T06:15:24.582 INFO:tasks.workunit.client.1.smithi117.stdout:9/526: mkdir d4/d46/da0 0 2024-03-22T06:15:24.585 INFO:tasks.workunit.client.1.smithi117.stdout:7/474: creat d8/d9/da/d10/d23/d41/f93 x:0 0 0 2024-03-22T06:15:24.587 INFO:tasks.workunit.client.1.smithi117.stdout:2/517: dwrite d2/d1b/d39/d77/f80 [0,4194304] 0 2024-03-22T06:15:24.594 INFO:tasks.workunit.client.1.smithi117.stdout:4/491: dread d0/d19/d28/f45 [0,4194304] 0 2024-03-22T06:15:24.594 INFO:tasks.workunit.client.1.smithi117.stdout:4/492: write d0/d4/d2a/d5b/fa6 [652169,1543] 0 2024-03-22T06:15:24.596 INFO:tasks.workunit.client.1.smithi117.stdout:0/542: mknod d1/d10/d18/d26/d49/cc1 0 2024-03-22T06:15:24.597 INFO:tasks.workunit.client.1.smithi117.stdout:5/544: creat dd/d16/d1a/d32/d73/db8/fb9 x:0 0 0 2024-03-22T06:15:24.598 INFO:tasks.workunit.client.1.smithi117.stdout:3/493: mknod d9/d1a/d39/d4c/d36/cb3 0 2024-03-22T06:15:24.599 INFO:tasks.workunit.client.1.smithi117.stdout:9/527: symlink d4/d46/la1 0 2024-03-22T06:15:24.603 INFO:tasks.workunit.client.1.smithi117.stdout:4/493: mknod d0/d19/d28/da3/cac 0 2024-03-22T06:15:24.605 INFO:tasks.workunit.client.1.smithi117.stdout:0/543: mknod d1/d3/d2b/d3e/de/d1f/d59/cc2 0 2024-03-22T06:15:24.605 INFO:tasks.workunit.client.1.smithi117.stdout:0/544: chown d1/d3/d2b/d3e/d3f/l9f 254 1 2024-03-22T06:15:24.607 INFO:tasks.workunit.client.1.smithi117.stdout:9/528: symlink d4/d8/la2 0 2024-03-22T06:15:24.608 INFO:tasks.workunit.client.1.smithi117.stdout:9/529: dread - d4/d8/d37/d9b/f9c zero size 2024-03-22T06:15:24.610 INFO:tasks.workunit.client.1.smithi117.stdout:5/545: write dd/df/d5e/d91/f23 [119043,77569] 0 2024-03-22T06:15:24.610 INFO:tasks.workunit.client.1.smithi117.stdout:2/518: dwrite d2/d5/db/f72 [0,4194304] 0 2024-03-22T06:15:24.611 INFO:tasks.workunit.client.1.smithi117.stdout:9/530: unlink d4/f21 0 2024-03-22T06:15:24.611 INFO:tasks.workunit.client.1.smithi117.stdout:4/494: getdents d0/d3f 0 2024-03-22T06:15:24.613 INFO:tasks.workunit.client.1.smithi117.stdout:5/546: creat dd/df/d5e/d91/d72/d35/d81/fba x:0 0 0 2024-03-22T06:15:24.615 INFO:tasks.workunit.client.1.smithi117.stdout:0/545: dread d1/d10/f11 [8388608,4194304] 0 2024-03-22T06:15:24.615 INFO:tasks.workunit.client.1.smithi117.stdout:0/546: write d1/d3/d2b/d3e/d3f/f71 [475614,4649] 0 2024-03-22T06:15:24.615 INFO:tasks.workunit.client.1.smithi117.stdout:9/531: rename d4/d7/f2a to d4/d8/d3f/fa3 0 2024-03-22T06:15:24.615 INFO:tasks.workunit.client.1.smithi117.stdout:9/532: fdatasync d4/d8/d3e/d7a/d2f/f39 0 2024-03-22T06:15:24.615 INFO:tasks.workunit.client.1.smithi117.stdout:9/533: creat d4/d46/fa4 x:0 0 0 2024-03-22T06:15:24.616 INFO:tasks.workunit.client.1.smithi117.stdout:4/495: mknod d0/d48/cad 0 2024-03-22T06:15:24.617 INFO:tasks.workunit.client.1.smithi117.stdout:9/534: dread d4/d8/f67 [0,4194304] 0 2024-03-22T06:15:24.617 INFO:tasks.workunit.client.1.smithi117.stdout:0/547: link d1/d10/l31 d1/d3/dae/lc3 0 2024-03-22T06:15:24.620 INFO:tasks.workunit.client.1.smithi117.stdout:2/519: dread d2/d5/d13/d3f/d1d/fa4 [0,4194304] 0 2024-03-22T06:15:24.621 INFO:tasks.workunit.client.1.smithi117.stdout:3/494: dwrite d9/d1a/d3a/f53 [0,4194304] 0 2024-03-22T06:15:24.622 INFO:tasks.workunit.client.1.smithi117.stdout:9/535: link d4/d7/d27/f2e d4/d47/d85/fa5 0 2024-03-22T06:15:24.624 INFO:tasks.workunit.client.1.smithi117.stdout:2/520: mkdir d2/d5/da5 0 2024-03-22T06:15:24.627 INFO:tasks.workunit.client.1.smithi117.stdout:3/495: creat d9/d93/fb4 x:0 0 0 2024-03-22T06:15:24.627 INFO:tasks.workunit.client.1.smithi117.stdout:3/496: fsync d9/d1a/d39/f7f 0 2024-03-22T06:15:24.628 INFO:tasks.workunit.client.1.smithi117.stdout:0/548: write d1/d10/d18/d26/d49/d7e/d39/d6a/f94 [528660,7078] 0 2024-03-22T06:15:24.628 INFO:tasks.workunit.client.1.smithi117.stdout:4/496: write d0/f11 [4046986,34421] 0 2024-03-22T06:15:24.629 INFO:tasks.workunit.client.1.smithi117.stdout:9/536: rename d4/d50/f88 to d4/d46/fa6 0 2024-03-22T06:15:24.629 INFO:tasks.workunit.client.1.smithi117.stdout:9/537: truncate d4/d7/d27/d5b/f78 965176 0 2024-03-22T06:15:24.629 INFO:tasks.workunit.client.1.smithi117.stdout:9/538: read d4/d8/f25 [1845190,44893] 0 2024-03-22T06:15:24.629 INFO:tasks.workunit.client.1.smithi117.stdout:4/497: dread d0/d19/d34/f77 [0,4194304] 0 2024-03-22T06:15:24.630 INFO:tasks.workunit.client.1.smithi117.stdout:3/497: creat d9/d1a/d39/d8a/da1/fb5 x:0 0 0 2024-03-22T06:15:24.630 INFO:tasks.workunit.client.1.smithi117.stdout:3/498: fsync d9/d1a/f1e 0 2024-03-22T06:15:24.631 INFO:tasks.workunit.client.1.smithi117.stdout:5/547: dwrite dd/d16/f6c [0,4194304] 0 2024-03-22T06:15:24.631 INFO:tasks.workunit.client.1.smithi117.stdout:5/548: write dd/df/d5e/d91/d72/d60/d6e/f85 [1707808,32509] 0 2024-03-22T06:15:24.631 INFO:tasks.workunit.client.1.smithi117.stdout:5/549: truncate dd/df/d5e/d91/f4f 4979612 0 2024-03-22T06:15:24.634 INFO:tasks.workunit.client.1.smithi117.stdout:0/549: rename d1/d10/d18/d26/d49/l5d to d1/d3/lc4 0 2024-03-22T06:15:24.637 INFO:tasks.workunit.client.1.smithi117.stdout:4/498: link d0/c8b d0/d19/d34/d43/d63/cae 0 2024-03-22T06:15:24.637 INFO:tasks.workunit.client.1.smithi117.stdout:4/499: stat d0/d19/d34/d43/d63/f74 0 2024-03-22T06:15:24.639 INFO:tasks.workunit.client.1.smithi117.stdout:3/499: mkdir d9/d1a/d39/d8a/da1/db6 0 2024-03-22T06:15:24.643 INFO:tasks.workunit.client.1.smithi117.stdout:5/550: mknod dd/df/d95/da0/cbb 0 2024-03-22T06:15:24.643 INFO:tasks.workunit.client.1.smithi117.stdout:5/551: creat dd/d16/fbc x:0 0 0 2024-03-22T06:15:24.649 INFO:tasks.workunit.client.1.smithi117.stdout:4/500: mkdir d0/d48/daf 0 2024-03-22T06:15:24.649 INFO:tasks.workunit.client.1.smithi117.stdout:4/501: truncate d0/d19/f49 575797 0 2024-03-22T06:15:24.649 INFO:tasks.workunit.client.1.smithi117.stdout:5/552: dread dd/d16/f64 [0,4194304] 0 2024-03-22T06:15:24.650 INFO:tasks.workunit.client.1.smithi117.stdout:9/539: dwrite d4/d7/f26 [0,4194304] 0 2024-03-22T06:15:24.651 INFO:tasks.workunit.client.1.smithi117.stdout:1/572: sync 2024-03-22T06:15:24.651 INFO:tasks.workunit.client.1.smithi117.stdout:8/485: sync 2024-03-22T06:15:24.651 INFO:tasks.workunit.client.1.smithi117.stdout:6/572: sync 2024-03-22T06:15:24.652 INFO:tasks.workunit.client.1.smithi117.stdout:1/573: chown d1/d3c/d4d/d5b/f96 2397 1 2024-03-22T06:15:24.658 INFO:tasks.workunit.client.1.smithi117.stdout:1/574: rmdir d1/d3c 39 2024-03-22T06:15:24.659 INFO:tasks.workunit.client.1.smithi117.stdout:4/502: write d0/d19/d34/d43/d63/d8d/f90 [562253,33750] 0 2024-03-22T06:15:24.660 INFO:tasks.workunit.client.1.smithi117.stdout:1/575: mknod d1/d22/d27/d4e/d92/ccc 0 2024-03-22T06:15:24.660 INFO:tasks.workunit.client.1.smithi117.stdout:1/576: chown d1/d62/ca7 0 1 2024-03-22T06:15:24.660 INFO:tasks.workunit.client.1.smithi117.stdout:1/577: truncate d1/d22/d27/d4e/d92/fab 501612 0 2024-03-22T06:15:24.661 INFO:tasks.workunit.client.1.smithi117.stdout:9/540: write d4/d8/d3f/fa3 [2438411,85294] 0 2024-03-22T06:15:24.661 INFO:tasks.workunit.client.1.smithi117.stdout:9/541: fdatasync d4/d7/d27/f2e 0 2024-03-22T06:15:24.662 INFO:tasks.workunit.client.1.smithi117.stdout:6/573: dread d3/d40/d46/d84/d49/f5b [0,4194304] 0 2024-03-22T06:15:24.664 INFO:tasks.workunit.client.1.smithi117.stdout:8/486: write d1/d3a/d44/d16/f7a [1120419,112234] 0 2024-03-22T06:15:24.664 INFO:tasks.workunit.client.1.smithi117.stdout:8/487: write d1/d28/d66/d7f/f92 [252634,79788] 0 2024-03-22T06:15:24.670 INFO:tasks.workunit.client.1.smithi117.stdout:6/574: rename d3/c13 to d3/d3d/cc8 0 2024-03-22T06:15:24.670 INFO:tasks.workunit.client.1.smithi117.stdout:6/575: chown d3/db4/d2d/d42/da2/f2f 23 1 2024-03-22T06:15:24.670 INFO:tasks.workunit.client.1.smithi117.stdout:6/576: chown d3/d40/d46/d84/d49/d79 122586919 1 2024-03-22T06:15:24.670 INFO:tasks.workunit.client.1.smithi117.stdout:9/542: write d4/d7/d27/f7d [4156120,75747] 0 2024-03-22T06:15:24.671 INFO:tasks.workunit.client.1.smithi117.stdout:9/543: fsync d4/d7/d27/f97 0 2024-03-22T06:15:24.671 INFO:tasks.workunit.client.1.smithi117.stdout:8/488: mkdir d1/d19/d98 0 2024-03-22T06:15:24.676 INFO:tasks.workunit.client.1.smithi117.stdout:5/553: dwrite dd/df/d5e/d91/d72/d33/f68 [8388608,4194304] 0 2024-03-22T06:15:24.676 INFO:tasks.workunit.client.1.smithi117.stdout:4/503: write d0/f11 [8270246,51466] 0 2024-03-22T06:15:24.678 INFO:tasks.workunit.client.1.smithi117.stdout:1/578: dread d1/f13 [0,4194304] 0 2024-03-22T06:15:24.679 INFO:tasks.workunit.client.1.smithi117.stdout:5/554: dread dd/df/d5e/d91/d36/f26 [0,4194304] 0 2024-03-22T06:15:24.679 INFO:tasks.workunit.client.1.smithi117.stdout:5/555: chown dd/df/lb3 1185 1 2024-03-22T06:15:24.680 INFO:tasks.workunit.client.1.smithi117.stdout:5/556: fsync dd/df/d5e/d91/f4f 0 2024-03-22T06:15:24.698 INFO:tasks.workunit.client.1.smithi117.stdout:9/544: dwrite d4/d7/d27/f97 [0,4194304] 0 2024-03-22T06:15:24.714 INFO:tasks.workunit.client.1.smithi117.stdout:9/545: dread d4/d8/d37/d58/f8a [0,4194304] 0 2024-03-22T06:15:24.716 INFO:tasks.workunit.client.1.smithi117.stdout:6/577: dwrite d3/db4/d2d/d53/d59/d74/da/db3/fa4 [0,4194304] 0 2024-03-22T06:15:24.734 INFO:tasks.workunit.client.1.smithi117.stdout:9/546: dwrite d4/d8/d3f/f89 [0,4194304] 0 2024-03-22T06:15:24.734 INFO:tasks.workunit.client.1.smithi117.stdout:9/547: chown d4/d8/d3e/d7a 207 1 2024-03-22T06:15:24.734 INFO:tasks.workunit.client.1.smithi117.stdout:6/578: dwrite d3/db4/d2d/d53/d75/fb0 [0,4194304] 0 2024-03-22T06:15:24.748 INFO:tasks.workunit.client.1.smithi117.stdout:9/548: dwrite d4/d8/d3e/f8f [0,4194304] 0 2024-03-22T06:15:24.759 INFO:tasks.workunit.client.1.smithi117.stdout:8/489: creat d1/d3a/d44/d11/d97/f99 x:0 0 0 2024-03-22T06:15:24.765 INFO:tasks.workunit.client.1.smithi117.stdout:4/504: stat d0/d19/c20 0 2024-03-22T06:15:24.770 INFO:tasks.workunit.client.1.smithi117.stdout:5/557: rmdir dd/df/d5e/d91/d36/d98 39 2024-03-22T06:15:24.812 INFO:tasks.workunit.client.1.smithi117.stdout:6/579: symlink d3/db4/d2d/d53/lc9 0 2024-03-22T06:15:24.834 INFO:tasks.workunit.client.1.smithi117.stdout:6/580: dwrite d3/d40/fa9 [0,4194304] 0 2024-03-22T06:15:24.900 INFO:tasks.workunit.client.1.smithi117.stdout:9/549: truncate d4/d46/f6d 1222129 0 2024-03-22T06:15:24.930 INFO:tasks.workunit.client.1.smithi117.stdout:5/558: symlink dd/df/d5e/d91/d72/d33/d9a/lbd 0 2024-03-22T06:15:24.938 INFO:tasks.workunit.client.1.smithi117.stdout:9/550: symlink d4/d8/d37/d58/la7 0 2024-03-22T06:15:24.940 INFO:tasks.workunit.client.1.smithi117.stdout:9/551: creat d4/d47/d85/fa8 x:0 0 0 2024-03-22T06:15:24.940 INFO:tasks.workunit.client.1.smithi117.stdout:9/552: fsync d4/d8/d3e/d7a/f63 0 2024-03-22T06:15:24.949 INFO:tasks.workunit.client.1.smithi117.stdout:9/553: dread d4/d46/f55 [0,4194304] 0 2024-03-22T06:15:24.960 INFO:tasks.workunit.client.1.smithi117.stdout:9/554: dread d4/d7/d27/f97 [0,4194304] 0 2024-03-22T06:15:24.978 INFO:tasks.workunit.client.1.smithi117.stdout:9/555: dwrite d4/d8/d3e/f8f [0,4194304] 0 2024-03-22T06:15:24.981 INFO:tasks.workunit.client.1.smithi117.stdout:9/556: mkdir d4/d8/d3e/d7a/da9 0 2024-03-22T06:15:24.984 INFO:tasks.workunit.client.1.smithi117.stdout:9/557: getdents d4/d8/d3f 0 2024-03-22T06:15:24.987 INFO:tasks.workunit.client.1.smithi117.stdout:9/558: rename d4/d8/d37/f4b to d4/d8/d3e/d82/faa 0 2024-03-22T06:15:24.990 INFO:tasks.workunit.client.1.smithi117.stdout:9/559: mkdir d4/d47/d85/dab 0 2024-03-22T06:15:24.992 INFO:tasks.workunit.client.1.smithi117.stdout:9/560: symlink d4/d8/d3e/d7a/d2f/lac 0 2024-03-22T06:15:25.000 INFO:tasks.workunit.client.1.smithi117.stdout:5/559: creat dd/df/d5e/d91/d72/fbe x:0 0 0 2024-03-22T06:15:25.002 INFO:tasks.workunit.client.1.smithi117.stdout:5/560: mkdir dd/d16/d1a/d32/dbf 0 2024-03-22T06:15:25.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:24 smithi117 ceph-mon[36514]: Reconfiguring grafana.smithi006 (dependencies changed)... 2024-03-22T06:15:25.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:24 smithi117 ceph-mon[36514]: Reconfiguring daemon grafana.smithi006 on smithi006 2024-03-22T06:15:25.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:24 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:15:25.018 INFO:tasks.workunit.client.1.smithi117.stdout:7/475: sync 2024-03-22T06:15:25.021 INFO:tasks.workunit.client.1.smithi117.stdout:7/476: symlink d8/d9/da/d10/d23/d27/l94 0 2024-03-22T06:15:25.021 INFO:tasks.workunit.client.1.smithi117.stdout:7/477: write d8/d9/d37/d5f/f79 [1281850,69861] 0 2024-03-22T06:15:25.021 INFO:tasks.workunit.client.1.smithi117.stdout:7/478: readlink d8/d9/d37/d5f/l63 0 2024-03-22T06:15:25.022 INFO:tasks.workunit.client.1.smithi117.stdout:7/479: truncate d8/d9/da/d10/d23/d48/f57 3143188 0 2024-03-22T06:15:25.023 INFO:tasks.workunit.client.1.smithi117.stdout:7/480: getdents d8/d9/da/d10/d23/d27/d39/d26 0 2024-03-22T06:15:25.028 INFO:tasks.workunit.client.1.smithi117.stdout:5/561: dwrite dd/df/d5e/d91/d90/fa4 [0,4194304] 0 2024-03-22T06:15:25.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:24 smithi006 ceph-mon[29301]: Reconfiguring grafana.smithi006 (dependencies changed)... 2024-03-22T06:15:25.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:24 smithi006 ceph-mon[29301]: Reconfiguring daemon grafana.smithi006 on smithi006 2024-03-22T06:15:25.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:24 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:15:25.041 INFO:tasks.workunit.client.1.smithi117.stdout:5/562: write dd/d16/f6c [1055270,9639] 0 2024-03-22T06:15:25.041 INFO:tasks.workunit.client.1.smithi117.stdout:5/563: write dd/d16/f48 [3508760,61941] 0 2024-03-22T06:15:25.048 INFO:tasks.workunit.client.1.smithi117.stdout:7/481: dwrite d8/d9/da/d2b/f6e [4194304,4194304] 0 2024-03-22T06:15:25.051 INFO:tasks.workunit.client.1.smithi117.stdout:7/482: creat d8/d9/d37/d5f/d82/f95 x:0 0 0 2024-03-22T06:15:25.051 INFO:tasks.workunit.client.1.smithi117.stdout:7/483: readlink d8/d50/l87 0 2024-03-22T06:15:25.051 INFO:tasks.workunit.client.1.smithi117.stdout:7/484: write d8/d9/da/f72 [940073,120487] 0 2024-03-22T06:15:25.053 INFO:tasks.workunit.client.1.smithi117.stdout:7/485: symlink d8/d6c/l96 0 2024-03-22T06:15:25.053 INFO:tasks.workunit.client.1.smithi117.stdout:7/486: chown d8/d9/da/d10/d23/d27/d39/d26/c33 1356345 1 2024-03-22T06:15:25.053 INFO:tasks.workunit.client.1.smithi117.stdout:5/564: dread dd/fe [0,4194304] 0 2024-03-22T06:15:25.054 INFO:tasks.workunit.client.1.smithi117.stdout:5/565: fsync dd/df/d5e/d91/d72/d35/fa5 0 2024-03-22T06:15:25.056 INFO:tasks.workunit.client.1.smithi117.stdout:7/487: mknod d8/d9/da/d2b/c97 0 2024-03-22T06:15:25.056 INFO:tasks.workunit.client.1.smithi117.stdout:7/488: readlink d8/d9/da/d10/d23/l6b 0 2024-03-22T06:15:25.058 INFO:tasks.workunit.client.1.smithi117.stdout:5/566: mknod dd/df/d88/cc0 0 2024-03-22T06:15:25.060 INFO:tasks.workunit.client.1.smithi117.stdout:5/567: mknod dd/df/dae/cc1 0 2024-03-22T06:15:25.060 INFO:tasks.workunit.client.1.smithi117.stdout:5/568: fsync dd/df/d5e/d91/f23 0 2024-03-22T06:15:25.077 INFO:tasks.workunit.client.1.smithi117.stdout:4/505: creat d0/d4/d2a/fb0 x:0 0 0 2024-03-22T06:15:25.089 INFO:tasks.workunit.client.1.smithi117.stdout:7/489: dwrite d8/d9/d37/d45/d61/f6f [0,4194304] 0 2024-03-22T06:15:25.090 INFO:tasks.workunit.client.1.smithi117.stdout:7/490: fdatasync d8/d9/da/d10/d23/d27/d39/f83 0 2024-03-22T06:15:25.090 INFO:tasks.workunit.client.1.smithi117.stdout:7/491: fdatasync d8/d9/da/d10/d23/d27/d39/f5d 0 2024-03-22T06:15:25.091 INFO:tasks.workunit.client.1.smithi117.stdout:5/569: dwrite dd/df/d5e/d91/d72/d35/d81/fba [0,4194304] 0 2024-03-22T06:15:25.091 INFO:tasks.workunit.client.1.smithi117.stdout:5/570: read dd/df/f2b [7469172,7124] 0 2024-03-22T06:15:25.091 INFO:tasks.workunit.client.1.smithi117.stdout:5/571: fdatasync dd/df/d5e/d91/d72/d60/d6e/f85 0 2024-03-22T06:15:25.093 INFO:tasks.workunit.client.1.smithi117.stdout:7/492: unlink d8/d9/da/d10/d23/d27/d39/d26/c33 0 2024-03-22T06:15:25.105 INFO:tasks.workunit.client.1.smithi117.stdout:4/506: dwrite d0/d3f/f54 [4194304,4194304] 0 2024-03-22T06:15:25.105 INFO:tasks.workunit.client.1.smithi117.stdout:4/507: readlink d0/d4/d2a/d5b/l73 0 2024-03-22T06:15:25.106 INFO:tasks.workunit.client.1.smithi117.stdout:4/508: fsync d0/d4/d2a/d5b/fa6 0 2024-03-22T06:15:25.110 INFO:tasks.workunit.client.1.smithi117.stdout:7/493: dread f7 [0,4194304] 0 2024-03-22T06:15:25.110 INFO:tasks.workunit.client.1.smithi117.stdout:7/494: truncate d8/d9/da/d10/d23/d27/d39/d26/f34 2080877 0 2024-03-22T06:15:25.110 INFO:tasks.workunit.client.1.smithi117.stdout:7/495: chown d8/d6c/l96 19 1 2024-03-22T06:15:25.113 INFO:tasks.workunit.client.1.smithi117.stdout:7/496: unlink d8/f2f 0 2024-03-22T06:15:25.113 INFO:tasks.workunit.client.1.smithi117.stdout:7/497: chown d8/d9/da/d10 1 1 2024-03-22T06:15:25.114 INFO:tasks.workunit.client.1.smithi117.stdout:5/572: dwrite dd/d16/d1a/d32/f51 [0,4194304] 0 2024-03-22T06:15:25.114 INFO:tasks.workunit.client.1.smithi117.stdout:5/573: write dd/df/d5e/d91/f23 [929625,115709] 0 2024-03-22T06:15:25.115 INFO:tasks.workunit.client.1.smithi117.stdout:7/498: symlink d8/d9/d37/d45/d88/l98 0 2024-03-22T06:15:25.122 INFO:tasks.workunit.client.1.smithi117.stdout:5/574: dread dd/d16/d1a/f62 [0,4194304] 0 2024-03-22T06:15:25.122 INFO:tasks.workunit.client.1.smithi117.stdout:8/490: mkdir d1/d19/d9a 0 2024-03-22T06:15:25.122 INFO:tasks.workunit.client.1.smithi117.stdout:8/491: dread - d1/d3a/d44/d11/f95 zero size 2024-03-22T06:15:25.123 INFO:tasks.workunit.client.1.smithi117.stdout:8/492: creat d1/d3a/d44/d6a/f9b x:0 0 0 2024-03-22T06:15:25.125 INFO:tasks.workunit.client.1.smithi117.stdout:4/509: dread d0/f25 [0,4194304] 0 2024-03-22T06:15:25.128 INFO:tasks.workunit.client.1.smithi117.stdout:4/510: mkdir d0/d19/d34/d43/d63/d6f/db1 0 2024-03-22T06:15:25.130 INFO:tasks.workunit.client.1.smithi117.stdout:5/575: dread dd/df/d5e/d91/d72/d33/f68 [4194304,4194304] 0 2024-03-22T06:15:25.134 INFO:tasks.workunit.client.1.smithi117.stdout:4/511: symlink d0/d19/d34/d43/d63/d6f/db1/lb2 0 2024-03-22T06:15:25.136 INFO:tasks.workunit.client.1.smithi117.stdout:5/576: symlink dd/df/d5e/d91/d72/d60/lc2 0 2024-03-22T06:15:25.137 INFO:tasks.workunit.client.1.smithi117.stdout:4/512: mknod d0/d19/d34/cb3 0 2024-03-22T06:15:25.137 INFO:tasks.workunit.client.1.smithi117.stdout:4/513: chown d0 529261 1 2024-03-22T06:15:25.139 INFO:tasks.workunit.client.1.smithi117.stdout:5/577: creat dd/df/d5e/d91/d72/d33/d9a/fc3 x:0 0 0 2024-03-22T06:15:25.139 INFO:tasks.workunit.client.1.smithi117.stdout:5/578: fdatasync dd/df/d5e/d91/d72/d60/d6e/f85 0 2024-03-22T06:15:25.140 INFO:tasks.workunit.client.1.smithi117.stdout:4/514: rename d0/d4/d2a/d5b/l73 to d0/d51/da4/lb4 0 2024-03-22T06:15:25.147 INFO:tasks.workunit.client.1.smithi117.stdout:8/493: dwrite d1/d19/f3f [4194304,4194304] 0 2024-03-22T06:15:25.147 INFO:tasks.workunit.client.1.smithi117.stdout:8/494: chown d1/d3a/d44/d11/d97/f99 92 1 2024-03-22T06:15:25.149 INFO:tasks.workunit.client.1.smithi117.stdout:8/495: mknod d1/c9c 0 2024-03-22T06:15:25.155 INFO:tasks.workunit.client.1.smithi117.stdout:7/499: link d8/d9/da/d10/d23/d27/d39/f5d d8/d9/f99 0 2024-03-22T06:15:25.157 INFO:tasks.workunit.client.1.smithi117.stdout:7/500: mknod d8/d9/da/d10/d23/d27/d56/c9a 0 2024-03-22T06:15:25.159 INFO:tasks.workunit.client.1.smithi117.stdout:7/501: write d8/d9/d37/d5f/f79 [23043,111674] 0 2024-03-22T06:15:25.159 INFO:tasks.workunit.client.1.smithi117.stdout:7/502: dread - d8/d9/da/d10/d23/f92 zero size 2024-03-22T06:15:25.159 INFO:tasks.workunit.client.1.smithi117.stdout:7/503: chown d8/d9/d37/d45/d61 0 1 2024-03-22T06:15:25.159 INFO:tasks.workunit.client.1.smithi117.stdout:7/504: chown l1 2 1 2024-03-22T06:15:25.159 INFO:tasks.workunit.client.1.smithi117.stdout:7/505: chown d8/d9/da/d10/d23/d48/f57 14304639 1 2024-03-22T06:15:25.160 INFO:tasks.workunit.client.1.smithi117.stdout:7/506: mkdir d8/d9/da/d2b/d9b 0 2024-03-22T06:15:25.160 INFO:tasks.workunit.client.1.smithi117.stdout:8/496: dread d1/d19/f51 [0,4194304] 0 2024-03-22T06:15:25.163 INFO:tasks.workunit.client.1.smithi117.stdout:7/507: rename d8/d9/f1e to d8/d9/da/d10/d23/d48/d91/f9c 0 2024-03-22T06:15:25.163 INFO:tasks.workunit.client.1.smithi117.stdout:7/508: read - d8/d9/da/d10/d23/d27/d39/d26/f7f zero size 2024-03-22T06:15:25.164 INFO:tasks.workunit.client.1.smithi117.stdout:8/497: mkdir d1/d28/d66/d7f/d9d 0 2024-03-22T06:15:25.165 INFO:tasks.workunit.client.1.smithi117.stdout:7/509: symlink d8/d9/d37/l9d 0 2024-03-22T06:15:25.165 INFO:tasks.workunit.client.1.smithi117.stdout:7/510: chown d8/d50/d66/l77 142 1 2024-03-22T06:15:25.166 INFO:tasks.workunit.client.1.smithi117.stdout:7/511: creat d8/d9/da/d10/d23/d27/d56/f9e x:0 0 0 2024-03-22T06:15:25.168 INFO:tasks.workunit.client.1.smithi117.stdout:7/512: mknod d8/d50/d5e/c9f 0 2024-03-22T06:15:25.168 INFO:tasks.workunit.client.1.smithi117.stdout:5/579: mknod dd/df/d5e/d91/cc4 0 2024-03-22T06:15:25.168 INFO:tasks.workunit.client.1.smithi117.stdout:5/580: chown dd/d16/d1a/d32/dbf 6479 1 2024-03-22T06:15:25.169 INFO:tasks.workunit.client.1.smithi117.stdout:5/581: creat dd/d16/d1a/d32/d73/db8/fc5 x:0 0 0 2024-03-22T06:15:25.172 INFO:tasks.workunit.client.1.smithi117.stdout:5/582: mknod dd/df/d5e/d91/d36/d98/cc6 0 2024-03-22T06:15:25.182 INFO:tasks.workunit.client.1.smithi117.stdout:7/513: getdents d8/d9/d37/d5f/d82 0 2024-03-22T06:15:25.194 INFO:tasks.workunit.client.1.smithi117.stdout:8/498: dwrite d1/d3a/d44/d11/d97/f99 [0,4194304] 0 2024-03-22T06:15:25.194 INFO:tasks.workunit.client.1.smithi117.stdout:8/499: chown d1/c9c 272951193 1 2024-03-22T06:15:25.194 INFO:tasks.workunit.client.1.smithi117.stdout:8/500: creat d1/d28/d66/d7f/f9e x:0 0 0 2024-03-22T06:15:25.194 INFO:tasks.workunit.client.1.smithi117.stdout:5/583: dwrite dd/f7d [0,4194304] 0 2024-03-22T06:15:25.206 INFO:tasks.workunit.client.1.smithi117.stdout:8/501: dread d1/d19/d38/f70 [0,4194304] 0 2024-03-22T06:15:25.206 INFO:tasks.workunit.client.1.smithi117.stdout:8/502: chown d1/f87 44 1 2024-03-22T06:15:25.208 INFO:tasks.workunit.client.1.smithi117.stdout:8/503: write d1/d19/f2d [3766984,11187] 0 2024-03-22T06:15:25.209 INFO:tasks.workunit.client.1.smithi117.stdout:7/514: dwrite d8/d9/da/d10/d23/d27/d39/f4a [0,4194304] 0 2024-03-22T06:15:25.209 INFO:tasks.workunit.client.1.smithi117.stdout:7/515: dread - d8/d9/d37/d45/f7e zero size 2024-03-22T06:15:25.209 INFO:tasks.workunit.client.1.smithi117.stdout:5/584: write dd/df/d5e/d91/d36/f31 [2789335,12134] 0 2024-03-22T06:15:25.209 INFO:tasks.workunit.client.1.smithi117.stdout:5/585: write dd/df/d5e/f92 [354755,53333] 0 2024-03-22T06:15:25.210 INFO:tasks.workunit.client.1.smithi117.stdout:5/586: write dd/df/d5e/f92 [177806,70787] 0 2024-03-22T06:15:25.215 INFO:tasks.workunit.client.1.smithi117.stdout:5/587: creat dd/df/d5e/d91/d72/d35/da2/fc7 x:0 0 0 2024-03-22T06:15:25.218 INFO:tasks.workunit.client.1.smithi117.stdout:8/504: read d1/d3a/d44/d11/d97/f99 [3650260,53819] 0 2024-03-22T06:15:25.219 INFO:tasks.workunit.client.1.smithi117.stdout:8/505: chown d1/d3a/d44/d16/c21 0 1 2024-03-22T06:15:25.219 INFO:tasks.workunit.client.1.smithi117.stdout:8/506: fdatasync d1/d19/d38/f8c 0 2024-03-22T06:15:25.219 INFO:tasks.workunit.client.1.smithi117.stdout:8/507: write d1/d28/d66/d7f/f92 [445631,18533] 0 2024-03-22T06:15:25.222 INFO:tasks.workunit.client.1.smithi117.stdout:2/521: sync 2024-03-22T06:15:25.222 INFO:tasks.workunit.client.1.smithi117.stdout:6/581: sync 2024-03-22T06:15:25.223 INFO:tasks.workunit.client.1.smithi117.stdout:1/579: sync 2024-03-22T06:15:25.223 INFO:tasks.workunit.client.1.smithi117.stdout:0/550: sync 2024-03-22T06:15:25.223 INFO:tasks.workunit.client.1.smithi117.stdout:3/500: sync 2024-03-22T06:15:25.223 INFO:tasks.workunit.client.1.smithi117.stdout:2/522: fsync d2/d5/d13/f7a 0 2024-03-22T06:15:25.227 INFO:tasks.workunit.client.1.smithi117.stdout:3/501: dread d9/d55/f67 [0,4194304] 0 2024-03-22T06:15:25.227 INFO:tasks.workunit.client.1.smithi117.stdout:3/502: creat d9/d93/fb7 x:0 0 0 2024-03-22T06:15:25.227 INFO:tasks.workunit.client.1.smithi117.stdout:5/588: mknod dd/df/d5e/d91/cc8 0 2024-03-22T06:15:25.227 INFO:tasks.workunit.client.1.smithi117.stdout:5/589: rename dd/df/d5e to dd/df/d5e/d91/d72/d35/dc9 22 2024-03-22T06:15:25.228 INFO:tasks.workunit.client.1.smithi117.stdout:8/508: unlink d1/d3a/d44/c58 0 2024-03-22T06:15:25.230 INFO:tasks.workunit.client.1.smithi117.stdout:1/580: creat d1/d3c/fcd x:0 0 0 2024-03-22T06:15:25.231 INFO:tasks.workunit.client.1.smithi117.stdout:6/582: creat d3/db4/d2d/d42/fca x:0 0 0 2024-03-22T06:15:25.232 INFO:tasks.workunit.client.1.smithi117.stdout:6/583: chown d3/db4/d2d/d53/d59/d74/l25 415916 1 2024-03-22T06:15:25.232 INFO:tasks.workunit.client.1.smithi117.stdout:6/584: write d3/db4/d2d/d53/d59/d74/da/fc6 [672238,93546] 0 2024-03-22T06:15:25.234 INFO:tasks.workunit.client.1.smithi117.stdout:7/516: dwrite d8/d9/d37/d45/f7e [0,4194304] 0 2024-03-22T06:15:25.234 INFO:tasks.workunit.client.1.smithi117.stdout:7/517: creat d8/d9/da/d10/d23/d27/d32/fa0 x:0 0 0 2024-03-22T06:15:25.234 INFO:tasks.workunit.client.1.smithi117.stdout:7/518: read f7 [3473276,95238] 0 2024-03-22T06:15:25.234 INFO:tasks.workunit.client.1.smithi117.stdout:3/503: symlink d9/d1a/d39/d4c/lb8 0 2024-03-22T06:15:25.234 INFO:tasks.workunit.client.1.smithi117.stdout:0/551: read d1/d10/f11 [11934781,20941] 0 2024-03-22T06:15:25.235 INFO:tasks.workunit.client.1.smithi117.stdout:3/504: write d9/d55/f7b [991879,125073] 0 2024-03-22T06:15:25.238 INFO:tasks.workunit.client.1.smithi117.stdout:8/509: unlink d1/d3a/d44/d16/l3d 0 2024-03-22T06:15:25.238 INFO:tasks.workunit.client.1.smithi117.stdout:8/510: creat d1/d19/d38/f9f x:0 0 0 2024-03-22T06:15:25.238 INFO:tasks.workunit.client.1.smithi117.stdout:6/585: getdents d3/d3d/d17 0 2024-03-22T06:15:25.243 INFO:tasks.workunit.client.1.smithi117.stdout:3/505: mknod d9/d1a/d39/d4c/d99/cb9 0 2024-03-22T06:15:25.243 INFO:tasks.workunit.client.1.smithi117.stdout:3/506: chown d9/d55/f7b 0 1 2024-03-22T06:15:25.245 INFO:tasks.workunit.client.1.smithi117.stdout:7/519: dread d8/d9/da/d10/d23/d27/f3e [0,4194304] 0 2024-03-22T06:15:25.246 INFO:tasks.workunit.client.1.smithi117.stdout:6/586: mkdir d3/d3d/dcb 0 2024-03-22T06:15:25.248 INFO:tasks.workunit.client.1.smithi117.stdout:3/507: creat d9/d1a/d39/d7d/d97/fba x:0 0 0 2024-03-22T06:15:25.249 INFO:tasks.workunit.client.1.smithi117.stdout:3/508: rmdir d9/d1a/d39/d82 39 2024-03-22T06:15:25.252 INFO:tasks.workunit.client.1.smithi117.stdout:9/561: mknod d4/d8/cad 0 2024-03-22T06:15:25.255 INFO:tasks.workunit.client.1.smithi117.stdout:9/562: mkdir d4/d7/d27/d5b/dae 0 2024-03-22T06:15:25.259 INFO:tasks.workunit.client.1.smithi117.stdout:7/520: dread d8/d9/d37/d45/f7e [0,4194304] 0 2024-03-22T06:15:25.263 INFO:tasks.workunit.client.1.smithi117.stdout:3/509: write d9/f18 [6663250,9079] 0 2024-03-22T06:15:25.263 INFO:tasks.workunit.client.1.smithi117.stdout:7/521: mkdir d8/d9/da/d10/da1 0 2024-03-22T06:15:25.264 INFO:tasks.workunit.client.1.smithi117.stdout:7/522: fdatasync d8/d9/f7c 0 2024-03-22T06:15:25.264 INFO:tasks.workunit.client.1.smithi117.stdout:7/523: write f7 [1227260,9233] 0 2024-03-22T06:15:25.268 INFO:tasks.workunit.client.1.smithi117.stdout:5/590: truncate dd/f7d 2701088 0 2024-03-22T06:15:25.270 INFO:tasks.workunit.client.1.smithi117.stdout:0/552: dwrite d1/f3d [0,4194304] 0 2024-03-22T06:15:25.270 INFO:tasks.workunit.client.1.smithi117.stdout:0/553: readlink d1/d10/d18/l1b 0 2024-03-22T06:15:25.270 INFO:tasks.workunit.client.1.smithi117.stdout:0/554: getdents d1/d3/d2b/d3e/dad 0 2024-03-22T06:15:25.280 INFO:tasks.workunit.client.1.smithi117.stdout:0/555: dread d1/f4f [0,4194304] 0 2024-03-22T06:15:25.281 INFO:tasks.workunit.client.1.smithi117.stdout:6/587: dwrite d3/db4/f5e [0,4194304] 0 2024-03-22T06:15:25.281 INFO:tasks.workunit.client.1.smithi117.stdout:1/581: dwrite d1/d14/f19 [0,4194304] 0 2024-03-22T06:15:25.282 INFO:tasks.workunit.client.1.smithi117.stdout:3/510: dread d9/d1a/d3a/f53 [0,4194304] 0 2024-03-22T06:15:25.282 INFO:tasks.workunit.client.1.smithi117.stdout:3/511: chown d9/d1a/d3a 835 1 2024-03-22T06:15:25.282 INFO:tasks.workunit.client.1.smithi117.stdout:0/556: unlink d1/d10/d18/d26/d49/d7e/d39/f90 0 2024-03-22T06:15:25.286 INFO:tasks.workunit.client.1.smithi117.stdout:1/582: rename d1/d30/d44/d9c/da9/cba to d1/d30/cce 0 2024-03-22T06:15:25.287 INFO:tasks.workunit.client.1.smithi117.stdout:6/588: mkdir d3/d3d/dcb/dcc 0 2024-03-22T06:15:25.288 INFO:tasks.workunit.client.1.smithi117.stdout:6/589: fdatasync d3/d3d/d17/d31/f78 0 2024-03-22T06:15:25.290 INFO:tasks.workunit.client.1.smithi117.stdout:0/557: truncate d1/d10/d5c/d40/f41 3270470 0 2024-03-22T06:15:25.291 INFO:tasks.workunit.client.1.smithi117.stdout:8/511: dwrite d1/d19/d38/f75 [0,4194304] 0 2024-03-22T06:15:25.293 INFO:tasks.workunit.client.1.smithi117.stdout:9/563: dwrite d4/d50/f6b [0,4194304] 0 2024-03-22T06:15:25.293 INFO:tasks.workunit.client.1.smithi117.stdout:6/590: mknod d3/db4/d2d/d53/d59/d74/da/db3/ccd 0 2024-03-22T06:15:25.302 INFO:tasks.workunit.client.1.smithi117.stdout:8/512: dread d1/d19/d38/f75 [0,4194304] 0 2024-03-22T06:15:25.302 INFO:tasks.workunit.client.1.smithi117.stdout:7/524: dwrite d8/d50/d5e/f8e [0,4194304] 0 2024-03-22T06:15:25.307 INFO:tasks.workunit.client.1.smithi117.stdout:9/564: rename d4/d46/la1 to d4/d8/d3e/d7a/da9/laf 0 2024-03-22T06:15:25.307 INFO:tasks.workunit.client.1.smithi117.stdout:9/565: readlink d4/d47/d85/l91 0 2024-03-22T06:15:25.312 INFO:tasks.workunit.client.1.smithi117.stdout:6/591: rmdir d3/d40/d46/d84/d49/d79/d9d/dbf 0 2024-03-22T06:15:25.312 INFO:tasks.workunit.client.1.smithi117.stdout:8/513: dread d1/d19/f3f [4194304,4194304] 0 2024-03-22T06:15:25.312 INFO:tasks.workunit.client.1.smithi117.stdout:8/514: write d1/d28/d66/d7f/f92 [1482616,12631] 0 2024-03-22T06:15:25.320 INFO:tasks.workunit.client.1.smithi117.stdout:6/592: unlink d3/db4/d2d/d53/d59/fb6 0 2024-03-22T06:15:25.321 INFO:tasks.workunit.client.1.smithi117.stdout:1/583: dwrite d1/f7 [4194304,4194304] 0 2024-03-22T06:15:25.321 INFO:tasks.workunit.client.1.smithi117.stdout:1/584: truncate d1/d22/d27/faf 480715 0 2024-03-22T06:15:25.322 INFO:tasks.workunit.client.1.smithi117.stdout:8/515: symlink d1/d3a/d44/d11/d97/la0 0 2024-03-22T06:15:25.329 INFO:tasks.workunit.client.1.smithi117.stdout:8/516: link d1/d3a/d44/l4 d1/d28/la1 0 2024-03-22T06:15:25.330 INFO:tasks.workunit.client.1.smithi117.stdout:8/517: creat d1/d28/d6b/fa2 x:0 0 0 2024-03-22T06:15:25.330 INFO:tasks.workunit.client.1.smithi117.stdout:9/566: dwrite d4/d7/d27/f2e [0,4194304] 0 2024-03-22T06:15:25.333 INFO:tasks.workunit.client.1.smithi117.stdout:9/567: dread d4/d8/d3e/d7a/d2f/f39 [0,4194304] 0 2024-03-22T06:15:25.333 INFO:tasks.workunit.client.1.smithi117.stdout:9/568: chown d4/d7/c44 6880530 1 2024-03-22T06:15:25.336 INFO:tasks.workunit.client.1.smithi117.stdout:9/569: rename d4/d8/f22 to d4/d8/d3e/d7a/d2f/fb0 0 2024-03-22T06:15:25.342 INFO:tasks.workunit.client.1.smithi117.stdout:9/570: dread d4/d8/d3e/d7a/f63 [0,4194304] 0 2024-03-22T06:15:25.342 INFO:tasks.workunit.client.1.smithi117.stdout:9/571: write d4/d8/d3e/d7a/d2f/f54 [435122,36240] 0 2024-03-22T06:15:25.343 INFO:tasks.workunit.client.1.smithi117.stdout:9/572: readlink d4/d7/d27/l90 0 2024-03-22T06:15:25.345 INFO:tasks.workunit.client.1.smithi117.stdout:9/573: mkdir d4/d8/d93/db1 0 2024-03-22T06:15:25.345 INFO:tasks.workunit.client.1.smithi117.stdout:9/574: dread - d4/d7/f7f zero size 2024-03-22T06:15:25.347 INFO:tasks.workunit.client.1.smithi117.stdout:9/575: creat d4/d8/d37/fb2 x:0 0 0 2024-03-22T06:15:25.348 INFO:tasks.workunit.client.1.smithi117.stdout:9/576: getdents d4/d7/d27/d5b/dae 0 2024-03-22T06:15:25.349 INFO:tasks.workunit.client.1.smithi117.stdout:9/577: creat d4/d47/d85/dab/fb3 x:0 0 0 2024-03-22T06:15:25.349 INFO:tasks.workunit.client.1.smithi117.stdout:9/578: dread - d4/d47/d85/fa8 zero size 2024-03-22T06:15:25.356 INFO:tasks.workunit.client.1.smithi117.stdout:9/579: dread d4/d8/d3e/d7a/f28 [0,4194304] 0 2024-03-22T06:15:25.357 INFO:tasks.workunit.client.1.smithi117.stdout:9/580: creat d4/d47/d64/fb4 x:0 0 0 2024-03-22T06:15:25.358 INFO:tasks.workunit.client.1.smithi117.stdout:9/581: symlink d4/d8/lb5 0 2024-03-22T06:15:25.359 INFO:tasks.workunit.client.1.smithi117.stdout:9/582: symlink d4/d8/d3e/d7a/da9/lb6 0 2024-03-22T06:15:25.366 INFO:tasks.workunit.client.1.smithi117.stdout:9/583: dread d4/d8/d3e/d7a/d3b/f57 [4194304,4194304] 0 2024-03-22T06:15:25.378 INFO:tasks.workunit.client.1.smithi117.stdout:4/515: sync 2024-03-22T06:15:25.378 INFO:tasks.workunit.client.1.smithi117.stdout:4/516: chown d0/d4/d2a/d5b/f6e 1911 1 2024-03-22T06:15:25.379 INFO:tasks.workunit.client.1.smithi117.stdout:8/518: dread f0 [0,4194304] 0 2024-03-22T06:15:25.380 INFO:tasks.workunit.client.1.smithi117.stdout:8/519: chown d1/d19/d38/f9f 56617743 1 2024-03-22T06:15:25.380 INFO:tasks.workunit.client.1.smithi117.stdout:8/520: truncate d1/d28/d66/d7f/f94 29025 0 2024-03-22T06:15:25.389 INFO:tasks.workunit.client.1.smithi117.stdout:4/517: dread d0/f83 [0,4194304] 0 2024-03-22T06:15:25.390 INFO:tasks.workunit.client.1.smithi117.stdout:4/518: mkdir d0/d19/d34/d43/d63/db5 0 2024-03-22T06:15:25.391 INFO:tasks.workunit.client.1.smithi117.stdout:4/519: mkdir d0/d19/d34/d43/d63/d6f/db1/db6 0 2024-03-22T06:15:25.391 INFO:tasks.workunit.client.1.smithi117.stdout:4/520: creat d0/d48/da0/fb7 x:0 0 0 2024-03-22T06:15:25.392 INFO:tasks.workunit.client.1.smithi117.stdout:4/521: dread d0/d19/d34/f77 [0,4194304] 0 2024-03-22T06:15:25.393 INFO:tasks.workunit.client.1.smithi117.stdout:4/522: truncate d0/d19/d28/f59 1228609 0 2024-03-22T06:15:25.393 INFO:tasks.workunit.client.1.smithi117.stdout:4/523: truncate d0/d51/f7e 1068675 0 2024-03-22T06:15:25.394 INFO:tasks.workunit.client.1.smithi117.stdout:4/524: mkdir d0/d4/d2a/d2f/db8 0 2024-03-22T06:15:25.396 INFO:tasks.workunit.client.1.smithi117.stdout:9/584: dwrite d4/d46/f6d [0,4194304] 0 2024-03-22T06:15:25.397 INFO:tasks.workunit.client.1.smithi117.stdout:8/521: dwrite d1/d3a/d44/d11/d97/f99 [0,4194304] 0 2024-03-22T06:15:25.414 INFO:tasks.workunit.client.1.smithi117.stdout:4/525: getdents d0/d48/da0 0 2024-03-22T06:15:25.414 INFO:tasks.workunit.client.1.smithi117.stdout:4/526: creat d0/d19/d34/d43/d63/d6f/d70/fb9 x:0 0 0 2024-03-22T06:15:25.418 INFO:tasks.workunit.client.1.smithi117.stdout:2/523: sync 2024-03-22T06:15:25.418 INFO:tasks.workunit.client.1.smithi117.stdout:5/591: sync 2024-03-22T06:15:25.418 INFO:tasks.workunit.client.1.smithi117.stdout:7/525: sync 2024-03-22T06:15:25.418 INFO:tasks.workunit.client.1.smithi117.stdout:0/558: sync 2024-03-22T06:15:25.418 INFO:tasks.workunit.client.1.smithi117.stdout:3/512: sync 2024-03-22T06:15:25.419 INFO:tasks.workunit.client.1.smithi117.stdout:5/592: mkdir dd/df/d5e/d91/d90/dca 0 2024-03-22T06:15:25.419 INFO:tasks.workunit.client.1.smithi117.stdout:5/593: write dd/d16/f64 [1354914,122869] 0 2024-03-22T06:15:25.419 INFO:tasks.workunit.client.1.smithi117.stdout:2/524: mkdir d2/d5/d13/d3f/d1d/d3a/da6 0 2024-03-22T06:15:25.420 INFO:tasks.workunit.client.1.smithi117.stdout:5/594: dread dd/df/d5e/d91/d72/d35/f6f [4194304,4194304] 0 2024-03-22T06:15:25.420 INFO:tasks.workunit.client.1.smithi117.stdout:7/526: rename d8/d9/c43 to d8/d9/da/d10/d23/d48/d91/ca2 0 2024-03-22T06:15:25.421 INFO:tasks.workunit.client.1.smithi117.stdout:0/559: mkdir d1/d10/d18/d26/d49/d7e/d39/dc5 0 2024-03-22T06:15:25.421 INFO:tasks.workunit.client.1.smithi117.stdout:8/522: dwrite d1/d3a/d44/fa [4194304,4194304] 0 2024-03-22T06:15:25.422 INFO:tasks.workunit.client.1.smithi117.stdout:5/595: rmdir dd/df/d5e/d91/d36/d98 39 2024-03-22T06:15:25.423 INFO:tasks.workunit.client.1.smithi117.stdout:3/513: rename d9/d1a/f1f to d9/d1a/d39/d4c/d99/fbb 0 2024-03-22T06:15:25.424 INFO:tasks.workunit.client.1.smithi117.stdout:9/585: dwrite d4/d7/d27/f2e [0,4194304] 0 2024-03-22T06:15:25.424 INFO:tasks.workunit.client.1.smithi117.stdout:9/586: truncate d4/d7/d27/d5b/f78 1394990 0 2024-03-22T06:15:25.424 INFO:tasks.workunit.client.1.smithi117.stdout:7/527: symlink d8/d9/da/d10/d23/la3 0 2024-03-22T06:15:25.424 INFO:tasks.workunit.client.1.smithi117.stdout:7/528: dread - d8/d9/da/f69 zero size 2024-03-22T06:15:25.425 INFO:tasks.workunit.client.1.smithi117.stdout:0/560: rmdir d1/d3 39 2024-03-22T06:15:25.425 INFO:tasks.workunit.client.1.smithi117.stdout:0/561: fsync d1/d3/d2b/d3e/de/f23 0 2024-03-22T06:15:25.426 INFO:tasks.workunit.client.1.smithi117.stdout:0/562: readlink d1/d10/d18/la5 0 2024-03-22T06:15:25.426 INFO:tasks.workunit.client.1.smithi117.stdout:9/587: dread d4/d7/d27/d5b/f78 [0,4194304] 0 2024-03-22T06:15:25.426 INFO:tasks.workunit.client.1.smithi117.stdout:9/588: write d4/d8/d37/d58/f92 [1160753,107046] 0 2024-03-22T06:15:25.426 INFO:tasks.workunit.client.1.smithi117.stdout:9/589: creat d4/d7/d27/d5b/fb7 x:0 0 0 2024-03-22T06:15:25.426 INFO:tasks.workunit.client.1.smithi117.stdout:9/590: chown d4/d8/d93 26 1 2024-03-22T06:15:25.426 INFO:tasks.workunit.client.1.smithi117.stdout:9/591: creat d4/d47/d85/fb8 x:0 0 0 2024-03-22T06:15:25.427 INFO:tasks.workunit.client.1.smithi117.stdout:4/527: dread d0/d4/d2a/f30 [0,4194304] 0 2024-03-22T06:15:25.429 INFO:tasks.workunit.client.1.smithi117.stdout:7/529: write d8/d9/da/f72 [739897,52566] 0 2024-03-22T06:15:25.429 INFO:tasks.workunit.client.1.smithi117.stdout:7/530: creat d8/d9/da/d10/d23/d27/d56/fa4 x:0 0 0 2024-03-22T06:15:25.429 INFO:tasks.workunit.client.1.smithi117.stdout:7/531: write d8/fb [9046450,111194] 0 2024-03-22T06:15:25.430 INFO:tasks.workunit.client.1.smithi117.stdout:8/523: write d1/d3a/d44/d11/d97/f99 [1725337,72501] 0 2024-03-22T06:15:25.431 INFO:tasks.workunit.client.1.smithi117.stdout:8/524: chown d1/d28/la1 5955 1 2024-03-22T06:15:25.433 INFO:tasks.workunit.client.1.smithi117.stdout:0/563: creat d1/d10/d5c/d40/d4e/dbc/fc6 x:0 0 0 2024-03-22T06:15:25.433 INFO:tasks.workunit.client.1.smithi117.stdout:0/564: creat d1/d10/d18/d8c/fc7 x:0 0 0 2024-03-22T06:15:25.439 INFO:tasks.workunit.client.1.smithi117.stdout:3/514: dwrite d9/f26 [0,4194304] 0 2024-03-22T06:15:25.439 INFO:tasks.workunit.client.1.smithi117.stdout:6/593: sync 2024-03-22T06:15:25.439 INFO:tasks.workunit.client.1.smithi117.stdout:1/585: sync 2024-03-22T06:15:25.439 INFO:tasks.workunit.client.1.smithi117.stdout:6/594: chown d3/db4/d2d/d53/d59/d74/l25 1 1 2024-03-22T06:15:25.440 INFO:tasks.workunit.client.1.smithi117.stdout:8/525: dread d1/d19/f51 [8388608,4194304] 0 2024-03-22T06:15:25.440 INFO:tasks.workunit.client.1.smithi117.stdout:6/595: dread d3/db4/d2d/d53/d59/d74/da/de/f47 [0,4194304] 0 2024-03-22T06:15:25.440 INFO:tasks.workunit.client.1.smithi117.stdout:2/525: rename d2/d5/d13/d57/f47 to d2/d5/da5/fa7 0 2024-03-22T06:15:25.448 INFO:tasks.workunit.client.1.smithi117.stdout:7/532: unlink d8/d9/da/d10/c8b 0 2024-03-22T06:15:25.448 INFO:tasks.workunit.client.1.smithi117.stdout:3/515: write d9/f19 [182786,116953] 0 2024-03-22T06:15:25.450 INFO:tasks.workunit.client.1.smithi117.stdout:3/516: dread d9/d1a/d3a/f7a [0,4194304] 0 2024-03-22T06:15:25.451 INFO:tasks.workunit.client.1.smithi117.stdout:0/565: creat d1/d10/d18/d26/d49/d7e/d39/dc5/fc8 x:0 0 0 2024-03-22T06:15:25.453 INFO:tasks.workunit.client.1.smithi117.stdout:6/596: dwrite d3/db4/d2d/d53/d75/fb0 [4194304,4194304] 0 2024-03-22T06:15:25.453 INFO:tasks.workunit.client.1.smithi117.stdout:9/592: mkdir d4/db9 0 2024-03-22T06:15:25.453 INFO:tasks.workunit.client.1.smithi117.stdout:9/593: fdatasync d4/d8/d3e/d7a/f63 0 2024-03-22T06:15:25.459 INFO:tasks.workunit.client.1.smithi117.stdout:1/586: symlink d1/d30/lcf 0 2024-03-22T06:15:25.459 INFO:tasks.workunit.client.1.smithi117.stdout:1/587: chown d1/d1f/f8f 0 1 2024-03-22T06:15:25.460 INFO:tasks.workunit.client.1.smithi117.stdout:8/526: unlink d1/d19/f3f 0 2024-03-22T06:15:25.462 INFO:tasks.workunit.client.1.smithi117.stdout:5/596: rename dd/df/d5e/c77 to dd/df/d5e/d91/d72/ccb 0 2024-03-22T06:15:25.462 INFO:tasks.workunit.client.1.smithi117.stdout:5/597: creat dd/df/d5e/d91/d72/d35/fcc x:0 0 0 2024-03-22T06:15:25.463 INFO:tasks.workunit.client.1.smithi117.stdout:2/526: mknod d2/d5/db/d2c/ca8 0 2024-03-22T06:15:25.463 INFO:tasks.workunit.client.1.smithi117.stdout:2/527: chown d2/fc 9 1 2024-03-22T06:15:25.463 INFO:tasks.workunit.client.1.smithi117.stdout:3/517: dwrite d9/d1a/f83 [0,4194304] 0 2024-03-22T06:15:25.463 INFO:tasks.workunit.client.1.smithi117.stdout:3/518: truncate d9/faa 628974 0 2024-03-22T06:15:25.465 INFO:tasks.workunit.client.1.smithi117.stdout:7/533: mkdir d8/d9/da/d2b/d9b/da5 0 2024-03-22T06:15:25.467 INFO:tasks.workunit.client.1.smithi117.stdout:0/566: rmdir d1 39 2024-03-22T06:15:25.472 INFO:tasks.workunit.client.1.smithi117.stdout:6/597: creat d3/d40/d46/d84/d49/d79/d9d/fce x:0 0 0 2024-03-22T06:15:25.474 INFO:tasks.workunit.client.1.smithi117.stdout:9/594: mknod d4/d8/d3f/cba 0 2024-03-22T06:15:25.477 INFO:tasks.workunit.client.1.smithi117.stdout:1/588: creat d1/d1f/d32/dc6/fd0 x:0 0 0 2024-03-22T06:15:25.479 INFO:tasks.workunit.client.1.smithi117.stdout:1/589: dread d1/d30/d44/f7b [0,4194304] 0 2024-03-22T06:15:25.482 INFO:tasks.workunit.client.1.smithi117.stdout:4/528: rename d0/d19/d28/da3/cac to d0/d19/d34/d43/cba 0 2024-03-22T06:15:25.482 INFO:tasks.workunit.client.1.smithi117.stdout:4/529: write d0/d19/d28/f2e [2308501,62623] 0 2024-03-22T06:15:25.482 INFO:tasks.workunit.client.1.smithi117.stdout:4/530: fdatasync d0/d4/d2a/d5b/fa6 0 2024-03-22T06:15:25.482 INFO:tasks.workunit.client.1.smithi117.stdout:4/531: truncate d0/d19/f7f 859506 0 2024-03-22T06:15:25.485 INFO:tasks.workunit.client.1.smithi117.stdout:0/567: dwrite d1/d3/d2b/d3e/de/d1f/d3b/f6b [0,4194304] 0 2024-03-22T06:15:25.486 INFO:tasks.workunit.client.1.smithi117.stdout:5/598: mkdir dd/df/dae/dcd 0 2024-03-22T06:15:25.486 INFO:tasks.workunit.client.1.smithi117.stdout:5/599: creat dd/d16/d1a/d32/d73/db8/fce x:0 0 0 2024-03-22T06:15:25.486 INFO:tasks.workunit.client.1.smithi117.stdout:5/600: creat dd/df/d5e/d91/d72/d35/da2/fcf x:0 0 0 2024-03-22T06:15:25.487 INFO:tasks.workunit.client.1.smithi117.stdout:2/528: symlink d2/d9a/la9 0 2024-03-22T06:15:25.493 INFO:tasks.workunit.client.1.smithi117.stdout:7/534: unlink d8/d9/da/d10/f13 0 2024-03-22T06:15:25.494 INFO:tasks.workunit.client.1.smithi117.stdout:6/598: creat d3/d40/d46/d84/d49/d79/fcf x:0 0 0 2024-03-22T06:15:25.495 INFO:tasks.workunit.client.1.smithi117.stdout:9/595: creat d4/d47/fbb x:0 0 0 2024-03-22T06:15:25.497 INFO:tasks.workunit.client.1.smithi117.stdout:8/527: rmdir d1/d19 39 2024-03-22T06:15:25.499 INFO:tasks.workunit.client.1.smithi117.stdout:1/590: link d1/d22/l57 d1/d84/ld1 0 2024-03-22T06:15:25.499 INFO:tasks.workunit.client.1.smithi117.stdout:1/591: creat d1/d84/d77/fd2 x:0 0 0 2024-03-22T06:15:25.499 INFO:tasks.workunit.client.1.smithi117.stdout:1/592: chown d1/d1f/f7f 22222 1 2024-03-22T06:15:25.499 INFO:tasks.workunit.client.1.smithi117.stdout:1/593: write d1/d3c/fcd [926708,15465] 0 2024-03-22T06:15:25.499 INFO:tasks.workunit.client.1.smithi117.stdout:1/594: chown d1/d30/d35/d58/f9e 12746 1 2024-03-22T06:15:25.499 INFO:tasks.workunit.client.1.smithi117.stdout:1/595: read d1/f7 [3249859,101605] 0 2024-03-22T06:15:25.501 INFO:tasks.workunit.client.1.smithi117.stdout:3/519: rename d9/c16 to d9/d1a/d39/d8a/da1/db6/cbc 0 2024-03-22T06:15:25.501 INFO:tasks.workunit.client.1.smithi117.stdout:3/520: creat d9/d66/fbd x:0 0 0 2024-03-22T06:15:25.502 INFO:tasks.workunit.client.1.smithi117.stdout:4/532: creat d0/d19/d28/d47/fbb x:0 0 0 2024-03-22T06:15:25.508 INFO:tasks.workunit.client.1.smithi117.stdout:0/568: creat d1/d3/d2b/d3e/dad/fc9 x:0 0 0 2024-03-22T06:15:25.508 INFO:tasks.workunit.client.1.smithi117.stdout:0/569: chown d1/d10/d5c/d40/d4e/dbc/fc6 197735 1 2024-03-22T06:15:25.509 INFO:tasks.workunit.client.1.smithi117.stdout:0/570: getdents d1/d10/d18/db0 0 2024-03-22T06:15:25.515 INFO:tasks.workunit.client.1.smithi117.stdout:5/601: rmdir dd/df/d5e/d91/d72/d33 39 2024-03-22T06:15:25.520 INFO:tasks.workunit.client.1.smithi117.stdout:0/571: write d1/d3/d2b/d3e/de/d1f/d3b/f42 [740517,103119] 0 2024-03-22T06:15:25.520 INFO:tasks.workunit.client.1.smithi117.stdout:0/572: fdatasync d1/d10/d18/d26/d49/d7e/f25 0 2024-03-22T06:15:25.520 INFO:tasks.workunit.client.1.smithi117.stdout:0/573: write d1/d3/d2b/fc0 [258678,3421] 0 2024-03-22T06:15:25.521 INFO:tasks.workunit.client.1.smithi117.stdout:6/599: link d3/d3d/f2e d3/d3d/d17/d60/fd0 0 2024-03-22T06:15:25.523 INFO:tasks.workunit.client.1.smithi117.stdout:9/596: mkdir d4/d8/d93/db1/dbc 0 2024-03-22T06:15:25.523 INFO:tasks.workunit.client.1.smithi117.stdout:9/597: chown d4/d7/c5e 4 1 2024-03-22T06:15:25.524 INFO:tasks.workunit.client.1.smithi117.stdout:1/596: dwrite d1/d62/f67 [4194304,4194304] 0 2024-03-22T06:15:25.525 INFO:tasks.workunit.client.1.smithi117.stdout:3/521: dwrite d9/d1a/d39/d4c/d37/f61 [0,4194304] 0 2024-03-22T06:15:25.525 INFO:tasks.workunit.client.1.smithi117.stdout:8/528: mkdir d1/d28/d6b/da3 0 2024-03-22T06:15:25.533 INFO:tasks.workunit.client.1.smithi117.stdout:5/602: mknod dd/d16/d1a/d32/d73/cd0 0 2024-03-22T06:15:25.535 INFO:tasks.workunit.client.1.smithi117.stdout:7/535: symlink d8/d9/da/d10/la6 0 2024-03-22T06:15:25.541 INFO:tasks.workunit.client.1.smithi117.stdout:6/600: mknod d3/d40/d46/d84/d49/cd1 0 2024-03-22T06:15:25.543 INFO:tasks.workunit.client.1.smithi117.stdout:9/598: rmdir d4/d8/d93 39 2024-03-22T06:15:25.543 INFO:tasks.workunit.client.1.smithi117.stdout:7/536: dread d8/d9/da/d10/f19 [0,4194304] 0 2024-03-22T06:15:25.546 INFO:tasks.workunit.client.1.smithi117.stdout:1/597: mkdir d1/d30/d44/dd3 0 2024-03-22T06:15:25.546 INFO:tasks.workunit.client.1.smithi117.stdout:1/598: chown d1/d3c/db6 4447 1 2024-03-22T06:15:25.546 INFO:tasks.workunit.client.1.smithi117.stdout:1/599: truncate d1/d22/d27/d4e/f6a 1185520 0 2024-03-22T06:15:25.548 INFO:tasks.workunit.client.1.smithi117.stdout:3/522: mknod d9/d1a/d39/d7d/cbe 0 2024-03-22T06:15:25.550 INFO:tasks.workunit.client.1.smithi117.stdout:8/529: symlink d1/d3a/d44/d9/la4 0 2024-03-22T06:15:25.550 INFO:tasks.workunit.client.1.smithi117.stdout:8/530: creat d1/d28/d6b/fa5 x:0 0 0 2024-03-22T06:15:25.552 INFO:tasks.workunit.client.1.smithi117.stdout:2/529: rename d2/d5/d13/d3f/d1d/d3a/d11 to d2/d5/d13/d3f/d1d/d3a/d89/daa 0 2024-03-22T06:15:25.552 INFO:tasks.workunit.client.1.smithi117.stdout:2/530: creat d2/d5/d13/d3f/d1d/d3a/d89/fab x:0 0 0 2024-03-22T06:15:25.552 INFO:tasks.workunit.client.1.smithi117.stdout:2/531: getdents d2/d5/d41/d83 0 2024-03-22T06:15:25.552 INFO:tasks.workunit.client.1.smithi117.stdout:2/532: creat d2/d1b/d39/fac x:0 0 0 2024-03-22T06:15:25.552 INFO:tasks.workunit.client.1.smithi117.stdout:2/533: truncate d2/d5/d13/d3f/d1d/d3a/d71/f9d 1255568 0 2024-03-22T06:15:25.552 INFO:tasks.workunit.client.1.smithi117.stdout:2/534: fsync d2/f26 0 2024-03-22T06:15:25.553 INFO:tasks.workunit.client.1.smithi117.stdout:4/533: rmdir d0/d19/d34/d43/d63/d6f/d70 39 2024-03-22T06:15:25.554 INFO:tasks.workunit.client.1.smithi117.stdout:5/603: symlink dd/df/d5e/d91/d72/d33/ld1 0 2024-03-22T06:15:25.556 INFO:tasks.workunit.client.1.smithi117.stdout:6/601: creat d3/db4/fd2 x:0 0 0 2024-03-22T06:15:25.556 INFO:tasks.workunit.client.1.smithi117.stdout:9/599: symlink d4/d47/d85/lbd 0 2024-03-22T06:15:25.558 INFO:tasks.workunit.client.1.smithi117.stdout:7/537: symlink d8/d9/da/d2b/d9b/da5/la7 0 2024-03-22T06:15:25.558 INFO:tasks.workunit.client.1.smithi117.stdout:3/523: mknod d9/d1a/d39/d8a/cbf 0 2024-03-22T06:15:25.558 INFO:tasks.workunit.client.1.smithi117.stdout:3/524: chown d9/d1a/l29 1194524035 1 2024-03-22T06:15:25.563 INFO:tasks.workunit.client.1.smithi117.stdout:8/531: rmdir d1/d19/d9a 0 2024-03-22T06:15:25.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/574: rename d1/c2 to d1/d10/d5c/d40/cca 0 2024-03-22T06:15:25.581 INFO:tasks.workunit.client.1.smithi117.stdout:2/535: creat d2/d5/d13/d3f/d1d/fad x:0 0 0 2024-03-22T06:15:25.583 INFO:tasks.workunit.client.1.smithi117.stdout:4/534: creat d0/d19/d34/d43/d63/d6f/fbc x:0 0 0 2024-03-22T06:15:25.583 INFO:tasks.workunit.client.1.smithi117.stdout:4/535: write d0/d4/d2a/f9e [934098,48506] 0 2024-03-22T06:15:25.585 INFO:tasks.workunit.client.1.smithi117.stdout:5/604: mkdir dd/d16/d5a/dd2 0 2024-03-22T06:15:25.585 INFO:tasks.workunit.client.1.smithi117.stdout:6/602: symlink d3/db4/ld3 0 2024-03-22T06:15:25.587 INFO:tasks.workunit.client.1.smithi117.stdout:9/600: symlink d4/d8/d3e/d7a/d2f/lbe 0 2024-03-22T06:15:25.588 INFO:tasks.workunit.client.1.smithi117.stdout:2/536: dread d2/d5/db/f72 [0,4194304] 0 2024-03-22T06:15:25.590 INFO:tasks.workunit.client.1.smithi117.stdout:3/525: mknod d9/d1a/d39/d4c/d36/cc0 0 2024-03-22T06:15:25.591 INFO:tasks.workunit.client.1.smithi117.stdout:8/532: creat d1/d3a/fa6 x:0 0 0 2024-03-22T06:15:25.591 INFO:tasks.workunit.client.1.smithi117.stdout:8/533: write d1/d3a/d44/d11/d97/f99 [1549145,119446] 0 2024-03-22T06:15:25.591 INFO:tasks.workunit.client.1.smithi117.stdout:8/534: write d1/d28/f7d [894319,112082] 0 2024-03-22T06:15:25.592 INFO:tasks.workunit.client.1.smithi117.stdout:6/603: dread d3/db4/d2d/d53/d75/f8e [0,4194304] 0 2024-03-22T06:15:25.592 INFO:tasks.workunit.client.1.smithi117.stdout:6/604: dread - d3/d3d/f8d zero size 2024-03-22T06:15:25.592 INFO:tasks.workunit.client.1.smithi117.stdout:7/538: rename d8/d50/d5e/f8e to d8/d9/da/d2b/d9b/fa8 0 2024-03-22T06:15:25.595 INFO:tasks.workunit.client.1.smithi117.stdout:0/575: mkdir d1/d3/d2b/d3e/de/d1f/d59/dcb 0 2024-03-22T06:15:25.595 INFO:tasks.workunit.client.1.smithi117.stdout:0/576: creat d1/d10/d18/d26/db6/fcc x:0 0 0 2024-03-22T06:15:25.595 INFO:tasks.workunit.client.1.smithi117.stdout:0/577: write d1/d3/d2b/fc0 [1162873,53807] 0 2024-03-22T06:15:25.599 INFO:tasks.workunit.client.1.smithi117.stdout:4/536: creat d0/d19/d34/d43/d63/d6f/db1/db6/fbd x:0 0 0 2024-03-22T06:15:25.600 INFO:tasks.workunit.client.1.smithi117.stdout:3/526: dread d9/d1a/f90 [0,4194304] 0 2024-03-22T06:15:25.600 INFO:tasks.workunit.client.1.smithi117.stdout:9/601: mkdir d4/d7/dbf 0 2024-03-22T06:15:25.600 INFO:tasks.workunit.client.1.smithi117.stdout:9/602: dread - d4/d47/d85/fb8 zero size 2024-03-22T06:15:25.603 INFO:tasks.workunit.client.1.smithi117.stdout:7/539: dread d8/d9/da/d10/d23/d48/f49 [0,4194304] 0 2024-03-22T06:15:25.603 INFO:tasks.workunit.client.1.smithi117.stdout:7/540: chown d8/d9/da/d10/d23/d48/c60 493410891 1 2024-03-22T06:15:25.603 INFO:tasks.workunit.client.1.smithi117.stdout:2/537: unlink d2/d5/l75 0 2024-03-22T06:15:25.606 INFO:tasks.workunit.client.1.smithi117.stdout:8/535: mknod d1/d3a/d44/d11/d97/ca7 0 2024-03-22T06:15:25.609 INFO:tasks.workunit.client.1.smithi117.stdout:9/603: dread d4/d7/f1b [0,4194304] 0 2024-03-22T06:15:25.610 INFO:tasks.workunit.client.1.smithi117.stdout:5/605: rename dd/df/d5e/d91/d36/f26 to dd/df/d5e/d91/d72/fd3 0 2024-03-22T06:15:25.610 INFO:tasks.workunit.client.1.smithi117.stdout:2/538: dread d2/d5/f33 [0,4194304] 0 2024-03-22T06:15:25.613 INFO:tasks.workunit.client.1.smithi117.stdout:4/537: truncate d0/d19/d28/f2e 1637883 0 2024-03-22T06:15:25.615 INFO:tasks.workunit.client.1.smithi117.stdout:3/527: unlink d9/d55/l72 0 2024-03-22T06:15:25.618 INFO:tasks.workunit.client.1.smithi117.stdout:0/578: dwrite d1/d10/d18/d26/d49/d7e/f99 [0,4194304] 0 2024-03-22T06:15:25.619 INFO:tasks.workunit.client.1.smithi117.stdout:7/541: symlink d8/d9/da/d10/d23/d27/d39/d5c/la9 0 2024-03-22T06:15:25.620 INFO:tasks.workunit.client.1.smithi117.stdout:6/605: dwrite d3/db4/d2d/d53/d75/fa0 [0,4194304] 0 2024-03-22T06:15:25.627 INFO:tasks.workunit.client.1.smithi117.stdout:8/536: rename d1/d3a/d44/d11/f95 to d1/d28/fa8 0 2024-03-22T06:15:25.627 INFO:tasks.workunit.client.1.smithi117.stdout:8/537: creat d1/fa9 x:0 0 0 2024-03-22T06:15:25.631 INFO:tasks.workunit.client.1.smithi117.stdout:5/606: truncate dd/df/d5e/d91/d72/d35/f6f 2085113 0 2024-03-22T06:15:25.634 INFO:tasks.workunit.client.1.smithi117.stdout:2/539: getdents d2/d9a 0 2024-03-22T06:15:25.636 INFO:tasks.workunit.client.1.smithi117.stdout:7/542: dwrite d8/d9/da/d10/d23/d27/d32/fa0 [0,4194304] 0 2024-03-22T06:15:25.636 INFO:tasks.workunit.client.1.smithi117.stdout:7/543: fsync d8/fb 0 2024-03-22T06:15:25.636 INFO:tasks.workunit.client.1.smithi117.stdout:7/544: write d8/d9/da/f28 [3682951,3755] 0 2024-03-22T06:15:25.636 INFO:tasks.workunit.client.1.smithi117.stdout:7/545: truncate d8/d9/da/d10/d23/d27/d39/d26/f34 2172122 0 2024-03-22T06:15:25.637 INFO:tasks.workunit.client.1.smithi117.stdout:4/538: creat d0/d19/d34/d43/d63/d6f/db1/fbe x:0 0 0 2024-03-22T06:15:25.637 INFO:tasks.workunit.client.1.smithi117.stdout:4/539: write d0/f6d [1466106,169] 0 2024-03-22T06:15:25.638 INFO:tasks.workunit.client.1.smithi117.stdout:3/528: mkdir d9/d1a/dc1 0 2024-03-22T06:15:25.647 INFO:tasks.workunit.client.1.smithi117.stdout:6/606: mkdir d3/db4/dd4 0 2024-03-22T06:15:25.653 INFO:tasks.workunit.client.1.smithi117.stdout:9/604: rename d4/d47/d64/d72/f99 to d4/d8/d37/fc0 0 2024-03-22T06:15:25.653 INFO:tasks.workunit.client.1.smithi117.stdout:9/605: creat d4/d46/fc1 x:0 0 0 2024-03-22T06:15:25.655 INFO:tasks.workunit.client.1.smithi117.stdout:8/538: creat d1/d19/d38/faa x:0 0 0 2024-03-22T06:15:25.655 INFO:tasks.workunit.client.1.smithi117.stdout:1/600: sync 2024-03-22T06:15:25.655 INFO:tasks.workunit.client.1.smithi117.stdout:1/601: write d1/d1f/d32/dc6/fd0 [876662,114179] 0 2024-03-22T06:15:25.656 INFO:tasks.workunit.client.1.smithi117.stdout:2/540: stat d2/d5/d13/d3f/d1d/d3a/c5c 0 2024-03-22T06:15:25.656 INFO:tasks.workunit.client.1.smithi117.stdout:6/607: dread d3/db4/f5e [0,4194304] 0 2024-03-22T06:15:25.661 INFO:tasks.workunit.client.1.smithi117.stdout:7/546: creat d8/d9/d37/d45/d88/faa x:0 0 0 2024-03-22T06:15:25.664 INFO:tasks.workunit.client.1.smithi117.stdout:3/529: truncate d9/d1a/d3a/f53 1334981 0 2024-03-22T06:15:25.664 INFO:tasks.workunit.client.1.smithi117.stdout:3/530: chown d9/f26 57053794 1 2024-03-22T06:15:25.664 INFO:tasks.workunit.client.1.smithi117.stdout:9/606: write d4/d7/f24 [1331353,111569] 0 2024-03-22T06:15:25.664 INFO:tasks.workunit.client.1.smithi117.stdout:9/607: chown d4/d7/d27/d5b 5 1 2024-03-22T06:15:25.666 INFO:tasks.workunit.client.1.smithi117.stdout:5/607: rename dd/f6d to dd/df/d5e/d91/d90/dca/fd4 0 2024-03-22T06:15:25.668 INFO:tasks.workunit.client.1.smithi117.stdout:8/539: symlink d1/d3a/d44/d4b/lab 0 2024-03-22T06:15:25.669 INFO:tasks.workunit.client.1.smithi117.stdout:1/602: getdents d1/d30/d44/d9c 0 2024-03-22T06:15:25.669 INFO:tasks.workunit.client.1.smithi117.stdout:1/603: fsync d1/d1f/f3e 0 2024-03-22T06:15:25.670 INFO:tasks.workunit.client.1.smithi117.stdout:6/608: mknod d3/d40/d46/d84/d49/da8/cd5 0 2024-03-22T06:15:25.670 INFO:tasks.workunit.client.1.smithi117.stdout:1/604: dread d1/d84/d77/f98 [0,4194304] 0 2024-03-22T06:15:25.671 INFO:tasks.workunit.client.1.smithi117.stdout:7/547: truncate d8/d9/d37/d5f/f79 40890 0 2024-03-22T06:15:25.673 INFO:tasks.workunit.client.1.smithi117.stdout:3/531: unlink d9/l23 0 2024-03-22T06:15:25.673 INFO:tasks.workunit.client.1.smithi117.stdout:3/532: chown d9/d1a/f4b 3 1 2024-03-22T06:15:25.675 INFO:tasks.workunit.client.1.smithi117.stdout:9/608: mknod d4/d46/da0/cc2 0 2024-03-22T06:15:25.677 INFO:tasks.workunit.client.1.smithi117.stdout:8/540: dread d1/d19/f64 [0,4194304] 0 2024-03-22T06:15:25.677 INFO:tasks.workunit.client.1.smithi117.stdout:8/541: fdatasync d1/d19/f51 0 2024-03-22T06:15:25.677 INFO:tasks.workunit.client.1.smithi117.stdout:8/542: chown d1/d19/c4a 45745553 1 2024-03-22T06:15:25.677 INFO:tasks.workunit.client.1.smithi117.stdout:8/543: creat d1/fac x:0 0 0 2024-03-22T06:15:25.677 INFO:tasks.workunit.client.1.smithi117.stdout:8/544: creat d1/d28/d6b/fad x:0 0 0 2024-03-22T06:15:25.677 INFO:tasks.workunit.client.1.smithi117.stdout:5/608: getdents dd/df/d5e/d91/d36/d98 0 2024-03-22T06:15:25.677 INFO:tasks.workunit.client.1.smithi117.stdout:5/609: truncate dd/df/d5e/d91/d72/d60/f6b 4293482 0 2024-03-22T06:15:25.678 INFO:tasks.workunit.client.1.smithi117.stdout:6/609: creat d3/db4/d2d/d53/fd6 x:0 0 0 2024-03-22T06:15:25.678 INFO:tasks.workunit.client.1.smithi117.stdout:2/541: dwrite d2/d5/da5/fa7 [0,4194304] 0 2024-03-22T06:15:25.680 INFO:tasks.workunit.client.1.smithi117.stdout:1/605: link d1/d30/d44/c5c d1/d1f/cd4 0 2024-03-22T06:15:25.680 INFO:tasks.workunit.client.1.smithi117.stdout:1/606: stat d1 0 2024-03-22T06:15:25.680 INFO:tasks.workunit.client.1.smithi117.stdout:1/607: write d1/d1f/f75 [2025127,121528] 0 2024-03-22T06:15:25.681 INFO:tasks.workunit.client.1.smithi117.stdout:7/548: rename d8/d9/da/d10/d23/d48/f49 to d8/d9/da/d10/d23/d41/fab 0 2024-03-22T06:15:25.682 INFO:tasks.workunit.client.1.smithi117.stdout:9/609: mkdir d4/d8/d3f/dc3 0 2024-03-22T06:15:25.682 INFO:tasks.workunit.client.1.smithi117.stdout:9/610: truncate d4/d46/fa6 229476 0 2024-03-22T06:15:25.685 INFO:tasks.workunit.client.1.smithi117.stdout:8/545: mkdir d1/d3a/d44/d4b/dae 0 2024-03-22T06:15:25.691 INFO:tasks.workunit.client.1.smithi117.stdout:6/610: dread d3/db4/d2d/d42/da2/f7a [0,4194304] 0 2024-03-22T06:15:25.695 INFO:tasks.workunit.client.1.smithi117.stdout:1/608: link d1/d62/ca7 d1/d30/d50/cd5 0 2024-03-22T06:15:25.697 INFO:tasks.workunit.client.1.smithi117.stdout:5/610: rename dd/d16/d1a/d32/d73/db8/fce to dd/df/d5e/d91/d90/fd5 0 2024-03-22T06:15:25.697 INFO:tasks.workunit.client.1.smithi117.stdout:4/540: getdents d0/d19/d34/d43/d63/d6f/db1 0 2024-03-22T06:15:25.700 INFO:tasks.workunit.client.1.smithi117.stdout:7/549: unlink d8/d9/da/d10/f19 0 2024-03-22T06:15:25.700 INFO:tasks.workunit.client.1.smithi117.stdout:7/550: fsync d8/d9/d37/d5f/d82/f95 0 2024-03-22T06:15:25.702 INFO:tasks.workunit.client.1.smithi117.stdout:9/611: mkdir d4/db9/dc4 0 2024-03-22T06:15:25.702 INFO:tasks.workunit.client.1.smithi117.stdout:9/612: write d4/d7/d27/f56 [4664791,99915] 0 2024-03-22T06:15:25.703 INFO:tasks.workunit.client.1.smithi117.stdout:8/546: symlink d1/d28/d66/d7f/laf 0 2024-03-22T06:15:25.705 INFO:tasks.workunit.client.1.smithi117.stdout:2/542: symlink d2/d5/d13/d57/lae 0 2024-03-22T06:15:25.705 INFO:tasks.workunit.client.1.smithi117.stdout:2/543: chown d2/d5/d13/d3f/d1d/f6c 124271 1 2024-03-22T06:15:25.705 INFO:tasks.workunit.client.1.smithi117.stdout:2/544: fdatasync d2/d5/d13/d3f/d1d/d3a/f64 0 2024-03-22T06:15:25.705 INFO:tasks.workunit.client.1.smithi117.stdout:4/541: dread d0/d51/f7e [0,4194304] 0 2024-03-22T06:15:25.705 INFO:tasks.workunit.client.1.smithi117.stdout:4/542: chown d0/d19/d28/c41 262592295 1 2024-03-22T06:15:25.708 INFO:tasks.workunit.client.1.smithi117.stdout:6/611: dwrite d3/d40/d46/d84/d49/d79/fcf [0,4194304] 0 2024-03-22T06:15:25.708 INFO:tasks.workunit.client.1.smithi117.stdout:6/612: fsync d3/db4/d2d/d53/d75/fb0 0 2024-03-22T06:15:25.711 INFO:tasks.workunit.client.1.smithi117.stdout:6/613: dread d3/d3d/d17/d31/f78 [0,4194304] 0 2024-03-22T06:15:25.712 INFO:tasks.workunit.client.1.smithi117.stdout:1/609: truncate d1/d30/d50/f63 1400047 0 2024-03-22T06:15:25.716 INFO:tasks.workunit.client.1.smithi117.stdout:7/551: mkdir d8/d9/da/d10/d23/d27/d39/dac 0 2024-03-22T06:15:25.717 INFO:tasks.workunit.client.1.smithi117.stdout:9/613: truncate d4/d7/d27/f2e 665115 0 2024-03-22T06:15:25.721 INFO:tasks.workunit.client.1.smithi117.stdout:8/547: rename d1/d28/c33 to d1/d28/d66/cb0 0 2024-03-22T06:15:25.723 INFO:tasks.workunit.client.1.smithi117.stdout:2/545: creat d2/d5/d13/d3f/d1d/d3a/d71/d76/faf x:0 0 0 2024-03-22T06:15:25.723 INFO:tasks.workunit.client.1.smithi117.stdout:2/546: creat d2/d5/d13/d3f/d1d/d3a/d71/d76/fb0 x:0 0 0 2024-03-22T06:15:25.725 INFO:tasks.workunit.client.1.smithi117.stdout:9/614: dread d4/d7/f24 [0,4194304] 0 2024-03-22T06:15:25.725 INFO:tasks.workunit.client.1.smithi117.stdout:9/615: fdatasync d4/d47/fbb 0 2024-03-22T06:15:25.725 INFO:tasks.workunit.client.1.smithi117.stdout:4/543: symlink d0/d19/d34/d43/d63/d6f/db1/lbf 0 2024-03-22T06:15:25.730 INFO:tasks.workunit.client.1.smithi117.stdout:2/547: dread d2/d5/d13/d3f/d1d/f6c [0,4194304] 0 2024-03-22T06:15:25.735 INFO:tasks.workunit.client.1.smithi117.stdout:1/610: creat d1/d3c/d4d/d5b/fd6 x:0 0 0 2024-03-22T06:15:25.735 INFO:tasks.workunit.client.1.smithi117.stdout:1/611: fsync d1/d22/d27/f34 0 2024-03-22T06:15:25.735 INFO:tasks.workunit.client.1.smithi117.stdout:1/612: chown d1/d3c 682888 1 2024-03-22T06:15:25.741 INFO:tasks.workunit.client.1.smithi117.stdout:8/548: dwrite d1/d19/f64 [4194304,4194304] 0 2024-03-22T06:15:25.741 INFO:tasks.workunit.client.1.smithi117.stdout:8/549: chown d1/d28/d6b/fad 62967465 1 2024-03-22T06:15:25.743 INFO:tasks.workunit.client.1.smithi117.stdout:7/552: rmdir d8/d9/da/d10/da1 0 2024-03-22T06:15:25.744 INFO:tasks.workunit.client.1.smithi117.stdout:2/548: dread d2/f4 [0,4194304] 0 2024-03-22T06:15:25.744 INFO:tasks.workunit.client.1.smithi117.stdout:2/549: fsync d2/d5/d13/d3f/f6b 0 2024-03-22T06:15:25.744 INFO:tasks.workunit.client.1.smithi117.stdout:2/550: read - d2/d5/d41/f5b zero size 2024-03-22T06:15:25.744 INFO:tasks.workunit.client.1.smithi117.stdout:2/551: write d2/d5/d13/d3f/d1d/d3a/d89/daa/f15 [1712106,86094] 0 2024-03-22T06:15:25.746 INFO:tasks.workunit.client.1.smithi117.stdout:9/616: mkdir d4/d47/d64/dc5 0 2024-03-22T06:15:25.750 INFO:tasks.workunit.client.1.smithi117.stdout:4/544: unlink d0/d48/f3d 0 2024-03-22T06:15:25.752 INFO:tasks.workunit.client.1.smithi117.stdout:6/614: mknod d3/cd7 0 2024-03-22T06:15:25.754 INFO:tasks.workunit.client.1.smithi117.stdout:1/613: mkdir d1/d30/d44/d99/dd7 0 2024-03-22T06:15:25.754 INFO:tasks.workunit.client.1.smithi117.stdout:1/614: chown d1/d1f/f75 62 1 2024-03-22T06:15:25.757 INFO:tasks.workunit.client.1.smithi117.stdout:7/553: symlink d8/d9/da/d10/d23/d48/d91/lad 0 2024-03-22T06:15:25.757 INFO:tasks.workunit.client.1.smithi117.stdout:0/579: sync 2024-03-22T06:15:25.758 INFO:tasks.workunit.client.1.smithi117.stdout:9/617: dread d4/d7/d27/f60 [0,4194304] 0 2024-03-22T06:15:25.763 INFO:tasks.workunit.client.1.smithi117.stdout:8/550: dwrite d1/d28/d6b/fad [0,4194304] 0 2024-03-22T06:15:25.763 INFO:tasks.workunit.client.1.smithi117.stdout:8/551: write d1/d3a/d44/d9/f8f [368812,122958] 0 2024-03-22T06:15:25.763 INFO:tasks.workunit.client.1.smithi117.stdout:8/552: write d1/d19/f86 [251591,110864] 0 2024-03-22T06:15:25.764 INFO:tasks.workunit.client.1.smithi117.stdout:4/545: mkdir d0/dc0 0 2024-03-22T06:15:25.766 INFO:tasks.workunit.client.1.smithi117.stdout:1/615: creat d1/d1f/d32/d33/da4/fd8 x:0 0 0 2024-03-22T06:15:25.766 INFO:tasks.workunit.client.1.smithi117.stdout:1/616: getdents d1/d30/d44/d99/dd7 0 2024-03-22T06:15:25.769 INFO:tasks.workunit.client.1.smithi117.stdout:7/554: unlink d8/d9/da/d10/f22 0 2024-03-22T06:15:25.771 INFO:tasks.workunit.client.1.smithi117.stdout:0/580: unlink d1/d3/d2b/d3e/d3f/faa 0 2024-03-22T06:15:25.775 INFO:tasks.workunit.client.1.smithi117.stdout:9/618: rename d4/d47/fbb to d4/d47/d64/d72/fc6 0 2024-03-22T06:15:25.777 INFO:tasks.workunit.client.1.smithi117.stdout:6/615: dwrite d3/db4/d2d/d53/d59/d74/da/f29 [0,4194304] 0 2024-03-22T06:15:25.777 INFO:tasks.workunit.client.1.smithi117.stdout:6/616: write d3/db4/d2d/d53/d59/fb7 [602130,62906] 0 2024-03-22T06:15:25.782 INFO:tasks.workunit.client.1.smithi117.stdout:8/553: unlink d1/d3a/d44/d11/f2a 0 2024-03-22T06:15:25.784 INFO:tasks.workunit.client.1.smithi117.stdout:4/546: getdents d0/d4/d2a/d2f 0 2024-03-22T06:15:25.786 INFO:tasks.workunit.client.1.smithi117.stdout:1/617: creat d1/d1f/d32/fd9 x:0 0 0 2024-03-22T06:15:25.788 INFO:tasks.workunit.client.1.smithi117.stdout:7/555: creat d8/d6c/fae x:0 0 0 2024-03-22T06:15:25.788 INFO:tasks.workunit.client.1.smithi117.stdout:7/556: dread - d8/d9/da/d10/f8d zero size 2024-03-22T06:15:25.790 INFO:tasks.workunit.client.1.smithi117.stdout:0/581: creat d1/d10/d5c/d40/dbb/fcd x:0 0 0 2024-03-22T06:15:25.790 INFO:tasks.workunit.client.1.smithi117.stdout:0/582: chown d1/f19 7655 1 2024-03-22T06:15:25.791 INFO:tasks.workunit.client.1.smithi117.stdout:9/619: rename d4/d50/f6b to d4/d8/d3e/d7a/da9/fc7 0 2024-03-22T06:15:25.792 INFO:tasks.workunit.client.1.smithi117.stdout:9/620: write d4/d47/d64/fb4 [759033,22134] 0 2024-03-22T06:15:25.792 INFO:tasks.workunit.client.1.smithi117.stdout:9/621: write d4/d7/f24 [4055855,112012] 0 2024-03-22T06:15:25.798 INFO:tasks.workunit.client.1.smithi117.stdout:6/617: symlink d3/d3d/d17/ld8 0 2024-03-22T06:15:25.800 INFO:tasks.workunit.client.1.smithi117.stdout:8/554: mknod d1/d3a/d44/d6a/cb1 0 2024-03-22T06:15:25.800 INFO:tasks.workunit.client.1.smithi117.stdout:0/583: dread d1/d10/d18/d26/d49/d7e/f47 [0,4194304] 0 2024-03-22T06:15:25.800 INFO:tasks.workunit.client.1.smithi117.stdout:0/584: chown d1/d10/d18/d26/d49/d7e/d39/f52 512248137 1 2024-03-22T06:15:25.800 INFO:tasks.workunit.client.1.smithi117.stdout:0/585: write d1/d10/d18/f7b [896156,107365] 0 2024-03-22T06:15:25.801 INFO:tasks.workunit.client.1.smithi117.stdout:0/586: chown d1/d10/d18/c35 347 1 2024-03-22T06:15:25.801 INFO:tasks.workunit.client.1.smithi117.stdout:0/587: getdents d1/d3/dae/dbe 0 2024-03-22T06:15:25.801 INFO:tasks.workunit.client.1.smithi117.stdout:7/557: unlink d8/l52 0 2024-03-22T06:15:25.801 INFO:tasks.workunit.client.1.smithi117.stdout:7/558: dread - d8/d9/da/d10/d23/d41/f93 zero size 2024-03-22T06:15:25.805 INFO:tasks.workunit.client.1.smithi117.stdout:8/555: mknod d1/d19/d98/cb2 0 2024-03-22T06:15:25.809 INFO:tasks.workunit.client.1.smithi117.stdout:4/547: dwrite d0/d4/d2a/d5b/f6e [0,4194304] 0 2024-03-22T06:15:25.809 INFO:tasks.workunit.client.1.smithi117.stdout:0/588: link d1/d10/d18/c5a d1/d10/d5c/d40/d4e/cce 0 2024-03-22T06:15:25.809 INFO:tasks.workunit.client.1.smithi117.stdout:0/589: chown d1/d10/d5c/d40/d4e/dbc 46 1 2024-03-22T06:15:25.811 INFO:tasks.workunit.client.1.smithi117.stdout:7/559: symlink d8/d6c/laf 0 2024-03-22T06:15:25.811 INFO:tasks.workunit.client.1.smithi117.stdout:7/560: readlink d8/d9/da/d10/d23/d27/d39/d5c/l8a 0 2024-03-22T06:15:25.811 INFO:tasks.workunit.client.1.smithi117.stdout:7/561: write d8/d9/da/f28 [1782389,67715] 0 2024-03-22T06:15:25.812 INFO:tasks.workunit.client.1.smithi117.stdout:6/618: dread d3/f4 [0,4194304] 0 2024-03-22T06:15:25.812 INFO:tasks.workunit.client.1.smithi117.stdout:6/619: read - d3/db4/d2d/d42/fca zero size 2024-03-22T06:15:25.812 INFO:tasks.workunit.client.1.smithi117.stdout:6/620: dread - d3/d40/d46/d84/d49/d79/d9d/fce zero size 2024-03-22T06:15:25.814 INFO:tasks.workunit.client.1.smithi117.stdout:1/618: dwrite d1/d22/d27/d4e/f89 [0,4194304] 0 2024-03-22T06:15:25.818 INFO:tasks.workunit.client.1.smithi117.stdout:6/621: mknod d3/d3d/cd9 0 2024-03-22T06:15:25.827 INFO:tasks.workunit.client.1.smithi117.stdout:0/590: read d1/f3d [1491418,86964] 0 2024-03-22T06:15:25.829 INFO:tasks.workunit.client.1.smithi117.stdout:8/556: dwrite d1/fa9 [0,4194304] 0 2024-03-22T06:15:25.831 INFO:tasks.workunit.client.1.smithi117.stdout:6/622: rmdir d3/db4/d2d/d53/dc7 0 2024-03-22T06:15:25.835 INFO:tasks.workunit.client.1.smithi117.stdout:0/591: link d1/d10/d18/d26/f75 d1/d10/d5c/d40/fcf 0 2024-03-22T06:15:25.835 INFO:tasks.workunit.client.1.smithi117.stdout:0/592: read - d1/d3/d2b/d3e/de/d1f/d3b/d55/f91 zero size 2024-03-22T06:15:25.835 INFO:tasks.workunit.client.1.smithi117.stdout:0/593: creat d1/d3/d2b/d3e/dad/fd0 x:0 0 0 2024-03-22T06:15:25.844 INFO:tasks.workunit.client.1.smithi117.stdout:9/622: rename d4/d47/d64/d72/fc6 to d4/d47/d64/d72/d8d/fc8 0 2024-03-22T06:15:25.844 INFO:tasks.workunit.client.1.smithi117.stdout:9/623: creat d4/d8/d3e/d7a/d2f/fc9 x:0 0 0 2024-03-22T06:15:25.846 INFO:tasks.workunit.client.1.smithi117.stdout:6/623: write d3/db4/d2d/d53/d59/d74/da/db3/f6d [488800,18896] 0 2024-03-22T06:15:25.849 INFO:tasks.workunit.client.1.smithi117.stdout:9/624: dread d4/d8/f2c [0,4194304] 0 2024-03-22T06:15:25.852 INFO:tasks.workunit.client.1.smithi117.stdout:7/562: dwrite d8/d9/da/d10/d23/d27/d32/f44 [0,4194304] 0 2024-03-22T06:15:25.854 INFO:tasks.workunit.client.1.smithi117.stdout:4/548: dwrite d0/d3f/f4b [0,4194304] 0 2024-03-22T06:15:25.854 INFO:tasks.workunit.client.1.smithi117.stdout:4/549: truncate d0/d51/f7e 1623621 0 2024-03-22T06:15:25.854 INFO:tasks.workunit.client.1.smithi117.stdout:8/557: dwrite d1/d19/d38/f9f [0,4194304] 0 2024-03-22T06:15:25.854 INFO:tasks.workunit.client.1.smithi117.stdout:8/558: write d1/d28/d66/d7f/f9e [516181,91871] 0 2024-03-22T06:15:25.854 INFO:tasks.workunit.client.1.smithi117.stdout:1/619: dwrite d1/fd [4194304,4194304] 0 2024-03-22T06:15:25.855 INFO:tasks.workunit.client.1.smithi117.stdout:8/559: read d1/d3a/d44/f3c [861300,30966] 0 2024-03-22T06:15:25.855 INFO:tasks.workunit.client.1.smithi117.stdout:8/560: chown d1/d3a/d44/d4b/dae 218067 1 2024-03-22T06:15:25.859 INFO:tasks.workunit.client.1.smithi117.stdout:6/624: unlink d3/db4/d2d/d53/d59/d74/da/l1e 0 2024-03-22T06:15:25.859 INFO:tasks.workunit.client.1.smithi117.stdout:3/533: sync 2024-03-22T06:15:25.863 INFO:tasks.workunit.client.1.smithi117.stdout:4/550: dread d0/f3 [0,4194304] 0 2024-03-22T06:15:25.863 INFO:tasks.workunit.client.1.smithi117.stdout:4/551: write d0/d19/f98 [774130,26273] 0 2024-03-22T06:15:25.863 INFO:tasks.workunit.client.1.smithi117.stdout:7/563: mkdir d8/d9/db0 0 2024-03-22T06:15:25.866 INFO:tasks.workunit.client.1.smithi117.stdout:9/625: dwrite d4/d7/f1b [0,4194304] 0 2024-03-22T06:15:25.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/620: rename d1/d30/d44/d99/lb3 to d1/d30/d44/lda 0 2024-03-22T06:15:25.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/621: creat d1/d1f/d32/fdb x:0 0 0 2024-03-22T06:15:25.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/622: chown d1/d30/d5d 3 1 2024-03-22T06:15:25.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/623: write d1/d22/d27/d4e/f7d [775058,65115] 0 2024-03-22T06:15:25.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/624: write d1/d30/d35/d58/fb0 [507614,112529] 0 2024-03-22T06:15:25.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/625: fsync d1/d22/f4b 0 2024-03-22T06:15:25.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/626: fdatasync d1/fd 0 2024-03-22T06:15:25.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/627: chown d1/d1f/db2/lc7 6 1 2024-03-22T06:15:25.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/628: write d1/fd [2299601,87866] 0 2024-03-22T06:15:25.873 INFO:tasks.workunit.client.1.smithi117.stdout:8/561: creat d1/d3a/d44/d4b/fb3 x:0 0 0 2024-03-22T06:15:25.874 INFO:tasks.workunit.client.1.smithi117.stdout:8/562: chown d1/d3a/d44/d9/f6c 9473 1 2024-03-22T06:15:25.874 INFO:tasks.workunit.client.1.smithi117.stdout:6/625: truncate d3/db4/d2d/d53/d59/d74/da/de/fa3 369515 0 2024-03-22T06:15:25.877 INFO:tasks.workunit.client.1.smithi117.stdout:4/552: creat d0/dc0/fc1 x:0 0 0 2024-03-22T06:15:25.877 INFO:tasks.workunit.client.1.smithi117.stdout:4/553: fdatasync d0/f2d 0 2024-03-22T06:15:25.878 INFO:tasks.workunit.client.1.smithi117.stdout:3/534: dread d9/d1a/d39/d7d/f9b [0,4194304] 0 2024-03-22T06:15:25.878 INFO:tasks.workunit.client.1.smithi117.stdout:7/564: mkdir d8/d9/d37/d45/d88/db1 0 2024-03-22T06:15:25.881 INFO:tasks.workunit.client.1.smithi117.stdout:9/626: truncate d4/d7/d27/f97 3585459 0 2024-03-22T06:15:25.881 INFO:tasks.workunit.client.1.smithi117.stdout:9/627: readlink d4/d7/d27/l38 0 2024-03-22T06:15:25.882 INFO:tasks.workunit.client.1.smithi117.stdout:9/628: creat d4/d8/d37/fca x:0 0 0 2024-03-22T06:15:25.884 INFO:tasks.workunit.client.1.smithi117.stdout:1/629: symlink d1/d30/d44/d99/ldc 0 2024-03-22T06:15:25.885 INFO:tasks.workunit.client.1.smithi117.stdout:6/626: symlink d3/d40/d46/d84/lda 0 2024-03-22T06:15:25.887 INFO:tasks.workunit.client.1.smithi117.stdout:3/535: mknod d9/d93/cc2 0 2024-03-22T06:15:25.894 INFO:tasks.workunit.client.1.smithi117.stdout:7/565: write d8/d9/da/d10/d23/d27/d32/f4d [1272984,5997] 0 2024-03-22T06:15:25.894 INFO:tasks.workunit.client.1.smithi117.stdout:7/566: creat d8/d50/d5e/fb2 x:0 0 0 2024-03-22T06:15:25.894 INFO:tasks.workunit.client.1.smithi117.stdout:7/567: stat d8/d9/da/f72 0 2024-03-22T06:15:25.894 INFO:tasks.workunit.client.1.smithi117.stdout:5/611: sync 2024-03-22T06:15:25.894 INFO:tasks.workunit.client.1.smithi117.stdout:9/629: unlink d4/d8/d3f/c6e 0 2024-03-22T06:15:25.894 INFO:tasks.workunit.client.1.smithi117.stdout:8/563: dwrite d1/d3a/d44/d9/f13 [0,4194304] 0 2024-03-22T06:15:25.894 INFO:tasks.workunit.client.1.smithi117.stdout:3/536: dread d9/d66/f6d [0,4194304] 0 2024-03-22T06:15:25.904 INFO:tasks.workunit.client.1.smithi117.stdout:3/537: dread d9/d1a/f83 [0,4194304] 0 2024-03-22T06:15:25.904 INFO:tasks.workunit.client.1.smithi117.stdout:3/538: chown d9/d93/cc2 758 1 2024-03-22T06:15:25.904 INFO:tasks.workunit.client.1.smithi117.stdout:3/539: creat d9/d1a/d39/d4c/d37/fc3 x:0 0 0 2024-03-22T06:15:25.906 INFO:tasks.workunit.client.1.smithi117.stdout:1/630: dwrite d1/d3c/f9a [0,4194304] 0 2024-03-22T06:15:25.908 INFO:tasks.workunit.client.1.smithi117.stdout:9/630: mknod d4/d46/da0/ccb 0 2024-03-22T06:15:25.911 INFO:tasks.workunit.client.1.smithi117.stdout:8/564: creat d1/d28/d66/d7f/d9d/fb4 x:0 0 0 2024-03-22T06:15:25.914 INFO:tasks.workunit.client.1.smithi117.stdout:2/552: sync 2024-03-22T06:15:25.915 INFO:tasks.workunit.client.1.smithi117.stdout:2/553: chown d2/d35/d38/c40 1204 1 2024-03-22T06:15:25.915 INFO:tasks.workunit.client.1.smithi117.stdout:3/540: link d9/l2f d9/d1a/d39/d4c/d37/d69/lc4 0 2024-03-22T06:15:25.919 INFO:tasks.workunit.client.1.smithi117.stdout:1/631: rmdir d1/d14/dcb 0 2024-03-22T06:15:25.919 INFO:tasks.workunit.client.1.smithi117.stdout:5/612: dwrite dd/d16/d1a/f37 [0,4194304] 0 2024-03-22T06:15:25.919 INFO:tasks.workunit.client.1.smithi117.stdout:7/568: dwrite d8/d9/da/d10/d23/d27/d56/fa4 [0,4194304] 0 2024-03-22T06:15:25.920 INFO:tasks.workunit.client.1.smithi117.stdout:9/631: creat d4/d8/d3f/dc3/fcc x:0 0 0 2024-03-22T06:15:25.920 INFO:tasks.workunit.client.1.smithi117.stdout:9/632: creat d4/d47/d64/fcd x:0 0 0 2024-03-22T06:15:25.920 INFO:tasks.workunit.client.1.smithi117.stdout:9/633: chown d4/d7/d27/d5b/f6a 0 1 2024-03-22T06:15:25.922 INFO:tasks.workunit.client.1.smithi117.stdout:8/565: creat d1/d3a/d44/d16/fb5 x:0 0 0 2024-03-22T06:15:25.922 INFO:tasks.workunit.client.1.smithi117.stdout:8/566: read d1/d3a/d44/d11/d97/f99 [1082833,49551] 0 2024-03-22T06:15:25.922 INFO:tasks.workunit.client.1.smithi117.stdout:8/567: dread - d1/fac zero size 2024-03-22T06:15:25.922 INFO:tasks.workunit.client.1.smithi117.stdout:2/554: symlink d2/d1b/d39/lb1 0 2024-03-22T06:15:25.925 INFO:tasks.workunit.client.1.smithi117.stdout:1/632: symlink d1/d62/d7c/ldd 0 2024-03-22T06:15:25.926 INFO:tasks.workunit.client.1.smithi117.stdout:5/613: creat dd/df/d5e/d91/d90/dca/fd6 x:0 0 0 2024-03-22T06:15:25.926 INFO:tasks.workunit.client.1.smithi117.stdout:5/614: chown dd/fe 1509 1 2024-03-22T06:15:25.927 INFO:tasks.workunit.client.1.smithi117.stdout:5/615: write dd/d16/d1a/d32/d73/db8/fb9 [882800,130111] 0 2024-03-22T06:15:25.929 INFO:tasks.workunit.client.1.smithi117.stdout:7/569: mknod d8/d9/da/d2b/cb3 0 2024-03-22T06:15:25.932 INFO:tasks.workunit.client.1.smithi117.stdout:8/568: truncate d1/d3a/d44/f1d 1878965 0 2024-03-22T06:15:25.934 INFO:tasks.workunit.client.1.smithi117.stdout:2/555: unlink d2/d5/d41/c59 0 2024-03-22T06:15:25.939 INFO:tasks.workunit.client.1.smithi117.stdout:8/569: dread d1/d28/d66/f74 [0,4194304] 0 2024-03-22T06:15:25.939 INFO:tasks.workunit.client.1.smithi117.stdout:8/570: chown d1/l41 14549 1 2024-03-22T06:15:25.940 INFO:tasks.workunit.client.1.smithi117.stdout:3/541: rmdir d9/d1a/d39/d4c/d37 39 2024-03-22T06:15:25.943 INFO:tasks.workunit.client.1.smithi117.stdout:1/633: link d1/d1f/d32/c69 d1/d30/d44/d99/dd7/cde 0 2024-03-22T06:15:25.943 INFO:tasks.workunit.client.1.smithi117.stdout:1/634: readlink d1/d22/l57 0 2024-03-22T06:15:25.945 INFO:tasks.workunit.client.1.smithi117.stdout:5/616: link dd/df/d5e/d91/f5d dd/d16/d1a/fd7 0 2024-03-22T06:15:25.945 INFO:tasks.workunit.client.1.smithi117.stdout:5/617: creat dd/df/d5e/d91/d72/d35/d81/fd8 x:0 0 0 2024-03-22T06:15:25.945 INFO:tasks.workunit.client.1.smithi117.stdout:5/618: write dd/df/d5e/d91/d72/f45 [4700152,98089] 0 2024-03-22T06:15:25.945 INFO:tasks.workunit.client.1.smithi117.stdout:5/619: fsync dd/d16/d1a/d32/d73/db8/fc5 0 2024-03-22T06:15:25.946 INFO:tasks.workunit.client.1.smithi117.stdout:7/570: write d8/d9/da/d10/d23/d27/d39/f3b [383370,14397] 0 2024-03-22T06:15:25.949 INFO:tasks.workunit.client.1.smithi117.stdout:2/556: creat d2/d5/db/d2c/d7d/fb2 x:0 0 0 2024-03-22T06:15:25.950 INFO:tasks.workunit.client.1.smithi117.stdout:8/571: symlink d1/d3a/d44/d4b/lb6 0 2024-03-22T06:15:25.950 INFO:tasks.workunit.client.1.smithi117.stdout:8/572: fdatasync d1/d19/d38/f9f 0 2024-03-22T06:15:25.951 INFO:tasks.workunit.client.1.smithi117.stdout:1/635: creat d1/d1f/d32/d33/fdf x:0 0 0 2024-03-22T06:15:25.953 INFO:tasks.workunit.client.1.smithi117.stdout:1/636: dread d1/d30/fa8 [0,4194304] 0 2024-03-22T06:15:25.955 INFO:tasks.workunit.client.1.smithi117.stdout:5/620: link dd/df/d5e/d91/d72/c7f dd/d16/d1a/d32/dbf/cd9 0 2024-03-22T06:15:25.955 INFO:tasks.workunit.client.1.smithi117.stdout:5/621: truncate dd/d16/d1a/d32/d73/db8/fc5 958315 0 2024-03-22T06:15:25.955 INFO:tasks.workunit.client.1.smithi117.stdout:5/622: fsync dd/d16/d5a/f97 0 2024-03-22T06:15:25.955 INFO:tasks.workunit.client.1.smithi117.stdout:5/623: truncate dd/df/d5e/d91/d72/d35/da2/fcf 558519 0 2024-03-22T06:15:25.958 INFO:tasks.workunit.client.1.smithi117.stdout:2/557: mknod d2/d5/d13/d3f/d1d/d3a/d89/d8d/cb3 0 2024-03-22T06:15:25.958 INFO:tasks.workunit.client.1.smithi117.stdout:2/558: dread - d2/d5/d13/d3f/f97 zero size 2024-03-22T06:15:25.958 INFO:tasks.workunit.client.1.smithi117.stdout:2/559: creat d2/d5/d41/fb4 x:0 0 0 2024-03-22T06:15:25.959 INFO:tasks.workunit.client.1.smithi117.stdout:8/573: mkdir d1/d3a/d44/d6a/db7 0 2024-03-22T06:15:25.961 INFO:tasks.workunit.client.1.smithi117.stdout:0/594: sync 2024-03-22T06:15:25.962 INFO:tasks.workunit.client.1.smithi117.stdout:5/624: mkdir dd/d16/d1a/dda 0 2024-03-22T06:15:25.964 INFO:tasks.workunit.client.1.smithi117.stdout:2/560: mknod d2/d5/d13/d3f/d1d/d3a/d89/daa/d4f/cb5 0 2024-03-22T06:15:25.965 INFO:tasks.workunit.client.1.smithi117.stdout:8/574: creat d1/d3a/d44/d4b/dae/fb8 x:0 0 0 2024-03-22T06:15:25.968 INFO:tasks.workunit.client.1.smithi117.stdout:3/542: dwrite d9/f18 [4194304,4194304] 0 2024-03-22T06:15:25.968 INFO:tasks.workunit.client.1.smithi117.stdout:7/571: dwrite d8/d9/d37/d45/d61/f71 [0,4194304] 0 2024-03-22T06:15:25.973 INFO:tasks.workunit.client.1.smithi117.stdout:1/637: rmdir d1/d22/d27 39 2024-03-22T06:15:25.973 INFO:tasks.workunit.client.1.smithi117.stdout:1/638: write d1/d1f/d32/fd9 [245996,112610] 0 2024-03-22T06:15:25.973 INFO:tasks.workunit.client.1.smithi117.stdout:1/639: creat d1/d3c/d4d/d5b/fe0 x:0 0 0 2024-03-22T06:15:25.975 INFO:tasks.workunit.client.1.smithi117.stdout:0/595: creat d1/d10/d5c/d40/dbb/dbf/fd1 x:0 0 0 2024-03-22T06:15:25.979 INFO:tasks.workunit.client.1.smithi117.stdout:7/572: mknod d8/d9/da/d10/d23/d27/d32/cb4 0 2024-03-22T06:15:25.988 INFO:tasks.workunit.client.1.smithi117.stdout:5/625: dwrite dd/df/f2b [8388608,4194304] 0 2024-03-22T06:15:25.988 INFO:tasks.workunit.client.1.smithi117.stdout:3/543: dwrite d9/d1a/d39/d4c/d36/f62 [0,4194304] 0 2024-03-22T06:15:25.989 INFO:tasks.workunit.client.1.smithi117.stdout:3/544: write d9/d1a/d39/d4c/d36/f9f [351924,70049] 0 2024-03-22T06:15:25.990 INFO:tasks.workunit.client.1.smithi117.stdout:1/640: symlink d1/d1f/d32/d33/le1 0 2024-03-22T06:15:25.990 INFO:tasks.workunit.client.1.smithi117.stdout:1/641: dread - d1/d14/fbf zero size 2024-03-22T06:15:25.991 INFO:tasks.workunit.client.1.smithi117.stdout:0/596: symlink d1/d3/dae/dbe/ld2 0 2024-03-22T06:15:25.992 INFO:tasks.workunit.client.1.smithi117.stdout:0/597: truncate d1/d3/d2b/f2e 391897 0 2024-03-22T06:15:25.994 INFO:tasks.workunit.client.1.smithi117.stdout:8/575: getdents d1/d19/d38 0 2024-03-22T06:15:25.994 INFO:tasks.workunit.client.1.smithi117.stdout:8/576: write d1/d19/f43 [4470222,118095] 0 2024-03-22T06:15:25.996 INFO:tasks.workunit.client.1.smithi117.stdout:7/573: creat d8/d9/d37/fb5 x:0 0 0 2024-03-22T06:15:26.002 INFO:tasks.workunit.client.1.smithi117.stdout:5/626: rename dd/df/d5e/d91/d72/d33/f7c to dd/df/d95/da0/fdb 0 2024-03-22T06:15:26.002 INFO:tasks.workunit.client.1.smithi117.stdout:5/627: chown dd/d16/d86/f9d 3184 1 2024-03-22T06:15:26.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:25 smithi117 ceph-mon[36514]: pgmap v26: 65 pgs: 65 active+clean; 2.7 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 156 MiB/s rd, 179 MiB/s wr, 378 op/s 2024-03-22T06:15:26.012 INFO:tasks.workunit.client.1.smithi117.stdout:1/642: link d1/d62/d7c/la2 d1/d84/le2 0 2024-03-22T06:15:26.015 INFO:tasks.workunit.client.1.smithi117.stdout:8/577: creat d1/d19/fb9 x:0 0 0 2024-03-22T06:15:26.015 INFO:tasks.workunit.client.1.smithi117.stdout:8/578: creat d1/d3a/d44/d6a/fba x:0 0 0 2024-03-22T06:15:26.015 INFO:tasks.workunit.client.1.smithi117.stdout:8/579: write d1/f87 [648305,10872] 0 2024-03-22T06:15:26.015 INFO:tasks.workunit.client.1.smithi117.stdout:8/580: write d1/d19/f2d [2643806,50954] 0 2024-03-22T06:15:26.016 INFO:tasks.workunit.client.1.smithi117.stdout:7/574: rmdir d8/d9/da/d10/d23/d48 39 2024-03-22T06:15:26.017 INFO:tasks.workunit.client.1.smithi117.stdout:3/545: dwrite d9/d1a/d39/d4c/d99/fbb [0,4194304] 0 2024-03-22T06:15:26.017 INFO:tasks.workunit.client.1.smithi117.stdout:3/546: readlink d9/d1a/l1c 0 2024-03-22T06:15:26.017 INFO:tasks.workunit.client.1.smithi117.stdout:5/628: truncate fa 2141577 0 2024-03-22T06:15:26.018 INFO:tasks.workunit.client.1.smithi117.stdout:0/598: dwrite d1/d10/d18/d26/f75 [0,4194304] 0 2024-03-22T06:15:26.018 INFO:tasks.workunit.client.1.smithi117.stdout:5/629: write dd/d16/d1a/d32/d73/db8/fc5 [761344,36401] 0 2024-03-22T06:15:26.018 INFO:tasks.workunit.client.1.smithi117.stdout:5/630: creat dd/df/d5e/fdc x:0 0 0 2024-03-22T06:15:26.019 INFO:tasks.workunit.client.1.smithi117.stdout:1/643: mkdir d1/d30/d44/d9c/da9/de3 0 2024-03-22T06:15:26.023 INFO:tasks.workunit.client.1.smithi117.stdout:5/631: write dd/df/d95/fab [1845413,13594] 0 2024-03-22T06:15:26.026 INFO:tasks.workunit.client.1.smithi117.stdout:8/581: truncate d1/d19/d38/f8c 493769 0 2024-03-22T06:15:26.026 INFO:tasks.workunit.client.1.smithi117.stdout:8/582: chown d1/d3a/d44/d4b/l59 1 1 2024-03-22T06:15:26.029 INFO:tasks.workunit.client.1.smithi117.stdout:7/575: mknod d8/d9/d37/d5f/cb6 0 2024-03-22T06:15:26.030 INFO:tasks.workunit.client.1.smithi117.stdout:3/547: symlink d9/d1a/d39/d8a/da1/db6/lc5 0 2024-03-22T06:15:26.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:25 smithi006 ceph-mon[29301]: pgmap v26: 65 pgs: 65 active+clean; 2.7 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 156 MiB/s rd, 179 MiB/s wr, 378 op/s 2024-03-22T06:15:26.032 INFO:tasks.workunit.client.1.smithi117.stdout:4/554: sync 2024-03-22T06:15:26.032 INFO:tasks.workunit.client.1.smithi117.stdout:4/555: stat d0/d19/d28/f55 0 2024-03-22T06:15:26.036 INFO:tasks.workunit.client.1.smithi117.stdout:1/644: dwrite d1/d1f/d32/fd9 [0,4194304] 0 2024-03-22T06:15:26.037 INFO:tasks.workunit.client.1.smithi117.stdout:0/599: symlink d1/d3/d2b/d3e/ld3 0 2024-03-22T06:15:26.037 INFO:tasks.workunit.client.1.smithi117.stdout:0/600: fdatasync d1/d10/d18/d26/d49/f77 0 2024-03-22T06:15:26.039 INFO:tasks.workunit.client.1.smithi117.stdout:5/632: link dd/d16/d1a/d32/dbf/cd9 dd/df/d5e/d91/d72/d33/cdd 0 2024-03-22T06:15:26.041 INFO:tasks.workunit.client.1.smithi117.stdout:7/576: creat d8/d9/da/fb7 x:0 0 0 2024-03-22T06:15:26.041 INFO:tasks.workunit.client.1.smithi117.stdout:7/577: fdatasync d8/d9/f7c 0 2024-03-22T06:15:26.042 INFO:tasks.workunit.client.1.smithi117.stdout:3/548: truncate d9/f19 1800640 0 2024-03-22T06:15:26.044 INFO:tasks.workunit.client.1.smithi117.stdout:4/556: rmdir d0/d19/d34 39 2024-03-22T06:15:26.053 INFO:tasks.workunit.client.1.smithi117.stdout:7/578: write d8/d9/da/d10/d23/d41/f75 [66142,60932] 0 2024-03-22T06:15:26.053 INFO:tasks.workunit.client.1.smithi117.stdout:7/579: fdatasync d8/d9/d37/d45/d61/f6f 0 2024-03-22T06:15:26.055 INFO:tasks.workunit.client.1.smithi117.stdout:1/645: rename d1/d3c/d4d/d5b/db8 to d1/d62/de4 0 2024-03-22T06:15:26.055 INFO:tasks.workunit.client.1.smithi117.stdout:1/646: truncate d1/d1f/f7f 135520 0 2024-03-22T06:15:26.058 INFO:tasks.workunit.client.1.smithi117.stdout:5/633: mkdir dd/df/d5e/d91/d72/dde 0 2024-03-22T06:15:26.058 INFO:tasks.workunit.client.1.smithi117.stdout:5/634: write dd/d16/f64 [2236071,87074] 0 2024-03-22T06:15:26.058 INFO:tasks.workunit.client.1.smithi117.stdout:5/635: chown dd/d16/d5a/d66 118 1 2024-03-22T06:15:26.061 INFO:tasks.workunit.client.1.smithi117.stdout:0/601: rename d1/d10/d5c/d40/fcf to d1/d3/dae/dbe/fd4 0 2024-03-22T06:15:26.061 INFO:tasks.workunit.client.1.smithi117.stdout:0/602: creat d1/d10/d18/d26/d49/d7e/d39/dc5/fd5 x:0 0 0 2024-03-22T06:15:26.063 INFO:tasks.workunit.client.1.smithi117.stdout:7/580: dread d8/d9/da/d10/f25 [0,4194304] 0 2024-03-22T06:15:26.063 INFO:tasks.workunit.client.1.smithi117.stdout:3/549: dwrite d9/d1a/d3a/f47 [0,4194304] 0 2024-03-22T06:15:26.063 INFO:tasks.workunit.client.1.smithi117.stdout:3/550: creat d9/d1a/fc6 x:0 0 0 2024-03-22T06:15:26.063 INFO:tasks.workunit.client.1.smithi117.stdout:3/551: dread - d9/f70 zero size 2024-03-22T06:15:26.063 INFO:tasks.workunit.client.1.smithi117.stdout:1/647: creat d1/d30/d44/d83/fe5 x:0 0 0 2024-03-22T06:15:26.063 INFO:tasks.workunit.client.1.smithi117.stdout:1/648: readlink d1/d3c/l45 0 2024-03-22T06:15:26.063 INFO:tasks.workunit.client.1.smithi117.stdout:1/649: truncate d1/d22/d27/d4e/f7a 1187157 0 2024-03-22T06:15:26.064 INFO:tasks.workunit.client.1.smithi117.stdout:5/636: mknod dd/cdf 0 2024-03-22T06:15:26.064 INFO:tasks.workunit.client.1.smithi117.stdout:5/637: dread - dd/df/d5e/d91/d72/d35/d81/fd8 zero size 2024-03-22T06:15:26.065 INFO:tasks.workunit.client.1.smithi117.stdout:4/557: getdents d0/d19/d34/d43/d63/d6f/d6a 0 2024-03-22T06:15:26.065 INFO:tasks.workunit.client.1.smithi117.stdout:4/558: creat d0/d19/d9b/fc2 x:0 0 0 2024-03-22T06:15:26.066 INFO:tasks.workunit.client.1.smithi117.stdout:4/559: creat d0/fc3 x:0 0 0 2024-03-22T06:15:26.066 INFO:tasks.workunit.client.1.smithi117.stdout:4/560: readlink d0/l18 0 2024-03-22T06:15:26.067 INFO:tasks.workunit.client.1.smithi117.stdout:9/634: sync 2024-03-22T06:15:26.067 INFO:tasks.workunit.client.1.smithi117.stdout:6/627: sync 2024-03-22T06:15:26.067 INFO:tasks.workunit.client.1.smithi117.stdout:9/635: chown d4/d46 10 1 2024-03-22T06:15:26.067 INFO:tasks.workunit.client.1.smithi117.stdout:9/636: chown d4/d8/d3e/f7e 4 1 2024-03-22T06:15:26.067 INFO:tasks.workunit.client.1.smithi117.stdout:9/637: fdatasync d4/d8/f2c 0 2024-03-22T06:15:26.067 INFO:tasks.workunit.client.1.smithi117.stdout:9/638: creat d4/d8/d37/d9b/fce x:0 0 0 2024-03-22T06:15:26.068 INFO:tasks.workunit.client.1.smithi117.stdout:0/603: link d1/d10/d18/d26/l44 d1/d10/d18/d26/d49/d7e/d39/ld6 0 2024-03-22T06:15:26.068 INFO:tasks.workunit.client.1.smithi117.stdout:0/604: readlink d1/d3/lc4 0 2024-03-22T06:15:26.069 INFO:tasks.workunit.client.1.smithi117.stdout:7/581: mkdir d8/d9/da/d2b/db8 0 2024-03-22T06:15:26.069 INFO:tasks.workunit.client.1.smithi117.stdout:7/582: readlink l1 0 2024-03-22T06:15:26.075 INFO:tasks.workunit.client.1.smithi117.stdout:6/628: dread d3/db4/d2d/d53/d75/fa0 [0,4194304] 0 2024-03-22T06:15:26.080 INFO:tasks.workunit.client.1.smithi117.stdout:4/561: dwrite d0/d19/f4a [0,4194304] 0 2024-03-22T06:15:26.082 INFO:tasks.workunit.client.1.smithi117.stdout:1/650: unlink d1/d3c/f88 0 2024-03-22T06:15:26.082 INFO:tasks.workunit.client.1.smithi117.stdout:1/651: chown d1/d30/d44/d99 8090521 1 2024-03-22T06:15:26.083 INFO:tasks.workunit.client.1.smithi117.stdout:5/638: mkdir dd/df/d5e/d91/d72/dde/de0 0 2024-03-22T06:15:26.087 INFO:tasks.workunit.client.1.smithi117.stdout:6/629: dwrite d3/db4/f5e [4194304,4194304] 0 2024-03-22T06:15:26.091 INFO:tasks.workunit.client.1.smithi117.stdout:2/561: sync 2024-03-22T06:15:26.091 INFO:tasks.workunit.client.1.smithi117.stdout:2/562: getdents d2/d5/d13/d3f/d1d/d86 0 2024-03-22T06:15:26.092 INFO:tasks.workunit.client.1.smithi117.stdout:2/563: creat d2/d5/db/d2c/d7d/fb6 x:0 0 0 2024-03-22T06:15:26.092 INFO:tasks.workunit.client.1.smithi117.stdout:2/564: readlink d2/d5/d13/d3f/d1d/d3a/d71/d76/l7b 0 2024-03-22T06:15:26.093 INFO:tasks.workunit.client.1.smithi117.stdout:9/639: mknod d4/d8/d3e/ccf 0 2024-03-22T06:15:26.093 INFO:tasks.workunit.client.1.smithi117.stdout:9/640: write d4/d46/f95 [743169,30634] 0 2024-03-22T06:15:26.097 INFO:tasks.workunit.client.1.smithi117.stdout:0/605: unlink d1/f19 0 2024-03-22T06:15:26.098 INFO:tasks.workunit.client.1.smithi117.stdout:7/583: creat d8/d9/da/d10/d23/d48/fb9 x:0 0 0 2024-03-22T06:15:26.099 INFO:tasks.workunit.client.1.smithi117.stdout:3/552: rmdir d9/d93 39 2024-03-22T06:15:26.103 INFO:tasks.workunit.client.1.smithi117.stdout:4/562: mknod d0/d19/d34/d43/d63/d6f/db1/db6/cc4 0 2024-03-22T06:15:26.104 INFO:tasks.workunit.client.1.smithi117.stdout:6/630: dwrite d3/db4/d2d/d53/d59/d74/da/fc6 [0,4194304] 0 2024-03-22T06:15:26.106 INFO:tasks.workunit.client.1.smithi117.stdout:5/639: creat dd/df/d5e/d91/d36/d70/fe1 x:0 0 0 2024-03-22T06:15:26.106 INFO:tasks.workunit.client.1.smithi117.stdout:5/640: fsync dd/d16/d5a/f97 0 2024-03-22T06:15:26.106 INFO:tasks.workunit.client.1.smithi117.stdout:5/641: truncate dd/df/d5e/d91/d90/fd5 928665 0 2024-03-22T06:15:26.113 INFO:tasks.workunit.client.1.smithi117.stdout:2/565: unlink d2/d35/d38/c9b 0 2024-03-22T06:15:26.114 INFO:tasks.workunit.client.1.smithi117.stdout:2/566: write d2/d5/db/f72 [4317539,81136] 0 2024-03-22T06:15:26.115 INFO:tasks.workunit.client.1.smithi117.stdout:7/584: mknod d8/d9/da/d10/d23/d41/cba 0 2024-03-22T06:15:26.115 INFO:tasks.workunit.client.1.smithi117.stdout:7/585: truncate d8/d9/da/f72 2066037 0 2024-03-22T06:15:26.123 INFO:tasks.workunit.client.1.smithi117.stdout:2/567: link d2/d5/d13/d3f/c2f d2/d5/d13/d3f/d1d/d3a/d89/daa/cb7 0 2024-03-22T06:15:26.124 INFO:tasks.workunit.client.1.smithi117.stdout:3/553: dwrite d9/f35 [0,4194304] 0 2024-03-22T06:15:26.124 INFO:tasks.workunit.client.1.smithi117.stdout:3/554: chown d9/f2a 39076985 1 2024-03-22T06:15:26.125 INFO:tasks.workunit.client.1.smithi117.stdout:7/586: creat d8/d9/da/d10/d23/fbb x:0 0 0 2024-03-22T06:15:26.128 INFO:tasks.workunit.client.1.smithi117.stdout:4/563: rmdir d0/d19/d34/d43/d63/d8d 39 2024-03-22T06:15:26.129 INFO:tasks.workunit.client.1.smithi117.stdout:2/568: creat d2/d5/d13/d3f/d1d/d3a/fb8 x:0 0 0 2024-03-22T06:15:26.131 INFO:tasks.workunit.client.1.smithi117.stdout:6/631: dwrite d3/db4/d2d/d53/d59/d74/d28/d85/f69 [4194304,4194304] 0 2024-03-22T06:15:26.132 INFO:tasks.workunit.client.1.smithi117.stdout:5/642: dwrite dd/d16/f64 [0,4194304] 0 2024-03-22T06:15:26.136 INFO:tasks.workunit.client.1.smithi117.stdout:4/564: symlink d0/d4/d2a/lc5 0 2024-03-22T06:15:26.137 INFO:tasks.workunit.client.1.smithi117.stdout:4/565: read d0/f42 [612352,30251] 0 2024-03-22T06:15:26.141 INFO:tasks.workunit.client.1.smithi117.stdout:2/569: rename d2/d5/db/lf to d2/d5/d13/lb9 0 2024-03-22T06:15:26.145 INFO:tasks.workunit.client.1.smithi117.stdout:5/643: dread dd/df/d5e/d91/d72/d35/d81/fba [0,4194304] 0 2024-03-22T06:15:26.145 INFO:tasks.workunit.client.1.smithi117.stdout:5/644: fdatasync dd/df/d5e/d91/d90/fa6 0 2024-03-22T06:15:26.145 INFO:tasks.workunit.client.1.smithi117.stdout:5/645: readlink dd/df/d5e/d91/l41 0 2024-03-22T06:15:26.145 INFO:tasks.workunit.client.1.smithi117.stdout:5/646: chown dd/df/d5e/d91/l96 255442984 1 2024-03-22T06:15:26.145 INFO:tasks.workunit.client.1.smithi117.stdout:6/632: mkdir d3/db4/d2d/d53/d59/d74/d28/ddb 0 2024-03-22T06:15:26.146 INFO:tasks.workunit.client.1.smithi117.stdout:6/633: write d3/db4/d2d/d42/da2/f7a [5682019,26871] 0 2024-03-22T06:15:26.146 INFO:tasks.workunit.client.1.smithi117.stdout:2/570: dread d2/d5/d13/d3f/d1d/d3a/d71/f9d [0,4194304] 0 2024-03-22T06:15:26.146 INFO:tasks.workunit.client.1.smithi117.stdout:4/566: dread d0/d19/d34/f4f [0,4194304] 0 2024-03-22T06:15:26.153 INFO:tasks.workunit.client.1.smithi117.stdout:4/567: write d0/d19/d28/f55 [2236697,47923] 0 2024-03-22T06:15:26.153 INFO:tasks.workunit.client.1.smithi117.stdout:4/568: write d0/d19/d34/d43/d63/f89 [874939,74916] 0 2024-03-22T06:15:26.156 INFO:tasks.workunit.client.1.smithi117.stdout:3/555: getdents d9/d66 0 2024-03-22T06:15:26.156 INFO:tasks.workunit.client.1.smithi117.stdout:3/556: chown d9/d1a/d39/d4c/d37/d69/lc4 15 1 2024-03-22T06:15:26.157 INFO:tasks.workunit.client.1.smithi117.stdout:5/647: creat dd/d16/d1a/d32/dbf/fe2 x:0 0 0 2024-03-22T06:15:26.159 INFO:tasks.workunit.client.1.smithi117.stdout:2/571: rename d2/d5/d13/d3f/d1d/d3a/f64 to d2/fba 0 2024-03-22T06:15:26.160 INFO:tasks.workunit.client.1.smithi117.stdout:3/557: mkdir d9/d1a/d39/dc7 0 2024-03-22T06:15:26.160 INFO:tasks.workunit.client.1.smithi117.stdout:3/558: read - d9/d66/fa9 zero size 2024-03-22T06:15:26.162 INFO:tasks.workunit.client.1.smithi117.stdout:5/648: creat dd/d16/fe3 x:0 0 0 2024-03-22T06:15:26.162 INFO:tasks.workunit.client.1.smithi117.stdout:5/649: truncate dd/df/d95/f9e 647700 0 2024-03-22T06:15:26.163 INFO:tasks.workunit.client.1.smithi117.stdout:4/569: rename d0/d51/l93 to d0/d19/d34/d43/d63/lc6 0 2024-03-22T06:15:26.165 INFO:tasks.workunit.client.1.smithi117.stdout:2/572: link d2/d1b/l2e d2/d5/d13/d3f/d1d/d3a/d89/daa/lbb 0 2024-03-22T06:15:26.167 INFO:tasks.workunit.client.1.smithi117.stdout:5/650: mknod dd/df/d5e/d91/d72/d35/da2/ce4 0 2024-03-22T06:15:26.169 INFO:tasks.workunit.client.1.smithi117.stdout:3/559: write d9/d1a/d39/f48 [193346,115137] 0 2024-03-22T06:15:26.170 INFO:tasks.workunit.client.1.smithi117.stdout:6/634: dwrite d3/db4/d2d/d53/d59/fb7 [0,4194304] 0 2024-03-22T06:15:26.171 INFO:tasks.workunit.client.1.smithi117.stdout:4/570: rename d0/l1f to d0/d19/d34/d43/d63/d6f/d6a/lc7 0 2024-03-22T06:15:26.172 INFO:tasks.workunit.client.1.smithi117.stdout:4/571: creat d0/d51/fc8 x:0 0 0 2024-03-22T06:15:26.172 INFO:tasks.workunit.client.1.smithi117.stdout:4/572: write d0/d19/f1a [5218219,19026] 0 2024-03-22T06:15:26.172 INFO:tasks.workunit.client.1.smithi117.stdout:4/573: readlink d0/d51/da4/lb4 0 2024-03-22T06:15:26.177 INFO:tasks.workunit.client.1.smithi117.stdout:2/573: read d2/f4a [2320377,105824] 0 2024-03-22T06:15:26.177 INFO:tasks.workunit.client.1.smithi117.stdout:2/574: readlink d2/d1b/d39/d45/l60 0 2024-03-22T06:15:26.185 INFO:tasks.workunit.client.1.smithi117.stdout:2/575: dread d2/d5/d13/d3f/d1d/fa4 [0,4194304] 0 2024-03-22T06:15:26.185 INFO:tasks.workunit.client.1.smithi117.stdout:2/576: write d2/d6e/f84 [174734,2563] 0 2024-03-22T06:15:26.185 INFO:tasks.workunit.client.1.smithi117.stdout:2/577: dread - d2/d5/d13/d3f/d1d/d3a/d89/fab zero size 2024-03-22T06:15:26.193 INFO:tasks.workunit.client.1.smithi117.stdout:8/583: sync 2024-03-22T06:15:26.193 INFO:tasks.workunit.client.1.smithi117.stdout:9/641: sync 2024-03-22T06:15:26.194 INFO:tasks.workunit.client.1.smithi117.stdout:5/651: dwrite dd/df/d5e/d91/d72/d35/da2/fcf [0,4194304] 0 2024-03-22T06:15:26.195 INFO:tasks.workunit.client.1.smithi117.stdout:3/560: creat d9/d1a/d39/d4c/fc8 x:0 0 0 2024-03-22T06:15:26.196 INFO:tasks.workunit.client.1.smithi117.stdout:6/635: symlink d3/d40/d46/ldc 0 2024-03-22T06:15:26.196 INFO:tasks.workunit.client.1.smithi117.stdout:6/636: chown d3/c4d 7281 1 2024-03-22T06:15:26.196 INFO:tasks.workunit.client.1.smithi117.stdout:6/637: read - d3/d3d/d17/fc4 zero size 2024-03-22T06:15:26.196 INFO:tasks.workunit.client.1.smithi117.stdout:6/638: dread - d3/d61/f9b zero size 2024-03-22T06:15:26.196 INFO:tasks.workunit.client.1.smithi117.stdout:6/639: readlink d3/db4/ld3 0 2024-03-22T06:15:26.199 INFO:tasks.workunit.client.1.smithi117.stdout:1/652: sync 2024-03-22T06:15:26.200 INFO:tasks.workunit.client.1.smithi117.stdout:1/653: dread d1/d30/d44/f7b [0,4194304] 0 2024-03-22T06:15:26.203 INFO:tasks.workunit.client.1.smithi117.stdout:9/642: dread d4/d7/f16 [0,4194304] 0 2024-03-22T06:15:26.203 INFO:tasks.workunit.client.1.smithi117.stdout:9/643: chown d4/d8/la2 25157763 1 2024-03-22T06:15:26.204 INFO:tasks.workunit.client.1.smithi117.stdout:4/574: creat d0/d48/fc9 x:0 0 0 2024-03-22T06:15:26.207 INFO:tasks.workunit.client.1.smithi117.stdout:2/578: mknod d2/d1b/d39/d45/cbc 0 2024-03-22T06:15:26.211 INFO:tasks.workunit.client.1.smithi117.stdout:3/561: dread f7 [0,4194304] 0 2024-03-22T06:15:26.217 INFO:tasks.workunit.client.1.smithi117.stdout:5/652: symlink dd/d16/d86/d93/le5 0 2024-03-22T06:15:26.217 INFO:tasks.workunit.client.1.smithi117.stdout:5/653: stat dd/df/d5e/d91/d90/fa6 0 2024-03-22T06:15:26.219 INFO:tasks.workunit.client.1.smithi117.stdout:9/644: creat d4/d8/d3e/d7a/fd0 x:0 0 0 2024-03-22T06:15:26.220 INFO:tasks.workunit.client.1.smithi117.stdout:4/575: creat d0/d19/d1c/fca x:0 0 0 2024-03-22T06:15:26.222 INFO:tasks.workunit.client.1.smithi117.stdout:1/654: dwrite d1/d30/fc4 [0,4194304] 0 2024-03-22T06:15:26.222 INFO:tasks.workunit.client.1.smithi117.stdout:6/640: dwrite d3/d3d/d17/d31/f87 [0,4194304] 0 2024-03-22T06:15:26.222 INFO:tasks.workunit.client.1.smithi117.stdout:2/579: unlink d2/d5/d13/d3f/d1d/d3a/d89/daa/d4f/l5f 0 2024-03-22T06:15:26.222 INFO:tasks.workunit.client.1.smithi117.stdout:2/580: truncate d2/d5/d13/d3f/d1d/d3a/d89/daa/d4f/f8c 1046805 0 2024-03-22T06:15:26.224 INFO:tasks.workunit.client.1.smithi117.stdout:5/654: dread fa [0,4194304] 0 2024-03-22T06:15:26.227 INFO:tasks.workunit.client.1.smithi117.stdout:9/645: mkdir d4/d8/d93/db1/dd1 0 2024-03-22T06:15:26.227 INFO:tasks.workunit.client.1.smithi117.stdout:7/587: sync 2024-03-22T06:15:26.227 INFO:tasks.workunit.client.1.smithi117.stdout:0/606: sync 2024-03-22T06:15:26.227 INFO:tasks.workunit.client.1.smithi117.stdout:0/607: dread - d1/d3/d2b/d3e/de/d1f/f8d zero size 2024-03-22T06:15:26.227 INFO:tasks.workunit.client.1.smithi117.stdout:0/608: fdatasync d1/d10/f86 0 2024-03-22T06:15:26.227 INFO:tasks.workunit.client.1.smithi117.stdout:7/588: write d8/d9/da/d10/d23/d27/d32/f54 [3788034,83892] 0 2024-03-22T06:15:26.228 INFO:tasks.workunit.client.1.smithi117.stdout:7/589: write d8/d6c/fae [1003848,116893] 0 2024-03-22T06:15:26.229 INFO:tasks.workunit.client.1.smithi117.stdout:4/576: rename d0/d19/d34/d43/f7c to d0/d19/d34/d43/d63/d6f/d5f/fcb 0 2024-03-22T06:15:26.232 INFO:tasks.workunit.client.1.smithi117.stdout:2/581: dread d2/d5/db/f72 [0,4194304] 0 2024-03-22T06:15:26.236 INFO:tasks.workunit.client.1.smithi117.stdout:1/655: symlink d1/d22/d27/le6 0 2024-03-22T06:15:26.239 INFO:tasks.workunit.client.1.smithi117.stdout:6/641: creat d3/d40/d46/d84/d49/da8/fdd x:0 0 0 2024-03-22T06:15:26.239 INFO:tasks.workunit.client.1.smithi117.stdout:6/642: chown d3/f4 104 1 2024-03-22T06:15:26.242 INFO:tasks.workunit.client.1.smithi117.stdout:5/655: unlink dd/df/d5e/d91/d72/c54 0 2024-03-22T06:15:26.242 INFO:tasks.workunit.client.1.smithi117.stdout:2/582: write d2/d5/d13/d3f/d1d/f46 [3232700,124473] 0 2024-03-22T06:15:26.243 INFO:tasks.workunit.client.1.smithi117.stdout:3/562: mkdir d9/d1a/d39/d7d/d97/dc9 0 2024-03-22T06:15:26.243 INFO:tasks.workunit.client.1.smithi117.stdout:3/563: chown d9/f19 2869839 1 2024-03-22T06:15:26.247 INFO:tasks.workunit.client.1.smithi117.stdout:9/646: unlink d4/d47/d64/fcd 0 2024-03-22T06:15:26.248 INFO:tasks.workunit.client.1.smithi117.stdout:9/647: chown d4/d8/d3e/d7a/d3b/f57 16 1 2024-03-22T06:15:26.251 INFO:tasks.workunit.client.1.smithi117.stdout:0/609: truncate d1/d3/dae/dbe/fd4 3413276 0 2024-03-22T06:15:26.255 INFO:tasks.workunit.client.1.smithi117.stdout:7/590: readlink d8/d9/da/d10/d23/d27/d39/d5c/l8a 0 2024-03-22T06:15:26.255 INFO:tasks.workunit.client.1.smithi117.stdout:5/656: dwrite dd/df/d88/fa9 [0,4194304] 0 2024-03-22T06:15:26.260 INFO:tasks.workunit.client.1.smithi117.stdout:6/643: mkdir d3/db4/d2d/d53/d59/d74/d28/ddb/dde 0 2024-03-22T06:15:26.266 INFO:tasks.workunit.client.1.smithi117.stdout:2/583: mknod d2/d5/da5/cbd 0 2024-03-22T06:15:26.270 INFO:tasks.workunit.client.1.smithi117.stdout:9/648: symlink d4/d8/d93/db1/dd1/ld2 0 2024-03-22T06:15:26.270 INFO:tasks.workunit.client.1.smithi117.stdout:9/649: chown d4 57334434 1 2024-03-22T06:15:26.273 INFO:tasks.workunit.client.1.smithi117.stdout:0/610: creat d1/d3/d2b/d3e/d3f/fd7 x:0 0 0 2024-03-22T06:15:26.276 INFO:tasks.workunit.client.1.smithi117.stdout:7/591: unlink d8/d9/da/d10/d23/d41/f93 0 2024-03-22T06:15:26.277 INFO:tasks.workunit.client.1.smithi117.stdout:5/657: dwrite dd/d16/d1a/f62 [0,4194304] 0 2024-03-22T06:15:26.290 INFO:tasks.workunit.client.1.smithi117.stdout:4/577: rename d0/d48/daf to d0/d19/dcc 0 2024-03-22T06:15:26.290 INFO:tasks.workunit.client.1.smithi117.stdout:4/578: write d0/d19/d34/d43/d63/d6f/db1/db6/fbd [238560,30277] 0 2024-03-22T06:15:26.291 INFO:tasks.workunit.client.1.smithi117.stdout:5/658: dwrite dd/df/d5e/d91/f23 [0,4194304] 0 2024-03-22T06:15:26.292 INFO:tasks.workunit.client.1.smithi117.stdout:2/584: symlink d2/lbe 0 2024-03-22T06:15:26.292 INFO:tasks.workunit.client.1.smithi117.stdout:2/585: write d2/d1b/d39/fac [865547,75844] 0 2024-03-22T06:15:26.295 INFO:tasks.workunit.client.1.smithi117.stdout:3/564: getdents d9/d1a/d39/d8a/da1 0 2024-03-22T06:15:26.295 INFO:tasks.workunit.client.1.smithi117.stdout:8/584: sync 2024-03-22T06:15:26.295 INFO:tasks.workunit.client.1.smithi117.stdout:8/585: chown d1/d3a/d44/d16/l82 505 1 2024-03-22T06:15:26.296 INFO:tasks.workunit.client.1.smithi117.stdout:3/565: dread d9/d55/f57 [0,4194304] 0 2024-03-22T06:15:26.296 INFO:tasks.workunit.client.1.smithi117.stdout:3/566: getdents d9/d1a/d39/d7d/d97/dc9 0 2024-03-22T06:15:26.296 INFO:tasks.workunit.client.1.smithi117.stdout:3/567: chown f2 1523016 1 2024-03-22T06:15:26.296 INFO:tasks.workunit.client.1.smithi117.stdout:3/568: stat d9/d55/fa0 0 2024-03-22T06:15:26.296 INFO:tasks.workunit.client.1.smithi117.stdout:9/650: rmdir d4/d8/d93 39 2024-03-22T06:15:26.298 INFO:tasks.workunit.client.1.smithi117.stdout:0/611: getdents d1/d10 0 2024-03-22T06:15:26.298 INFO:tasks.workunit.client.1.smithi117.stdout:7/592: truncate d8/d9/da/d2b/f6e 7890121 0 2024-03-22T06:15:26.299 INFO:tasks.workunit.client.1.smithi117.stdout:7/593: write d8/d9/da/d10/d23/d27/d56/f9e [878140,117016] 0 2024-03-22T06:15:26.302 INFO:tasks.workunit.client.1.smithi117.stdout:1/656: rename d1/d14/f6c to d1/d1f/d32/d33/fe7 0 2024-03-22T06:15:26.302 INFO:tasks.workunit.client.1.smithi117.stdout:1/657: stat d1/l4 0 2024-03-22T06:15:26.303 INFO:tasks.workunit.client.1.smithi117.stdout:1/658: write d1/d1f/d32/d33/da4/fd8 [516642,84799] 0 2024-03-22T06:15:26.304 INFO:tasks.workunit.client.1.smithi117.stdout:4/579: creat d0/d19/d34/d43/d63/d8d/fcd x:0 0 0 2024-03-22T06:15:26.304 INFO:tasks.workunit.client.1.smithi117.stdout:4/580: readlink d0/d19/d28/d47/l8a 0 2024-03-22T06:15:26.305 INFO:tasks.workunit.client.1.smithi117.stdout:2/586: chown d2/d5/d13/d57/f36 47786 1 2024-03-22T06:15:26.305 INFO:tasks.workunit.client.1.smithi117.stdout:2/587: creat d2/d5/d41/fbf x:0 0 0 2024-03-22T06:15:26.308 INFO:tasks.workunit.client.1.smithi117.stdout:8/586: creat d1/d28/d6b/da3/fbb x:0 0 0 2024-03-22T06:15:26.309 INFO:tasks.workunit.client.1.smithi117.stdout:3/569: creat d9/d1a/d39/d8a/fca x:0 0 0 2024-03-22T06:15:26.309 INFO:tasks.workunit.client.1.smithi117.stdout:3/570: truncate d9/d1a/d39/d8a/da1/fb5 227395 0 2024-03-22T06:15:26.310 INFO:tasks.workunit.client.1.smithi117.stdout:3/571: chown d9/d1a/d39/d4c/d36/c9e 25685924 1 2024-03-22T06:15:26.313 INFO:tasks.workunit.client.1.smithi117.stdout:9/651: unlink d4/d8/d37/fca 0 2024-03-22T06:15:26.317 INFO:tasks.workunit.client.1.smithi117.stdout:0/612: symlink d1/d3/ld8 0 2024-03-22T06:15:26.319 INFO:tasks.workunit.client.1.smithi117.stdout:4/581: write d0/d4/d2a/f5d [2951191,47285] 0 2024-03-22T06:15:26.319 INFO:tasks.workunit.client.1.smithi117.stdout:4/582: chown d0/d4/d2a/d5b/f97 2 1 2024-03-22T06:15:26.319 INFO:tasks.workunit.client.1.smithi117.stdout:7/594: unlink d8/d9/d37/d45/d88/faa 0 2024-03-22T06:15:26.319 INFO:tasks.workunit.client.1.smithi117.stdout:7/595: write d8/d9/d37/fb5 [125152,62355] 0 2024-03-22T06:15:26.322 INFO:tasks.workunit.client.1.smithi117.stdout:0/613: creat d1/d3/dae/dbe/fd9 x:0 0 0 2024-03-22T06:15:26.325 INFO:tasks.workunit.client.1.smithi117.stdout:9/652: read d4/d8/d37/d58/f8a [1763697,120505] 0 2024-03-22T06:15:26.326 INFO:tasks.workunit.client.1.smithi117.stdout:1/659: dwrite d1/d1f/fb1 [0,4194304] 0 2024-03-22T06:15:26.331 INFO:tasks.workunit.client.1.smithi117.stdout:7/596: write d8/d9/d37/d45/d61/f71 [3021895,4334] 0 2024-03-22T06:15:26.335 INFO:tasks.workunit.client.1.smithi117.stdout:0/614: rename d1/d10/d18/d26/d49/f77 to d1/d10/d18/fda 0 2024-03-22T06:15:26.340 INFO:tasks.workunit.client.1.smithi117.stdout:2/588: dwrite d2/d5/da5/fa7 [0,4194304] 0 2024-03-22T06:15:26.341 INFO:tasks.workunit.client.1.smithi117.stdout:0/615: mkdir d1/d10/d5c/d40/d4e/dbc/ddb 0 2024-03-22T06:15:26.341 INFO:tasks.workunit.client.1.smithi117.stdout:0/616: chown d1/d10/d18/ca0 2358 1 2024-03-22T06:15:26.341 INFO:tasks.workunit.client.1.smithi117.stdout:0/617: write d1/d3/d2b/d3e/de/d1f/f9b [647603,84934] 0 2024-03-22T06:15:26.341 INFO:tasks.workunit.client.1.smithi117.stdout:0/618: chown d1/d3/l20 19357669 1 2024-03-22T06:15:26.349 INFO:tasks.workunit.client.1.smithi117.stdout:3/572: dwrite d9/f30 [0,4194304] 0 2024-03-22T06:15:26.349 INFO:tasks.workunit.client.1.smithi117.stdout:8/587: dwrite f0 [0,4194304] 0 2024-03-22T06:15:26.350 INFO:tasks.workunit.client.1.smithi117.stdout:1/660: rename d1/d1f/db2 to d1/d3c/d4d/de8 0 2024-03-22T06:15:26.352 INFO:tasks.workunit.client.1.smithi117.stdout:9/653: dwrite d4/d7/d27/f97 [0,4194304] 0 2024-03-22T06:15:26.352 INFO:tasks.workunit.client.1.smithi117.stdout:9/654: creat d4/d8/d3f/dc3/fd3 x:0 0 0 2024-03-22T06:15:26.355 INFO:tasks.workunit.client.1.smithi117.stdout:8/588: creat d1/d19/d98/fbc x:0 0 0 2024-03-22T06:15:26.355 INFO:tasks.workunit.client.1.smithi117.stdout:8/589: fdatasync d1/d19/d98/fbc 0 2024-03-22T06:15:26.361 INFO:tasks.workunit.client.1.smithi117.stdout:9/655: dread d4/d7/d27/f56 [0,4194304] 0 2024-03-22T06:15:26.362 INFO:tasks.workunit.client.1.smithi117.stdout:9/656: fdatasync d4/d8/d3e/d7a/f70 0 2024-03-22T06:15:26.362 INFO:tasks.workunit.client.1.smithi117.stdout:9/657: dread - d4/d8/d3e/d7a/d2f/f9f zero size 2024-03-22T06:15:26.362 INFO:tasks.workunit.client.1.smithi117.stdout:7/597: rename d8/d9/da/d10/d23/d27/l90 to d8/d9/da/d2b/db8/lbc 0 2024-03-22T06:15:26.362 INFO:tasks.workunit.client.1.smithi117.stdout:7/598: chown d8/d9/da/d2b/d9b 451 1 2024-03-22T06:15:26.364 INFO:tasks.workunit.client.1.smithi117.stdout:1/661: mknod d1/d30/d44/d83/ce9 0 2024-03-22T06:15:26.364 INFO:tasks.workunit.client.1.smithi117.stdout:1/662: chown d1/d22 53 1 2024-03-22T06:15:26.365 INFO:tasks.workunit.client.1.smithi117.stdout:0/619: dwrite d1/d10/f89 [0,4194304] 0 2024-03-22T06:15:26.365 INFO:tasks.workunit.client.1.smithi117.stdout:3/573: dread f7 [0,4194304] 0 2024-03-22T06:15:26.367 INFO:tasks.workunit.client.1.smithi117.stdout:8/590: mknod d1/d3a/d44/d11/cbd 0 2024-03-22T06:15:26.367 INFO:tasks.workunit.client.1.smithi117.stdout:8/591: write d1/d19/d38/faa [663863,53573] 0 2024-03-22T06:15:26.369 INFO:tasks.workunit.client.1.smithi117.stdout:9/658: link d4/d8/d3e/c7c d4/d47/d85/dab/cd4 0 2024-03-22T06:15:26.371 INFO:tasks.workunit.client.1.smithi117.stdout:7/599: unlink d8/d50/d5e/fb2 0 2024-03-22T06:15:26.371 INFO:tasks.workunit.client.1.smithi117.stdout:7/600: creat d8/d50/d5e/fbd x:0 0 0 2024-03-22T06:15:26.371 INFO:tasks.workunit.client.1.smithi117.stdout:7/601: creat d8/d50/d5e/fbe x:0 0 0 2024-03-22T06:15:26.371 INFO:tasks.workunit.client.1.smithi117.stdout:7/602: read d8/d9/d37/f4e [2416847,18161] 0 2024-03-22T06:15:26.371 INFO:tasks.workunit.client.1.smithi117.stdout:7/603: creat d8/d9/d37/d45/d88/fbf x:0 0 0 2024-03-22T06:15:26.372 INFO:tasks.workunit.client.1.smithi117.stdout:1/663: unlink d1/d30/d44/d99/ldc 0 2024-03-22T06:15:26.373 INFO:tasks.workunit.client.1.smithi117.stdout:1/664: write d1/d30/fa8 [559721,104136] 0 2024-03-22T06:15:26.376 INFO:tasks.workunit.client.1.smithi117.stdout:0/620: symlink d1/d10/d5c/d40/dbb/ldc 0 2024-03-22T06:15:26.378 INFO:tasks.workunit.client.1.smithi117.stdout:3/574: dread d9/d1a/d39/d4c/d99/fbb [0,4194304] 0 2024-03-22T06:15:26.378 INFO:tasks.workunit.client.1.smithi117.stdout:2/589: dwrite d2/d5/d13/d57/f63 [0,4194304] 0 2024-03-22T06:15:26.379 INFO:tasks.workunit.client.1.smithi117.stdout:2/590: write d2/d5/db/f72 [4653337,75464] 0 2024-03-22T06:15:26.379 INFO:tasks.workunit.client.1.smithi117.stdout:2/591: readlink d2/d5/d13/d3f/l9e 0 2024-03-22T06:15:26.382 INFO:tasks.workunit.client.1.smithi117.stdout:9/659: write d4/d7/d27/f97 [2813359,68453] 0 2024-03-22T06:15:26.385 INFO:tasks.workunit.client.1.smithi117.stdout:0/621: mkdir d1/d3/d2b/d3e/de/d1f/ddd 0 2024-03-22T06:15:26.386 INFO:tasks.workunit.client.1.smithi117.stdout:8/592: rmdir d1/d3a/d44/d6a 39 2024-03-22T06:15:26.390 INFO:tasks.workunit.client.1.smithi117.stdout:1/665: dwrite d1/d22/d27/faf [0,4194304] 0 2024-03-22T06:15:26.390 INFO:tasks.workunit.client.1.smithi117.stdout:1/666: creat d1/d30/d50/fea x:0 0 0 2024-03-22T06:15:26.390 INFO:tasks.workunit.client.1.smithi117.stdout:1/667: dread - d1/d3c/d4d/d5b/fd6 zero size 2024-03-22T06:15:26.390 INFO:tasks.workunit.client.1.smithi117.stdout:8/593: dread d1/d3a/d44/d9/f6c [0,4194304] 0 2024-03-22T06:15:26.395 INFO:tasks.workunit.client.1.smithi117.stdout:3/575: mknod d9/d1a/d39/dc7/ccb 0 2024-03-22T06:15:26.399 INFO:tasks.workunit.client.1.smithi117.stdout:2/592: mkdir d2/d5/d13/d3f/d1d/d3a/d89/dc0 0 2024-03-22T06:15:26.399 INFO:tasks.workunit.client.1.smithi117.stdout:2/593: readlink d2/d1b/l2e 0 2024-03-22T06:15:26.402 INFO:tasks.workunit.client.1.smithi117.stdout:9/660: link d4/d7/f24 d4/d8/d3e/d7a/fd5 0 2024-03-22T06:15:26.403 INFO:tasks.workunit.client.1.smithi117.stdout:6/644: sync 2024-03-22T06:15:26.407 INFO:tasks.workunit.client.1.smithi117.stdout:7/604: rename d8/d9/da/d10/d23/d27/d39/d26/l2d to d8/d9/da/d10/d23/d48/lc0 0 2024-03-22T06:15:26.407 INFO:tasks.workunit.client.1.smithi117.stdout:7/605: chown d8/d9/da/d10/d23/d27/d32/l3f 124660768 1 2024-03-22T06:15:26.407 INFO:tasks.workunit.client.1.smithi117.stdout:2/594: dread d2/d1b/d39/f65 [0,4194304] 0 2024-03-22T06:15:26.411 INFO:tasks.workunit.client.1.smithi117.stdout:2/595: read d2/fba [3137651,54802] 0 2024-03-22T06:15:26.412 INFO:tasks.workunit.client.1.smithi117.stdout:2/596: dread - d2/d5/d41/d83/f87 zero size 2024-03-22T06:15:26.413 INFO:tasks.workunit.client.1.smithi117.stdout:0/622: symlink d1/d10/d18/d26/d49/lde 0 2024-03-22T06:15:26.414 INFO:tasks.workunit.client.1.smithi117.stdout:6/645: read d3/db4/d2d/d53/d59/d74/d28/d85/f69 [5215982,44890] 0 2024-03-22T06:15:26.424 INFO:tasks.workunit.client.1.smithi117.stdout:9/661: dwrite d4/d8/d3e/d7a/d2f/f7b [0,4194304] 0 2024-03-22T06:15:26.426 INFO:tasks.workunit.client.1.smithi117.stdout:1/668: mkdir d1/d62/deb 0 2024-03-22T06:15:26.426 INFO:tasks.workunit.client.1.smithi117.stdout:1/669: stat d1 0 2024-03-22T06:15:26.428 INFO:tasks.workunit.client.1.smithi117.stdout:5/659: sync 2024-03-22T06:15:26.428 INFO:tasks.workunit.client.1.smithi117.stdout:5/660: write dd/df/d5e/f92 [888330,36148] 0 2024-03-22T06:15:26.428 INFO:tasks.workunit.client.1.smithi117.stdout:5/661: read - dd/f8a zero size 2024-03-22T06:15:26.428 INFO:tasks.workunit.client.1.smithi117.stdout:5/662: fsync dd/d16/fbc 0 2024-03-22T06:15:26.428 INFO:tasks.workunit.client.1.smithi117.stdout:5/663: creat dd/df/d95/fe6 x:0 0 0 2024-03-22T06:15:26.428 INFO:tasks.workunit.client.1.smithi117.stdout:8/594: getdents d1/d3a/d44/d6a 0 2024-03-22T06:15:26.431 INFO:tasks.workunit.client.1.smithi117.stdout:3/576: mkdir d9/d1a/d39/d4c/d37/dcc 0 2024-03-22T06:15:26.431 INFO:tasks.workunit.client.1.smithi117.stdout:3/577: write d9/d1a/d39/d4c/d99/f9a [1339508,105543] 0 2024-03-22T06:15:26.434 INFO:tasks.workunit.client.1.smithi117.stdout:2/597: mknod d2/d5/d13/d3f/d1d/d3a/da6/cc1 0 2024-03-22T06:15:26.436 INFO:tasks.workunit.client.1.smithi117.stdout:8/595: dread d1/d3a/d44/d9/f13 [0,4194304] 0 2024-03-22T06:15:26.436 INFO:tasks.workunit.client.1.smithi117.stdout:8/596: creat d1/d28/fbe x:0 0 0 2024-03-22T06:15:26.436 INFO:tasks.workunit.client.1.smithi117.stdout:8/597: write d1/d28/fbe [148792,62426] 0 2024-03-22T06:15:26.438 INFO:tasks.workunit.client.1.smithi117.stdout:0/623: creat d1/d3/dae/fdf x:0 0 0 2024-03-22T06:15:26.442 INFO:tasks.workunit.client.1.smithi117.stdout:6/646: mknod d3/db4/d2d/d53/cdf 0 2024-03-22T06:15:26.442 INFO:tasks.workunit.client.1.smithi117.stdout:6/647: readlink d3/db4/d2d/d42/l58 0 2024-03-22T06:15:26.442 INFO:tasks.workunit.client.1.smithi117.stdout:6/648: chown d3/db4/d2d/d53/cdf 1385615990 1 2024-03-22T06:15:26.446 INFO:tasks.workunit.client.1.smithi117.stdout:0/624: dread d1/d3/d2b/d3e/de/f23 [0,4194304] 0 2024-03-22T06:15:26.447 INFO:tasks.workunit.client.1.smithi117.stdout:9/662: unlink d4/d7/d27/f7d 0 2024-03-22T06:15:26.447 INFO:tasks.workunit.client.1.smithi117.stdout:9/663: write d4/d46/fa6 [304247,38256] 0 2024-03-22T06:15:26.447 INFO:tasks.workunit.client.1.smithi117.stdout:9/664: write d4/d7/d27/d5b/f6a [1108694,117824] 0 2024-03-22T06:15:26.448 INFO:tasks.workunit.client.1.smithi117.stdout:4/583: sync 2024-03-22T06:15:26.448 INFO:tasks.workunit.client.1.smithi117.stdout:1/670: dwrite d1/d62/f67 [4194304,4194304] 0 2024-03-22T06:15:26.449 INFO:tasks.workunit.client.1.smithi117.stdout:1/671: fsync d1/d62/f67 0 2024-03-22T06:15:26.450 INFO:tasks.workunit.client.1.smithi117.stdout:4/584: dread d0/d19/d1c/f84 [0,4194304] 0 2024-03-22T06:15:26.453 INFO:tasks.workunit.client.1.smithi117.stdout:5/664: mknod dd/df/d95/da0/da8/ce7 0 2024-03-22T06:15:26.456 INFO:tasks.workunit.client.1.smithi117.stdout:3/578: creat d9/d93/fcd x:0 0 0 2024-03-22T06:15:26.456 INFO:tasks.workunit.client.1.smithi117.stdout:3/579: fdatasync d9/d1a/d39/d4c/d36/f50 0 2024-03-22T06:15:26.457 INFO:tasks.workunit.client.1.smithi117.stdout:3/580: write d9/d1a/d39/d4c/d36/f9f [1450495,33107] 0 2024-03-22T06:15:26.457 INFO:tasks.workunit.client.1.smithi117.stdout:3/581: write d9/d93/fb4 [520493,11375] 0 2024-03-22T06:15:26.457 INFO:tasks.workunit.client.1.smithi117.stdout:3/582: chown d9/d93/cad 77 1 2024-03-22T06:15:26.457 INFO:tasks.workunit.client.1.smithi117.stdout:3/583: chown d9/d1a/d39/f45 30776 1 2024-03-22T06:15:26.457 INFO:tasks.workunit.client.1.smithi117.stdout:2/598: mknod d2/cc2 0 2024-03-22T06:15:26.457 INFO:tasks.workunit.client.1.smithi117.stdout:6/649: dread d3/db4/d2d/d42/da2/f7a [0,4194304] 0 2024-03-22T06:15:26.457 INFO:tasks.workunit.client.1.smithi117.stdout:6/650: fsync d3/d40/d46/d84/f7b 0 2024-03-22T06:15:26.457 INFO:tasks.workunit.client.1.smithi117.stdout:6/651: dread - d3/db4/d2d/d53/fd6 zero size 2024-03-22T06:15:26.457 INFO:tasks.workunit.client.1.smithi117.stdout:3/584: write d9/d55/f7c [465746,41102] 0 2024-03-22T06:15:26.458 INFO:tasks.workunit.client.1.smithi117.stdout:8/598: dwrite d1/d3a/d44/f3c [0,4194304] 0 2024-03-22T06:15:26.458 INFO:tasks.workunit.client.1.smithi117.stdout:8/599: creat d1/d3a/d44/d4b/fbf x:0 0 0 2024-03-22T06:15:26.458 INFO:tasks.workunit.client.1.smithi117.stdout:5/665: dread dd/f29 [0,4194304] 0 2024-03-22T06:15:26.459 INFO:tasks.workunit.client.1.smithi117.stdout:1/672: write d1/d22/d27/f34 [1759764,25775] 0 2024-03-22T06:15:26.459 INFO:tasks.workunit.client.1.smithi117.stdout:0/625: rename d1/d10/f11 to d1/d10/d18/d26/d49/d7e/fe0 0 2024-03-22T06:15:26.462 INFO:tasks.workunit.client.1.smithi117.stdout:4/585: unlink d0/d51/l72 0 2024-03-22T06:15:26.463 INFO:tasks.workunit.client.1.smithi117.stdout:6/652: symlink d3/d40/d46/d84/d49/le0 0 2024-03-22T06:15:26.465 INFO:tasks.workunit.client.1.smithi117.stdout:3/585: fdatasync d9/d1a/d39/d4c/d36/d41/f65 0 2024-03-22T06:15:26.466 INFO:tasks.workunit.client.1.smithi117.stdout:4/586: dread d0/d19/d34/d43/f62 [0,4194304] 0 2024-03-22T06:15:26.466 INFO:tasks.workunit.client.1.smithi117.stdout:4/587: dread - d0/d4/d2a/d5b/f8e zero size 2024-03-22T06:15:26.468 INFO:tasks.workunit.client.1.smithi117.stdout:1/673: read d1/fc [9734557,109742] 0 2024-03-22T06:15:26.482 INFO:tasks.workunit.client.1.smithi117.stdout:8/600: symlink d1/d28/d66/d7f/d9d/lc0 0 2024-03-22T06:15:26.484 INFO:tasks.workunit.client.1.smithi117.stdout:8/601: dread d1/d28/f7d [0,4194304] 0 2024-03-22T06:15:26.484 INFO:tasks.workunit.client.1.smithi117.stdout:8/602: fdatasync d1/d19/d98/fbc 0 2024-03-22T06:15:26.484 INFO:tasks.workunit.client.1.smithi117.stdout:5/666: mknod dd/df/d5e/d91/d72/d35/da2/ce8 0 2024-03-22T06:15:26.489 INFO:tasks.workunit.client.1.smithi117.stdout:3/586: link d9/d1a/d39/d4c/d36/d41/l76 d9/d1a/d3a/lce 0 2024-03-22T06:15:26.490 INFO:tasks.workunit.client.1.smithi117.stdout:4/588: creat d0/d19/d34/d43/fce x:0 0 0 2024-03-22T06:15:26.491 INFO:tasks.workunit.client.1.smithi117.stdout:4/589: creat d0/d19/d34/d43/d63/d6f/db1/db6/fcf x:0 0 0 2024-03-22T06:15:26.494 INFO:tasks.workunit.client.1.smithi117.stdout:1/674: mkdir d1/d3c/d4d/de8/dec 0 2024-03-22T06:15:26.495 INFO:tasks.workunit.client.1.smithi117.stdout:7/606: sync 2024-03-22T06:15:26.495 INFO:tasks.workunit.client.1.smithi117.stdout:7/607: write d8/d9/d37/d45/d88/fbf [925165,19895] 0 2024-03-22T06:15:26.495 INFO:tasks.workunit.client.1.smithi117.stdout:8/603: mknod d1/d3a/cc1 0 2024-03-22T06:15:26.495 INFO:tasks.workunit.client.1.smithi117.stdout:8/604: stat d1/d28/d66/d7f/f92 0 2024-03-22T06:15:26.506 INFO:tasks.workunit.client.1.smithi117.stdout:1/675: creat d1/d84/fed x:0 0 0 2024-03-22T06:15:26.506 INFO:tasks.workunit.client.1.smithi117.stdout:1/676: dread - d1/d30/d44/d99/fb7 zero size 2024-03-22T06:15:26.506 INFO:tasks.workunit.client.1.smithi117.stdout:1/677: fsync d1/d84/d77/fd2 0 2024-03-22T06:15:26.506 INFO:tasks.workunit.client.1.smithi117.stdout:1/678: chown d1/d1f 357023180 1 2024-03-22T06:15:26.506 INFO:tasks.workunit.client.1.smithi117.stdout:1/679: truncate d1/f13 4799166 0 2024-03-22T06:15:26.506 INFO:tasks.workunit.client.1.smithi117.stdout:7/608: truncate d8/d9/da/d10/d23/d27/d32/f44 3283827 0 2024-03-22T06:15:26.508 INFO:tasks.workunit.client.1.smithi117.stdout:9/665: rename d4/d8/d37 to d4/d8/d93/db1/dd1/dd6 0 2024-03-22T06:15:26.508 INFO:tasks.workunit.client.1.smithi117.stdout:8/605: write d1/d28/f31 [1919090,34274] 0 2024-03-22T06:15:26.508 INFO:tasks.workunit.client.1.smithi117.stdout:8/606: fsync d1/d19/d38/f9f 0 2024-03-22T06:15:26.508 INFO:tasks.workunit.client.1.smithi117.stdout:8/607: read d1/d28/d66/f74 [4001897,127251] 0 2024-03-22T06:15:26.509 INFO:tasks.workunit.client.1.smithi117.stdout:5/667: dwrite dd/d16/d1a/d32/d73/db8/fb9 [0,4194304] 0 2024-03-22T06:15:26.509 INFO:tasks.workunit.client.1.smithi117.stdout:4/590: dwrite d0/d19/f98 [0,4194304] 0 2024-03-22T06:15:26.509 INFO:tasks.workunit.client.1.smithi117.stdout:4/591: fsync d0/d4/d2a/d5b/f8e 0 2024-03-22T06:15:26.510 INFO:tasks.workunit.client.1.smithi117.stdout:1/680: unlink d1/d1f/d32/d33/f3d 0 2024-03-22T06:15:26.510 INFO:tasks.workunit.client.1.smithi117.stdout:1/681: fdatasync d1/d1f/f8f 0 2024-03-22T06:15:26.515 INFO:tasks.workunit.client.1.smithi117.stdout:7/609: symlink d8/d9/d37/d45/lc1 0 2024-03-22T06:15:26.516 INFO:tasks.workunit.client.1.smithi117.stdout:7/610: dread d8/d9/d37/fb5 [0,4194304] 0 2024-03-22T06:15:26.519 INFO:tasks.workunit.client.1.smithi117.stdout:9/666: creat d4/d8/d3f/fd7 x:0 0 0 2024-03-22T06:15:26.519 INFO:tasks.workunit.client.1.smithi117.stdout:8/608: write d1/fa9 [1442034,61812] 0 2024-03-22T06:15:26.529 INFO:tasks.workunit.client.1.smithi117.stdout:4/592: creat d0/d4/d2a/d2f/fd0 x:0 0 0 2024-03-22T06:15:26.531 INFO:tasks.workunit.client.1.smithi117.stdout:2/599: rename d2/f4 to d2/d5/d13/d57/fc3 0 2024-03-22T06:15:26.531 INFO:tasks.workunit.client.1.smithi117.stdout:0/626: rename d1/d10/d18 to d1/d10/d18/d26/de1 22 2024-03-22T06:15:26.533 INFO:tasks.workunit.client.1.smithi117.stdout:9/667: mkdir d4/d8/d93/db1/dd1/dd6/d9b/dd8 0 2024-03-22T06:15:26.539 INFO:tasks.workunit.client.1.smithi117.stdout:6/653: rename d3/d3d/l2b to d3/db4/d2d/d53/le1 0 2024-03-22T06:15:26.540 INFO:tasks.workunit.client.1.smithi117.stdout:0/627: symlink d1/d3/d2b/d3e/d3f/d8b/le2 0 2024-03-22T06:15:26.541 INFO:tasks.workunit.client.1.smithi117.stdout:4/593: dread d0/d19/d34/f4f [0,4194304] 0 2024-03-22T06:15:26.541 INFO:tasks.workunit.client.1.smithi117.stdout:4/594: write d0/d19/d34/d43/d63/f74 [635287,123020] 0 2024-03-22T06:15:26.541 INFO:tasks.workunit.client.1.smithi117.stdout:4/595: truncate d0/d19/d34/d43/d63/d6f/d6a/f9d 361675 0 2024-03-22T06:15:26.543 INFO:tasks.workunit.client.1.smithi117.stdout:3/587: rename d9/d1a/l33 to d9/d1a/d39/d4c/d99/lcf 0 2024-03-22T06:15:26.548 INFO:tasks.workunit.client.1.smithi117.stdout:1/682: dwrite d1/f13 [0,4194304] 0 2024-03-22T06:15:26.548 INFO:tasks.workunit.client.1.smithi117.stdout:1/683: chown d1/d3c/d4d/d5b/fd6 482349811 1 2024-03-22T06:15:26.549 INFO:tasks.workunit.client.1.smithi117.stdout:1/684: read - d1/d30/d35/fc5 zero size 2024-03-22T06:15:26.551 INFO:tasks.workunit.client.1.smithi117.stdout:2/600: rename d2/d1b/l74 to d2/d5/db/lc4 0 2024-03-22T06:15:26.556 INFO:tasks.workunit.client.1.smithi117.stdout:5/668: dwrite dd/df/d5e/d91/d36/f25 [0,4194304] 0 2024-03-22T06:15:26.557 INFO:tasks.workunit.client.1.smithi117.stdout:7/611: dwrite d8/d9/da/d10/d23/d27/d32/f81 [0,4194304] 0 2024-03-22T06:15:26.557 INFO:tasks.workunit.client.1.smithi117.stdout:1/685: link d1/d22/d27/d4e/d92/fab d1/d3c/d4d/de8/fee 0 2024-03-22T06:15:26.558 INFO:tasks.workunit.client.1.smithi117.stdout:8/609: dwrite d1/d19/d38/faa [0,4194304] 0 2024-03-22T06:15:26.558 INFO:tasks.workunit.client.1.smithi117.stdout:8/610: readlink d1/d3a/d44/d16/l82 0 2024-03-22T06:15:26.559 INFO:tasks.workunit.client.1.smithi117.stdout:2/601: rmdir d2/d35/d38/d98 0 2024-03-22T06:15:26.566 INFO:tasks.workunit.client.1.smithi117.stdout:1/686: creat d1/d62/fef x:0 0 0 2024-03-22T06:15:26.571 INFO:tasks.workunit.client.1.smithi117.stdout:2/602: creat d2/d5/d13/d3f/d1d/d3a/d89/daa/fc5 x:0 0 0 2024-03-22T06:15:26.571 INFO:tasks.workunit.client.1.smithi117.stdout:5/669: dread dd/df/d5e/d91/d36/f39 [0,4194304] 0 2024-03-22T06:15:26.572 INFO:tasks.workunit.client.1.smithi117.stdout:1/687: rmdir d1/d30/d35 39 2024-03-22T06:15:26.573 INFO:tasks.workunit.client.1.smithi117.stdout:1/688: creat d1/d30/d44/d99/ff0 x:0 0 0 2024-03-22T06:15:26.577 INFO:tasks.workunit.client.1.smithi117.stdout:7/612: write d8/d9/d37/d45/d88/fbf [697084,25889] 0 2024-03-22T06:15:26.577 INFO:tasks.workunit.client.1.smithi117.stdout:9/668: dwrite d4/d47/d64/d72/d8d/fc8 [0,4194304] 0 2024-03-22T06:15:26.577 INFO:tasks.workunit.client.1.smithi117.stdout:7/613: creat d8/d6c/fc2 x:0 0 0 2024-03-22T06:15:26.577 INFO:tasks.workunit.client.1.smithi117.stdout:5/670: creat dd/d16/d1a/fe9 x:0 0 0 2024-03-22T06:15:26.584 INFO:tasks.workunit.client.1.smithi117.stdout:6/654: dwrite d3/db4/d2d/d53/fd6 [0,4194304] 0 2024-03-22T06:15:26.584 INFO:tasks.workunit.client.1.smithi117.stdout:0/628: dwrite d1/d3/d2b/d3e/dad/fc9 [0,4194304] 0 2024-03-22T06:15:26.584 INFO:tasks.workunit.client.1.smithi117.stdout:8/611: dwrite d1/d3a/d44/d9/f13 [4194304,4194304] 0 2024-03-22T06:15:26.585 INFO:tasks.workunit.client.1.smithi117.stdout:3/588: dwrite d9/d1a/d39/d4c/f4e [0,4194304] 0 2024-03-22T06:15:26.585 INFO:tasks.workunit.client.1.smithi117.stdout:3/589: stat d9/d55/fa0 0 2024-03-22T06:15:26.585 INFO:tasks.workunit.client.1.smithi117.stdout:3/590: chown d9/d1a/d3a/f47 876224 1 2024-03-22T06:15:26.588 INFO:tasks.workunit.client.1.smithi117.stdout:9/669: mkdir d4/d8/d93/db1/dd1/dd9 0 2024-03-22T06:15:26.588 INFO:tasks.workunit.client.1.smithi117.stdout:9/670: chown d4/d47/d64/d72/d8d/fc8 10728351 1 2024-03-22T06:15:26.596 INFO:tasks.workunit.client.1.smithi117.stdout:5/671: dwrite dd/d16/fbc [0,4194304] 0 2024-03-22T06:15:26.597 INFO:tasks.workunit.client.1.smithi117.stdout:2/603: rename d2/d1b to d2/d5/d13/d3f/d1d/d3a/d89/dc6 0 2024-03-22T06:15:26.598 INFO:tasks.workunit.client.1.smithi117.stdout:2/604: write d2/d5/d13/d3f/d1d/f46 [482835,19645] 0 2024-03-22T06:15:26.600 INFO:tasks.workunit.client.1.smithi117.stdout:1/689: dwrite d1/d3c/d4d/d5b/fd6 [0,4194304] 0 2024-03-22T06:15:26.605 INFO:tasks.workunit.client.1.smithi117.stdout:7/614: dread d8/d9/da/d10/d23/d27/d39/f5d [0,4194304] 0 2024-03-22T06:15:26.605 INFO:tasks.workunit.client.1.smithi117.stdout:6/655: link d3/d3d/d17/f9a d3/d40/d46/d84/dbe/fe2 0 2024-03-22T06:15:26.612 INFO:tasks.workunit.client.1.smithi117.stdout:7/615: dread d8/d9/d37/d45/d61/f71 [0,4194304] 0 2024-03-22T06:15:26.612 INFO:tasks.workunit.client.1.smithi117.stdout:7/616: readlink d8/d9/da/d10/l7d 0 2024-03-22T06:15:26.613 INFO:tasks.workunit.client.1.smithi117.stdout:9/671: dwrite d4/d8/d3f/fd7 [0,4194304] 0 2024-03-22T06:15:26.614 INFO:tasks.workunit.client.1.smithi117.stdout:9/672: chown d4/d7/d27/d5b/dae 3415906 1 2024-03-22T06:15:26.616 INFO:tasks.workunit.client.1.smithi117.stdout:0/629: symlink d1/d3/d2b/d3e/de/d1f/d3b/le3 0 2024-03-22T06:15:26.616 INFO:tasks.workunit.client.1.smithi117.stdout:0/630: truncate d1/d10/d5c/d40/dbb/dbf/fd1 15814 0 2024-03-22T06:15:26.625 INFO:tasks.workunit.client.1.smithi117.stdout:1/690: dwrite d1/d22/d27/f68 [0,4194304] 0 2024-03-22T06:15:26.631 INFO:tasks.workunit.client.1.smithi117.stdout:5/672: rename dd/l42 to dd/df/d5e/d91/d90/dca/lea 0 2024-03-22T06:15:26.636 INFO:tasks.workunit.client.1.smithi117.stdout:2/605: creat d2/d5/da5/fc7 x:0 0 0 2024-03-22T06:15:26.637 INFO:tasks.workunit.client.1.smithi117.stdout:5/673: dread dd/df/d5e/d91/d36/f25 [4194304,4194304] 0 2024-03-22T06:15:26.644 INFO:tasks.workunit.client.1.smithi117.stdout:6/656: symlink d3/d3d/dcb/dcc/le3 0 2024-03-22T06:15:26.645 INFO:tasks.workunit.client.1.smithi117.stdout:6/657: fdatasync d3/d40/d46/d84/d49/d79/d9d/fce 0 2024-03-22T06:15:26.652 INFO:tasks.workunit.client.1.smithi117.stdout:9/673: mkdir d4/d47/d64/dc5/dda 0 2024-03-22T06:15:26.654 INFO:tasks.workunit.client.1.smithi117.stdout:6/658: write d3/d40/d46/d84/f7b [2702844,36694] 0 2024-03-22T06:15:26.662 INFO:tasks.workunit.client.1.smithi117.stdout:9/674: write d4/d8/d3e/d7a/f63 [3544496,97994] 0 2024-03-22T06:15:26.664 INFO:tasks.workunit.client.1.smithi117.stdout:8/612: rmdir d1/d3a/d44 39 2024-03-22T06:15:26.664 INFO:tasks.workunit.client.1.smithi117.stdout:6/659: dread d3/d40/d46/d84/d49/d79/fcf [0,4194304] 0 2024-03-22T06:15:26.665 INFO:tasks.workunit.client.1.smithi117.stdout:3/591: truncate d9/d1a/d39/d4c/f4e 2619270 0 2024-03-22T06:15:26.665 INFO:tasks.workunit.client.1.smithi117.stdout:3/592: fsync d9/d1a/f34 0 2024-03-22T06:15:26.665 INFO:tasks.workunit.client.1.smithi117.stdout:3/593: stat d9/f21 0 2024-03-22T06:15:26.668 INFO:tasks.workunit.client.1.smithi117.stdout:1/691: mknod d1/d30/d44/d9c/cf1 0 2024-03-22T06:15:26.670 INFO:tasks.workunit.client.1.smithi117.stdout:2/606: creat d2/d5/d13/d3f/fc8 x:0 0 0 2024-03-22T06:15:26.671 INFO:tasks.workunit.client.1.smithi117.stdout:5/674: mkdir dd/df/d5e/d91/deb 0 2024-03-22T06:15:26.674 INFO:tasks.workunit.client.1.smithi117.stdout:9/675: symlink d4/d8/d93/db1/dd1/ldb 0 2024-03-22T06:15:26.678 INFO:tasks.workunit.client.1.smithi117.stdout:8/613: mknod d1/d3a/d44/cc2 0 2024-03-22T06:15:26.678 INFO:tasks.workunit.client.1.smithi117.stdout:8/614: truncate d1/d3a/d44/d11/d97/f99 4923296 0 2024-03-22T06:15:26.680 INFO:tasks.workunit.client.1.smithi117.stdout:6/660: creat d3/d3d/dcb/dcc/fe4 x:0 0 0 2024-03-22T06:15:26.680 INFO:tasks.workunit.client.1.smithi117.stdout:4/596: sync 2024-03-22T06:15:26.683 INFO:tasks.workunit.client.1.smithi117.stdout:3/594: mkdir d9/d1a/dc1/dd0 0 2024-03-22T06:15:26.684 INFO:tasks.workunit.client.1.smithi117.stdout:3/595: truncate d9/d55/fa0 57569 0 2024-03-22T06:15:26.687 INFO:tasks.workunit.client.1.smithi117.stdout:1/692: symlink d1/d30/d44/dd3/lf2 0 2024-03-22T06:15:26.687 INFO:tasks.workunit.client.1.smithi117.stdout:1/693: write d1/d30/d35/d58/fa1 [31294,83929] 0 2024-03-22T06:15:26.688 INFO:tasks.workunit.client.1.smithi117.stdout:2/607: mkdir d2/d5/d13/d3f/d1d/d3a/d89/d8d/d96/dc9 0 2024-03-22T06:15:26.692 INFO:tasks.workunit.client.1.smithi117.stdout:6/661: write d3/d40/d46/d84/d49/f5b [3535494,92338] 0 2024-03-22T06:15:26.693 INFO:tasks.workunit.client.1.smithi117.stdout:3/596: dread d9/f21 [0,4194304] 0 2024-03-22T06:15:26.693 INFO:tasks.workunit.client.1.smithi117.stdout:3/597: chown d9/d1a/d39/d4c/d36/d41/c85 102314279 1 2024-03-22T06:15:26.694 INFO:tasks.workunit.client.1.smithi117.stdout:5/675: mkdir dd/df/d5e/d91/d72/d60/dec 0 2024-03-22T06:15:26.695 INFO:tasks.workunit.client.1.smithi117.stdout:9/676: mknod d4/d47/d64/d72/cdc 0 2024-03-22T06:15:26.697 INFO:tasks.workunit.client.1.smithi117.stdout:2/608: getdents d2/d5/d13/d3f/d1d/d3a/d89/dc6/d39/d45/d7c 0 2024-03-22T06:15:26.702 INFO:tasks.workunit.client.1.smithi117.stdout:1/694: dread d1/d22/d27/f34 [0,4194304] 0 2024-03-22T06:15:26.702 INFO:tasks.workunit.client.1.smithi117.stdout:1/695: chown d1/d3c/l45 1081778 1 2024-03-22T06:15:26.702 INFO:tasks.workunit.client.1.smithi117.stdout:5/676: mknod dd/d16/d86/ced 0 2024-03-22T06:15:26.704 INFO:tasks.workunit.client.1.smithi117.stdout:8/615: dwrite d1/d19/d98/fbc [0,4194304] 0 2024-03-22T06:15:26.704 INFO:tasks.workunit.client.1.smithi117.stdout:3/598: dread d9/f63 [0,4194304] 0 2024-03-22T06:15:26.705 INFO:tasks.workunit.client.1.smithi117.stdout:9/677: rename d4/d8/d93/db1/dd1/ld2 to d4/d47/d64/dc5/ldd 0 2024-03-22T06:15:26.705 INFO:tasks.workunit.client.1.smithi117.stdout:6/662: read d3/db4/d2d/d53/d59/d74/da/f29 [2864827,116561] 0 2024-03-22T06:15:26.706 INFO:tasks.workunit.client.1.smithi117.stdout:1/696: rmdir d1/d30/d44/dd3 39 2024-03-22T06:15:26.709 INFO:tasks.workunit.client.1.smithi117.stdout:2/609: dread d2/fc [0,4194304] 0 2024-03-22T06:15:26.715 INFO:tasks.workunit.client.1.smithi117.stdout:8/616: dread d1/d28/f31 [0,4194304] 0 2024-03-22T06:15:26.715 INFO:tasks.workunit.client.1.smithi117.stdout:8/617: readlink d1/d3a/d44/d4b/l80 0 2024-03-22T06:15:26.718 INFO:tasks.workunit.client.1.smithi117.stdout:3/599: creat d9/d1a/d39/d4c/d37/dcc/fd1 x:0 0 0 2024-03-22T06:15:26.719 INFO:tasks.workunit.client.1.smithi117.stdout:3/600: chown d9/fa 40141140 1 2024-03-22T06:15:26.719 INFO:tasks.workunit.client.1.smithi117.stdout:3/601: write d9/f19 [2489617,88873] 0 2024-03-22T06:15:26.719 INFO:tasks.workunit.client.1.smithi117.stdout:3/602: write d9/faa [1089766,61954] 0 2024-03-22T06:15:26.719 INFO:tasks.workunit.client.1.smithi117.stdout:3/603: chown d9/f25 29 1 2024-03-22T06:15:26.719 INFO:tasks.workunit.client.1.smithi117.stdout:3/604: write d9/d1a/d39/d82/f84 [2613639,65903] 0 2024-03-22T06:15:26.719 INFO:tasks.workunit.client.1.smithi117.stdout:3/605: write d9/d66/f6d [1403387,58946] 0 2024-03-22T06:15:26.719 INFO:tasks.workunit.client.1.smithi117.stdout:3/606: read - d9/d1a/d39/d4c/d37/fc3 zero size 2024-03-22T06:15:26.719 INFO:tasks.workunit.client.1.smithi117.stdout:3/607: creat d9/d1a/d39/fd2 x:0 0 0 2024-03-22T06:15:26.721 INFO:tasks.workunit.client.1.smithi117.stdout:6/663: write d3/f4 [489615,119602] 0 2024-03-22T06:15:26.724 INFO:tasks.workunit.client.1.smithi117.stdout:2/610: mknod d2/d5/d13/d3f/d1d/d3a/d89/dc6/d39/d45/d88/cca 0 2024-03-22T06:15:26.724 INFO:tasks.workunit.client.1.smithi117.stdout:1/697: dread d1/d1f/d32/d33/fe7 [0,4194304] 0 2024-03-22T06:15:26.726 INFO:tasks.workunit.client.1.smithi117.stdout:7/617: sync 2024-03-22T06:15:26.727 INFO:tasks.workunit.client.1.smithi117.stdout:7/618: chown d8/d9/da/d10/d23/d27/d32 1 1 2024-03-22T06:15:26.727 INFO:tasks.workunit.client.1.smithi117.stdout:7/619: truncate d8/d9/da/d10/d23/d27/d39/d26/f7f 4310673 0 2024-03-22T06:15:26.727 INFO:tasks.workunit.client.1.smithi117.stdout:7/620: chown d8/d9/da/d10/d23/d27/d32/c36 66017996 1 2024-03-22T06:15:26.727 INFO:tasks.workunit.client.1.smithi117.stdout:5/677: dwrite dd/df/d5e/d91/f4f [4194304,4194304] 0 2024-03-22T06:15:26.730 INFO:tasks.workunit.client.1.smithi117.stdout:1/698: dread d1/d22/d27/f8e [0,4194304] 0 2024-03-22T06:15:26.730 INFO:tasks.workunit.client.1.smithi117.stdout:1/699: write d1/d1f/d32/dc6/fd0 [1997106,64574] 0 2024-03-22T06:15:26.731 INFO:tasks.workunit.client.1.smithi117.stdout:8/618: dread d1/d28/d6b/fad [0,4194304] 0 2024-03-22T06:15:26.733 INFO:tasks.workunit.client.1.smithi117.stdout:3/608: rename d9/d1a/d39/d4c/d36/d41/f94 to d9/d1a/d39/d4c/d99/fd3 0 2024-03-22T06:15:26.736 INFO:tasks.workunit.client.1.smithi117.stdout:9/678: rmdir d4/d46 39 2024-03-22T06:15:26.737 INFO:tasks.workunit.client.1.smithi117.stdout:5/678: dread dd/df/d5e/d91/f57 [0,4194304] 0 2024-03-22T06:15:26.740 INFO:tasks.workunit.client.1.smithi117.stdout:2/611: unlink d2/f4a 0 2024-03-22T06:15:26.772 INFO:tasks.workunit.client.1.smithi117.stdout:7/621: getdents d8/d9/da/d10/d23/d27/d39/dac 0 2024-03-22T06:15:26.777 INFO:tasks.workunit.client.1.smithi117.stdout:1/700: mknod d1/d22/d27/cf3 0 2024-03-22T06:15:26.777 INFO:tasks.workunit.client.1.smithi117.stdout:1/701: dread - d1/d30/d44/d99/fb7 zero size 2024-03-22T06:15:26.778 INFO:tasks.workunit.client.1.smithi117.stdout:8/619: creat d1/d3a/d44/d9/fc3 x:0 0 0 2024-03-22T06:15:26.779 INFO:tasks.workunit.client.1.smithi117.stdout:8/620: chown d1/d19/d38/c85 1412974 1 2024-03-22T06:15:26.781 INFO:tasks.workunit.client.1.smithi117.stdout:6/664: rename d3/c15 to d3/db4/d2d/d53/d75/ce5 0 2024-03-22T06:15:26.783 INFO:tasks.workunit.client.1.smithi117.stdout:3/609: readlink d9/d1a/d39/d7d/l91 0 2024-03-22T06:15:26.783 INFO:tasks.workunit.client.1.smithi117.stdout:3/610: chown d9/d1a/f4b 269276807 1 2024-03-22T06:15:26.783 INFO:tasks.workunit.client.1.smithi117.stdout:0/631: sync 2024-03-22T06:15:26.783 INFO:tasks.workunit.client.1.smithi117.stdout:0/632: dread - d1/f87 zero size 2024-03-22T06:15:26.784 INFO:tasks.workunit.client.1.smithi117.stdout:9/679: link d4/d8/d3e/d7a/d3b/l43 d4/d46/da0/lde 0 2024-03-22T06:15:26.784 INFO:tasks.workunit.client.1.smithi117.stdout:9/680: write d4/d8/f25 [2148166,101772] 0 2024-03-22T06:15:26.785 INFO:tasks.workunit.client.1.smithi117.stdout:9/681: chown d4/d7/l62 889243 1 2024-03-22T06:15:26.785 INFO:tasks.workunit.client.1.smithi117.stdout:9/682: fsync d4/d8/d3e/d7a/d2f/fc9 0 2024-03-22T06:15:26.785 INFO:tasks.workunit.client.1.smithi117.stdout:9/683: fdatasync d4/d47/d64/f84 0 2024-03-22T06:15:26.786 INFO:tasks.workunit.client.1.smithi117.stdout:5/679: mkdir dd/d16/d86/dee 0 2024-03-22T06:15:26.789 INFO:tasks.workunit.client.1.smithi117.stdout:2/612: unlink d2/d5/l92 0 2024-03-22T06:15:26.789 INFO:tasks.workunit.client.1.smithi117.stdout:2/613: readlink d2/d5/l94 0 2024-03-22T06:15:26.797 INFO:tasks.workunit.client.1.smithi117.stdout:6/665: dread d3/db4/d2d/d53/fd6 [0,4194304] 0 2024-03-22T06:15:26.797 INFO:tasks.workunit.client.1.smithi117.stdout:9/684: dread d4/d8/d3e/d7a/fd5 [0,4194304] 0 2024-03-22T06:15:26.798 INFO:tasks.workunit.client.1.smithi117.stdout:7/622: link d8/d9/da/d10/l15 d8/d50/d5e/lc3 0 2024-03-22T06:15:26.798 INFO:tasks.workunit.client.1.smithi117.stdout:7/623: fsync d8/d9/d37/d5f/f79 0 2024-03-22T06:15:26.798 INFO:tasks.workunit.client.1.smithi117.stdout:7/624: write d8/d9/da/d10/f8d [628338,33400] 0 2024-03-22T06:15:26.798 INFO:tasks.workunit.client.1.smithi117.stdout:7/625: chown d8/d9/da/d10/l15 271607 1 2024-03-22T06:15:26.803 INFO:tasks.workunit.client.1.smithi117.stdout:3/611: symlink d9/d66/ld4 0 2024-03-22T06:15:26.803 INFO:tasks.workunit.client.1.smithi117.stdout:8/621: truncate d1/d3a/d44/fa 1680231 0 2024-03-22T06:15:26.803 INFO:tasks.workunit.client.1.smithi117.stdout:0/633: symlink d1/d3/d2b/d3e/de/d1f/d59/le4 0 2024-03-22T06:15:26.806 INFO:tasks.workunit.client.1.smithi117.stdout:6/666: mknod d3/db4/d2d/d53/ce6 0 2024-03-22T06:15:26.806 INFO:tasks.workunit.client.1.smithi117.stdout:6/667: chown d3/db4/d2d/l72 1 1 2024-03-22T06:15:26.807 INFO:tasks.workunit.client.1.smithi117.stdout:9/685: truncate d4/d7/f26 3405523 0 2024-03-22T06:15:26.807 INFO:tasks.workunit.client.1.smithi117.stdout:9/686: chown d4/d8/d3e/d7a/d3b/l43 126392692 1 2024-03-22T06:15:26.807 INFO:tasks.workunit.client.1.smithi117.stdout:9/687: fdatasync d4/d8/f41 0 2024-03-22T06:15:26.810 INFO:tasks.workunit.client.1.smithi117.stdout:1/702: getdents d1/d30/d35/d58 0 2024-03-22T06:15:26.813 INFO:tasks.workunit.client.1.smithi117.stdout:8/622: unlink d1/d3a/d44/d16/l82 0 2024-03-22T06:15:26.813 INFO:tasks.workunit.client.1.smithi117.stdout:8/623: creat d1/d28/fc4 x:0 0 0 2024-03-22T06:15:26.813 INFO:tasks.workunit.client.1.smithi117.stdout:8/624: dread - d1/d28/fa8 zero size 2024-03-22T06:15:26.813 INFO:tasks.workunit.client.1.smithi117.stdout:8/625: truncate d1/d3a/d44/d4b/dae/fb8 569886 0 2024-03-22T06:15:26.813 INFO:tasks.workunit.client.1.smithi117.stdout:8/626: dread - d1/fac zero size 2024-03-22T06:15:26.813 INFO:tasks.workunit.client.1.smithi117.stdout:8/627: dread - d1/d28/fa8 zero size 2024-03-22T06:15:26.813 INFO:tasks.workunit.client.1.smithi117.stdout:0/634: dread d1/f4f [0,4194304] 0 2024-03-22T06:15:26.813 INFO:tasks.workunit.client.1.smithi117.stdout:0/635: fsync d1/d3/d2b/d3e/de/f51 0 2024-03-22T06:15:26.814 INFO:tasks.workunit.client.1.smithi117.stdout:6/668: mkdir d3/db4/d2d/d53/d59/d74/d28/ddb/de7 0 2024-03-22T06:15:26.819 INFO:tasks.workunit.client.1.smithi117.stdout:5/680: dwrite dd/df/d5e/f9c [0,4194304] 0 2024-03-22T06:15:26.822 INFO:tasks.workunit.client.1.smithi117.stdout:9/688: read d4/d46/f79 [4144600,58076] 0 2024-03-22T06:15:26.822 INFO:tasks.workunit.client.1.smithi117.stdout:9/689: chown d4/d8/d3e/l75 0 1 2024-03-22T06:15:26.822 INFO:tasks.workunit.client.1.smithi117.stdout:8/628: rmdir d1/d3a/d44/d4b/dae 39 2024-03-22T06:15:26.825 INFO:tasks.workunit.client.1.smithi117.stdout:0/636: mknod d1/d3/d2b/d3e/de/d1f/d59/ce5 0 2024-03-22T06:15:26.826 INFO:tasks.workunit.client.1.smithi117.stdout:6/669: creat d3/d40/fe8 x:0 0 0 2024-03-22T06:15:26.827 INFO:tasks.workunit.client.1.smithi117.stdout:1/703: getdents d1/d1f/d32/dc6 0 2024-03-22T06:15:26.827 INFO:tasks.workunit.client.1.smithi117.stdout:4/597: sync 2024-03-22T06:15:26.827 INFO:tasks.workunit.client.1.smithi117.stdout:4/598: fdatasync d0/f2d 0 2024-03-22T06:15:26.827 INFO:tasks.workunit.client.1.smithi117.stdout:4/599: truncate d0/d51/fa9 247048 0 2024-03-22T06:15:26.827 INFO:tasks.workunit.client.1.smithi117.stdout:4/600: fsync d0/d19/d34/d43/d63/f89 0 2024-03-22T06:15:26.828 INFO:tasks.workunit.client.1.smithi117.stdout:3/612: dwrite f2 [0,4194304] 0 2024-03-22T06:15:26.828 INFO:tasks.workunit.client.1.smithi117.stdout:7/626: dwrite d8/d9/da/f72 [0,4194304] 0 2024-03-22T06:15:26.829 INFO:tasks.workunit.client.1.smithi117.stdout:6/670: dread d3/d3d/d17/d31/f78 [0,4194304] 0 2024-03-22T06:15:26.829 INFO:tasks.workunit.client.1.smithi117.stdout:6/671: truncate d3/db4/d2d/d42/da2/f7a 6413796 0 2024-03-22T06:15:26.829 INFO:tasks.workunit.client.1.smithi117.stdout:6/672: dread - d3/db4/fd2 zero size 2024-03-22T06:15:26.830 INFO:tasks.workunit.client.1.smithi117.stdout:6/673: read - d3/db4/fd2 zero size 2024-03-22T06:15:26.833 INFO:tasks.workunit.client.1.smithi117.stdout:9/690: rename d4/l40 to d4/d8/d93/db1/dd1/dd9/ldf 0 2024-03-22T06:15:26.836 INFO:tasks.workunit.client.1.smithi117.stdout:8/629: symlink d1/d3a/d44/d4b/dae/lc5 0 2024-03-22T06:15:26.838 INFO:tasks.workunit.client.1.smithi117.stdout:1/704: dread d1/f13 [0,4194304] 0 2024-03-22T06:15:26.838 INFO:tasks.workunit.client.1.smithi117.stdout:1/705: chown d1/d3c/fcd 5949 1 2024-03-22T06:15:26.838 INFO:tasks.workunit.client.1.smithi117.stdout:1/706: write d1/d1f/f3e [4622386,51925] 0 2024-03-22T06:15:26.838 INFO:tasks.workunit.client.1.smithi117.stdout:8/630: read d1/d28/d66/d7f/f94 [15396,122351] 0 2024-03-22T06:15:26.838 INFO:tasks.workunit.client.1.smithi117.stdout:8/631: fdatasync d1/d28/d66/d7f/f9e 0 2024-03-22T06:15:26.838 INFO:tasks.workunit.client.1.smithi117.stdout:8/632: write d1/d3a/d44/d4b/fbf [567332,68578] 0 2024-03-22T06:15:26.838 INFO:tasks.workunit.client.1.smithi117.stdout:0/637: symlink d1/d3/dae/dbe/le6 0 2024-03-22T06:15:26.838 INFO:tasks.workunit.client.1.smithi117.stdout:0/638: chown d1/d3/d2b/d3e/de/d1f/d59/ce5 5073 1 2024-03-22T06:15:26.838 INFO:tasks.workunit.client.1.smithi117.stdout:7/627: dread d8/d9/da/d10/d23/d27/d32/f81 [0,4194304] 0 2024-03-22T06:15:26.838 INFO:tasks.workunit.client.1.smithi117.stdout:0/639: creat d1/d10/d5c/d40/d4e/dbc/fe7 x:0 0 0 2024-03-22T06:15:26.839 INFO:tasks.workunit.client.1.smithi117.stdout:8/633: dread d1/d19/d38/f8c [0,4194304] 0 2024-03-22T06:15:26.839 INFO:tasks.workunit.client.1.smithi117.stdout:8/634: chown d1/d3a/d44/d4b/c47 989 1 2024-03-22T06:15:26.839 INFO:tasks.workunit.client.1.smithi117.stdout:8/635: stat d1/d28/d66/c88 0 2024-03-22T06:15:26.844 INFO:tasks.workunit.client.1.smithi117.stdout:3/613: mkdir d9/d1a/d39/d4c/d37/dd5 0 2024-03-22T06:15:26.844 INFO:tasks.workunit.client.1.smithi117.stdout:3/614: truncate d9/faa 1312755 0 2024-03-22T06:15:26.844 INFO:tasks.workunit.client.1.smithi117.stdout:3/615: chown d9/c5f 439721610 1 2024-03-22T06:15:26.851 INFO:tasks.workunit.client.1.smithi117.stdout:6/674: unlink d3/db4/d2d/d53/d59/c96 0 2024-03-22T06:15:26.851 INFO:tasks.workunit.client.1.smithi117.stdout:6/675: write d3/d40/d46/d84/d49/d79/d9d/fce [228597,65937] 0 2024-03-22T06:15:26.857 INFO:tasks.workunit.client.1.smithi117.stdout:9/691: truncate d4/d8/f67 281999 0 2024-03-22T06:15:26.861 INFO:tasks.workunit.client.1.smithi117.stdout:6/676: dread d3/d40/f22 [0,4194304] 0 2024-03-22T06:15:26.862 INFO:tasks.workunit.client.1.smithi117.stdout:6/677: write d3/d3d/d17/d60/fd0 [521793,29557] 0 2024-03-22T06:15:26.862 INFO:tasks.workunit.client.1.smithi117.stdout:6/678: write d3/db4/d2d/d42/da2/f2f [2464326,120943] 0 2024-03-22T06:15:26.863 INFO:tasks.workunit.client.1.smithi117.stdout:1/707: mknod d1/d3c/db6/cf4 0 2024-03-22T06:15:26.873 INFO:tasks.workunit.client.1.smithi117.stdout:7/628: rename d8/d9/da/d10/d23/d27/d39 to d8/d9/da/d2b/dc4 0 2024-03-22T06:15:26.873 INFO:tasks.workunit.client.1.smithi117.stdout:7/629: creat d8/d9/da/d10/d23/d27/d32/fc5 x:0 0 0 2024-03-22T06:15:26.873 INFO:tasks.workunit.client.1.smithi117.stdout:7/630: write d8/d9/da/d10/d23/f38 [3070883,57282] 0 2024-03-22T06:15:26.876 INFO:tasks.workunit.client.1.smithi117.stdout:0/640: mkdir d1/d10/d18/d26/d49/d7e/d39/dc5/de8 0 2024-03-22T06:15:26.878 INFO:tasks.workunit.client.1.smithi117.stdout:0/641: dread d1/f4f [0,4194304] 0 2024-03-22T06:15:26.878 INFO:tasks.workunit.client.1.smithi117.stdout:8/636: mknod d1/d3a/d44/d11/d97/cc6 0 2024-03-22T06:15:26.879 INFO:tasks.workunit.client.1.smithi117.stdout:8/637: dread d1/d3a/d44/d4b/dae/fb8 [0,4194304] 0 2024-03-22T06:15:26.883 INFO:tasks.workunit.client.1.smithi117.stdout:3/616: creat d9/d1a/d39/d4c/d37/dd5/fd6 x:0 0 0 2024-03-22T06:15:26.888 INFO:tasks.workunit.client.1.smithi117.stdout:1/708: rename d1/d1f/d32/fdb to d1/d84/d77/ff5 0 2024-03-22T06:15:26.890 INFO:tasks.workunit.client.1.smithi117.stdout:9/692: dwrite d4/d47/d64/fb4 [0,4194304] 0 2024-03-22T06:15:26.890 INFO:tasks.workunit.client.1.smithi117.stdout:9/693: creat d4/d8/d3e/d7a/d2f/fe0 x:0 0 0 2024-03-22T06:15:26.892 INFO:tasks.workunit.client.1.smithi117.stdout:0/642: dread d1/d10/f6c [0,4194304] 0 2024-03-22T06:15:26.892 INFO:tasks.workunit.client.1.smithi117.stdout:1/709: write d1/f13 [4458325,39553] 0 2024-03-22T06:15:26.893 INFO:tasks.workunit.client.1.smithi117.stdout:7/631: rmdir d8/d9/d37/d45/d61 39 2024-03-22T06:15:26.893 INFO:tasks.workunit.client.1.smithi117.stdout:7/632: dread - d8/d9/d37/d5f/d82/f95 zero size 2024-03-22T06:15:26.893 INFO:tasks.workunit.client.1.smithi117.stdout:7/633: truncate d8/d6c/fc2 721473 0 2024-03-22T06:15:26.893 INFO:tasks.workunit.client.1.smithi117.stdout:6/679: dwrite d3/d3d/dcb/dcc/fe4 [0,4194304] 0 2024-03-22T06:15:26.893 INFO:tasks.workunit.client.1.smithi117.stdout:3/617: dread d9/d1a/f83 [0,4194304] 0 2024-03-22T06:15:26.893 INFO:tasks.workunit.client.1.smithi117.stdout:3/618: chown d9/f63 272453850 1 2024-03-22T06:15:26.895 INFO:tasks.workunit.client.1.smithi117.stdout:4/601: rmdir d0/d19/d28/da3 0 2024-03-22T06:15:26.895 INFO:tasks.workunit.client.1.smithi117.stdout:4/602: write d0/d4/d2a/fb0 [895728,80053] 0 2024-03-22T06:15:26.895 INFO:tasks.workunit.client.1.smithi117.stdout:4/603: truncate d0/d19/d34/d43/d63/d6f/d6a/f8c 927341 0 2024-03-22T06:15:26.895 INFO:tasks.workunit.client.1.smithi117.stdout:4/604: chown d0/d19/d34/d43/d63/d8d/f90 2053 1 2024-03-22T06:15:26.895 INFO:tasks.workunit.client.1.smithi117.stdout:4/605: dread - d0/d19/d34/d43/d63/d6f/d5f/f75 zero size 2024-03-22T06:15:26.896 INFO:tasks.workunit.client.1.smithi117.stdout:8/638: dread d1/d19/f51 [8388608,4194304] 0 2024-03-22T06:15:26.901 INFO:tasks.workunit.client.1.smithi117.stdout:0/643: write d1/d3/d2b/d3e/f60 [2015251,91218] 0 2024-03-22T06:15:26.902 INFO:tasks.workunit.client.1.smithi117.stdout:9/694: read d4/d8/f1a [224985,7727] 0 2024-03-22T06:15:26.902 INFO:tasks.workunit.client.1.smithi117.stdout:8/639: dread d1/d3a/d44/d4b/dae/fb8 [0,4194304] 0 2024-03-22T06:15:26.909 INFO:tasks.workunit.client.1.smithi117.stdout:1/710: dwrite d1/d30/d50/fea [0,4194304] 0 2024-03-22T06:15:26.912 INFO:tasks.workunit.client.1.smithi117.stdout:7/634: mkdir d8/d9/d37/d45/d88/db1/dc6 0 2024-03-22T06:15:26.914 INFO:tasks.workunit.client.1.smithi117.stdout:6/680: link d3/l1d d3/d40/d46/le9 0 2024-03-22T06:15:26.914 INFO:tasks.workunit.client.1.smithi117.stdout:6/681: truncate d3/db4/d2d/d53/d59/d74/d28/d85/fba 682500 0 2024-03-22T06:15:26.914 INFO:tasks.workunit.client.1.smithi117.stdout:6/682: creat d3/d3d/dcb/dcc/fea x:0 0 0 2024-03-22T06:15:26.930 INFO:tasks.workunit.client.1.smithi117.stdout:3/619: unlink d9/f70 0 2024-03-22T06:15:26.940 INFO:tasks.workunit.client.1.smithi117.stdout:9/695: link d4/d7/f26 d4/d8/d93/db1/fe1 0 2024-03-22T06:15:26.940 INFO:tasks.workunit.client.1.smithi117.stdout:9/696: creat d4/d50/fe2 x:0 0 0 2024-03-22T06:15:26.940 INFO:tasks.workunit.client.1.smithi117.stdout:9/697: dread - d4/d46/fa4 zero size 2024-03-22T06:15:26.940 INFO:tasks.workunit.client.1.smithi117.stdout:9/698: write d4/d47/d85/fb8 [892093,121287] 0 2024-03-22T06:15:26.943 INFO:tasks.workunit.client.1.smithi117.stdout:8/640: unlink d1/d19/c4a 0 2024-03-22T06:15:26.946 INFO:tasks.workunit.client.1.smithi117.stdout:2/614: sync 2024-03-22T06:15:26.946 INFO:tasks.workunit.client.1.smithi117.stdout:2/615: truncate d2/d5/d13/d3f/d1d/d3a/d89/fab 64635 0 2024-03-22T06:15:26.959 INFO:tasks.workunit.client.1.smithi117.stdout:7/635: rename d8/d9/da/d2b/db8/lbc to d8/d9/da/d2b/dc4/dac/lc7 0 2024-03-22T06:15:26.964 INFO:tasks.workunit.client.1.smithi117.stdout:2/616: dwrite d2/d5/d13/d3f/d1d/f6d [0,4194304] 0 2024-03-22T06:15:26.965 INFO:tasks.workunit.client.1.smithi117.stdout:2/617: write d2/d5/d13/d3f/f6b [592521,11015] 0 2024-03-22T06:15:26.965 INFO:tasks.workunit.client.1.smithi117.stdout:6/683: creat d3/db4/d2d/d42/da2/feb x:0 0 0 2024-03-22T06:15:26.966 INFO:tasks.workunit.client.1.smithi117.stdout:3/620: mknod d9/d93/cd7 0 2024-03-22T06:15:26.988 INFO:tasks.workunit.client.1.smithi117.stdout:4/606: rmdir d0 39 2024-03-22T06:15:26.989 INFO:tasks.workunit.client.1.smithi117.stdout:4/607: fsync d0/d4/d2a/d2f/fd0 0 2024-03-22T06:15:26.998 INFO:tasks.workunit.client.1.smithi117.stdout:8/641: mkdir d1/d3a/d44/d11/dc7 0 2024-03-22T06:15:27.002 INFO:tasks.workunit.client.1.smithi117.stdout:1/711: mkdir d1/d22/d27/d4e/df6 0 2024-03-22T06:15:27.005 INFO:tasks.workunit.client.1.smithi117.stdout:9/699: rename d4/d8/c33 to d4/d8/d93/db1/dd1/dd9/ce3 0 2024-03-22T06:15:27.005 INFO:tasks.workunit.client.1.smithi117.stdout:6/684: rename d3/db4/d2d/d53 to d3/db4/d2d/d53/d59/d74/dec 22 2024-03-22T06:15:27.005 INFO:tasks.workunit.client.1.smithi117.stdout:6/685: read - d3/d3d/f8d zero size 2024-03-22T06:15:27.006 INFO:tasks.workunit.client.1.smithi117.stdout:7/636: mkdir d8/d9/da/d10/d23/d48/dc8 0 2024-03-22T06:15:27.009 INFO:tasks.workunit.client.1.smithi117.stdout:2/618: mknod d2/d5/d13/d3f/d1d/d3a/ccb 0 2024-03-22T06:15:27.010 INFO:tasks.workunit.client.1.smithi117.stdout:3/621: fdatasync d9/d1a/d39/d7d/f7e 0 2024-03-22T06:15:27.017 INFO:tasks.workunit.client.1.smithi117.stdout:4/608: mkdir d0/dd1 0 2024-03-22T06:15:27.018 INFO:tasks.workunit.client.1.smithi117.stdout:8/642: symlink d1/d3a/d44/d6a/db7/lc8 0 2024-03-22T06:15:27.020 INFO:tasks.workunit.client.1.smithi117.stdout:1/712: mkdir d1/d30/d44/d99/df7 0 2024-03-22T06:15:27.020 INFO:tasks.workunit.client.1.smithi117.stdout:1/713: write d1/d3c/d4d/de8/fbe [1036766,121243] 0 2024-03-22T06:15:27.021 INFO:tasks.workunit.client.1.smithi117.stdout:9/700: link d4/d8/d3e/d7a/d2f/fc9 d4/d8/d3e/d7a/d3b/fe4 0 2024-03-22T06:15:27.024 INFO:tasks.workunit.client.1.smithi117.stdout:6/686: creat d3/d40/d46/d84/fed x:0 0 0 2024-03-22T06:15:27.026 INFO:tasks.workunit.client.1.smithi117.stdout:7/637: symlink d8/d9/db0/lc9 0 2024-03-22T06:15:27.028 INFO:tasks.workunit.client.1.smithi117.stdout:0/644: sync 2024-03-22T06:15:27.028 INFO:tasks.workunit.client.1.smithi117.stdout:5/681: sync 2024-03-22T06:15:27.028 INFO:tasks.workunit.client.1.smithi117.stdout:5/682: chown dd/d16/d1a/d32 105590 1 2024-03-22T06:15:27.032 INFO:tasks.workunit.client.1.smithi117.stdout:2/619: creat d2/d5/d41/d83/fcc x:0 0 0 2024-03-22T06:15:27.032 INFO:tasks.workunit.client.1.smithi117.stdout:3/622: mknod d9/d1a/d39/d4c/d99/cd8 0 2024-03-22T06:15:27.033 INFO:tasks.workunit.client.1.smithi117.stdout:2/620: chown d2/d5/d13/d3f/d1d/d3a/d89/d8d/cb3 82227 1 2024-03-22T06:15:27.037 INFO:tasks.workunit.client.1.smithi117.stdout:4/609: rename d0/d19/d34/d43/d63/d6f/db1 to d0/d48/da0/dd2 0 2024-03-22T06:15:27.037 INFO:tasks.workunit.client.1.smithi117.stdout:4/610: fsync d0/d19/d34/d43/d63/d6f/d5f/f9a 0 2024-03-22T06:15:27.038 INFO:tasks.workunit.client.1.smithi117.stdout:8/643: mkdir d1/d3a/d44/d4b/dc9 0 2024-03-22T06:15:27.040 INFO:tasks.workunit.client.1.smithi117.stdout:1/714: link d1/d30/d44/f7b d1/d22/ff8 0 2024-03-22T06:15:27.041 INFO:tasks.workunit.client.1.smithi117.stdout:9/701: rmdir d4/d8/d93/db1 39 2024-03-22T06:15:27.044 INFO:tasks.workunit.client.1.smithi117.stdout:0/645: link d1/d10/d18/c2a d1/d3/d2b/d3e/de/d1f/d3b/d55/ce9 0 2024-03-22T06:15:27.044 INFO:tasks.workunit.client.1.smithi117.stdout:0/646: dread - d1/d10/d18/d26/d49/fab zero size 2024-03-22T06:15:27.044 INFO:tasks.workunit.client.1.smithi117.stdout:0/647: dread - d1/d10/d18/d8c/fc7 zero size 2024-03-22T06:15:27.045 INFO:tasks.workunit.client.1.smithi117.stdout:3/623: stat d9/d1a/d39/d4c/d37/c51 0 2024-03-22T06:15:27.045 INFO:tasks.workunit.client.1.smithi117.stdout:3/624: fsync d9/f2a 0 2024-03-22T06:15:27.047 INFO:tasks.workunit.client.1.smithi117.stdout:2/621: symlink d2/d35/lcd 0 2024-03-22T06:15:27.049 INFO:tasks.workunit.client.1.smithi117.stdout:4/611: mknod d0/d4/d2a/cd3 0 2024-03-22T06:15:27.051 INFO:tasks.workunit.client.1.smithi117.stdout:8/644: mkdir d1/d3a/d44/d4b/dca 0 2024-03-22T06:15:27.051 INFO:tasks.workunit.client.1.smithi117.stdout:8/645: truncate d1/d28/d6b/da3/fbb 473785 0 2024-03-22T06:15:27.051 INFO:tasks.workunit.client.1.smithi117.stdout:8/646: readlink d1/d3a/d44/l4 0 2024-03-22T06:15:27.051 INFO:tasks.workunit.client.1.smithi117.stdout:8/647: chown d1/d19/d38/f9f 23952005 1 2024-03-22T06:15:27.051 INFO:tasks.workunit.client.1.smithi117.stdout:8/648: chown d1/d19/d98/fbc 15169539 1 2024-03-22T06:15:27.053 INFO:tasks.workunit.client.1.smithi117.stdout:6/687: dwrite d3/db4/d2d/d42/fca [0,4194304] 0 2024-03-22T06:15:27.053 INFO:tasks.workunit.client.1.smithi117.stdout:6/688: truncate d3/db4/d2d/d42/fca 5053974 0 2024-03-22T06:15:27.053 INFO:tasks.workunit.client.1.smithi117.stdout:1/715: creat d1/d30/d44/d9c/da9/de3/ff9 x:0 0 0 2024-03-22T06:15:27.057 INFO:tasks.workunit.client.1.smithi117.stdout:9/702: rename d4/d8/d3e/d7a/d2f/f9f to d4/d8/d93/db1/dd1/dd6/d58/fe5 0 2024-03-22T06:15:27.057 INFO:tasks.workunit.client.1.smithi117.stdout:7/638: dwrite d8/d9/da/f69 [0,4194304] 0 2024-03-22T06:15:27.059 INFO:tasks.workunit.client.1.smithi117.stdout:0/648: truncate d1/d10/d5c/d40/d69/d72/f76 274694 0 2024-03-22T06:15:27.059 INFO:tasks.workunit.client.1.smithi117.stdout:0/649: truncate d1/d3/d2b/d3e/d3f/f78 1107748 0 2024-03-22T06:15:27.061 INFO:tasks.workunit.client.1.smithi117.stdout:5/683: dwrite dd/d16/d1a/d32/d73/db8/fb9 [0,4194304] 0 2024-03-22T06:15:27.061 INFO:tasks.workunit.client.1.smithi117.stdout:5/684: fsync dd/df/d5e/d91/d72/d33/d9a/fc3 0 2024-03-22T06:15:27.061 INFO:tasks.workunit.client.1.smithi117.stdout:5/685: dread - dd/df/d5e/d91/d72/d35/f9f zero size 2024-03-22T06:15:27.061 INFO:tasks.workunit.client.1.smithi117.stdout:5/686: chown dd/df/d5e/d91/d90/dca 72133722 1 2024-03-22T06:15:27.062 INFO:tasks.workunit.client.1.smithi117.stdout:3/625: symlink d9/d1a/dc1/dd0/ld9 0 2024-03-22T06:15:27.062 INFO:tasks.workunit.client.1.smithi117.stdout:3/626: write d9/d1a/d39/d8a/fca [1022680,97564] 0 2024-03-22T06:15:27.062 INFO:tasks.workunit.client.1.smithi117.stdout:3/627: stat f2 0 2024-03-22T06:15:27.064 INFO:tasks.workunit.client.1.smithi117.stdout:4/612: unlink d0/c21 0 2024-03-22T06:15:27.065 INFO:tasks.workunit.client.1.smithi117.stdout:3/628: write d9/d66/f6d [727521,16444] 0 2024-03-22T06:15:27.066 INFO:tasks.workunit.client.1.smithi117.stdout:8/649: stat d1/d28/d66/cb0 0 2024-03-22T06:15:27.067 INFO:tasks.workunit.client.1.smithi117.stdout:9/703: write d4/d8/d3e/d7a/f42 [2973650,94215] 0 2024-03-22T06:15:27.072 INFO:tasks.workunit.client.1.smithi117.stdout:6/689: rename d3/d40/fa9 to d3/d61/d73/fee 0 2024-03-22T06:15:27.072 INFO:tasks.workunit.client.1.smithi117.stdout:6/690: creat d3/db4/d2d/d53/fef x:0 0 0 2024-03-22T06:15:27.072 INFO:tasks.workunit.client.1.smithi117.stdout:6/691: dread - d3/d40/d46/d84/fed zero size 2024-03-22T06:15:27.073 INFO:tasks.workunit.client.1.smithi117.stdout:6/692: chown d3/db4/d2d/d53/d59/d74/l18 270669 1 2024-03-22T06:15:27.075 INFO:tasks.workunit.client.1.smithi117.stdout:8/650: dread d1/d3a/d44/f3c [0,4194304] 0 2024-03-22T06:15:27.082 INFO:tasks.workunit.client.1.smithi117.stdout:7/639: mknod d8/d9/d37/cca 0 2024-03-22T06:15:27.082 INFO:tasks.workunit.client.1.smithi117.stdout:7/640: write d8/d9/da/d10/d23/d27/d56/f9e [1853269,66903] 0 2024-03-22T06:15:27.087 INFO:tasks.workunit.client.1.smithi117.stdout:0/650: rmdir d1/d10/d18/d26 39 2024-03-22T06:15:27.091 INFO:tasks.workunit.client.1.smithi117.stdout:1/716: dwrite d1/d30/d44/f7b [0,4194304] 0 2024-03-22T06:15:27.091 INFO:tasks.workunit.client.1.smithi117.stdout:5/687: dwrite dd/d16/d1a/fe9 [0,4194304] 0 2024-03-22T06:15:27.092 INFO:tasks.workunit.client.1.smithi117.stdout:2/622: dwrite d2/d5/d13/d3f/d1d/f8f [0,4194304] 0 2024-03-22T06:15:27.096 INFO:tasks.workunit.client.1.smithi117.stdout:7/641: dread d8/d9/da/d10/d23/d41/f75 [0,4194304] 0 2024-03-22T06:15:27.096 INFO:tasks.workunit.client.1.smithi117.stdout:3/629: dwrite d9/f18 [0,4194304] 0 2024-03-22T06:15:27.096 INFO:tasks.workunit.client.1.smithi117.stdout:3/630: fdatasync d9/faa 0 2024-03-22T06:15:27.097 INFO:tasks.workunit.client.1.smithi117.stdout:4/613: creat d0/d19/dcc/fd4 x:0 0 0 2024-03-22T06:15:27.097 INFO:tasks.workunit.client.1.smithi117.stdout:4/614: truncate d0/dc0/fc1 428249 0 2024-03-22T06:15:27.102 INFO:tasks.workunit.client.1.smithi117.stdout:6/693: stat d3/d3d/cc8 0 2024-03-22T06:15:27.103 INFO:tasks.workunit.client.1.smithi117.stdout:6/694: fdatasync d3/db4/fc5 0 2024-03-22T06:15:27.103 INFO:tasks.workunit.client.1.smithi117.stdout:6/695: chown d3/db4/d2d/d53/d59/d74/da/c26 504 1 2024-03-22T06:15:27.103 INFO:tasks.workunit.client.1.smithi117.stdout:5/688: dread dd/d16/d1a/f59 [0,4194304] 0 2024-03-22T06:15:27.104 INFO:tasks.workunit.client.1.smithi117.stdout:7/642: dread d8/d9/da/d10/d23/d27/d32/f54 [0,4194304] 0 2024-03-22T06:15:27.104 INFO:tasks.workunit.client.1.smithi117.stdout:7/643: chown d8/d9/da/d2b/dc4/f5d 5124 1 2024-03-22T06:15:27.105 INFO:tasks.workunit.client.1.smithi117.stdout:8/651: rmdir d1/d3a/d44/d4b/dca 0 2024-03-22T06:15:27.105 INFO:tasks.workunit.client.1.smithi117.stdout:8/652: creat d1/d3a/fcb x:0 0 0 2024-03-22T06:15:27.114 INFO:tasks.workunit.client.1.smithi117.stdout:1/717: mknod d1/d62/d7c/cfa 0 2024-03-22T06:15:27.115 INFO:tasks.workunit.client.1.smithi117.stdout:9/704: dwrite d4/d8/d3e/d7a/f42 [0,4194304] 0 2024-03-22T06:15:27.115 INFO:tasks.workunit.client.1.smithi117.stdout:0/651: dwrite d1/d10/d18/d26/d53/f93 [0,4194304] 0 2024-03-22T06:15:27.115 INFO:tasks.workunit.client.1.smithi117.stdout:0/652: stat d1/d10/d5c/d40/dbb/ldc 0 2024-03-22T06:15:27.119 INFO:tasks.workunit.client.1.smithi117.stdout:2/623: mknod d2/d5/d13/d3f/d1d/cce 0 2024-03-22T06:15:27.120 INFO:tasks.workunit.client.1.smithi117.stdout:8/653: write d1/d3a/d44/f8d [2518155,111637] 0 2024-03-22T06:15:27.120 INFO:tasks.workunit.client.1.smithi117.stdout:8/654: write d1/d28/fbe [1237897,8630] 0 2024-03-22T06:15:27.128 INFO:tasks.workunit.client.1.smithi117.stdout:7/644: dwrite d8/d9/da/d10/d23/d27/d32/f4d [0,4194304] 0 2024-03-22T06:15:27.133 INFO:tasks.workunit.client.1.smithi117.stdout:3/631: rename d9/d1a/l1c to d9/d1a/d39/d4c/lda 0 2024-03-22T06:15:27.133 INFO:tasks.workunit.client.1.smithi117.stdout:3/632: write d9/d1a/d39/f71 [138224,76940] 0 2024-03-22T06:15:27.136 INFO:tasks.workunit.client.1.smithi117.stdout:4/615: creat d0/d19/d1c/fd5 x:0 0 0 2024-03-22T06:15:27.138 INFO:tasks.workunit.client.1.smithi117.stdout:6/696: unlink d3/db4/d2d/d53/f70 0 2024-03-22T06:15:27.138 INFO:tasks.workunit.client.1.smithi117.stdout:6/697: chown d3/d61/d73/fee 16 1 2024-03-22T06:15:27.149 INFO:tasks.workunit.client.1.smithi117.stdout:9/705: symlink d4/d47/d64/dc5/dda/le6 0 2024-03-22T06:15:27.153 INFO:tasks.workunit.client.1.smithi117.stdout:6/698: dread d3/db4/d2d/d42/fca [0,4194304] 0 2024-03-22T06:15:27.153 INFO:tasks.workunit.client.1.smithi117.stdout:1/718: dwrite d1/d30/d5d/d6f/f95 [4194304,4194304] 0 2024-03-22T06:15:27.153 INFO:tasks.workunit.client.1.smithi117.stdout:6/699: chown d3/c2a 2133654 1 2024-03-22T06:15:27.153 INFO:tasks.workunit.client.1.smithi117.stdout:6/700: fsync d3/db4/d2d/d42/da2/f7a 0 2024-03-22T06:15:27.154 INFO:tasks.workunit.client.1.smithi117.stdout:2/624: dwrite d2/f26 [0,4194304] 0 2024-03-22T06:15:27.179 INFO:tasks.workunit.client.1.smithi117.stdout:8/655: symlink d1/d19/lcc 0 2024-03-22T06:15:27.181 INFO:tasks.workunit.client.1.smithi117.stdout:8/656: dread d1/d19/f86 [0,4194304] 0 2024-03-22T06:15:27.193 INFO:tasks.workunit.client.1.smithi117.stdout:7/645: unlink d8/d9/da/d2b/c31 0 2024-03-22T06:15:27.194 INFO:tasks.workunit.client.1.smithi117.stdout:3/633: link d9/d66/fa9 d9/d1a/dc1/dd0/fdb 0 2024-03-22T06:15:27.195 INFO:tasks.workunit.client.1.smithi117.stdout:3/634: write d9/d1a/d39/d4c/d36/f75 [369637,51810] 0 2024-03-22T06:15:27.195 INFO:tasks.workunit.client.1.smithi117.stdout:3/635: creat d9/d93/fdc x:0 0 0 2024-03-22T06:15:27.196 INFO:tasks.workunit.client.1.smithi117.stdout:3/636: readlink d9/d1a/d39/d4c/d37/d69/l80 0 2024-03-22T06:15:27.196 INFO:tasks.workunit.client.1.smithi117.stdout:3/637: write d9/d1a/d39/f96 [878447,113389] 0 2024-03-22T06:15:27.198 INFO:tasks.workunit.client.1.smithi117.stdout:4/616: rmdir d0/d4/d2a/d2f/db8 0 2024-03-22T06:15:27.200 INFO:tasks.workunit.client.1.smithi117.stdout:9/706: rename d4/db9 to d4/d7/d27/de7 0 2024-03-22T06:15:27.204 INFO:tasks.workunit.client.1.smithi117.stdout:1/719: mknod d1/d22/d27/d4e/d92/cfb 0 2024-03-22T06:15:27.208 INFO:tasks.workunit.client.1.smithi117.stdout:6/701: creat d3/db4/d2d/d53/d59/d74/da/de/ff0 x:0 0 0 2024-03-22T06:15:27.219 INFO:tasks.workunit.client.1.smithi117.stdout:6/702: dread d3/d40/d46/d84/f7b [0,4194304] 0 2024-03-22T06:15:27.223 INFO:tasks.workunit.client.1.smithi117.stdout:8/657: symlink d1/d28/d66/d7f/d9d/lcd 0 2024-03-22T06:15:27.224 INFO:tasks.workunit.client.1.smithi117.stdout:8/658: chown d1/f1e 205835923 1 2024-03-22T06:15:27.224 INFO:tasks.workunit.client.1.smithi117.stdout:8/659: dread - d1/d3a/d44/d6a/f9b zero size 2024-03-22T06:15:27.229 INFO:tasks.workunit.client.1.smithi117.stdout:7/646: truncate d8/d9/da/f72 2224762 0 2024-03-22T06:15:27.236 INFO:tasks.workunit.client.1.smithi117.stdout:3/638: mkdir d9/d1a/d39/d7d/d97/dc9/ddd 0 2024-03-22T06:15:27.236 INFO:tasks.workunit.client.1.smithi117.stdout:3/639: write d9/d93/fdc [631575,20535] 0 2024-03-22T06:15:27.236 INFO:tasks.workunit.client.1.smithi117.stdout:3/640: chown d9/d1a/d39/d4c/d37/fc3 0 1 2024-03-22T06:15:27.242 INFO:tasks.workunit.client.1.smithi117.stdout:0/653: sync 2024-03-22T06:15:27.242 INFO:tasks.workunit.client.1.smithi117.stdout:5/689: sync 2024-03-22T06:15:27.242 INFO:tasks.workunit.client.1.smithi117.stdout:0/654: truncate d1/d3/d2b/d3e/de/d1f/d3b/f46 1501923 0 2024-03-22T06:15:27.243 INFO:tasks.workunit.client.1.smithi117.stdout:4/617: mkdir d0/d19/d28/d47/dd6 0 2024-03-22T06:15:27.250 INFO:tasks.workunit.client.1.smithi117.stdout:2/625: rename d2/d5/db/f24 to d2/d35/d4c/fcf 0 2024-03-22T06:15:27.253 INFO:tasks.workunit.client.1.smithi117.stdout:6/703: mknod d3/d40/d46/d84/dbe/cf1 0 2024-03-22T06:15:27.253 INFO:tasks.workunit.client.1.smithi117.stdout:6/704: fsync d3/db4/d2d/d53/d75/fc1 0 2024-03-22T06:15:27.253 INFO:tasks.workunit.client.1.smithi117.stdout:6/705: creat d3/db4/d2d/d53/ff2 x:0 0 0 2024-03-22T06:15:27.253 INFO:tasks.workunit.client.1.smithi117.stdout:6/706: write d3/f4 [5217401,116212] 0 2024-03-22T06:15:27.253 INFO:tasks.workunit.client.1.smithi117.stdout:6/707: chown d3/db4/d2d/d53/d59/d74/da/de/f19 2889899 1 2024-03-22T06:15:27.256 INFO:tasks.workunit.client.1.smithi117.stdout:3/641: dwrite f2 [0,4194304] 0 2024-03-22T06:15:27.257 INFO:tasks.workunit.client.1.smithi117.stdout:3/642: chown d9/d1a/d39/d4c/d36/cc0 28 1 2024-03-22T06:15:27.257 INFO:tasks.workunit.client.1.smithi117.stdout:3/643: chown d9/d1a/d39/d4c/l6a 3578604 1 2024-03-22T06:15:27.258 INFO:tasks.workunit.client.1.smithi117.stdout:8/660: unlink d1/d3a/d44/d4b/fbf 0 2024-03-22T06:15:27.258 INFO:tasks.workunit.client.1.smithi117.stdout:8/661: write d1/d3a/d44/d9/f81 [309723,98223] 0 2024-03-22T06:15:27.258 INFO:tasks.workunit.client.1.smithi117.stdout:8/662: dread - d1/d28/fc4 zero size 2024-03-22T06:15:27.258 INFO:tasks.workunit.client.1.smithi117.stdout:5/690: dwrite dd/d16/fe3 [0,4194304] 0 2024-03-22T06:15:27.261 INFO:tasks.workunit.client.1.smithi117.stdout:0/655: truncate d1/d10/d18/f7b 581714 0 2024-03-22T06:15:27.264 INFO:tasks.workunit.client.1.smithi117.stdout:4/618: unlink d0/d4/d2a/f5d 0 2024-03-22T06:15:27.305 INFO:tasks.workunit.client.1.smithi117.stdout:9/707: rename d4/d8/f41 to d4/d47/d64/fe8 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:2/626: dwrite d2/d5/d13/d3f/d1d/d3a/d89/daa/f15 [0,4194304] 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:4/619: dread d0/d19/d28/f2e [0,4194304] 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:6/708: truncate d3/d3d/d17/d31/f87 1966512 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:0/656: dwrite d1/f3d [0,4194304] 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:5/691: creat dd/df/dae/dcd/fef x:0 0 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:5/692: write dd/f7d [2746941,116892] 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:2/627: dwrite d2/d35/d4c/fcf [0,4194304] 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:7/647: getdents d8/d9/d37/d45/d88 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:1/720: rename d1/d62/ca7 to d1/d30/d44/d99/dd7/cfc 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:1/721: write d1/d84/fed [503878,121078] 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:2/628: dwrite d2/d5/d13/d3f/d1d/d3a/d89/daa/d4f/f62 [0,4194304] 0 2024-03-22T06:15:27.306 INFO:tasks.workunit.client.1.smithi117.stdout:1/722: dread d1/d1f/d32/d33/fe7 [0,4194304] 0 2024-03-22T06:15:27.307 INFO:tasks.workunit.client.1.smithi117.stdout:9/708: creat d4/d8/d93/db1/dd1/fe9 x:0 0 0 2024-03-22T06:15:27.309 INFO:tasks.workunit.client.1.smithi117.stdout:4/620: symlink d0/ld7 0 2024-03-22T06:15:27.311 INFO:tasks.workunit.client.1.smithi117.stdout:6/709: creat d3/db4/d2d/d53/d59/d74/ff3 x:0 0 0 2024-03-22T06:15:27.311 INFO:tasks.workunit.client.1.smithi117.stdout:6/710: fsync d3/d40/d46/f6b 0 2024-03-22T06:15:27.314 INFO:tasks.workunit.client.1.smithi117.stdout:0/657: rmdir d1/d10/d5c/d40/d69/d72 39 2024-03-22T06:15:27.322 INFO:tasks.workunit.client.1.smithi117.stdout:7/648: creat d8/d9/da/d2b/d9b/fcb x:0 0 0 2024-03-22T06:15:27.326 INFO:tasks.workunit.client.1.smithi117.stdout:3/644: rename d9 to d9/d1a/d39/d4c/d36/d41/dde 22 2024-03-22T06:15:27.342 INFO:tasks.workunit.client.1.smithi117.stdout:1/723: creat d1/d1f/d32/dc6/ffd x:0 0 0 2024-03-22T06:15:27.343 INFO:tasks.workunit.client.1.smithi117.stdout:9/709: unlink d4/d46/da0/ccb 0 2024-03-22T06:15:27.344 INFO:tasks.workunit.client.1.smithi117.stdout:3/645: dwrite d9/d1a/d3a/f53 [0,4194304] 0 2024-03-22T06:15:27.344 INFO:tasks.workunit.client.1.smithi117.stdout:3/646: truncate d9/d1a/faf 1005872 0 2024-03-22T06:15:27.344 INFO:tasks.workunit.client.1.smithi117.stdout:3/647: chown d9/l27 8103 1 2024-03-22T06:15:27.351 INFO:tasks.workunit.client.1.smithi117.stdout:4/621: creat d0/d48/fd8 x:0 0 0 2024-03-22T06:15:27.351 INFO:tasks.workunit.client.1.smithi117.stdout:4/622: dread - d0/d48/da0/fb7 zero size 2024-03-22T06:15:27.354 INFO:tasks.workunit.client.1.smithi117.stdout:6/711: mkdir d3/db4/dd4/df4 0 2024-03-22T06:15:27.357 INFO:tasks.workunit.client.1.smithi117.stdout:4/623: write d0/d4/d2a/d2f/f56 [3960674,26697] 0 2024-03-22T06:15:27.358 INFO:tasks.workunit.client.1.smithi117.stdout:0/658: unlink d1/d10/d5c/d40/f41 0 2024-03-22T06:15:27.364 INFO:tasks.workunit.client.1.smithi117.stdout:5/693: rmdir dd/d16/d1a/d32 39 2024-03-22T06:15:27.366 INFO:tasks.workunit.client.1.smithi117.stdout:7/649: mknod d8/d9/da/d10/d23/d27/d56/ccc 0 2024-03-22T06:15:27.368 INFO:tasks.workunit.client.1.smithi117.stdout:8/663: rename d1/d28/l5f to d1/d3a/d44/d11/lce 0 2024-03-22T06:15:27.378 INFO:tasks.workunit.client.1.smithi117.stdout:2/629: rmdir d2/d5/d13 39 2024-03-22T06:15:27.384 INFO:tasks.workunit.client.1.smithi117.stdout:1/724: mkdir d1/d3c/d4d/dfe 0 2024-03-22T06:15:27.390 INFO:tasks.workunit.client.1.smithi117.stdout:9/710: getdents d4/d8/d93 0 2024-03-22T06:15:27.394 INFO:tasks.workunit.client.1.smithi117.stdout:6/712: mkdir d3/d3d/d17/d31/df5 0 2024-03-22T06:15:27.397 INFO:tasks.workunit.client.1.smithi117.stdout:4/624: rename d0/f2d to d0/fd9 0 2024-03-22T06:15:27.398 INFO:tasks.workunit.client.1.smithi117.stdout:5/694: mknod dd/d16/d1a/dda/cf0 0 2024-03-22T06:15:27.402 INFO:tasks.workunit.client.1.smithi117.stdout:2/630: dwrite d2/d5/d13/d3f/d1d/d3a/d89/dc6/d39/fac [0,4194304] 0 2024-03-22T06:15:27.404 INFO:tasks.workunit.client.1.smithi117.stdout:8/664: truncate d1/d3a/d44/f8d 1967394 0 2024-03-22T06:15:27.404 INFO:tasks.workunit.client.1.smithi117.stdout:8/665: dread - d1/fac zero size 2024-03-22T06:15:27.405 INFO:tasks.workunit.client.1.smithi117.stdout:4/625: dread d0/d19/f1a [0,4194304] 0 2024-03-22T06:15:27.410 INFO:tasks.workunit.client.1.smithi117.stdout:8/666: write d1/d3a/d44/f3c [2701348,41470] 0 2024-03-22T06:15:27.422 INFO:tasks.workunit.client.1.smithi117.stdout:5/695: link dd/d16/d1a/l63 dd/df/d5e/d91/d72/d35/da2/lf1 0 2024-03-22T06:15:27.422 INFO:tasks.workunit.client.1.smithi117.stdout:2/631: dwrite d2/d5/d41/d83/fcc [0,4194304] 0 2024-03-22T06:15:27.425 INFO:tasks.workunit.client.1.smithi117.stdout:8/667: write d1/d28/d6b/fad [1161266,39313] 0 2024-03-22T06:15:27.426 INFO:tasks.workunit.client.1.smithi117.stdout:3/648: rename d9/d1a/dc1/dd0/fdb to d9/d1a/d39/d7d/fdf 0 2024-03-22T06:15:27.427 INFO:tasks.workunit.client.1.smithi117.stdout:3/649: fsync d9/d1a/d39/d4c/d37/f8c 0 2024-03-22T06:15:27.427 INFO:tasks.workunit.client.1.smithi117.stdout:3/650: creat d9/d1a/d39/d4c/d37/dcc/fe0 x:0 0 0 2024-03-22T06:15:27.429 INFO:tasks.workunit.client.1.smithi117.stdout:3/651: dread d9/d93/fdc [0,4194304] 0 2024-03-22T06:15:27.452 INFO:tasks.workunit.client.1.smithi117.stdout:5/696: mknod dd/d16/d1a/dda/cf2 0 2024-03-22T06:15:27.452 INFO:tasks.workunit.client.1.smithi117.stdout:5/697: creat dd/df/d5e/d91/d72/d35/da2/ff3 x:0 0 0 2024-03-22T06:15:27.458 INFO:tasks.workunit.client.1.smithi117.stdout:5/698: write dd/df/d88/fa9 [1373128,78086] 0 2024-03-22T06:15:27.459 INFO:tasks.workunit.client.1.smithi117.stdout:8/668: creat d1/d3a/d44/d11/fcf x:0 0 0 2024-03-22T06:15:27.459 INFO:tasks.workunit.client.1.smithi117.stdout:8/669: creat d1/d28/d6b/fd0 x:0 0 0 2024-03-22T06:15:27.462 INFO:tasks.workunit.client.1.smithi117.stdout:0/659: rename d1/d10/d18/d26/db6/fb9 to d1/d10/d5c/d40/d4e/dbc/ddb/fea 0 2024-03-22T06:15:27.462 INFO:tasks.workunit.client.1.smithi117.stdout:0/660: chown d1/d3/d2b/d3e/de/d1f/d3b/d55/l68 4887 1 2024-03-22T06:15:27.465 INFO:tasks.workunit.client.1.smithi117.stdout:3/652: creat d9/d1a/d3a/fe1 x:0 0 0 2024-03-22T06:15:27.474 INFO:tasks.workunit.client.1.smithi117.stdout:8/670: dwrite d1/fa9 [0,4194304] 0 2024-03-22T06:15:27.475 INFO:tasks.workunit.client.1.smithi117.stdout:2/632: truncate d2/d5/d13/d3f/d1d/d3a/d89/daa/d4f/f62 755355 0 2024-03-22T06:15:27.480 INFO:tasks.workunit.client.1.smithi117.stdout:5/699: mknod dd/d16/d86/cf4 0 2024-03-22T06:15:27.480 INFO:tasks.workunit.client.1.smithi117.stdout:5/700: fsync dd/df/d5e/d91/d72/d33/d9a/fc3 0 2024-03-22T06:15:27.480 INFO:tasks.workunit.client.1.smithi117.stdout:5/701: truncate dd/d16/d5a/f97 1409190 0 2024-03-22T06:15:27.480 INFO:tasks.workunit.client.1.smithi117.stdout:5/702: readlink dd/df/d5e/d91/d72/l69 0 2024-03-22T06:15:27.480 INFO:tasks.workunit.client.1.smithi117.stdout:5/703: chown l5 15559 1 2024-03-22T06:15:27.486 INFO:tasks.workunit.client.1.smithi117.stdout:7/650: rename d8/d9/da/d10/d23/d41/fab to d8/d9/da/d10/d23/fcd 0 2024-03-22T06:15:27.489 INFO:tasks.workunit.client.1.smithi117.stdout:0/661: getdents d1/d10/d18 0 2024-03-22T06:15:27.492 INFO:tasks.workunit.client.1.smithi117.stdout:3/653: write d9/d1a/d39/d7d/f98 [2297157,104200] 0 2024-03-22T06:15:27.492 INFO:tasks.workunit.client.1.smithi117.stdout:3/654: creat d9/d1a/fe2 x:0 0 0 2024-03-22T06:15:27.492 INFO:tasks.workunit.client.1.smithi117.stdout:3/655: creat d9/d1a/d3a/fe3 x:0 0 0 2024-03-22T06:15:27.495 INFO:tasks.workunit.client.1.smithi117.stdout:8/671: symlink d1/d28/d6b/da3/ld1 0 2024-03-22T06:15:27.498 INFO:tasks.workunit.client.1.smithi117.stdout:3/656: dread f2 [0,4194304] 0 2024-03-22T06:15:27.502 INFO:tasks.workunit.client.1.smithi117.stdout:9/711: rename d4/d8/d3f/f89 to d4/d7/d27/d5b/dae/fea 0 2024-03-22T06:15:27.503 INFO:tasks.workunit.client.1.smithi117.stdout:9/712: chown d4/d8/d3e/d7a/f9e 0 1 2024-03-22T06:15:27.503 INFO:tasks.workunit.client.1.smithi117.stdout:9/713: fdatasync d4/d8/f1a 0 2024-03-22T06:15:27.503 INFO:tasks.workunit.client.1.smithi117.stdout:9/714: chown d4/d7/d27/f60 59470 1 2024-03-22T06:15:27.508 INFO:tasks.workunit.client.1.smithi117.stdout:3/657: dread d9/d1a/f20 [0,4194304] 0 2024-03-22T06:15:27.508 INFO:tasks.workunit.client.1.smithi117.stdout:7/651: unlink d8/d9/da/d2b/dc4/dac/lc7 0 2024-03-22T06:15:27.509 INFO:tasks.workunit.client.1.smithi117.stdout:0/662: getdents d1/d10/d5c/d40/d4e/dbc/ddb 0 2024-03-22T06:15:27.509 INFO:tasks.workunit.client.1.smithi117.stdout:0/663: dread - d1/d10/d5c/d40/d4e/f79 zero size 2024-03-22T06:15:27.513 INFO:tasks.workunit.client.1.smithi117.stdout:1/725: sync 2024-03-22T06:15:27.513 INFO:tasks.workunit.client.1.smithi117.stdout:1/726: dread - d1/d30/d44/d9c/da9/de3/ff9 zero size 2024-03-22T06:15:27.514 INFO:tasks.workunit.client.1.smithi117.stdout:9/715: dread d4/d47/d64/fb4 [0,4194304] 0 2024-03-22T06:15:27.517 INFO:tasks.workunit.client.1.smithi117.stdout:0/664: write d1/f3d [3769676,89975] 0 2024-03-22T06:15:27.518 INFO:tasks.workunit.client.1.smithi117.stdout:8/672: creat d1/d3a/d44/d6a/db7/fd2 x:0 0 0 2024-03-22T06:15:27.527 INFO:tasks.workunit.client.1.smithi117.stdout:6/713: rename d3/d3d/f8d to d3/d3d/dcb/dcc/ff6 0 2024-03-22T06:15:27.527 INFO:tasks.workunit.client.1.smithi117.stdout:4/626: sync 2024-03-22T06:15:27.529 INFO:tasks.workunit.client.1.smithi117.stdout:7/652: mkdir d8/d9/d37/d5f/dce 0 2024-03-22T06:15:27.535 INFO:tasks.workunit.client.1.smithi117.stdout:1/727: mknod d1/d62/de4/cff 0 2024-03-22T06:15:27.535 INFO:tasks.workunit.client.1.smithi117.stdout:1/728: write d1/d3c/d4d/d5b/fe0 [972475,83607] 0 2024-03-22T06:15:27.535 INFO:tasks.workunit.client.1.smithi117.stdout:1/729: readlink d1/d30/lcf 0 2024-03-22T06:15:27.535 INFO:tasks.workunit.client.1.smithi117.stdout:3/658: dwrite d9/d1a/d39/d4c/d37/dd5/fd6 [0,4194304] 0 2024-03-22T06:15:27.535 INFO:tasks.workunit.client.1.smithi117.stdout:3/659: fsync d9/d1a/f4b 0 2024-03-22T06:15:27.535 INFO:tasks.workunit.client.1.smithi117.stdout:0/665: symlink d1/d10/d5c/leb 0 2024-03-22T06:15:27.535 INFO:tasks.workunit.client.1.smithi117.stdout:0/666: fdatasync d1/d10/d5c/d40/f73 0 2024-03-22T06:15:27.537 INFO:tasks.workunit.client.1.smithi117.stdout:8/673: unlink d1/d3a/d44/d9/f81 0 2024-03-22T06:15:27.537 INFO:tasks.workunit.client.1.smithi117.stdout:8/674: chown d1/d28/d66/d7f/f92 169500 1 2024-03-22T06:15:27.537 INFO:tasks.workunit.client.1.smithi117.stdout:8/675: chown d1/d3a/d44/d4b/dc9 101 1 2024-03-22T06:15:27.537 INFO:tasks.workunit.client.1.smithi117.stdout:8/676: write d1/d19/d38/f8c [878399,6790] 0 2024-03-22T06:15:27.538 INFO:tasks.workunit.client.1.smithi117.stdout:6/714: mkdir d3/db4/d2d/d53/d59/d74/d28/ddb/de7/df7 0 2024-03-22T06:15:27.539 INFO:tasks.workunit.client.1.smithi117.stdout:1/730: creat d1/d22/d27/f100 x:0 0 0 2024-03-22T06:15:27.539 INFO:tasks.workunit.client.1.smithi117.stdout:1/731: fsync d1/d30/d44/d83/fe5 0 2024-03-22T06:15:27.539 INFO:tasks.workunit.client.1.smithi117.stdout:4/627: dread d0/d19/d34/d43/d63/d6f/d5f/fcb [0,4194304] 0 2024-03-22T06:15:27.540 INFO:tasks.workunit.client.1.smithi117.stdout:3/660: dread d9/d1a/d39/d7d/f9b [0,4194304] 0 2024-03-22T06:15:27.540 INFO:tasks.workunit.client.1.smithi117.stdout:8/677: dread d1/d28/d66/d7f/f9e [0,4194304] 0 2024-03-22T06:15:27.545 INFO:tasks.workunit.client.1.smithi117.stdout:7/653: dwrite d8/d9/da/f69 [4194304,4194304] 0 2024-03-22T06:15:27.545 INFO:tasks.workunit.client.1.smithi117.stdout:0/667: truncate d1/d3/d2b/d3e/de/f51 555570 0 2024-03-22T06:15:27.549 INFO:tasks.workunit.client.1.smithi117.stdout:1/732: dread d1/d1f/f8f [0,4194304] 0 2024-03-22T06:15:27.550 INFO:tasks.workunit.client.1.smithi117.stdout:2/633: rename d2/d5/d13/d3f to d2/d5/d13/d57/dd0 0 2024-03-22T06:15:27.554 INFO:tasks.workunit.client.1.smithi117.stdout:6/715: truncate d3/d61/f66 385548 0 2024-03-22T06:15:27.556 INFO:tasks.workunit.client.1.smithi117.stdout:8/678: symlink d1/d3a/ld3 0 2024-03-22T06:15:27.563 INFO:tasks.workunit.client.1.smithi117.stdout:7/654: unlink d8/d9/da/d10/d23/d27/d32/f54 0 2024-03-22T06:15:27.567 INFO:tasks.workunit.client.1.smithi117.stdout:5/704: rename dd/df/d5e/d91/d72/d35/da2/ff3 to dd/df/d5e/d91/d72/d33/ff5 0 2024-03-22T06:15:27.570 INFO:tasks.workunit.client.1.smithi117.stdout:2/634: truncate d2/d5/d13/f30 798020 0 2024-03-22T06:15:27.570 INFO:tasks.workunit.client.1.smithi117.stdout:2/635: write d2/d5/d13/d57/dd0/d1d/d3a/f3e [894676,59295] 0 2024-03-22T06:15:27.570 INFO:tasks.workunit.client.1.smithi117.stdout:2/636: chown d2/d5/db/d2c/d7d/la3 86062202 1 2024-03-22T06:15:27.570 INFO:tasks.workunit.client.1.smithi117.stdout:7/655: dread d8/d9/f99 [0,4194304] 0 2024-03-22T06:15:27.573 INFO:tasks.workunit.client.1.smithi117.stdout:8/679: symlink d1/d3a/d44/d4b/ld4 0 2024-03-22T06:15:27.576 INFO:tasks.workunit.client.1.smithi117.stdout:9/716: rename d4/d8/d3e/d7a/d3b/c4f to d4/d47/d64/d72/d8d/ceb 0 2024-03-22T06:15:27.577 INFO:tasks.workunit.client.1.smithi117.stdout:2/637: symlink d2/d9a/ld1 0 2024-03-22T06:15:27.577 INFO:tasks.workunit.client.1.smithi117.stdout:7/656: mkdir d8/d9/db0/dcf 0 2024-03-22T06:15:27.580 INFO:tasks.workunit.client.1.smithi117.stdout:8/680: dread d1/d28/d66/d7f/f92 [0,4194304] 0 2024-03-22T06:15:27.580 INFO:tasks.workunit.client.1.smithi117.stdout:4/628: rename d0/d19/d34/d43/d63/d6f/d70/fb9 to d0/dc0/fda 0 2024-03-22T06:15:27.580 INFO:tasks.workunit.client.1.smithi117.stdout:4/629: write d0/d48/da0/fb7 [515405,71171] 0 2024-03-22T06:15:27.581 INFO:tasks.workunit.client.1.smithi117.stdout:4/630: dread - d0/d19/d34/d43/d63/d6f/d70/fa2 zero size 2024-03-22T06:15:27.581 INFO:tasks.workunit.client.1.smithi117.stdout:4/631: chown d0/d4/d2a/d2f 225358604 1 2024-03-22T06:15:27.582 INFO:tasks.workunit.client.1.smithi117.stdout:3/661: dwrite d9/d1a/d39/d8a/da1/fb5 [0,4194304] 0 2024-03-22T06:15:27.583 INFO:tasks.workunit.client.1.smithi117.stdout:6/716: dwrite d3/d40/fe8 [0,4194304] 0 2024-03-22T06:15:27.583 INFO:tasks.workunit.client.1.smithi117.stdout:6/717: stat d3/db4/d2d/d42/da2/cb9 0 2024-03-22T06:15:27.583 INFO:tasks.workunit.client.1.smithi117.stdout:6/718: write d3/f5 [4935826,116581] 0 2024-03-22T06:15:27.584 INFO:tasks.workunit.client.1.smithi117.stdout:5/705: write dd/df/d5e/d91/d90/fa4 [1648505,24079] 0 2024-03-22T06:15:27.584 INFO:tasks.workunit.client.1.smithi117.stdout:5/706: read dd/d16/d1a/d32/d73/db8/fc5 [673919,57494] 0 2024-03-22T06:15:27.588 INFO:tasks.workunit.client.1.smithi117.stdout:2/638: dread d2/d5/d41/d83/fcc [0,4194304] 0 2024-03-22T06:15:27.595 INFO:tasks.workunit.client.1.smithi117.stdout:7/657: symlink d8/d9/db0/ld0 0 2024-03-22T06:15:27.595 INFO:tasks.workunit.client.1.smithi117.stdout:7/658: write d8/d9/da/d10/d23/d27/d32/fc5 [837775,91350] 0 2024-03-22T06:15:27.595 INFO:tasks.workunit.client.1.smithi117.stdout:7/659: write d8/d9/da/d10/d23/fbb [1002753,59029] 0 2024-03-22T06:15:27.599 INFO:tasks.workunit.client.1.smithi117.stdout:8/681: unlink d1/d19/d38/l57 0 2024-03-22T06:15:27.602 INFO:tasks.workunit.client.1.smithi117.stdout:7/660: write d8/d9/da/d10/d23/d27/d32/f81 [3711198,126578] 0 2024-03-22T06:15:27.602 INFO:tasks.workunit.client.1.smithi117.stdout:7/661: stat d8/d9/da/d10/d23/f38 0 2024-03-22T06:15:27.602 INFO:tasks.workunit.client.1.smithi117.stdout:0/668: rename d1/d10/d18/fda to d1/d3/d2b/d3e/de/d1f/d3b/d55/fec 0 2024-03-22T06:15:27.602 INFO:tasks.workunit.client.1.smithi117.stdout:0/669: write d1/d10/d18/d8c/fc7 [123280,40595] 0 2024-03-22T06:15:27.607 INFO:tasks.workunit.client.1.smithi117.stdout:4/632: link d0/d3f/c5a d0/d19/dcc/cdb 0 2024-03-22T06:15:27.607 INFO:tasks.workunit.client.1.smithi117.stdout:4/633: readlink d0/ld7 0 2024-03-22T06:15:27.607 INFO:tasks.workunit.client.1.smithi117.stdout:6/719: dwrite d3/d40/d46/f6b [0,4194304] 0 2024-03-22T06:15:27.607 INFO:tasks.workunit.client.1.smithi117.stdout:6/720: fsync d3/db4/d2d/d53/d75/fa0 0 2024-03-22T06:15:27.607 INFO:tasks.workunit.client.1.smithi117.stdout:6/721: chown d3/c2a 19 1 2024-03-22T06:15:27.609 INFO:tasks.workunit.client.1.smithi117.stdout:0/670: write d1/d10/d18/d26/d49/d7e/d39/f52 [1937207,94856] 0 2024-03-22T06:15:27.617 INFO:tasks.workunit.client.1.smithi117.stdout:5/707: mknod dd/d16/d1a/cf6 0 2024-03-22T06:15:27.628 INFO:tasks.workunit.client.1.smithi117.stdout:1/733: rename d1/d62/f67 to d1/d30/d44/d83/f101 0 2024-03-22T06:15:27.631 INFO:tasks.workunit.client.1.smithi117.stdout:4/634: link d0/d19/l3e d0/d51/da4/ldc 0 2024-03-22T06:15:27.631 INFO:tasks.workunit.client.1.smithi117.stdout:4/635: readlink d0/d19/d34/d43/d63/d6f/d6a/lc7 0 2024-03-22T06:15:27.631 INFO:tasks.workunit.client.1.smithi117.stdout:4/636: creat d0/d48/fdd x:0 0 0 2024-03-22T06:15:27.634 INFO:tasks.workunit.client.1.smithi117.stdout:6/722: symlink d3/d3d/dcb/lf8 0 2024-03-22T06:15:27.634 INFO:tasks.workunit.client.1.smithi117.stdout:6/723: fdatasync d3/d61/f9b 0 2024-03-22T06:15:27.635 INFO:tasks.workunit.client.1.smithi117.stdout:6/724: truncate d3/db4/fc5 982444 0 2024-03-22T06:15:27.637 INFO:tasks.workunit.client.1.smithi117.stdout:0/671: link d1/d3/d2b/d3e/de/d1f/d3b/l96 d1/d10/d18/d26/db6/led 0 2024-03-22T06:15:27.637 INFO:tasks.workunit.client.1.smithi117.stdout:0/672: fsync d1/d10/d18/d26/d49/d7e/d39/dc5/fd5 0 2024-03-22T06:15:27.637 INFO:tasks.workunit.client.1.smithi117.stdout:0/673: write d1/d3/d2b/d3e/d3f/fd7 [571062,13728] 0 2024-03-22T06:15:27.638 INFO:tasks.workunit.client.1.smithi117.stdout:0/674: chown d1/d3/l56 0 1 2024-03-22T06:15:27.638 INFO:tasks.workunit.client.1.smithi117.stdout:0/675: chown d1/d10/d18/d26/d53/c7c 11 1 2024-03-22T06:15:27.638 INFO:tasks.workunit.client.1.smithi117.stdout:0/676: dread - d1/d10/d5c/d40/dbb/fcd zero size 2024-03-22T06:15:27.638 INFO:tasks.workunit.client.1.smithi117.stdout:0/677: chown d1/d10/f86 28997642 1 2024-03-22T06:15:27.638 INFO:tasks.workunit.client.1.smithi117.stdout:5/708: creat dd/d16/d1a/d32/ff7 x:0 0 0 2024-03-22T06:15:27.639 INFO:tasks.workunit.client.1.smithi117.stdout:1/734: creat d1/d22/d27/d4e/f102 x:0 0 0 2024-03-22T06:15:27.641 INFO:tasks.workunit.client.1.smithi117.stdout:9/717: rename d4/d8/d93/db1/dd1/dd6/d9b/fce to d4/d8/d3e/d7a/d3b/fec 0 2024-03-22T06:15:27.643 INFO:tasks.workunit.client.1.smithi117.stdout:4/637: mkdir d0/d48/da0/dd2/db6/dde 0 2024-03-22T06:15:27.643 INFO:tasks.workunit.client.1.smithi117.stdout:4/638: readlink d0/d4/d2a/d2f/la1 0 2024-03-22T06:15:27.643 INFO:tasks.workunit.client.1.smithi117.stdout:4/639: stat d0/d19/d34/d43/d63/d6f/d70/fa2 0 2024-03-22T06:15:27.645 INFO:tasks.workunit.client.1.smithi117.stdout:6/725: unlink d3/db4/d2d/d53/d59/d74/c7c 0 2024-03-22T06:15:27.645 INFO:tasks.workunit.client.1.smithi117.stdout:6/726: dread - d3/db4/fd2 zero size 2024-03-22T06:15:27.646 INFO:tasks.workunit.client.1.smithi117.stdout:6/727: truncate d3/d61/f9b 919789 0 2024-03-22T06:15:27.647 INFO:tasks.workunit.client.1.smithi117.stdout:5/709: mknod dd/df/dae/cf8 0 2024-03-22T06:15:27.647 INFO:tasks.workunit.client.1.smithi117.stdout:5/710: read - dd/df/d1e/fb2 zero size 2024-03-22T06:15:27.647 INFO:tasks.workunit.client.1.smithi117.stdout:5/711: readlink dd/d16/d5a/d66/lb5 0 2024-03-22T06:15:27.647 INFO:tasks.workunit.client.1.smithi117.stdout:5/712: chown dd/df/d5e/d91/d72/d33/cdd 459 1 2024-03-22T06:15:27.648 INFO:tasks.workunit.client.1.smithi117.stdout:1/735: creat d1/d30/d44/d9c/da9/f103 x:0 0 0 2024-03-22T06:15:27.651 INFO:tasks.workunit.client.1.smithi117.stdout:3/662: rename d9/d1a/d39/f48 to d9/d1a/d39/d4c/d37/d69/fe4 0 2024-03-22T06:15:27.652 INFO:tasks.workunit.client.1.smithi117.stdout:3/663: creat d9/d1a/d39/d4c/d37/dd5/fe5 x:0 0 0 2024-03-22T06:15:27.652 INFO:tasks.workunit.client.1.smithi117.stdout:1/736: dread d1/d1f/f8f [0,4194304] 0 2024-03-22T06:15:27.653 INFO:tasks.workunit.client.1.smithi117.stdout:9/718: truncate d4/d8/d3e/f8f 3126180 0 2024-03-22T06:15:27.653 INFO:tasks.workunit.client.1.smithi117.stdout:9/719: write d4/d8/d3e/d7a/f70 [9042329,86246] 0 2024-03-22T06:15:27.653 INFO:tasks.workunit.client.1.smithi117.stdout:0/678: dwrite d1/d10/d18/d26/d49/d7e/d39/f6f [0,4194304] 0 2024-03-22T06:15:27.656 INFO:tasks.workunit.client.1.smithi117.stdout:6/728: link d3/d40/d46/d84/f7b d3/d3d/d17/d31/ff9 0 2024-03-22T06:15:27.657 INFO:tasks.workunit.client.1.smithi117.stdout:5/713: getdents dd/d16/d1a/dda 0 2024-03-22T06:15:27.661 INFO:tasks.workunit.client.1.smithi117.stdout:2/639: rename d2/d5/d41/c54 to d2/d35/d4c/cd2 0 2024-03-22T06:15:27.662 INFO:tasks.workunit.client.1.smithi117.stdout:4/640: dwrite d0/d19/d28/f59 [0,4194304] 0 2024-03-22T06:15:27.664 INFO:tasks.workunit.client.1.smithi117.stdout:3/664: symlink d9/d66/le6 0 2024-03-22T06:15:27.664 INFO:tasks.workunit.client.1.smithi117.stdout:3/665: write d9/d1a/d39/d4c/d37/fc3 [609648,63435] 0 2024-03-22T06:15:27.665 INFO:tasks.workunit.client.1.smithi117.stdout:1/737: dread d1/fd [4194304,4194304] 0 2024-03-22T06:15:27.675 INFO:tasks.workunit.client.1.smithi117.stdout:6/729: mkdir d3/d3d/d17/d31/df5/dfa 0 2024-03-22T06:15:27.675 INFO:tasks.workunit.client.1.smithi117.stdout:6/730: chown d3/db4/d2d/d53/d59/d74/da/fc6 142249554 1 2024-03-22T06:15:27.675 INFO:tasks.workunit.client.1.smithi117.stdout:6/731: dread - d3/d3d/d17/d60/f8f zero size 2024-03-22T06:15:27.676 INFO:tasks.workunit.client.1.smithi117.stdout:0/679: dwrite d1/d3/d2b/d3e/d3f/f6d [0,4194304] 0 2024-03-22T06:15:27.676 INFO:tasks.workunit.client.1.smithi117.stdout:5/714: getdents dd/d16/d86/d93 0 2024-03-22T06:15:27.678 INFO:tasks.workunit.client.1.smithi117.stdout:8/682: rename d1/l61 to d1/d3a/d44/d6a/db7/ld5 0 2024-03-22T06:15:27.679 INFO:tasks.workunit.client.1.smithi117.stdout:8/683: stat d1/d3a/d44/d11 0 2024-03-22T06:15:27.679 INFO:tasks.workunit.client.1.smithi117.stdout:8/684: chown d1/d3a/d44/d9/c7c 1 1 2024-03-22T06:15:27.680 INFO:tasks.workunit.client.1.smithi117.stdout:2/640: mkdir d2/d9a/dd3 0 2024-03-22T06:15:27.680 INFO:tasks.workunit.client.1.smithi117.stdout:9/720: dwrite d4/d47/d64/f84 [0,4194304] 0 2024-03-22T06:15:27.690 INFO:tasks.workunit.client.1.smithi117.stdout:4/641: link d0/d19/d1c/fd5 d0/d19/d34/d43/d63/d6f/d70/fdf 0 2024-03-22T06:15:27.690 INFO:tasks.workunit.client.1.smithi117.stdout:4/642: write d0/d48/da0/fb7 [998724,119948] 0 2024-03-22T06:15:27.695 INFO:tasks.workunit.client.1.smithi117.stdout:6/732: mknod d3/d3d/d17/cfb 0 2024-03-22T06:15:27.700 INFO:tasks.workunit.client.1.smithi117.stdout:5/715: mknod dd/df/d5e/d91/d72/d35/d81/cf9 0 2024-03-22T06:15:27.700 INFO:tasks.workunit.client.1.smithi117.stdout:5/716: truncate dd/df/d5e/d91/d36/f99 1047468 0 2024-03-22T06:15:27.701 INFO:tasks.workunit.client.1.smithi117.stdout:8/685: dwrite d1/d19/d38/f75 [0,4194304] 0 2024-03-22T06:15:27.701 INFO:tasks.workunit.client.1.smithi117.stdout:8/686: creat d1/d3a/d44/d6a/db7/fd6 x:0 0 0 2024-03-22T06:15:27.702 INFO:tasks.workunit.client.1.smithi117.stdout:8/687: chown d1/d19/d38/f70 503974 1 2024-03-22T06:15:27.702 INFO:tasks.workunit.client.1.smithi117.stdout:8/688: write d1/d28/fa8 [744316,28393] 0 2024-03-22T06:15:27.702 INFO:tasks.workunit.client.1.smithi117.stdout:0/680: dwrite d1/d10/d18/d26/d49/d7e/d39/f52 [0,4194304] 0 2024-03-22T06:15:27.711 INFO:tasks.workunit.client.1.smithi117.stdout:2/641: unlink d2/d5/d13/d57/dd0/d1d/d3a/da6/cc1 0 2024-03-22T06:15:27.725 INFO:tasks.workunit.client.1.smithi117.stdout:8/689: dwrite d1/d3a/d44/d6a/fba [0,4194304] 0 2024-03-22T06:15:27.735 INFO:tasks.workunit.client.1.smithi117.stdout:8/690: dread d1/f1e [0,4194304] 0 2024-03-22T06:15:27.735 INFO:tasks.workunit.client.1.smithi117.stdout:8/691: chown d1/d3a/d44/d6a/c78 3196301 1 2024-03-22T06:15:27.737 INFO:tasks.workunit.client.1.smithi117.stdout:4/643: creat d0/d3f/fe0 x:0 0 0 2024-03-22T06:15:27.738 INFO:tasks.workunit.client.1.smithi117.stdout:3/666: symlink d9/d1a/d39/d4c/d36/d41/le7 0 2024-03-22T06:15:27.740 INFO:tasks.workunit.client.1.smithi117.stdout:6/733: mknod d3/d40/d46/d84/d49/d79/cfc 0 2024-03-22T06:15:27.742 INFO:tasks.workunit.client.1.smithi117.stdout:5/717: creat dd/df/d5e/d91/d72/d60/dec/ffa x:0 0 0 2024-03-22T06:15:27.748 INFO:tasks.workunit.client.1.smithi117.stdout:5/718: dread dd/df/d5e/d91/d72/d35/f6f [0,4194304] 0 2024-03-22T06:15:27.749 INFO:tasks.workunit.client.1.smithi117.stdout:5/719: chown dd/df/d5e/d91/f34 1455465979 1 2024-03-22T06:15:27.753 INFO:tasks.workunit.client.1.smithi117.stdout:0/681: creat d1/d3/d2b/d3e/de/d1f/fee x:0 0 0 2024-03-22T06:15:27.759 INFO:tasks.workunit.client.1.smithi117.stdout:2/642: write d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d77/f95 [992759,98847] 0 2024-03-22T06:15:27.773 INFO:tasks.workunit.client.1.smithi117.stdout:6/734: dwrite d3/db4/d2d/d53/d59/d74/da/fc6 [0,4194304] 0 2024-03-22T06:15:27.773 INFO:tasks.workunit.client.1.smithi117.stdout:6/735: chown d3/db4/d2d/d42/f76 6 1 2024-03-22T06:15:27.773 INFO:tasks.workunit.client.1.smithi117.stdout:3/667: dwrite d9/d1a/d3a/fe1 [0,4194304] 0 2024-03-22T06:15:27.816 INFO:tasks.workunit.client.1.smithi117.stdout:9/721: rmdir d4 39 2024-03-22T06:15:27.830 INFO:tasks.workunit.client.1.smithi117.stdout:8/692: creat d1/d3a/d44/d4b/dc9/fd7 x:0 0 0 2024-03-22T06:15:27.832 INFO:tasks.workunit.client.1.smithi117.stdout:4/644: creat d0/d19/dcc/fe1 x:0 0 0 2024-03-22T06:15:27.836 INFO:tasks.workunit.client.1.smithi117.stdout:5/720: symlink dd/d16/d86/dee/lfb 0 2024-03-22T06:15:27.836 INFO:tasks.workunit.client.1.smithi117.stdout:5/721: write dd/df/d5e/d91/d72/fd3 [712357,95506] 0 2024-03-22T06:15:27.839 INFO:tasks.workunit.client.1.smithi117.stdout:0/682: creat d1/d3/d2b/fef x:0 0 0 2024-03-22T06:15:27.839 INFO:tasks.workunit.client.1.smithi117.stdout:0/683: creat d1/d10/d18/d8c/ff0 x:0 0 0 2024-03-22T06:15:27.840 INFO:tasks.workunit.client.1.smithi117.stdout:0/684: fdatasync d1/d10/d18/f38 0 2024-03-22T06:15:27.840 INFO:tasks.workunit.client.1.smithi117.stdout:7/662: rename d8/d9/da/d10/d23/d27/d32 to d8/d9/dd1 0 2024-03-22T06:15:27.843 INFO:tasks.workunit.client.1.smithi117.stdout:2/643: unlink d2/d5/db/d2c/c67 0 2024-03-22T06:15:27.843 INFO:tasks.workunit.client.1.smithi117.stdout:2/644: fsync d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/f65 0 2024-03-22T06:15:27.845 INFO:tasks.workunit.client.1.smithi117.stdout:6/736: rmdir d3/db4/d2d/d53/d59/d74/d28/ddb/de7 39 2024-03-22T06:15:27.849 INFO:tasks.workunit.client.1.smithi117.stdout:9/722: creat d4/d46/fed x:0 0 0 2024-03-22T06:15:27.852 INFO:tasks.workunit.client.1.smithi117.stdout:4/645: rmdir d0/d19/d34/d43/d63/db5 0 2024-03-22T06:15:27.855 INFO:tasks.workunit.client.1.smithi117.stdout:5/722: mkdir dd/d16/d1a/d32/dfc 0 2024-03-22T06:15:27.855 INFO:tasks.workunit.client.1.smithi117.stdout:8/693: dwrite d1/d19/d98/fbc [0,4194304] 0 2024-03-22T06:15:27.855 INFO:tasks.workunit.client.1.smithi117.stdout:8/694: creat d1/d19/d98/fd8 x:0 0 0 2024-03-22T06:15:27.868 INFO:tasks.workunit.client.1.smithi117.stdout:3/668: rename d9/d1a/d39/d4c/d37/d69/lc4 to d9/d1a/d39/d7d/d97/le8 0 2024-03-22T06:15:27.868 INFO:tasks.workunit.client.1.smithi117.stdout:3/669: stat d9/d66/fbd 0 2024-03-22T06:15:27.868 INFO:tasks.workunit.client.1.smithi117.stdout:3/670: creat d9/d1a/d39/d4c/d37/dcc/fe9 x:0 0 0 2024-03-22T06:15:27.870 INFO:tasks.workunit.client.1.smithi117.stdout:7/663: symlink d8/d9/da/d10/d23/d27/ld2 0 2024-03-22T06:15:27.873 INFO:tasks.workunit.client.1.smithi117.stdout:2/645: mkdir d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4 0 2024-03-22T06:15:27.874 INFO:tasks.workunit.client.1.smithi117.stdout:6/737: creat d3/db4/d2d/ffd x:0 0 0 2024-03-22T06:15:27.876 INFO:tasks.workunit.client.1.smithi117.stdout:9/723: mkdir d4/d50/dee 0 2024-03-22T06:15:27.878 INFO:tasks.workunit.client.1.smithi117.stdout:4/646: dwrite d0/dc0/fda [0,4194304] 0 2024-03-22T06:15:27.878 INFO:tasks.workunit.client.1.smithi117.stdout:4/647: truncate d0/d19/d9b/fc2 397996 0 2024-03-22T06:15:27.886 INFO:tasks.workunit.client.1.smithi117.stdout:5/723: symlink dd/df/d5e/d91/d90/dca/lfd 0 2024-03-22T06:15:27.886 INFO:tasks.workunit.client.1.smithi117.stdout:5/724: chown dd/df/d5e/d91/d72/l7b 13 1 2024-03-22T06:15:27.886 INFO:tasks.workunit.client.1.smithi117.stdout:5/725: write dd/df/d5e/d91/d72/d35/fcc [877140,36783] 0 2024-03-22T06:15:27.887 INFO:tasks.workunit.client.1.smithi117.stdout:5/726: readlink dd/df/d1e/l3b 0 2024-03-22T06:15:27.889 INFO:tasks.workunit.client.1.smithi117.stdout:8/695: symlink d1/d28/ld9 0 2024-03-22T06:15:27.893 INFO:tasks.workunit.client.1.smithi117.stdout:1/738: sync 2024-03-22T06:15:27.893 INFO:tasks.workunit.client.1.smithi117.stdout:1/739: chown d1/d30/d44/dd3 52770 1 2024-03-22T06:15:27.893 INFO:tasks.workunit.client.1.smithi117.stdout:7/664: dwrite d8/d9/da/d10/d23/d27/d56/fa4 [4194304,4194304] 0 2024-03-22T06:15:27.899 INFO:tasks.workunit.client.1.smithi117.stdout:2/646: mknod d2/d35/d4c/cd5 0 2024-03-22T06:15:27.901 INFO:tasks.workunit.client.1.smithi117.stdout:6/738: mknod d3/db4/d2d/d53/d59/cfe 0 2024-03-22T06:15:27.901 INFO:tasks.workunit.client.1.smithi117.stdout:6/739: fdatasync d3/d3d/d17/f33 0 2024-03-22T06:15:27.903 INFO:tasks.workunit.client.1.smithi117.stdout:9/724: getdents d4/d47/d64/dc5/dda 0 2024-03-22T06:15:27.907 INFO:tasks.workunit.client.1.smithi117.stdout:4/648: dwrite d0/f83 [0,4194304] 0 2024-03-22T06:15:27.907 INFO:tasks.workunit.client.1.smithi117.stdout:4/649: fsync d0/d19/d28/d47/fbb 0 2024-03-22T06:15:27.909 INFO:tasks.workunit.client.1.smithi117.stdout:5/727: mkdir dd/d16/d86/d93/dfe 0 2024-03-22T06:15:27.909 INFO:tasks.workunit.client.1.smithi117.stdout:5/728: fsync dd/df/d5e/d91/d72/d33/d9a/fc3 0 2024-03-22T06:15:27.910 INFO:tasks.workunit.client.1.smithi117.stdout:3/671: read d9/d1a/d39/d7d/f88 [2790632,93013] 0 2024-03-22T06:15:27.910 INFO:tasks.workunit.client.1.smithi117.stdout:3/672: stat d9/d1a/d39/d7d/fdf 0 2024-03-22T06:15:27.910 INFO:tasks.workunit.client.1.smithi117.stdout:3/673: write d9/d1a/d39/d4c/d37/dd5/fe5 [983589,51990] 0 2024-03-22T06:15:27.913 INFO:tasks.workunit.client.1.smithi117.stdout:8/696: rename d1/d3a/d44/d11/fcf to d1/d3a/d44/d4b/dc9/fda 0 2024-03-22T06:15:27.913 INFO:tasks.workunit.client.1.smithi117.stdout:8/697: write d1/d28/d6b/fad [2837331,48811] 0 2024-03-22T06:15:27.915 INFO:tasks.workunit.client.1.smithi117.stdout:0/685: link d1/d9d/la1 d1/d9d/lf1 0 2024-03-22T06:15:27.917 INFO:tasks.workunit.client.1.smithi117.stdout:1/740: rmdir d1/d62/deb 0 2024-03-22T06:15:27.921 INFO:tasks.workunit.client.1.smithi117.stdout:8/698: write d1/d19/d98/fbc [2844669,47883] 0 2024-03-22T06:15:27.922 INFO:tasks.workunit.client.1.smithi117.stdout:6/740: mknod d3/d3d/d17/cff 0 2024-03-22T06:15:27.923 INFO:tasks.workunit.client.1.smithi117.stdout:0/686: dread d1/d3/d2b/d3e/d3f/f6d [0,4194304] 0 2024-03-22T06:15:27.924 INFO:tasks.workunit.client.1.smithi117.stdout:0/687: write d1/d3/d2b/d3e/d3f/f71 [325916,31924] 0 2024-03-22T06:15:27.924 INFO:tasks.workunit.client.1.smithi117.stdout:0/688: creat d1/d10/d5c/d40/dbb/ff2 x:0 0 0 2024-03-22T06:15:27.924 INFO:tasks.workunit.client.1.smithi117.stdout:0/689: write d1/d10/d5c/d40/d4e/dbc/fe7 [230550,46534] 0 2024-03-22T06:15:27.924 INFO:tasks.workunit.client.1.smithi117.stdout:9/725: creat d4/d7/d27/fef x:0 0 0 2024-03-22T06:15:27.927 INFO:tasks.workunit.client.1.smithi117.stdout:4/650: unlink d0/d48/da0/dd2/fbe 0 2024-03-22T06:15:27.928 INFO:tasks.workunit.client.1.smithi117.stdout:5/729: rmdir dd/df/d5e/d91/d36/d98 39 2024-03-22T06:15:27.930 INFO:tasks.workunit.client.1.smithi117.stdout:3/674: truncate d9/d1a/d39/d4c/d36/f62 909190 0 2024-03-22T06:15:27.930 INFO:tasks.workunit.client.1.smithi117.stdout:3/675: fsync d9/d1a/d39/d4c/d36/f9f 0 2024-03-22T06:15:27.934 INFO:tasks.workunit.client.1.smithi117.stdout:2/647: dread d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d77/f80 [0,4194304] 0 2024-03-22T06:15:27.934 INFO:tasks.workunit.client.1.smithi117.stdout:9/726: write d4/d47/d64/f84 [4017421,11371] 0 2024-03-22T06:15:27.936 INFO:tasks.workunit.client.1.smithi117.stdout:6/741: dread d3/db4/d2d/d53/d59/d74/da/db3/fa4 [0,4194304] 0 2024-03-22T06:15:27.936 INFO:tasks.workunit.client.1.smithi117.stdout:6/742: write d3/db4/d2d/d53/d59/d74/da/de/ff0 [500195,59577] 0 2024-03-22T06:15:27.937 INFO:tasks.workunit.client.1.smithi117.stdout:6/743: creat d3/d3d/d17/d60/f100 x:0 0 0 2024-03-22T06:15:27.937 INFO:tasks.workunit.client.1.smithi117.stdout:6/744: dread - d3/db4/fd2 zero size 2024-03-22T06:15:27.942 INFO:tasks.workunit.client.1.smithi117.stdout:5/730: mkdir dd/d16/d1a/d32/dfc/dff 0 2024-03-22T06:15:27.942 INFO:tasks.workunit.client.1.smithi117.stdout:5/731: write dd/df/d95/fe6 [564307,11580] 0 2024-03-22T06:15:27.947 INFO:tasks.workunit.client.1.smithi117.stdout:7/665: rename d8/d9/da/d2b/dc4/l42 to d8/d9/d37/d45/d88/db1/ld3 0 2024-03-22T06:15:27.948 INFO:tasks.workunit.client.1.smithi117.stdout:7/666: read d8/d9/da/d10/f8d [577247,50480] 0 2024-03-22T06:15:27.948 INFO:tasks.workunit.client.1.smithi117.stdout:7/667: write d8/d9/da/d10/f8d [1688738,59515] 0 2024-03-22T06:15:27.949 INFO:tasks.workunit.client.1.smithi117.stdout:3/676: dwrite d9/fa6 [4194304,4194304] 0 2024-03-22T06:15:27.953 INFO:tasks.workunit.client.1.smithi117.stdout:2/648: link d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d45/d88/cca d2/d5/d13/d57/dd0/d1d/d86/cd6 0 2024-03-22T06:15:27.953 INFO:tasks.workunit.client.1.smithi117.stdout:3/677: read d9/f63 [1054318,60856] 0 2024-03-22T06:15:27.955 INFO:tasks.workunit.client.1.smithi117.stdout:9/727: getdents d4/d47/d85 0 2024-03-22T06:15:27.958 INFO:tasks.workunit.client.1.smithi117.stdout:9/728: write d4/d7/d27/f56 [23255,28617] 0 2024-03-22T06:15:27.960 INFO:tasks.workunit.client.1.smithi117.stdout:4/651: dwrite d0/d4/d2a/d5b/fa6 [0,4194304] 0 2024-03-22T06:15:27.960 INFO:tasks.workunit.client.1.smithi117.stdout:5/732: mkdir dd/d16/d100 0 2024-03-22T06:15:27.960 INFO:tasks.workunit.client.1.smithi117.stdout:5/733: creat dd/df/d5e/d91/d72/d33/d9a/f101 x:0 0 0 2024-03-22T06:15:27.961 INFO:tasks.workunit.client.1.smithi117.stdout:3/678: write d9/d1a/d3a/f47 [207227,110210] 0 2024-03-22T06:15:27.962 INFO:tasks.workunit.client.1.smithi117.stdout:3/679: readlink d9/d1a/d39/d4c/d99/lcf 0 2024-03-22T06:15:27.975 INFO:tasks.workunit.client.1.smithi117.stdout:1/741: rename d1/d30/d35/f1a to d1/d14/f104 0 2024-03-22T06:15:27.981 INFO:tasks.workunit.client.1.smithi117.stdout:7/668: getdents d8/d9/da/d2b/d9b/da5 0 2024-03-22T06:15:27.985 INFO:tasks.workunit.client.1.smithi117.stdout:2/649: symlink d2/d5/db/ld7 0 2024-03-22T06:15:27.985 INFO:tasks.workunit.client.1.smithi117.stdout:2/650: fsync d2/d5/d13/d57/f63 0 2024-03-22T06:15:27.985 INFO:tasks.workunit.client.1.smithi117.stdout:2/651: chown d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/c91 143939 1 2024-03-22T06:15:27.986 INFO:tasks.workunit.client.1.smithi117.stdout:9/729: symlink d4/d7/d27/d5b/lf0 0 2024-03-22T06:15:27.989 INFO:tasks.workunit.client.1.smithi117.stdout:4/652: creat d0/d19/d34/d43/d63/fe2 x:0 0 0 2024-03-22T06:15:27.991 INFO:tasks.workunit.client.1.smithi117.stdout:5/734: mknod dd/df/d5e/d91/d72/d35/da2/c102 0 2024-03-22T06:15:27.992 INFO:tasks.workunit.client.1.smithi117.stdout:3/680: unlink d9/d1a/d39/fd2 0 2024-03-22T06:15:27.992 INFO:tasks.workunit.client.1.smithi117.stdout:3/681: chown d9/d1a/d39/d7d/d97/fba 53625 1 2024-03-22T06:15:27.995 INFO:tasks.workunit.client.1.smithi117.stdout:0/690: rename d1/d3/d2b/d3e/de/d1f/d3b/d55/f8e to d1/d3/dae/ff3 0 2024-03-22T06:15:27.995 INFO:tasks.workunit.client.1.smithi117.stdout:0/691: truncate d1/d10/d5c/d40/dbb/dbf/fd1 111347 0 2024-03-22T06:15:27.995 INFO:tasks.workunit.client.1.smithi117.stdout:0/692: truncate d1/d10/d18/d8c/ff0 267107 0 2024-03-22T06:15:27.996 INFO:tasks.workunit.client.1.smithi117.stdout:1/742: creat d1/d1f/f105 x:0 0 0 2024-03-22T06:15:27.997 INFO:tasks.workunit.client.1.smithi117.stdout:2/652: creat d2/d5/d13/d57/dd0/d1d/d3a/d71/fd8 x:0 0 0 2024-03-22T06:15:27.998 INFO:tasks.workunit.client.1.smithi117.stdout:9/730: link d4/d46/f55 d4/d8/d3f/ff1 0 2024-03-22T06:15:28.001 INFO:tasks.workunit.client.1.smithi117.stdout:4/653: creat d0/d19/d34/d43/fe3 x:0 0 0 2024-03-22T06:15:28.001 INFO:tasks.workunit.client.1.smithi117.stdout:4/654: getdents d0/d19/d28/d47/dd6 0 2024-03-22T06:15:28.001 INFO:tasks.workunit.client.1.smithi117.stdout:4/655: write d0/d19/d1c/fd5 [502306,75890] 0 2024-03-22T06:15:28.002 INFO:tasks.workunit.client.1.smithi117.stdout:3/682: symlink d9/d1a/d39/d4c/d37/lea 0 2024-03-22T06:15:28.002 INFO:tasks.workunit.client.1.smithi117.stdout:3/683: creat d9/d1a/d39/d4c/d37/dd5/feb x:0 0 0 2024-03-22T06:15:28.004 INFO:tasks.workunit.client.1.smithi117.stdout:0/693: mkdir d1/d10/d5c/d40/df4 0 2024-03-22T06:15:28.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:27 smithi117 ceph-mon[36514]: pgmap v27: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 224 MiB/s rd, 269 MiB/s wr, 496 op/s 2024-03-22T06:15:28.007 INFO:tasks.workunit.client.1.smithi117.stdout:5/735: dwrite dd/d16/d86/f9d [0,4194304] 0 2024-03-22T06:15:28.007 INFO:tasks.workunit.client.1.smithi117.stdout:5/736: chown dd/df/lb3 0 1 2024-03-22T06:15:28.007 INFO:tasks.workunit.client.1.smithi117.stdout:2/653: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/fd9 x:0 0 0 2024-03-22T06:15:28.011 INFO:tasks.workunit.client.1.smithi117.stdout:9/731: symlink d4/d47/lf2 0 2024-03-22T06:15:28.011 INFO:tasks.workunit.client.1.smithi117.stdout:9/732: dread - d4/d8/d3e/d7a/d2f/fe0 zero size 2024-03-22T06:15:28.012 INFO:tasks.workunit.client.1.smithi117.stdout:4/656: dread d0/d4/f17 [0,4194304] 0 2024-03-22T06:15:28.012 INFO:tasks.workunit.client.1.smithi117.stdout:4/657: fdatasync d0/d19/d34/d43/d63/d6f/d70/f7d 0 2024-03-22T06:15:28.016 INFO:tasks.workunit.client.1.smithi117.stdout:9/733: dread d4/d8/d3e/f8f [0,4194304] 0 2024-03-22T06:15:28.017 INFO:tasks.workunit.client.1.smithi117.stdout:3/684: symlink d9/d1a/d39/d7d/d97/dc9/lec 0 2024-03-22T06:15:28.025 INFO:tasks.workunit.client.1.smithi117.stdout:0/694: link d1/d3/d2b/d3e/de/d1f/d59/ca3 d1/d10/d5c/d40/cf5 0 2024-03-22T06:15:28.026 INFO:tasks.workunit.client.1.smithi117.stdout:1/743: getdents d1/d1f/d32/dc6 0 2024-03-22T06:15:28.027 INFO:tasks.workunit.client.1.smithi117.stdout:9/734: dread d4/d8/d93/db1/dd1/dd6/d58/f8a [0,4194304] 0 2024-03-22T06:15:28.029 INFO:tasks.workunit.client.1.smithi117.stdout:5/737: truncate dd/df/d88/fa9 14092 0 2024-03-22T06:15:28.029 INFO:tasks.workunit.client.1.smithi117.stdout:5/738: creat dd/df/d1e/f103 x:0 0 0 2024-03-22T06:15:28.029 INFO:tasks.workunit.client.1.smithi117.stdout:2/654: mknod d2/d5/db/cda 0 2024-03-22T06:15:28.030 INFO:tasks.workunit.client.1.smithi117.stdout:2/655: read d2/d5/d13/f7a [266758,54540] 0 2024-03-22T06:15:28.030 INFO:tasks.workunit.client.1.smithi117.stdout:2/656: creat d2/d5/db/d2c/d7d/fdb x:0 0 0 2024-03-22T06:15:28.030 INFO:tasks.workunit.client.1.smithi117.stdout:2/657: fsync d2/d5/f33 0 2024-03-22T06:15:28.031 INFO:tasks.workunit.client.1.smithi117.stdout:4/658: mkdir d0/d48/de4 0 2024-03-22T06:15:28.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:27 smithi006 ceph-mon[29301]: pgmap v27: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 224 MiB/s rd, 269 MiB/s wr, 496 op/s 2024-03-22T06:15:28.038 INFO:tasks.workunit.client.1.smithi117.stdout:6/745: rename d3/db4/d2d/d42 to d3/db4/dd4/df4/d101 0 2024-03-22T06:15:28.038 INFO:tasks.workunit.client.1.smithi117.stdout:6/746: fsync d3/d40/d46/d84/f7b 0 2024-03-22T06:15:28.039 INFO:tasks.workunit.client.1.smithi117.stdout:6/747: truncate d3/d40/f95 777131 0 2024-03-22T06:15:28.039 INFO:tasks.workunit.client.1.smithi117.stdout:6/748: write d3/db4/d2d/d53/d59/d74/da/db3/f6d [2611864,71156] 0 2024-03-22T06:15:28.044 INFO:tasks.workunit.client.1.smithi117.stdout:0/695: symlink d1/d10/d18/db0/lf6 0 2024-03-22T06:15:28.047 INFO:tasks.workunit.client.1.smithi117.stdout:9/735: truncate d4/d7/f26 1212294 0 2024-03-22T06:15:28.053 INFO:tasks.workunit.client.1.smithi117.stdout:2/658: truncate d2/d5/d13/d57/dd0/d1d/f6d 3152111 0 2024-03-22T06:15:28.056 INFO:tasks.workunit.client.1.smithi117.stdout:4/659: mkdir d0/d19/d34/d43/d63/d6f/de5 0 2024-03-22T06:15:28.058 INFO:tasks.workunit.client.1.smithi117.stdout:7/669: rename d8/d9/c2a to d8/d9/d37/d5f/dce/cd4 0 2024-03-22T06:15:28.062 INFO:tasks.workunit.client.1.smithi117.stdout:6/749: truncate d3/db4/d2d/d53/fd6 1138800 0 2024-03-22T06:15:28.062 INFO:tasks.workunit.client.1.smithi117.stdout:6/750: fdatasync d3/db4/dd4/df4/d101/da2/f7a 0 2024-03-22T06:15:28.064 INFO:tasks.workunit.client.1.smithi117.stdout:4/660: dread d0/f25 [0,4194304] 0 2024-03-22T06:15:28.071 INFO:tasks.workunit.client.1.smithi117.stdout:1/744: rmdir d1/d1f 39 2024-03-22T06:15:28.074 INFO:tasks.workunit.client.1.smithi117.stdout:9/736: creat d4/d8/ff3 x:0 0 0 2024-03-22T06:15:28.074 INFO:tasks.workunit.client.1.smithi117.stdout:9/737: write d4/d7/d27/f97 [5167786,32536] 0 2024-03-22T06:15:28.079 INFO:tasks.workunit.client.1.smithi117.stdout:5/739: rename dd/d16/d5a/d66 to dd/df/d5e/d91/d72/d35/d81/d104 0 2024-03-22T06:15:28.081 INFO:tasks.workunit.client.1.smithi117.stdout:6/751: dwrite d3/db4/d2d/d53/d59/d74/da/fc6 [4194304,4194304] 0 2024-03-22T06:15:28.081 INFO:tasks.workunit.client.1.smithi117.stdout:5/740: dread dd/df/d5e/d91/d72/f45 [4194304,4194304] 0 2024-03-22T06:15:28.082 INFO:tasks.workunit.client.1.smithi117.stdout:7/670: creat d8/d9/da/d10/fd5 x:0 0 0 2024-03-22T06:15:28.085 INFO:tasks.workunit.client.1.smithi117.stdout:4/661: creat d0/d51/fe6 x:0 0 0 2024-03-22T06:15:28.090 INFO:tasks.workunit.client.1.smithi117.stdout:8/699: sync 2024-03-22T06:15:28.091 INFO:tasks.workunit.client.1.smithi117.stdout:9/738: mknod d4/d47/d64/d72/cf4 0 2024-03-22T06:15:28.096 INFO:tasks.workunit.client.1.smithi117.stdout:0/696: rename d1/d10/d18/d26/d53/f82 to d1/d10/d5c/d40/dac/ff7 0 2024-03-22T06:15:28.096 INFO:tasks.workunit.client.1.smithi117.stdout:0/697: creat d1/d10/d18/d26/d49/ff8 x:0 0 0 2024-03-22T06:15:28.100 INFO:tasks.workunit.client.1.smithi117.stdout:6/752: truncate d3/db4/d2d/d53/d59/d74/f11 14130 0 2024-03-22T06:15:28.100 INFO:tasks.workunit.client.1.smithi117.stdout:6/753: write d3/d40/d46/d84/d49/d79/fcf [5207819,67509] 0 2024-03-22T06:15:28.100 INFO:tasks.workunit.client.1.smithi117.stdout:7/671: unlink d8/d9/d37/l40 0 2024-03-22T06:15:28.102 INFO:tasks.workunit.client.1.smithi117.stdout:2/659: rename d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/l25 to d2/d5/db/d2c/ldc 0 2024-03-22T06:15:28.106 INFO:tasks.workunit.client.1.smithi117.stdout:6/754: unlink d3/d40/d46/d84/f7b 0 2024-03-22T06:15:28.106 INFO:tasks.workunit.client.1.smithi117.stdout:5/741: dwrite dd/df/d5e/d91/f5d [0,4194304] 0 2024-03-22T06:15:28.108 INFO:tasks.workunit.client.1.smithi117.stdout:1/745: rename d1/d22/d27/d4e/d92 to d1/d22/d106 0 2024-03-22T06:15:28.110 INFO:tasks.workunit.client.1.smithi117.stdout:2/660: getdents d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f 0 2024-03-22T06:15:28.115 INFO:tasks.workunit.client.1.smithi117.stdout:9/739: rename d4/d47/d64/d72/d8d/ceb to d4/d8/d93/db1/dd1/dd6/d58/cf5 0 2024-03-22T06:15:28.116 INFO:tasks.workunit.client.1.smithi117.stdout:6/755: symlink d3/l102 0 2024-03-22T06:15:28.117 INFO:tasks.workunit.client.1.smithi117.stdout:4/662: dwrite d0/d48/da0/fb7 [0,4194304] 0 2024-03-22T06:15:28.118 INFO:tasks.workunit.client.1.smithi117.stdout:9/740: link d4/d8/d3e/d7a/da9/fc7 d4/d8/d3e/d7a/ff6 0 2024-03-22T06:15:28.120 INFO:tasks.workunit.client.1.smithi117.stdout:8/700: dwrite d1/d28/f7d [0,4194304] 0 2024-03-22T06:15:28.120 INFO:tasks.workunit.client.1.smithi117.stdout:8/701: stat d1/d19/d38/f42 0 2024-03-22T06:15:28.120 INFO:tasks.workunit.client.1.smithi117.stdout:8/702: creat d1/d3a/d44/d9/fdb x:0 0 0 2024-03-22T06:15:28.120 INFO:tasks.workunit.client.1.smithi117.stdout:8/703: read - d1/d3a/d44/d6a/db7/fd6 zero size 2024-03-22T06:15:28.120 INFO:tasks.workunit.client.1.smithi117.stdout:8/704: fdatasync d1/d19/d38/f70 0 2024-03-22T06:15:28.120 INFO:tasks.workunit.client.1.smithi117.stdout:6/756: mknod d3/db4/c103 0 2024-03-22T06:15:28.123 INFO:tasks.workunit.client.1.smithi117.stdout:1/746: write d1/d22/d27/d4e/f89 [556155,102540] 0 2024-03-22T06:15:28.123 INFO:tasks.workunit.client.1.smithi117.stdout:1/747: fdatasync d1/d14/f19 0 2024-03-22T06:15:28.124 INFO:tasks.workunit.client.1.smithi117.stdout:4/663: symlink d0/d19/d1c/le7 0 2024-03-22T06:15:28.125 INFO:tasks.workunit.client.1.smithi117.stdout:4/664: stat d0/d19/d1c 0 2024-03-22T06:15:28.125 INFO:tasks.workunit.client.1.smithi117.stdout:2/661: write d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/f65 [2783314,18371] 0 2024-03-22T06:15:28.125 INFO:tasks.workunit.client.1.smithi117.stdout:2/662: write d2/d5/db/d2c/d7d/fb6 [147042,72355] 0 2024-03-22T06:15:28.125 INFO:tasks.workunit.client.1.smithi117.stdout:2/663: chown d2/d35/d38 0 1 2024-03-22T06:15:28.125 INFO:tasks.workunit.client.1.smithi117.stdout:2/664: read d2/d5/d13/d57/dd0/d1d/d3a/d89/fab [29773,112435] 0 2024-03-22T06:15:28.125 INFO:tasks.workunit.client.1.smithi117.stdout:9/741: rename d4/d47/d64/dc5/ldd to d4/d47/d85/dab/lf7 0 2024-03-22T06:15:28.126 INFO:tasks.workunit.client.1.smithi117.stdout:7/672: dwrite d8/d9/d37/f4e [0,4194304] 0 2024-03-22T06:15:28.126 INFO:tasks.workunit.client.1.smithi117.stdout:7/673: chown d8/d9/d37/d45/d61/f6f 829220 1 2024-03-22T06:15:28.126 INFO:tasks.workunit.client.1.smithi117.stdout:7/674: readlink l1 0 2024-03-22T06:15:28.129 INFO:tasks.workunit.client.1.smithi117.stdout:5/742: dwrite dd/f7d [0,4194304] 0 2024-03-22T06:15:28.129 INFO:tasks.workunit.client.1.smithi117.stdout:5/743: creat dd/df/d5e/d91/d36/d70/f105 x:0 0 0 2024-03-22T06:15:28.131 INFO:tasks.workunit.client.1.smithi117.stdout:6/757: link d3/d40/d46/d84/c4c d3/db4/d2d/d53/d59/d74/d28/ddb/de7/df7/c104 0 2024-03-22T06:15:28.131 INFO:tasks.workunit.client.1.smithi117.stdout:6/758: fsync d3/db4/d2d/d53/d59/d74/d28/d85/f8a 0 2024-03-22T06:15:28.131 INFO:tasks.workunit.client.1.smithi117.stdout:6/759: creat d3/d40/f105 x:0 0 0 2024-03-22T06:15:28.133 INFO:tasks.workunit.client.1.smithi117.stdout:1/748: mkdir d1/d30/d44/d9c/d107 0 2024-03-22T06:15:28.133 INFO:tasks.workunit.client.1.smithi117.stdout:1/749: write d1/d22/f4b [3947555,113385] 0 2024-03-22T06:15:28.136 INFO:tasks.workunit.client.1.smithi117.stdout:6/760: dread d3/db4/dd4/df4/d101/da2/f7a [4194304,4194304] 0 2024-03-22T06:15:28.137 INFO:tasks.workunit.client.1.smithi117.stdout:6/761: dread d3/d61/f9b [0,4194304] 0 2024-03-22T06:15:28.137 INFO:tasks.workunit.client.1.smithi117.stdout:6/762: fsync d3/db4/d2d/d53/ff2 0 2024-03-22T06:15:28.137 INFO:tasks.workunit.client.1.smithi117.stdout:4/665: mkdir d0/d19/d1c/de8 0 2024-03-22T06:15:28.142 INFO:tasks.workunit.client.1.smithi117.stdout:2/665: mkdir d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d45/ddd 0 2024-03-22T06:15:28.145 INFO:tasks.workunit.client.1.smithi117.stdout:8/705: rename d1/d19/f51 to d1/d28/d6b/fdc 0 2024-03-22T06:15:28.146 INFO:tasks.workunit.client.1.smithi117.stdout:8/706: chown d1/d3a/d44/d6a 256 1 2024-03-22T06:15:28.147 INFO:tasks.workunit.client.1.smithi117.stdout:7/675: dwrite d8/fb [4194304,4194304] 0 2024-03-22T06:15:28.149 INFO:tasks.workunit.client.1.smithi117.stdout:7/676: dread d8/d9/dd1/fc5 [0,4194304] 0 2024-03-22T06:15:28.154 INFO:tasks.workunit.client.1.smithi117.stdout:5/744: mkdir dd/d16/d106 0 2024-03-22T06:15:28.155 INFO:tasks.workunit.client.1.smithi117.stdout:7/677: dread d8/d9/dd1/f81 [0,4194304] 0 2024-03-22T06:15:28.159 INFO:tasks.workunit.client.1.smithi117.stdout:1/750: symlink d1/d30/d35/d58/l108 0 2024-03-22T06:15:28.164 INFO:tasks.workunit.client.1.smithi117.stdout:2/666: mkdir d2/d5/d13/dde 0 2024-03-22T06:15:28.164 INFO:tasks.workunit.client.1.smithi117.stdout:2/667: readlink d2/d5/d13/d57/dd0/d1d/d3a/d89/la1 0 2024-03-22T06:15:28.165 INFO:tasks.workunit.client.1.smithi117.stdout:1/751: dread d1/d3c/d4d/de8/fbe [0,4194304] 0 2024-03-22T06:15:28.170 INFO:tasks.workunit.client.1.smithi117.stdout:3/685: sync 2024-03-22T06:15:28.172 INFO:tasks.workunit.client.1.smithi117.stdout:4/666: rename d0/c8b to d0/d4/d2a/ce9 0 2024-03-22T06:15:28.172 INFO:tasks.workunit.client.1.smithi117.stdout:4/667: read - d0/d51/fe6 zero size 2024-03-22T06:15:28.172 INFO:tasks.workunit.client.1.smithi117.stdout:4/668: dread d0/d19/d34/d43/d63/d6f/d6a/f8c [0,4194304] 0 2024-03-22T06:15:28.174 INFO:tasks.workunit.client.1.smithi117.stdout:8/707: symlink d1/d28/ldd 0 2024-03-22T06:15:28.174 INFO:tasks.workunit.client.1.smithi117.stdout:8/708: fdatasync d1/d3a/d44/f8d 0 2024-03-22T06:15:28.176 INFO:tasks.workunit.client.1.smithi117.stdout:5/745: mkdir dd/df/d5e/d91/d72/dde/de0/d107 0 2024-03-22T06:15:28.177 INFO:tasks.workunit.client.1.smithi117.stdout:7/678: mknod d8/d9/d37/d45/d61/cd6 0 2024-03-22T06:15:28.178 INFO:tasks.workunit.client.1.smithi117.stdout:7/679: write d8/d9/da/d10/d23/fbb [890630,31106] 0 2024-03-22T06:15:28.181 INFO:tasks.workunit.client.1.smithi117.stdout:2/668: mknod d2/d35/d38/cdf 0 2024-03-22T06:15:28.181 INFO:tasks.workunit.client.1.smithi117.stdout:8/709: dread d1/d28/d6b/fdc [0,4194304] 0 2024-03-22T06:15:28.188 INFO:tasks.workunit.client.1.smithi117.stdout:3/686: mknod d9/d1a/d39/d4c/d37/d69/ced 0 2024-03-22T06:15:28.189 INFO:tasks.workunit.client.1.smithi117.stdout:4/669: creat d0/dc0/fea x:0 0 0 2024-03-22T06:15:28.191 INFO:tasks.workunit.client.1.smithi117.stdout:5/746: rmdir dd/df/dae 39 2024-03-22T06:15:28.193 INFO:tasks.workunit.client.1.smithi117.stdout:8/710: mknod d1/d3a/d44/d9/cde 0 2024-03-22T06:15:28.196 INFO:tasks.workunit.client.1.smithi117.stdout:2/669: dwrite d2/d5/d13/d57/dd0/d1d/d3a/d89/f93 [0,4194304] 0 2024-03-22T06:15:28.196 INFO:tasks.workunit.client.1.smithi117.stdout:2/670: chown d2/d35/d4c/cd5 9 1 2024-03-22T06:15:28.196 INFO:tasks.workunit.client.1.smithi117.stdout:2/671: chown d2/d5/d13/f30 394 1 2024-03-22T06:15:28.198 INFO:tasks.workunit.client.1.smithi117.stdout:1/752: rename d1/f7 to d1/d1f/d32/d33/da4/f109 0 2024-03-22T06:15:28.198 INFO:tasks.workunit.client.1.smithi117.stdout:1/753: creat d1/d30/d44/d99/f10a x:0 0 0 2024-03-22T06:15:28.200 INFO:tasks.workunit.client.1.smithi117.stdout:4/670: symlink d0/d4/leb 0 2024-03-22T06:15:28.202 INFO:tasks.workunit.client.1.smithi117.stdout:5/747: creat dd/df/d5e/d91/d72/dde/f108 x:0 0 0 2024-03-22T06:15:28.202 INFO:tasks.workunit.client.1.smithi117.stdout:5/748: fsync dd/df/d5e/d91/d72/d33/d9a/fc3 0 2024-03-22T06:15:28.202 INFO:tasks.workunit.client.1.smithi117.stdout:5/749: write dd/df/d5e/d91/d72/dde/f108 [809618,32450] 0 2024-03-22T06:15:28.207 INFO:tasks.workunit.client.1.smithi117.stdout:2/672: creat d2/d5/d13/dde/fe0 x:0 0 0 2024-03-22T06:15:28.215 INFO:tasks.workunit.client.1.smithi117.stdout:1/754: mknod d1/d30/d44/dd3/c10b 0 2024-03-22T06:15:28.216 INFO:tasks.workunit.client.1.smithi117.stdout:5/750: symlink dd/df/d5e/d91/d72/d35/l109 0 2024-03-22T06:15:28.218 INFO:tasks.workunit.client.1.smithi117.stdout:2/673: creat d2/d5/d13/fe1 x:0 0 0 2024-03-22T06:15:28.219 INFO:tasks.workunit.client.1.smithi117.stdout:1/755: creat d1/d3c/d4d/de8/f10c x:0 0 0 2024-03-22T06:15:28.219 INFO:tasks.workunit.client.1.smithi117.stdout:1/756: getdents d1/d30/d44/d99/dd7 0 2024-03-22T06:15:28.220 INFO:tasks.workunit.client.1.smithi117.stdout:5/751: read dd/df/d95/da0/fdb [368543,13455] 0 2024-03-22T06:15:28.225 INFO:tasks.workunit.client.1.smithi117.stdout:8/711: dwrite d1/d28/f31 [0,4194304] 0 2024-03-22T06:15:28.225 INFO:tasks.workunit.client.1.smithi117.stdout:8/712: stat d1/d19/f43 0 2024-03-22T06:15:28.226 INFO:tasks.workunit.client.1.smithi117.stdout:1/757: dread d1/d30/fa8 [0,4194304] 0 2024-03-22T06:15:28.226 INFO:tasks.workunit.client.1.smithi117.stdout:1/758: chown d1/d1f/d32/dc6/fd0 305132 1 2024-03-22T06:15:28.228 INFO:tasks.workunit.client.1.smithi117.stdout:4/671: dwrite d0/d19/f98 [0,4194304] 0 2024-03-22T06:15:28.235 INFO:tasks.workunit.client.1.smithi117.stdout:8/713: unlink d1/d3a/l6e 0 2024-03-22T06:15:28.237 INFO:tasks.workunit.client.1.smithi117.stdout:1/759: mknod d1/d62/de4/c10d 0 2024-03-22T06:15:28.240 INFO:tasks.workunit.client.1.smithi117.stdout:4/672: creat d0/d4/d2a/d2f/fec x:0 0 0 2024-03-22T06:15:28.241 INFO:tasks.workunit.client.1.smithi117.stdout:6/763: sync 2024-03-22T06:15:28.241 INFO:tasks.workunit.client.1.smithi117.stdout:0/698: sync 2024-03-22T06:15:28.241 INFO:tasks.workunit.client.1.smithi117.stdout:9/742: sync 2024-03-22T06:15:28.242 INFO:tasks.workunit.client.1.smithi117.stdout:9/743: chown d4/d8/f2c 522395805 1 2024-03-22T06:15:28.243 INFO:tasks.workunit.client.1.smithi117.stdout:1/760: creat d1/d30/d44/dd3/f10e x:0 0 0 2024-03-22T06:15:28.244 INFO:tasks.workunit.client.1.smithi117.stdout:1/761: creat d1/d3c/d4d/d5b/f10f x:0 0 0 2024-03-22T06:15:28.244 INFO:tasks.workunit.client.1.smithi117.stdout:1/762: dread - d1/f76 zero size 2024-03-22T06:15:28.244 INFO:tasks.workunit.client.1.smithi117.stdout:4/673: creat d0/d19/d34/d43/d63/d8d/fed x:0 0 0 2024-03-22T06:15:28.245 INFO:tasks.workunit.client.1.smithi117.stdout:8/714: getdents d1/d19/d98 0 2024-03-22T06:15:28.246 INFO:tasks.workunit.client.1.smithi117.stdout:6/764: rename d3/d3d/d17/cfb to d3/db4/dd4/df4/d101/da2/c106 0 2024-03-22T06:15:28.248 INFO:tasks.workunit.client.1.smithi117.stdout:9/744: dread d4/d8/d3e/f7e [0,4194304] 0 2024-03-22T06:15:28.248 INFO:tasks.workunit.client.1.smithi117.stdout:0/699: mkdir d1/d10/d5c/d40/d4e/dbc/ddb/df9 0 2024-03-22T06:15:28.251 INFO:tasks.workunit.client.1.smithi117.stdout:8/715: getdents d1/d3a/d44/d11/d97 0 2024-03-22T06:15:28.252 INFO:tasks.workunit.client.1.smithi117.stdout:8/716: fsync d1/f77 0 2024-03-22T06:15:28.252 INFO:tasks.workunit.client.1.smithi117.stdout:8/717: readlink d1/d3a/d44/d16/l4f 0 2024-03-22T06:15:28.254 INFO:tasks.workunit.client.1.smithi117.stdout:9/745: rmdir d4/d7/d27/de7/dc4 0 2024-03-22T06:15:28.255 INFO:tasks.workunit.client.1.smithi117.stdout:9/746: truncate d4/d47/d85/fa8 443961 0 2024-03-22T06:15:28.255 INFO:tasks.workunit.client.1.smithi117.stdout:9/747: write d4/d8/d3e/d7a/f42 [4958209,130902] 0 2024-03-22T06:15:28.255 INFO:tasks.workunit.client.1.smithi117.stdout:0/700: symlink d1/d10/d18/d26/d53/lfa 0 2024-03-22T06:15:28.257 INFO:tasks.workunit.client.1.smithi117.stdout:4/674: getdents d0/d48 0 2024-03-22T06:15:28.257 INFO:tasks.workunit.client.1.smithi117.stdout:4/675: write d0/d19/dcc/fd4 [1021641,13351] 0 2024-03-22T06:15:28.257 INFO:tasks.workunit.client.1.smithi117.stdout:5/752: dwrite dd/df/d5e/d91/d72/d35/f27 [0,4194304] 0 2024-03-22T06:15:28.257 INFO:tasks.workunit.client.1.smithi117.stdout:5/753: chown dd/df/c58 94635350 1 2024-03-22T06:15:28.259 INFO:tasks.workunit.client.1.smithi117.stdout:8/718: symlink d1/d3a/d44/d11/d97/ldf 0 2024-03-22T06:15:28.259 INFO:tasks.workunit.client.1.smithi117.stdout:2/674: dwrite d2/d6e/f84 [0,4194304] 0 2024-03-22T06:15:28.263 INFO:tasks.workunit.client.1.smithi117.stdout:6/765: dread d3/d40/f22 [0,4194304] 0 2024-03-22T06:15:28.263 INFO:tasks.workunit.client.1.smithi117.stdout:9/748: mkdir d4/d8/d93/db1/dd1/df8 0 2024-03-22T06:15:28.265 INFO:tasks.workunit.client.1.smithi117.stdout:0/701: creat d1/d10/d18/d26/d49/d7e/ffb x:0 0 0 2024-03-22T06:15:28.283 INFO:tasks.workunit.client.1.smithi117.stdout:2/675: creat d2/d5/fe2 x:0 0 0 2024-03-22T06:15:28.287 INFO:tasks.workunit.client.1.smithi117.stdout:1/763: rename d1/c6 to d1/d22/d27/d4e/c110 0 2024-03-22T06:15:28.287 INFO:tasks.workunit.client.1.smithi117.stdout:4/676: dwrite d0/d19/d34/d43/f62 [0,4194304] 0 2024-03-22T06:15:28.287 INFO:tasks.workunit.client.1.smithi117.stdout:4/677: getdents d0/d51/d92 0 2024-03-22T06:15:28.292 INFO:tasks.workunit.client.1.smithi117.stdout:4/678: rmdir d0/d48/da0/dd2/db6 39 2024-03-22T06:15:28.293 INFO:tasks.workunit.client.1.smithi117.stdout:0/702: rmdir d1/d10/d18/d26/d49/d7e/d39/d6a 39 2024-03-22T06:15:28.296 INFO:tasks.workunit.client.1.smithi117.stdout:4/679: dread d0/fd9 [0,4194304] 0 2024-03-22T06:15:28.297 INFO:tasks.workunit.client.1.smithi117.stdout:0/703: symlink d1/d3/dae/lfc 0 2024-03-22T06:15:28.297 INFO:tasks.workunit.client.1.smithi117.stdout:0/704: read - d1/d10/d5c/d40/d4e/dbc/fc6 zero size 2024-03-22T06:15:28.299 INFO:tasks.workunit.client.1.smithi117.stdout:4/680: creat d0/d48/fee x:0 0 0 2024-03-22T06:15:28.299 INFO:tasks.workunit.client.1.smithi117.stdout:4/681: fdatasync d0/f3 0 2024-03-22T06:15:28.302 INFO:tasks.workunit.client.1.smithi117.stdout:0/705: unlink d1/d10/d5c/d40/f73 0 2024-03-22T06:15:28.303 INFO:tasks.workunit.client.1.smithi117.stdout:4/682: mknod d0/d19/d34/cef 0 2024-03-22T06:15:28.304 INFO:tasks.workunit.client.1.smithi117.stdout:6/766: dwrite d3/db4/fc5 [0,4194304] 0 2024-03-22T06:15:28.304 INFO:tasks.workunit.client.1.smithi117.stdout:8/719: dwrite d1/d28/d66/f74 [0,4194304] 0 2024-03-22T06:15:28.305 INFO:tasks.workunit.client.1.smithi117.stdout:0/706: link d1/d10/c21 d1/d3/d2b/d3e/dad/cfd 0 2024-03-22T06:15:28.305 INFO:tasks.workunit.client.1.smithi117.stdout:0/707: write d1/d3/d2b/d3e/de/fb8 [685260,64973] 0 2024-03-22T06:15:28.305 INFO:tasks.workunit.client.1.smithi117.stdout:0/708: fdatasync d1/d10/d5c/d40/d4e/dbc/fc6 0 2024-03-22T06:15:28.311 INFO:tasks.workunit.client.1.smithi117.stdout:6/767: stat d3/d40/d46/c50 0 2024-03-22T06:15:28.314 INFO:tasks.workunit.client.1.smithi117.stdout:4/683: write d0/d19/d34/f37 [517184,54567] 0 2024-03-22T06:15:28.316 INFO:tasks.workunit.client.1.smithi117.stdout:1/764: dwrite d1/d30/fa8 [0,4194304] 0 2024-03-22T06:15:28.316 INFO:tasks.workunit.client.1.smithi117.stdout:2/676: dwrite d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/f15 [4194304,4194304] 0 2024-03-22T06:15:28.317 INFO:tasks.workunit.client.1.smithi117.stdout:8/720: mkdir d1/d19/de0 0 2024-03-22T06:15:28.317 INFO:tasks.workunit.client.1.smithi117.stdout:8/721: fsync d1/d3a/d44/d4b/dc9/fd7 0 2024-03-22T06:15:28.322 INFO:tasks.workunit.client.1.smithi117.stdout:0/709: rename d1/d3/d2b/d3e/fb to d1/d3/d2b/d3e/ffe 0 2024-03-22T06:15:28.322 INFO:tasks.workunit.client.1.smithi117.stdout:0/710: creat d1/d3/d2b/d3e/de/d1f/fff x:0 0 0 2024-03-22T06:15:28.327 INFO:tasks.workunit.client.1.smithi117.stdout:3/687: sync 2024-03-22T06:15:28.327 INFO:tasks.workunit.client.1.smithi117.stdout:7/680: sync 2024-03-22T06:15:28.327 INFO:tasks.workunit.client.1.smithi117.stdout:7/681: readlink l1 0 2024-03-22T06:15:28.327 INFO:tasks.workunit.client.1.smithi117.stdout:7/682: chown d8/d9/da/d2b/d9b/da5 32823584 1 2024-03-22T06:15:28.327 INFO:tasks.workunit.client.1.smithi117.stdout:3/688: creat d9/d1a/d39/d4c/d99/fee x:0 0 0 2024-03-22T06:15:28.329 INFO:tasks.workunit.client.1.smithi117.stdout:4/684: rename d0/d48/da0/fb7 to d0/d48/da0/dd2/db6/dde/ff0 0 2024-03-22T06:15:28.329 INFO:tasks.workunit.client.1.smithi117.stdout:4/685: readlink d0/d19/d34/d43/l4e 0 2024-03-22T06:15:28.338 INFO:tasks.workunit.client.1.smithi117.stdout:1/765: link d1/d30/d44/lda d1/d3c/l111 0 2024-03-22T06:15:28.345 INFO:tasks.workunit.client.1.smithi117.stdout:6/768: mkdir d3/d61/d73/d107 0 2024-03-22T06:15:28.345 INFO:tasks.workunit.client.1.smithi117.stdout:6/769: fdatasync d3/db4/dd4/df4/d101/da2/feb 0 2024-03-22T06:15:28.350 INFO:tasks.workunit.client.1.smithi117.stdout:7/683: symlink d8/d6c/ld7 0 2024-03-22T06:15:28.350 INFO:tasks.workunit.client.1.smithi117.stdout:7/684: write d8/d9/d37/d45/d88/fbf [1075012,58278] 0 2024-03-22T06:15:28.350 INFO:tasks.workunit.client.1.smithi117.stdout:3/689: mkdir d9/d1a/d39/d4c/d99/def 0 2024-03-22T06:15:28.350 INFO:tasks.workunit.client.1.smithi117.stdout:3/690: stat d9/d1a/f34 0 2024-03-22T06:15:28.353 INFO:tasks.workunit.client.1.smithi117.stdout:2/677: rename d2/d35/d38/c40 to d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/ce3 0 2024-03-22T06:15:28.354 INFO:tasks.workunit.client.1.smithi117.stdout:0/711: dwrite d1/d3/f12 [0,4194304] 0 2024-03-22T06:15:28.358 INFO:tasks.workunit.client.1.smithi117.stdout:2/678: creat d2/d5/d13/d57/dd0/d1d/d3a/da6/fe4 x:0 0 0 2024-03-22T06:15:28.358 INFO:tasks.workunit.client.1.smithi117.stdout:2/679: read - d2/d5/d41/fb4 zero size 2024-03-22T06:15:28.360 INFO:tasks.workunit.client.1.smithi117.stdout:4/686: dwrite d0/d19/d34/d43/fce [0,4194304] 0 2024-03-22T06:15:28.361 INFO:tasks.workunit.client.1.smithi117.stdout:0/712: rename d1/d10/d18/d26/d49/d7e/d39/f52 to d1/d10/d5c/d40/df4/f100 0 2024-03-22T06:15:28.363 INFO:tasks.workunit.client.1.smithi117.stdout:7/685: dread d8/f16 [0,4194304] 0 2024-03-22T06:15:28.363 INFO:tasks.workunit.client.1.smithi117.stdout:8/722: dwrite d1/fac [0,4194304] 0 2024-03-22T06:15:28.363 INFO:tasks.workunit.client.1.smithi117.stdout:8/723: chown d1/d3a/d44/d9/fc3 395045576 1 2024-03-22T06:15:28.366 INFO:tasks.workunit.client.1.smithi117.stdout:3/691: write d9/d1a/d39/d4c/d99/fd3 [161241,23197] 0 2024-03-22T06:15:28.366 INFO:tasks.workunit.client.1.smithi117.stdout:3/692: dread - d9/d1a/d39/d4c/d36/d41/f65 zero size 2024-03-22T06:15:28.371 INFO:tasks.workunit.client.1.smithi117.stdout:1/766: dwrite d1/d1f/f75 [0,4194304] 0 2024-03-22T06:15:28.372 INFO:tasks.workunit.client.1.smithi117.stdout:0/713: read d1/d3/f12 [2479617,80874] 0 2024-03-22T06:15:28.373 INFO:tasks.workunit.client.1.smithi117.stdout:1/767: dread d1/d3c/d4d/d5b/fe0 [0,4194304] 0 2024-03-22T06:15:28.375 INFO:tasks.workunit.client.1.smithi117.stdout:6/770: rmdir d3/db4 39 2024-03-22T06:15:28.377 INFO:tasks.workunit.client.1.smithi117.stdout:2/680: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d45/d88/fe5 x:0 0 0 2024-03-22T06:15:28.377 INFO:tasks.workunit.client.1.smithi117.stdout:2/681: dread - d2/d5/d41/f8a zero size 2024-03-22T06:15:28.378 INFO:tasks.workunit.client.1.smithi117.stdout:5/754: sync 2024-03-22T06:15:28.378 INFO:tasks.workunit.client.1.smithi117.stdout:5/755: chown dd/d16/fe3 747 1 2024-03-22T06:15:28.383 INFO:tasks.workunit.client.1.smithi117.stdout:7/686: creat d8/d9/d37/d45/d88/db1/fd8 x:0 0 0 2024-03-22T06:15:28.385 INFO:tasks.workunit.client.1.smithi117.stdout:8/724: mknod d1/d3a/ce1 0 2024-03-22T06:15:28.385 INFO:tasks.workunit.client.1.smithi117.stdout:8/725: stat d1/d3a/d44/d11 0 2024-03-22T06:15:28.388 INFO:tasks.workunit.client.1.smithi117.stdout:3/693: symlink d9/d1a/d39/dc7/lf0 0 2024-03-22T06:15:28.388 INFO:tasks.workunit.client.1.smithi117.stdout:0/714: write d1/d10/f6c [3432974,94906] 0 2024-03-22T06:15:28.388 INFO:tasks.workunit.client.1.smithi117.stdout:0/715: fdatasync d1/d3/d2b/fb2 0 2024-03-22T06:15:28.388 INFO:tasks.workunit.client.1.smithi117.stdout:1/768: rename d1/d62/d7c to d1/d30/d44/d9c/da9/de3/d112 0 2024-03-22T06:15:28.389 INFO:tasks.workunit.client.1.smithi117.stdout:1/769: truncate d1/d30/d35/d58/f9e 1117543 0 2024-03-22T06:15:28.390 INFO:tasks.workunit.client.1.smithi117.stdout:6/771: mkdir d3/db4/d108 0 2024-03-22T06:15:28.390 INFO:tasks.workunit.client.1.smithi117.stdout:6/772: truncate d3/db4/d2d/d53/d59/d74/da/de/fa3 1029915 0 2024-03-22T06:15:28.390 INFO:tasks.workunit.client.1.smithi117.stdout:7/687: dread d8/d9/d37/d45/d61/f71 [0,4194304] 0 2024-03-22T06:15:28.390 INFO:tasks.workunit.client.1.smithi117.stdout:4/687: dwrite d0/d48/da0/dd2/db6/dde/ff0 [0,4194304] 0 2024-03-22T06:15:28.391 INFO:tasks.workunit.client.1.smithi117.stdout:4/688: dread d0/d19/d34/f77 [0,4194304] 0 2024-03-22T06:15:28.391 INFO:tasks.workunit.client.1.smithi117.stdout:7/688: write d8/d9/da/d10/f8d [961641,94095] 0 2024-03-22T06:15:28.391 INFO:tasks.workunit.client.1.smithi117.stdout:7/689: creat d8/d9/da/d2b/d9b/fd9 x:0 0 0 2024-03-22T06:15:28.392 INFO:tasks.workunit.client.1.smithi117.stdout:7/690: chown d8/d9/da/d10/f8d 7763 1 2024-03-22T06:15:28.392 INFO:tasks.workunit.client.1.smithi117.stdout:7/691: write d8/d9/d37/d45/d88/db1/fd8 [994208,126041] 0 2024-03-22T06:15:28.393 INFO:tasks.workunit.client.1.smithi117.stdout:5/756: truncate dd/df/d95/da0/fdb 1173000 0 2024-03-22T06:15:28.393 INFO:tasks.workunit.client.1.smithi117.stdout:8/726: mknod d1/d28/d6b/d91/ce2 0 2024-03-22T06:15:28.393 INFO:tasks.workunit.client.1.smithi117.stdout:8/727: write d1/d3a/d44/d4b/dc9/fda [392035,125772] 0 2024-03-22T06:15:28.393 INFO:tasks.workunit.client.1.smithi117.stdout:8/728: chown d1/d3a/d44/d16/l96 1571922 1 2024-03-22T06:15:28.394 INFO:tasks.workunit.client.1.smithi117.stdout:3/694: mknod d9/d1a/d39/d4c/d99/def/cf1 0 2024-03-22T06:15:28.401 INFO:tasks.workunit.client.1.smithi117.stdout:2/682: rename d2/d5/d13/d57/dd0/d1d/f8f to d2/d5/d41/fe6 0 2024-03-22T06:15:28.401 INFO:tasks.workunit.client.1.smithi117.stdout:2/683: truncate d2/d5/d13/d57/dd0/fc8 849986 0 2024-03-22T06:15:28.404 INFO:tasks.workunit.client.1.smithi117.stdout:4/689: symlink d0/d19/d34/d43/d63/d8d/lf1 0 2024-03-22T06:15:28.404 INFO:tasks.workunit.client.1.smithi117.stdout:6/773: write d3/db4/f4e [1112348,126207] 0 2024-03-22T06:15:28.408 INFO:tasks.workunit.client.1.smithi117.stdout:5/757: write dd/df/d5e/d91/d72/d35/da2/fcf [3716173,59756] 0 2024-03-22T06:15:28.408 INFO:tasks.workunit.client.1.smithi117.stdout:7/692: creat d8/d9/d37/d5f/fda x:0 0 0 2024-03-22T06:15:28.410 INFO:tasks.workunit.client.1.smithi117.stdout:8/729: unlink d1/d19/f43 0 2024-03-22T06:15:28.412 INFO:tasks.workunit.client.1.smithi117.stdout:3/695: creat d9/ff2 x:0 0 0 2024-03-22T06:15:28.412 INFO:tasks.workunit.client.1.smithi117.stdout:3/696: chown d9/d1a/d39/d7d/d97 13545 1 2024-03-22T06:15:28.413 INFO:tasks.workunit.client.1.smithi117.stdout:0/716: rmdir d1/d3/d2b/d3e/de/d1f/d3b/d55 39 2024-03-22T06:15:28.413 INFO:tasks.workunit.client.1.smithi117.stdout:9/749: sync 2024-03-22T06:15:28.414 INFO:tasks.workunit.client.1.smithi117.stdout:1/770: dwrite d1/d84/d77/fb9 [0,4194304] 0 2024-03-22T06:15:28.417 INFO:tasks.workunit.client.1.smithi117.stdout:2/684: mknod d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d45/ce7 0 2024-03-22T06:15:28.420 INFO:tasks.workunit.client.1.smithi117.stdout:4/690: rename d0/d19/d34/d43/d63/d8d/fcd to d0/d4/d2a/ff2 0 2024-03-22T06:15:28.422 INFO:tasks.workunit.client.1.smithi117.stdout:6/774: creat d3/d3d/d17/d31/df5/dfa/f109 x:0 0 0 2024-03-22T06:15:28.426 INFO:tasks.workunit.client.1.smithi117.stdout:5/758: mknod dd/df/d1e/c10a 0 2024-03-22T06:15:28.429 INFO:tasks.workunit.client.1.smithi117.stdout:7/693: mkdir d8/d9/db0/ddb 0 2024-03-22T06:15:28.433 INFO:tasks.workunit.client.1.smithi117.stdout:8/730: symlink d1/d28/d6b/le3 0 2024-03-22T06:15:28.435 INFO:tasks.workunit.client.1.smithi117.stdout:0/717: creat d1/d3/d2b/d3e/de/d1f/ddd/f101 x:0 0 0 2024-03-22T06:15:28.437 INFO:tasks.workunit.client.1.smithi117.stdout:9/750: rmdir d4/d8/d3f 39 2024-03-22T06:15:28.437 INFO:tasks.workunit.client.1.smithi117.stdout:9/751: chown d4/d7/d27/f2e 0 1 2024-03-22T06:15:28.437 INFO:tasks.workunit.client.1.smithi117.stdout:9/752: stat d4/d50/dee 0 2024-03-22T06:15:28.440 INFO:tasks.workunit.client.1.smithi117.stdout:2/685: link d2/d5/d13/d57/dd0/d1d/d3a/d71/f9d d2/d5/d13/d57/dd0/d1d/fe8 0 2024-03-22T06:15:28.446 INFO:tasks.workunit.client.1.smithi117.stdout:4/691: dwrite d0/d19/f98 [4194304,4194304] 0 2024-03-22T06:15:28.456 INFO:tasks.workunit.client.1.smithi117.stdout:3/697: rename d9/d1a/d39/f96 to d9/d1a/d39/d8a/da1/db6/ff3 0 2024-03-22T06:15:28.457 INFO:tasks.workunit.client.1.smithi117.stdout:3/698: dread - d9/d66/fbd zero size 2024-03-22T06:15:28.457 INFO:tasks.workunit.client.1.smithi117.stdout:3/699: chown d9/d1a/d39/d4c/d37/d69/fa8 6567747 1 2024-03-22T06:15:28.462 INFO:tasks.workunit.client.1.smithi117.stdout:5/759: unlink dd/df/d1e/f103 0 2024-03-22T06:15:28.463 INFO:tasks.workunit.client.1.smithi117.stdout:7/694: mkdir d8/d9/da/d2b/dc4/dac/ddc 0 2024-03-22T06:15:28.463 INFO:tasks.workunit.client.1.smithi117.stdout:7/695: creat d8/d9/da/d10/d23/d27/d56/fdd x:0 0 0 2024-03-22T06:15:28.466 INFO:tasks.workunit.client.1.smithi117.stdout:8/731: creat d1/d28/d66/fe4 x:0 0 0 2024-03-22T06:15:28.467 INFO:tasks.workunit.client.1.smithi117.stdout:8/732: readlink d1/d3a/d44/d9/la4 0 2024-03-22T06:15:28.471 INFO:tasks.workunit.client.1.smithi117.stdout:0/718: mknod d1/d10/d18/d26/d53/c102 0 2024-03-22T06:15:28.475 INFO:tasks.workunit.client.1.smithi117.stdout:8/733: dread d1/f77 [0,4194304] 0 2024-03-22T06:15:28.480 INFO:tasks.workunit.client.1.smithi117.stdout:4/692: creat d0/d19/d34/ff3 x:0 0 0 2024-03-22T06:15:28.480 INFO:tasks.workunit.client.1.smithi117.stdout:4/693: readlink d0/d19/d28/d47/l8a 0 2024-03-22T06:15:28.487 INFO:tasks.workunit.client.1.smithi117.stdout:9/753: rename d4/d7/f26 to d4/d7/d27/de7/ff9 0 2024-03-22T06:15:28.494 INFO:tasks.workunit.client.1.smithi117.stdout:2/686: rename d2/d35/d4c/cd2 to d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d45/ddd/ce9 0 2024-03-22T06:15:28.494 INFO:tasks.workunit.client.1.smithi117.stdout:2/687: write d2/d5/db/d2c/f42 [2071230,81681] 0 2024-03-22T06:15:28.494 INFO:tasks.workunit.client.1.smithi117.stdout:2/688: fsync d2/d5/d13/f30 0 2024-03-22T06:15:28.497 INFO:tasks.workunit.client.1.smithi117.stdout:6/775: dwrite d3/d61/d73/fee [0,4194304] 0 2024-03-22T06:15:28.497 INFO:tasks.workunit.client.1.smithi117.stdout:6/776: write d3/db4/d2d/d53/d59/d74/da/de/f19 [1216308,109453] 0 2024-03-22T06:15:28.497 INFO:tasks.workunit.client.1.smithi117.stdout:6/777: chown d3/d40/d46/d84/d49/d79/d9d/lae 970307 1 2024-03-22T06:15:28.498 INFO:tasks.workunit.client.1.smithi117.stdout:7/696: dwrite d8/d9/d37/d5f/d82/f95 [0,4194304] 0 2024-03-22T06:15:28.498 INFO:tasks.workunit.client.1.smithi117.stdout:4/694: dwrite d0/d19/d34/d43/fe3 [0,4194304] 0 2024-03-22T06:15:28.498 INFO:tasks.workunit.client.1.smithi117.stdout:4/695: readlink d0/d19/d34/d43/l4e 0 2024-03-22T06:15:28.500 INFO:tasks.workunit.client.1.smithi117.stdout:3/700: rename d9/d1a/d39/d7d/l91 to d9/d93/lf4 0 2024-03-22T06:15:28.500 INFO:tasks.workunit.client.1.smithi117.stdout:3/701: chown d9/d1a/d39/d4c/d37/d69/f87 204078 1 2024-03-22T06:15:28.501 INFO:tasks.workunit.client.1.smithi117.stdout:3/702: write d9/d1a/d39/d7d/f7e [801265,127974] 0 2024-03-22T06:15:28.501 INFO:tasks.workunit.client.1.smithi117.stdout:3/703: stat d9/d1a/d3a/fe1 0 2024-03-22T06:15:28.501 INFO:tasks.workunit.client.1.smithi117.stdout:3/704: fsync d9/f35 0 2024-03-22T06:15:28.501 INFO:tasks.workunit.client.1.smithi117.stdout:2/689: creat d2/d5/d13/d57/dd0/d1d/d3a/fea x:0 0 0 2024-03-22T06:15:28.503 INFO:tasks.workunit.client.1.smithi117.stdout:3/705: dread d9/d1a/d3a/f7a [0,4194304] 0 2024-03-22T06:15:28.507 INFO:tasks.workunit.client.1.smithi117.stdout:6/778: write d3/db4/dd4/df4/d101/fca [3123838,130008] 0 2024-03-22T06:15:28.511 INFO:tasks.workunit.client.1.smithi117.stdout:3/706: write d9/fa6 [5937370,7449] 0 2024-03-22T06:15:28.511 INFO:tasks.workunit.client.1.smithi117.stdout:3/707: creat d9/d1a/d39/d4c/d37/dcc/ff5 x:0 0 0 2024-03-22T06:15:28.511 INFO:tasks.workunit.client.1.smithi117.stdout:3/708: chown d9/d1a/d39/d7d/d97 22859966 1 2024-03-22T06:15:28.521 INFO:tasks.workunit.client.1.smithi117.stdout:4/696: symlink d0/d19/d34/d43/d63/d6f/d70/lf4 0 2024-03-22T06:15:28.523 INFO:tasks.workunit.client.1.smithi117.stdout:8/734: rename d1/d28/fbe to d1/d3a/d44/d4b/dc9/fe5 0 2024-03-22T06:15:28.526 INFO:tasks.workunit.client.1.smithi117.stdout:1/771: sync 2024-03-22T06:15:28.526 INFO:tasks.workunit.client.1.smithi117.stdout:1/772: dread - d1/d30/d35/fc5 zero size 2024-03-22T06:15:28.526 INFO:tasks.workunit.client.1.smithi117.stdout:1/773: creat d1/d30/d44/d9c/da9/f113 x:0 0 0 2024-03-22T06:15:28.529 INFO:tasks.workunit.client.1.smithi117.stdout:6/779: mkdir d3/d3d/d17/d31/d10a 0 2024-03-22T06:15:28.529 INFO:tasks.workunit.client.1.smithi117.stdout:6/780: chown d3/d3d/f43 1 1 2024-03-22T06:15:28.529 INFO:tasks.workunit.client.1.smithi117.stdout:6/781: stat d3/d40/d46/d84/fed 0 2024-03-22T06:15:28.533 INFO:tasks.workunit.client.1.smithi117.stdout:3/709: unlink d9/d1a/d39/d4c/d36/l4d 0 2024-03-22T06:15:28.534 INFO:tasks.workunit.client.1.smithi117.stdout:9/754: rename d4/d8/d3e/d7a/d3b/l43 to d4/d8/d3f/lfa 0 2024-03-22T06:15:28.536 INFO:tasks.workunit.client.1.smithi117.stdout:4/697: dwrite d0/d19/d1c/f4d [4194304,4194304] 0 2024-03-22T06:15:28.537 INFO:tasks.workunit.client.1.smithi117.stdout:1/774: getdents d1/d1f 0 2024-03-22T06:15:28.538 INFO:tasks.workunit.client.1.smithi117.stdout:2/690: getdents d2/d6e 0 2024-03-22T06:15:28.547 INFO:tasks.workunit.client.1.smithi117.stdout:6/782: symlink d3/d40/l10b 0 2024-03-22T06:15:28.549 INFO:tasks.workunit.client.1.smithi117.stdout:3/710: getdents d9/d1a/d39/dc7 0 2024-03-22T06:15:28.551 INFO:tasks.workunit.client.1.smithi117.stdout:9/755: link d4/d7/c44 d4/d8/d93/db1/dd1/dd9/cfb 0 2024-03-22T06:15:28.558 INFO:tasks.workunit.client.1.smithi117.stdout:2/691: creat d2/d35/d38/feb x:0 0 0 2024-03-22T06:15:28.558 INFO:tasks.workunit.client.1.smithi117.stdout:4/698: dwrite d0/d19/d34/f37 [0,4194304] 0 2024-03-22T06:15:28.558 INFO:tasks.workunit.client.1.smithi117.stdout:4/699: stat d0/d48/da0/dd2 0 2024-03-22T06:15:28.558 INFO:tasks.workunit.client.1.smithi117.stdout:2/692: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/fec x:0 0 0 2024-03-22T06:15:28.558 INFO:tasks.workunit.client.1.smithi117.stdout:2/693: stat d2/d5/db/c81 0 2024-03-22T06:15:28.558 INFO:tasks.workunit.client.1.smithi117.stdout:2/694: stat d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/dc9 0 2024-03-22T06:15:28.558 INFO:tasks.workunit.client.1.smithi117.stdout:1/775: rename d1/d3c/d4d/d5b to d1/d22/d27/d114 0 2024-03-22T06:15:28.560 INFO:tasks.workunit.client.1.smithi117.stdout:4/700: creat d0/d19/d34/d43/d63/d6f/d5f/ff5 x:0 0 0 2024-03-22T06:15:28.564 INFO:tasks.workunit.client.1.smithi117.stdout:6/783: dread d3/d40/d46/f6b [0,4194304] 0 2024-03-22T06:15:28.564 INFO:tasks.workunit.client.1.smithi117.stdout:3/711: dread d9/d1a/d39/d4c/f42 [0,4194304] 0 2024-03-22T06:15:28.564 INFO:tasks.workunit.client.1.smithi117.stdout:3/712: chown d9/d1a/d3a/f56 87741 1 2024-03-22T06:15:28.566 INFO:tasks.workunit.client.1.smithi117.stdout:6/784: link d3/db4/dd4/df4/d101/da2/f2f d3/db4/d2d/d53/f10c 0 2024-03-22T06:15:28.569 INFO:tasks.workunit.client.1.smithi117.stdout:6/785: symlink d3/d3d/d17/d31/l10d 0 2024-03-22T06:15:28.571 INFO:tasks.workunit.client.1.smithi117.stdout:5/760: sync 2024-03-22T06:15:28.573 INFO:tasks.workunit.client.1.smithi117.stdout:6/786: symlink d3/d3d/d17/d31/l10e 0 2024-03-22T06:15:28.575 INFO:tasks.workunit.client.1.smithi117.stdout:3/713: write d9/d1a/f1e [976026,97218] 0 2024-03-22T06:15:28.578 INFO:tasks.workunit.client.1.smithi117.stdout:6/787: unlink d3/d3d/d17/d31/l10e 0 2024-03-22T06:15:28.578 INFO:tasks.workunit.client.1.smithi117.stdout:6/788: chown d3/d3d/d17/d31/df5/dfa 10836638 1 2024-03-22T06:15:28.578 INFO:tasks.workunit.client.1.smithi117.stdout:6/789: read d3/d40/d46/d84/d49/f5b [2288060,69181] 0 2024-03-22T06:15:28.578 INFO:tasks.workunit.client.1.smithi117.stdout:6/790: write d3/d3d/d17/d60/f80 [1012262,94876] 0 2024-03-22T06:15:28.581 INFO:tasks.workunit.client.1.smithi117.stdout:3/714: mknod d9/d1a/dc1/cf6 0 2024-03-22T06:15:28.581 INFO:tasks.workunit.client.1.smithi117.stdout:3/715: creat d9/d1a/d39/d4c/d36/ff7 x:0 0 0 2024-03-22T06:15:28.582 INFO:tasks.workunit.client.1.smithi117.stdout:9/756: dwrite d4/d8/d3e/d7a/d2f/f54 [0,4194304] 0 2024-03-22T06:15:28.582 INFO:tasks.workunit.client.1.smithi117.stdout:9/757: chown d4/d47/d85/dab/lf7 1 1 2024-03-22T06:15:28.582 INFO:tasks.workunit.client.1.smithi117.stdout:9/758: write d4/d8/d3e/d7a/d2f/f39 [1770651,37380] 0 2024-03-22T06:15:28.582 INFO:tasks.workunit.client.1.smithi117.stdout:6/791: mkdir d3/db4/d2d/d10f 0 2024-03-22T06:15:28.585 INFO:tasks.workunit.client.1.smithi117.stdout:3/716: dread d9/d93/fdc [0,4194304] 0 2024-03-22T06:15:28.585 INFO:tasks.workunit.client.1.smithi117.stdout:3/717: readlink d9/d1a/d39/d4c/l8b 0 2024-03-22T06:15:28.588 INFO:tasks.workunit.client.1.smithi117.stdout:6/792: creat d3/db4/d2d/f110 x:0 0 0 2024-03-22T06:15:28.591 INFO:tasks.workunit.client.1.smithi117.stdout:3/718: link d9/d93/lac d9/d1a/d39/d4c/d99/lf8 0 2024-03-22T06:15:28.595 INFO:tasks.workunit.client.1.smithi117.stdout:3/719: symlink d9/d1a/dc1/dd0/lf9 0 2024-03-22T06:15:28.595 INFO:tasks.workunit.client.1.smithi117.stdout:3/720: chown d9/d1a/d3a/f47 37 1 2024-03-22T06:15:28.597 INFO:tasks.workunit.client.1.smithi117.stdout:3/721: mknod d9/d1a/d39/cfa 0 2024-03-22T06:15:28.600 INFO:tasks.workunit.client.1.smithi117.stdout:3/722: mknod d9/d1a/d39/d4c/d37/d69/cfb 0 2024-03-22T06:15:28.600 INFO:tasks.workunit.client.1.smithi117.stdout:6/793: dread d3/db4/fc5 [0,4194304] 0 2024-03-22T06:15:28.600 INFO:tasks.workunit.client.1.smithi117.stdout:3/723: dread - d9/d1a/d39/d4c/d37/dcc/ff5 zero size 2024-03-22T06:15:28.603 INFO:tasks.workunit.client.1.smithi117.stdout:3/724: rename d9/fa to d9/d1a/d39/d82/ffc 0 2024-03-22T06:15:28.603 INFO:tasks.workunit.client.1.smithi117.stdout:3/725: truncate d9/d1a/d39/d7d/fdf 959627 0 2024-03-22T06:15:28.604 INFO:tasks.workunit.client.1.smithi117.stdout:1/776: dwrite d1/d30/d44/d99/fb7 [0,4194304] 0 2024-03-22T06:15:28.605 INFO:tasks.workunit.client.1.smithi117.stdout:2/695: dwrite d2/fba [0,4194304] 0 2024-03-22T06:15:28.605 INFO:tasks.workunit.client.1.smithi117.stdout:5/761: dwrite dd/df/d5e/d91/d72/d60/f6b [4194304,4194304] 0 2024-03-22T06:15:28.605 INFO:tasks.workunit.client.1.smithi117.stdout:3/726: rmdir d9/d1a/dc1 39 2024-03-22T06:15:28.605 INFO:tasks.workunit.client.1.smithi117.stdout:3/727: write d9/f25 [4438033,15019] 0 2024-03-22T06:15:28.605 INFO:tasks.workunit.client.1.smithi117.stdout:5/762: write dd/df/d5e/d91/f7e [4539343,76763] 0 2024-03-22T06:15:28.606 INFO:tasks.workunit.client.1.smithi117.stdout:3/728: truncate d9/d1a/d39/d82/ffc 4963555 0 2024-03-22T06:15:28.606 INFO:tasks.workunit.client.1.smithi117.stdout:9/759: dwrite d4/d8/d93/db1/dd1/dd6/d58/f92 [0,4194304] 0 2024-03-22T06:15:28.606 INFO:tasks.workunit.client.1.smithi117.stdout:9/760: write d4/d8/d93/db1/dd1/dd6/fb2 [308603,97013] 0 2024-03-22T06:15:28.606 INFO:tasks.workunit.client.1.smithi117.stdout:9/761: fdatasync d4/d7/d27/f60 0 2024-03-22T06:15:28.606 INFO:tasks.workunit.client.1.smithi117.stdout:9/762: fsync d4/d8/d3e/d7a/f6c 0 2024-03-22T06:15:28.607 INFO:tasks.workunit.client.1.smithi117.stdout:6/794: write d3/d3d/d17/d31/f87 [13692,67156] 0 2024-03-22T06:15:28.607 INFO:tasks.workunit.client.1.smithi117.stdout:4/701: dwrite d0/d19/f22 [0,4194304] 0 2024-03-22T06:15:28.607 INFO:tasks.workunit.client.1.smithi117.stdout:4/702: read d0/f42 [61026,27413] 0 2024-03-22T06:15:28.615 INFO:tasks.workunit.client.1.smithi117.stdout:4/703: dread d0/d19/d34/d43/d63/d6f/d6a/f9d [0,4194304] 0 2024-03-22T06:15:28.615 INFO:tasks.workunit.client.1.smithi117.stdout:4/704: fsync d0/d48/da0/dd2/db6/fcf 0 2024-03-22T06:15:28.615 INFO:tasks.workunit.client.1.smithi117.stdout:4/705: dread - d0/d19/d34/d43/d63/d6f/f91 zero size 2024-03-22T06:15:28.620 INFO:tasks.workunit.client.1.smithi117.stdout:6/795: write d3/d40/f22 [1508631,65410] 0 2024-03-22T06:15:28.620 INFO:tasks.workunit.client.1.smithi117.stdout:6/796: dread - d3/d40/f105 zero size 2024-03-22T06:15:28.620 INFO:tasks.workunit.client.1.smithi117.stdout:6/797: chown d3/db4/d2d/d53/d59/d74/d28/d85/ca5 260055012 1 2024-03-22T06:15:28.620 INFO:tasks.workunit.client.1.smithi117.stdout:6/798: dread - d3/d3d/d17/f9a zero size 2024-03-22T06:15:28.635 INFO:tasks.workunit.client.1.smithi117.stdout:1/777: dwrite d1/d22/ff8 [4194304,4194304] 0 2024-03-22T06:15:28.636 INFO:tasks.workunit.client.1.smithi117.stdout:5/763: dwrite dd/d16/d1a/d32/dbf/fe2 [0,4194304] 0 2024-03-22T06:15:28.636 INFO:tasks.workunit.client.1.smithi117.stdout:9/763: getdents d4/d8/d3e/d7a/da9 0 2024-03-22T06:15:28.644 INFO:tasks.workunit.client.1.smithi117.stdout:5/764: write dd/d16/d1a/fe9 [890361,27148] 0 2024-03-22T06:15:28.644 INFO:tasks.workunit.client.1.smithi117.stdout:1/778: dread d1/d22/d27/faf [0,4194304] 0 2024-03-22T06:15:28.652 INFO:tasks.workunit.client.1.smithi117.stdout:9/764: dwrite d4/d8/d3f/f52 [0,4194304] 0 2024-03-22T06:15:28.656 INFO:tasks.workunit.client.1.smithi117.stdout:2/696: mknod d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d77/ced 0 2024-03-22T06:15:28.658 INFO:tasks.workunit.client.1.smithi117.stdout:6/799: creat d3/db4/dd4/df4/d101/f111 x:0 0 0 2024-03-22T06:15:28.661 INFO:tasks.workunit.client.1.smithi117.stdout:3/729: rename d9/d1a/d39/d4c/d36/ff7 to d9/d1a/d39/ffd 0 2024-03-22T06:15:28.661 INFO:tasks.workunit.client.1.smithi117.stdout:3/730: fdatasync d9/d1a/d39/d4c/d37/dcc/fe0 0 2024-03-22T06:15:28.661 INFO:tasks.workunit.client.1.smithi117.stdout:3/731: readlink d9/d1a/d39/d4c/d37/l5c 0 2024-03-22T06:15:28.661 INFO:tasks.workunit.client.1.smithi117.stdout:3/732: dread - d9/d1a/d39/d4c/d36/fb2 zero size 2024-03-22T06:15:28.661 INFO:tasks.workunit.client.1.smithi117.stdout:3/733: chown d9/d1a/d39/d4c/l6a 1716545840 1 2024-03-22T06:15:28.662 INFO:tasks.workunit.client.1.smithi117.stdout:1/779: dwrite d1/d3c/d4d/de8/fbe [0,4194304] 0 2024-03-22T06:15:28.662 INFO:tasks.workunit.client.1.smithi117.stdout:1/780: readlink d1/d22/l2e 0 2024-03-22T06:15:28.675 INFO:tasks.workunit.client.1.smithi117.stdout:5/765: link dd/d16/d86/dee/lfb dd/d16/d1a/d32/d73/db8/l10b 0 2024-03-22T06:15:28.677 INFO:tasks.workunit.client.1.smithi117.stdout:4/706: getdents d0/d51 0 2024-03-22T06:15:28.677 INFO:tasks.workunit.client.1.smithi117.stdout:7/697: sync 2024-03-22T06:15:28.678 INFO:tasks.workunit.client.1.smithi117.stdout:0/719: sync 2024-03-22T06:15:28.678 INFO:tasks.workunit.client.1.smithi117.stdout:0/720: chown d1/d9d 29414 1 2024-03-22T06:15:28.678 INFO:tasks.workunit.client.1.smithi117.stdout:8/735: sync 2024-03-22T06:15:28.678 INFO:tasks.workunit.client.1.smithi117.stdout:0/721: dread - d1/d10/d5c/d40/d4e/dbc/ddb/fea zero size 2024-03-22T06:15:28.678 INFO:tasks.workunit.client.1.smithi117.stdout:0/722: dread - d1/d10/f86 zero size 2024-03-22T06:15:28.678 INFO:tasks.workunit.client.1.smithi117.stdout:0/723: creat d1/d10/d18/d26/d49/f103 x:0 0 0 2024-03-22T06:15:28.681 INFO:tasks.workunit.client.1.smithi117.stdout:2/697: creat d2/d6e/fee x:0 0 0 2024-03-22T06:15:28.681 INFO:tasks.workunit.client.1.smithi117.stdout:2/698: write d2/d5/f33 [2066452,103139] 0 2024-03-22T06:15:28.681 INFO:tasks.workunit.client.1.smithi117.stdout:3/734: dwrite d9/d1a/d39/d4c/d36/f50 [0,4194304] 0 2024-03-22T06:15:28.683 INFO:tasks.workunit.client.1.smithi117.stdout:6/800: link d3/db4/d2d/d53/d59/d74/d28/ddb/de7/df7/c104 d3/db4/d2d/d53/d59/d74/da/de/c112 0 2024-03-22T06:15:28.684 INFO:tasks.workunit.client.1.smithi117.stdout:6/801: truncate d3/d40/d46/d84/d49/d79/d9d/fce 493624 0 2024-03-22T06:15:28.684 INFO:tasks.workunit.client.1.smithi117.stdout:6/802: write d3/db4/fd2 [575627,92276] 0 2024-03-22T06:15:28.687 INFO:tasks.workunit.client.1.smithi117.stdout:9/765: rename d4/d7/d27/f5a to d4/d47/d64/d72/d8d/ffc 0 2024-03-22T06:15:28.687 INFO:tasks.workunit.client.1.smithi117.stdout:9/766: truncate d4/d46/f95 1787769 0 2024-03-22T06:15:28.687 INFO:tasks.workunit.client.1.smithi117.stdout:9/767: chown d4/d7/d27/d5b/dae/fea 15 1 2024-03-22T06:15:28.691 INFO:tasks.workunit.client.1.smithi117.stdout:2/699: dread d2/d5/d13/d57/fc3 [0,4194304] 0 2024-03-22T06:15:28.691 INFO:tasks.workunit.client.1.smithi117.stdout:1/781: truncate d1/d22/d27/d4e/f89 3122693 0 2024-03-22T06:15:28.692 INFO:tasks.workunit.client.1.smithi117.stdout:5/766: creat dd/df/d5e/d91/d72/d35/d81/d104/f10c x:0 0 0 2024-03-22T06:15:28.692 INFO:tasks.workunit.client.1.smithi117.stdout:5/767: stat dd/d16/d1a/d32/d73/db8 0 2024-03-22T06:15:28.694 INFO:tasks.workunit.client.1.smithi117.stdout:4/707: mknod d0/d48/da0/cf6 0 2024-03-22T06:15:28.695 INFO:tasks.workunit.client.1.smithi117.stdout:7/698: symlink d8/d9/d37/d5f/d82/lde 0 2024-03-22T06:15:28.700 INFO:tasks.workunit.client.1.smithi117.stdout:0/724: mkdir d1/d10/d18/d8c/d104 0 2024-03-22T06:15:28.701 INFO:tasks.workunit.client.1.smithi117.stdout:8/736: dwrite d1/d28/f7d [0,4194304] 0 2024-03-22T06:15:28.702 INFO:tasks.workunit.client.1.smithi117.stdout:6/803: dwrite d3/db4/d2d/d53/d59/f88 [0,4194304] 0 2024-03-22T06:15:28.705 INFO:tasks.workunit.client.1.smithi117.stdout:7/699: dread d8/d9/dd1/f4d [0,4194304] 0 2024-03-22T06:15:28.705 INFO:tasks.workunit.client.1.smithi117.stdout:7/700: read - d8/d9/da/d10/d23/f92 zero size 2024-03-22T06:15:28.711 INFO:tasks.workunit.client.1.smithi117.stdout:3/735: creat d9/d1a/d39/d4c/d36/d41/ffe x:0 0 0 2024-03-22T06:15:28.711 INFO:tasks.workunit.client.1.smithi117.stdout:3/736: chown d9/d1a/d39/d7d/d97/dc9/lec 145 1 2024-03-22T06:15:28.714 INFO:tasks.workunit.client.1.smithi117.stdout:3/737: dread d9/d55/f7b [0,4194304] 0 2024-03-22T06:15:28.714 INFO:tasks.workunit.client.1.smithi117.stdout:9/768: stat d4/c18 0 2024-03-22T06:15:28.716 INFO:tasks.workunit.client.1.smithi117.stdout:2/700: getdents d2/d35 0 2024-03-22T06:15:28.717 INFO:tasks.workunit.client.1.smithi117.stdout:2/701: write d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/fb0 [331422,110788] 0 2024-03-22T06:15:28.718 INFO:tasks.workunit.client.1.smithi117.stdout:1/782: link d1/d84/cad d1/d30/d5d/d6f/c115 0 2024-03-22T06:15:28.718 INFO:tasks.workunit.client.1.smithi117.stdout:1/783: readlink d1/d3c/l45 0 2024-03-22T06:15:28.721 INFO:tasks.workunit.client.1.smithi117.stdout:5/768: link dd/df/d5e/d91/d72/d35/d81/d104/lb5 dd/df/d5e/d91/d72/d33/l10d 0 2024-03-22T06:15:28.721 INFO:tasks.workunit.client.1.smithi117.stdout:5/769: chown dd/df/d5e/d91/d90/dca/lea 6 1 2024-03-22T06:15:28.721 INFO:tasks.workunit.client.1.smithi117.stdout:3/738: read d9/d1a/d39/d4c/d36/f50 [3314054,9769] 0 2024-03-22T06:15:28.723 INFO:tasks.workunit.client.1.smithi117.stdout:4/708: rmdir d0/d19/d28 39 2024-03-22T06:15:28.723 INFO:tasks.workunit.client.1.smithi117.stdout:4/709: write d0/d4/f17 [4014643,125316] 0 2024-03-22T06:15:28.723 INFO:tasks.workunit.client.1.smithi117.stdout:4/710: creat d0/d19/d1c/ff7 x:0 0 0 2024-03-22T06:15:28.723 INFO:tasks.workunit.client.1.smithi117.stdout:0/725: symlink d1/d3/d2b/d3e/de/d1f/d59/l105 0 2024-03-22T06:15:28.724 INFO:tasks.workunit.client.1.smithi117.stdout:0/726: stat d1/d10/d5c/d40/d4e/dbc 0 2024-03-22T06:15:28.729 INFO:tasks.workunit.client.1.smithi117.stdout:8/737: creat d1/d19/d98/fe6 x:0 0 0 2024-03-22T06:15:28.730 INFO:tasks.workunit.client.1.smithi117.stdout:8/738: readlink d1/d3a/d44/d9/ld 0 2024-03-22T06:15:28.738 INFO:tasks.workunit.client.1.smithi117.stdout:6/804: symlink d3/d40/d46/d84/d49/d79/d9d/l113 0 2024-03-22T06:15:28.738 INFO:tasks.workunit.client.1.smithi117.stdout:6/805: write d3/db4/f4e [4313898,96879] 0 2024-03-22T06:15:28.738 INFO:tasks.workunit.client.1.smithi117.stdout:6/806: stat d3/db4/fc5 0 2024-03-22T06:15:28.742 INFO:tasks.workunit.client.1.smithi117.stdout:6/807: dread d3/db4/dd4/df4/d101/da2/f2f [0,4194304] 0 2024-03-22T06:15:28.743 INFO:tasks.workunit.client.1.smithi117.stdout:9/769: rename d4/d8/d3e/d7a/c9d to d4/d8/d93/db1/dd1/dd6/d9b/dd8/cfd 0 2024-03-22T06:15:28.744 INFO:tasks.workunit.client.1.smithi117.stdout:2/702: mknod d2/d5/d13/d57/dd0/d1d/d3a/d89/cef 0 2024-03-22T06:15:28.746 INFO:tasks.workunit.client.1.smithi117.stdout:1/784: creat d1/d30/d44/d99/dd7/f116 x:0 0 0 2024-03-22T06:15:28.746 INFO:tasks.workunit.client.1.smithi117.stdout:1/785: fdatasync d1/fc 0 2024-03-22T06:15:28.748 INFO:tasks.workunit.client.1.smithi117.stdout:5/770: link dd/d16/d1a/cf6 dd/df/d5e/d91/d72/d60/c10e 0 2024-03-22T06:15:28.748 INFO:tasks.workunit.client.1.smithi117.stdout:5/771: write dd/df/d95/fab [3017913,23983] 0 2024-03-22T06:15:28.749 INFO:tasks.workunit.client.1.smithi117.stdout:7/701: write d8/d9/da/f4b [199070,83056] 0 2024-03-22T06:15:28.749 INFO:tasks.workunit.client.1.smithi117.stdout:5/772: chown dd/df/d5e/d91/d90/dca/lea 4332922 1 2024-03-22T06:15:28.749 INFO:tasks.workunit.client.1.smithi117.stdout:7/702: dread - d8/d9/da/d10/d23/f92 zero size 2024-03-22T06:15:28.751 INFO:tasks.workunit.client.1.smithi117.stdout:4/711: unlink d0/d4/c16 0 2024-03-22T06:15:28.751 INFO:tasks.workunit.client.1.smithi117.stdout:4/712: chown d0/d48 3106 1 2024-03-22T06:15:28.752 INFO:tasks.workunit.client.1.smithi117.stdout:2/703: dread d2/d5/d13/d57/dd0/d1d/d3a/d71/f9d [0,4194304] 0 2024-03-22T06:15:28.753 INFO:tasks.workunit.client.1.smithi117.stdout:0/727: mknod d1/d3/d2b/d3e/de/d1f/d59/dcb/c106 0 2024-03-22T06:15:28.754 INFO:tasks.workunit.client.1.smithi117.stdout:6/808: getdents d3/d40/d46/d84/dbe 0 2024-03-22T06:15:28.755 INFO:tasks.workunit.client.1.smithi117.stdout:1/786: dread d1/d30/d5d/d6f/f95 [4194304,4194304] 0 2024-03-22T06:15:28.755 INFO:tasks.workunit.client.1.smithi117.stdout:1/787: truncate d1/d22/d27/f100 140948 0 2024-03-22T06:15:28.756 INFO:tasks.workunit.client.1.smithi117.stdout:3/739: rename d9/d1a/l2c to d9/d1a/d39/d4c/d37/lff 0 2024-03-22T06:15:28.756 INFO:tasks.workunit.client.1.smithi117.stdout:3/740: creat d9/d1a/d3a/f100 x:0 0 0 2024-03-22T06:15:28.756 INFO:tasks.workunit.client.1.smithi117.stdout:3/741: chown d9/d1a/d39/d7d/d97/lb1 5959 1 2024-03-22T06:15:28.757 INFO:tasks.workunit.client.1.smithi117.stdout:9/770: getdents d4/d8/d3f 0 2024-03-22T06:15:28.757 INFO:tasks.workunit.client.1.smithi117.stdout:9/771: dread - d4/d8/ff3 zero size 2024-03-22T06:15:28.757 INFO:tasks.workunit.client.1.smithi117.stdout:9/772: readlink d4/d7/d27/l90 0 2024-03-22T06:15:28.770 INFO:tasks.workunit.client.1.smithi117.stdout:7/703: symlink d8/d9/db0/ddb/ldf 0 2024-03-22T06:15:28.770 INFO:tasks.workunit.client.1.smithi117.stdout:7/704: chown d8/d9/da/d2b/dc4/f3b 45921 1 2024-03-22T06:15:28.770 INFO:tasks.workunit.client.1.smithi117.stdout:7/705: fsync d8/d9/da/d10/d23/d27/d56/fdd 0 2024-03-22T06:15:28.771 INFO:tasks.workunit.client.1.smithi117.stdout:0/728: symlink d1/d10/d18/d26/d49/d7e/d39/dc5/de8/l107 0 2024-03-22T06:15:28.772 INFO:tasks.workunit.client.1.smithi117.stdout:4/713: dwrite d0/dc0/fda [4194304,4194304] 0 2024-03-22T06:15:28.774 INFO:tasks.workunit.client.1.smithi117.stdout:6/809: mknod d3/d3d/d17/d31/df5/dfa/c114 0 2024-03-22T06:15:28.776 INFO:tasks.workunit.client.1.smithi117.stdout:1/788: symlink d1/d1f/l117 0 2024-03-22T06:15:28.779 INFO:tasks.workunit.client.1.smithi117.stdout:8/739: rename d1/d3a/d44/d6a/db7/fd2 to d1/d3a/d44/fe7 0 2024-03-22T06:15:28.782 INFO:tasks.workunit.client.1.smithi117.stdout:3/742: mkdir d9/d1a/d39/d4c/d37/d69/d101 0 2024-03-22T06:15:28.784 INFO:tasks.workunit.client.1.smithi117.stdout:8/740: dread d1/d3a/d44/d11/d97/f99 [0,4194304] 0 2024-03-22T06:15:28.784 INFO:tasks.workunit.client.1.smithi117.stdout:8/741: read d1/d28/d66/d7f/f94 [3748,82957] 0 2024-03-22T06:15:28.784 INFO:tasks.workunit.client.1.smithi117.stdout:8/742: fsync d1/d28/d66/d7f/f94 0 2024-03-22T06:15:28.784 INFO:tasks.workunit.client.1.smithi117.stdout:8/743: chown d1/d19/d98/cb2 210423801 1 2024-03-22T06:15:28.784 INFO:tasks.workunit.client.1.smithi117.stdout:8/744: dread - d1/d3a/d44/d6a/db7/fd6 zero size 2024-03-22T06:15:28.787 INFO:tasks.workunit.client.1.smithi117.stdout:5/773: rmdir dd/df/d5e/d91/d90 39 2024-03-22T06:15:28.787 INFO:tasks.workunit.client.1.smithi117.stdout:5/774: write dd/d16/f6c [1386753,110718] 0 2024-03-22T06:15:28.794 INFO:tasks.workunit.client.1.smithi117.stdout:7/706: dwrite d8/d6c/fc2 [0,4194304] 0 2024-03-22T06:15:28.795 INFO:tasks.workunit.client.1.smithi117.stdout:0/729: creat d1/d10/d5c/d40/dac/f108 x:0 0 0 2024-03-22T06:15:28.795 INFO:tasks.workunit.client.1.smithi117.stdout:0/730: creat d1/d10/d18/d26/d49/d7e/d39/dc5/f109 x:0 0 0 2024-03-22T06:15:28.795 INFO:tasks.workunit.client.1.smithi117.stdout:0/731: write d1/d10/d18/d26/d49/d7e/d39/d6a/fb3 [112550,68172] 0 2024-03-22T06:15:28.795 INFO:tasks.workunit.client.1.smithi117.stdout:0/732: creat d1/d10/d5c/d40/dbb/f10a x:0 0 0 2024-03-22T06:15:28.797 INFO:tasks.workunit.client.1.smithi117.stdout:4/714: mknod d0/d19/d28/cf8 0 2024-03-22T06:15:28.798 INFO:tasks.workunit.client.1.smithi117.stdout:6/810: creat d3/d3d/d17/d31/d10a/f115 x:0 0 0 2024-03-22T06:15:28.799 INFO:tasks.workunit.client.1.smithi117.stdout:2/704: rename d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/cb3 to d2/d5/d13/d57/dd0/d1d/d3a/cf0 0 2024-03-22T06:15:28.800 INFO:tasks.workunit.client.1.smithi117.stdout:3/743: mkdir d9/d1a/dc1/d102 0 2024-03-22T06:15:28.805 INFO:tasks.workunit.client.1.smithi117.stdout:5/775: mkdir dd/d16/d86/d93/d10f 0 2024-03-22T06:15:28.810 INFO:tasks.workunit.client.1.smithi117.stdout:7/707: mkdir d8/d9/d37/d45/d88/db1/dc6/de0 0 2024-03-22T06:15:28.811 INFO:tasks.workunit.client.1.smithi117.stdout:7/708: truncate d8/d9/dd1/fc5 1164926 0 2024-03-22T06:15:28.811 INFO:tasks.workunit.client.1.smithi117.stdout:2/705: write d2/d5/d13/d57/dd0/d1d/f6d [1379533,27973] 0 2024-03-22T06:15:28.816 INFO:tasks.workunit.client.1.smithi117.stdout:6/811: mknod d3/d61/c116 0 2024-03-22T06:15:28.818 INFO:tasks.workunit.client.1.smithi117.stdout:1/789: rename d1/d30/d44/lda to d1/d30/d44/d9c/d107/l118 0 2024-03-22T06:15:28.819 INFO:tasks.workunit.client.1.smithi117.stdout:8/745: rmdir d1/d3a/d44/d4b 39 2024-03-22T06:15:28.822 INFO:tasks.workunit.client.1.smithi117.stdout:1/790: mknod d1/d30/d44/d9c/da9/de3/d112/c119 0 2024-03-22T06:15:28.823 INFO:tasks.workunit.client.1.smithi117.stdout:6/812: truncate d3/d40/f22 912483 0 2024-03-22T06:15:28.823 INFO:tasks.workunit.client.1.smithi117.stdout:6/813: creat d3/d40/d46/f117 x:0 0 0 2024-03-22T06:15:28.823 INFO:tasks.workunit.client.1.smithi117.stdout:6/814: creat d3/d40/f118 x:0 0 0 2024-03-22T06:15:28.824 INFO:tasks.workunit.client.1.smithi117.stdout:2/706: dread d2/d5/d13/d57/f36 [0,4194304] 0 2024-03-22T06:15:28.824 INFO:tasks.workunit.client.1.smithi117.stdout:2/707: readlink d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/l6a 0 2024-03-22T06:15:28.824 INFO:tasks.workunit.client.1.smithi117.stdout:1/791: creat d1/d30/d5d/d6f/f11a x:0 0 0 2024-03-22T06:15:28.828 INFO:tasks.workunit.client.1.smithi117.stdout:2/708: link d2/d35/lcd d2/d5/da5/lf1 0 2024-03-22T06:15:28.832 INFO:tasks.workunit.client.1.smithi117.stdout:1/792: rmdir d1/d1f/d32/dc6 39 2024-03-22T06:15:28.834 INFO:tasks.workunit.client.1.smithi117.stdout:3/744: dwrite d9/d1a/fc6 [0,4194304] 0 2024-03-22T06:15:28.835 INFO:tasks.workunit.client.1.smithi117.stdout:5/776: dwrite dd/df/d5e/d91/d72/d60/f6b [4194304,4194304] 0 2024-03-22T06:15:28.836 INFO:tasks.workunit.client.1.smithi117.stdout:1/793: mknod d1/d22/d27/c11b 0 2024-03-22T06:15:28.836 INFO:tasks.workunit.client.1.smithi117.stdout:0/733: dwrite d1/d10/d18/f95 [0,4194304] 0 2024-03-22T06:15:28.836 INFO:tasks.workunit.client.1.smithi117.stdout:0/734: read d1/d10/d18/f38 [86029,118858] 0 2024-03-22T06:15:28.836 INFO:tasks.workunit.client.1.smithi117.stdout:0/735: readlink d1/d10/d18/d26/d53/l5b 0 2024-03-22T06:15:28.836 INFO:tasks.workunit.client.1.smithi117.stdout:0/736: dread - d1/d10/d5c/d40/f7f zero size 2024-03-22T06:15:28.837 INFO:tasks.workunit.client.1.smithi117.stdout:2/709: link d2/d5/da5/fa7 d2/d5/d13/d57/ff2 0 2024-03-22T06:15:28.837 INFO:tasks.workunit.client.1.smithi117.stdout:2/710: readlink d2/d5/d13/d57/dd0/l9e 0 2024-03-22T06:15:28.837 INFO:tasks.workunit.client.1.smithi117.stdout:2/711: dread - d2/d6e/fee zero size 2024-03-22T06:15:28.840 INFO:tasks.workunit.client.1.smithi117.stdout:5/777: getdents dd/d16/d1a/d32/dfc 0 2024-03-22T06:15:28.842 INFO:tasks.workunit.client.1.smithi117.stdout:4/715: write d0/d19/f49 [1248463,51886] 0 2024-03-22T06:15:28.844 INFO:tasks.workunit.client.1.smithi117.stdout:3/745: write d9/d1a/f20 [1114644,786] 0 2024-03-22T06:15:28.844 INFO:tasks.workunit.client.1.smithi117.stdout:3/746: fsync d9/f17 0 2024-03-22T06:15:28.847 INFO:tasks.workunit.client.1.smithi117.stdout:6/815: dwrite d3/d3d/dcb/dcc/ff6 [0,4194304] 0 2024-03-22T06:15:28.848 INFO:tasks.workunit.client.1.smithi117.stdout:1/794: symlink d1/d84/d77/l11c 0 2024-03-22T06:15:28.854 INFO:tasks.workunit.client.1.smithi117.stdout:0/737: rmdir d1/d3/dae/dbe 39 2024-03-22T06:15:28.854 INFO:tasks.workunit.client.1.smithi117.stdout:0/738: dread - d1/d3/d2b/d3e/f98 zero size 2024-03-22T06:15:28.854 INFO:tasks.workunit.client.1.smithi117.stdout:0/739: truncate d1/d10/d5c/d40/dbb/f10a 285730 0 2024-03-22T06:15:28.857 INFO:tasks.workunit.client.1.smithi117.stdout:6/816: dread d3/db4/d2d/d53/d75/f8e [0,4194304] 0 2024-03-22T06:15:28.858 INFO:tasks.workunit.client.1.smithi117.stdout:5/778: symlink dd/df/dae/l110 0 2024-03-22T06:15:28.860 INFO:tasks.workunit.client.1.smithi117.stdout:3/747: unlink d9/d66/fa9 0 2024-03-22T06:15:28.860 INFO:tasks.workunit.client.1.smithi117.stdout:4/716: dwrite d0/d19/d1c/fd5 [0,4194304] 0 2024-03-22T06:15:28.861 INFO:tasks.workunit.client.1.smithi117.stdout:2/712: getdents d2/d5/d13/d57 0 2024-03-22T06:15:28.865 INFO:tasks.workunit.client.1.smithi117.stdout:0/740: dread d1/d10/d5c/d40/df4/f100 [0,4194304] 0 2024-03-22T06:15:28.865 INFO:tasks.workunit.client.1.smithi117.stdout:0/741: chown d1/d10/d18/d26/d49/d7e/ffb 8240549 1 2024-03-22T06:15:28.865 INFO:tasks.workunit.client.1.smithi117.stdout:0/742: creat d1/d3/d2b/d3e/de/d1f/f10b x:0 0 0 2024-03-22T06:15:28.866 INFO:tasks.workunit.client.1.smithi117.stdout:0/743: write d1/d3/d2b/d3e/d3f/fd7 [432130,104972] 0 2024-03-22T06:15:28.867 INFO:tasks.workunit.client.1.smithi117.stdout:1/795: mkdir d1/d1f/d11d 0 2024-03-22T06:15:28.869 INFO:tasks.workunit.client.1.smithi117.stdout:6/817: creat d3/db4/d2d/d53/d59/d74/d28/ddb/dde/f119 x:0 0 0 2024-03-22T06:15:28.870 INFO:tasks.workunit.client.1.smithi117.stdout:5/779: write dd/df/f2a [1569094,80084] 0 2024-03-22T06:15:28.874 INFO:tasks.workunit.client.1.smithi117.stdout:9/773: sync 2024-03-22T06:15:28.874 INFO:tasks.workunit.client.1.smithi117.stdout:9/774: fsync d4/d47/d64/f84 0 2024-03-22T06:15:28.874 INFO:tasks.workunit.client.1.smithi117.stdout:9/775: write d4/d8/d93/db1/dd1/dd6/fb2 [208583,14513] 0 2024-03-22T06:15:28.879 INFO:tasks.workunit.client.1.smithi117.stdout:4/717: truncate d0/d19/d34/d43/d63/d6f/d5f/fcb 5691354 0 2024-03-22T06:15:28.882 INFO:tasks.workunit.client.1.smithi117.stdout:0/744: read d1/d10/d18/d26/d49/d7e/fe0 [4150866,76015] 0 2024-03-22T06:15:28.882 INFO:tasks.workunit.client.1.smithi117.stdout:0/745: chown d1/d10/d5c/d40/d69/fa2 894189808 1 2024-03-22T06:15:28.884 INFO:tasks.workunit.client.1.smithi117.stdout:2/713: unlink d2/d5/db/d2c/d7d/fdb 0 2024-03-22T06:15:28.890 INFO:tasks.workunit.client.1.smithi117.stdout:3/748: dwrite d9/d1a/d39/d7d/f98 [0,4194304] 0 2024-03-22T06:15:28.890 INFO:tasks.workunit.client.1.smithi117.stdout:3/749: readlink d9/d66/le6 0 2024-03-22T06:15:28.890 INFO:tasks.workunit.client.1.smithi117.stdout:6/818: dwrite d3/db4/dd4/df4/d101/da2/f2f [0,4194304] 0 2024-03-22T06:15:28.890 INFO:tasks.workunit.client.1.smithi117.stdout:3/750: write d9/d66/f6d [2037656,33909] 0 2024-03-22T06:15:28.892 INFO:tasks.workunit.client.1.smithi117.stdout:5/780: symlink dd/d16/d1a/d32/d73/l111 0 2024-03-22T06:15:28.892 INFO:tasks.workunit.client.1.smithi117.stdout:4/718: dread d0/d19/f4a [0,4194304] 0 2024-03-22T06:15:28.893 INFO:tasks.workunit.client.1.smithi117.stdout:9/776: link d4/d8/d3e/d7a/d3b/fec d4/d8/d3f/dc3/ffe 0 2024-03-22T06:15:28.895 INFO:tasks.workunit.client.1.smithi117.stdout:0/746: mkdir d1/d3/d2b/d3e/de/d1f/d10c 0 2024-03-22T06:15:28.896 INFO:tasks.workunit.client.1.smithi117.stdout:4/719: write d0/d51/f7e [1099355,84626] 0 2024-03-22T06:15:28.897 INFO:tasks.workunit.client.1.smithi117.stdout:2/714: symlink d2/d5/d13/d57/dd0/d1d/d3a/d89/lf3 0 2024-03-22T06:15:28.911 INFO:tasks.workunit.client.1.smithi117.stdout:9/777: dwrite d4/d7/d27/d5b/fb7 [0,4194304] 0 2024-03-22T06:15:28.912 INFO:tasks.workunit.client.1.smithi117.stdout:6/819: mkdir d3/db4/d2d/d53/d11a 0 2024-03-22T06:15:28.912 INFO:tasks.workunit.client.1.smithi117.stdout:6/820: readlink d3/d3d/dcb/dcc/le3 0 2024-03-22T06:15:28.913 INFO:tasks.workunit.client.1.smithi117.stdout:9/778: dread d4/d8/d3e/d7a/f28 [4194304,4194304] 0 2024-03-22T06:15:28.915 INFO:tasks.workunit.client.1.smithi117.stdout:3/751: mknod d9/d1a/d39/d8a/da1/c103 0 2024-03-22T06:15:28.919 INFO:tasks.workunit.client.1.smithi117.stdout:5/781: symlink dd/d16/d86/d93/dfe/l112 0 2024-03-22T06:15:28.919 INFO:tasks.workunit.client.1.smithi117.stdout:5/782: creat dd/df/d5e/d91/d72/f113 x:0 0 0 2024-03-22T06:15:28.919 INFO:tasks.workunit.client.1.smithi117.stdout:5/783: fsync dd/d16/d1a/d32/d73/db8/fc5 0 2024-03-22T06:15:28.931 INFO:tasks.workunit.client.1.smithi117.stdout:4/720: unlink d0/d3f/l81 0 2024-03-22T06:15:28.932 INFO:tasks.workunit.client.1.smithi117.stdout:7/709: sync 2024-03-22T06:15:28.932 INFO:tasks.workunit.client.1.smithi117.stdout:8/746: sync 2024-03-22T06:15:28.932 INFO:tasks.workunit.client.1.smithi117.stdout:0/747: creat d1/d10/f10d x:0 0 0 2024-03-22T06:15:28.934 INFO:tasks.workunit.client.1.smithi117.stdout:2/715: unlink d2/d5/d13/d57/l31 0 2024-03-22T06:15:28.934 INFO:tasks.workunit.client.1.smithi117.stdout:7/710: read d8/d9/da/d10/d23/d27/d56/f9e [453028,80872] 0 2024-03-22T06:15:28.935 INFO:tasks.workunit.client.1.smithi117.stdout:5/784: dwrite dd/df/d5e/d91/d36/f47 [4194304,4194304] 0 2024-03-22T06:15:28.937 INFO:tasks.workunit.client.1.smithi117.stdout:6/821: mkdir d3/db4/d2d/d53/d11a/d11b 0 2024-03-22T06:15:28.939 INFO:tasks.workunit.client.1.smithi117.stdout:9/779: unlink d4/d8/d93/db1/dd1/dd6/d58/la7 0 2024-03-22T06:15:28.941 INFO:tasks.workunit.client.1.smithi117.stdout:3/752: mkdir d9/d1a/d39/d4c/d104 0 2024-03-22T06:15:28.946 INFO:tasks.workunit.client.1.smithi117.stdout:4/721: creat d0/d19/d34/d43/d63/d6f/de5/ff9 x:0 0 0 2024-03-22T06:15:28.950 INFO:tasks.workunit.client.1.smithi117.stdout:8/747: link d1/d3a/d44/d6a/l83 d1/d3a/d44/d11/d97/le8 0 2024-03-22T06:15:28.952 INFO:tasks.workunit.client.1.smithi117.stdout:9/780: dread d4/d7/f24 [0,4194304] 0 2024-03-22T06:15:28.954 INFO:tasks.workunit.client.1.smithi117.stdout:5/785: dwrite dd/f29 [0,4194304] 0 2024-03-22T06:15:28.954 INFO:tasks.workunit.client.1.smithi117.stdout:2/716: dread - d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d77/f99 zero size 2024-03-22T06:15:28.957 INFO:tasks.workunit.client.1.smithi117.stdout:7/711: symlink d8/d9/da/d10/d23/d41/le1 0 2024-03-22T06:15:28.958 INFO:tasks.workunit.client.1.smithi117.stdout:7/712: chown d8/d9/c20 2083854 1 2024-03-22T06:15:28.962 INFO:tasks.workunit.client.1.smithi117.stdout:1/796: sync 2024-03-22T06:15:28.963 INFO:tasks.workunit.client.1.smithi117.stdout:1/797: write d1/d1f/d32/dc6/fd0 [249715,58227] 0 2024-03-22T06:15:28.965 INFO:tasks.workunit.client.1.smithi117.stdout:6/822: rename d3/d40/d46/d84/l94 to d3/db4/d2d/d53/d59/d74/d28/ddb/de7/df7/l11c 0 2024-03-22T06:15:28.966 INFO:tasks.workunit.client.1.smithi117.stdout:4/722: symlink d0/d19/d28/d47/dd6/lfa 0 2024-03-22T06:15:28.969 INFO:tasks.workunit.client.1.smithi117.stdout:8/748: mkdir d1/d19/d98/de9 0 2024-03-22T06:15:28.972 INFO:tasks.workunit.client.1.smithi117.stdout:5/786: truncate dd/df/d5e/d91/d90/dca/fd4 879471 0 2024-03-22T06:15:28.974 INFO:tasks.workunit.client.1.smithi117.stdout:4/723: stat d0/d4/d2a/d5b/f85 0 2024-03-22T06:15:28.974 INFO:tasks.workunit.client.1.smithi117.stdout:4/724: fdatasync d0/d19/d34/d43/d63/f74 0 2024-03-22T06:15:28.974 INFO:tasks.workunit.client.1.smithi117.stdout:4/725: chown d0/d19/f4a 23820705 1 2024-03-22T06:15:28.977 INFO:tasks.workunit.client.1.smithi117.stdout:5/787: unlink dd/df/d5e/d91/d72/d33/l52 0 2024-03-22T06:15:28.977 INFO:tasks.workunit.client.1.smithi117.stdout:8/749: truncate d1/d19/f86 213976 0 2024-03-22T06:15:28.977 INFO:tasks.workunit.client.1.smithi117.stdout:5/788: chown dd/df/d5e/d91/d72/d60/dec/ffa 445 1 2024-03-22T06:15:28.978 INFO:tasks.workunit.client.1.smithi117.stdout:6/823: read d3/db4/dd4/df4/d101/f76 [103130,89742] 0 2024-03-22T06:15:28.978 INFO:tasks.workunit.client.1.smithi117.stdout:6/824: readlink d3/db4/d2d/d53/lc9 0 2024-03-22T06:15:28.978 INFO:tasks.workunit.client.1.smithi117.stdout:7/713: rmdir d8/d9/da/d2b/d9b 39 2024-03-22T06:15:28.981 INFO:tasks.workunit.client.1.smithi117.stdout:8/750: creat d1/d3a/d44/d16/fea x:0 0 0 2024-03-22T06:15:28.981 INFO:tasks.workunit.client.1.smithi117.stdout:8/751: write d1/d3a/d44/d16/fea [954597,22589] 0 2024-03-22T06:15:28.981 INFO:tasks.workunit.client.1.smithi117.stdout:8/752: creat d1/d3a/feb x:0 0 0 2024-03-22T06:15:28.981 INFO:tasks.workunit.client.1.smithi117.stdout:6/825: mknod d3/d3d/d17/d31/d10a/c11d 0 2024-03-22T06:15:28.982 INFO:tasks.workunit.client.1.smithi117.stdout:7/714: creat d8/d9/da/d2b/dc4/dac/ddc/fe2 x:0 0 0 2024-03-22T06:15:28.985 INFO:tasks.workunit.client.1.smithi117.stdout:5/789: dread fa [0,4194304] 0 2024-03-22T06:15:28.985 INFO:tasks.workunit.client.1.smithi117.stdout:5/790: stat dd/d16/d1a/d32/f40 0 2024-03-22T06:15:28.986 INFO:tasks.workunit.client.1.smithi117.stdout:3/753: dwrite d9/d93/fdc [0,4194304] 0 2024-03-22T06:15:28.986 INFO:tasks.workunit.client.1.smithi117.stdout:8/753: rename d1/d19/d38/faa to d1/d3a/d44/d9/fec 0 2024-03-22T06:15:28.988 INFO:tasks.workunit.client.1.smithi117.stdout:4/726: write d0/d4/f7 [2392665,118962] 0 2024-03-22T06:15:28.988 INFO:tasks.workunit.client.1.smithi117.stdout:2/717: dwrite d2/d5/da5/fc7 [0,4194304] 0 2024-03-22T06:15:28.988 INFO:tasks.workunit.client.1.smithi117.stdout:0/748: dwrite d1/d3/d2b/d3e/de/d1f/d3b/fa6 [0,4194304] 0 2024-03-22T06:15:28.989 INFO:tasks.workunit.client.1.smithi117.stdout:9/781: dwrite d4/d8/d3e/d7a/d2f/f39 [0,4194304] 0 2024-03-22T06:15:28.989 INFO:tasks.workunit.client.1.smithi117.stdout:9/782: readlink d4/d47/lf2 0 2024-03-22T06:15:28.989 INFO:tasks.workunit.client.1.smithi117.stdout:9/783: write d4/f59 [282356,99951] 0 2024-03-22T06:15:28.990 INFO:tasks.workunit.client.1.smithi117.stdout:7/715: unlink d8/d9/da/d10/d23/d27/d56/fdd 0 2024-03-22T06:15:28.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.990+0000 7f858f162700 1 -- 172.21.15.6:0/3364475902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85880fbb30 msgr2=0x7f85880fbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:28.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.990+0000 7f858f162700 1 --2- 172.21.15.6:0/3364475902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85880fbb30 0x7f85880fbf40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8580009ce0 tx=0x7f8580004d10 comp rx=0 tx=0).stop 2024-03-22T06:15:28.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.991+0000 7f858f162700 1 -- 172.21.15.6:0/3364475902 shutdown_connections 2024-03-22T06:15:28.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.991+0000 7f858f162700 1 --2- 172.21.15.6:0/3364475902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85880fcde0 0x7f85880fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:28.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.991+0000 7f858f162700 1 --2- 172.21.15.6:0/3364475902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85880fbb30 0x7f85880fbf40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:28.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.991+0000 7f858f162700 1 -- 172.21.15.6:0/3364475902 >> 172.21.15.6:0/3364475902 conn(0x7f85880f7290 msgr2=0x7f85880f96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:28.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.991+0000 7f858f162700 1 -- 172.21.15.6:0/3364475902 shutdown_connections 2024-03-22T06:15:28.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.991+0000 7f858f162700 1 -- 172.21.15.6:0/3364475902 wait complete. 2024-03-22T06:15:28.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.991+0000 7f858f162700 1 Processor -- start 2024-03-22T06:15:28.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858f162700 1 -- start start 2024-03-22T06:15:28.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858f162700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85880fbb30 0x7f858810d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:28.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858f162700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85880fcde0 0x7f858810df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:28.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858f162700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858810e520 con 0x7f85880fcde0 2024-03-22T06:15:28.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858f162700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858810ed70 con 0x7f85880fbb30 2024-03-22T06:15:28.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858e160700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85880fbb30 0x7f858810d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:28.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858e160700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85880fbb30 0x7f858810d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:42786/0 (socket says 172.21.15.6:42786) 2024-03-22T06:15:28.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858e160700 1 -- 172.21.15.6:0/3466157172 learned_addr learned my addr 172.21.15.6:0/3466157172 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:15:28.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858d95f700 1 --2- 172.21.15.6:0/3466157172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85880fcde0 0x7f858810df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:28.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858e160700 1 -- 172.21.15.6:0/3466157172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85880fcde0 msgr2=0x7f858810df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:28.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858e160700 1 --2- 172.21.15.6:0/3466157172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85880fcde0 0x7f858810df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:28.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.992+0000 7f858e160700 1 -- 172.21.15.6:0/3466157172 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8580009940 con 0x7f85880fbb30 2024-03-22T06:15:28.995 INFO:tasks.workunit.client.1.smithi117.stdout:5/791: mknod dd/df/d1e/c114 0 2024-03-22T06:15:28.995 INFO:tasks.workunit.client.1.smithi117.stdout:1/798: dwrite d1/d22/d27/d4e/f89 [0,4194304] 0 2024-03-22T06:15:28.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.993+0000 7f858e160700 1 --2- 172.21.15.6:0/3466157172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85880fbb30 0x7f858810d9e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8580004e60 tx=0x7f8580004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:28.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.993+0000 7f857effd700 1 -- 172.21.15.6:0/3466157172 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f858000e040 con 0x7f85880fbb30 2024-03-22T06:15:28.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.993+0000 7f858f162700 1 -- 172.21.15.6:0/3466157172 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858810eff0 con 0x7f85880fbb30 2024-03-22T06:15:28.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.993+0000 7f857effd700 1 -- 172.21.15.6:0/3466157172 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f858001cb20 con 0x7f85880fbb30 2024-03-22T06:15:28.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.993+0000 7f857effd700 1 -- 172.21.15.6:0/3466157172 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8580012b90 con 0x7f85880fbb30 2024-03-22T06:15:28.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.993+0000 7f858f162700 1 -- 172.21.15.6:0/3466157172 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858810f530 con 0x7f85880fbb30 2024-03-22T06:15:28.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.994+0000 7f858f162700 1 -- 172.21.15.6:0/3466157172 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85881071c0 con 0x7f85880fbb30 2024-03-22T06:15:28.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.995+0000 7f857effd700 1 -- 172.21.15.6:0/3466157172 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f858001c650 con 0x7f85880fbb30 2024-03-22T06:15:28.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.996+0000 7f857effd700 1 --2- 172.21.15.6:0/3466157172 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f857406bbe0 0x7f857406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:28.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.996+0000 7f857effd700 1 -- 172.21.15.6:0/3466157172 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f858008cb20 con 0x7f85880fbb30 2024-03-22T06:15:28.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.997+0000 7f858d95f700 1 --2- 172.21.15.6:0/3466157172 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f857406bbe0 0x7f857406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:28.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:28.999+0000 7f858d95f700 1 --2- 172.21.15.6:0/3466157172 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f857406bbe0 0x7f857406e090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8584009b80 tx=0x7f858400b040 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:29.000 INFO:tasks.workunit.client.1.smithi117.stdout:8/754: rename d1/d3a/d44/d11/c8a to d1/d28/d6b/da3/ced 0 2024-03-22T06:15:29.000 INFO:tasks.workunit.client.1.smithi117.stdout:8/755: creat d1/d28/d66/fee x:0 0 0 2024-03-22T06:15:29.001 INFO:tasks.workunit.client.1.smithi117.stdout:0/749: write d1/d10/d18/d26/f75 [501080,84866] 0 2024-03-22T06:15:29.002 INFO:tasks.workunit.client.1.smithi117.stdout:6/826: mknod d3/d61/d73/c11e 0 2024-03-22T06:15:29.002 INFO:tasks.workunit.client.1.smithi117.stdout:6/827: dread - d3/d40/f118 zero size 2024-03-22T06:15:29.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.002+0000 7f857effd700 1 -- 172.21.15.6:0/3466157172 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f8580059db0 con 0x7f85880fbb30 2024-03-22T06:15:29.006 INFO:tasks.workunit.client.1.smithi117.stdout:5/792: dread dd/d16/f64 [0,4194304] 0 2024-03-22T06:15:29.006 INFO:tasks.workunit.client.1.smithi117.stdout:5/793: write dd/d16/d1a/d32/ff7 [900379,23573] 0 2024-03-22T06:15:29.008 INFO:tasks.workunit.client.1.smithi117.stdout:1/799: dread d1/d14/f104 [0,4194304] 0 2024-03-22T06:15:29.014 INFO:tasks.workunit.client.1.smithi117.stdout:3/754: dwrite d9/d1a/d39/d7d/f7e [0,4194304] 0 2024-03-22T06:15:29.014 INFO:tasks.workunit.client.1.smithi117.stdout:3/755: write d9/d1a/d3a/f7a [493623,15113] 0 2024-03-22T06:15:29.020 INFO:tasks.workunit.client.1.smithi117.stdout:9/784: link d4/d8/f25 d4/d8/d3f/fff 0 2024-03-22T06:15:29.022 INFO:tasks.workunit.client.1.smithi117.stdout:3/756: write d9/d55/f59 [3987114,87830] 0 2024-03-22T06:15:29.022 INFO:tasks.workunit.client.1.smithi117.stdout:3/757: read d9/fa6 [2364386,98069] 0 2024-03-22T06:15:29.022 INFO:tasks.workunit.client.1.smithi117.stdout:3/758: truncate d9/d1a/d39/d4c/d37/dcc/fe9 314124 0 2024-03-22T06:15:29.023 INFO:tasks.workunit.client.1.smithi117.stdout:7/716: fdatasync d8/d9/da/d2b/dc4/f83 0 2024-03-22T06:15:29.024 INFO:tasks.workunit.client.1.smithi117.stdout:3/759: dread d9/d1a/d39/f71 [0,4194304] 0 2024-03-22T06:15:29.027 INFO:tasks.workunit.client.1.smithi117.stdout:5/794: dwrite dd/df/d5e/d91/d72/d33/f68 [8388608,4194304] 0 2024-03-22T06:15:29.030 INFO:tasks.workunit.client.1.smithi117.stdout:4/727: rename d0/d4/d2a/d5b/f85 to d0/d19/d28/d47/ffb 0 2024-03-22T06:15:29.033 INFO:tasks.workunit.client.1.smithi117.stdout:8/756: creat d1/d3a/d44/d4b/fef x:0 0 0 2024-03-22T06:15:29.034 INFO:tasks.workunit.client.1.smithi117.stdout:8/757: dread d1/d3a/d44/d11/d97/f99 [4194304,4194304] 0 2024-03-22T06:15:29.037 INFO:tasks.workunit.client.1.smithi117.stdout:6/828: mkdir d3/d40/d46/d84/d49/d11f 0 2024-03-22T06:15:29.038 INFO:tasks.workunit.client.1.smithi117.stdout:1/800: symlink d1/d1f/d32/l11e 0 2024-03-22T06:15:29.045 INFO:tasks.workunit.client.1.smithi117.stdout:7/717: mknod d8/d50/d66/ce3 0 2024-03-22T06:15:29.048 INFO:tasks.workunit.client.1.smithi117.stdout:1/801: read d1/d1f/d32/fd9 [3803186,111469] 0 2024-03-22T06:15:29.048 INFO:tasks.workunit.client.1.smithi117.stdout:1/802: chown d1/d30/d44/c5c 410262 1 2024-03-22T06:15:29.048 INFO:tasks.workunit.client.1.smithi117.stdout:1/803: fdatasync d1/d30/fa8 0 2024-03-22T06:15:29.048 INFO:tasks.workunit.client.1.smithi117.stdout:1/804: chown d1/d30/d35/d58/fa1 337688158 1 2024-03-22T06:15:29.048 INFO:tasks.workunit.client.1.smithi117.stdout:5/795: link dd/df/d5e/d91/l41 dd/d16/d106/l115 0 2024-03-22T06:15:29.056 INFO:tasks.workunit.client.1.smithi117.stdout:2/718: rename d2/d5/d41/d83/fcc to d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/ff4 0 2024-03-22T06:15:29.056 INFO:tasks.workunit.client.1.smithi117.stdout:2/719: dread - d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/fec zero size 2024-03-22T06:15:29.058 INFO:tasks.workunit.client.1.smithi117.stdout:4/728: mknod d0/d19/d34/d43/d63/d6f/d6a/cfc 0 2024-03-22T06:15:29.058 INFO:tasks.workunit.client.1.smithi117.stdout:4/729: write d0/fd9 [344324,45727] 0 2024-03-22T06:15:29.062 INFO:tasks.workunit.client.1.smithi117.stdout:6/829: mknod d3/d40/d46/d84/d49/d79/c120 0 2024-03-22T06:15:29.064 INFO:tasks.workunit.client.1.smithi117.stdout:7/718: creat d8/d9/d37/d45/d88/db1/dc6/fe4 x:0 0 0 2024-03-22T06:15:29.064 INFO:tasks.workunit.client.1.smithi117.stdout:7/719: chown d8/d9/da/d10 46 1 2024-03-22T06:15:29.064 INFO:tasks.workunit.client.1.smithi117.stdout:7/720: stat d8/d9/da/d10/d23/fbb 0 2024-03-22T06:15:29.064 INFO:tasks.workunit.client.1.smithi117.stdout:7/721: truncate d8/d9/d37/d5f/f79 291962 0 2024-03-22T06:15:29.065 INFO:tasks.workunit.client.1.smithi117.stdout:4/730: dread d0/d19/d1c/f4d [4194304,4194304] 0 2024-03-22T06:15:29.067 INFO:tasks.workunit.client.1.smithi117.stdout:5/796: mkdir dd/d16/d1a/d32/d73/db8/d116 0 2024-03-22T06:15:29.070 INFO:tasks.workunit.client.1.smithi117.stdout:0/750: rename d1/d3/d2b/d3e/de/l57 to d1/d10/d18/db0/l10e 0 2024-03-22T06:15:29.071 INFO:tasks.workunit.client.1.smithi117.stdout:2/720: mkdir d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/df5 0 2024-03-22T06:15:29.077 INFO:tasks.workunit.client.1.smithi117.stdout:6/830: unlink d3/db4/d2d/d53/d59/d74/d28/l82 0 2024-03-22T06:15:29.077 INFO:tasks.workunit.client.1.smithi117.stdout:6/831: readlink d3/db4/d2d/d53/le1 0 2024-03-22T06:15:29.077 INFO:tasks.workunit.client.1.smithi117.stdout:6/832: chown d3/db4/dd4/df4/d101/da2/l62 37164 1 2024-03-22T06:15:29.079 INFO:tasks.workunit.client.1.smithi117.stdout:7/722: unlink d8/d9/d37/l5a 0 2024-03-22T06:15:29.081 INFO:tasks.workunit.client.1.smithi117.stdout:4/731: symlink d0/d19/d9b/lfd 0 2024-03-22T06:15:29.081 INFO:tasks.workunit.client.1.smithi117.stdout:4/732: dread - d0/d19/d34/d43/d63/d8d/fed zero size 2024-03-22T06:15:29.081 INFO:tasks.workunit.client.1.smithi117.stdout:4/733: write d0/d19/d28/f45 [4680063,69539] 0 2024-03-22T06:15:29.083 INFO:tasks.workunit.client.1.smithi117.stdout:0/751: dwrite d1/f87 [0,4194304] 0 2024-03-22T06:15:29.083 INFO:tasks.workunit.client.1.smithi117.stdout:0/752: read d1/d3/d2b/d3e/d3f/f45 [2444880,102449] 0 2024-03-22T06:15:29.085 INFO:tasks.workunit.client.1.smithi117.stdout:5/797: symlink dd/d16/d1a/d32/d73/db8/d116/l117 0 2024-03-22T06:15:29.089 INFO:tasks.workunit.client.1.smithi117.stdout:7/723: truncate d8/d9/da/d2b/f6e 4677893 0 2024-03-22T06:15:29.094 INFO:tasks.workunit.client.1.smithi117.stdout:4/734: read d0/d4/d2a/d5b/f6e [3917052,86499] 0 2024-03-22T06:15:29.097 INFO:tasks.workunit.client.1.smithi117.stdout:0/753: dwrite d1/d10/d5c/d40/dbb/dbf/fd1 [0,4194304] 0 2024-03-22T06:15:29.097 INFO:tasks.workunit.client.1.smithi117.stdout:0/754: write d1/d10/d18/d26/d49/d7e/fe0 [1721291,42869] 0 2024-03-22T06:15:29.100 INFO:tasks.workunit.client.1.smithi117.stdout:7/724: dread d8/d9/dd1/f44 [0,4194304] 0 2024-03-22T06:15:29.107 INFO:tasks.workunit.client.1.smithi117.stdout:5/798: link dd/df/d95/da0/cbb dd/d16/d1a/d32/dfc/c118 0 2024-03-22T06:15:29.107 INFO:tasks.workunit.client.1.smithi117.stdout:5/799: write dd/df/d5e/d91/d72/d35/fa5 [80270,124342] 0 2024-03-22T06:15:29.107 INFO:tasks.workunit.client.1.smithi117.stdout:5/800: write dd/df/d5e/d91/f5d [5000938,15951] 0 2024-03-22T06:15:29.114 INFO:tasks.workunit.client.1.smithi117.stdout:9/785: rename d4/d8/d93/db1 to d4/d7/dbf/d100 0 2024-03-22T06:15:29.118 INFO:tasks.workunit.client.1.smithi117.stdout:4/735: creat d0/d19/d34/d43/d63/d6f/ffe x:0 0 0 2024-03-22T06:15:29.118 INFO:tasks.workunit.client.1.smithi117.stdout:4/736: write d0/d19/d34/d43/d63/d6f/fbc [148538,1318] 0 2024-03-22T06:15:29.119 INFO:tasks.workunit.client.1.smithi117.stdout:0/755: dwrite d1/d3/d2b/d3e/de/d1f/d3b/fa6 [4194304,4194304] 0 2024-03-22T06:15:29.119 INFO:tasks.workunit.client.1.smithi117.stdout:0/756: readlink d1/d10/d18/d26/d49/d7e/d39/dc5/de8/l107 0 2024-03-22T06:15:29.119 INFO:tasks.workunit.client.1.smithi117.stdout:0/757: stat d1/d3/d2b/d3e/de/fa4 0 2024-03-22T06:15:29.123 INFO:tasks.workunit.client.1.smithi117.stdout:5/801: dwrite dd/d16/d1a/d32/dbf/fe2 [4194304,4194304] 0 2024-03-22T06:15:29.128 INFO:tasks.workunit.client.1.smithi117.stdout:7/725: mknod d8/d9/da/d2b/d9b/ce5 0 2024-03-22T06:15:29.134 INFO:tasks.workunit.client.1.smithi117.stdout:6/833: rename d3/d3d/d17/d31/df5/dfa/c114 to d3/d3d/d17/c121 0 2024-03-22T06:15:29.134 INFO:tasks.workunit.client.1.smithi117.stdout:6/834: chown d3/db4/d2d/d53/d59/f88 673711 1 2024-03-22T06:15:29.136 INFO:tasks.workunit.client.1.smithi117.stdout:4/737: dwrite d0/d19/d34/d43/d63/d6f/d5f/f75 [0,4194304] 0 2024-03-22T06:15:29.136 INFO:tasks.workunit.client.1.smithi117.stdout:7/726: dread d8/d9/da/d10/d23/d48/d91/f9c [0,4194304] 0 2024-03-22T06:15:29.136 INFO:tasks.workunit.client.1.smithi117.stdout:7/727: write d8/d9/da/d10/d23/fbb [146462,111943] 0 2024-03-22T06:15:29.136 INFO:tasks.workunit.client.1.smithi117.stdout:7/728: stat d8/d9/da 0 2024-03-22T06:15:29.138 INFO:tasks.workunit.client.1.smithi117.stdout:3/760: sync 2024-03-22T06:15:29.139 INFO:tasks.workunit.client.1.smithi117.stdout:6/835: dread d3/f4 [4194304,4194304] 0 2024-03-22T06:15:29.139 INFO:tasks.workunit.client.1.smithi117.stdout:9/786: truncate d4/d8/d3e/d7a/f42 70720 0 2024-03-22T06:15:29.143 INFO:tasks.workunit.client.1.smithi117.stdout:0/758: mknod d1/d10/d5c/d40/dac/c10f 0 2024-03-22T06:15:29.146 INFO:tasks.workunit.client.1.smithi117.stdout:3/761: dread d9/d1a/fc6 [0,4194304] 0 2024-03-22T06:15:29.149 INFO:tasks.workunit.client.1.smithi117.stdout:9/787: dread d4/d7/f1b [0,4194304] 0 2024-03-22T06:15:29.149 INFO:tasks.workunit.client.1.smithi117.stdout:9/788: stat d4/c18 0 2024-03-22T06:15:29.157 INFO:tasks.workunit.client.1.smithi117.stdout:5/802: rename dd/df/cb7 to dd/d16/d1a/d32/dfc/dff/c119 0 2024-03-22T06:15:29.159 INFO:tasks.workunit.client.1.smithi117.stdout:1/805: sync 2024-03-22T06:15:29.160 INFO:tasks.workunit.client.1.smithi117.stdout:8/758: sync 2024-03-22T06:15:29.160 INFO:tasks.workunit.client.1.smithi117.stdout:1/806: getdents d1/d22/d27/d4e/df6 0 2024-03-22T06:15:29.163 INFO:tasks.workunit.client.1.smithi117.stdout:4/738: truncate d0/d19/d34/f4f 852207 0 2024-03-22T06:15:29.166 INFO:tasks.workunit.client.1.smithi117.stdout:0/759: dwrite d1/d10/d5c/d40/dbb/fcd [0,4194304] 0 2024-03-22T06:15:29.166 INFO:tasks.workunit.client.1.smithi117.stdout:0/760: truncate d1/d10/d18/d26/d49/fab 1022750 0 2024-03-22T06:15:29.166 INFO:tasks.workunit.client.1.smithi117.stdout:7/729: mkdir d8/d9/da/d10/d23/d48/de6 0 2024-03-22T06:15:29.167 INFO:tasks.workunit.client.1.smithi117.stdout:6/836: symlink d3/d3d/d17/l122 0 2024-03-22T06:15:29.171 INFO:tasks.workunit.client.1.smithi117.stdout:3/762: creat d9/d1a/d39/d4c/d37/f105 x:0 0 0 2024-03-22T06:15:29.172 INFO:tasks.workunit.client.1.smithi117.stdout:9/789: mkdir d4/d8/d3e/d7a/d2f/d101 0 2024-03-22T06:15:29.172 INFO:tasks.workunit.client.1.smithi117.stdout:9/790: creat d4/d46/f102 x:0 0 0 2024-03-22T06:15:29.173 INFO:tasks.workunit.client.1.smithi117.stdout:2/721: sync 2024-03-22T06:15:29.177 INFO:tasks.workunit.client.1.smithi117.stdout:2/722: write d2/fc [9370037,69609] 0 2024-03-22T06:15:29.182 INFO:tasks.workunit.client.1.smithi117.stdout:5/803: symlink dd/df/d5e/d91/d36/l11a 0 2024-03-22T06:15:29.183 INFO:tasks.workunit.client.1.smithi117.stdout:5/804: stat dd/df/d5e/d91/d72/d35/da2/fc7 0 2024-03-22T06:15:29.183 INFO:tasks.workunit.client.1.smithi117.stdout:5/805: write dd/df/d88/fa9 [827841,63546] 0 2024-03-22T06:15:29.186 INFO:tasks.workunit.client.1.smithi117.stdout:5/806: dread dd/df/d5e/d91/d72/d60/d6e/f85 [0,4194304] 0 2024-03-22T06:15:29.198 INFO:tasks.workunit.client.1.smithi117.stdout:4/739: dwrite d0/d48/fd8 [0,4194304] 0 2024-03-22T06:15:29.199 INFO:tasks.workunit.client.1.smithi117.stdout:4/740: fdatasync d0/d48/fee 0 2024-03-22T06:15:29.199 INFO:tasks.workunit.client.1.smithi117.stdout:4/741: truncate d0/fc3 1014029 0 2024-03-22T06:15:29.199 INFO:tasks.workunit.client.1.smithi117.stdout:2/723: link d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/la2 d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/lf6 0 2024-03-22T06:15:29.199 INFO:tasks.workunit.client.1.smithi117.stdout:2/724: chown d2/d35/d4c/fcf 12882 1 2024-03-22T06:15:29.199 INFO:tasks.workunit.client.1.smithi117.stdout:1/807: getdents d1/d30/d44/d99/dd7 0 2024-03-22T06:15:29.204 INFO:tasks.workunit.client.1.smithi117.stdout:3/763: getdents d9/d1a/d39/d4c/d37/dd5 0 2024-03-22T06:15:29.206 INFO:tasks.workunit.client.1.smithi117.stdout:7/730: dwrite d8/d9/f99 [0,4194304] 0 2024-03-22T06:15:29.214 INFO:tasks.workunit.client.1.smithi117.stdout:4/742: mknod d0/d19/d9b/cff 0 2024-03-22T06:15:29.214 INFO:tasks.workunit.client.1.smithi117.stdout:4/743: readlink d0/d19/d28/d47/l8a 0 2024-03-22T06:15:29.214 INFO:tasks.workunit.client.1.smithi117.stdout:4/744: write d0/d48/fdd [269201,71401] 0 2024-03-22T06:15:29.215 INFO:tasks.workunit.client.1.smithi117.stdout:0/761: dwrite d1/d3/d2b/d3e/de/d1f/fff [0,4194304] 0 2024-03-22T06:15:29.215 INFO:tasks.workunit.client.1.smithi117.stdout:2/725: truncate d2/d5/d13/d57/f36 1281310 0 2024-03-22T06:15:29.216 INFO:tasks.workunit.client.1.smithi117.stdout:2/726: write d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/f62 [994702,114997] 0 2024-03-22T06:15:29.216 INFO:tasks.workunit.client.1.smithi117.stdout:2/727: creat d2/d5/db/ff7 x:0 0 0 2024-03-22T06:15:29.217 INFO:tasks.workunit.client.1.smithi117.stdout:9/791: dwrite d4/d8/d3e/d82/faa [0,4194304] 0 2024-03-22T06:15:29.218 INFO:tasks.workunit.client.1.smithi117.stdout:5/807: dwrite dd/df/d5e/d91/f34 [0,4194304] 0 2024-03-22T06:15:29.219 INFO:tasks.workunit.client.1.smithi117.stdout:4/745: mkdir d0/d51/d100 0 2024-03-22T06:15:29.231 INFO:tasks.workunit.client.1.smithi117.stdout:9/792: dread d4/d7/f2d [0,4194304] 0 2024-03-22T06:15:29.232 INFO:tasks.workunit.client.1.smithi117.stdout:9/793: chown d4/d8/d3e/d7a/d2f/f54 6395478 1 2024-03-22T06:15:29.235 INFO:tasks.workunit.client.1.smithi117.stdout:0/762: unlink d1/d10/f6c 0 2024-03-22T06:15:29.236 INFO:tasks.workunit.client.1.smithi117.stdout:3/764: dwrite d9/d1a/d39/d4c/d37/f105 [0,4194304] 0 2024-03-22T06:15:29.237 INFO:tasks.workunit.client.1.smithi117.stdout:3/765: chown d9/d1a/d39/dc7/ccb 122 1 2024-03-22T06:15:29.238 INFO:tasks.workunit.client.1.smithi117.stdout:1/808: dwrite d1/fc [4194304,4194304] 0 2024-03-22T06:15:29.238 INFO:tasks.workunit.client.1.smithi117.stdout:5/808: dwrite dd/d16/d1a/f59 [4194304,4194304] 0 2024-03-22T06:15:29.238 INFO:tasks.workunit.client.1.smithi117.stdout:5/809: stat dd/cdf 0 2024-03-22T06:15:29.243 INFO:tasks.workunit.client.1.smithi117.stdout:2/728: link d2/d5/db/cda d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/cf8 0 2024-03-22T06:15:29.243 INFO:tasks.workunit.client.1.smithi117.stdout:2/729: fsync d2/d5/db/d2c/f85 0 2024-03-22T06:15:29.248 INFO:tasks.workunit.client.1.smithi117.stdout:6/837: dwrite d3/d40/f95 [0,4194304] 0 2024-03-22T06:15:29.248 INFO:tasks.workunit.client.1.smithi117.stdout:6/838: dread - d3/d40/d46/d84/dbe/fe2 zero size 2024-03-22T06:15:29.252 INFO:tasks.workunit.client.1.smithi117.stdout:7/731: getdents d8 0 2024-03-22T06:15:29.252 INFO:tasks.workunit.client.1.smithi117.stdout:7/732: write d8/d9/d37/d45/d88/fbf [744438,11528] 0 2024-03-22T06:15:29.259 INFO:tasks.workunit.client.1.smithi117.stdout:9/794: unlink d4/d7/l3d 0 2024-03-22T06:15:29.259 INFO:tasks.workunit.client.1.smithi117.stdout:9/795: chown d4/d8/d3e/d7a/fd5 63980 1 2024-03-22T06:15:29.259 INFO:tasks.workunit.client.1.smithi117.stdout:9/796: fsync d4/d8/d3e/d7a/f9e 0 2024-03-22T06:15:29.271 INFO:tasks.workunit.client.1.smithi117.stdout:3/766: creat d9/d1a/d39/d4c/d37/d69/d101/f106 x:0 0 0 2024-03-22T06:15:29.295 INFO:tasks.workunit.client.1.smithi117.stdout:5/810: rmdir dd/d16/d1a/d32/dfc/dff 39 2024-03-22T06:15:29.299 INFO:tasks.workunit.client.1.smithi117.stdout:2/730: unlink d2/d5/d13/d57/dd0/d1d/f6c 0 2024-03-22T06:15:29.300 INFO:tasks.workunit.client.1.smithi117.stdout:8/759: sync 2024-03-22T06:15:29.307 INFO:tasks.workunit.client.1.smithi117.stdout:6/839: link d3/d61/f9b d3/d3d/d17/d31/df5/dfa/f123 0 2024-03-22T06:15:29.310 INFO:tasks.workunit.client.1.smithi117.stdout:7/733: truncate d8/d9/da/d2b/dc4/f83 296642 0 2024-03-22T06:15:29.312 INFO:tasks.workunit.client.1.smithi117.stdout:4/746: symlink d0/d19/l101 0 2024-03-22T06:15:29.312 INFO:tasks.workunit.client.1.smithi117.stdout:4/747: truncate d0/d48/da0/dd2/db6/fcf 199823 0 2024-03-22T06:15:29.316 INFO:tasks.workunit.client.1.smithi117.stdout:9/797: mkdir d4/d8/d93/d103 0 2024-03-22T06:15:29.319 INFO:tasks.workunit.client.1.smithi117.stdout:3/767: mknod d9/d1a/d39/dc7/c107 0 2024-03-22T06:15:29.327 INFO:tasks.workunit.client.1.smithi117.stdout:5/811: mknod dd/c11b 0 2024-03-22T06:15:29.327 INFO:tasks.workunit.client.1.smithi117.stdout:1/809: sync 2024-03-22T06:15:29.327 INFO:tasks.workunit.client.1.smithi117.stdout:0/763: sync 2024-03-22T06:15:29.327 INFO:tasks.workunit.client.1.smithi117.stdout:0/764: dread - d1/d10/d18/d26/f58 zero size 2024-03-22T06:15:29.327 INFO:tasks.workunit.client.1.smithi117.stdout:0/765: read d1/f3d [1152021,85412] 0 2024-03-22T06:15:29.330 INFO:tasks.workunit.client.1.smithi117.stdout:2/731: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/ff9 x:0 0 0 2024-03-22T06:15:29.332 INFO:tasks.workunit.client.1.smithi117.stdout:8/760: truncate d1/d3a/d44/d4b/dc9/fe5 554149 0 2024-03-22T06:15:29.332 INFO:tasks.workunit.client.1.smithi117.stdout:8/761: chown d1/d3a/d44/d16/l4f 0 1 2024-03-22T06:15:29.332 INFO:tasks.workunit.client.1.smithi117.stdout:8/762: dread - d1/d28/d6b/fd0 zero size 2024-03-22T06:15:29.332 INFO:tasks.workunit.client.1.smithi117.stdout:8/763: fdatasync d1/d3a/d44/d11/d97/f99 0 2024-03-22T06:15:29.332 INFO:tasks.workunit.client.1.smithi117.stdout:8/764: dread - d1/d28/d66/fe4 zero size 2024-03-22T06:15:29.332 INFO:tasks.workunit.client.1.smithi117.stdout:8/765: write d1/d3a/d44/d4b/dae/fb8 [696215,61352] 0 2024-03-22T06:15:29.332 INFO:tasks.workunit.client.1.smithi117.stdout:8/766: write d1/d3a/d44/d9/fc3 [24049,25688] 0 2024-03-22T06:15:29.333 INFO:tasks.workunit.client.1.smithi117.stdout:2/732: dread d2/d5/d13/d57/dd0/d1d/d3a/f3e [0,4194304] 0 2024-03-22T06:15:29.333 INFO:tasks.workunit.client.1.smithi117.stdout:2/733: write d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d77/f95 [1006057,26781] 0 2024-03-22T06:15:29.333 INFO:tasks.workunit.client.1.smithi117.stdout:2/734: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/ffa x:0 0 0 2024-03-22T06:15:29.333 INFO:tasks.workunit.client.1.smithi117.stdout:6/840: creat d3/db4/d2d/d53/d59/d74/d28/f124 x:0 0 0 2024-03-22T06:15:29.333 INFO:tasks.workunit.client.1.smithi117.stdout:6/841: fdatasync d3/d3d/d17/d60/f80 0 2024-03-22T06:15:29.333 INFO:tasks.workunit.client.1.smithi117.stdout:6/842: write d3/db4/d2d/d53/d59/d74/da/f29 [1557255,74229] 0 2024-03-22T06:15:29.334 INFO:tasks.workunit.client.1.smithi117.stdout:8/767: dread d1/d28/fa8 [0,4194304] 0 2024-03-22T06:15:29.334 INFO:tasks.workunit.client.1.smithi117.stdout:7/734: truncate d8/d9/d37/d5f/d82/f95 1881144 0 2024-03-22T06:15:29.339 INFO:tasks.workunit.client.1.smithi117.stdout:4/748: creat d0/d51/d92/f102 x:0 0 0 2024-03-22T06:15:29.347 INFO:tasks.workunit.client.1.smithi117.stdout:4/749: dread d0/d4/d2a/d5b/f6e [0,4194304] 0 2024-03-22T06:15:29.347 INFO:tasks.workunit.client.1.smithi117.stdout:4/750: write d0/d4/d2a/d5b/f6e [534610,83992] 0 2024-03-22T06:15:29.349 INFO:tasks.workunit.client.1.smithi117.stdout:3/768: chown d9/d1a/d39/d7d/d97/lb1 308 1 2024-03-22T06:15:29.353 INFO:tasks.workunit.client.1.smithi117.stdout:1/810: link d1/d22/l2e d1/d30/d5d/d6f/l11f 0 2024-03-22T06:15:29.353 INFO:tasks.workunit.client.1.smithi117.stdout:1/811: chown d1/d3c/db6/cf4 195373 1 2024-03-22T06:15:29.353 INFO:tasks.workunit.client.1.smithi117.stdout:0/766: symlink d1/d3/dae/l110 0 2024-03-22T06:15:29.354 INFO:tasks.workunit.client.1.smithi117.stdout:3/769: dread d9/d55/f7b [0,4194304] 0 2024-03-22T06:15:29.357 INFO:tasks.workunit.client.1.smithi117.stdout:6/843: dwrite d3/db4/d2d/d53/d59/d74/da/fbd [0,4194304] 0 2024-03-22T06:15:29.357 INFO:tasks.workunit.client.1.smithi117.stdout:6/844: fdatasync d3/db4/d2d/d53/d59/d74/da/de/fa3 0 2024-03-22T06:15:29.358 INFO:tasks.workunit.client.1.smithi117.stdout:8/768: symlink d1/d28/lf0 0 2024-03-22T06:15:29.362 INFO:tasks.workunit.client.1.smithi117.stdout:4/751: unlink d0/d19/d34/d43/d63/d6f/d70/fdf 0 2024-03-22T06:15:29.366 INFO:tasks.workunit.client.1.smithi117.stdout:5/812: unlink dd/df/c2d 0 2024-03-22T06:15:29.366 INFO:tasks.workunit.client.1.smithi117.stdout:5/813: readlink dd/d16/d86/d93/dfe/l112 0 2024-03-22T06:15:29.366 INFO:tasks.workunit.client.1.smithi117.stdout:3/770: dread d9/d1a/f20 [0,4194304] 0 2024-03-22T06:15:29.366 INFO:tasks.workunit.client.1.smithi117.stdout:7/735: dwrite d8/d9/da/d2b/d9b/fa8 [0,4194304] 0 2024-03-22T06:15:29.366 INFO:tasks.workunit.client.1.smithi117.stdout:7/736: chown d8/d9/da/d10 791271 1 2024-03-22T06:15:29.369 INFO:tasks.workunit.client.1.smithi117.stdout:1/812: link d1/d1f/l117 d1/d22/d27/d4e/df6/l120 0 2024-03-22T06:15:29.374 INFO:tasks.workunit.client.1.smithi117.stdout:6/845: mknod d3/d61/d73/d107/c125 0 2024-03-22T06:15:29.375 INFO:tasks.workunit.client.1.smithi117.stdout:2/735: rmdir d2/d5/d13/d57/dd0/d1d/d3a/d71 39 2024-03-22T06:15:29.378 INFO:tasks.workunit.client.1.smithi117.stdout:7/737: dread d8/d9/f99 [0,4194304] 0 2024-03-22T06:15:29.382 INFO:tasks.workunit.client.1.smithi117.stdout:8/769: link d1/d19/f86 d1/d28/d66/d7f/d9d/ff1 0 2024-03-22T06:15:29.382 INFO:tasks.workunit.client.1.smithi117.stdout:4/752: rename d0/d4/d2a/fb0 to d0/d51/d92/f103 0 2024-03-22T06:15:29.382 INFO:tasks.workunit.client.1.smithi117.stdout:5/814: mknod dd/d16/d1a/d32/dbf/c11c 0 2024-03-22T06:15:29.382 INFO:tasks.workunit.client.1.smithi117.stdout:8/770: dread d1/d28/d66/d7f/f9e [0,4194304] 0 2024-03-22T06:15:29.384 INFO:tasks.workunit.client.1.smithi117.stdout:3/771: dwrite d9/d1a/d39/d4c/d37/d69/fe4 [4194304,4194304] 0 2024-03-22T06:15:29.384 INFO:tasks.workunit.client.1.smithi117.stdout:3/772: truncate d9/d55/f67 2116081 0 2024-03-22T06:15:29.384 INFO:tasks.workunit.client.1.smithi117.stdout:3/773: chown d9/d1a/faf 5644 1 2024-03-22T06:15:29.384 INFO:tasks.workunit.client.1.smithi117.stdout:3/774: getdents d9/d1a/d39/d4c/d104 0 2024-03-22T06:15:29.384 INFO:tasks.workunit.client.1.smithi117.stdout:3/775: chown d9/d1a/d39/d4c/d99/cb9 1 1 2024-03-22T06:15:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.386+0000 7f858f162700 1 -- 172.21.15.6:0/3466157172 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8588104790 con 0x7f857406bbe0 2024-03-22T06:15:29.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.388+0000 7f857effd700 1 -- 172.21.15.6:0/3466157172 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f8588104790 con 0x7f857406bbe0 2024-03-22T06:15:29.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.392+0000 7f858f162700 1 -- 172.21.15.6:0/3466157172 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f857406bbe0 msgr2=0x7f857406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:29.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.392+0000 7f858f162700 1 --2- 172.21.15.6:0/3466157172 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f857406bbe0 0x7f857406e090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8584009b80 tx=0x7f858400b040 comp rx=0 tx=0).stop 2024-03-22T06:15:29.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.392+0000 7f858f162700 1 -- 172.21.15.6:0/3466157172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85880fbb30 msgr2=0x7f858810d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:29.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.392+0000 7f858f162700 1 --2- 172.21.15.6:0/3466157172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85880fbb30 0x7f858810d9e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8580004e60 tx=0x7f8580004d10 comp rx=0 tx=0).stop 2024-03-22T06:15:29.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.392+0000 7f858f162700 1 -- 172.21.15.6:0/3466157172 shutdown_connections 2024-03-22T06:15:29.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.392+0000 7f858f162700 1 --2- 172.21.15.6:0/3466157172 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f857406bbe0 0x7f857406e090 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8584009b80 tx=0x7f858400b040 comp rx=0 tx=0).stop 2024-03-22T06:15:29.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.392+0000 7f858f162700 1 --2- 172.21.15.6:0/3466157172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85880fbb30 0x7f858810d9e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:29.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.392+0000 7f858f162700 1 --2- 172.21.15.6:0/3466157172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85880fcde0 0x7f858810df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:29.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.392+0000 7f858f162700 1 -- 172.21.15.6:0/3466157172 >> 172.21.15.6:0/3466157172 conn(0x7f85880f7290 msgr2=0x7f85880f96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:29.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.394+0000 7f858f162700 1 -- 172.21.15.6:0/3466157172 shutdown_connections 2024-03-22T06:15:29.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.394+0000 7f858f162700 1 -- 172.21.15.6:0/3466157172 wait complete. 2024-03-22T06:15:29.397 INFO:tasks.workunit.client.1.smithi117.stdout:6/846: mkdir d3/db4/d2d/d53/d75/d126 0 2024-03-22T06:15:29.400 INFO:tasks.workunit.client.1.smithi117.stdout:2/736: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/dc9/ffb x:0 0 0 2024-03-22T06:15:29.400 INFO:tasks.workunit.client.1.smithi117.stdout:2/737: chown d2/d6e/f84 9 1 2024-03-22T06:15:29.400 INFO:tasks.workunit.client.1.smithi117.stdout:5/815: dwrite dd/df/d5e/d91/f57 [4194304,4194304] 0 2024-03-22T06:15:29.402 INFO:tasks.workunit.client.1.smithi117.stdout:4/753: dwrite d0/d48/da0/dd2/db6/dde/ff0 [0,4194304] 0 2024-03-22T06:15:29.402 INFO:tasks.workunit.client.1.smithi117.stdout:4/754: chown d0/d48/da0/dd2/db6/cc4 108129017 1 2024-03-22T06:15:29.403 INFO:tasks.workunit.client.1.smithi117.stdout:7/738: write d8/d9/da/d2b/dc4/f7b [4486418,7782] 0 2024-03-22T06:15:29.405 INFO:tasks.workunit.client.1.smithi117.stdout:1/813: rename d1/d14/f104 to d1/d3c/d4d/dfe/f121 0 2024-03-22T06:15:29.405 INFO:tasks.workunit.client.1.smithi117.stdout:1/814: creat d1/d3c/f122 x:0 0 0 2024-03-22T06:15:29.407 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:15:29.409 INFO:tasks.workunit.client.1.smithi117.stdout:8/771: symlink d1/d3a/d44/d9/lf2 0 2024-03-22T06:15:29.410 INFO:tasks.workunit.client.1.smithi117.stdout:5/816: dread dd/d16/fbc [0,4194304] 0 2024-03-22T06:15:29.410 INFO:tasks.workunit.client.1.smithi117.stdout:5/817: dread - dd/df/d5e/d91/d36/d70/f105 zero size 2024-03-22T06:15:29.411 INFO:tasks.workunit.client.1.smithi117.stdout:5/818: stat dd/d16/d1a/d32/d73 0 2024-03-22T06:15:29.413 INFO:tasks.workunit.client.1.smithi117.stdout:3/776: creat d9/d1a/d39/d4c/d104/f108 x:0 0 0 2024-03-22T06:15:29.415 INFO:tasks.workunit.client.1.smithi117.stdout:2/738: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/ffc x:0 0 0 2024-03-22T06:15:29.421 INFO:tasks.workunit.client.1.smithi117.stdout:3/777: dread d9/d55/f81 [0,4194304] 0 2024-03-22T06:15:29.425 INFO:tasks.workunit.client.1.smithi117.stdout:9/798: sync 2024-03-22T06:15:29.430 INFO:tasks.workunit.client.1.smithi117.stdout:4/755: truncate d0/d4/f7 3999850 0 2024-03-22T06:15:29.431 INFO:tasks.workunit.client.1.smithi117.stdout:4/756: chown d0/d48/c5e 25998447 1 2024-03-22T06:15:29.439 INFO:tasks.workunit.client.1.smithi117.stdout:7/739: creat d8/d9/da/d2b/db8/fe7 x:0 0 0 2024-03-22T06:15:29.443 INFO:tasks.workunit.client.1.smithi117.stdout:1/815: unlink d1/d3c/d4d/l74 0 2024-03-22T06:15:29.444 INFO:tasks.workunit.client.1.smithi117.stdout:8/772: symlink d1/d28/d66/d7f/d9d/lf3 0 2024-03-22T06:15:29.444 INFO:tasks.workunit.client.1.smithi117.stdout:8/773: fdatasync d1/d28/d6b/fad 0 2024-03-22T06:15:29.445 INFO:tasks.workunit.client.1.smithi117.stdout:5/819: truncate dd/df/d5e/d91/d36/f39 4063000 0 2024-03-22T06:15:29.447 INFO:tasks.workunit.client.1.smithi117.stdout:1/816: write d1/d3c/d4d/dfe/f121 [3630083,66552] 0 2024-03-22T06:15:29.447 INFO:tasks.workunit.client.1.smithi117.stdout:1/817: truncate d1/d84/d77/fd2 574463 0 2024-03-22T06:15:29.447 INFO:tasks.workunit.client.1.smithi117.stdout:6/847: symlink d3/d61/d73/l127 0 2024-03-22T06:15:29.448 INFO:tasks.workunit.client.1.smithi117.stdout:8/774: mkdir d1/d28/d6b/df4 0 2024-03-22T06:15:29.451 INFO:tasks.workunit.client.1.smithi117.stdout:2/739: dwrite d2/d5/d13/d57/dd0/f51 [0,4194304] 0 2024-03-22T06:15:29.453 INFO:tasks.workunit.client.1.smithi117.stdout:3/778: rename d9/d1a/d3a/c73 to d9/d1a/d39/d7d/c109 0 2024-03-22T06:15:29.454 INFO:tasks.workunit.client.1.smithi117.stdout:0/767: sync 2024-03-22T06:15:29.454 INFO:tasks.workunit.client.1.smithi117.stdout:8/775: creat d1/d28/d6b/df4/ff5 x:0 0 0 2024-03-22T06:15:29.455 INFO:tasks.workunit.client.1.smithi117.stdout:8/776: readlink d1/d3a/d44/d11/l4e 0 2024-03-22T06:15:29.455 INFO:tasks.workunit.client.1.smithi117.stdout:0/768: write d1/d3/d2b/d3e/de/d1f/ddd/f101 [912505,8611] 0 2024-03-22T06:15:29.455 INFO:tasks.workunit.client.1.smithi117.stdout:4/757: write d0/d4/d2a/f30 [3651994,122518] 0 2024-03-22T06:15:29.455 INFO:tasks.workunit.client.1.smithi117.stdout:4/758: dread - d0/d19/d28/d47/ffb zero size 2024-03-22T06:15:29.455 INFO:tasks.workunit.client.1.smithi117.stdout:9/799: dwrite d4/d47/d85/fa8 [0,4194304] 0 2024-03-22T06:15:29.455 INFO:tasks.workunit.client.1.smithi117.stdout:1/818: dread d1/d1f/f79 [0,4194304] 0 2024-03-22T06:15:29.455 INFO:tasks.workunit.client.1.smithi117.stdout:1/819: write d1/d1f/d32/d33/da4/f109 [1226769,89449] 0 2024-03-22T06:15:29.455 INFO:tasks.workunit.client.1.smithi117.stdout:5/820: getdents dd/d16/d1a 0 2024-03-22T06:15:29.459 INFO:tasks.workunit.client.1.smithi117.stdout:6/848: dread d3/db4/d2d/d53/d59/d74/da/f29 [0,4194304] 0 2024-03-22T06:15:29.462 INFO:tasks.workunit.client.1.smithi117.stdout:2/740: mknod d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/cfd 0 2024-03-22T06:15:29.464 INFO:tasks.workunit.client.1.smithi117.stdout:7/740: rename d8/d9/dd1/f4d to d8/d9/da/d10/d23/d48/dc8/fe8 0 2024-03-22T06:15:29.465 INFO:tasks.workunit.client.1.smithi117.stdout:4/759: dread d0/d4/d2a/d2f/f56 [0,4194304] 0 2024-03-22T06:15:29.468 INFO:tasks.workunit.client.1.smithi117.stdout:3/779: mkdir d9/d1a/d39/d82/d10a 0 2024-03-22T06:15:29.479 INFO:tasks.workunit.client.1.smithi117.stdout:8/777: dwrite f0 [4194304,4194304] 0 2024-03-22T06:15:29.479 INFO:tasks.workunit.client.1.smithi117.stdout:8/778: read - d1/d3a/d44/d9/fdb zero size 2024-03-22T06:15:29.480 INFO:tasks.workunit.client.1.smithi117.stdout:9/800: getdents d4/d8/d3e 0 2024-03-22T06:15:29.480 INFO:tasks.workunit.client.1.smithi117.stdout:9/801: chown d4/d8/d3f/dc3/fd3 60 1 2024-03-22T06:15:29.486 INFO:tasks.workunit.client.1.smithi117.stdout:3/780: dread d9/d1a/d39/d7d/f7e [0,4194304] 0 2024-03-22T06:15:29.486 INFO:tasks.workunit.client.1.smithi117.stdout:5/821: dwrite dd/df/d5e/d91/d72/d35/da2/fc7 [0,4194304] 0 2024-03-22T06:15:29.493 INFO:tasks.workunit.client.1.smithi117.stdout:1/820: rmdir d1/d62/de4 39 2024-03-22T06:15:29.499 INFO:tasks.workunit.client.1.smithi117.stdout:1/821: dread d1/d1f/d32/d33/da4/fd8 [0,4194304] 0 2024-03-22T06:15:29.499 INFO:tasks.workunit.client.1.smithi117.stdout:2/741: truncate d2/d5/d13/d57/dd0/d1d/fe8 3467463 0 2024-03-22T06:15:29.502 INFO:tasks.workunit.client.1.smithi117.stdout:2/742: write d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d77/f95 [302985,53277] 0 2024-03-22T06:15:29.502 INFO:tasks.workunit.client.1.smithi117.stdout:2/743: readlink d2/d5/d13/d57/dd0/d1d/d3a/l5a 0 2024-03-22T06:15:29.503 INFO:tasks.workunit.client.1.smithi117.stdout:2/744: write d2/d5/f33 [1364294,83330] 0 2024-03-22T06:15:29.507 INFO:tasks.workunit.client.1.smithi117.stdout:8/779: dwrite d1/d28/d66/fe4 [0,4194304] 0 2024-03-22T06:15:29.508 INFO:tasks.workunit.client.1.smithi117.stdout:9/802: dwrite d4/d8/f1a [0,4194304] 0 2024-03-22T06:15:29.508 INFO:tasks.workunit.client.1.smithi117.stdout:9/803: write d4/d8/d3f/fff [2795343,38507] 0 2024-03-22T06:15:29.510 INFO:tasks.workunit.client.1.smithi117.stdout:6/849: rename d3/d40/d46/d84/dbe/cf1 to d3/d3d/dcb/dcc/c128 0 2024-03-22T06:15:29.515 INFO:tasks.workunit.client.1.smithi117.stdout:4/760: truncate d0/d3f/f54 1241228 0 2024-03-22T06:15:29.515 INFO:tasks.workunit.client.1.smithi117.stdout:4/761: write d0/d19/d1c/fca [442532,100947] 0 2024-03-22T06:15:29.538 INFO:tasks.workunit.client.1.smithi117.stdout:9/804: dwrite d4/d7/dbf/d100/dd1/dd6/d9b/f9c [0,4194304] 0 2024-03-22T06:15:29.542 INFO:tasks.workunit.client.1.smithi117.stdout:8/780: dwrite d1/d28/d6b/fdc [8388608,4194304] 0 2024-03-22T06:15:29.544 INFO:tasks.workunit.client.1.smithi117.stdout:3/781: mknod d9/d1a/d39/d8a/c10b 0 2024-03-22T06:15:29.548 INFO:tasks.workunit.client.1.smithi117.stdout:2/745: truncate d2/d35/d38/f5e 5438230 0 2024-03-22T06:15:29.569 INFO:tasks.workunit.client.1.smithi117.stdout:5/822: rename dd/df/d5e/d91/d72/d35/da2/lf1 to dd/df/d5e/d91/d72/dde/de0/d107/l11d 0 2024-03-22T06:15:29.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.569+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2306476402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d40fcf80 msgr2=0x7f46d40fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:29.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.569+0000 7f46d8bc4700 1 --2- 172.21.15.6:0/2306476402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d40fcf80 0x7f46d40fd3f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f46c8004940 tx=0x7f46c8009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:29.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.569+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2306476402 shutdown_connections 2024-03-22T06:15:29.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.569+0000 7f46d8bc4700 1 --2- 172.21.15.6:0/2306476402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d40fcf80 0x7f46d40fd3f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:29.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.569+0000 7f46d8bc4700 1 --2- 172.21.15.6:0/2306476402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d40fbcd0 0x7f46d40fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:29.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.569+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2306476402 >> 172.21.15.6:0/2306476402 conn(0x7f46d40f7430 msgr2=0x7f46d40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:29.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.569+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2306476402 shutdown_connections 2024-03-22T06:15:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.570+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2306476402 wait complete. 2024-03-22T06:15:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.570+0000 7f46d8bc4700 1 Processor -- start 2024-03-22T06:15:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.570+0000 7f46d8bc4700 1 -- start start 2024-03-22T06:15:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.570+0000 7f46d8bc4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d40fbcd0 0x7f46d40ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:29.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.571+0000 7f46d8bc4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d40fcf80 0x7f46d40fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:29.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.571+0000 7f46d8bc4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4111610 con 0x7f46d40fcf80 2024-03-22T06:15:29.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.571+0000 7f46d8bc4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4111780 con 0x7f46d40fbcd0 2024-03-22T06:15:29.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.571+0000 7f46d1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d40fcf80 0x7f46d40fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:29.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.571+0000 7f46d1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d40fcf80 0x7f46d40fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47082/0 (socket says 172.21.15.6:47082) 2024-03-22T06:15:29.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.571+0000 7f46d1d9b700 1 -- 172.21.15.6:0/2818352734 learned_addr learned my addr 172.21.15.6:0/2818352734 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:15:29.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.571+0000 7f46d1d9b700 1 -- 172.21.15.6:0/2818352734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d40fbcd0 msgr2=0x7f46d40ffdf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-22T06:15:29.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.571+0000 7f46d1d9b700 1 --2- 172.21.15.6:0/2818352734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d40fbcd0 0x7f46d40ffdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:29.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.571+0000 7f46d1d9b700 1 -- 172.21.15.6:0/2818352734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c8005040 con 0x7f46d40fcf80 2024-03-22T06:15:29.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.571+0000 7f46d1d9b700 1 --2- 172.21.15.6:0/2818352734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d40fcf80 0x7f46d40fe2c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f46c800b3b0 tx=0x7f46c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:29.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.572+0000 7f46c2ffd700 1 -- 172.21.15.6:0/2818352734 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46c800e040 con 0x7f46d40fcf80 2024-03-22T06:15:29.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.572+0000 7f46c2ffd700 1 -- 172.21.15.6:0/2818352734 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46c8004290 con 0x7f46d40fcf80 2024-03-22T06:15:29.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.572+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2818352734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d40fe8c0 con 0x7f46d40fcf80 2024-03-22T06:15:29.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.572+0000 7f46c2ffd700 1 -- 172.21.15.6:0/2818352734 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46c801c630 con 0x7f46d40fcf80 2024-03-22T06:15:29.574 INFO:tasks.workunit.client.1.smithi117.stdout:6/850: symlink d3/d3d/d17/d60/l129 0 2024-03-22T06:15:29.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.572+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2818352734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d40fed70 con 0x7f46d40fcf80 2024-03-22T06:15:29.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.573+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2818352734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d4101b10 con 0x7f46d40fcf80 2024-03-22T06:15:29.577 INFO:tasks.workunit.client.1.smithi117.stdout:0/769: sync 2024-03-22T06:15:29.577 INFO:tasks.workunit.client.1.smithi117.stdout:7/741: sync 2024-03-22T06:15:29.577 INFO:tasks.workunit.client.1.smithi117.stdout:0/770: fsync d1/d3/d2b/d3e/de/d1f/d3b/d55/f92 0 2024-03-22T06:15:29.577 INFO:tasks.workunit.client.1.smithi117.stdout:7/742: write d8/d50/d5e/fbd [242467,21692] 0 2024-03-22T06:15:29.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.576+0000 7f46c2ffd700 1 -- 172.21.15.6:0/2818352734 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f46c800bbf0 con 0x7f46d40fcf80 2024-03-22T06:15:29.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.577+0000 7f46c2ffd700 1 --2- 172.21.15.6:0/2818352734 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f46bc06bbe0 0x7f46bc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:29.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.577+0000 7f46c2ffd700 1 -- 172.21.15.6:0/2818352734 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f46c808bc40 con 0x7f46d40fcf80 2024-03-22T06:15:29.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.579+0000 7f46d259c700 1 --2- 172.21.15.6:0/2818352734 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f46bc06bbe0 0x7f46bc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:29.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.580+0000 7f46d259c700 1 --2- 172.21.15.6:0/2818352734 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f46bc06bbe0 0x7f46bc06e090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009b80 tx=0x7f46c4009d60 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:29.581 INFO:tasks.workunit.client.1.smithi117.stdout:9/805: mkdir d4/d47/d64/dc5/dda/d104 0 2024-03-22T06:15:29.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.587+0000 7f46c2ffd700 1 -- 172.21.15.6:0/2818352734 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f46c8055420 con 0x7f46d40fcf80 2024-03-22T06:15:29.587 INFO:tasks.workunit.client.1.smithi117.stdout:2/746: truncate d2/d5/d13/d57/f63 3323490 0 2024-03-22T06:15:29.588 INFO:tasks.workunit.client.1.smithi117.stdout:2/747: stat d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6/d39/d45/d7c/l9c 0 2024-03-22T06:15:29.590 INFO:tasks.workunit.client.1.smithi117.stdout:0/771: dread d1/d3/d2b/d3e/ffe [0,4194304] 0 2024-03-22T06:15:29.591 INFO:tasks.workunit.client.1.smithi117.stdout:0/772: read d1/d3/d2b/d3e/d3f/f78 [88173,62510] 0 2024-03-22T06:15:29.597 INFO:tasks.workunit.client.1.smithi117.stdout:6/851: dwrite d3/db4/d2d/d53/d75/fc1 [4194304,4194304] 0 2024-03-22T06:15:29.598 INFO:tasks.workunit.client.1.smithi117.stdout:9/806: dwrite d4/d46/fed [0,4194304] 0 2024-03-22T06:15:29.599 INFO:tasks.workunit.client.1.smithi117.stdout:5/823: dwrite dd/df/d5e/d91/d72/d33/ff5 [0,4194304] 0 2024-03-22T06:15:29.606 INFO:tasks.workunit.client.1.smithi117.stdout:0/773: dread d1/d10/d18/f95 [0,4194304] 0 2024-03-22T06:15:29.612 INFO:tasks.workunit.client.1.smithi117.stdout:4/762: rmdir d0/d19/d34/d43/d63 39 2024-03-22T06:15:29.614 INFO:tasks.workunit.client.1.smithi117.stdout:5/824: dread dd/df/d5e/d91/d72/d33/f68 [0,4194304] 0 2024-03-22T06:15:29.614 INFO:tasks.workunit.client.1.smithi117.stdout:9/807: dread d4/d47/d64/fb4 [0,4194304] 0 2024-03-22T06:15:29.614 INFO:tasks.workunit.client.1.smithi117.stdout:9/808: chown d4/d8/la2 235 1 2024-03-22T06:15:29.614 INFO:tasks.workunit.client.1.smithi117.stdout:7/743: truncate d8/d9/da/d2b/dc4/f5d 156514 0 2024-03-22T06:15:29.614 INFO:tasks.workunit.client.1.smithi117.stdout:7/744: dread - d8/d9/da/d10/fd5 zero size 2024-03-22T06:15:29.614 INFO:tasks.workunit.client.1.smithi117.stdout:7/745: readlink d8/d9/da/d10/l7d 0 2024-03-22T06:15:29.617 INFO:tasks.workunit.client.1.smithi117.stdout:8/781: getdents d1/d3a/d44/d4b 0 2024-03-22T06:15:29.620 INFO:tasks.workunit.client.1.smithi117.stdout:3/782: link d9/d1a/d39/d4c/d99/lcf d9/d1a/d39/d4c/d36/d41/l10c 0 2024-03-22T06:15:29.624 INFO:tasks.workunit.client.1.smithi117.stdout:2/748: mkdir d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/dfe 0 2024-03-22T06:15:29.626 INFO:tasks.workunit.client.1.smithi117.stdout:1/822: sync 2024-03-22T06:15:29.631 INFO:tasks.workunit.client.1.smithi117.stdout:6/852: creat d3/db4/d2d/d53/d59/d74/d28/ddb/f12a x:0 0 0 2024-03-22T06:15:29.632 INFO:tasks.workunit.client.1.smithi117.stdout:1/823: write d1/d1f/f3e [1294642,60955] 0 2024-03-22T06:15:29.633 INFO:tasks.workunit.client.1.smithi117.stdout:1/824: write d1/d1f/fae [775179,73483] 0 2024-03-22T06:15:29.639 INFO:tasks.workunit.client.1.smithi117.stdout:0/774: symlink d1/d10/d18/d26/d49/d7e/d39/dc5/l111 0 2024-03-22T06:15:29.640 INFO:tasks.workunit.client.1.smithi117.stdout:0/775: stat d1/d3/dae/lfc 0 2024-03-22T06:15:29.640 INFO:tasks.workunit.client.1.smithi117.stdout:0/776: creat d1/d3/d2b/d3e/d3f/f112 x:0 0 0 2024-03-22T06:15:29.645 INFO:tasks.workunit.client.1.smithi117.stdout:4/763: rename d0/d19/d34/d43/d63/l78 to d0/d48/de4/l104 0 2024-03-22T06:15:29.647 INFO:tasks.workunit.client.1.smithi117.stdout:5/825: creat dd/df/d5e/d91/deb/f11e x:0 0 0 2024-03-22T06:15:29.647 INFO:tasks.workunit.client.1.smithi117.stdout:5/826: creat dd/df/d95/f11f x:0 0 0 2024-03-22T06:15:29.648 INFO:tasks.workunit.client.1.smithi117.stdout:0/777: read d1/f87 [1873765,43164] 0 2024-03-22T06:15:29.649 INFO:tasks.workunit.client.1.smithi117.stdout:0/778: fdatasync d1/d10/d18/f38 0 2024-03-22T06:15:29.649 INFO:tasks.workunit.client.1.smithi117.stdout:0/779: readlink d1/d9d/lf1 0 2024-03-22T06:15:29.649 INFO:tasks.workunit.client.1.smithi117.stdout:9/809: mkdir d4/d46/d8c/d105 0 2024-03-22T06:15:29.651 INFO:tasks.workunit.client.1.smithi117.stdout:5/827: read dd/d16/fbc [2953399,47080] 0 2024-03-22T06:15:29.651 INFO:tasks.workunit.client.1.smithi117.stdout:5/828: chown dd/df/d5e/d91/d72/d33/d9a/lbd 412 1 2024-03-22T06:15:29.651 INFO:tasks.workunit.client.1.smithi117.stdout:7/746: creat d8/d50/d66/fe9 x:0 0 0 2024-03-22T06:15:29.653 INFO:tasks.workunit.client.1.smithi117.stdout:8/782: symlink d1/d3a/d44/d6a/db7/lf6 0 2024-03-22T06:15:29.654 INFO:tasks.workunit.client.1.smithi117.stdout:8/783: dread - d1/d3a/fa6 zero size 2024-03-22T06:15:29.657 INFO:tasks.workunit.client.1.smithi117.stdout:1/825: rmdir d1/d1f/d32 39 2024-03-22T06:15:29.666 INFO:tasks.workunit.client.1.smithi117.stdout:5/829: dwrite dd/df/d5e/d91/d36/f25 [4194304,4194304] 0 2024-03-22T06:15:29.675 INFO:tasks.workunit.client.1.smithi117.stdout:0/780: mkdir d1/d10/d5c/d40/d113 0 2024-03-22T06:15:29.738 INFO:tasks.workunit.client.1.smithi117.stdout:9/810: symlink d4/d46/d8c/l106 0 2024-03-22T06:15:29.741 INFO:tasks.workunit.client.1.smithi117.stdout:7/747: creat d8/d9/d37/d45/d88/fea x:0 0 0 2024-03-22T06:15:29.741 INFO:tasks.workunit.client.1.smithi117.stdout:7/748: write d8/d9/da/d2b/dc4/f83 [1202490,106631] 0 2024-03-22T06:15:29.747 INFO:tasks.workunit.client.1.smithi117.stdout:3/783: symlink d9/d1a/d39/d4c/d36/d41/l10d 0 2024-03-22T06:15:29.747 INFO:tasks.workunit.client.1.smithi117.stdout:3/784: dread - d9/d1a/d3a/f100 zero size 2024-03-22T06:15:29.756 INFO:tasks.workunit.client.1.smithi117.stdout:5/830: getdents dd/df/d5e/d91/d72/dde/de0/d107 0 2024-03-22T06:15:29.756 INFO:tasks.workunit.client.1.smithi117.stdout:5/831: chown dd/d16/d1a/l63 1542646032 1 2024-03-22T06:15:29.756 INFO:tasks.workunit.client.1.smithi117.stdout:5/832: stat dd/df/d1e/c114 0 2024-03-22T06:15:29.756 INFO:tasks.workunit.client.1.smithi117.stdout:5/833: readlink dd/d16/d1a/d32/d73/db8/l10b 0 2024-03-22T06:15:29.766 INFO:tasks.workunit.client.1.smithi117.stdout:2/749: rename d2/d5/d13/d57/dd0/d1d/d3a/d89/dc6 to d2/d5/db/d2c/d7d/dff 0 2024-03-22T06:15:29.767 INFO:tasks.workunit.client.1.smithi117.stdout:2/750: dread d2/d5/d13/d57/dd0/d1d/d3a/d89/fab [0,4194304] 0 2024-03-22T06:15:29.770 INFO:tasks.workunit.client.1.smithi117.stdout:4/764: rmdir d0/dc0 39 2024-03-22T06:15:29.770 INFO:tasks.workunit.client.1.smithi117.stdout:4/765: fsync d0/d4/d2a/d2f/f53 0 2024-03-22T06:15:29.770 INFO:tasks.workunit.client.1.smithi117.stdout:4/766: fsync d0/d4/d2a/f30 0 2024-03-22T06:15:29.770 INFO:tasks.workunit.client.1.smithi117.stdout:4/767: chown d0/d51/fe6 487113144 1 2024-03-22T06:15:29.773 INFO:tasks.workunit.client.1.smithi117.stdout:0/781: mknod d1/d10/d18/d26/d53/c114 0 2024-03-22T06:15:29.773 INFO:tasks.workunit.client.1.smithi117.stdout:0/782: write d1/d10/d5c/d40/d69/fa8 [2766733,103442] 0 2024-03-22T06:15:29.895 INFO:tasks.workunit.client.1.smithi117.stdout:9/811: link d4/d7/dbf/d100/dd1/dd6/c77 d4/d8/d3f/c107 0 2024-03-22T06:15:29.895 INFO:tasks.workunit.client.1.smithi117.stdout:9/812: chown d4/d8/d3e/d7a/da9 6 1 2024-03-22T06:15:29.899 INFO:tasks.workunit.client.1.smithi117.stdout:7/749: creat d8/d9/da/d2b/dc4/d5c/feb x:0 0 0 2024-03-22T06:15:29.902 INFO:tasks.workunit.client.1.smithi117.stdout:8/784: getdents d1/d28/d6b 0 2024-03-22T06:15:29.906 INFO:tasks.workunit.client.1.smithi117.stdout:3/785: creat d9/d1a/d39/d8a/da1/db6/f10e x:0 0 0 2024-03-22T06:15:29.906 INFO:tasks.workunit.client.1.smithi117.stdout:3/786: write d9/d55/f67 [2690458,2588] 0 2024-03-22T06:15:29.906 INFO:tasks.workunit.client.1.smithi117.stdout:3/787: write d9/d66/f74 [196179,55398] 0 2024-03-22T06:15:29.908 INFO:tasks.workunit.client.1.smithi117.stdout:5/834: creat dd/d16/d86/d93/d10f/f120 x:0 0 0 2024-03-22T06:15:29.912 INFO:tasks.workunit.client.1.smithi117.stdout:2/751: symlink d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/dc9/l100 0 2024-03-22T06:15:29.914 INFO:tasks.workunit.client.1.smithi117.stdout:4/768: unlink d0/d19/d34/d43/d63/d8d/f90 0 2024-03-22T06:15:29.918 INFO:tasks.workunit.client.1.smithi117.stdout:9/813: creat d4/d47/d64/dc5/dda/d104/f108 x:0 0 0 2024-03-22T06:15:29.921 INFO:tasks.workunit.client.1.smithi117.stdout:8/785: mkdir d1/d3a/d44/d4b/df7 0 2024-03-22T06:15:29.921 INFO:tasks.workunit.client.1.smithi117.stdout:8/786: creat d1/d28/d6b/da3/ff8 x:0 0 0 2024-03-22T06:15:29.922 INFO:tasks.workunit.client.1.smithi117.stdout:8/787: fsync d1/d19/d98/fe6 0 2024-03-22T06:15:29.923 INFO:tasks.workunit.client.1.smithi117.stdout:3/788: truncate d9/d1a/d39/d82/f84 3807570 0 2024-03-22T06:15:29.923 INFO:tasks.workunit.client.1.smithi117.stdout:3/789: write d9/d1a/d3a/f56 [8918693,77849] 0 2024-03-22T06:15:29.925 INFO:tasks.workunit.client.1.smithi117.stdout:5/835: creat dd/df/d95/da0/da8/f121 x:0 0 0 2024-03-22T06:15:29.926 INFO:tasks.workunit.client.1.smithi117.stdout:4/769: mkdir d0/d48/da0/dd2/d105 0 2024-03-22T06:15:29.927 INFO:tasks.workunit.client.1.smithi117.stdout:9/814: truncate d4/d7/dbf/d100/dd1/dd6/d9b/f9c 533537 0 2024-03-22T06:15:29.927 INFO:tasks.workunit.client.1.smithi117.stdout:9/815: fsync d4/d7/d27/f56 0 2024-03-22T06:15:29.932 INFO:tasks.workunit.client.1.smithi117.stdout:5/836: symlink dd/df/d5e/d91/d90/l122 0 2024-03-22T06:15:29.932 INFO:tasks.workunit.client.1.smithi117.stdout:5/837: chown dd/df/d88 7347320 1 2024-03-22T06:15:29.933 INFO:tasks.workunit.client.1.smithi117.stdout:5/838: write dd/d16/f65 [19536,80091] 0 2024-03-22T06:15:29.933 INFO:tasks.workunit.client.1.smithi117.stdout:4/770: dread d0/d4/f12 [0,4194304] 0 2024-03-22T06:15:29.933 INFO:tasks.workunit.client.1.smithi117.stdout:9/816: link d4/d7/d27/d5b/l94 d4/d47/d85/dab/l109 0 2024-03-22T06:15:29.934 INFO:tasks.workunit.client.1.smithi117.stdout:4/771: creat d0/d19/d9b/f106 x:0 0 0 2024-03-22T06:15:29.939 INFO:tasks.workunit.client.1.smithi117.stdout:4/772: write d0/d19/d34/f4f [613271,53519] 0 2024-03-22T06:15:29.941 INFO:tasks.workunit.client.1.smithi117.stdout:4/773: mknod d0/d4/d2a/d5b/c107 0 2024-03-22T06:15:29.941 INFO:tasks.workunit.client.1.smithi117.stdout:4/774: chown d0/fc3 516 1 2024-03-22T06:15:29.943 INFO:tasks.workunit.client.1.smithi117.stdout:5/839: dread dd/df/d5e/d91/d72/f45 [0,4194304] 0 2024-03-22T06:15:29.943 INFO:tasks.workunit.client.1.smithi117.stdout:5/840: chown dd/df/lb3 879861439 1 2024-03-22T06:15:29.943 INFO:tasks.workunit.client.1.smithi117.stdout:5/841: stat dd/d16/d86/f9d 0 2024-03-22T06:15:29.943 INFO:tasks.workunit.client.1.smithi117.stdout:5/842: dread - dd/df/d5e/fdc zero size 2024-03-22T06:15:29.943 INFO:tasks.workunit.client.1.smithi117.stdout:9/817: write d4/d8/d3f/f52 [3438535,70604] 0 2024-03-22T06:15:29.945 INFO:tasks.workunit.client.1.smithi117.stdout:9/818: dread d4/d46/f95 [0,4194304] 0 2024-03-22T06:15:29.946 INFO:tasks.workunit.client.1.smithi117.stdout:4/775: dread d0/d4/f7 [0,4194304] 0 2024-03-22T06:15:29.948 INFO:tasks.workunit.client.1.smithi117.stdout:5/843: creat dd/d16/d106/f123 x:0 0 0 2024-03-22T06:15:29.954 INFO:tasks.workunit.client.1.smithi117.stdout:5/844: symlink dd/df/d5e/d91/d90/dca/l124 0 2024-03-22T06:15:29.954 INFO:tasks.workunit.client.1.smithi117.stdout:5/845: stat dd/df/d5e/d91/d36/f25 0 2024-03-22T06:15:29.954 INFO:tasks.workunit.client.1.smithi117.stdout:5/846: write dd/d16/d1a/f3e [977585,6769] 0 2024-03-22T06:15:29.954 INFO:tasks.workunit.client.1.smithi117.stdout:5/847: write dd/df/d5e/d91/f7e [2674122,60054] 0 2024-03-22T06:15:29.967 INFO:tasks.workunit.client.1.smithi117.stdout:4/776: dread d0/d3f/f95 [0,4194304] 0 2024-03-22T06:15:29.973 INFO:tasks.workunit.client.1.smithi117.stdout:1/826: sync 2024-03-22T06:15:29.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.980+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2818352734 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46d4104830 con 0x7f46bc06bbe0 2024-03-22T06:15:29.981 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:29 smithi006 ceph-mon[29301]: pgmap v28: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 207 MiB/s rd, 246 MiB/s wr, 366 op/s 2024-03-22T06:15:29.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.982+0000 7f46c2ffd700 1 -- 172.21.15.6:0/2818352734 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f46d4104830 con 0x7f46bc06bbe0 2024-03-22T06:15:29.986 INFO:tasks.workunit.client.1.smithi117.stdout:9/819: dwrite d4/d8/d3f/f52 [4194304,4194304] 0 2024-03-22T06:15:29.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.986+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2818352734 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f46bc06bbe0 msgr2=0x7f46bc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:29.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.986+0000 7f46d8bc4700 1 --2- 172.21.15.6:0/2818352734 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f46bc06bbe0 0x7f46bc06e090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009b80 tx=0x7f46c4009d60 comp rx=0 tx=0).stop 2024-03-22T06:15:29.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.986+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2818352734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d40fcf80 msgr2=0x7f46d40fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:29.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.986+0000 7f46d8bc4700 1 --2- 172.21.15.6:0/2818352734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d40fcf80 0x7f46d40fe2c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f46c800b3b0 tx=0x7f46c8009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:29.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.986+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2818352734 shutdown_connections 2024-03-22T06:15:29.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.986+0000 7f46d8bc4700 1 --2- 172.21.15.6:0/2818352734 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f46bc06bbe0 0x7f46bc06e090 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:29.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.986+0000 7f46d8bc4700 1 --2- 172.21.15.6:0/2818352734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d40fbcd0 0x7f46d40ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:29.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.986+0000 7f46d8bc4700 1 --2- 172.21.15.6:0/2818352734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d40fcf80 0x7f46d40fe2c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:29.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.986+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2818352734 >> 172.21.15.6:0/2818352734 conn(0x7f46d40f7430 msgr2=0x7f46d4065ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:29.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.986+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2818352734 shutdown_connections 2024-03-22T06:15:29.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:29.986+0000 7f46d8bc4700 1 -- 172.21.15.6:0/2818352734 wait complete. 2024-03-22T06:15:29.991 INFO:tasks.workunit.client.1.smithi117.stdout:9/820: creat d4/d8/d3e/d7a/d2f/d101/f10a x:0 0 0 2024-03-22T06:15:29.991 INFO:tasks.workunit.client.1.smithi117.stdout:9/821: write d4/d8/d3f/dc3/fcc [588770,60906] 0 2024-03-22T06:15:29.999 INFO:tasks.workunit.client.1.smithi117.stdout:9/822: mkdir d4/d50/dee/d10b 0 2024-03-22T06:15:29.999 INFO:tasks.workunit.client.1.smithi117.stdout:9/823: write d4/d8/d3e/d7a/f63 [5078597,44533] 0 2024-03-22T06:15:30.003 INFO:tasks.workunit.client.1.smithi117.stdout:9/824: mkdir d4/d47/d85/dab/d10c 0 2024-03-22T06:15:30.003 INFO:tasks.workunit.client.1.smithi117.stdout:9/825: dread - d4/d8/d3e/d7a/d3b/fec zero size 2024-03-22T06:15:30.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:29 smithi117 ceph-mon[36514]: pgmap v28: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 207 MiB/s rd, 246 MiB/s wr, 366 op/s 2024-03-22T06:15:30.036 INFO:tasks.workunit.client.1.smithi117.stdout:9/826: dwrite d4/d8/f2c [0,4194304] 0 2024-03-22T06:15:30.041 INFO:tasks.workunit.client.1.smithi117.stdout:9/827: getdents d4/d47/d64/d72/d8d 0 2024-03-22T06:15:30.066 INFO:tasks.workunit.client.1.smithi117.stdout:3/790: fsync d9/d1a/d39/d8a/da1/db6/f10e 0 2024-03-22T06:15:30.067 INFO:tasks.workunit.client.1.smithi117.stdout:3/791: read d9/faa [186047,75336] 0 2024-03-22T06:15:30.068 INFO:tasks.workunit.client.1.smithi117.stdout:3/792: fsync d9/d1a/d39/d4c/d36/f43 0 2024-03-22T06:15:30.080 INFO:tasks.workunit.client.1.smithi117.stdout:9/828: link d4/d7/dbf/d100/dd1/dd6/d9b/f9c d4/d50/f10d 0 2024-03-22T06:15:30.082 INFO:tasks.workunit.client.1.smithi117.stdout:9/829: symlink d4/d7/dbf/d100/dbc/l10e 0 2024-03-22T06:15:30.091 INFO:tasks.workunit.client.1.smithi117.stdout:7/750: creat d8/d50/fec x:0 0 0 2024-03-22T06:15:30.096 INFO:tasks.workunit.client.1.smithi117.stdout:3/793: mkdir d9/d1a/d39/d4c/d10f 0 2024-03-22T06:15:30.101 INFO:tasks.workunit.client.1.smithi117.stdout:6/853: rename d3/d40/d46/d84/fed to d3/db4/dd4/df4/d101/f12b 0 2024-03-22T06:15:30.103 INFO:tasks.workunit.client.1.smithi117.stdout:2/752: rename d2/d5/d13/d57/dd0/d1d/d3a/d89/lf3 to d2/d5/l101 0 2024-03-22T06:15:30.105 INFO:tasks.workunit.client.1.smithi117.stdout:6/854: link d3/d40/d46/d84/lda d3/db4/d2d/d53/d59/d74/da/de/l12c 0 2024-03-22T06:15:30.107 INFO:tasks.workunit.client.1.smithi117.stdout:8/788: rename d1/d3a/d44/d9/cde to d1/d19/d38/cf9 0 2024-03-22T06:15:30.109 INFO:tasks.workunit.client.1.smithi117.stdout:6/855: symlink d3/db4/dd4/df4/d101/da2/l12d 0 2024-03-22T06:15:30.118 INFO:tasks.workunit.client.1.smithi117.stdout:8/789: dread d1/d28/d66/fe4 [0,4194304] 0 2024-03-22T06:15:30.121 INFO:tasks.workunit.client.1.smithi117.stdout:8/790: mkdir d1/d3a/d44/d6a/dfa 0 2024-03-22T06:15:30.121 INFO:tasks.workunit.client.1.smithi117.stdout:8/791: readlink d1/d3a/ld3 0 2024-03-22T06:15:30.122 INFO:tasks.workunit.client.1.smithi117.stdout:8/792: write d1/f87 [152067,82373] 0 2024-03-22T06:15:30.122 INFO:tasks.workunit.client.1.smithi117.stdout:8/793: readlink d1/d28/d6b/da3/ld1 0 2024-03-22T06:15:30.122 INFO:tasks.workunit.client.1.smithi117.stdout:8/794: readlink d1/d28/la1 0 2024-03-22T06:15:30.122 INFO:tasks.workunit.client.1.smithi117.stdout:8/795: truncate d1/d19/f86 685903 0 2024-03-22T06:15:30.124 INFO:tasks.workunit.client.1.smithi117.stdout:8/796: getdents d1/d3a/d44 0 2024-03-22T06:15:30.124 INFO:tasks.workunit.client.1.smithi117.stdout:8/797: dread - d1/d3a/d44/d4b/fef zero size 2024-03-22T06:15:30.126 INFO:tasks.workunit.client.1.smithi117.stdout:5/848: symlink dd/df/d5e/l125 0 2024-03-22T06:15:30.126 INFO:tasks.workunit.client.1.smithi117.stdout:5/849: chown dd/d16/c53 150949 1 2024-03-22T06:15:30.126 INFO:tasks.workunit.client.1.smithi117.stdout:5/850: chown dd/d16/d1a/d32/ff7 764498 1 2024-03-22T06:15:30.128 INFO:tasks.workunit.client.1.smithi117.stdout:4/777: rename d0/d19/d34/d43/d63 to d0/d48/da0/dd2/db6/dde/d108 0 2024-03-22T06:15:30.128 INFO:tasks.workunit.client.1.smithi117.stdout:6/856: dwrite d3/d3d/f43 [0,4194304] 0 2024-03-22T06:15:30.130 INFO:tasks.workunit.client.1.smithi117.stdout:2/753: dwrite d2/d5/d13/d57/dd0/f6b [0,4194304] 0 2024-03-22T06:15:30.130 INFO:tasks.workunit.client.1.smithi117.stdout:2/754: chown d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/fb0 0 1 2024-03-22T06:15:30.132 INFO:tasks.workunit.client.1.smithi117.stdout:5/851: mkdir dd/df/d5e/d91/d72/dde/de0/d126 0 2024-03-22T06:15:30.135 INFO:tasks.workunit.client.1.smithi117.stdout:9/830: rename d4/d8/d3e/d7a/d2f/f54 to d4/d47/f10f 0 2024-03-22T06:15:30.147 INFO:tasks.workunit.client.1.smithi117.stdout:0/783: sync 2024-03-22T06:15:30.154 INFO:tasks.workunit.client.1.smithi117.stdout:3/794: rename d9/d1a/d39/d4c/d37/dcc/fd1 to d9/d1a/d39/d4c/d99/def/f110 0 2024-03-22T06:15:30.154 INFO:tasks.workunit.client.1.smithi117.stdout:4/778: rename d0 to d0/d48/da0/dd2/db6/dde/d108/d6f/d70/d109 22 2024-03-22T06:15:30.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.154+0000 7f193e911700 1 -- 172.21.15.6:0/1120691672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19380fcf70 msgr2=0x7f19380fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:30.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.154+0000 7f193e911700 1 --2- 172.21.15.6:0/1120691672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19380fcf70 0x7f19380fd3e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f192c004940 tx=0x7f192c009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:30.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.154+0000 7f193e911700 1 -- 172.21.15.6:0/1120691672 shutdown_connections 2024-03-22T06:15:30.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.154+0000 7f193e911700 1 --2- 172.21.15.6:0/1120691672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19380fcf70 0x7f19380fd3e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:30.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.154+0000 7f193e911700 1 --2- 172.21.15.6:0/1120691672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19380fbe00 0x7f19380fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:30.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.154+0000 7f193e911700 1 -- 172.21.15.6:0/1120691672 >> 172.21.15.6:0/1120691672 conn(0x7f19380f74e0 msgr2=0x7f19380f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:30.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.154+0000 7f193e911700 1 -- 172.21.15.6:0/1120691672 shutdown_connections 2024-03-22T06:15:30.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.154+0000 7f193e911700 1 -- 172.21.15.6:0/1120691672 wait complete. 2024-03-22T06:15:30.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.155+0000 7f193e911700 1 Processor -- start 2024-03-22T06:15:30.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.155+0000 7f193e911700 1 -- start start 2024-03-22T06:15:30.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.155+0000 7f193e911700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19380fbe00 0x7f1938114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:30.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.156+0000 7f193e911700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19380fcf70 0x7f19381145c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:30.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.156+0000 7f193e911700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1938114c50 con 0x7f19380fbe00 2024-03-22T06:15:30.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.156+0000 7f1937fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19380fbe00 0x7f1938114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:30.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.156+0000 7f1937fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19380fbe00 0x7f1938114080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47106/0 (socket says 172.21.15.6:47106) 2024-03-22T06:15:30.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.156+0000 7f1937fff700 1 -- 172.21.15.6:0/1317724409 learned_addr learned my addr 172.21.15.6:0/1317724409 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:15:30.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.156+0000 7f193e911700 1 -- 172.21.15.6:0/1317724409 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1938115440 con 0x7f19380fcf70 2024-03-22T06:15:30.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.156+0000 7f19377fe700 1 --2- 172.21.15.6:0/1317724409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19380fcf70 0x7f19381145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:30.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.156+0000 7f1937fff700 1 -- 172.21.15.6:0/1317724409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19380fcf70 msgr2=0x7f19381145c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:30.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.156+0000 7f1937fff700 1 --2- 172.21.15.6:0/1317724409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19380fcf70 0x7f19381145c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:30.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.156+0000 7f1937fff700 1 -- 172.21.15.6:0/1317724409 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c005040 con 0x7f19380fbe00 2024-03-22T06:15:30.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.157+0000 7f1937fff700 1 --2- 172.21.15.6:0/1317724409 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19380fbe00 0x7f1938114080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f192800b790 tx=0x7f1928009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:30.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.157+0000 7f1934ff9700 1 -- 172.21.15.6:0/1317724409 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f192800f6f0 con 0x7f19380fbe00 2024-03-22T06:15:30.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.157+0000 7f193e911700 1 -- 172.21.15.6:0/1317724409 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1938115720 con 0x7f19380fbe00 2024-03-22T06:15:30.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.157+0000 7f1934ff9700 1 -- 172.21.15.6:0/1317724409 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f192800fd20 con 0x7f19380fbe00 2024-03-22T06:15:30.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.157+0000 7f193e911700 1 -- 172.21.15.6:0/1317724409 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1938115ce0 con 0x7f19380fbe00 2024-03-22T06:15:30.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.157+0000 7f1934ff9700 1 -- 172.21.15.6:0/1317724409 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19280103a0 con 0x7f19380fbe00 2024-03-22T06:15:30.159 INFO:tasks.workunit.client.1.smithi117.stdout:0/784: symlink d1/d3/d2b/d3e/de/l115 0 2024-03-22T06:15:30.159 INFO:tasks.workunit.client.1.smithi117.stdout:0/785: readlink d1/d3/d2b/d3e/de/d1f/d59/le4 0 2024-03-22T06:15:30.159 INFO:tasks.workunit.client.1.smithi117.stdout:0/786: fdatasync d1/d10/d18/d8c/fb5 0 2024-03-22T06:15:30.159 INFO:tasks.workunit.client.1.smithi117.stdout:0/787: dread d1/d10/d18/f7b [0,4194304] 0 2024-03-22T06:15:30.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.159+0000 7f193e911700 1 -- 172.21.15.6:0/1317724409 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19240052e0 con 0x7f19380fbe00 2024-03-22T06:15:30.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.160+0000 7f1934ff9700 1 -- 172.21.15.6:0/1317724409 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f192800f850 con 0x7f19380fbe00 2024-03-22T06:15:30.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.160+0000 7f1934ff9700 1 --2- 172.21.15.6:0/1317724409 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f192006bb10 0x7f192006dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:30.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.160+0000 7f1934ff9700 1 -- 172.21.15.6:0/1317724409 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f192808a2f0 con 0x7f19380fbe00 2024-03-22T06:15:30.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.160+0000 7f19377fe700 1 --2- 172.21.15.6:0/1317724409 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f192006bb10 0x7f192006dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:30.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.161+0000 7f19377fe700 1 --2- 172.21.15.6:0/1317724409 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f192006bb10 0x7f192006dfc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f192c009fd0 tx=0x7f192c009e30 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:30.164 INFO:tasks.workunit.client.1.smithi117.stdout:6/857: dwrite d3/d3d/dcb/dcc/fea [0,4194304] 0 2024-03-22T06:15:30.165 INFO:tasks.workunit.client.1.smithi117.stdout:0/788: mknod d1/d3/d2b/d3e/de/c116 0 2024-03-22T06:15:30.165 INFO:tasks.workunit.client.1.smithi117.stdout:0/789: write d1/d10/d18/d26/d49/f103 [805910,35331] 0 2024-03-22T06:15:30.166 INFO:tasks.workunit.client.1.smithi117.stdout:4/779: dread d0/d19/d34/d43/fce [0,4194304] 0 2024-03-22T06:15:30.167 INFO:tasks.workunit.client.1.smithi117.stdout:2/755: dwrite d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/fb0 [0,4194304] 0 2024-03-22T06:15:30.167 INFO:tasks.workunit.client.1.smithi117.stdout:9/831: dread d4/d8/d3e/d7a/da9/fc7 [0,4194304] 0 2024-03-22T06:15:30.167 INFO:tasks.workunit.client.1.smithi117.stdout:2/756: chown d2/d5/da5/fc7 3 1 2024-03-22T06:15:30.167 INFO:tasks.workunit.client.1.smithi117.stdout:2/757: fdatasync d2/d35/d4c/f9f 0 2024-03-22T06:15:30.167 INFO:tasks.workunit.client.1.smithi117.stdout:5/852: dwrite dd/df/d5e/d91/d36/d70/f105 [0,4194304] 0 2024-03-22T06:15:30.168 INFO:tasks.workunit.client.1.smithi117.stdout:1/827: sync 2024-03-22T06:15:30.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.170+0000 7f1934ff9700 1 -- 172.21.15.6:0/1317724409 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f19280577d0 con 0x7f19380fbe00 2024-03-22T06:15:30.171 INFO:tasks.workunit.client.1.smithi117.stdout:0/790: dread d1/d10/d18/f38 [0,4194304] 0 2024-03-22T06:15:30.172 INFO:tasks.workunit.client.1.smithi117.stdout:6/858: mkdir d3/d40/d46/d84/d49/d12e 0 2024-03-22T06:15:30.174 INFO:tasks.workunit.client.1.smithi117.stdout:4/780: dread d0/d4/d2a/d5b/f6e [0,4194304] 0 2024-03-22T06:15:30.177 INFO:tasks.workunit.client.1.smithi117.stdout:9/832: mknod d4/d8/d3e/d7a/d2f/d101/c110 0 2024-03-22T06:15:30.183 INFO:tasks.workunit.client.1.smithi117.stdout:4/781: dread d0/f25 [0,4194304] 0 2024-03-22T06:15:30.183 INFO:tasks.workunit.client.1.smithi117.stdout:4/782: write d0/d4/d2a/f30 [1523736,44331] 0 2024-03-22T06:15:30.186 INFO:tasks.workunit.client.1.smithi117.stdout:0/791: mkdir d1/d3/dae/dbe/d117 0 2024-03-22T06:15:30.186 INFO:tasks.workunit.client.1.smithi117.stdout:5/853: dwrite dd/df/d5e/d91/f23 [0,4194304] 0 2024-03-22T06:15:30.186 INFO:tasks.workunit.client.1.smithi117.stdout:5/854: write dd/df/d5e/d91/d72/d33/f68 [7647734,94937] 0 2024-03-22T06:15:30.187 INFO:tasks.workunit.client.1.smithi117.stdout:3/795: dwrite d9/d1a/d39/d4c/d36/fb2 [0,4194304] 0 2024-03-22T06:15:30.187 INFO:tasks.workunit.client.1.smithi117.stdout:9/833: dread d4/d8/d3e/d7a/ff6 [0,4194304] 0 2024-03-22T06:15:30.187 INFO:tasks.workunit.client.1.smithi117.stdout:6/859: unlink d3/db4/d2d/d53/d59/d74/da/db3/c39 0 2024-03-22T06:15:30.188 INFO:tasks.workunit.client.1.smithi117.stdout:6/860: stat d3/db4/d2d/d53/ce6 0 2024-03-22T06:15:30.192 INFO:tasks.workunit.client.1.smithi117.stdout:2/758: rename d2/d5/d41 to d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d102 0 2024-03-22T06:15:30.198 INFO:tasks.workunit.client.1.smithi117.stdout:1/828: unlink d1/d22/d27/d4e/l5f 0 2024-03-22T06:15:30.198 INFO:tasks.workunit.client.1.smithi117.stdout:3/796: dread d9/d1a/f20 [0,4194304] 0 2024-03-22T06:15:30.199 INFO:tasks.workunit.client.1.smithi117.stdout:9/834: write d4/d8/d3e/d7a/ff6 [1679586,40670] 0 2024-03-22T06:15:30.199 INFO:tasks.workunit.client.1.smithi117.stdout:9/835: chown d4/d8/d3e/d7a/d3b/fe4 241198845 1 2024-03-22T06:15:30.199 INFO:tasks.workunit.client.1.smithi117.stdout:9/836: write d4/d8/d3f/fff [2783272,122792] 0 2024-03-22T06:15:30.199 INFO:tasks.workunit.client.1.smithi117.stdout:9/837: fsync d4/d8/f67 0 2024-03-22T06:15:30.199 INFO:tasks.workunit.client.1.smithi117.stdout:3/797: read d9/d55/f67 [1686259,122258] 0 2024-03-22T06:15:30.202 INFO:tasks.workunit.client.1.smithi117.stdout:1/829: dread d1/d22/d27/d4e/f7d [0,4194304] 0 2024-03-22T06:15:30.202 INFO:tasks.workunit.client.1.smithi117.stdout:6/861: dread d3/db4/f44 [0,4194304] 0 2024-03-22T06:15:30.202 INFO:tasks.workunit.client.1.smithi117.stdout:0/792: mkdir d1/d10/d118 0 2024-03-22T06:15:30.202 INFO:tasks.workunit.client.1.smithi117.stdout:0/793: fsync d1/d3/d2b/fef 0 2024-03-22T06:15:30.219 INFO:tasks.workunit.client.1.smithi117.stdout:6/862: dwrite d3/db4/d2d/d53/d59/d74/d28/ddb/f12a [0,4194304] 0 2024-03-22T06:15:30.219 INFO:tasks.workunit.client.1.smithi117.stdout:6/863: fsync d3/db4/d2d/d53/d59/d74/da/f29 0 2024-03-22T06:15:30.219 INFO:tasks.workunit.client.1.smithi117.stdout:6/864: dread - d3/d40/f105 zero size 2024-03-22T06:15:30.220 INFO:tasks.workunit.client.1.smithi117.stdout:2/759: mknod d2/d5/d13/d57/dd0/d1d/d3a/d71/c103 0 2024-03-22T06:15:30.220 INFO:tasks.workunit.client.1.smithi117.stdout:2/760: creat d2/d35/d38/f104 x:0 0 0 2024-03-22T06:15:30.224 INFO:tasks.workunit.client.1.smithi117.stdout:9/838: stat d4/d7/c98 0 2024-03-22T06:15:30.227 INFO:tasks.workunit.client.1.smithi117.stdout:3/798: symlink d9/d1a/d39/d8a/l111 0 2024-03-22T06:15:30.240 INFO:tasks.workunit.client.1.smithi117.stdout:5/855: getdents dd/df/d5e/d91/d72/d35/da2 0 2024-03-22T06:15:30.247 INFO:tasks.workunit.client.1.smithi117.stdout:6/865: rename d3/d40/d46/d84/l5f to d3/d40/d46/d84/dbe/l12f 0 2024-03-22T06:15:30.252 INFO:tasks.workunit.client.1.smithi117.stdout:2/761: rmdir d2/d5/d13/d57/dd0/d1d/d86 39 2024-03-22T06:15:30.252 INFO:tasks.workunit.client.1.smithi117.stdout:2/762: dread - d2/d5/d13/d57/dd0/d1d/d3a/da6/fe4 zero size 2024-03-22T06:15:30.256 INFO:tasks.workunit.client.1.smithi117.stdout:1/830: getdents d1/d84/d77 0 2024-03-22T06:15:30.262 INFO:tasks.workunit.client.1.smithi117.stdout:5/856: dwrite dd/df/d5e/d91/d72/d33/d9a/f101 [0,4194304] 0 2024-03-22T06:15:30.262 INFO:tasks.workunit.client.1.smithi117.stdout:5/857: dread - dd/d16/d1a/d32/f8b zero size 2024-03-22T06:15:30.262 INFO:tasks.workunit.client.1.smithi117.stdout:5/858: chown dd/d16/d106/f123 43495093 1 2024-03-22T06:15:30.263 INFO:tasks.workunit.client.1.smithi117.stdout:5/859: write dd/d16/f6c [4144442,77504] 0 2024-03-22T06:15:30.272 INFO:tasks.workunit.client.1.smithi117.stdout:0/794: rmdir d1/d10 39 2024-03-22T06:15:30.272 INFO:tasks.workunit.client.1.smithi117.stdout:0/795: chown d1/d10/d5c/d40/dac/f108 4 1 2024-03-22T06:15:30.277 INFO:tasks.workunit.client.1.smithi117.stdout:9/839: rename d4/d47/d85/dab/d10c to d4/d7/dbf/d100/d111 0 2024-03-22T06:15:30.277 INFO:tasks.workunit.client.1.smithi117.stdout:9/840: chown d4/d47/d85 305918578 1 2024-03-22T06:15:30.277 INFO:tasks.workunit.client.1.smithi117.stdout:9/841: chown d4/d8/lb 0 1 2024-03-22T06:15:30.281 INFO:tasks.workunit.client.1.smithi117.stdout:6/866: mkdir d3/db4/d2d/d53/d11a/d11b/d130 0 2024-03-22T06:15:30.283 INFO:tasks.workunit.client.1.smithi117.stdout:7/751: sync 2024-03-22T06:15:30.283 INFO:tasks.workunit.client.1.smithi117.stdout:8/798: sync 2024-03-22T06:15:30.283 INFO:tasks.workunit.client.1.smithi117.stdout:8/799: write d1/d19/d38/f70 [2750942,53745] 0 2024-03-22T06:15:30.283 INFO:tasks.workunit.client.1.smithi117.stdout:8/800: readlink d1/d3a/d44/d6a/l73 0 2024-03-22T06:15:30.283 INFO:tasks.workunit.client.1.smithi117.stdout:7/752: write d8/d9/da/d10/d23/d27/d56/f9e [2513724,90436] 0 2024-03-22T06:15:30.286 INFO:tasks.workunit.client.1.smithi117.stdout:2/763: mknod d2/d5/d13/d57/dd0/d1d/d3a/da6/c105 0 2024-03-22T06:15:30.286 INFO:tasks.workunit.client.1.smithi117.stdout:2/764: chown d2/d5/d13/d57/dd0/d1d/d3a/c3c 1618 1 2024-03-22T06:15:30.288 INFO:tasks.workunit.client.1.smithi117.stdout:1/831: link d1/d3c/db6/cf4 d1/d1f/c123 0 2024-03-22T06:15:30.288 INFO:tasks.workunit.client.1.smithi117.stdout:1/832: stat d1/d22/d106/fab 0 2024-03-22T06:15:30.301 INFO:tasks.workunit.client.1.smithi117.stdout:2/765: dwrite d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d102/fb4 [0,4194304] 0 2024-03-22T06:15:30.302 INFO:tasks.workunit.client.1.smithi117.stdout:4/783: sync 2024-03-22T06:15:30.303 INFO:tasks.workunit.client.1.smithi117.stdout:5/860: mknod dd/df/d5e/d91/d36/d98/c127 0 2024-03-22T06:15:30.303 INFO:tasks.workunit.client.1.smithi117.stdout:5/861: creat dd/d16/d86/d93/d10f/f128 x:0 0 0 2024-03-22T06:15:30.304 INFO:tasks.workunit.client.1.smithi117.stdout:0/796: creat d1/d3/dae/dbe/d117/f119 x:0 0 0 2024-03-22T06:15:30.304 INFO:tasks.workunit.client.1.smithi117.stdout:0/797: fdatasync d1/d3/d2b/d3e/d3f/f45 0 2024-03-22T06:15:30.304 INFO:tasks.workunit.client.1.smithi117.stdout:0/798: chown d1/d10/d18/d26/d53/c102 111142 1 2024-03-22T06:15:30.304 INFO:tasks.workunit.client.1.smithi117.stdout:0/799: readlink d1/d3/lc4 0 2024-03-22T06:15:30.304 INFO:tasks.workunit.client.1.smithi117.stdout:0/800: fdatasync d1/d3/d2b/d3e/de/d1f/d3b/fa6 0 2024-03-22T06:15:30.305 INFO:tasks.workunit.client.1.smithi117.stdout:4/784: write d0/d19/d34/d43/fce [608970,27624] 0 2024-03-22T06:15:30.305 INFO:tasks.workunit.client.1.smithi117.stdout:4/785: chown d0/d19/d34/d43/fe3 5 1 2024-03-22T06:15:30.307 INFO:tasks.workunit.client.1.smithi117.stdout:3/799: rename d9/d1a/d39/d4c/d99/def/f110 to d9/d1a/d39/d8a/da1/f112 0 2024-03-22T06:15:30.307 INFO:tasks.workunit.client.1.smithi117.stdout:3/800: write d9/d1a/d39/d4c/d104/f108 [237374,108993] 0 2024-03-22T06:15:30.307 INFO:tasks.workunit.client.1.smithi117.stdout:3/801: fdatasync d9/d1a/d39/d4c/d37/dd5/fd6 0 2024-03-22T06:15:30.308 INFO:tasks.workunit.client.1.smithi117.stdout:9/842: symlink d4/d7/d27/d5b/dae/l112 0 2024-03-22T06:15:30.310 INFO:tasks.workunit.client.1.smithi117.stdout:6/867: rmdir d3/db4/d2d/d53/d59/d74/da/de 39 2024-03-22T06:15:30.310 INFO:tasks.workunit.client.1.smithi117.stdout:6/868: dread - d3/d40/f118 zero size 2024-03-22T06:15:30.311 INFO:tasks.workunit.client.1.smithi117.stdout:7/753: rmdir d8/d9/d37/d5f/dce 39 2024-03-22T06:15:30.311 INFO:tasks.workunit.client.1.smithi117.stdout:7/754: readlink d8/d9/da/d2b/dc4/d5c/la9 0 2024-03-22T06:15:30.314 INFO:tasks.workunit.client.1.smithi117.stdout:8/801: symlink d1/d28/d66/lfb 0 2024-03-22T06:15:30.318 INFO:tasks.workunit.client.1.smithi117.stdout:2/766: truncate d2/d5/da5/fa7 716966 0 2024-03-22T06:15:30.318 INFO:tasks.workunit.client.1.smithi117.stdout:2/767: dread - d2/d5/d13/d57/dd0/d1d/d3a/da6/fe4 zero size 2024-03-22T06:15:30.318 INFO:tasks.workunit.client.1.smithi117.stdout:2/768: chown d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/ffc 94667250 1 2024-03-22T06:15:30.318 INFO:tasks.workunit.client.1.smithi117.stdout:2/769: truncate d2/d5/db/d2c/d7d/dff/d39/f4b 908993 0 2024-03-22T06:15:30.319 INFO:tasks.workunit.client.1.smithi117.stdout:2/770: stat d2/d5/db/d2c/d7d/fb2 0 2024-03-22T06:15:30.326 INFO:tasks.workunit.client.1.smithi117.stdout:5/862: symlink dd/df/d5e/d91/d72/dde/de0/d126/l129 0 2024-03-22T06:15:30.326 INFO:tasks.workunit.client.1.smithi117.stdout:5/863: read dd/df/d5e/d91/d36/f31 [3866379,38320] 0 2024-03-22T06:15:30.326 INFO:tasks.workunit.client.1.smithi117.stdout:5/864: fsync dd/d16/d1a/d32/d73/db8/fb9 0 2024-03-22T06:15:30.326 INFO:tasks.workunit.client.1.smithi117.stdout:5/865: write dd/df/d5e/d91/d36/f31 [5560778,27187] 0 2024-03-22T06:15:30.327 INFO:tasks.workunit.client.1.smithi117.stdout:0/801: mknod d1/d10/d18/d26/d49/d7e/d39/d6a/c11a 0 2024-03-22T06:15:30.327 INFO:tasks.workunit.client.1.smithi117.stdout:0/802: chown d1/d10/d18/d26/db6 31888 1 2024-03-22T06:15:30.336 INFO:tasks.workunit.client.1.smithi117.stdout:9/843: mkdir d4/d7/dbf/d100/dd1/dd6/d9b/dd8/d113 0 2024-03-22T06:15:30.336 INFO:tasks.workunit.client.1.smithi117.stdout:9/844: readlink d4/d8/d3e/d7a/d2f/lbe 0 2024-03-22T06:15:30.338 INFO:tasks.workunit.client.1.smithi117.stdout:7/755: rmdir d8/d9/d37/d45 39 2024-03-22T06:15:30.338 INFO:tasks.workunit.client.1.smithi117.stdout:7/756: write d8/d9/da/d10/d23/d48/dc8/fe8 [4725982,6294] 0 2024-03-22T06:15:30.338 INFO:tasks.workunit.client.1.smithi117.stdout:7/757: truncate d8/d9/da/d2b/dc4/f5d 326262 0 2024-03-22T06:15:30.341 INFO:tasks.workunit.client.1.smithi117.stdout:8/802: creat d1/d3a/d44/d6a/dfa/ffc x:0 0 0 2024-03-22T06:15:30.347 INFO:tasks.workunit.client.1.smithi117.stdout:2/771: mknod d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/c106 0 2024-03-22T06:15:30.349 INFO:tasks.workunit.client.1.smithi117.stdout:1/833: rename d1/c12 to d1/d22/d27/d4e/df6/c124 0 2024-03-22T06:15:30.349 INFO:tasks.workunit.client.1.smithi117.stdout:1/834: write d1/d3c/db6/fca [1020505,120619] 0 2024-03-22T06:15:30.353 INFO:tasks.workunit.client.1.smithi117.stdout:9/845: unlink d4/d7/d27/de7/ff9 0 2024-03-22T06:15:30.353 INFO:tasks.workunit.client.1.smithi117.stdout:9/846: chown d4/d7/dbf/d100/dd1/dd6/d58/f8a 9560 1 2024-03-22T06:15:30.353 INFO:tasks.workunit.client.1.smithi117.stdout:9/847: creat d4/d46/f114 x:0 0 0 2024-03-22T06:15:30.354 INFO:tasks.workunit.client.1.smithi117.stdout:5/866: dwrite dd/df/d5e/d91/deb/f11e [0,4194304] 0 2024-03-22T06:15:30.355 INFO:tasks.workunit.client.1.smithi117.stdout:7/758: symlink d8/d9/da/d2b/led 0 2024-03-22T06:15:30.356 INFO:tasks.workunit.client.1.smithi117.stdout:8/803: mkdir d1/d3a/d44/d6a/dfd 0 2024-03-22T06:15:30.356 INFO:tasks.workunit.client.1.smithi117.stdout:8/804: stat d1/d28/d66/d7f/laf 0 2024-03-22T06:15:30.356 INFO:tasks.workunit.client.1.smithi117.stdout:8/805: creat d1/d3a/d44/d6a/db7/ffe x:0 0 0 2024-03-22T06:15:30.356 INFO:tasks.workunit.client.1.smithi117.stdout:0/803: dwrite d1/d10/d5c/d40/d4e/f79 [0,4194304] 0 2024-03-22T06:15:30.356 INFO:tasks.workunit.client.1.smithi117.stdout:0/804: chown d1/d3/d2b/d3e/de/d1f/d59/c7a 0 1 2024-03-22T06:15:30.359 INFO:tasks.workunit.client.1.smithi117.stdout:4/786: rename d0/d48/da0/dd2/db6/dde/d108/d6f/d5f/ff5 to d0/d48/da0/dd2/f10a 0 2024-03-22T06:15:30.361 INFO:tasks.workunit.client.1.smithi117.stdout:3/802: rename d9/d1a/d39/d4c/d99 to d9/d1a/d39/d4c/d113 0 2024-03-22T06:15:30.364 INFO:tasks.workunit.client.1.smithi117.stdout:1/835: write d1/d22/d27/f68 [2910390,130672] 0 2024-03-22T06:15:30.364 INFO:tasks.workunit.client.1.smithi117.stdout:1/836: chown d1/d1f/d32/d33 9151 1 2024-03-22T06:15:30.364 INFO:tasks.workunit.client.1.smithi117.stdout:9/848: symlink d4/d7/d27/de7/l115 0 2024-03-22T06:15:30.370 INFO:tasks.workunit.client.1.smithi117.stdout:5/867: write dd/df/d5e/d91/d36/d70/f105 [1210223,16118] 0 2024-03-22T06:15:30.374 INFO:tasks.workunit.client.1.smithi117.stdout:8/806: unlink d1/d3a/d44/d11/d97/la0 0 2024-03-22T06:15:30.374 INFO:tasks.workunit.client.1.smithi117.stdout:8/807: write d1/d28/d6b/fd0 [907428,7904] 0 2024-03-22T06:15:30.387 INFO:tasks.workunit.client.1.smithi117.stdout:7/759: dwrite d8/d6c/fc2 [4194304,4194304] 0 2024-03-22T06:15:30.388 INFO:tasks.workunit.client.1.smithi117.stdout:2/772: rmdir d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4 39 2024-03-22T06:15:30.389 INFO:tasks.workunit.client.1.smithi117.stdout:0/805: dwrite d1/d10/d18/d8c/fc7 [0,4194304] 0 2024-03-22T06:15:30.389 INFO:tasks.workunit.client.1.smithi117.stdout:0/806: chown d1/d10/d5c/d40/d4e/f79 4707 1 2024-03-22T06:15:30.389 INFO:tasks.workunit.client.1.smithi117.stdout:1/837: dwrite d1/d30/d44/d9c/da9/de3/d112/f93 [0,4194304] 0 2024-03-22T06:15:30.389 INFO:tasks.workunit.client.1.smithi117.stdout:9/849: dread d4/d8/d3e/d7a/d2f/f7b [0,4194304] 0 2024-03-22T06:15:30.389 INFO:tasks.workunit.client.1.smithi117.stdout:4/787: dwrite d0/d48/da0/dd2/db6/dde/d108/d6f/fbc [0,4194304] 0 2024-03-22T06:15:30.392 INFO:tasks.workunit.client.1.smithi117.stdout:3/803: rename d9/d1a/d39/d7d/f7e to d9/d1a/d39/d4c/d104/f114 0 2024-03-22T06:15:30.392 INFO:tasks.workunit.client.1.smithi117.stdout:3/804: readlink d9/d1a/d39/d7d/d97/dc9/lec 0 2024-03-22T06:15:30.392 INFO:tasks.workunit.client.1.smithi117.stdout:3/805: chown d9/d1a/d39/d82/f84 38189855 1 2024-03-22T06:15:30.392 INFO:tasks.workunit.client.1.smithi117.stdout:3/806: chown d9/d66/f6d 10677 1 2024-03-22T06:15:30.394 INFO:tasks.workunit.client.1.smithi117.stdout:1/838: dread d1/d30/d50/f63 [0,4194304] 0 2024-03-22T06:15:30.397 INFO:tasks.workunit.client.1.smithi117.stdout:8/808: truncate d1/d3a/d44/d16/f7a 3625424 0 2024-03-22T06:15:30.397 INFO:tasks.workunit.client.1.smithi117.stdout:8/809: creat d1/d3a/d44/d11/fff x:0 0 0 2024-03-22T06:15:30.397 INFO:tasks.workunit.client.1.smithi117.stdout:8/810: creat d1/d28/d6b/da3/f100 x:0 0 0 2024-03-22T06:15:30.398 INFO:tasks.workunit.client.1.smithi117.stdout:0/807: dread d1/d10/f89 [0,4194304] 0 2024-03-22T06:15:30.401 INFO:tasks.workunit.client.1.smithi117.stdout:7/760: truncate f7 914056 0 2024-03-22T06:15:30.403 INFO:tasks.workunit.client.1.smithi117.stdout:2/773: mknod d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/c107 0 2024-03-22T06:15:30.409 INFO:tasks.workunit.client.1.smithi117.stdout:8/811: write d1/d19/d38/f9f [3713696,100220] 0 2024-03-22T06:15:30.410 INFO:tasks.workunit.client.1.smithi117.stdout:4/788: dwrite d0/d19/d34/f37 [0,4194304] 0 2024-03-22T06:15:30.410 INFO:tasks.workunit.client.1.smithi117.stdout:4/789: fdatasync d0/d19/d28/d47/ffb 0 2024-03-22T06:15:30.415 INFO:tasks.workunit.client.1.smithi117.stdout:7/761: dwrite d8/d9/d37/d45/d61/f71 [0,4194304] 0 2024-03-22T06:15:30.423 INFO:tasks.workunit.client.1.smithi117.stdout:9/850: creat d4/d46/da0/f116 x:0 0 0 2024-03-22T06:15:30.428 INFO:tasks.workunit.client.1.smithi117.stdout:3/807: link d9/d1a/d39/d4c/d37/dd5/fd6 d9/d1a/d39/dc7/f115 0 2024-03-22T06:15:30.429 INFO:tasks.workunit.client.1.smithi117.stdout:1/839: mknod d1/d3c/d4d/de8/dec/c125 0 2024-03-22T06:15:30.433 INFO:tasks.workunit.client.1.smithi117.stdout:0/808: link d1/d3/d2b/fef d1/d10/f11b 0 2024-03-22T06:15:30.433 INFO:tasks.workunit.client.1.smithi117.stdout:6/869: sync 2024-03-22T06:15:30.436 INFO:tasks.workunit.client.1.smithi117.stdout:9/851: dwrite d4/d7/f7f [0,4194304] 0 2024-03-22T06:15:30.444 INFO:tasks.workunit.client.1.smithi117.stdout:8/812: creat d1/d3a/d44/d11/dc7/f101 x:0 0 0 2024-03-22T06:15:30.447 INFO:tasks.workunit.client.1.smithi117.stdout:4/790: link d0/d51/da4/lb4 d0/d48/da0/dd2/db6/dde/d108/l10b 0 2024-03-22T06:15:30.451 INFO:tasks.workunit.client.1.smithi117.stdout:3/808: stat d9/d1a/d39/d4c/d37/l78 0 2024-03-22T06:15:30.454 INFO:tasks.workunit.client.1.smithi117.stdout:1/840: link d1/d84/d77/ff5 d1/d30/d44/d9c/da9/f126 0 2024-03-22T06:15:30.457 INFO:tasks.workunit.client.1.smithi117.stdout:0/809: rmdir d1/d10/d18/d26/d49/d7e/d39/dc5/de8 39 2024-03-22T06:15:30.457 INFO:tasks.workunit.client.1.smithi117.stdout:0/810: write d1/f4f [2244438,96810] 0 2024-03-22T06:15:30.457 INFO:tasks.workunit.client.1.smithi117.stdout:0/811: write d1/d10/d18/d26/d49/d7e/d39/d6a/fb3 [897351,41240] 0 2024-03-22T06:15:30.457 INFO:tasks.workunit.client.1.smithi117.stdout:0/812: stat d1/l15 0 2024-03-22T06:15:30.458 INFO:tasks.workunit.client.1.smithi117.stdout:6/870: mknod d3/d40/d46/d84/d49/da8/c131 0 2024-03-22T06:15:30.463 INFO:tasks.workunit.client.1.smithi117.stdout:2/774: getdents d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96 0 2024-03-22T06:15:30.463 INFO:tasks.workunit.client.1.smithi117.stdout:2/775: write d2/d6e/fee [239280,2612] 0 2024-03-22T06:15:30.463 INFO:tasks.workunit.client.1.smithi117.stdout:2/776: readlink d2/d5/db/ld7 0 2024-03-22T06:15:30.463 INFO:tasks.workunit.client.1.smithi117.stdout:2/777: creat d2/d5/db/d2c/d7d/f108 x:0 0 0 2024-03-22T06:15:30.463 INFO:tasks.workunit.client.1.smithi117.stdout:2/778: fdatasync d2/d5/db/d2c/d7d/dff/d39/d77/f80 0 2024-03-22T06:15:30.472 INFO:tasks.workunit.client.1.smithi117.stdout:6/871: dwrite d3/d3d/f2e [0,4194304] 0 2024-03-22T06:15:30.476 INFO:tasks.workunit.client.1.smithi117.stdout:9/852: rmdir d4/d46/d8c/d105 0 2024-03-22T06:15:30.481 INFO:tasks.workunit.client.1.smithi117.stdout:8/813: rename d1/d3a/d44/d11/d97/ca7 to d1/d3a/d44/d11/dc7/c102 0 2024-03-22T06:15:30.483 INFO:tasks.workunit.client.1.smithi117.stdout:4/791: mknod d0/d19/d9b/c10c 0 2024-03-22T06:15:30.483 INFO:tasks.workunit.client.1.smithi117.stdout:4/792: chown d0/d48/da0/dd2/db6/dde/d108/d6f/ffe 195407754 1 2024-03-22T06:15:30.490 INFO:tasks.workunit.client.1.smithi117.stdout:5/868: sync 2024-03-22T06:15:30.490 INFO:tasks.workunit.client.1.smithi117.stdout:7/762: rmdir d8/d50 39 2024-03-22T06:15:30.490 INFO:tasks.workunit.client.1.smithi117.stdout:7/763: chown d8/d9/da/d2b/dc4/f83 512005 1 2024-03-22T06:15:30.491 INFO:tasks.workunit.client.1.smithi117.stdout:1/841: creat d1/d22/d27/f127 x:0 0 0 2024-03-22T06:15:30.494 INFO:tasks.workunit.client.1.smithi117.stdout:2/779: getdents d2/d5/d13/d57/dd0 0 2024-03-22T06:15:30.496 INFO:tasks.workunit.client.1.smithi117.stdout:6/872: symlink d3/d40/d46/d84/d49/d12e/l132 0 2024-03-22T06:15:30.498 INFO:tasks.workunit.client.1.smithi117.stdout:9/853: creat d4/d7/dbf/f117 x:0 0 0 2024-03-22T06:15:30.502 INFO:tasks.workunit.client.1.smithi117.stdout:8/814: unlink d1/d3a/d44/d9/c7c 0 2024-03-22T06:15:30.502 INFO:tasks.workunit.client.1.smithi117.stdout:2/780: dread d2/d5/d13/d57/dd0/d1d/f6d [0,4194304] 0 2024-03-22T06:15:30.503 INFO:tasks.workunit.client.1.smithi117.stdout:4/793: symlink d0/dc0/l10d 0 2024-03-22T06:15:30.504 INFO:tasks.workunit.client.1.smithi117.stdout:5/869: mknod dd/df/d5e/d91/d72/d60/d6e/c12a 0 2024-03-22T06:15:30.506 INFO:tasks.workunit.client.1.smithi117.stdout:7/764: symlink d8/d9/da/d10/d23/d27/d56/lee 0 2024-03-22T06:15:30.510 INFO:tasks.workunit.client.1.smithi117.stdout:3/809: rename d9/d1a/d39/d8a/cbf to d9/d1a/d39/d7d/c116 0 2024-03-22T06:15:30.510 INFO:tasks.workunit.client.1.smithi117.stdout:3/810: read d9/d1a/d39/d4c/d104/f108 [245425,3810] 0 2024-03-22T06:15:30.510 INFO:tasks.workunit.client.1.smithi117.stdout:8/815: symlink d1/d3a/d44/l103 0 2024-03-22T06:15:30.514 INFO:tasks.workunit.client.1.smithi117.stdout:2/781: stat d2/d5/db/d2c/d7d/dff/d39/d45/l60 0 2024-03-22T06:15:30.517 INFO:tasks.workunit.client.1.smithi117.stdout:5/870: symlink dd/df/d5e/d91/d72/l12b 0 2024-03-22T06:15:30.517 INFO:tasks.workunit.client.1.smithi117.stdout:7/765: readlink d8/d9/da/d10/l1d 0 2024-03-22T06:15:30.517 INFO:tasks.workunit.client.1.smithi117.stdout:7/766: chown d8/d9/d37/d5f/d82 46 1 2024-03-22T06:15:30.518 INFO:tasks.workunit.client.1.smithi117.stdout:6/873: dwrite d3/db4/d2d/d53/d59/f88 [0,4194304] 0 2024-03-22T06:15:30.518 INFO:tasks.workunit.client.1.smithi117.stdout:6/874: read - d3/db4/d2d/d53/d59/d74/d28/d85/f8a zero size 2024-03-22T06:15:30.518 INFO:tasks.workunit.client.1.smithi117.stdout:9/854: dwrite d4/d8/d3e/d7a/d2f/fe0 [0,4194304] 0 2024-03-22T06:15:30.522 INFO:tasks.workunit.client.1.smithi117.stdout:0/813: rename d1/d10/d18/d26/db6/led to d1/d3/d2b/d3e/de/d1f/d59/dcb/l11c 0 2024-03-22T06:15:30.523 INFO:tasks.workunit.client.1.smithi117.stdout:8/816: truncate d1/d19/f64 1616479 0 2024-03-22T06:15:30.523 INFO:tasks.workunit.client.1.smithi117.stdout:7/767: dread d8/d9/dd1/f81 [0,4194304] 0 2024-03-22T06:15:30.526 INFO:tasks.workunit.client.1.smithi117.stdout:2/782: creat d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/dfe/f109 x:0 0 0 2024-03-22T06:15:30.526 INFO:tasks.workunit.client.1.smithi117.stdout:5/871: mkdir dd/df/d5e/d91/d72/d12c 0 2024-03-22T06:15:30.532 INFO:tasks.workunit.client.1.smithi117.stdout:9/855: write d4/d7/dbf/d100/dd1/dd6/fb2 [78779,99108] 0 2024-03-22T06:15:30.535 INFO:tasks.workunit.client.1.smithi117.stdout:1/842: rename d1/d1f/f8f to d1/d3c/d4d/f128 0 2024-03-22T06:15:30.537 INFO:tasks.workunit.client.1.smithi117.stdout:8/817: mkdir d1/d28/d104 0 2024-03-22T06:15:30.538 INFO:tasks.workunit.client.1.smithi117.stdout:2/783: symlink d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/dc9/l10a 0 2024-03-22T06:15:30.538 INFO:tasks.workunit.client.1.smithi117.stdout:2/784: dread - d2/d5/db/d2c/f85 zero size 2024-03-22T06:15:30.538 INFO:tasks.workunit.client.1.smithi117.stdout:2/785: fsync d2/d35/d38/f104 0 2024-03-22T06:15:30.539 INFO:tasks.workunit.client.1.smithi117.stdout:5/872: symlink dd/df/d5e/d91/d72/d33/d9a/l12d 0 2024-03-22T06:15:30.540 INFO:tasks.workunit.client.1.smithi117.stdout:9/856: creat d4/d8/d3e/d7a/da9/f118 x:0 0 0 2024-03-22T06:15:30.542 INFO:tasks.workunit.client.1.smithi117.stdout:9/857: dread d4/d46/f95 [0,4194304] 0 2024-03-22T06:15:30.542 INFO:tasks.workunit.client.1.smithi117.stdout:9/858: write d4/d8/d3f/f52 [3574557,5176] 0 2024-03-22T06:15:30.543 INFO:tasks.workunit.client.1.smithi117.stdout:8/818: mkdir d1/d3a/d44/d105 0 2024-03-22T06:15:30.544 INFO:tasks.workunit.client.1.smithi117.stdout:2/786: mknod d2/d5/db/d2c/d7d/dff/d39/d45/d7c/c10b 0 2024-03-22T06:15:30.545 INFO:tasks.workunit.client.1.smithi117.stdout:5/873: creat dd/df/dae/f12e x:0 0 0 2024-03-22T06:15:30.546 INFO:tasks.workunit.client.1.smithi117.stdout:3/811: dwrite d9/d1a/d39/d4c/d37/dcc/fe0 [0,4194304] 0 2024-03-22T06:15:30.547 INFO:tasks.workunit.client.1.smithi117.stdout:9/859: mkdir d4/d46/d119 0 2024-03-22T06:15:30.547 INFO:tasks.workunit.client.1.smithi117.stdout:9/860: readlink d4/d46/d8c/l106 0 2024-03-22T06:15:30.547 INFO:tasks.workunit.client.1.smithi117.stdout:9/861: write d4/d46/da0/f116 [162040,52702] 0 2024-03-22T06:15:30.548 INFO:tasks.workunit.client.1.smithi117.stdout:8/819: truncate d1/d3a/d44/d4b/dae/fb8 334333 0 2024-03-22T06:15:30.548 INFO:tasks.workunit.client.1.smithi117.stdout:6/875: dwrite d3/db4/d2d/d53/d59/f88 [0,4194304] 0 2024-03-22T06:15:30.548 INFO:tasks.workunit.client.1.smithi117.stdout:6/876: write d3/d3d/f43 [4976024,39013] 0 2024-03-22T06:15:30.548 INFO:tasks.workunit.client.1.smithi117.stdout:6/877: fdatasync d3/d40/fe8 0 2024-03-22T06:15:30.549 INFO:tasks.workunit.client.1.smithi117.stdout:7/768: dwrite d8/d9/dd1/fa0 [0,4194304] 0 2024-03-22T06:15:30.549 INFO:tasks.workunit.client.1.smithi117.stdout:7/769: chown d8/d9/d37/f4e 1073175710 1 2024-03-22T06:15:30.550 INFO:tasks.workunit.client.1.smithi117.stdout:5/874: rename dd/df/d5e/d91/d72/ccb to dd/df/d95/da0/da8/c12f 0 2024-03-22T06:15:30.550 INFO:tasks.workunit.client.1.smithi117.stdout:5/875: getdents dd/d16/d1a 0 2024-03-22T06:15:30.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.552+0000 7f193e911700 1 -- 172.21.15.6:0/1317724409 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1924000bf0 con 0x7f192006bb10 2024-03-22T06:15:30.555 INFO:tasks.workunit.client.1.smithi117.stdout:1/843: dwrite d1/d22/d27/f100 [0,4194304] 0 2024-03-22T06:15:30.556 INFO:tasks.workunit.client.1.smithi117.stdout:1/844: truncate d1/d1f/d32/dc6/ffd 680325 0 2024-03-22T06:15:30.557 INFO:tasks.workunit.client.1.smithi117.stdout:9/862: link d4/d8/d3f/fa3 d4/d46/da0/f11a 0 2024-03-22T06:15:30.557 INFO:tasks.workunit.client.1.smithi117.stdout:9/863: dread - d4/d7/dbf/d100/dd1/fe9 zero size 2024-03-22T06:15:30.561 INFO:tasks.workunit.client.1.smithi117.stdout:8/820: symlink d1/d28/d6b/l106 0 2024-03-22T06:15:30.561 INFO:tasks.workunit.client.1.smithi117.stdout:8/821: fdatasync d1/d3a/d44/d9/f6c 0 2024-03-22T06:15:30.562 INFO:tasks.workunit.client.1.smithi117.stdout:6/878: mknod d3/db4/d2d/d53/d59/d74/c133 0 2024-03-22T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 starting - - - - 2024-03-22T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (8m) 30s ago 8m 6656k - 17.2.7-726-g9c4e0c34 837124da804f a49ab95d42e6 2024-03-22T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (7m) 43s ago 7m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0a36af1f8ed0 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 running (7m) 30s ago 8m 86.7M - 9.4.7 954c08fa6188 8b365bc3fd8b 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (3m) 30s ago 3m 137M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (3m) 30s ago 3m 15.7M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (3m) 43s ago 3m 13.0M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (2m) 43s ago 2m 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:9283 running (9m) 30s ago 9m 338M - 17.2.7-726-g9c4e0c34 837124da804f 2cea0b1a5c60 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283 running (70s) 43s ago 7m 489M - 19.0.0-2299-gea01217f aa461e7c5e7f ff1047688325 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (9m) 30s ago 10m 50.5M 2048M 17.2.7-726-g9c4e0c34 837124da804f b2fe36c6b005 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (7m) 43s ago 7m 39.9M 2048M 17.2.7-726-g9c4e0c34 837124da804f 0559848da9ab 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 starting - - - - 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (7m) 43s ago 7m 14.9M - 1.5.0 0da6a335fe13 58a93ac942d3 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (6m) 30s ago 6m 104M 1559M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (6m) 30s ago 6m 111M 1559M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (5m) 30s ago 5m 106M 1559M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (5m) 43s ago 5m 125M 2581M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:15:30.567 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (5m) 43s ago 5m 108M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:15:30.567 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (4m) 43s ago 4m 120M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:15:30.567 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (6m) 30s ago 8m 56.7M - 2.43.0 a07b618ecd1d 59d1a24d5af8 2024-03-22T06:15:30.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.563+0000 7f1934ff9700 1 -- 172.21.15.6:0/1317724409 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f1924000bf0 con 0x7f192006bb10 2024-03-22T06:15:30.567 INFO:tasks.workunit.client.1.smithi117.stdout:7/770: symlink d8/d9/db0/lef 0 2024-03-22T06:15:30.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.567+0000 7f193e911700 1 -- 172.21.15.6:0/1317724409 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f192006bb10 msgr2=0x7f192006dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:30.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.567+0000 7f193e911700 1 --2- 172.21.15.6:0/1317724409 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f192006bb10 0x7f192006dfc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f192c009fd0 tx=0x7f192c009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:30.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.567+0000 7f193e911700 1 -- 172.21.15.6:0/1317724409 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19380fbe00 msgr2=0x7f1938114080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:30.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.567+0000 7f193e911700 1 --2- 172.21.15.6:0/1317724409 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19380fbe00 0x7f1938114080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f192800b790 tx=0x7f1928009250 comp rx=0 tx=0).stop 2024-03-22T06:15:30.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.567+0000 7f193e911700 1 -- 172.21.15.6:0/1317724409 shutdown_connections 2024-03-22T06:15:30.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.567+0000 7f193e911700 1 --2- 172.21.15.6:0/1317724409 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f192006bb10 0x7f192006dfc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:30.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.567+0000 7f193e911700 1 --2- 172.21.15.6:0/1317724409 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19380fbe00 0x7f1938114080 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:30.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.567+0000 7f193e911700 1 --2- 172.21.15.6:0/1317724409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19380fcf70 0x7f19381145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:30.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.567+0000 7f193e911700 1 -- 172.21.15.6:0/1317724409 >> 172.21.15.6:0/1317724409 conn(0x7f19380f74e0 msgr2=0x7f19380f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:30.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.568+0000 7f193e911700 1 -- 172.21.15.6:0/1317724409 shutdown_connections 2024-03-22T06:15:30.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.568+0000 7f193e911700 1 -- 172.21.15.6:0/1317724409 wait complete. 2024-03-22T06:15:30.570 INFO:tasks.workunit.client.1.smithi117.stdout:5/876: symlink dd/df/d5e/d91/d72/d60/dec/l130 0 2024-03-22T06:15:30.571 INFO:tasks.workunit.client.1.smithi117.stdout:3/812: dwrite d9/d1a/d39/f71 [0,4194304] 0 2024-03-22T06:15:30.575 INFO:tasks.workunit.client.1.smithi117.stdout:6/879: write d3/db4/d2d/d53/d59/d74/da/db3/fa4 [7632750,36954] 0 2024-03-22T06:15:30.580 INFO:tasks.workunit.client.1.smithi117.stdout:1/845: symlink d1/d30/d5d/l129 0 2024-03-22T06:15:30.580 INFO:tasks.workunit.client.1.smithi117.stdout:1/846: write d1/d22/d27/d114/f60 [458445,108642] 0 2024-03-22T06:15:30.588 INFO:tasks.workunit.client.1.smithi117.stdout:9/864: creat d4/d7/dbf/d100/f11b x:0 0 0 2024-03-22T06:15:30.591 INFO:tasks.workunit.client.1.smithi117.stdout:9/865: dread d4/d7/d27/f2e [0,4194304] 0 2024-03-22T06:15:30.592 INFO:tasks.workunit.client.1.smithi117.stdout:0/814: write d1/d3/d2b/f2e [385478,116751] 0 2024-03-22T06:15:30.592 INFO:tasks.workunit.client.1.smithi117.stdout:0/815: chown d1/d3/d2b/fb2 23 1 2024-03-22T06:15:30.593 INFO:tasks.workunit.client.1.smithi117.stdout:7/771: creat d8/d9/d37/d45/d61/ff0 x:0 0 0 2024-03-22T06:15:30.593 INFO:tasks.workunit.client.1.smithi117.stdout:7/772: write d8/d6c/fae [2116560,117789] 0 2024-03-22T06:15:30.594 INFO:tasks.workunit.client.1.smithi117.stdout:5/877: mknod dd/d16/d1a/d32/dfc/c131 0 2024-03-22T06:15:30.603 INFO:tasks.workunit.client.1.smithi117.stdout:3/813: unlink d9/d1a/d39/d8a/c10b 0 2024-03-22T06:15:30.609 INFO:tasks.workunit.client.1.smithi117.stdout:6/880: symlink d3/db4/d2d/d53/d59/d74/da/de/l134 0 2024-03-22T06:15:30.621 INFO:tasks.workunit.client.1.smithi117.stdout:1/847: creat d1/d30/d35/d58/f12a x:0 0 0 2024-03-22T06:15:30.627 INFO:tasks.workunit.client.1.smithi117.stdout:3/814: dwrite d9/d1a/d39/f71 [0,4194304] 0 2024-03-22T06:15:30.627 INFO:tasks.workunit.client.1.smithi117.stdout:9/866: mknod d4/d47/d64/d72/d8d/c11c 0 2024-03-22T06:15:30.630 INFO:tasks.workunit.client.1.smithi117.stdout:0/816: symlink d1/d3/d2b/d3e/dad/l11d 0 2024-03-22T06:15:30.635 INFO:tasks.workunit.client.1.smithi117.stdout:7/773: mkdir d8/d9/da/df1 0 2024-03-22T06:15:30.635 INFO:tasks.workunit.client.1.smithi117.stdout:7/774: dread - d8/d50/d5e/fbe zero size 2024-03-22T06:15:30.637 INFO:tasks.workunit.client.1.smithi117.stdout:5/878: rmdir dd/df/d5e/d91/d72/d60 39 2024-03-22T06:15:30.647 INFO:tasks.workunit.client.1.smithi117.stdout:1/848: dwrite d1/d22/d27/f68 [0,4194304] 0 2024-03-22T06:15:30.654 INFO:tasks.workunit.client.1.smithi117.stdout:9/867: symlink d4/d47/d85/l11d 0 2024-03-22T06:15:30.658 INFO:tasks.workunit.client.1.smithi117.stdout:0/817: mknod d1/d10/d5c/d40/c11e 0 2024-03-22T06:15:30.658 INFO:tasks.workunit.client.1.smithi117.stdout:0/818: chown d1/d10/l31 247 1 2024-03-22T06:15:30.659 INFO:tasks.workunit.client.1.smithi117.stdout:7/775: creat d8/d9/ff2 x:0 0 0 2024-03-22T06:15:30.660 INFO:tasks.workunit.client.1.smithi117.stdout:7/776: chown d8/d9/da/d2b/d9b/da5/la7 151876033 1 2024-03-22T06:15:30.660 INFO:tasks.workunit.client.1.smithi117.stdout:7/777: chown d8/d9/da/d10/d23/c64 3864 1 2024-03-22T06:15:30.664 INFO:tasks.workunit.client.1.smithi117.stdout:5/879: creat dd/df/d5e/d91/d72/d60/d6e/f132 x:0 0 0 2024-03-22T06:15:30.664 INFO:tasks.workunit.client.1.smithi117.stdout:5/880: chown dd/df/d5e/d91/d72/d33/c9b 114803 1 2024-03-22T06:15:30.664 INFO:tasks.workunit.client.1.smithi117.stdout:5/881: readlink dd/df/d5e/d91/d90/dca/lea 0 2024-03-22T06:15:30.664 INFO:tasks.workunit.client.1.smithi117.stdout:5/882: truncate dd/d16/d86/d93/d10f/f120 140165 0 2024-03-22T06:15:30.664 INFO:tasks.workunit.client.1.smithi117.stdout:5/883: readlink dd/l74 0 2024-03-22T06:15:30.670 INFO:tasks.workunit.client.1.smithi117.stdout:4/794: sync 2024-03-22T06:15:30.674 INFO:tasks.workunit.client.1.smithi117.stdout:1/849: mknod d1/d30/d5d/c12b 0 2024-03-22T06:15:30.674 INFO:tasks.workunit.client.1.smithi117.stdout:1/850: write d1/d14/fbf [60209,35161] 0 2024-03-22T06:15:30.674 INFO:tasks.workunit.client.1.smithi117.stdout:1/851: creat d1/d30/d35/d58/f12c x:0 0 0 2024-03-22T06:15:30.674 INFO:tasks.workunit.client.1.smithi117.stdout:1/852: dread - d1/d22/d27/d114/f10f zero size 2024-03-22T06:15:30.679 INFO:tasks.workunit.client.1.smithi117.stdout:0/819: dwrite d1/d3/d2b/d3e/d3f/f78 [0,4194304] 0 2024-03-22T06:15:30.690 INFO:tasks.workunit.client.1.smithi117.stdout:6/881: rename d3/d3d/d17/d60 to d3/d3d/d17/d31/d135 0 2024-03-22T06:15:30.690 INFO:tasks.workunit.client.1.smithi117.stdout:4/795: dwrite d0/d4/d2a/d2f/fec [0,4194304] 0 2024-03-22T06:15:30.694 INFO:tasks.workunit.client.1.smithi117.stdout:1/853: creat d1/d30/d44/d99/dd7/f12d x:0 0 0 2024-03-22T06:15:30.696 INFO:tasks.workunit.client.1.smithi117.stdout:0/820: dwrite d1/d3/f12 [0,4194304] 0 2024-03-22T06:15:30.701 INFO:tasks.workunit.client.1.smithi117.stdout:6/882: dread d3/db4/d2d/d53/d75/f8e [0,4194304] 0 2024-03-22T06:15:30.701 INFO:tasks.workunit.client.1.smithi117.stdout:6/883: chown d3/d40/d46/d84/d49/d79/cfc 1053966879 1 2024-03-22T06:15:30.703 INFO:tasks.workunit.client.1.smithi117.stdout:7/778: getdents d8/d9/d37/d45/d61 0 2024-03-22T06:15:30.703 INFO:tasks.workunit.client.1.smithi117.stdout:7/779: truncate d8/d50/d66/fe9 559844 0 2024-03-22T06:15:30.705 INFO:tasks.workunit.client.1.smithi117.stdout:3/815: rename d9/d1a/d39/d4c/d36/l6b to d9/d1a/d39/d4c/d10f/l117 0 2024-03-22T06:15:30.710 INFO:tasks.workunit.client.1.smithi117.stdout:1/854: dwrite d1/d22/d27/d4e/f102 [0,4194304] 0 2024-03-22T06:15:30.713 INFO:tasks.workunit.client.1.smithi117.stdout:6/884: mknod d3/db4/d2d/d53/d59/d74/da/c136 0 2024-03-22T06:15:30.719 INFO:tasks.workunit.client.1.smithi117.stdout:9/868: rename d4/d8/d3e/d7a/d2f/lac to d4/d8/l11e 0 2024-03-22T06:15:30.719 INFO:tasks.workunit.client.1.smithi117.stdout:6/885: dread d3/d3d/d17/d31/f87 [0,4194304] 0 2024-03-22T06:15:30.719 INFO:tasks.workunit.client.1.smithi117.stdout:6/886: readlink d3/db4/dd4/df4/d101/da2/l62 0 2024-03-22T06:15:30.720 INFO:tasks.workunit.client.1.smithi117.stdout:7/780: dwrite d8/d9/dd1/fa0 [0,4194304] 0 2024-03-22T06:15:30.721 INFO:tasks.workunit.client.1.smithi117.stdout:3/816: rename d9/d1a/d39/d4c/d37/d69/l80 to d9/d1a/d39/d4c/d37/d69/l118 0 2024-03-22T06:15:30.727 INFO:tasks.workunit.client.1.smithi117.stdout:1/855: mkdir d1/d1f/d32/d12e 0 2024-03-22T06:15:30.730 INFO:tasks.workunit.client.1.smithi117.stdout:0/821: rename d1/d3/d2b/d3e/de/d1f/d3b/l96 to d1/d3/d2b/d3e/de/l11f 0 2024-03-22T06:15:30.733 INFO:tasks.workunit.client.1.smithi117.stdout:6/887: symlink d3/db4/d2d/d53/d59/d74/l137 0 2024-03-22T06:15:30.738 INFO:tasks.workunit.client.1.smithi117.stdout:7/781: rmdir d8/d9/db0/ddb 39 2024-03-22T06:15:30.738 INFO:tasks.workunit.client.1.smithi117.stdout:7/782: write d8/d9/da/d10/d23/d48/fb9 [812568,37650] 0 2024-03-22T06:15:30.738 INFO:tasks.workunit.client.1.smithi117.stdout:7/783: truncate d8/d9/d37/fb5 256753 0 2024-03-22T06:15:30.738 INFO:tasks.workunit.client.1.smithi117.stdout:7/784: read d8/d9/da/d10/d23/f38 [743789,13606] 0 2024-03-22T06:15:30.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.739+0000 7fb9af182700 1 -- 172.21.15.6:0/1526708212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a81078e0 msgr2=0x7fb9a8109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:30.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.739+0000 7fb9af182700 1 --2- 172.21.15.6:0/1526708212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a81078e0 0x7fb9a8109cc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb99c004940 tx=0x7fb99c009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:30.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.739+0000 7fb9af182700 1 -- 172.21.15.6:0/1526708212 shutdown_connections 2024-03-22T06:15:30.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.739+0000 7fb9af182700 1 --2- 172.21.15.6:0/1526708212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a81078e0 0x7fb9a8109cc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:30.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.739+0000 7fb9af182700 1 --2- 172.21.15.6:0/1526708212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8104fc0 0x7fb9a81073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:30.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.739+0000 7fb9af182700 1 -- 172.21.15.6:0/1526708212 >> 172.21.15.6:0/1526708212 conn(0x7fb9a806a380 msgr2=0x7fb9a806c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:30.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.740+0000 7fb9af182700 1 -- 172.21.15.6:0/1526708212 shutdown_connections 2024-03-22T06:15:30.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.740+0000 7fb9af182700 1 -- 172.21.15.6:0/1526708212 wait complete. 2024-03-22T06:15:30.742 INFO:tasks.workunit.client.1.smithi117.stdout:0/822: mknod d1/d3/d2b/d3e/de/d1f/d3b/d55/c120 0 2024-03-22T06:15:30.742 INFO:tasks.workunit.client.1.smithi117.stdout:0/823: chown d1/d10/d18/d26/d49/d7e/l50 60201 1 2024-03-22T06:15:30.742 INFO:tasks.workunit.client.1.smithi117.stdout:0/824: chown d1/d10/d18/d26/f58 3 1 2024-03-22T06:15:30.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.741+0000 7fb9af182700 1 Processor -- start 2024-03-22T06:15:30.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.741+0000 7fb9af182700 1 -- start start 2024-03-22T06:15:30.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.741+0000 7fb9af182700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8104fc0 0x7fb9a8113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:30.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.741+0000 7fb9af182700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a81078e0 0x7fb9a81144b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.741+0000 7fb9af182700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a8114ab0 con 0x7fb9a81078e0 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.741+0000 7fb9af182700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a8114c20 con 0x7fb9a8104fc0 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.741+0000 7fb9a7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a81078e0 0x7fb9a81144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.741+0000 7fb9a7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a81078e0 0x7fb9a81144b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47122/0 (socket says 172.21.15.6:47122) 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.741+0000 7fb9a7fff700 1 -- 172.21.15.6:0/2441314913 learned_addr learned my addr 172.21.15.6:0/2441314913 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.742+0000 7fb9a7fff700 1 -- 172.21.15.6:0/2441314913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8104fc0 msgr2=0x7fb9a8113f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.742+0000 7fb9a7fff700 1 --2- 172.21.15.6:0/2441314913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8104fc0 0x7fb9a8113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.742+0000 7fb9a7fff700 1 -- 172.21.15.6:0/2441314913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c005040 con 0x7fb9a81078e0 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.742+0000 7fb9a7fff700 1 --2- 172.21.15.6:0/2441314913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a81078e0 0x7fb9a81144b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb99c000c00 tx=0x7fb99c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.742+0000 7fb9a57fa700 1 -- 172.21.15.6:0/2441314913 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb99c00e070 con 0x7fb9a81078e0 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.743+0000 7fb9af182700 1 -- 172.21.15.6:0/2441314913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a8115600 con 0x7fb9a81078e0 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.743+0000 7fb9af182700 1 -- 172.21.15.6:0/2441314913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a8115bc0 con 0x7fb9a81078e0 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.743+0000 7fb9a57fa700 1 -- 172.21.15.6:0/2441314913 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb99c00bbf0 con 0x7fb9a81078e0 2024-03-22T06:15:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.743+0000 7fb9a57fa700 1 -- 172.21.15.6:0/2441314913 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb99c013e10 con 0x7fb9a81078e0 2024-03-22T06:15:30.745 INFO:tasks.workunit.client.1.smithi117.stdout:9/869: rmdir d4/d7/d27/de7 39 2024-03-22T06:15:30.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.744+0000 7fb9af182700 1 -- 172.21.15.6:0/2441314913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9940052e0 con 0x7fb9a81078e0 2024-03-22T06:15:30.746 INFO:tasks.workunit.client.1.smithi117.stdout:3/817: dwrite d9/f35 [4194304,4194304] 0 2024-03-22T06:15:30.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.745+0000 7fb9a57fa700 1 -- 172.21.15.6:0/2441314913 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7fb99c013420 con 0x7fb9a81078e0 2024-03-22T06:15:30.746 INFO:tasks.workunit.client.1.smithi117.stdout:1/856: dwrite d1/d22/ff8 [4194304,4194304] 0 2024-03-22T06:15:30.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.746+0000 7fb9a57fa700 1 --2- 172.21.15.6:0/2441314913 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb99006bb70 0x7fb99006e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:30.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.746+0000 7fb9a57fa700 1 -- 172.21.15.6:0/2441314913 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fb99c08c2a0 con 0x7fb9a81078e0 2024-03-22T06:15:30.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.746+0000 7fb9acf1e700 1 --2- 172.21.15.6:0/2441314913 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb99006bb70 0x7fb99006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:30.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.747+0000 7fb9acf1e700 1 --2- 172.21.15.6:0/2441314913 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb99006bb70 0x7fb99006e020 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0004ed0 tx=0x7fb9a0009b50 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:30.748 INFO:tasks.workunit.client.1.smithi117.stdout:7/785: link d8/d9/da/f4b d8/d9/d37/d45/d88/db1/dc6/de0/ff3 0 2024-03-22T06:15:30.749 INFO:tasks.workunit.client.1.smithi117.stdout:0/825: rmdir d1/d10/d18/d26/d53 39 2024-03-22T06:15:30.750 INFO:tasks.workunit.client.1.smithi117.stdout:0/826: chown d1/d10/d18/l81 57956 1 2024-03-22T06:15:30.751 INFO:tasks.workunit.client.1.smithi117.stdout:9/870: creat d4/d7/dbf/d100/dd1/dd6/d58/f11f x:0 0 0 2024-03-22T06:15:30.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:30.752+0000 7fb9a57fa700 1 -- 172.21.15.6:0/2441314913 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7fb99c0595b0 con 0x7fb9a81078e0 2024-03-22T06:15:30.755 INFO:tasks.workunit.client.1.smithi117.stdout:6/888: rename d3/d3d/dcb to d3/db4/d2d/d53/d59/d74/d28/ddb/d138 0 2024-03-22T06:15:30.755 INFO:tasks.workunit.client.1.smithi117.stdout:6/889: write d3/db4/d2d/d53/d59/d74/d28/d85/fba [1291463,127874] 0 2024-03-22T06:15:30.755 INFO:tasks.workunit.client.1.smithi117.stdout:6/890: creat d3/d3d/d17/d31/d10a/f139 x:0 0 0 2024-03-22T06:15:30.755 INFO:tasks.workunit.client.1.smithi117.stdout:6/891: chown d3/d3d/d17/cff 263670920 1 2024-03-22T06:15:30.755 INFO:tasks.workunit.client.1.smithi117.stdout:6/892: chown d3/db4/d2d/d53/d59/d74/da/db3 310609 1 2024-03-22T06:15:30.755 INFO:tasks.workunit.client.1.smithi117.stdout:6/893: chown d3/db4/dd4/df4/d101/da2/lb2 19832 1 2024-03-22T06:15:30.757 INFO:tasks.workunit.client.1.smithi117.stdout:1/857: write d1/d30/fa8 [1168501,99913] 0 2024-03-22T06:15:30.757 INFO:tasks.workunit.client.1.smithi117.stdout:1/858: chown d1/d30/d44/d99 486671210 1 2024-03-22T06:15:30.757 INFO:tasks.workunit.client.1.smithi117.stdout:0/827: creat d1/d10/d5c/d40/d4e/dbc/f121 x:0 0 0 2024-03-22T06:15:30.757 INFO:tasks.workunit.client.1.smithi117.stdout:1/859: write d1/d30/d44/dd3/f10e [264582,113644] 0 2024-03-22T06:15:30.758 INFO:tasks.workunit.client.1.smithi117.stdout:9/871: mkdir d4/d8/d3e/d7a/d3b/d120 0 2024-03-22T06:15:30.759 INFO:tasks.workunit.client.1.smithi117.stdout:3/818: dread d9/d1a/d39/f71 [0,4194304] 0 2024-03-22T06:15:30.759 INFO:tasks.workunit.client.1.smithi117.stdout:6/894: rmdir d3/db4 39 2024-03-22T06:15:30.760 INFO:tasks.workunit.client.1.smithi117.stdout:0/828: creat d1/d10/d5c/d40/d69/f122 x:0 0 0 2024-03-22T06:15:30.765 INFO:tasks.workunit.client.1.smithi117.stdout:1/860: write d1/d1f/f79 [1291301,82428] 0 2024-03-22T06:15:30.766 INFO:tasks.workunit.client.1.smithi117.stdout:1/861: read d1/d84/d77/fd2 [538869,43322] 0 2024-03-22T06:15:30.767 INFO:tasks.workunit.client.1.smithi117.stdout:8/822: sync 2024-03-22T06:15:30.767 INFO:tasks.workunit.client.1.smithi117.stdout:2/787: sync 2024-03-22T06:15:30.769 INFO:tasks.workunit.client.1.smithi117.stdout:0/829: dread d1/d3/d2b/d3e/de/f23 [0,4194304] 0 2024-03-22T06:15:30.769 INFO:tasks.workunit.client.1.smithi117.stdout:9/872: mkdir d4/d7/dbf/d100/dd1/dd6/d9b/dd8/d113/d121 0 2024-03-22T06:15:30.770 INFO:tasks.workunit.client.1.smithi117.stdout:3/819: getdents d9/d1a/d39/d8a/da1 0 2024-03-22T06:15:30.774 INFO:tasks.workunit.client.1.smithi117.stdout:6/895: rename d3/db4/d2d/d53/d59/d74/d28/d85/f8a to d3/db4/f13a 0 2024-03-22T06:15:30.780 INFO:tasks.workunit.client.1.smithi117.stdout:3/820: write d9/d1a/d39/d4c/d36/f9f [1824605,17704] 0 2024-03-22T06:15:30.781 INFO:tasks.workunit.client.1.smithi117.stdout:1/862: unlink d1/d30/d44/d9c/da9/f113 0 2024-03-22T06:15:30.782 INFO:tasks.workunit.client.1.smithi117.stdout:3/821: dread d9/d1a/d39/d8a/fca [0,4194304] 0 2024-03-22T06:15:30.785 INFO:tasks.workunit.client.1.smithi117.stdout:8/823: mkdir d1/d3a/d44/d6a/dfa/d107 0 2024-03-22T06:15:30.787 INFO:tasks.workunit.client.1.smithi117.stdout:2/788: mknod d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/c10c 0 2024-03-22T06:15:30.790 INFO:tasks.workunit.client.1.smithi117.stdout:0/830: creat d1/d3/d2b/d3e/de/d1f/d59/f123 x:0 0 0 2024-03-22T06:15:30.791 INFO:tasks.workunit.client.1.smithi117.stdout:0/831: chown d1/d3/f12 16 1 2024-03-22T06:15:30.791 INFO:tasks.workunit.client.1.smithi117.stdout:0/832: chown d1/d10/d18 15839280 1 2024-03-22T06:15:30.791 INFO:tasks.workunit.client.1.smithi117.stdout:0/833: fdatasync d1/d3/d2b/d3e/de/d1f/f3c 0 2024-03-22T06:15:30.796 INFO:tasks.workunit.client.1.smithi117.stdout:6/896: rmdir d3/d40/d46/d84/d49/d12e 39 2024-03-22T06:15:30.796 INFO:tasks.workunit.client.1.smithi117.stdout:6/897: truncate d3/db4/d2d/f110 400641 0 2024-03-22T06:15:30.796 INFO:tasks.workunit.client.1.smithi117.stdout:3/822: read d9/d1a/d39/d4c/f95 [178266,98992] 0 2024-03-22T06:15:30.796 INFO:tasks.workunit.client.1.smithi117.stdout:3/823: chown d9/d1a/d39/d8a/da1/f112 1770 1 2024-03-22T06:15:30.803 INFO:tasks.workunit.client.1.smithi117.stdout:1/863: mkdir d1/d62/d12f 0 2024-03-22T06:15:30.803 INFO:tasks.workunit.client.1.smithi117.stdout:1/864: chown d1/d30/d5d/d6f/l11f 6282116 1 2024-03-22T06:15:30.809 INFO:tasks.workunit.client.1.smithi117.stdout:8/824: mkdir d1/d3a/d44/d16/d108 0 2024-03-22T06:15:30.814 INFO:tasks.workunit.client.1.smithi117.stdout:2/789: symlink d2/d35/l10d 0 2024-03-22T06:15:30.819 INFO:tasks.workunit.client.1.smithi117.stdout:1/865: write d1/d22/d27/d4e/f102 [2007567,97572] 0 2024-03-22T06:15:30.819 INFO:tasks.workunit.client.1.smithi117.stdout:1/866: write d1/d30/d44/d99/dd7/f12d [105476,100862] 0 2024-03-22T06:15:30.819 INFO:tasks.workunit.client.1.smithi117.stdout:6/898: creat d3/db4/d2d/d53/d59/d74/da/db3/f13b x:0 0 0 2024-03-22T06:15:30.822 INFO:tasks.workunit.client.1.smithi117.stdout:0/834: dwrite d1/d3/d2b/d3e/de/d1f/d59/f123 [0,4194304] 0 2024-03-22T06:15:30.822 INFO:tasks.workunit.client.1.smithi117.stdout:0/835: stat d1/d3/l20 0 2024-03-22T06:15:30.841 INFO:tasks.workunit.client.1.smithi117.stdout:8/825: dwrite d1/d28/d66/d7f/d9d/ff1 [0,4194304] 0 2024-03-22T06:15:30.845 INFO:tasks.workunit.client.1.smithi117.stdout:2/790: unlink d2/d5/db/c81 0 2024-03-22T06:15:30.845 INFO:tasks.workunit.client.1.smithi117.stdout:2/791: creat d2/d6e/f10e x:0 0 0 2024-03-22T06:15:30.845 INFO:tasks.workunit.client.1.smithi117.stdout:2/792: write d2/d35/d38/f104 [1008220,19197] 0 2024-03-22T06:15:30.850 INFO:tasks.workunit.client.1.smithi117.stdout:8/826: dread d1/fa9 [0,4194304] 0 2024-03-22T06:15:30.857 INFO:tasks.workunit.client.1.smithi117.stdout:1/867: dwrite d1/d30/d35/d58/f12a [0,4194304] 0 2024-03-22T06:15:30.857 INFO:tasks.workunit.client.1.smithi117.stdout:1/868: fsync d1/d30/d44/d99/fb7 0 2024-03-22T06:15:30.858 INFO:tasks.workunit.client.1.smithi117.stdout:1/869: write d1/d1f/f20 [4528391,49823] 0 2024-03-22T06:15:30.860 INFO:tasks.workunit.client.1.smithi117.stdout:1/870: dread d1/d22/d27/d4e/f6a [0,4194304] 0 2024-03-22T06:15:30.861 INFO:tasks.workunit.client.1.smithi117.stdout:5/884: sync 2024-03-22T06:15:30.863 INFO:tasks.workunit.client.1.smithi117.stdout:6/899: unlink d3/d40/d46/fa1 0 2024-03-22T06:15:30.870 INFO:tasks.workunit.client.1.smithi117.stdout:0/836: link d1/f3d d1/d10/d18/d26/d53/f124 0 2024-03-22T06:15:30.873 INFO:tasks.workunit.client.1.smithi117.stdout:3/824: rename d9/d1a/d39/d7d/c109 to d9/d1a/d39/d4c/d36/d41/c119 0 2024-03-22T06:15:30.879 INFO:tasks.workunit.client.1.smithi117.stdout:2/793: mkdir d2/d5/d13/d57/dd0/d1d/d3a/d10f 0 2024-03-22T06:15:30.879 INFO:tasks.workunit.client.1.smithi117.stdout:2/794: getdents d2/d9a/dd3 0 2024-03-22T06:15:30.879 INFO:tasks.workunit.client.1.smithi117.stdout:2/795: creat d2/d6e/f110 x:0 0 0 2024-03-22T06:15:30.881 INFO:tasks.workunit.client.1.smithi117.stdout:8/827: mkdir d1/d28/d66/d109 0 2024-03-22T06:15:30.881 INFO:tasks.workunit.client.1.smithi117.stdout:8/828: fsync d1/d28/d66/d7f/d9d/fb4 0 2024-03-22T06:15:30.888 INFO:tasks.workunit.client.1.smithi117.stdout:3/825: dread d9/d1a/d39/d4c/d37/f61 [0,4194304] 0 2024-03-22T06:15:30.888 INFO:tasks.workunit.client.1.smithi117.stdout:3/826: truncate d9/d55/f7c 646128 0 2024-03-22T06:15:30.889 INFO:tasks.workunit.client.1.smithi117.stdout:3/827: creat d9/d55/f11a x:0 0 0 2024-03-22T06:15:30.900 INFO:tasks.workunit.client.1.smithi117.stdout:0/837: dwrite d1/d3/d2b/d3e/f60 [0,4194304] 0 2024-03-22T06:15:30.902 INFO:tasks.workunit.client.1.smithi117.stdout:6/900: dwrite d3/db4/dd4/df4/d101/f111 [0,4194304] 0 2024-03-22T06:15:30.902 INFO:tasks.workunit.client.1.smithi117.stdout:6/901: fdatasync d3/d3d/d17/d31/d135/f80 0 2024-03-22T06:15:30.902 INFO:tasks.workunit.client.1.smithi117.stdout:6/902: dread - d3/db4/d2d/ffd zero size 2024-03-22T06:15:30.902 INFO:tasks.workunit.client.1.smithi117.stdout:2/796: dwrite d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/ffc [0,4194304] 0 2024-03-22T06:15:30.902 INFO:tasks.workunit.client.1.smithi117.stdout:2/797: write d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/fc5 [496243,77489] 0 2024-03-22T06:15:30.902 INFO:tasks.workunit.client.1.smithi117.stdout:2/798: getdents d2/d5/d13/d57/dd0/d1d/d3a/d10f 0 2024-03-22T06:15:30.903 INFO:tasks.workunit.client.1.smithi117.stdout:1/871: unlink d1/d30/d44/d9c/da9/de3/d112/ldd 0 2024-03-22T06:15:30.906 INFO:tasks.workunit.client.1.smithi117.stdout:5/885: rename dd/df/d5e/d91/deb/f11e to dd/df/d5e/d91/d72/dde/de0/f133 0 2024-03-22T06:15:30.915 INFO:tasks.workunit.client.1.smithi117.stdout:8/829: mknod d1/d3a/d44/d4b/dae/c10a 0 2024-03-22T06:15:30.915 INFO:tasks.workunit.client.1.smithi117.stdout:8/830: fdatasync d1/d28/d6b/fa2 0 2024-03-22T06:15:30.915 INFO:tasks.workunit.client.1.smithi117.stdout:6/903: dwrite d3/db4/d2d/d53/d59/d74/da/db3/f13b [0,4194304] 0 2024-03-22T06:15:30.919 INFO:tasks.workunit.client.1.smithi117.stdout:3/828: symlink d9/d66/l11b 0 2024-03-22T06:15:30.919 INFO:tasks.workunit.client.1.smithi117.stdout:3/829: stat d9/d1a/d39/d4c/d36/d41/ffe 0 2024-03-22T06:15:30.919 INFO:tasks.workunit.client.1.smithi117.stdout:3/830: dread - d9/d1a/d39/d4c/f46 zero size 2024-03-22T06:15:30.919 INFO:tasks.workunit.client.1.smithi117.stdout:3/831: chown d9/d1a/f4b 460858937 1 2024-03-22T06:15:30.919 INFO:tasks.workunit.client.1.smithi117.stdout:3/832: dread - d9/d1a/d3a/f100 zero size 2024-03-22T06:15:30.924 INFO:tasks.workunit.client.1.smithi117.stdout:0/838: creat d1/d3/d2b/f125 x:0 0 0 2024-03-22T06:15:30.924 INFO:tasks.workunit.client.1.smithi117.stdout:7/786: sync 2024-03-22T06:15:30.924 INFO:tasks.workunit.client.1.smithi117.stdout:4/796: sync 2024-03-22T06:15:30.924 INFO:tasks.workunit.client.1.smithi117.stdout:9/873: sync 2024-03-22T06:15:30.924 INFO:tasks.workunit.client.1.smithi117.stdout:7/787: chown d8/d9/da/d2b/dc4/f7b 393149443 1 2024-03-22T06:15:30.924 INFO:tasks.workunit.client.1.smithi117.stdout:7/788: creat d8/d9/da/d2b/db8/ff4 x:0 0 0 2024-03-22T06:15:30.924 INFO:tasks.workunit.client.1.smithi117.stdout:7/789: stat d8/d9/d37/d45/d61 0 2024-03-22T06:15:30.925 INFO:tasks.workunit.client.1.smithi117.stdout:5/886: dwrite dd/df/d5e/d91/d72/d60/dec/ffa [0,4194304] 0 2024-03-22T06:15:30.926 INFO:tasks.workunit.client.1.smithi117.stdout:5/887: truncate dd/d16/d86/d93/d10f/f120 260925 0 2024-03-22T06:15:30.926 INFO:tasks.workunit.client.1.smithi117.stdout:5/888: readlink dd/df/d5e/d91/d72/d60/l8d 0 2024-03-22T06:15:30.926 INFO:tasks.workunit.client.1.smithi117.stdout:5/889: fsync dd/d16/d1a/d32/dbf/fe2 0 2024-03-22T06:15:30.926 INFO:tasks.workunit.client.1.smithi117.stdout:5/890: readlink dd/df/d1e/l44 0 2024-03-22T06:15:30.926 INFO:tasks.workunit.client.1.smithi117.stdout:5/891: fdatasync dd/df/d5e/d91/d72/d35/d81/fba 0 2024-03-22T06:15:30.959 INFO:tasks.workunit.client.1.smithi117.stdout:5/892: dwrite dd/df/d95/f9e [0,4194304] 0 2024-03-22T06:15:30.959 INFO:tasks.workunit.client.1.smithi117.stdout:9/874: dwrite d4/d50/fe2 [0,4194304] 0 2024-03-22T06:15:30.959 INFO:tasks.workunit.client.1.smithi117.stdout:4/797: dwrite d0/d51/fe6 [0,4194304] 0 2024-03-22T06:15:30.963 INFO:tasks.workunit.client.1.smithi117.stdout:8/831: mknod d1/d19/de0/c10b 0 2024-03-22T06:15:30.968 INFO:tasks.workunit.client.1.smithi117.stdout:9/875: dread d4/d47/d64/d72/d8d/fc8 [0,4194304] 0 2024-03-22T06:15:30.980 INFO:tasks.workunit.client.1.smithi117.stdout:2/799: write d2/d35/d38/f5e [1973595,2537] 0 2024-03-22T06:15:30.980 INFO:tasks.workunit.client.1.smithi117.stdout:2/800: write d2/d35/d38/feb [343445,100281] 0 2024-03-22T06:15:30.980 INFO:tasks.workunit.client.1.smithi117.stdout:2/801: chown d2/d5/db/lc4 0 1 2024-03-22T06:15:30.980 INFO:tasks.workunit.client.1.smithi117.stdout:2/802: truncate d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/f8c 1426920 0 2024-03-22T06:15:30.985 INFO:tasks.workunit.client.1.smithi117.stdout:4/798: dwrite d0/d4/d2a/d2f/fec [0,4194304] 0 2024-03-22T06:15:30.985 INFO:tasks.workunit.client.1.smithi117.stdout:0/839: link d1/d3/d2b/d3e/de/l115 d1/d3/d2b/d3e/d3f/d8b/l126 0 2024-03-22T06:15:30.985 INFO:tasks.workunit.client.1.smithi117.stdout:0/840: dread - d1/d10/d18/d26/d49/d7e/d39/dc5/fd5 zero size 2024-03-22T06:15:30.985 INFO:tasks.workunit.client.1.smithi117.stdout:0/841: truncate d1/d10/d5c/d40/dac/f108 927788 0 2024-03-22T06:15:30.991 INFO:tasks.workunit.client.1.smithi117.stdout:7/790: creat d8/ff5 x:0 0 0 2024-03-22T06:15:30.997 INFO:tasks.workunit.client.1.smithi117.stdout:5/893: mknod dd/df/d5e/d91/c134 0 2024-03-22T06:15:30.998 INFO:tasks.workunit.client.1.smithi117.stdout:2/803: dwrite d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/f15 [0,4194304] 0 2024-03-22T06:15:31.002 INFO:tasks.workunit.client.1.smithi117.stdout:1/872: rename d1/c66 to d1/c130 0 2024-03-22T06:15:31.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:30 smithi117 ceph-mon[36514]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:15:31.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:30 smithi117 ceph-mon[36514]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:15:31.008 INFO:tasks.workunit.client.1.smithi117.stdout:2/804: dread d2/d5/db/d2c/d7d/dff/d39/fac [0,4194304] 0 2024-03-22T06:15:31.009 INFO:tasks.workunit.client.1.smithi117.stdout:8/832: rename d1/d28/d66/d7f/f92 to d1/d3a/d44/d9/f10c 0 2024-03-22T06:15:31.009 INFO:tasks.workunit.client.1.smithi117.stdout:8/833: chown d1/d28/c60 165 1 2024-03-22T06:15:31.012 INFO:tasks.workunit.client.1.smithi117.stdout:9/876: mkdir d4/d8/d3e/d7a/da9/d122 0 2024-03-22T06:15:31.014 INFO:tasks.workunit.client.1.smithi117.stdout:6/904: symlink d3/l13c 0 2024-03-22T06:15:31.018 INFO:tasks.workunit.client.1.smithi117.stdout:3/833: rmdir d9/d1a/d39/d4c/d104 39 2024-03-22T06:15:31.018 INFO:tasks.workunit.client.1.smithi117.stdout:3/834: fsync d9/d66/fbd 0 2024-03-22T06:15:31.018 INFO:tasks.workunit.client.1.smithi117.stdout:3/835: write d9/d1a/d3a/f47 [697291,17760] 0 2024-03-22T06:15:31.018 INFO:tasks.workunit.client.1.smithi117.stdout:3/836: creat d9/d1a/d39/d4c/d37/d69/d101/f11c x:0 0 0 2024-03-22T06:15:31.024 INFO:tasks.workunit.client.1.smithi117.stdout:4/799: rename d0/d48/da0/dd2/db6/dde/d108/d6f/d70 to d0/d48/da0/dd2/db6/dde/d108/d6f/d10e 0 2024-03-22T06:15:31.024 INFO:tasks.workunit.client.1.smithi117.stdout:4/800: write d0/d48/fdd [503375,39986] 0 2024-03-22T06:15:31.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/842: truncate d1/d3/d2b/d3e/de/d1f/d3b/f6b 2138809 0 2024-03-22T06:15:31.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/843: stat d1/d10/d18/d26/d49/f103 0 2024-03-22T06:15:31.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/844: fdatasync d1/d10/d5c/d40/d4e/dbc/ddb/fea 0 2024-03-22T06:15:31.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/845: write d1/f87 [4449072,111255] 0 2024-03-22T06:15:31.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/846: chown d1/d3/d2b/d3e/de/d1f/d3b/f6b 37661801 1 2024-03-22T06:15:31.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/847: readlink d1/d3/d2b/d3e/de/l11f 0 2024-03-22T06:15:31.027 INFO:tasks.workunit.client.1.smithi117.stdout:7/791: symlink d8/d9/da/lf6 0 2024-03-22T06:15:31.027 INFO:tasks.workunit.client.1.smithi117.stdout:7/792: fsync d8/d9/da/d2b/dc4/d26/f7f 0 2024-03-22T06:15:31.027 INFO:tasks.workunit.client.1.smithi117.stdout:7/793: chown d8/ff5 52196 1 2024-03-22T06:15:31.028 INFO:tasks.workunit.client.1.smithi117.stdout:7/794: write d8/d9/dd1/fc5 [367214,14985] 0 2024-03-22T06:15:31.028 INFO:tasks.workunit.client.1.smithi117.stdout:7/795: fdatasync d8/d9/da/d2b/dc4/d26/f7f 0 2024-03-22T06:15:31.029 INFO:tasks.workunit.client.1.smithi117.stdout:5/894: link dd/df/d5e/d91/d72/fd3 dd/df/d5e/d91/d72/dde/f135 0 2024-03-22T06:15:31.029 INFO:tasks.workunit.client.1.smithi117.stdout:5/895: write dd/df/d5e/d91/d72/d35/f9f [824670,64621] 0 2024-03-22T06:15:31.030 INFO:tasks.workunit.client.1.smithi117.stdout:1/873: creat d1/d84/d77/f131 x:0 0 0 2024-03-22T06:15:31.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:30 smithi006 ceph-mon[29301]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:15:31.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:30 smithi006 ceph-mon[29301]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:15:31.032 INFO:tasks.workunit.client.1.smithi117.stdout:6/905: creat d3/d40/d46/d84/f13d x:0 0 0 2024-03-22T06:15:31.033 INFO:tasks.workunit.client.1.smithi117.stdout:6/906: fsync d3/db4/d2d/d53/d59/d74/da/de/ff0 0 2024-03-22T06:15:31.033 INFO:tasks.workunit.client.1.smithi117.stdout:2/805: dwrite d2/d5/f33 [0,4194304] 0 2024-03-22T06:15:31.033 INFO:tasks.workunit.client.1.smithi117.stdout:2/806: chown d2/d5/da5/lf1 7058 1 2024-03-22T06:15:31.034 INFO:tasks.workunit.client.1.smithi117.stdout:8/834: dwrite d1/d3a/d44/d6a/f9b [0,4194304] 0 2024-03-22T06:15:31.045 INFO:tasks.workunit.client.1.smithi117.stdout:9/877: dwrite d4/d8/d3e/f7e [0,4194304] 0 2024-03-22T06:15:31.045 INFO:tasks.workunit.client.1.smithi117.stdout:0/848: mknod d1/d10/d5c/d40/c127 0 2024-03-22T06:15:31.047 INFO:tasks.workunit.client.1.smithi117.stdout:7/796: getdents d8/d9/db0 0 2024-03-22T06:15:31.048 INFO:tasks.workunit.client.1.smithi117.stdout:6/907: dread d3/db4/dd4/df4/d101/da2/f7a [0,4194304] 0 2024-03-22T06:15:31.050 INFO:tasks.workunit.client.1.smithi117.stdout:5/896: rename dd/df/f2c to dd/df/d5e/f136 0 2024-03-22T06:15:31.051 INFO:tasks.workunit.client.1.smithi117.stdout:2/807: dwrite d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/ffc [4194304,4194304] 0 2024-03-22T06:15:31.055 INFO:tasks.workunit.client.1.smithi117.stdout:8/835: creat d1/d3a/d44/d6a/dfd/f10d x:0 0 0 2024-03-22T06:15:31.056 INFO:tasks.workunit.client.1.smithi117.stdout:3/837: mknod d9/d1a/d39/d7d/c11d 0 2024-03-22T06:15:31.057 INFO:tasks.workunit.client.1.smithi117.stdout:4/801: symlink d0/d19/l10f 0 2024-03-22T06:15:31.058 INFO:tasks.workunit.client.1.smithi117.stdout:4/802: dread - d0/d48/da0/dd2/db6/dde/d108/d6f/d10e/fa2 zero size 2024-03-22T06:15:31.058 INFO:tasks.workunit.client.1.smithi117.stdout:4/803: dread - d0/d48/fee zero size 2024-03-22T06:15:31.059 INFO:tasks.workunit.client.1.smithi117.stdout:9/878: getdents d4/d8/d3f/dc3 0 2024-03-22T06:15:31.062 INFO:tasks.workunit.client.1.smithi117.stdout:7/797: unlink d8/d9/da/d2b/led 0 2024-03-22T06:15:31.066 INFO:tasks.workunit.client.1.smithi117.stdout:6/908: unlink d3/db4/ld3 0 2024-03-22T06:15:31.067 INFO:tasks.workunit.client.1.smithi117.stdout:1/874: dwrite d1/d22/d27/d114/f96 [0,4194304] 0 2024-03-22T06:15:31.067 INFO:tasks.workunit.client.1.smithi117.stdout:1/875: readlink d1/d22/d27/d4e/l6d 0 2024-03-22T06:15:31.070 INFO:tasks.workunit.client.1.smithi117.stdout:0/849: dwrite d1/d10/d18/f7b [0,4194304] 0 2024-03-22T06:15:31.072 INFO:tasks.workunit.client.1.smithi117.stdout:8/836: symlink d1/l10e 0 2024-03-22T06:15:31.072 INFO:tasks.workunit.client.1.smithi117.stdout:5/897: dread dd/df/d5e/d91/d72/d35/f9f [0,4194304] 0 2024-03-22T06:15:31.074 INFO:tasks.workunit.client.1.smithi117.stdout:4/804: creat d0/d19/f110 x:0 0 0 2024-03-22T06:15:31.076 INFO:tasks.workunit.client.1.smithi117.stdout:9/879: creat d4/d7/dbf/d100/dd1/dd6/d9b/dd8/f123 x:0 0 0 2024-03-22T06:15:31.076 INFO:tasks.workunit.client.1.smithi117.stdout:9/880: truncate d4/d46/f114 443211 0 2024-03-22T06:15:31.080 INFO:tasks.workunit.client.1.smithi117.stdout:6/909: rename d3/db4/dd4/df4/d101/da2/c106 to d3/db4/d2d/d53/c13e 0 2024-03-22T06:15:31.080 INFO:tasks.workunit.client.1.smithi117.stdout:6/910: chown d3/db4/f5e 189737 1 2024-03-22T06:15:31.082 INFO:tasks.workunit.client.1.smithi117.stdout:1/876: write d1/d30/d35/d58/f12a [2712544,68088] 0 2024-03-22T06:15:31.082 INFO:tasks.workunit.client.1.smithi117.stdout:2/808: dwrite d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/ff4 [4194304,4194304] 0 2024-03-22T06:15:31.088 INFO:tasks.workunit.client.1.smithi117.stdout:0/850: read d1/d10/d18/d26/d53/f93 [3916919,95919] 0 2024-03-22T06:15:31.091 INFO:tasks.workunit.client.1.smithi117.stdout:3/838: dwrite d9/f19 [0,4194304] 0 2024-03-22T06:15:31.091 INFO:tasks.workunit.client.1.smithi117.stdout:3/839: dread - d9/d1a/d39/d4c/d113/fee zero size 2024-03-22T06:15:31.097 INFO:tasks.workunit.client.1.smithi117.stdout:8/837: unlink d1/d28/c60 0 2024-03-22T06:15:31.099 INFO:tasks.workunit.client.1.smithi117.stdout:4/805: dwrite d0/d19/d1c/f4d [0,4194304] 0 2024-03-22T06:15:31.099 INFO:tasks.workunit.client.1.smithi117.stdout:4/806: readlink d0/d48/da0/dd2/db6/dde/d108/d8d/l99 0 2024-03-22T06:15:31.099 INFO:tasks.workunit.client.1.smithi117.stdout:9/881: dwrite d4/d46/fa6 [0,4194304] 0 2024-03-22T06:15:31.099 INFO:tasks.workunit.client.1.smithi117.stdout:9/882: write d4/d7/dbf/d100/dd1/dd6/d58/f11f [619396,103023] 0 2024-03-22T06:15:31.102 INFO:tasks.workunit.client.1.smithi117.stdout:5/898: getdents dd/df/d95 0 2024-03-22T06:15:31.111 INFO:tasks.workunit.client.1.smithi117.stdout:1/877: link d1/d1f/fb1 d1/d3c/db6/f132 0 2024-03-22T06:15:31.111 INFO:tasks.workunit.client.1.smithi117.stdout:1/878: write d1/d30/d44/d99/dd7/f116 [199057,73056] 0 2024-03-22T06:15:31.111 INFO:tasks.workunit.client.1.smithi117.stdout:1/879: creat d1/d30/d44/d99/f133 x:0 0 0 2024-03-22T06:15:31.113 INFO:tasks.workunit.client.1.smithi117.stdout:4/807: dread d0/d19/d28/f59 [0,4194304] 0 2024-03-22T06:15:31.113 INFO:tasks.workunit.client.1.smithi117.stdout:4/808: read d0/d48/da0/dd2/db6/dde/d108/d6f/d5f/fcb [1238706,110973] 0 2024-03-22T06:15:31.113 INFO:tasks.workunit.client.1.smithi117.stdout:4/809: chown d0/d19/d28/d47/dd6 3 1 2024-03-22T06:15:31.118 INFO:tasks.workunit.client.1.smithi117.stdout:2/809: mknod d2/d5/db/d2c/d7d/dff/d39/d45/c111 0 2024-03-22T06:15:31.124 INFO:tasks.workunit.client.1.smithi117.stdout:3/840: mknod d9/d1a/d3a/c11e 0 2024-03-22T06:15:31.128 INFO:tasks.workunit.client.1.smithi117.stdout:8/838: mknod d1/d3a/d44/d11/d97/c10f 0 2024-03-22T06:15:31.130 INFO:tasks.workunit.client.1.smithi117.stdout:8/839: write d1/d28/d6b/fd0 [80177,67814] 0 2024-03-22T06:15:31.134 INFO:tasks.workunit.client.1.smithi117.stdout:9/883: rename d4/d47/d64/d72/d8d to d4/d46/da0/d124 0 2024-03-22T06:15:31.135 INFO:tasks.workunit.client.1.smithi117.stdout:5/899: stat dd/d16/d1a/l63 0 2024-03-22T06:15:31.135 INFO:tasks.workunit.client.1.smithi117.stdout:5/900: readlink dd/l49 0 2024-03-22T06:15:31.149 INFO:tasks.workunit.client.1.smithi117.stdout:5/901: dwrite dd/d16/d1a/f59 [4194304,4194304] 0 2024-03-22T06:15:31.149 INFO:tasks.workunit.client.1.smithi117.stdout:5/902: chown dd/df/d5e/d91/d36/f39 31550 1 2024-03-22T06:15:31.149 INFO:tasks.workunit.client.1.smithi117.stdout:5/903: chown dd/df/d95 8 1 2024-03-22T06:15:31.149 INFO:tasks.workunit.client.1.smithi117.stdout:5/904: truncate dd/df/d5e/d91/d36/d70/fe1 874992 0 2024-03-22T06:15:31.158 INFO:tasks.workunit.client.1.smithi117.stdout:1/880: symlink d1/d30/d44/dd3/l134 0 2024-03-22T06:15:31.159 INFO:tasks.workunit.client.1.smithi117.stdout:1/881: dread - d1/d30/d44/d99/f10a zero size 2024-03-22T06:15:31.159 INFO:tasks.workunit.client.1.smithi117.stdout:4/810: mknod d0/d19/dcc/c111 0 2024-03-22T06:15:31.164 INFO:tasks.workunit.client.1.smithi117.stdout:2/810: creat d2/d5/db/d2c/d7d/dff/d39/df5/f112 x:0 0 0 2024-03-22T06:15:31.164 INFO:tasks.workunit.client.1.smithi117.stdout:2/811: fsync d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/fd9 0 2024-03-22T06:15:31.164 INFO:tasks.workunit.client.1.smithi117.stdout:2/812: chown d2/d35/d38/feb 171 1 2024-03-22T06:15:31.166 INFO:tasks.workunit.client.1.smithi117.stdout:0/851: getdents d1/d10/d5c/d40/d4e/dbc 0 2024-03-22T06:15:31.166 INFO:tasks.workunit.client.1.smithi117.stdout:0/852: chown d1/d10/d18/d26/db6/fcc 23692787 1 2024-03-22T06:15:31.171 INFO:tasks.workunit.client.1.smithi117.stdout:8/840: symlink d1/d28/d66/d7f/l110 0 2024-03-22T06:15:31.171 INFO:tasks.workunit.client.1.smithi117.stdout:8/841: chown d1/d19/d38/f8c 48242 1 2024-03-22T06:15:31.174 INFO:tasks.workunit.client.1.smithi117.stdout:4/811: rmdir d0/d48/de4 39 2024-03-22T06:15:31.175 INFO:tasks.workunit.client.1.smithi117.stdout:2/813: mkdir d2/d35/d38/d113 0 2024-03-22T06:15:31.176 INFO:tasks.workunit.client.1.smithi117.stdout:0/853: mkdir d1/d10/d5c/d40/dac/d128 0 2024-03-22T06:15:31.177 INFO:tasks.workunit.client.1.smithi117.stdout:8/842: symlink d1/d3a/d44/d9/l111 0 2024-03-22T06:15:31.177 INFO:tasks.workunit.client.1.smithi117.stdout:8/843: write d1/d3a/fcb [125163,44647] 0 2024-03-22T06:15:31.179 INFO:tasks.workunit.client.1.smithi117.stdout:5/905: symlink dd/df/l137 0 2024-03-22T06:15:31.179 INFO:tasks.workunit.client.1.smithi117.stdout:5/906: chown dd/df/d5e/d91/d72/d60/d6e/c12a 1638 1 2024-03-22T06:15:31.181 INFO:tasks.workunit.client.1.smithi117.stdout:4/812: unlink d0/d3f/fe0 0 2024-03-22T06:15:31.181 INFO:tasks.workunit.client.1.smithi117.stdout:4/813: write d0/d19/d28/d47/fbb [516652,76277] 0 2024-03-22T06:15:31.181 INFO:tasks.workunit.client.1.smithi117.stdout:4/814: write d0/d51/d92/f102 [385315,124719] 0 2024-03-22T06:15:31.181 INFO:tasks.workunit.client.1.smithi117.stdout:4/815: write d0/d19/dcc/fe1 [289551,110282] 0 2024-03-22T06:15:31.184 INFO:tasks.workunit.client.1.smithi117.stdout:2/814: unlink d2/d5/db/d2c/d7d/dff/d39/fac 0 2024-03-22T06:15:31.186 INFO:tasks.workunit.client.1.smithi117.stdout:8/844: write d1/d28/f31 [1303190,125137] 0 2024-03-22T06:15:31.186 INFO:tasks.workunit.client.1.smithi117.stdout:0/854: mkdir d1/d3/d129 0 2024-03-22T06:15:31.187 INFO:tasks.workunit.client.1.smithi117.stdout:5/907: symlink dd/df/d5e/d91/d72/d12c/l138 0 2024-03-22T06:15:31.189 INFO:tasks.workunit.client.1.smithi117.stdout:4/816: link d0/d48/da0/dd2/db6/dde/d108/d6f/f91 d0/d51/d92/f112 0 2024-03-22T06:15:31.202 INFO:tasks.workunit.client.1.smithi117.stdout:0/855: dwrite d1/d3/d2b/d3e/f98 [0,4194304] 0 2024-03-22T06:15:31.205 INFO:tasks.workunit.client.1.smithi117.stdout:2/815: link d2/lbe d2/d5/d13/d57/dd0/d1d/d3a/d10f/l114 0 2024-03-22T06:15:31.210 INFO:tasks.workunit.client.1.smithi117.stdout:8/845: link d1/d28/c54 d1/d28/d6b/df4/c112 0 2024-03-22T06:15:31.211 INFO:tasks.workunit.client.1.smithi117.stdout:8/846: write d1/f87 [1143307,12993] 0 2024-03-22T06:15:31.211 INFO:tasks.workunit.client.1.smithi117.stdout:8/847: chown d1/d28/d6b/da3 933 1 2024-03-22T06:15:31.211 INFO:tasks.workunit.client.1.smithi117.stdout:5/908: mkdir dd/d16/d139 0 2024-03-22T06:15:31.211 INFO:tasks.workunit.client.1.smithi117.stdout:5/909: chown dd/d16/d86/d93/d10f/f120 3 1 2024-03-22T06:15:31.223 INFO:tasks.workunit.client.1.smithi117.stdout:5/910: dread dd/df/d95/f9e [0,4194304] 0 2024-03-22T06:15:31.224 INFO:tasks.workunit.client.1.smithi117.stdout:5/911: creat dd/d16/d86/d93/d10f/f13a x:0 0 0 2024-03-22T06:15:31.224 INFO:tasks.workunit.client.1.smithi117.stdout:5/912: chown dd/df/d5e/d91/d90/fa4 1289516 1 2024-03-22T06:15:31.224 INFO:tasks.workunit.client.1.smithi117.stdout:5/913: fsync dd/d16/fe3 0 2024-03-22T06:15:31.224 INFO:tasks.workunit.client.1.smithi117.stdout:0/856: creat d1/d10/d18/f12a x:0 0 0 2024-03-22T06:15:31.224 INFO:tasks.workunit.client.1.smithi117.stdout:0/857: truncate d1/d10/d18/d26/d49/d7e/d39/f88 5051625 0 2024-03-22T06:15:31.225 INFO:tasks.workunit.client.1.smithi117.stdout:0/858: chown d1/d3/d2b/d3e/de/f23 521 1 2024-03-22T06:15:31.225 INFO:tasks.workunit.client.1.smithi117.stdout:0/859: read d1/d10/d18/d26/d49/fab [373685,118730] 0 2024-03-22T06:15:31.228 INFO:tasks.workunit.client.1.smithi117.stdout:2/816: mkdir d2/d5/d13/d57/dd0/d1d/d3a/d89/d115 0 2024-03-22T06:15:31.228 INFO:tasks.workunit.client.1.smithi117.stdout:7/798: sync 2024-03-22T06:15:31.228 INFO:tasks.workunit.client.1.smithi117.stdout:6/911: sync 2024-03-22T06:15:31.235 INFO:tasks.workunit.client.1.smithi117.stdout:7/799: read d8/d6c/fc2 [7899171,92028] 0 2024-03-22T06:15:31.235 INFO:tasks.workunit.client.1.smithi117.stdout:4/817: dwrite d0/d19/d1c/fd5 [4194304,4194304] 0 2024-03-22T06:15:31.236 INFO:tasks.workunit.client.1.smithi117.stdout:4/818: write d0/d19/d34/f4f [47794,105904] 0 2024-03-22T06:15:31.242 INFO:tasks.workunit.client.1.smithi117.stdout:5/914: creat dd/d16/d1a/f13b x:0 0 0 2024-03-22T06:15:31.242 INFO:tasks.workunit.client.1.smithi117.stdout:5/915: fsync dd/d16/d1a/f37 0 2024-03-22T06:15:31.243 INFO:tasks.workunit.client.1.smithi117.stdout:7/800: dread d8/d9/da/d10/f25 [0,4194304] 0 2024-03-22T06:15:31.243 INFO:tasks.workunit.client.1.smithi117.stdout:7/801: write d8/ff5 [958272,18255] 0 2024-03-22T06:15:31.244 INFO:tasks.workunit.client.1.smithi117.stdout:0/860: mknod d1/d10/d18/db0/c12b 0 2024-03-22T06:15:31.244 INFO:tasks.workunit.client.1.smithi117.stdout:0/861: write d1/d10/d18/d26/d49/d7e/d39/d6a/f94 [1672594,50216] 0 2024-03-22T06:15:31.244 INFO:tasks.workunit.client.1.smithi117.stdout:0/862: dread - d1/d3/d2b/fef zero size 2024-03-22T06:15:31.244 INFO:tasks.workunit.client.1.smithi117.stdout:2/817: creat d2/d5/db/d2c/d7d/dff/d39/d45/ddd/f116 x:0 0 0 2024-03-22T06:15:31.247 INFO:tasks.workunit.client.1.smithi117.stdout:4/819: mkdir d0/d48/da0/dd2/db6/dde/d108/d6f/de5/d113 0 2024-03-22T06:15:31.247 INFO:tasks.workunit.client.1.smithi117.stdout:4/820: fdatasync d0/d19/d34/d43/fe3 0 2024-03-22T06:15:31.248 INFO:tasks.workunit.client.1.smithi117.stdout:5/916: mknod dd/df/d88/c13c 0 2024-03-22T06:15:31.248 INFO:tasks.workunit.client.1.smithi117.stdout:5/917: fsync dd/df/d5e/d91/f4f 0 2024-03-22T06:15:31.248 INFO:tasks.workunit.client.1.smithi117.stdout:5/918: readlink dd/df/d5e/d91/d72/d33/d9a/l12d 0 2024-03-22T06:15:31.250 INFO:tasks.workunit.client.1.smithi117.stdout:7/802: mkdir d8/d6c/df7 0 2024-03-22T06:15:31.251 INFO:tasks.workunit.client.1.smithi117.stdout:6/912: dwrite d3/db4/d2d/d53/d59/d74/d28/d85/fba [0,4194304] 0 2024-03-22T06:15:31.251 INFO:tasks.workunit.client.1.smithi117.stdout:6/913: write d3/d40/d46/d84/f13d [438360,103536] 0 2024-03-22T06:15:31.251 INFO:tasks.workunit.client.1.smithi117.stdout:1/882: sync 2024-03-22T06:15:31.251 INFO:tasks.workunit.client.1.smithi117.stdout:9/884: sync 2024-03-22T06:15:31.251 INFO:tasks.workunit.client.1.smithi117.stdout:3/841: sync 2024-03-22T06:15:31.251 INFO:tasks.workunit.client.1.smithi117.stdout:1/883: dread - d1/d62/fef zero size 2024-03-22T06:15:31.251 INFO:tasks.workunit.client.1.smithi117.stdout:1/884: fdatasync d1/d30/d35/d58/fb0 0 2024-03-22T06:15:31.251 INFO:tasks.workunit.client.1.smithi117.stdout:9/885: write d4/d47/d64/fb4 [4195307,5438] 0 2024-03-22T06:15:31.252 INFO:tasks.workunit.client.1.smithi117.stdout:0/863: creat d1/d10/d5c/d40/d4e/dbc/ddb/f12c x:0 0 0 2024-03-22T06:15:31.252 INFO:tasks.workunit.client.1.smithi117.stdout:9/886: dread d4/d46/da0/f116 [0,4194304] 0 2024-03-22T06:15:31.252 INFO:tasks.workunit.client.1.smithi117.stdout:9/887: fsync d4/d46/fa4 0 2024-03-22T06:15:31.252 INFO:tasks.workunit.client.1.smithi117.stdout:9/888: chown d4/d46/da0/f116 29 1 2024-03-22T06:15:31.253 INFO:tasks.workunit.client.1.smithi117.stdout:5/919: getdents dd 0 2024-03-22T06:15:31.255 INFO:tasks.workunit.client.1.smithi117.stdout:7/803: mknod d8/d9/da/d10/d23/d27/d56/cf8 0 2024-03-22T06:15:31.261 INFO:tasks.workunit.client.1.smithi117.stdout:6/914: truncate d3/db4/d2d/d53/d59/d74/da/f29 2423142 0 2024-03-22T06:15:31.261 INFO:tasks.workunit.client.1.smithi117.stdout:6/915: fdatasync d3/d40/d46/f117 0 2024-03-22T06:15:31.264 INFO:tasks.workunit.client.1.smithi117.stdout:3/842: rename d9/d1a/d39/d4c/d113/cb9 to d9/d1a/d39/d4c/d10f/c11f 0 2024-03-22T06:15:31.268 INFO:tasks.workunit.client.1.smithi117.stdout:1/885: getdents d1/d3c 0 2024-03-22T06:15:31.268 INFO:tasks.workunit.client.1.smithi117.stdout:1/886: chown d1/d30/d44/d9c/da9/de3/d112/la2 85133 1 2024-03-22T06:15:31.279 INFO:tasks.workunit.client.1.smithi117.stdout:0/864: link d1/d10/f89 d1/d10/d5c/d40/f12d 0 2024-03-22T06:15:31.282 INFO:tasks.workunit.client.1.smithi117.stdout:9/889: link d4/d46/da0/lde d4/d8/d93/d103/l125 0 2024-03-22T06:15:31.287 INFO:tasks.workunit.client.1.smithi117.stdout:2/818: link d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d102/d83/f87 d2/d5/db/d2c/d7d/dff/d39/d77/f117 0 2024-03-22T06:15:31.288 INFO:tasks.workunit.client.1.smithi117.stdout:5/920: creat dd/d16/d1a/d32/dfc/f13d x:0 0 0 2024-03-22T06:15:31.288 INFO:tasks.workunit.client.1.smithi117.stdout:5/921: fdatasync dd/d16/d1a/d32/d73/db8/fb9 0 2024-03-22T06:15:31.289 INFO:tasks.workunit.client.1.smithi117.stdout:5/922: chown dd/d16/d5a/f97 7 1 2024-03-22T06:15:31.290 INFO:tasks.workunit.client.1.smithi117.stdout:5/923: dread dd/d16/d5a/f83 [0,4194304] 0 2024-03-22T06:15:31.290 INFO:tasks.workunit.client.1.smithi117.stdout:5/924: fdatasync dd/df/dae/dcd/fef 0 2024-03-22T06:15:31.290 INFO:tasks.workunit.client.1.smithi117.stdout:5/925: write dd/df/d5e/d91/d72/dde/f135 [1334405,18682] 0 2024-03-22T06:15:31.291 INFO:tasks.workunit.client.1.smithi117.stdout:7/804: unlink d8/d9/da/d10/fd5 0 2024-03-22T06:15:31.292 INFO:tasks.workunit.client.1.smithi117.stdout:6/916: creat d3/db4/d2d/d53/d59/d74/d28/ddb/de7/f13f x:0 0 0 2024-03-22T06:15:31.293 INFO:tasks.workunit.client.1.smithi117.stdout:3/843: truncate d9/d66/f6d 1730150 0 2024-03-22T06:15:31.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.293+0000 7fb9af182700 1 -- 172.21.15.6:0/2441314913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb994005860 con 0x7fb9a81078e0 2024-03-22T06:15:31.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.294+0000 7fb9a57fa700 1 -- 172.21.15.6:0/2441314913 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fb99c059140 con 0x7fb9a81078e0 2024-03-22T06:15:31.296 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:15:31.296 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:15:31.296 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T06:15:31.296 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:15:31.296 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:15:31.296 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T06:15:31.296 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 1 2024-03-22T06:15:31.296 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:15:31.296 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:15:31.296 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T06:15:31.296 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:15:31.297 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:15:31.297 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:15:31.297 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:15:31.297 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:15:31.297 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-22T06:15:31.297 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 1 2024-03-22T06:15:31.297 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:15:31.297 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:15:31.297 INFO:tasks.workunit.client.1.smithi117.stdout:1/887: mkdir d1/d62/d135 0 2024-03-22T06:15:31.297 INFO:tasks.workunit.client.1.smithi117.stdout:7/805: dread d8/d9/f7c [0,4194304] 0 2024-03-22T06:15:31.297 INFO:tasks.workunit.client.1.smithi117.stdout:7/806: chown d8/d9/d37/d45/d61/f6f 198382068 1 2024-03-22T06:15:31.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.297+0000 7fb9af182700 1 -- 172.21.15.6:0/2441314913 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb99006bb70 msgr2=0x7fb99006e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:31.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.297+0000 7fb9af182700 1 --2- 172.21.15.6:0/2441314913 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb99006bb70 0x7fb99006e020 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0004ed0 tx=0x7fb9a0009b50 comp rx=0 tx=0).stop 2024-03-22T06:15:31.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.297+0000 7fb9af182700 1 -- 172.21.15.6:0/2441314913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a81078e0 msgr2=0x7fb9a81144b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:31.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.297+0000 7fb9af182700 1 --2- 172.21.15.6:0/2441314913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a81078e0 0x7fb9a81144b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb99c000c00 tx=0x7fb99c009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:31.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.297+0000 7fb9af182700 1 -- 172.21.15.6:0/2441314913 shutdown_connections 2024-03-22T06:15:31.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.297+0000 7fb9af182700 1 --2- 172.21.15.6:0/2441314913 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fb99006bb70 0x7fb99006e020 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:31.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.297+0000 7fb9af182700 1 --2- 172.21.15.6:0/2441314913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8104fc0 0x7fb9a8113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:31.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.297+0000 7fb9af182700 1 --2- 172.21.15.6:0/2441314913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a81078e0 0x7fb9a81144b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:31.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.297+0000 7fb9af182700 1 -- 172.21.15.6:0/2441314913 >> 172.21.15.6:0/2441314913 conn(0x7fb9a806a380 msgr2=0x7fb9a8108030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:31.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.297+0000 7fb9af182700 1 -- 172.21.15.6:0/2441314913 shutdown_connections 2024-03-22T06:15:31.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.298+0000 7fb9af182700 1 -- 172.21.15.6:0/2441314913 wait complete. 2024-03-22T06:15:31.299 INFO:tasks.workunit.client.1.smithi117.stdout:0/865: mkdir d1/d10/d18/d12e 0 2024-03-22T06:15:31.303 INFO:tasks.workunit.client.1.smithi117.stdout:9/890: stat d4/d7/dbf/d100/dd1/dd6/d58/cf5 0 2024-03-22T06:15:31.303 INFO:tasks.workunit.client.1.smithi117.stdout:9/891: write d4/d7/d27/fef [740687,7588] 0 2024-03-22T06:15:31.303 INFO:tasks.workunit.client.1.smithi117.stdout:9/892: stat d4/d7/dbf/d100/dd1 0 2024-03-22T06:15:31.303 INFO:tasks.workunit.client.1.smithi117.stdout:9/893: fsync d4/d47/d85/fa5 0 2024-03-22T06:15:31.305 INFO:tasks.workunit.client.1.smithi117.stdout:0/866: dread d1/d10/d18/d8c/fc7 [0,4194304] 0 2024-03-22T06:15:31.310 INFO:tasks.workunit.client.1.smithi117.stdout:5/926: mknod dd/d16/d1a/d32/d73/c13e 0 2024-03-22T06:15:31.317 INFO:tasks.workunit.client.1.smithi117.stdout:3/844: creat d9/d1a/dc1/d102/f120 x:0 0 0 2024-03-22T06:15:31.320 INFO:tasks.workunit.client.1.smithi117.stdout:1/888: rename d1/d3c/d4d/de8/fc8 to d1/d30/f136 0 2024-03-22T06:15:31.320 INFO:tasks.workunit.client.1.smithi117.stdout:1/889: creat d1/d30/d35/d58/f137 x:0 0 0 2024-03-22T06:15:31.326 INFO:tasks.workunit.client.1.smithi117.stdout:5/927: dwrite dd/df/d5e/d91/d36/f25 [4194304,4194304] 0 2024-03-22T06:15:31.326 INFO:tasks.workunit.client.1.smithi117.stdout:5/928: dread - dd/d16/d1a/f13b zero size 2024-03-22T06:15:31.326 INFO:tasks.workunit.client.1.smithi117.stdout:5/929: chown dd/df/dae/l110 55972 1 2024-03-22T06:15:31.330 INFO:tasks.workunit.client.1.smithi117.stdout:9/894: rmdir d4/d7/dbf/d100/dd1/dd9 39 2024-03-22T06:15:31.337 INFO:tasks.workunit.client.1.smithi117.stdout:9/895: dread d4/d46/da0/f11a [0,4194304] 0 2024-03-22T06:15:31.337 INFO:tasks.workunit.client.1.smithi117.stdout:9/896: creat d4/d46/da0/f126 x:0 0 0 2024-03-22T06:15:31.338 INFO:tasks.workunit.client.1.smithi117.stdout:0/867: mknod d1/d10/d5c/d40/d4e/c12f 0 2024-03-22T06:15:31.344 INFO:tasks.workunit.client.1.smithi117.stdout:2/819: rmdir d2/d5/db/d2c/d7d/dff/d39 39 2024-03-22T06:15:31.345 INFO:tasks.workunit.client.1.smithi117.stdout:2/820: write d2/d5/d13/d57/dd0/d1d/d3a/f3e [704979,30495] 0 2024-03-22T06:15:31.345 INFO:tasks.workunit.client.1.smithi117.stdout:2/821: chown d2/d5/db/d2c/d7d 714665 1 2024-03-22T06:15:31.348 INFO:tasks.workunit.client.1.smithi117.stdout:6/917: getdents d3/d40/d46/d84/d49/d79/d9d 0 2024-03-22T06:15:31.350 INFO:tasks.workunit.client.1.smithi117.stdout:3/845: unlink d9/d1a/d39/d4c/f46 0 2024-03-22T06:15:31.353 INFO:tasks.workunit.client.1.smithi117.stdout:2/822: dread d2/d5/d13/d57/f63 [0,4194304] 0 2024-03-22T06:15:31.353 INFO:tasks.workunit.client.1.smithi117.stdout:2/823: chown d2/d5/d13/d57/f36 1 1 2024-03-22T06:15:31.354 INFO:tasks.workunit.client.1.smithi117.stdout:1/890: mknod d1/d1f/d11d/c138 0 2024-03-22T06:15:31.355 INFO:tasks.workunit.client.1.smithi117.stdout:7/807: rmdir d8/d6c 39 2024-03-22T06:15:31.359 INFO:tasks.workunit.client.1.smithi117.stdout:3/846: write d9/d1a/d39/d4c/d37/f61 [1255493,118883] 0 2024-03-22T06:15:31.359 INFO:tasks.workunit.client.1.smithi117.stdout:3/847: fdatasync d9/d1a/d3a/f53 0 2024-03-22T06:15:31.359 INFO:tasks.workunit.client.1.smithi117.stdout:3/848: fdatasync d9/fa6 0 2024-03-22T06:15:31.360 INFO:tasks.workunit.client.1.smithi117.stdout:9/897: unlink d4/d8/d3e/c86 0 2024-03-22T06:15:31.362 INFO:tasks.workunit.client.1.smithi117.stdout:0/868: unlink d1/d3/dae/dbe/fd4 0 2024-03-22T06:15:31.366 INFO:tasks.workunit.client.1.smithi117.stdout:6/918: rename d3/d3d/d17/d31/df5/dfa/f109 to d3/db4/d2d/f140 0 2024-03-22T06:15:31.368 INFO:tasks.workunit.client.1.smithi117.stdout:1/891: symlink d1/d84/l139 0 2024-03-22T06:15:31.372 INFO:tasks.workunit.client.1.smithi117.stdout:5/930: rmdir dd/df/d5e/d91/d72/d60/d6e 39 2024-03-22T06:15:31.374 INFO:tasks.workunit.client.1.smithi117.stdout:3/849: mkdir d9/d1a/d39/d8a/da1/db6/d121 0 2024-03-22T06:15:31.375 INFO:tasks.workunit.client.1.smithi117.stdout:9/898: symlink d4/d8/d3e/d7a/d3b/l127 0 2024-03-22T06:15:31.377 INFO:tasks.workunit.client.1.smithi117.stdout:0/869: dwrite d1/d3/d2b/d3e/de/d1f/fff [0,4194304] 0 2024-03-22T06:15:31.377 INFO:tasks.workunit.client.1.smithi117.stdout:0/870: write d1/d3/d2b/d3e/de/d1f/f3c [1121215,49504] 0 2024-03-22T06:15:31.377 INFO:tasks.workunit.client.1.smithi117.stdout:0/871: stat d1/d10/d5c/leb 0 2024-03-22T06:15:31.377 INFO:tasks.workunit.client.1.smithi117.stdout:9/899: write d4/d8/d3e/d7a/d2f/f6f [305081,44273] 0 2024-03-22T06:15:31.377 INFO:tasks.workunit.client.1.smithi117.stdout:9/900: readlink d4/d7/d27/d5b/dae/l112 0 2024-03-22T06:15:31.378 INFO:tasks.workunit.client.1.smithi117.stdout:6/919: creat d3/d3d/f141 x:0 0 0 2024-03-22T06:15:31.381 INFO:tasks.workunit.client.1.smithi117.stdout:2/824: rename d2/d5/db/d2c/d7d/dff/d39/d77/f117 to d2/d5/db/d2c/d7d/dff/d39/d77/f118 0 2024-03-22T06:15:31.387 INFO:tasks.workunit.client.1.smithi117.stdout:1/892: mknod d1/d84/d77/c13a 0 2024-03-22T06:15:31.391 INFO:tasks.workunit.client.1.smithi117.stdout:3/850: mkdir d9/d1a/d39/d4c/d37/d69/d122 0 2024-03-22T06:15:31.395 INFO:tasks.workunit.client.1.smithi117.stdout:6/920: mkdir d3/db4/d2d/d53/d75/d142 0 2024-03-22T06:15:31.396 INFO:tasks.workunit.client.1.smithi117.stdout:2/825: symlink d2/d35/d38/l119 0 2024-03-22T06:15:31.397 INFO:tasks.workunit.client.1.smithi117.stdout:9/901: dwrite d4/d8/d3f/dc3/fcc [0,4194304] 0 2024-03-22T06:15:31.397 INFO:tasks.workunit.client.1.smithi117.stdout:0/872: dwrite d1/d10/f86 [0,4194304] 0 2024-03-22T06:15:31.397 INFO:tasks.workunit.client.1.smithi117.stdout:0/873: creat d1/d3/dae/dbe/d117/f130 x:0 0 0 2024-03-22T06:15:31.398 INFO:tasks.workunit.client.1.smithi117.stdout:2/826: read d2/d35/d4c/f8e [552528,36871] 0 2024-03-22T06:15:31.399 INFO:tasks.workunit.client.1.smithi117.stdout:1/893: creat d1/d1f/f13b x:0 0 0 2024-03-22T06:15:31.399 INFO:tasks.workunit.client.1.smithi117.stdout:1/894: dread - d1/d1f/f105 zero size 2024-03-22T06:15:31.400 INFO:tasks.workunit.client.1.smithi117.stdout:8/848: sync 2024-03-22T06:15:31.402 INFO:tasks.workunit.client.1.smithi117.stdout:4/821: sync 2024-03-22T06:15:31.406 INFO:tasks.workunit.client.1.smithi117.stdout:3/851: dwrite d9/d55/f7c [0,4194304] 0 2024-03-22T06:15:31.406 INFO:tasks.workunit.client.1.smithi117.stdout:9/902: rename d4/d46/fed to d4/d46/f128 0 2024-03-22T06:15:31.406 INFO:tasks.workunit.client.1.smithi117.stdout:6/921: dread d3/db4/d2d/d53/d59/d74/d28/ddb/d138/dcc/fe4 [0,4194304] 0 2024-03-22T06:15:31.413 INFO:tasks.workunit.client.1.smithi117.stdout:6/922: dread d3/d3d/f43 [0,4194304] 0 2024-03-22T06:15:31.415 INFO:tasks.workunit.client.1.smithi117.stdout:0/874: mkdir d1/d3/dae/dbe/d131 0 2024-03-22T06:15:31.417 INFO:tasks.workunit.client.1.smithi117.stdout:2/827: unlink d2/d5/d13/d57/dd0/d1d/d3a/da6/c105 0 2024-03-22T06:15:31.419 INFO:tasks.workunit.client.1.smithi117.stdout:1/895: creat d1/d30/d44/f13c x:0 0 0 2024-03-22T06:15:31.421 INFO:tasks.workunit.client.1.smithi117.stdout:8/849: mknod d1/d28/d104/c113 0 2024-03-22T06:15:31.422 INFO:tasks.workunit.client.1.smithi117.stdout:4/822: mknod d0/c114 0 2024-03-22T06:15:31.428 INFO:tasks.workunit.client.1.smithi117.stdout:3/852: truncate d9/d1a/d39/d4c/f4e 875991 0 2024-03-22T06:15:31.434 INFO:tasks.workunit.client.1.smithi117.stdout:9/903: symlink d4/d8/d3f/l129 0 2024-03-22T06:15:31.435 INFO:tasks.workunit.client.1.smithi117.stdout:9/904: creat d4/d8/d3e/d7a/d2f/d101/f12a x:0 0 0 2024-03-22T06:15:31.435 INFO:tasks.workunit.client.1.smithi117.stdout:9/905: fsync d4/d8/f1a 0 2024-03-22T06:15:31.437 INFO:tasks.workunit.client.1.smithi117.stdout:0/875: mkdir d1/d3/d2b/d132 0 2024-03-22T06:15:31.437 INFO:tasks.workunit.client.1.smithi117.stdout:0/876: write d1/d10/d5c/d40/d4e/dbc/fe7 [292843,83390] 0 2024-03-22T06:15:31.438 INFO:tasks.workunit.client.1.smithi117.stdout:2/828: rename d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/f8c to d2/d5/db/d2c/d7d/dff/d39/d45/d88/f11a 0 2024-03-22T06:15:31.439 INFO:tasks.workunit.client.1.smithi117.stdout:0/877: dread d1/d10/d18/d26/d49/d7e/d39/d6a/fb3 [0,4194304] 0 2024-03-22T06:15:31.440 INFO:tasks.workunit.client.1.smithi117.stdout:1/896: creat d1/d1f/d32/f13d x:0 0 0 2024-03-22T06:15:31.442 INFO:tasks.workunit.client.1.smithi117.stdout:4/823: rmdir d0/dc0 39 2024-03-22T06:15:31.442 INFO:tasks.workunit.client.1.smithi117.stdout:1/897: dread d1/d1f/fae [0,4194304] 0 2024-03-22T06:15:31.442 INFO:tasks.workunit.client.1.smithi117.stdout:1/898: fsync d1/d3c/d4d/de8/fbe 0 2024-03-22T06:15:31.444 INFO:tasks.workunit.client.1.smithi117.stdout:3/853: write d9/d1a/d39/d4c/f4e [1721449,68549] 0 2024-03-22T06:15:31.450 INFO:tasks.workunit.client.1.smithi117.stdout:2/829: mknod d2/d35/c11b 0 2024-03-22T06:15:31.453 INFO:tasks.workunit.client.1.smithi117.stdout:0/878: truncate d1/d10/d18/f38 467930 0 2024-03-22T06:15:31.453 INFO:tasks.workunit.client.1.smithi117.stdout:0/879: creat d1/d3/dae/dbe/d117/f133 x:0 0 0 2024-03-22T06:15:31.456 INFO:tasks.workunit.client.1.smithi117.stdout:4/824: mkdir d0/d4/d115 0 2024-03-22T06:15:31.457 INFO:tasks.workunit.client.1.smithi117.stdout:1/899: mkdir d1/d30/d35/d58/d13e 0 2024-03-22T06:15:31.458 INFO:tasks.workunit.client.1.smithi117.stdout:1/900: fsync d1/d30/d44/f13c 0 2024-03-22T06:15:31.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.459+0000 7f18a777b700 1 -- 172.21.15.6:0/835881263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a01078e0 msgr2=0x7f18a0109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:31.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.459+0000 7f18a777b700 1 --2- 172.21.15.6:0/835881263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a01078e0 0x7f18a0109cc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1890004940 tx=0x7f1890009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:31.461 INFO:tasks.workunit.client.1.smithi117.stdout:9/906: rename d4/d47/d85/lbd to d4/l12b 0 2024-03-22T06:15:31.461 INFO:tasks.workunit.client.1.smithi117.stdout:9/907: chown d4/d8/c34 12064 1 2024-03-22T06:15:31.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.462+0000 7f18a777b700 1 -- 172.21.15.6:0/835881263 shutdown_connections 2024-03-22T06:15:31.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.462+0000 7f18a777b700 1 --2- 172.21.15.6:0/835881263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a01078e0 0x7f18a0109cc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:31.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.462+0000 7f18a777b700 1 --2- 172.21.15.6:0/835881263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a0104fc0 0x7f18a01073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:31.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.462+0000 7f18a777b700 1 -- 172.21.15.6:0/835881263 >> 172.21.15.6:0/835881263 conn(0x7f18a006a380 msgr2=0x7f18a006c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:31.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.462+0000 7f18a777b700 1 -- 172.21.15.6:0/835881263 shutdown_connections 2024-03-22T06:15:31.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.462+0000 7f18a777b700 1 -- 172.21.15.6:0/835881263 wait complete. 2024-03-22T06:15:31.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.462+0000 7f18a777b700 1 Processor -- start 2024-03-22T06:15:31.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.463+0000 7f18a777b700 1 -- start start 2024-03-22T06:15:31.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.463+0000 7f18a777b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a0104fc0 0x7f18a01182c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:31.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.463+0000 7f18a777b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a01078e0 0x7f18a0118800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:31.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.463+0000 7f18a777b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a0118e00 con 0x7f18a0104fc0 2024-03-22T06:15:31.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.463+0000 7f18a777b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a0118f70 con 0x7f18a01078e0 2024-03-22T06:15:31.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.463+0000 7f18a5517700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a0104fc0 0x7f18a01182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:31.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.463+0000 7f18a4d16700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a01078e0 0x7f18a0118800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:31.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.463+0000 7f18a4d16700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a01078e0 0x7f18a0118800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:42852/0 (socket says 172.21.15.6:42852) 2024-03-22T06:15:31.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.463+0000 7f18a4d16700 1 -- 172.21.15.6:0/3772418066 learned_addr learned my addr 172.21.15.6:0/3772418066 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:15:31.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.463+0000 7f18a5517700 1 --2- 172.21.15.6:0/3772418066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a0104fc0 0x7f18a01182c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47136/0 (socket says 172.21.15.6:47136) 2024-03-22T06:15:31.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.464+0000 7f18a5517700 1 -- 172.21.15.6:0/3772418066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a01078e0 msgr2=0x7f18a0118800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:31.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.464+0000 7f18a5517700 1 --2- 172.21.15.6:0/3772418066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a01078e0 0x7f18a0118800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:31.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.464+0000 7f18a5517700 1 -- 172.21.15.6:0/3772418066 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1890005040 con 0x7f18a0104fc0 2024-03-22T06:15:31.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.465+0000 7f18a5517700 1 --2- 172.21.15.6:0/3772418066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a0104fc0 0x7f18a01182c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f189c0094a0 tx=0x7f189c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:31.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.465+0000 7f1895ffb700 1 -- 172.21.15.6:0/3772418066 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f189c00e970 con 0x7f18a0104fc0 2024-03-22T06:15:31.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.465+0000 7f18a777b700 1 -- 172.21.15.6:0/3772418066 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a011d970 con 0x7f18a0104fc0 2024-03-22T06:15:31.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.465+0000 7f1895ffb700 1 -- 172.21.15.6:0/3772418066 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f189c011450 con 0x7f18a0104fc0 2024-03-22T06:15:31.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.465+0000 7f1895ffb700 1 -- 172.21.15.6:0/3772418066 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f189c01a5e0 con 0x7f18a0104fc0 2024-03-22T06:15:31.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.465+0000 7f18a777b700 1 -- 172.21.15.6:0/3772418066 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a011df30 con 0x7f18a0104fc0 2024-03-22T06:15:31.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.466+0000 7f18a777b700 1 -- 172.21.15.6:0/3772418066 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a004c710 con 0x7f18a0104fc0 2024-03-22T06:15:31.467 INFO:tasks.workunit.client.1.smithi117.stdout:2/830: mkdir d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/d11c 0 2024-03-22T06:15:31.467 INFO:tasks.workunit.client.1.smithi117.stdout:3/854: dwrite d9/d93/fdc [0,4194304] 0 2024-03-22T06:15:31.467 INFO:tasks.workunit.client.1.smithi117.stdout:3/855: chown d9/d1a 76048 1 2024-03-22T06:15:31.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.467+0000 7f1895ffb700 1 -- 172.21.15.6:0/3772418066 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f189c0115c0 con 0x7f18a0104fc0 2024-03-22T06:15:31.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.467+0000 7f1895ffb700 1 --2- 172.21.15.6:0/3772418066 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f188c06bb20 0x7f188c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:31.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.468+0000 7f1895ffb700 1 -- 172.21.15.6:0/3772418066 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f189c08ade0 con 0x7f18a0104fc0 2024-03-22T06:15:31.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.468+0000 7f18a4d16700 1 --2- 172.21.15.6:0/3772418066 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f188c06bb20 0x7f188c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:31.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.468+0000 7f18a4d16700 1 --2- 172.21.15.6:0/3772418066 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f188c06bb20 0x7f188c06dfd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1890009fd0 tx=0x7f1890009e30 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:31.471 INFO:tasks.workunit.client.1.smithi117.stdout:0/880: dwrite d1/d3/d2b/d3e/de/d1f/d3b/f42 [0,4194304] 0 2024-03-22T06:15:31.471 INFO:tasks.workunit.client.1.smithi117.stdout:0/881: dread - d1/d10/d18/d26/d49/d7e/d39/dc5/fd5 zero size 2024-03-22T06:15:31.472 INFO:tasks.workunit.client.1.smithi117.stdout:4/825: symlink d0/d48/da0/dd2/d105/l116 0 2024-03-22T06:15:31.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.474+0000 7f1895ffb700 1 -- 172.21.15.6:0/3772418066 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f189c058110 con 0x7f18a0104fc0 2024-03-22T06:15:31.475 INFO:tasks.workunit.client.1.smithi117.stdout:1/901: mkdir d1/d30/d44/dd3/d13f 0 2024-03-22T06:15:31.478 INFO:tasks.workunit.client.1.smithi117.stdout:9/908: symlink d4/d7/d27/de7/l12c 0 2024-03-22T06:15:31.484 INFO:tasks.workunit.client.1.smithi117.stdout:7/808: sync 2024-03-22T06:15:31.485 INFO:tasks.workunit.client.1.smithi117.stdout:1/902: dread d1/d22/d27/d4e/f89 [0,4194304] 0 2024-03-22T06:15:31.487 INFO:tasks.workunit.client.1.smithi117.stdout:3/856: symlink d9/d1a/d39/d4c/l123 0 2024-03-22T06:15:31.493 INFO:tasks.workunit.client.1.smithi117.stdout:0/882: unlink d1/d3/d2b/d3e/c97 0 2024-03-22T06:15:31.495 INFO:tasks.workunit.client.1.smithi117.stdout:1/903: dread d1/d1f/d32/d33/fe7 [0,4194304] 0 2024-03-22T06:15:31.496 INFO:tasks.workunit.client.1.smithi117.stdout:4/826: mkdir d0/d19/d28/d47/d117 0 2024-03-22T06:15:31.502 INFO:tasks.workunit.client.1.smithi117.stdout:9/909: creat d4/d46/d8c/f12d x:0 0 0 2024-03-22T06:15:31.502 INFO:tasks.workunit.client.1.smithi117.stdout:4/827: write d0/f25 [2424102,29767] 0 2024-03-22T06:15:31.504 INFO:tasks.workunit.client.1.smithi117.stdout:2/831: symlink d2/d5/db/d2c/d7d/dff/d39/d77/l11d 0 2024-03-22T06:15:31.506 INFO:tasks.workunit.client.1.smithi117.stdout:7/809: truncate d8/d9/f99 255389 0 2024-03-22T06:15:31.506 INFO:tasks.workunit.client.1.smithi117.stdout:7/810: creat d8/d9/da/d2b/dc4/dac/ddc/ff9 x:0 0 0 2024-03-22T06:15:31.508 INFO:tasks.workunit.client.1.smithi117.stdout:3/857: truncate d9/f63 4825144 0 2024-03-22T06:15:31.510 INFO:tasks.workunit.client.1.smithi117.stdout:0/883: symlink d1/d10/d5c/d40/dac/d128/l134 0 2024-03-22T06:15:31.511 INFO:tasks.workunit.client.1.smithi117.stdout:9/910: write d4/d7/f1b [1354779,20808] 0 2024-03-22T06:15:31.513 INFO:tasks.workunit.client.1.smithi117.stdout:2/832: read d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/ffc [3798850,111538] 0 2024-03-22T06:15:31.513 INFO:tasks.workunit.client.1.smithi117.stdout:2/833: write d2/d5/d13/d57/dd0/d1d/d3a/fea [548425,122508] 0 2024-03-22T06:15:31.513 INFO:tasks.workunit.client.1.smithi117.stdout:1/904: symlink d1/d30/d35/d58/l140 0 2024-03-22T06:15:31.515 INFO:tasks.workunit.client.1.smithi117.stdout:5/931: sync 2024-03-22T06:15:31.516 INFO:tasks.workunit.client.1.smithi117.stdout:5/932: write dd/d16/f48 [575677,13366] 0 2024-03-22T06:15:31.516 INFO:tasks.workunit.client.1.smithi117.stdout:5/933: stat dd/d16/d86/d93/d10f/f120 0 2024-03-22T06:15:31.516 INFO:tasks.workunit.client.1.smithi117.stdout:5/934: fsync dd/d16/d86/f9d 0 2024-03-22T06:15:31.516 INFO:tasks.workunit.client.1.smithi117.stdout:5/935: chown dd/df/d5e/d91/d36/f39 5724733 1 2024-03-22T06:15:31.518 INFO:tasks.workunit.client.1.smithi117.stdout:4/828: dwrite d0/d48/da0/dd2/db6/dde/d108/fe2 [0,4194304] 0 2024-03-22T06:15:31.518 INFO:tasks.workunit.client.1.smithi117.stdout:4/829: fdatasync d0/f1d 0 2024-03-22T06:15:31.519 INFO:tasks.workunit.client.1.smithi117.stdout:2/834: dread d2/d35/d38/feb [0,4194304] 0 2024-03-22T06:15:31.524 INFO:tasks.workunit.client.1.smithi117.stdout:7/811: mkdir d8/d9/da/d2b/dc4/dac/dfa 0 2024-03-22T06:15:31.524 INFO:tasks.workunit.client.1.smithi117.stdout:8/850: sync 2024-03-22T06:15:31.524 INFO:tasks.workunit.client.1.smithi117.stdout:6/923: sync 2024-03-22T06:15:31.525 INFO:tasks.workunit.client.1.smithi117.stdout:8/851: stat d1/d28/d6b/le3 0 2024-03-22T06:15:31.525 INFO:tasks.workunit.client.1.smithi117.stdout:8/852: read - d1/d3a/d44/d6a/db7/ffe zero size 2024-03-22T06:15:31.525 INFO:tasks.workunit.client.1.smithi117.stdout:6/924: write d3/db4/d2d/d53/d59/d74/da/de/ff0 [1105009,32751] 0 2024-03-22T06:15:31.529 INFO:tasks.workunit.client.1.smithi117.stdout:4/830: write d0/d51/d92/f112 [2645042,20926] 0 2024-03-22T06:15:31.530 INFO:tasks.workunit.client.1.smithi117.stdout:3/858: mkdir d9/d1a/d39/d4c/d37/d69/d101/d124 0 2024-03-22T06:15:31.531 INFO:tasks.workunit.client.1.smithi117.stdout:3/859: fsync d9/d1a/d39/d7d/d97/fba 0 2024-03-22T06:15:31.533 INFO:tasks.workunit.client.1.smithi117.stdout:0/884: creat d1/d10/d5c/d40/d4e/dbc/f135 x:0 0 0 2024-03-22T06:15:31.537 INFO:tasks.workunit.client.1.smithi117.stdout:0/885: dread d1/d10/d18/d8c/fc7 [0,4194304] 0 2024-03-22T06:15:31.548 INFO:tasks.workunit.client.1.smithi117.stdout:5/936: symlink dd/d16/d100/l13f 0 2024-03-22T06:15:31.548 INFO:tasks.workunit.client.1.smithi117.stdout:5/937: creat dd/df/d88/f140 x:0 0 0 2024-03-22T06:15:31.556 INFO:tasks.workunit.client.1.smithi117.stdout:6/925: dwrite d3/db4/d2d/d53/d59/d74/d28/d85/fba [0,4194304] 0 2024-03-22T06:15:31.558 INFO:tasks.workunit.client.1.smithi117.stdout:2/835: mkdir d2/d5/db/d2c/d7d/dff/d11e 0 2024-03-22T06:15:31.562 INFO:tasks.workunit.client.1.smithi117.stdout:7/812: rename d8/d9/da/d10/d23/d48/d91/f9c to d8/d50/d5e/ffb 0 2024-03-22T06:15:31.564 INFO:tasks.workunit.client.1.smithi117.stdout:4/831: symlink d0/d48/da0/dd2/db6/dde/l118 0 2024-03-22T06:15:31.566 INFO:tasks.workunit.client.1.smithi117.stdout:0/886: write d1/d3/d2b/d3e/de/fa4 [1113353,126915] 0 2024-03-22T06:15:31.566 INFO:tasks.workunit.client.1.smithi117.stdout:0/887: write d1/f3d [4641361,115213] 0 2024-03-22T06:15:31.569 INFO:tasks.workunit.client.1.smithi117.stdout:8/853: dwrite d1/d3a/d44/d9/f13 [4194304,4194304] 0 2024-03-22T06:15:31.570 INFO:tasks.workunit.client.1.smithi117.stdout:3/860: dwrite d9/d1a/d39/d4c/d37/f61 [0,4194304] 0 2024-03-22T06:15:31.570 INFO:tasks.workunit.client.1.smithi117.stdout:3/861: dread - d9/d1a/fe2 zero size 2024-03-22T06:15:31.570 INFO:tasks.workunit.client.1.smithi117.stdout:3/862: fdatasync d9/d1a/d39/d8a/fca 0 2024-03-22T06:15:31.570 INFO:tasks.workunit.client.1.smithi117.stdout:3/863: stat d9/d1a/fa7 0 2024-03-22T06:15:31.572 INFO:tasks.workunit.client.1.smithi117.stdout:5/938: dwrite dd/df/d5e/d91/d36/f39 [0,4194304] 0 2024-03-22T06:15:31.577 INFO:tasks.workunit.client.1.smithi117.stdout:6/926: read d3/db4/d2d/d53/d59/d74/d28/d85/fba [2489466,29357] 0 2024-03-22T06:15:31.578 INFO:tasks.workunit.client.1.smithi117.stdout:7/813: dread d8/d9/da/d10/d23/d48/dc8/fe8 [0,4194304] 0 2024-03-22T06:15:31.578 INFO:tasks.workunit.client.1.smithi117.stdout:7/814: chown d8/d9/da/d10/la6 46 1 2024-03-22T06:15:31.578 INFO:tasks.workunit.client.1.smithi117.stdout:7/815: chown d8/d9/d37/d5f 10082 1 2024-03-22T06:15:31.583 INFO:tasks.workunit.client.1.smithi117.stdout:3/864: truncate d9/d1a/d39/d4c/d36/d41/f54 2231533 0 2024-03-22T06:15:31.584 INFO:tasks.workunit.client.1.smithi117.stdout:3/865: write d9/d1a/d39/d8a/fca [816208,102354] 0 2024-03-22T06:15:31.584 INFO:tasks.workunit.client.1.smithi117.stdout:3/866: chown d9/ff2 10 1 2024-03-22T06:15:31.586 INFO:tasks.workunit.client.1.smithi117.stdout:5/939: link dd/d16/f6c dd/d16/d86/dee/f141 0 2024-03-22T06:15:31.586 INFO:tasks.workunit.client.1.smithi117.stdout:5/940: write dd/df/d5e/d91/d72/d35/fcc [1750019,48992] 0 2024-03-22T06:15:31.586 INFO:tasks.workunit.client.1.smithi117.stdout:5/941: fsync dd/df/d5e/d91/d90/fa4 0 2024-03-22T06:15:31.588 INFO:tasks.workunit.client.1.smithi117.stdout:6/927: mknod d3/db4/d2d/d53/d59/d74/d28/d85/c143 0 2024-03-22T06:15:31.588 INFO:tasks.workunit.client.1.smithi117.stdout:6/928: truncate d3/db4/d2d/d53/d59/d74/f11 221745 0 2024-03-22T06:15:31.588 INFO:tasks.workunit.client.1.smithi117.stdout:6/929: chown d3/db4/dd4/df4/d101/f76 406513 1 2024-03-22T06:15:31.589 INFO:tasks.workunit.client.1.smithi117.stdout:7/816: truncate d8/d9/d37/d5f/f79 265273 0 2024-03-22T06:15:31.590 INFO:tasks.workunit.client.1.smithi117.stdout:3/867: symlink d9/d1a/d39/d8a/da1/l125 0 2024-03-22T06:15:31.591 INFO:tasks.workunit.client.1.smithi117.stdout:3/868: dread d9/d1a/faf [0,4194304] 0 2024-03-22T06:15:31.591 INFO:tasks.workunit.client.1.smithi117.stdout:3/869: dread - d9/d1a/d39/d4c/d37/d69/d101/f106 zero size 2024-03-22T06:15:31.591 INFO:tasks.workunit.client.1.smithi117.stdout:3/870: fsync d9/d1a/d39/d4c/d37/f8c 0 2024-03-22T06:15:31.592 INFO:tasks.workunit.client.1.smithi117.stdout:5/942: mknod dd/df/d5e/d91/d90/dca/c142 0 2024-03-22T06:15:31.592 INFO:tasks.workunit.client.1.smithi117.stdout:5/943: fsync dd/d16/d106/f123 0 2024-03-22T06:15:31.593 INFO:tasks.workunit.client.1.smithi117.stdout:4/832: dwrite d0/d19/d34/ff3 [0,4194304] 0 2024-03-22T06:15:31.593 INFO:tasks.workunit.client.1.smithi117.stdout:4/833: dread - d0/dc0/fea zero size 2024-03-22T06:15:31.593 INFO:tasks.workunit.client.1.smithi117.stdout:8/854: dwrite d1/d3a/d44/d11/fff [0,4194304] 0 2024-03-22T06:15:31.594 INFO:tasks.workunit.client.1.smithi117.stdout:0/888: dwrite d1/d10/f89 [0,4194304] 0 2024-03-22T06:15:31.595 INFO:tasks.workunit.client.1.smithi117.stdout:2/836: rename d2/d6e to d2/d35/d38/d11f 0 2024-03-22T06:15:31.595 INFO:tasks.workunit.client.1.smithi117.stdout:2/837: readlink d2/d5/db/d2c/d7d/dff/d39/d45/d7c/l9c 0 2024-03-22T06:15:31.595 INFO:tasks.workunit.client.1.smithi117.stdout:2/838: chown d2/d5/d13/d57/dd0/d1d/d3a/d89/dc0 29752793 1 2024-03-22T06:15:31.599 INFO:tasks.workunit.client.1.smithi117.stdout:6/930: dread d3/db4/fc5 [0,4194304] 0 2024-03-22T06:15:31.604 INFO:tasks.workunit.client.1.smithi117.stdout:5/944: mknod dd/d16/d86/d93/c143 0 2024-03-22T06:15:31.604 INFO:tasks.workunit.client.1.smithi117.stdout:5/945: chown dd/d16/d1a/d32/dbf/fe2 983 1 2024-03-22T06:15:31.607 INFO:tasks.workunit.client.1.smithi117.stdout:2/839: read d2/d5/d13/d57/dd0/d1d/d3a/d71/f9d [1679173,47717] 0 2024-03-22T06:15:31.609 INFO:tasks.workunit.client.1.smithi117.stdout:6/931: dread d3/db4/d2d/d53/d59/d74/f1b [0,4194304] 0 2024-03-22T06:15:31.609 INFO:tasks.workunit.client.1.smithi117.stdout:6/932: truncate d3/d40/d46/d84/f13d 643911 0 2024-03-22T06:15:31.616 INFO:tasks.workunit.client.1.smithi117.stdout:4/834: symlink d0/d19/d28/l119 0 2024-03-22T06:15:31.620 INFO:tasks.workunit.client.1.smithi117.stdout:8/855: dwrite d1/d3a/d44/d4b/dc9/fda [0,4194304] 0 2024-03-22T06:15:31.629 INFO:tasks.workunit.client.1.smithi117.stdout:3/871: dwrite d9/d1a/fc6 [0,4194304] 0 2024-03-22T06:15:31.629 INFO:tasks.workunit.client.1.smithi117.stdout:3/872: chown d9/d1a/d39/d82 3324664 1 2024-03-22T06:15:31.629 INFO:tasks.workunit.client.1.smithi117.stdout:5/946: dwrite dd/df/d5e/d91/d72/d60/f6b [0,4194304] 0 2024-03-22T06:15:31.629 INFO:tasks.workunit.client.1.smithi117.stdout:5/947: chown dd/df/d5e/d91/d72/d35/da2/ce4 1655 1 2024-03-22T06:15:31.635 INFO:tasks.workunit.client.1.smithi117.stdout:7/817: mknod d8/d9/da/d10/cfc 0 2024-03-22T06:15:31.641 INFO:tasks.workunit.client.1.smithi117.stdout:6/933: rename d3/db4/d2d/d53/d59/d74/f1b to d3/db4/d2d/d53/f144 0 2024-03-22T06:15:31.642 INFO:tasks.workunit.client.1.smithi117.stdout:4/835: stat d0/d48/da0/dd2/db6/dde/d108/d6f/d10e/caa 0 2024-03-22T06:15:31.646 INFO:tasks.workunit.client.1.smithi117.stdout:8/856: creat d1/d28/d6b/d91/f114 x:0 0 0 2024-03-22T06:15:31.652 INFO:tasks.workunit.client.1.smithi117.stdout:3/873: dwrite d9/d1a/d3a/f7a [0,4194304] 0 2024-03-22T06:15:31.652 INFO:tasks.workunit.client.1.smithi117.stdout:3/874: write d9/d55/f57 [681109,2763] 0 2024-03-22T06:15:31.652 INFO:tasks.workunit.client.1.smithi117.stdout:3/875: readlink d9/d1a/d39/d4c/d37/l52 0 2024-03-22T06:15:31.653 INFO:tasks.workunit.client.1.smithi117.stdout:5/948: creat dd/df/d5e/d91/d72/d60/d6e/f144 x:0 0 0 2024-03-22T06:15:31.656 INFO:tasks.workunit.client.1.smithi117.stdout:7/818: link d8/d9/d37/d45/c8c d8/d6c/cfd 0 2024-03-22T06:15:31.656 INFO:tasks.workunit.client.1.smithi117.stdout:7/819: fdatasync d8/d9/da/f4b 0 2024-03-22T06:15:31.656 INFO:tasks.workunit.client.1.smithi117.stdout:7/820: write d8/d9/ff2 [448566,78157] 0 2024-03-22T06:15:31.658 INFO:tasks.workunit.client.1.smithi117.stdout:4/836: dwrite d0/d4/d2a/f30 [0,4194304] 0 2024-03-22T06:15:31.659 INFO:tasks.workunit.client.1.smithi117.stdout:2/840: getdents d2/d5/db/d2c/d7d 0 2024-03-22T06:15:31.659 INFO:tasks.workunit.client.1.smithi117.stdout:9/911: sync 2024-03-22T06:15:31.660 INFO:tasks.workunit.client.1.smithi117.stdout:1/905: sync 2024-03-22T06:15:31.660 INFO:tasks.workunit.client.1.smithi117.stdout:9/912: chown d4/d7/dbf/d100/dd1/dd6/d9b/dd8/f123 484718 1 2024-03-22T06:15:31.662 INFO:tasks.workunit.client.1.smithi117.stdout:6/934: mknod d3/db4/d2d/d53/d59/d74/d28/d85/c145 0 2024-03-22T06:15:31.672 INFO:tasks.workunit.client.1.smithi117.stdout:2/841: dwrite d2/d5/d13/d57/f90 [0,4194304] 0 2024-03-22T06:15:31.674 INFO:tasks.workunit.client.1.smithi117.stdout:5/949: rename dd/df/d88/cc0 to dd/df/d5e/d91/d90/dca/c145 0 2024-03-22T06:15:31.674 INFO:tasks.workunit.client.1.smithi117.stdout:5/950: read - dd/df/d5e/d91/d72/d33/d9a/fc3 zero size 2024-03-22T06:15:31.677 INFO:tasks.workunit.client.1.smithi117.stdout:7/821: mknod d8/d9/d37/d5f/dce/cfe 0 2024-03-22T06:15:31.677 INFO:tasks.workunit.client.1.smithi117.stdout:7/822: chown d8/d9/da/d10/d23/d27/d56/ccc 2 1 2024-03-22T06:15:31.682 INFO:tasks.workunit.client.1.smithi117.stdout:4/837: creat d0/d19/d28/f11a x:0 0 0 2024-03-22T06:15:31.682 INFO:tasks.workunit.client.1.smithi117.stdout:4/838: fdatasync d0/d19/d1c/f84 0 2024-03-22T06:15:31.683 INFO:tasks.workunit.client.1.smithi117.stdout:1/906: stat d1/d30/d50/c52 0 2024-03-22T06:15:31.685 INFO:tasks.workunit.client.1.smithi117.stdout:9/913: creat d4/d7/f12e x:0 0 0 2024-03-22T06:15:31.689 INFO:tasks.workunit.client.1.smithi117.stdout:6/935: symlink d3/d40/d46/d84/d49/d79/d9d/l146 0 2024-03-22T06:15:31.691 INFO:tasks.workunit.client.1.smithi117.stdout:8/857: rmdir d1/d3a/d44/d6a/dfa 39 2024-03-22T06:15:31.691 INFO:tasks.workunit.client.1.smithi117.stdout:8/858: chown d1/d3a/d44/fa 846246483 1 2024-03-22T06:15:31.694 INFO:tasks.workunit.client.1.smithi117.stdout:3/876: dwrite d9/d1a/d39/d7d/f98 [0,4194304] 0 2024-03-22T06:15:31.695 INFO:tasks.workunit.client.1.smithi117.stdout:2/842: symlink d2/d5/db/d2c/d7d/dff/d39/l120 0 2024-03-22T06:15:31.695 INFO:tasks.workunit.client.1.smithi117.stdout:2/843: chown d2/d5/d13/d57/dd0/d1d/d3a/l5a 1028926711 1 2024-03-22T06:15:31.697 INFO:tasks.workunit.client.1.smithi117.stdout:4/839: mkdir d0/d48/da0/dd2/db6/dde/d108/d6f/d11b 0 2024-03-22T06:15:31.698 INFO:tasks.workunit.client.1.smithi117.stdout:1/907: rename d1/d84/c37 to d1/d30/d44/c141 0 2024-03-22T06:15:31.699 INFO:tasks.workunit.client.1.smithi117.stdout:9/914: truncate d4/d7/d27/d5b/fb7 2363148 0 2024-03-22T06:15:31.699 INFO:tasks.workunit.client.1.smithi117.stdout:9/915: dread - d4/d8/d3e/d7a/d3b/fec zero size 2024-03-22T06:15:31.701 INFO:tasks.workunit.client.1.smithi117.stdout:6/936: symlink d3/d3d/d17/d31/l147 0 2024-03-22T06:15:31.701 INFO:tasks.workunit.client.1.smithi117.stdout:6/937: stat d3/db4/d2d/d53/d59/d74/d28 0 2024-03-22T06:15:31.701 INFO:tasks.workunit.client.1.smithi117.stdout:6/938: stat d3/db4/dd4/df4/d101/fca 0 2024-03-22T06:15:31.701 INFO:tasks.workunit.client.1.smithi117.stdout:6/939: stat d3/db4/dd4/df4/d101/da2/cb9 0 2024-03-22T06:15:31.701 INFO:tasks.workunit.client.1.smithi117.stdout:8/859: truncate d1/d28/d66/f74 1869771 0 2024-03-22T06:15:31.703 INFO:tasks.workunit.client.1.smithi117.stdout:4/840: rmdir d0/d19/d34 39 2024-03-22T06:15:31.703 INFO:tasks.workunit.client.1.smithi117.stdout:4/841: chown d0/d48 70 1 2024-03-22T06:15:31.704 INFO:tasks.workunit.client.1.smithi117.stdout:3/877: write d9/d1a/d39/d7d/f9b [155660,86192] 0 2024-03-22T06:15:31.704 INFO:tasks.workunit.client.1.smithi117.stdout:4/842: chown d0/d48/da0/dd2/db6/fcf 1751 1 2024-03-22T06:15:31.705 INFO:tasks.workunit.client.1.smithi117.stdout:6/940: symlink d3/db4/d2d/d53/d59/d74/d28/l148 0 2024-03-22T06:15:31.711 INFO:tasks.workunit.client.1.smithi117.stdout:8/860: mkdir d1/d3a/d44/d11/d97/d115 0 2024-03-22T06:15:31.712 INFO:tasks.workunit.client.1.smithi117.stdout:5/951: dwrite dd/df/d5e/d91/d72/d35/f27 [0,4194304] 0 2024-03-22T06:15:31.712 INFO:tasks.workunit.client.1.smithi117.stdout:7/823: dwrite d8/d6c/fc2 [8388608,4194304] 0 2024-03-22T06:15:31.713 INFO:tasks.workunit.client.1.smithi117.stdout:9/916: mknod d4/d7/dbf/d100/dbc/c12f 0 2024-03-22T06:15:31.714 INFO:tasks.workunit.client.1.smithi117.stdout:6/941: creat d3/db4/dd4/df4/d101/f149 x:0 0 0 2024-03-22T06:15:31.717 INFO:tasks.workunit.client.1.smithi117.stdout:4/843: write d0/dc0/fda [6160640,55257] 0 2024-03-22T06:15:31.718 INFO:tasks.workunit.client.1.smithi117.stdout:8/861: rename d1/d3a/d44/d9/lf2 to d1/d28/d104/l116 0 2024-03-22T06:15:31.722 INFO:tasks.workunit.client.1.smithi117.stdout:6/942: dread d3/db4/d2d/d53/d75/fb0 [0,4194304] 0 2024-03-22T06:15:31.725 INFO:tasks.workunit.client.1.smithi117.stdout:3/878: dwrite d9/d1a/d39/d4c/d37/d69/d101/f106 [0,4194304] 0 2024-03-22T06:15:31.726 INFO:tasks.workunit.client.1.smithi117.stdout:5/952: unlink dd/df/d5e/l125 0 2024-03-22T06:15:31.735 INFO:tasks.workunit.client.1.smithi117.stdout:9/917: rename d4/d7/dbf/d100/dd1/dd6/d9b/dd8/f123 to d4/d8/d3e/d7a/d3b/f130 0 2024-03-22T06:15:31.736 INFO:tasks.workunit.client.1.smithi117.stdout:9/918: chown d4/d8/d3f/c107 1050 1 2024-03-22T06:15:31.738 INFO:tasks.workunit.client.1.smithi117.stdout:8/862: creat d1/d28/d66/d109/f117 x:0 0 0 2024-03-22T06:15:31.743 INFO:tasks.workunit.client.1.smithi117.stdout:4/844: dwrite d0/d19/d34/f37 [0,4194304] 0 2024-03-22T06:15:31.751 INFO:tasks.workunit.client.1.smithi117.stdout:9/919: mknod d4/d8/d3e/d7a/d2f/c131 0 2024-03-22T06:15:31.751 INFO:tasks.workunit.client.1.smithi117.stdout:8/863: mknod d1/d3a/d44/d4b/df7/c118 0 2024-03-22T06:15:31.754 INFO:tasks.workunit.client.1.smithi117.stdout:6/943: dwrite d3/d61/d73/fee [0,4194304] 0 2024-03-22T06:15:31.754 INFO:tasks.workunit.client.1.smithi117.stdout:9/920: creat d4/d47/d64/d72/f132 x:0 0 0 2024-03-22T06:15:31.755 INFO:tasks.workunit.client.1.smithi117.stdout:8/864: symlink d1/d3a/d44/l119 0 2024-03-22T06:15:31.756 INFO:tasks.workunit.client.1.smithi117.stdout:4/845: getdents d0/d48/da0/dd2 0 2024-03-22T06:15:31.756 INFO:tasks.workunit.client.1.smithi117.stdout:4/846: creat d0/d19/d1c/f11c x:0 0 0 2024-03-22T06:15:31.759 INFO:tasks.workunit.client.1.smithi117.stdout:6/944: mknod d3/db4/d2d/d53/d75/d126/c14a 0 2024-03-22T06:15:31.759 INFO:tasks.workunit.client.1.smithi117.stdout:5/953: dwrite dd/df/d95/fab [0,4194304] 0 2024-03-22T06:15:31.761 INFO:tasks.workunit.client.1.smithi117.stdout:9/921: creat d4/d47/d64/f133 x:0 0 0 2024-03-22T06:15:31.761 INFO:tasks.workunit.client.1.smithi117.stdout:9/922: write d4/d8/d3e/d7a/d2f/fb0 [4819343,91028] 0 2024-03-22T06:15:31.764 INFO:tasks.workunit.client.1.smithi117.stdout:8/865: creat d1/d19/d38/f11a x:0 0 0 2024-03-22T06:15:31.765 INFO:tasks.workunit.client.1.smithi117.stdout:0/889: sync 2024-03-22T06:15:31.765 INFO:tasks.workunit.client.1.smithi117.stdout:0/890: chown d1/d10/d5c/d40/dbb/dbf 114007 1 2024-03-22T06:15:31.766 INFO:tasks.workunit.client.1.smithi117.stdout:5/954: dread dd/d16/d1a/f3e [0,4194304] 0 2024-03-22T06:15:31.766 INFO:tasks.workunit.client.1.smithi117.stdout:2/844: dwrite d2/d5/db/d2c/d7d/dff/d39/d45/d88/f11a [0,4194304] 0 2024-03-22T06:15:31.767 INFO:tasks.workunit.client.1.smithi117.stdout:3/879: dwrite d9/f2a [0,4194304] 0 2024-03-22T06:15:31.768 INFO:tasks.workunit.client.1.smithi117.stdout:9/923: rename d4/d47/f10f to d4/d7/dbf/d100/dd1/dd9/f134 0 2024-03-22T06:15:31.768 INFO:tasks.workunit.client.1.smithi117.stdout:9/924: stat d4/d8/f25 0 2024-03-22T06:15:31.768 INFO:tasks.workunit.client.1.smithi117.stdout:9/925: readlink d4/d47/d85/l11d 0 2024-03-22T06:15:31.769 INFO:tasks.workunit.client.1.smithi117.stdout:9/926: dread - d4/d47/d85/dab/fb3 zero size 2024-03-22T06:15:31.769 INFO:tasks.workunit.client.1.smithi117.stdout:9/927: creat d4/d8/d3e/d7a/d2f/d101/f135 x:0 0 0 2024-03-22T06:15:31.769 INFO:tasks.workunit.client.1.smithi117.stdout:9/928: chown d4/d8/d3e/d82 298221027 1 2024-03-22T06:15:31.769 INFO:tasks.workunit.client.1.smithi117.stdout:9/929: dread - d4/d8/d3e/d7a/fd0 zero size 2024-03-22T06:15:31.769 INFO:tasks.workunit.client.1.smithi117.stdout:9/930: fdatasync d4/d46/da0/f11a 0 2024-03-22T06:15:31.772 INFO:tasks.workunit.client.1.smithi117.stdout:7/824: truncate d8/d6c/fc2 8411963 0 2024-03-22T06:15:31.773 INFO:tasks.workunit.client.1.smithi117.stdout:6/945: dread d3/d40/f95 [0,4194304] 0 2024-03-22T06:15:31.774 INFO:tasks.workunit.client.1.smithi117.stdout:6/946: readlink d3/db4/d2d/d53/d59/d74/l21 0 2024-03-22T06:15:31.774 INFO:tasks.workunit.client.1.smithi117.stdout:2/845: creat d2/d5/d13/d57/dd0/d1d/f121 x:0 0 0 2024-03-22T06:15:31.776 INFO:tasks.workunit.client.1.smithi117.stdout:8/866: dread d1/d28/d66/d7f/d9d/ff1 [0,4194304] 0 2024-03-22T06:15:31.777 INFO:tasks.workunit.client.1.smithi117.stdout:4/847: dwrite d0/d48/da0/dd2/db6/dde/d108/d6f/f91 [0,4194304] 0 2024-03-22T06:15:31.778 INFO:tasks.workunit.client.1.smithi117.stdout:0/891: dread d1/d3/dae/ff3 [0,4194304] 0 2024-03-22T06:15:31.778 INFO:tasks.workunit.client.1.smithi117.stdout:4/848: creat d0/d19/d1c/f11d x:0 0 0 2024-03-22T06:15:31.778 INFO:tasks.workunit.client.1.smithi117.stdout:3/880: creat d9/d1a/d39/d4c/d37/d69/d101/d124/f126 x:0 0 0 2024-03-22T06:15:31.779 INFO:tasks.workunit.client.1.smithi117.stdout:5/955: dread dd/d16/f22 [0,4194304] 0 2024-03-22T06:15:31.780 INFO:tasks.workunit.client.1.smithi117.stdout:9/931: dread d4/d8/d3e/f8f [0,4194304] 0 2024-03-22T06:15:31.780 INFO:tasks.workunit.client.1.smithi117.stdout:9/932: readlink d4/d8/d3f/l129 0 2024-03-22T06:15:31.780 INFO:tasks.workunit.client.1.smithi117.stdout:9/933: chown d4/d50/dee/d10b 221841374 1 2024-03-22T06:15:31.782 INFO:tasks.workunit.client.1.smithi117.stdout:7/825: read d8/d9/da/d2b/dc4/f4a [375987,97009] 0 2024-03-22T06:15:31.784 INFO:tasks.workunit.client.1.smithi117.stdout:2/846: symlink d2/d5/db/d2c/d7d/l122 0 2024-03-22T06:15:31.784 INFO:tasks.workunit.client.1.smithi117.stdout:2/847: readlink d2/d5/db/d2c/d7d/dff/d39/d45/l60 0 2024-03-22T06:15:31.788 INFO:tasks.workunit.client.1.smithi117.stdout:8/867: truncate d1/d3a/d44/fa 384171 0 2024-03-22T06:15:31.789 INFO:tasks.workunit.client.1.smithi117.stdout:0/892: dread d1/d3/d2b/d3e/de/d1f/d59/f123 [0,4194304] 0 2024-03-22T06:15:31.791 INFO:tasks.workunit.client.1.smithi117.stdout:3/881: read d9/d1a/d39/d7d/f98 [151077,29683] 0 2024-03-22T06:15:31.791 INFO:tasks.workunit.client.1.smithi117.stdout:3/882: chown d9/d1a/d39/dc7/f115 50900 1 2024-03-22T06:15:31.795 INFO:tasks.workunit.client.1.smithi117.stdout:6/947: dwrite d3/db4/d2d/d53/d59/d74/d28/f124 [0,4194304] 0 2024-03-22T06:15:31.795 INFO:tasks.workunit.client.1.smithi117.stdout:6/948: chown d3/db4/d2d/d53/ce6 673227 1 2024-03-22T06:15:31.795 INFO:tasks.workunit.client.1.smithi117.stdout:6/949: fsync d3/db4/d2d/d53/fef 0 2024-03-22T06:15:31.795 INFO:tasks.workunit.client.1.smithi117.stdout:6/950: readlink d3/db4/l54 0 2024-03-22T06:15:31.805 INFO:tasks.workunit.client.1.smithi117.stdout:8/868: dwrite d1/d3a/d44/d16/f7a [0,4194304] 0 2024-03-22T06:15:31.805 INFO:tasks.workunit.client.1.smithi117.stdout:8/869: fdatasync f0 0 2024-03-22T06:15:31.806 INFO:tasks.workunit.client.1.smithi117.stdout:4/849: creat d0/d48/da0/dd2/db6/dde/d108/d6f/d11b/f11e x:0 0 0 2024-03-22T06:15:31.806 INFO:tasks.workunit.client.1.smithi117.stdout:4/850: readlink d0/d19/d28/d47/dd6/lfa 0 2024-03-22T06:15:31.806 INFO:tasks.workunit.client.1.smithi117.stdout:4/851: dread - d0/d48/da0/dd2/db6/dde/d108/d6f/d5f/f9a zero size 2024-03-22T06:15:31.809 INFO:tasks.workunit.client.1.smithi117.stdout:5/956: mknod dd/df/d5e/d91/d90/dca/c146 0 2024-03-22T06:15:31.811 INFO:tasks.workunit.client.1.smithi117.stdout:9/934: creat d4/d8/d93/f136 x:0 0 0 2024-03-22T06:15:31.811 INFO:tasks.workunit.client.1.smithi117.stdout:9/935: creat d4/d8/d3e/d7a/da9/f137 x:0 0 0 2024-03-22T06:15:31.815 INFO:tasks.workunit.client.1.smithi117.stdout:2/848: rename d2/d5/db/d2c/d7d/dff/d39/d45/cbc to d2/c123 0 2024-03-22T06:15:31.817 INFO:tasks.workunit.client.1.smithi117.stdout:5/957: dread dd/df/d5e/d91/d36/f39 [0,4194304] 0 2024-03-22T06:15:31.819 INFO:tasks.workunit.client.1.smithi117.stdout:3/883: link d9/faa d9/d1a/d39/d4c/d113/f127 0 2024-03-22T06:15:31.824 INFO:tasks.workunit.client.1.smithi117.stdout:6/951: link d3/f3c d3/db4/dd4/df4/f14b 0 2024-03-22T06:15:31.825 INFO:tasks.workunit.client.1.smithi117.stdout:8/870: symlink d1/l11b 0 2024-03-22T06:15:31.825 INFO:tasks.workunit.client.1.smithi117.stdout:8/871: fsync d1/d19/f64 0 2024-03-22T06:15:31.826 INFO:tasks.workunit.client.1.smithi117.stdout:1/908: sync 2024-03-22T06:15:31.826 INFO:tasks.workunit.client.1.smithi117.stdout:1/909: readlink d1/d22/d106/la6 0 2024-03-22T06:15:31.826 INFO:tasks.workunit.client.1.smithi117.stdout:1/910: chown d1/d84/le2 0 1 2024-03-22T06:15:31.827 INFO:tasks.workunit.client.1.smithi117.stdout:4/852: dread - d0/d4/d2a/d5b/f97 zero size 2024-03-22T06:15:31.831 INFO:tasks.workunit.client.1.smithi117.stdout:9/936: mkdir d4/d47/d64/d138 0 2024-03-22T06:15:31.831 INFO:tasks.workunit.client.1.smithi117.stdout:9/937: dread - d4/d7/dbf/d100/dd1/dd6/fc0 zero size 2024-03-22T06:15:31.833 INFO:tasks.workunit.client.1.smithi117.stdout:7/826: getdents d8/d9/da/d2b/dc4/dac/ddc 0 2024-03-22T06:15:31.835 INFO:tasks.workunit.client.1.smithi117.stdout:0/893: rename d1/f3d to d1/d10/d18/db0/f136 0 2024-03-22T06:15:31.835 INFO:tasks.workunit.client.1.smithi117.stdout:0/894: truncate d1/d3/dae/dbe/d117/f130 363199 0 2024-03-22T06:15:31.836 INFO:tasks.workunit.client.1.smithi117.stdout:2/849: stat d2/d5/d13/d57/dd0/l79 0 2024-03-22T06:15:31.837 INFO:tasks.workunit.client.1.smithi117.stdout:7/827: dread d8/d9/f7c [0,4194304] 0 2024-03-22T06:15:31.838 INFO:tasks.workunit.client.1.smithi117.stdout:5/958: link dd/df/d5e/d91/d72/d35/f6f dd/df/d5e/d91/d36/d70/f147 0 2024-03-22T06:15:31.838 INFO:tasks.workunit.client.1.smithi117.stdout:5/959: dread - dd/df/dae/f12e zero size 2024-03-22T06:15:31.841 INFO:tasks.workunit.client.1.smithi117.stdout:3/884: dwrite d9/d1a/d39/d4c/d37/d69/d101/f106 [0,4194304] 0 2024-03-22T06:15:31.843 INFO:tasks.workunit.client.1.smithi117.stdout:2/850: dread d2/d5/d13/d57/f90 [0,4194304] 0 2024-03-22T06:15:31.844 INFO:tasks.workunit.client.1.smithi117.stdout:6/952: mkdir d3/db4/d2d/d53/d75/d142/d14c 0 2024-03-22T06:15:31.846 INFO:tasks.workunit.client.1.smithi117.stdout:4/853: mknod d0/d19/d34/c11f 0 2024-03-22T06:15:31.846 INFO:tasks.workunit.client.1.smithi117.stdout:4/854: chown d0/d4/d2a/d5b 106482204 1 2024-03-22T06:15:31.849 INFO:tasks.workunit.client.1.smithi117.stdout:9/938: truncate d4/d8/d3f/ff1 2167076 0 2024-03-22T06:15:31.853 INFO:tasks.workunit.client.1.smithi117.stdout:8/872: rename d1/d3a/d44/d4b/lb6 to d1/d28/d6b/l11c 0 2024-03-22T06:15:31.853 INFO:tasks.workunit.client.1.smithi117.stdout:8/873: write d1/d19/d98/fe6 [971017,110493] 0 2024-03-22T06:15:31.853 INFO:tasks.workunit.client.1.smithi117.stdout:8/874: dread - d1/d3a/d44/d6a/db7/fd6 zero size 2024-03-22T06:15:31.854 INFO:tasks.workunit.client.1.smithi117.stdout:0/895: mknod d1/d10/d5c/d40/d69/d72/c137 0 2024-03-22T06:15:31.857 INFO:tasks.workunit.client.1.smithi117.stdout:7/828: truncate d8/d9/da/d2b/d9b/fa8 1754130 0 2024-03-22T06:15:31.860 INFO:tasks.workunit.client.1.smithi117.stdout:3/885: dwrite d9/d1a/d3a/f53 [0,4194304] 0 2024-03-22T06:15:31.861 INFO:tasks.workunit.client.1.smithi117.stdout:5/960: rmdir dd/df/d5e/d91/d90/dca 39 2024-03-22T06:15:31.863 INFO:tasks.workunit.client.1.smithi117.stdout:6/953: link d3/d40/d46/d84/d49/d79/d9d/lac d3/d3d/d17/d31/df5/l14d 0 2024-03-22T06:15:31.868 INFO:tasks.workunit.client.1.smithi117.stdout:4/855: creat d0/d48/da0/dd2/db6/dde/d108/d6f/d10e/f120 x:0 0 0 2024-03-22T06:15:31.870 INFO:tasks.workunit.client.1.smithi117.stdout:8/875: mknod d1/d3a/d44/c11d 0 2024-03-22T06:15:31.872 INFO:tasks.workunit.client.1.smithi117.stdout:0/896: symlink d1/d3/d2b/d3e/d3f/d8b/l138 0 2024-03-22T06:15:31.872 INFO:tasks.workunit.client.1.smithi117.stdout:0/897: chown d1/d10/d18/d26/l44 178537343 1 2024-03-22T06:15:31.874 INFO:tasks.workunit.client.1.smithi117.stdout:7/829: dwrite d8/d9/da/d2b/dc4/d26/f7f [0,4194304] 0 2024-03-22T06:15:31.874 INFO:tasks.workunit.client.1.smithi117.stdout:7/830: dread - d8/d9/d37/d45/d88/fea zero size 2024-03-22T06:15:31.877 INFO:tasks.workunit.client.1.smithi117.stdout:3/886: mknod d9/d1a/d39/d7d/d97/dc9/ddd/c128 0 2024-03-22T06:15:31.880 INFO:tasks.workunit.client.1.smithi117.stdout:5/961: creat dd/df/d5e/d91/d72/d35/f148 x:0 0 0 2024-03-22T06:15:31.882 INFO:tasks.workunit.client.1.smithi117.stdout:6/954: creat d3/db4/d2d/d53/d11a/d11b/f14e x:0 0 0 2024-03-22T06:15:31.882 INFO:tasks.workunit.client.1.smithi117.stdout:6/955: fsync d3/d3d/d17/d31/f78 0 2024-03-22T06:15:31.883 INFO:tasks.workunit.client.1.smithi117.stdout:4/856: truncate d0/d19/d28/f2c 227692 0 2024-03-22T06:15:31.884 INFO:tasks.workunit.client.1.smithi117.stdout:8/876: symlink d1/d28/l11e 0 2024-03-22T06:15:31.886 INFO:tasks.workunit.client.1.smithi117.stdout:0/898: creat d1/d10/d18/d26/d49/d7e/d39/dc5/f139 x:0 0 0 2024-03-22T06:15:31.888 INFO:tasks.workunit.client.1.smithi117.stdout:7/831: symlink d8/d6c/df7/lff 0 2024-03-22T06:15:31.889 INFO:tasks.workunit.client.1.smithi117.stdout:3/887: getdents d9/d1a/d39/d4c/d104 0 2024-03-22T06:15:31.889 INFO:tasks.workunit.client.1.smithi117.stdout:3/888: truncate d9/d93/fb7 99512 0 2024-03-22T06:15:31.895 INFO:tasks.workunit.client.1.smithi117.stdout:3/889: dread d9/d1a/f90 [0,4194304] 0 2024-03-22T06:15:31.896 INFO:tasks.workunit.client.1.smithi117.stdout:5/962: link dd/df/d5e/d91/d90/dca/lea dd/df/d5e/d91/deb/l149 0 2024-03-22T06:15:31.900 INFO:tasks.workunit.client.1.smithi117.stdout:8/877: dwrite d1/d3a/d44/d6a/db7/ffe [0,4194304] 0 2024-03-22T06:15:31.904 INFO:tasks.workunit.client.1.smithi117.stdout:3/890: mkdir d9/d1a/d39/d4c/d113/def/d129 0 2024-03-22T06:15:31.904 INFO:tasks.workunit.client.1.smithi117.stdout:3/891: write d9/d1a/d39/d7d/f98 [1328728,53609] 0 2024-03-22T06:15:31.904 INFO:tasks.workunit.client.1.smithi117.stdout:3/892: truncate d9/d1a/faf 1675943 0 2024-03-22T06:15:31.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.905+0000 7f18a777b700 1 -- 172.21.15.6:0/3772418066 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f18a011e370 con 0x7f18a0104fc0 2024-03-22T06:15:31.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.906+0000 7f1895ffb700 1 -- 172.21.15.6:0/3772418066 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1805 (secure 0 0 0) 0x7f189c015070 con 0x7f18a0104fc0 2024-03-22T06:15:31.906 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:15:31.906 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:15:31.906 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:15:31.906 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:15:31.907 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:15:31.908 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:15:31.909 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:15:31.909 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:15:31.909 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:15:31.909 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:15:31.909 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:15:31.909 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:15:31.909 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:15:31.909 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:15:31.909 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:15:31.909 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:15:31.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.909+0000 7f18a777b700 1 -- 172.21.15.6:0/3772418066 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f188c06bb20 msgr2=0x7f188c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:31.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.909+0000 7f18a777b700 1 --2- 172.21.15.6:0/3772418066 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f188c06bb20 0x7f188c06dfd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1890009fd0 tx=0x7f1890009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:31.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.909+0000 7f18a777b700 1 -- 172.21.15.6:0/3772418066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a0104fc0 msgr2=0x7f18a01182c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:31.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.909+0000 7f18a777b700 1 --2- 172.21.15.6:0/3772418066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a0104fc0 0x7f18a01182c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f189c0094a0 tx=0x7f189c009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:31.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.909+0000 7f18a777b700 1 -- 172.21.15.6:0/3772418066 shutdown_connections 2024-03-22T06:15:31.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.909+0000 7f18a777b700 1 --2- 172.21.15.6:0/3772418066 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f188c06bb20 0x7f188c06dfd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:31.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.909+0000 7f18a777b700 1 --2- 172.21.15.6:0/3772418066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a0104fc0 0x7f18a01182c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:31.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.909+0000 7f18a777b700 1 --2- 172.21.15.6:0/3772418066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a01078e0 0x7f18a0118800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:31.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.909+0000 7f18a777b700 1 -- 172.21.15.6:0/3772418066 >> 172.21.15.6:0/3772418066 conn(0x7f18a006a380 msgr2=0x7f18a0107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:31.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.909+0000 7f18a777b700 1 -- 172.21.15.6:0/3772418066 shutdown_connections 2024-03-22T06:15:31.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:31.909+0000 7f18a777b700 1 -- 172.21.15.6:0/3772418066 wait complete. 2024-03-22T06:15:31.911 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:15:31.925 INFO:tasks.workunit.client.1.smithi117.stdout:6/956: rename d3/db4/d2d/d53/d59/d74/da/cb to d3/c14f 0 2024-03-22T06:15:31.925 INFO:tasks.workunit.client.1.smithi117.stdout:6/957: dread - d3/db4/d2d/d53/d59/d74/ff3 zero size 2024-03-22T06:15:31.925 INFO:tasks.workunit.client.1.smithi117.stdout:6/958: truncate d3/db4/d2d/d53/fd6 1812457 0 2024-03-22T06:15:31.925 INFO:tasks.workunit.client.1.smithi117.stdout:6/959: dread - d3/db4/d2d/d53/d59/d74/ff3 zero size 2024-03-22T06:15:31.929 INFO:tasks.workunit.client.1.smithi117.stdout:8/878: truncate d1/d3a/d44/d4b/dc9/fda 1750980 0 2024-03-22T06:15:31.931 INFO:tasks.workunit.client.1.smithi117.stdout:6/960: write d3/db4/f44 [1626839,54532] 0 2024-03-22T06:15:31.936 INFO:tasks.workunit.client.1.smithi117.stdout:0/899: write d1/d10/d5c/d40/d69/d72/f76 [1120822,116731] 0 2024-03-22T06:15:31.937 INFO:tasks.workunit.client.1.smithi117.stdout:7/832: link d8/d9/da/d2b/dc4/c29 d8/d9/d37/d45/d88/db1/dc6/c100 0 2024-03-22T06:15:31.948 INFO:tasks.workunit.client.1.smithi117.stdout:6/961: dwrite d3/db4/d2d/ffd [0,4194304] 0 2024-03-22T06:15:31.952 INFO:tasks.workunit.client.1.smithi117.stdout:1/911: sync 2024-03-22T06:15:31.952 INFO:tasks.workunit.client.1.smithi117.stdout:1/912: write d1/d30/fb4 [955186,36376] 0 2024-03-22T06:15:31.952 INFO:tasks.workunit.client.1.smithi117.stdout:1/913: write d1/d30/d44/d99/ff0 [779022,114084] 0 2024-03-22T06:15:31.953 INFO:tasks.workunit.client.1.smithi117.stdout:8/879: mknod d1/d3a/d44/d4b/c11f 0 2024-03-22T06:15:31.958 INFO:tasks.workunit.client.1.smithi117.stdout:6/962: mknod d3/d40/d46/d84/d49/da8/c150 0 2024-03-22T06:15:31.958 INFO:tasks.workunit.client.1.smithi117.stdout:6/963: chown d3/d61/cc3 332206932 1 2024-03-22T06:15:31.959 INFO:tasks.workunit.client.1.smithi117.stdout:4/857: rename d0/d19/d1c to d0/d48/da0/dd2/d121 0 2024-03-22T06:15:31.961 INFO:tasks.workunit.client.1.smithi117.stdout:6/964: unlink d3/db4/f4e 0 2024-03-22T06:15:31.961 INFO:tasks.workunit.client.1.smithi117.stdout:6/965: dread - d3/db4/dd4/df4/d101/da2/feb zero size 2024-03-22T06:15:31.965 INFO:tasks.workunit.client.1.smithi117.stdout:4/858: unlink d0/d19/dcc/fd4 0 2024-03-22T06:15:31.967 INFO:tasks.workunit.client.1.smithi117.stdout:6/966: mkdir d3/db4/d2d/d53/d59/d74/d28/ddb/d138/dcc/d151 0 2024-03-22T06:15:31.967 INFO:tasks.workunit.client.1.smithi117.stdout:6/967: dread - d3/d3d/d17/d31/d135/f100 zero size 2024-03-22T06:15:31.969 INFO:tasks.workunit.client.1.smithi117.stdout:4/859: creat d0/d48/da0/f122 x:0 0 0 2024-03-22T06:15:31.969 INFO:tasks.workunit.client.1.smithi117.stdout:4/860: dread - d0/d19/d9b/f106 zero size 2024-03-22T06:15:31.969 INFO:tasks.workunit.client.1.smithi117.stdout:4/861: chown d0/d4/f12 14059868 1 2024-03-22T06:15:31.969 INFO:tasks.workunit.client.1.smithi117.stdout:4/862: fsync d0/f6d 0 2024-03-22T06:15:31.969 INFO:tasks.workunit.client.1.smithi117.stdout:4/863: chown d0/dc0 19 1 2024-03-22T06:15:31.969 INFO:tasks.workunit.client.1.smithi117.stdout:7/833: dwrite d8/d9/da/d10/d23/d27/d56/fa4 [0,4194304] 0 2024-03-22T06:15:31.970 INFO:tasks.workunit.client.1.smithi117.stdout:6/968: mkdir d3/db4/dd4/d152 0 2024-03-22T06:15:31.971 INFO:tasks.workunit.client.1.smithi117.stdout:4/864: creat d0/d48/da0/dd2/db6/dde/f123 x:0 0 0 2024-03-22T06:15:31.971 INFO:tasks.workunit.client.1.smithi117.stdout:4/865: truncate d0/dc0/fea 737502 0 2024-03-22T06:15:31.974 INFO:tasks.workunit.client.1.smithi117.stdout:6/969: rename d3/db4/d2d/d53/d59/d74/d28/d85/c63 to d3/d40/d46/d84/d49/c153 0 2024-03-22T06:15:31.974 INFO:tasks.workunit.client.1.smithi117.stdout:6/970: readlink d3/db4/d2d/d53/d59/d74/da/de/l134 0 2024-03-22T06:15:31.976 INFO:tasks.workunit.client.1.smithi117.stdout:0/900: dwrite d1/d10/d18/f7b [0,4194304] 0 2024-03-22T06:15:31.976 INFO:tasks.workunit.client.1.smithi117.stdout:0/901: chown d1/d3/d2b/d3e/de/d1f/d3b 161 1 2024-03-22T06:15:31.978 INFO:tasks.workunit.client.1.smithi117.stdout:6/971: unlink d3/db4/d2d/d53/le1 0 2024-03-22T06:15:31.978 INFO:tasks.workunit.client.1.smithi117.stdout:1/914: dwrite d1/d1f/f3e [0,4194304] 0 2024-03-22T06:15:31.982 INFO:tasks.workunit.client.1.smithi117.stdout:6/972: dread d3/d3d/d17/d31/f87 [0,4194304] 0 2024-03-22T06:15:31.982 INFO:tasks.workunit.client.1.smithi117.stdout:6/973: truncate d3/d40/d46/f117 528982 0 2024-03-22T06:15:31.982 INFO:tasks.workunit.client.1.smithi117.stdout:6/974: chown d3/db4/d2d/d53/d11a/d11b/f14e 1937282796 1 2024-03-22T06:15:31.982 INFO:tasks.workunit.client.1.smithi117.stdout:6/975: creat d3/d40/d46/d84/f154 x:0 0 0 2024-03-22T06:15:31.982 INFO:tasks.workunit.client.1.smithi117.stdout:0/902: mknod d1/d10/d18/d26/d49/d7e/d39/dc5/de8/c13a 0 2024-03-22T06:15:31.987 INFO:tasks.workunit.client.1.smithi117.stdout:1/915: unlink d1/d1f/f3e 0 2024-03-22T06:15:31.991 INFO:tasks.workunit.client.1.smithi117.stdout:8/880: dwrite d1/d28/f31 [0,4194304] 0 2024-03-22T06:15:31.991 INFO:tasks.workunit.client.1.smithi117.stdout:6/976: mkdir d3/d61/d73/d107/d155 0 2024-03-22T06:15:31.991 INFO:tasks.workunit.client.1.smithi117.stdout:6/977: fsync d3/d3d/d17/d31/d135/f100 0 2024-03-22T06:15:31.993 INFO:tasks.workunit.client.1.smithi117.stdout:0/903: mknod d1/d10/d5c/c13b 0 2024-03-22T06:15:31.997 INFO:tasks.workunit.client.1.smithi117.stdout:1/916: creat d1/d30/d44/d83/f142 x:0 0 0 2024-03-22T06:15:31.999 INFO:tasks.workunit.client.1.smithi117.stdout:4/866: dwrite d0/d48/da0/dd2/db6/fcf [0,4194304] 0 2024-03-22T06:15:31.999 INFO:tasks.workunit.client.1.smithi117.stdout:7/834: dwrite d8/d9/da/d10/d23/d48/dc8/fe8 [4194304,4194304] 0 2024-03-22T06:15:31.999 INFO:tasks.workunit.client.1.smithi117.stdout:4/867: dread - d0/d19/d28/f11a zero size 2024-03-22T06:15:31.999 INFO:tasks.workunit.client.1.smithi117.stdout:4/868: chown d0/d19/d9b/cff 936 1 2024-03-22T06:15:31.999 INFO:tasks.workunit.client.1.smithi117.stdout:7/835: write d8/d50/fec [774195,85003] 0 2024-03-22T06:15:31.999 INFO:tasks.workunit.client.1.smithi117.stdout:7/836: fsync d8/f16 0 2024-03-22T06:15:32.002 INFO:tasks.workunit.client.1.smithi117.stdout:4/869: dread d0/d19/d9b/fa8 [0,4194304] 0 2024-03-22T06:15:32.004 INFO:tasks.workunit.client.1.smithi117.stdout:6/978: write d3/db4/d2d/d53/d59/d74/da/fbd [2570931,126016] 0 2024-03-22T06:15:32.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:31 smithi117 ceph-mon[36514]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:15:32.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:31 smithi117 ceph-mon[36514]: pgmap v29: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 290 MiB/s rd, 331 MiB/s wr, 532 op/s 2024-03-22T06:15:32.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:31 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2441314913' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:15:32.007 INFO:tasks.workunit.client.1.smithi117.stdout:8/881: rmdir d1/d3a 39 2024-03-22T06:15:32.007 INFO:tasks.workunit.client.1.smithi117.stdout:8/882: chown d1/d19/d98/fbc 2231933 1 2024-03-22T06:15:32.007 INFO:tasks.workunit.client.1.smithi117.stdout:8/883: write d1/d19/d38/f70 [1067639,89257] 0 2024-03-22T06:15:32.008 INFO:tasks.workunit.client.1.smithi117.stdout:1/917: getdents d1/d3c/d4d/dfe 0 2024-03-22T06:15:32.012 INFO:tasks.workunit.client.1.smithi117.stdout:4/870: symlink d0/d48/da0/l124 0 2024-03-22T06:15:32.012 INFO:tasks.workunit.client.1.smithi117.stdout:6/979: read d3/db4/d2d/d53/d75/fc1 [2597470,109224] 0 2024-03-22T06:15:32.014 INFO:tasks.workunit.client.1.smithi117.stdout:8/884: rename d1/d28/d6b/fa5 to d1/d28/d6b/d91/f120 0 2024-03-22T06:15:32.014 INFO:tasks.workunit.client.1.smithi117.stdout:8/885: creat d1/d28/d6b/da3/f121 x:0 0 0 2024-03-22T06:15:32.016 INFO:tasks.workunit.client.1.smithi117.stdout:1/918: creat d1/d22/d27/d114/f143 x:0 0 0 2024-03-22T06:15:32.016 INFO:tasks.workunit.client.1.smithi117.stdout:1/919: readlink d1/d1f/d32/d33/l51 0 2024-03-22T06:15:32.019 INFO:tasks.workunit.client.1.smithi117.stdout:4/871: dread d0/d19/d28/f2c [0,4194304] 0 2024-03-22T06:15:32.020 INFO:tasks.workunit.client.1.smithi117.stdout:6/980: rename d3/db4/d2d/d53/d59/d74/d28/d85/c145 to d3/db4/d2d/d53/d59/d74/d28/ddb/dde/c156 0 2024-03-22T06:15:32.020 INFO:tasks.workunit.client.1.smithi117.stdout:6/981: stat d3/db4/c103 0 2024-03-22T06:15:32.020 INFO:tasks.workunit.client.1.smithi117.stdout:6/982: write d3/db4/dd4/df4/d101/f111 [5102800,76462] 0 2024-03-22T06:15:32.020 INFO:tasks.workunit.client.1.smithi117.stdout:6/983: dread - d3/d3d/d17/f9a zero size 2024-03-22T06:15:32.024 INFO:tasks.workunit.client.1.smithi117.stdout:7/837: dwrite d8/d9/d37/d5f/fda [0,4194304] 0 2024-03-22T06:15:32.024 INFO:tasks.workunit.client.1.smithi117.stdout:8/886: mkdir d1/d19/d38/d122 0 2024-03-22T06:15:32.024 INFO:tasks.workunit.client.1.smithi117.stdout:8/887: fsync d1/d28/d66/fee 0 2024-03-22T06:15:32.024 INFO:tasks.workunit.client.1.smithi117.stdout:8/888: stat d1/d28/d6b/da3/f100 0 2024-03-22T06:15:32.025 INFO:tasks.workunit.client.1.smithi117.stdout:4/872: creat d0/dd1/f125 x:0 0 0 2024-03-22T06:15:32.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:31 smithi006 ceph-mon[29301]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:15:32.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:31 smithi006 ceph-mon[29301]: pgmap v29: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 290 MiB/s rd, 331 MiB/s wr, 532 op/s 2024-03-22T06:15:32.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:31 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2441314913' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:15:32.032 INFO:tasks.workunit.client.1.smithi117.stdout:7/838: creat d8/d9/d37/d5f/dce/f101 x:0 0 0 2024-03-22T06:15:32.035 INFO:tasks.workunit.client.1.smithi117.stdout:8/889: symlink d1/d3a/d44/d6a/dfa/l123 0 2024-03-22T06:15:32.036 INFO:tasks.workunit.client.1.smithi117.stdout:8/890: truncate d1/d19/fb9 600040 0 2024-03-22T06:15:32.036 INFO:tasks.workunit.client.1.smithi117.stdout:7/839: mkdir d8/d9/d37/d45/d102 0 2024-03-22T06:15:32.038 INFO:tasks.workunit.client.1.smithi117.stdout:8/891: link d1/d28/d66/fee d1/f124 0 2024-03-22T06:15:32.038 INFO:tasks.workunit.client.1.smithi117.stdout:4/873: read d0/d48/da0/dd2/db6/dde/ff0 [3484931,130175] 0 2024-03-22T06:15:32.038 INFO:tasks.workunit.client.1.smithi117.stdout:7/840: symlink d8/d9/da/d2b/l103 0 2024-03-22T06:15:32.039 INFO:tasks.workunit.client.1.smithi117.stdout:8/892: truncate d1/d28/d66/fe4 2986278 0 2024-03-22T06:15:32.039 INFO:tasks.workunit.client.1.smithi117.stdout:7/841: read d8/d9/da/d2b/dc4/f5d [20077,130261] 0 2024-03-22T06:15:32.041 INFO:tasks.workunit.client.1.smithi117.stdout:4/874: mknod d0/d4/d115/c126 0 2024-03-22T06:15:32.041 INFO:tasks.workunit.client.1.smithi117.stdout:8/893: write d1/d3a/d44/d9/f6c [508679,80777] 0 2024-03-22T06:15:32.042 INFO:tasks.workunit.client.1.smithi117.stdout:7/842: mkdir d8/d9/da/d2b/db8/d104 0 2024-03-22T06:15:32.042 INFO:tasks.workunit.client.1.smithi117.stdout:7/843: fdatasync d8/d9/da/d10/d23/d27/d56/fa4 0 2024-03-22T06:15:32.046 INFO:tasks.workunit.client.1.smithi117.stdout:8/894: symlink d1/d3a/d44/l125 0 2024-03-22T06:15:32.046 INFO:tasks.workunit.client.1.smithi117.stdout:2/851: sync 2024-03-22T06:15:32.046 INFO:tasks.workunit.client.1.smithi117.stdout:9/939: sync 2024-03-22T06:15:32.046 INFO:tasks.workunit.client.1.smithi117.stdout:2/852: truncate d2/d5/db/d2c/d7d/f108 347589 0 2024-03-22T06:15:32.047 INFO:tasks.workunit.client.1.smithi117.stdout:1/920: dwrite d1/d14/f19 [0,4194304] 0 2024-03-22T06:15:32.047 INFO:tasks.workunit.client.1.smithi117.stdout:1/921: stat d1/d84/d77/c13a 0 2024-03-22T06:15:32.049 INFO:tasks.workunit.client.1.smithi117.stdout:6/984: dwrite d3/db4/d2d/d53/d59/d74/d28/ddb/d138/dcc/ff6 [4194304,4194304] 0 2024-03-22T06:15:32.051 INFO:tasks.workunit.client.1.smithi117.stdout:5/963: sync 2024-03-22T06:15:32.051 INFO:tasks.workunit.client.1.smithi117.stdout:3/893: sync 2024-03-22T06:15:32.051 INFO:tasks.workunit.client.1.smithi117.stdout:3/894: creat d9/d1a/d39/d7d/d97/f12a x:0 0 0 2024-03-22T06:15:32.052 INFO:tasks.workunit.client.1.smithi117.stdout:8/895: symlink d1/d3a/d44/d6a/dfa/l126 0 2024-03-22T06:15:32.052 INFO:tasks.workunit.client.1.smithi117.stdout:9/940: mknod d4/d7/d27/de7/c139 0 2024-03-22T06:15:32.053 INFO:tasks.workunit.client.1.smithi117.stdout:4/875: read d0/d3f/f95 [1037172,19077] 0 2024-03-22T06:15:32.056 INFO:tasks.workunit.client.1.smithi117.stdout:3/895: dread d9/d55/f7b [0,4194304] 0 2024-03-22T06:15:32.056 INFO:tasks.workunit.client.1.smithi117.stdout:1/922: symlink d1/d1f/l144 0 2024-03-22T06:15:32.056 INFO:tasks.workunit.client.1.smithi117.stdout:7/844: dread f7 [0,4194304] 0 2024-03-22T06:15:32.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.058+0000 7fde6e96f700 1 -- 172.21.15.6:0/3763049999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde68066ef0 msgr2=0x7fde680ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:32.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.058+0000 7fde6e96f700 1 --2- 172.21.15.6:0/3763049999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde68066ef0 0x7fde680ffa70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fde58004940 tx=0x7fde58009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:32.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.058+0000 7fde6e96f700 1 -- 172.21.15.6:0/3763049999 shutdown_connections 2024-03-22T06:15:32.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.058+0000 7fde6e96f700 1 --2- 172.21.15.6:0/3763049999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde680fffb0 0x7fde68102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:32.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.058+0000 7fde6e96f700 1 --2- 172.21.15.6:0/3763049999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde68066ef0 0x7fde680ffa70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:32.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.058+0000 7fde6e96f700 1 -- 172.21.15.6:0/3763049999 >> 172.21.15.6:0/3763049999 conn(0x7fde680f7440 msgr2=0x7fde680f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:32.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.058+0000 7fde6e96f700 1 -- 172.21.15.6:0/3763049999 shutdown_connections 2024-03-22T06:15:32.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.058+0000 7fde6e96f700 1 -- 172.21.15.6:0/3763049999 wait complete. 2024-03-22T06:15:32.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.059+0000 7fde6e96f700 1 Processor -- start 2024-03-22T06:15:32.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.059+0000 7fde6e96f700 1 -- start start 2024-03-22T06:15:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.059+0000 7fde6e96f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde68066ef0 0x7fde680fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.059+0000 7fde6e96f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde680fffb0 0x7fde680fb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.059+0000 7fde6e96f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde680fd900 con 0x7fde680fffb0 2024-03-22T06:15:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.059+0000 7fde6e96f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde680fbee0 con 0x7fde68066ef0 2024-03-22T06:15:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.060+0000 7fde677fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde680fffb0 0x7fde680fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.060+0000 7fde677fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde680fffb0 0x7fde680fb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47156/0 (socket says 172.21.15.6:47156) 2024-03-22T06:15:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.060+0000 7fde677fe700 1 -- 172.21.15.6:0/3047717008 learned_addr learned my addr 172.21.15.6:0/3047717008 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:15:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.060+0000 7fde677fe700 1 -- 172.21.15.6:0/3047717008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde68066ef0 msgr2=0x7fde680fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.060+0000 7fde677fe700 1 --2- 172.21.15.6:0/3047717008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde68066ef0 0x7fde680fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.060+0000 7fde677fe700 1 -- 172.21.15.6:0/3047717008 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde58005040 con 0x7fde680fffb0 2024-03-22T06:15:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.060+0000 7fde677fe700 1 --2- 172.21.15.6:0/3047717008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde680fffb0 0x7fde680fb970 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00b790 tx=0x7fde5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:32.062 INFO:tasks.workunit.client.1.smithi117.stdout:5/964: write dd/df/d5e/d91/d72/d60/fb4 [1003852,107024] 0 2024-03-22T06:15:32.062 INFO:tasks.workunit.client.1.smithi117.stdout:5/965: creat dd/d16/d1a/f14a x:0 0 0 2024-03-22T06:15:32.063 INFO:tasks.workunit.client.1.smithi117.stdout:5/966: stat dd/d16/d1a/d32/dfc/dff 0 2024-03-22T06:15:32.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.060+0000 7fde64ff9700 1 -- 172.21.15.6:0/3047717008 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde5c00f770 con 0x7fde680fffb0 2024-03-22T06:15:32.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.061+0000 7fde6e96f700 1 -- 172.21.15.6:0/3047717008 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde680fc1c0 con 0x7fde680fffb0 2024-03-22T06:15:32.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.061+0000 7fde6e96f700 1 -- 172.21.15.6:0/3047717008 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde680fc780 con 0x7fde680fffb0 2024-03-22T06:15:32.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.061+0000 7fde64ff9700 1 -- 172.21.15.6:0/3047717008 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde5c00fda0 con 0x7fde680fffb0 2024-03-22T06:15:32.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.062+0000 7fde64ff9700 1 -- 172.21.15.6:0/3047717008 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde5c00d310 con 0x7fde680fffb0 2024-03-22T06:15:32.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.063+0000 7fde6e96f700 1 -- 172.21.15.6:0/3047717008 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde540052e0 con 0x7fde680fffb0 2024-03-22T06:15:32.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.064+0000 7fde64ff9700 1 -- 172.21.15.6:0/3047717008 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7fde5c00f8d0 con 0x7fde680fffb0 2024-03-22T06:15:32.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.065+0000 7fde64ff9700 1 --2- 172.21.15.6:0/3047717008 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fde5006b910 0x7fde5006ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:32.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.065+0000 7fde64ff9700 1 -- 172.21.15.6:0/3047717008 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fde5c08ada0 con 0x7fde680fffb0 2024-03-22T06:15:32.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.066+0000 7fde67fff700 1 --2- 172.21.15.6:0/3047717008 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fde5006b910 0x7fde5006ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:32.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.067+0000 7fde67fff700 1 --2- 172.21.15.6:0/3047717008 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fde5006b910 0x7fde5006ddc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fde58004f90 tx=0x7fde58009e30 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:32.068 INFO:tasks.workunit.client.1.smithi117.stdout:9/941: mkdir d4/d8/d3e/d7a/d2f/d13a 0 2024-03-22T06:15:32.069 INFO:tasks.workunit.client.1.smithi117.stdout:4/876: mkdir d0/d19/d34/d43/d127 0 2024-03-22T06:15:32.072 INFO:tasks.workunit.client.1.smithi117.stdout:3/896: mkdir d9/d66/d12b 0 2024-03-22T06:15:32.072 INFO:tasks.workunit.client.1.smithi117.stdout:3/897: readlink d9/d1a/d39/d4c/d37/lff 0 2024-03-22T06:15:32.072 INFO:tasks.workunit.client.1.smithi117.stdout:3/898: fdatasync d9/d1a/d3a/f53 0 2024-03-22T06:15:32.072 INFO:tasks.workunit.client.1.smithi117.stdout:3/899: fsync d9/d1a/d39/d4c/d36/d41/f65 0 2024-03-22T06:15:32.072 INFO:tasks.workunit.client.1.smithi117.stdout:3/900: fsync d9/d1a/fa7 0 2024-03-22T06:15:32.073 INFO:tasks.workunit.client.1.smithi117.stdout:3/901: write d9/d55/f11a [1029958,89592] 0 2024-03-22T06:15:32.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.072+0000 7fde64ff9700 1 -- 172.21.15.6:0/3047717008 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7fde5c058280 con 0x7fde680fffb0 2024-03-22T06:15:32.073 INFO:tasks.workunit.client.1.smithi117.stdout:1/923: mkdir d1/d3c/d4d/de8/d145 0 2024-03-22T06:15:32.076 INFO:tasks.workunit.client.1.smithi117.stdout:7/845: creat d8/d9/da/d2b/f105 x:0 0 0 2024-03-22T06:15:32.076 INFO:tasks.workunit.client.1.smithi117.stdout:7/846: chown d8/d9/da/d2b/dc4/d5c 467 1 2024-03-22T06:15:32.076 INFO:tasks.workunit.client.1.smithi117.stdout:7/847: chown d8/d9/d37/d45 481 1 2024-03-22T06:15:32.076 INFO:tasks.workunit.client.1.smithi117.stdout:7/848: chown d8/d50/fec 78377108 1 2024-03-22T06:15:32.080 INFO:tasks.workunit.client.1.smithi117.stdout:4/877: dread d0/d19/d34/d43/f62 [0,4194304] 0 2024-03-22T06:15:32.080 INFO:tasks.workunit.client.1.smithi117.stdout:9/942: mkdir d4/d13b 0 2024-03-22T06:15:32.082 INFO:tasks.workunit.client.1.smithi117.stdout:9/943: symlink d4/d7/dbf/d100/dd1/dd6/d9b/dd8/d113/l13c 0 2024-03-22T06:15:32.082 INFO:tasks.workunit.client.1.smithi117.stdout:6/985: dwrite d3/db4/fd2 [0,4194304] 0 2024-03-22T06:15:32.082 INFO:tasks.workunit.client.1.smithi117.stdout:6/986: stat d3/db4/d2d/d53/d59/d74/d28/ddb/dde/f119 0 2024-03-22T06:15:32.083 INFO:tasks.workunit.client.1.smithi117.stdout:9/944: mknod d4/d46/d8c/c13d 0 2024-03-22T06:15:32.083 INFO:tasks.workunit.client.1.smithi117.stdout:9/945: dread - d4/d8/d3e/d7a/d2f/d101/f135 zero size 2024-03-22T06:15:32.086 INFO:tasks.workunit.client.1.smithi117.stdout:6/987: mkdir d3/d3d/d17/d31/df5/dfa/d157 0 2024-03-22T06:15:32.086 INFO:tasks.workunit.client.1.smithi117.stdout:6/988: write d3/db4/d2d/d53/d75/f8e [1125248,62913] 0 2024-03-22T06:15:32.088 INFO:tasks.workunit.client.1.smithi117.stdout:6/989: mkdir d3/db4/d2d/d53/d75/d142/d14c/d158 0 2024-03-22T06:15:32.090 INFO:tasks.workunit.client.1.smithi117.stdout:8/896: dwrite d1/d3a/d44/d11/d97/f99 [0,4194304] 0 2024-03-22T06:15:32.090 INFO:tasks.workunit.client.1.smithi117.stdout:6/990: symlink d3/d40/d46/d84/d49/l159 0 2024-03-22T06:15:32.094 INFO:tasks.workunit.client.1.smithi117.stdout:8/897: rename d1/d28/c54 to d1/d3a/d44/d6a/db7/c127 0 2024-03-22T06:15:32.094 INFO:tasks.workunit.client.1.smithi117.stdout:8/898: creat d1/d3a/d44/d11/dc7/f128 x:0 0 0 2024-03-22T06:15:32.094 INFO:tasks.workunit.client.1.smithi117.stdout:6/991: stat d3/d3d/d17/c121 0 2024-03-22T06:15:32.095 INFO:tasks.workunit.client.1.smithi117.stdout:9/946: dread d4/d8/d3f/fd7 [0,4194304] 0 2024-03-22T06:15:32.095 INFO:tasks.workunit.client.1.smithi117.stdout:9/947: chown d4/d7/d27/c71 0 1 2024-03-22T06:15:32.096 INFO:tasks.workunit.client.1.smithi117.stdout:5/967: dwrite dd/df/d5e/d91/d72/d35/f148 [0,4194304] 0 2024-03-22T06:15:32.098 INFO:tasks.workunit.client.1.smithi117.stdout:9/948: rename d4/d47/d85/dab/lf7 to d4/d8/d3f/l13e 0 2024-03-22T06:15:32.102 INFO:tasks.workunit.client.1.smithi117.stdout:3/902: dwrite d9/d1a/dc1/d102/f120 [0,4194304] 0 2024-03-22T06:15:32.102 INFO:tasks.workunit.client.1.smithi117.stdout:1/924: dwrite d1/d1f/f7f [0,4194304] 0 2024-03-22T06:15:32.102 INFO:tasks.workunit.client.1.smithi117.stdout:8/899: write d1/d3a/d44/d4b/dc9/fda [329766,1619] 0 2024-03-22T06:15:32.106 INFO:tasks.workunit.client.1.smithi117.stdout:9/949: dread d4/d8/d3f/fd7 [0,4194304] 0 2024-03-22T06:15:32.106 INFO:tasks.workunit.client.1.smithi117.stdout:6/992: dread d3/db4/d2d/d53/d59/d74/d28/f124 [0,4194304] 0 2024-03-22T06:15:32.106 INFO:tasks.workunit.client.1.smithi117.stdout:6/993: write d3/db4/dd4/df4/d101/f149 [633455,71574] 0 2024-03-22T06:15:32.109 INFO:tasks.workunit.client.1.smithi117.stdout:7/849: dwrite d8/d9/d37/d5f/fda [0,4194304] 0 2024-03-22T06:15:32.111 INFO:tasks.workunit.client.1.smithi117.stdout:5/968: link dd/df/d5e/d91/d72/f45 dd/df/d88/f14b 0 2024-03-22T06:15:32.116 INFO:tasks.workunit.client.1.smithi117.stdout:3/903: creat d9/d1a/d39/d82/d10a/f12c x:0 0 0 2024-03-22T06:15:32.120 INFO:tasks.workunit.client.1.smithi117.stdout:1/925: creat d1/d30/d44/dd3/d13f/f146 x:0 0 0 2024-03-22T06:15:32.122 INFO:tasks.workunit.client.1.smithi117.stdout:8/900: mkdir d1/d3a/d44/d6a/d129 0 2024-03-22T06:15:32.126 INFO:tasks.workunit.client.1.smithi117.stdout:6/994: dwrite d3/db4/d2d/d53/d59/d74/d28/d85/f69 [4194304,4194304] 0 2024-03-22T06:15:32.132 INFO:tasks.workunit.client.1.smithi117.stdout:1/926: write d1/d1f/d32/d33/da4/f109 [5794361,119927] 0 2024-03-22T06:15:32.132 INFO:tasks.workunit.client.1.smithi117.stdout:1/927: write d1/fd [4067530,59429] 0 2024-03-22T06:15:32.133 INFO:tasks.workunit.client.1.smithi117.stdout:7/850: creat d8/d6c/f106 x:0 0 0 2024-03-22T06:15:32.137 INFO:tasks.workunit.client.1.smithi117.stdout:5/969: rename dd/df/f38 to dd/d16/d1a/f14c 0 2024-03-22T06:15:32.139 INFO:tasks.workunit.client.1.smithi117.stdout:8/901: symlink d1/d3a/d44/d16/l12a 0 2024-03-22T06:15:32.141 INFO:tasks.workunit.client.1.smithi117.stdout:9/950: rmdir d4/d46 39 2024-03-22T06:15:32.143 INFO:tasks.workunit.client.1.smithi117.stdout:6/995: rmdir d3/d40/d46/d84/d49/d79/d9d 39 2024-03-22T06:15:32.148 INFO:tasks.workunit.client.1.smithi117.stdout:5/970: write dd/df/d5e/d91/d72/d35/f27 [415871,62581] 0 2024-03-22T06:15:32.148 INFO:tasks.workunit.client.1.smithi117.stdout:5/971: write dd/d16/d1a/d32/f8b [107362,46073] 0 2024-03-22T06:15:32.148 INFO:tasks.workunit.client.1.smithi117.stdout:5/972: dread - dd/d16/d106/f123 zero size 2024-03-22T06:15:32.149 INFO:tasks.workunit.client.1.smithi117.stdout:3/904: rename d9/fa6 to d9/d1a/dc1/dd0/f12d 0 2024-03-22T06:15:32.150 INFO:tasks.workunit.client.1.smithi117.stdout:8/902: link d1/d3a/d44/d11/fff d1/d3a/d44/d4b/df7/f12b 0 2024-03-22T06:15:32.153 INFO:tasks.workunit.client.1.smithi117.stdout:6/996: mknod d3/db4/d2d/d53/d59/d74/da/db3/c15a 0 2024-03-22T06:15:32.161 INFO:tasks.workunit.client.1.smithi117.stdout:6/997: dread d3/d40/d46/d84/d49/d79/f7d [4194304,4194304] 0 2024-03-22T06:15:32.161 INFO:tasks.workunit.client.1.smithi117.stdout:6/998: write d3/db4/dd4/df4/f14b [597803,18149] 0 2024-03-22T06:15:32.161 INFO:tasks.workunit.client.1.smithi117.stdout:6/999: chown d3/c14f 13252653 1 2024-03-22T06:15:32.166 INFO:tasks.workunit.client.1.smithi117.stdout:1/928: rmdir d1/d30/d44 39 2024-03-22T06:15:32.168 INFO:tasks.workunit.client.1.smithi117.stdout:5/973: symlink dd/d16/d1a/d32/dfc/dff/l14d 0 2024-03-22T06:15:32.169 INFO:tasks.workunit.client.1.smithi117.stdout:0/904: sync 2024-03-22T06:15:32.171 INFO:tasks.workunit.client.1.smithi117.stdout:7/851: rename d8/c6d to d8/d9/da/d10/d23/d41/c107 0 2024-03-22T06:15:32.174 INFO:tasks.workunit.client.1.smithi117.stdout:3/905: dwrite d9/f17 [0,4194304] 0 2024-03-22T06:15:32.174 INFO:tasks.workunit.client.1.smithi117.stdout:3/906: dread - d9/d66/fbd zero size 2024-03-22T06:15:32.177 INFO:tasks.workunit.client.1.smithi117.stdout:8/903: mkdir d1/d19/d98/d12c 0 2024-03-22T06:15:32.178 INFO:tasks.workunit.client.1.smithi117.stdout:2/853: sync 2024-03-22T06:15:32.180 INFO:tasks.workunit.client.1.smithi117.stdout:1/929: creat d1/d1f/d32/dc6/f147 x:0 0 0 2024-03-22T06:15:32.180 INFO:tasks.workunit.client.1.smithi117.stdout:1/930: dread - d1/d1f/d32/d33/fdf zero size 2024-03-22T06:15:32.180 INFO:tasks.workunit.client.1.smithi117.stdout:1/931: write d1/d22/d27/d114/fe0 [977296,25069] 0 2024-03-22T06:15:32.180 INFO:tasks.workunit.client.1.smithi117.stdout:1/932: fsync d1/d22/d27/d4e/f7a 0 2024-03-22T06:15:32.182 INFO:tasks.workunit.client.1.smithi117.stdout:5/974: mknod dd/df/d5e/d91/d72/dde/c14e 0 2024-03-22T06:15:32.184 INFO:tasks.workunit.client.1.smithi117.stdout:8/904: write d1/d28/d66/fe4 [2317561,79625] 0 2024-03-22T06:15:32.185 INFO:tasks.workunit.client.1.smithi117.stdout:8/905: read d1/d3a/d44/d4b/dc9/fe5 [339606,792] 0 2024-03-22T06:15:32.185 INFO:tasks.workunit.client.1.smithi117.stdout:4/878: sync 2024-03-22T06:15:32.187 INFO:tasks.workunit.client.1.smithi117.stdout:0/905: creat d1/d10/d18/d26/d49/d7e/f13c x:0 0 0 2024-03-22T06:15:32.191 INFO:tasks.workunit.client.1.smithi117.stdout:9/951: rename d4/d8/d3f/ff1 to d4/d50/dee/d10b/f13f 0 2024-03-22T06:15:32.197 INFO:tasks.workunit.client.1.smithi117.stdout:3/907: mknod d9/d1a/d39/d8a/da1/db6/d121/c12e 0 2024-03-22T06:15:32.197 INFO:tasks.workunit.client.1.smithi117.stdout:3/908: getdents d9/d1a/d39/d4c/d37/dd5 0 2024-03-22T06:15:32.201 INFO:tasks.workunit.client.1.smithi117.stdout:9/952: write d4/d8/d3e/d7a/d2f/fe0 [471091,38795] 0 2024-03-22T06:15:32.201 INFO:tasks.workunit.client.1.smithi117.stdout:2/854: mkdir d2/d5/db/d2c/d7d/dff/d39/d45/d88/d124 0 2024-03-22T06:15:32.202 INFO:tasks.workunit.client.1.smithi117.stdout:5/975: mkdir dd/df/d5e/d91/d72/dde/d14f 0 2024-03-22T06:15:32.205 INFO:tasks.workunit.client.1.smithi117.stdout:8/906: truncate f0 1642148 0 2024-03-22T06:15:32.206 INFO:tasks.workunit.client.1.smithi117.stdout:4/879: symlink d0/d48/da0/dd2/db6/dde/d108/d6f/d5f/l128 0 2024-03-22T06:15:32.212 INFO:tasks.workunit.client.1.smithi117.stdout:1/933: rename d1/d30/d44/d9c/da9/f103 to d1/d30/d44/d9c/da9/de3/d112/f148 0 2024-03-22T06:15:32.213 INFO:tasks.workunit.client.1.smithi117.stdout:0/906: dwrite d1/d3/d2b/d3e/d3f/f78 [0,4194304] 0 2024-03-22T06:15:32.216 INFO:tasks.workunit.client.1.smithi117.stdout:2/855: read d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/fb0 [410904,71634] 0 2024-03-22T06:15:32.219 INFO:tasks.workunit.client.1.smithi117.stdout:9/953: link d4/d8/d3e/d7a/da9/fc7 d4/d7/f140 0 2024-03-22T06:15:32.221 INFO:tasks.workunit.client.1.smithi117.stdout:5/976: symlink dd/df/d5e/d91/d72/d35/da2/l150 0 2024-03-22T06:15:32.230 INFO:tasks.workunit.client.1.smithi117.stdout:4/880: rename d0/d48/da0/dd2/db6/dde/d108/d6f/d5f/c96 to d0/d48/da0/dd2/db6/dde/d108/d6f/c129 0 2024-03-22T06:15:32.237 INFO:tasks.workunit.client.1.smithi117.stdout:9/954: dwrite d4/d7/d27/f60 [0,4194304] 0 2024-03-22T06:15:32.238 INFO:tasks.workunit.client.1.smithi117.stdout:0/907: truncate d1/d10/d18/d26/d49/d7e/f99 570312 0 2024-03-22T06:15:32.240 INFO:tasks.workunit.client.1.smithi117.stdout:2/856: symlink d2/d5/db/d2c/d7d/dff/d39/d45/d7c/l125 0 2024-03-22T06:15:32.240 INFO:tasks.workunit.client.1.smithi117.stdout:2/857: truncate d2/d5/d13/d57/dd0/d1d/d3a/d89/fab 150772 0 2024-03-22T06:15:32.240 INFO:tasks.workunit.client.1.smithi117.stdout:2/858: creat d2/d5/d13/dde/f126 x:0 0 0 2024-03-22T06:15:32.240 INFO:tasks.workunit.client.1.smithi117.stdout:2/859: write d2/d35/d38/d11f/f10e [189899,24536] 0 2024-03-22T06:15:32.242 INFO:tasks.workunit.client.1.smithi117.stdout:5/977: symlink dd/df/d5e/d91/l151 0 2024-03-22T06:15:32.246 INFO:tasks.workunit.client.1.smithi117.stdout:4/881: creat d0/dc0/f12a x:0 0 0 2024-03-22T06:15:32.250 INFO:tasks.workunit.client.1.smithi117.stdout:4/882: readlink d0/d48/da0/dd2/d105/l116 0 2024-03-22T06:15:32.250 INFO:tasks.workunit.client.1.smithi117.stdout:1/934: getdents d1/d30/d5d/d6f 0 2024-03-22T06:15:32.259 INFO:tasks.workunit.client.1.smithi117.stdout:4/883: mkdir d0/d51/d12b 0 2024-03-22T06:15:32.261 INFO:tasks.workunit.client.1.smithi117.stdout:1/935: getdents d1/d1f/d32/d33 0 2024-03-22T06:15:32.262 INFO:tasks.workunit.client.1.smithi117.stdout:5/978: dread dd/df/d5e/d91/f34 [0,4194304] 0 2024-03-22T06:15:32.263 INFO:tasks.workunit.client.1.smithi117.stdout:4/884: truncate d0/d4/d2a/f30 929825 0 2024-03-22T06:15:32.267 INFO:tasks.workunit.client.1.smithi117.stdout:1/936: mknod d1/d62/de4/c149 0 2024-03-22T06:15:32.269 INFO:tasks.workunit.client.1.smithi117.stdout:4/885: symlink d0/d48/da0/dd2/db6/dde/l12c 0 2024-03-22T06:15:32.269 INFO:tasks.workunit.client.1.smithi117.stdout:4/886: write d0/d48/da0/dd2/db6/dde/d108/d6f/d5f/fcb [4446409,8438] 0 2024-03-22T06:15:32.271 INFO:tasks.workunit.client.1.smithi117.stdout:1/937: mknod d1/d1f/d32/d12e/c14a 0 2024-03-22T06:15:32.271 INFO:tasks.workunit.client.1.smithi117.stdout:0/908: dwrite d1/d10/d18/d26/f75 [0,4194304] 0 2024-03-22T06:15:32.273 INFO:tasks.workunit.client.1.smithi117.stdout:9/955: dwrite d4/d8/d3e/d7a/f42 [0,4194304] 0 2024-03-22T06:15:32.274 INFO:tasks.workunit.client.1.smithi117.stdout:2/860: dwrite d2/d5/fe2 [0,4194304] 0 2024-03-22T06:15:32.275 INFO:tasks.workunit.client.1.smithi117.stdout:2/861: fdatasync d2/d5/d13/d57/dd0/d1d/d3a/f78 0 2024-03-22T06:15:32.278 INFO:tasks.workunit.client.1.smithi117.stdout:0/909: mkdir d1/d3/d2b/d3e/d3f/d13d 0 2024-03-22T06:15:32.280 INFO:tasks.workunit.client.1.smithi117.stdout:4/887: write d0/d4/d2a/d5b/fa6 [2689391,82648] 0 2024-03-22T06:15:32.280 INFO:tasks.workunit.client.1.smithi117.stdout:4/888: dread - d0/d19/d9b/f106 zero size 2024-03-22T06:15:32.281 INFO:tasks.workunit.client.1.smithi117.stdout:9/956: creat d4/d7/dbf/d100/dd1/dd6/d9b/dd8/d113/d121/f141 x:0 0 0 2024-03-22T06:15:32.281 INFO:tasks.workunit.client.1.smithi117.stdout:9/957: chown d4/d47/d64/fe8 4 1 2024-03-22T06:15:32.285 INFO:tasks.workunit.client.1.smithi117.stdout:5/979: dwrite dd/d16/d86/d93/d10f/f128 [0,4194304] 0 2024-03-22T06:15:32.295 INFO:tasks.workunit.client.1.smithi117.stdout:2/862: truncate d2/d5/d13/d57/dd0/d1d/d3a/d89/f93 75619 0 2024-03-22T06:15:32.297 INFO:tasks.workunit.client.1.smithi117.stdout:0/910: creat d1/d3/d2b/f13e x:0 0 0 2024-03-22T06:15:32.298 INFO:tasks.workunit.client.1.smithi117.stdout:4/889: mkdir d0/d48/da0/dd2/db6/dde/d108/d6f/d5f/d12d 0 2024-03-22T06:15:32.299 INFO:tasks.workunit.client.1.smithi117.stdout:4/890: write d0/f42 [1752260,32073] 0 2024-03-22T06:15:32.300 INFO:tasks.workunit.client.1.smithi117.stdout:9/958: symlink d4/d46/da0/l142 0 2024-03-22T06:15:32.300 INFO:tasks.workunit.client.1.smithi117.stdout:9/959: fsync d4/d46/fa4 0 2024-03-22T06:15:32.300 INFO:tasks.workunit.client.1.smithi117.stdout:9/960: chown d4/d8/f1a 55 1 2024-03-22T06:15:32.301 INFO:tasks.workunit.client.1.smithi117.stdout:4/891: dread d0/d48/da0/dd2/db6/dde/d108/f74 [0,4194304] 0 2024-03-22T06:15:32.302 INFO:tasks.workunit.client.1.smithi117.stdout:4/892: read d0/d19/f49 [369049,31589] 0 2024-03-22T06:15:32.306 INFO:tasks.workunit.client.1.smithi117.stdout:5/980: rename dd/df/d5e/d91/d72/d60/lc2 to dd/df/d5e/d91/d72/d60/dec/l152 0 2024-03-22T06:15:32.306 INFO:tasks.workunit.client.1.smithi117.stdout:5/981: chown dd/cdf 3480662 1 2024-03-22T06:15:32.307 INFO:tasks.workunit.client.1.smithi117.stdout:7/852: sync 2024-03-22T06:15:32.308 INFO:tasks.workunit.client.1.smithi117.stdout:2/863: getdents d2/d5/d13/d57/dd0/d1d/d86 0 2024-03-22T06:15:32.314 INFO:tasks.workunit.client.1.smithi117.stdout:9/961: link d4/d8/d3e/d7a/d2f/d101/c110 d4/d7/dbf/d100/dd1/dd6/c143 0 2024-03-22T06:15:32.315 INFO:tasks.workunit.client.1.smithi117.stdout:0/911: dwrite d1/d3/dae/fdf [0,4194304] 0 2024-03-22T06:15:32.315 INFO:tasks.workunit.client.1.smithi117.stdout:4/893: creat d0/f12e x:0 0 0 2024-03-22T06:15:32.320 INFO:tasks.workunit.client.1.smithi117.stdout:2/864: rename d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/c107 to d2/d9a/c127 0 2024-03-22T06:15:32.320 INFO:tasks.workunit.client.1.smithi117.stdout:5/982: write dd/f7d [3596175,112810] 0 2024-03-22T06:15:32.320 INFO:tasks.workunit.client.1.smithi117.stdout:5/983: stat dd/df/d5e/d91/d36/d70/f105 0 2024-03-22T06:15:32.322 INFO:tasks.workunit.client.1.smithi117.stdout:3/909: sync 2024-03-22T06:15:32.323 INFO:tasks.workunit.client.1.smithi117.stdout:7/853: dwrite d8/fb [4194304,4194304] 0 2024-03-22T06:15:32.323 INFO:tasks.workunit.client.1.smithi117.stdout:7/854: write d8/d6c/fae [2769383,66727] 0 2024-03-22T06:15:32.324 INFO:tasks.workunit.client.1.smithi117.stdout:7/855: creat d8/d9/da/f108 x:0 0 0 2024-03-22T06:15:32.324 INFO:tasks.workunit.client.1.smithi117.stdout:7/856: write d8/d9/da/d2b/dc4/f3b [1194392,26422] 0 2024-03-22T06:15:32.324 INFO:tasks.workunit.client.1.smithi117.stdout:7/857: truncate d8/d50/d5e/fbd 404692 0 2024-03-22T06:15:32.325 INFO:tasks.workunit.client.1.smithi117.stdout:0/912: truncate d1/d3/d2b/d3e/d3f/f45 953885 0 2024-03-22T06:15:32.329 INFO:tasks.workunit.client.1.smithi117.stdout:9/962: rename d4/d8/d3f/dc3/ffe to d4/d47/f144 0 2024-03-22T06:15:32.331 INFO:tasks.workunit.client.1.smithi117.stdout:2/865: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/f128 x:0 0 0 2024-03-22T06:15:32.331 INFO:tasks.workunit.client.1.smithi117.stdout:2/866: fsync d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/dfe/f109 0 2024-03-22T06:15:32.334 INFO:tasks.workunit.client.1.smithi117.stdout:3/910: creat d9/d1a/d39/dc7/f12f x:0 0 0 2024-03-22T06:15:32.341 INFO:tasks.workunit.client.1.smithi117.stdout:9/963: write d4/d8/d3e/d7a/f28 [3661789,12054] 0 2024-03-22T06:15:32.342 INFO:tasks.workunit.client.1.smithi117.stdout:2/867: dread d2/d5/d13/d57/dd0/d1d/d3a/d89/fab [0,4194304] 0 2024-03-22T06:15:32.342 INFO:tasks.workunit.client.1.smithi117.stdout:2/868: truncate d2/d5/db/d2c/f85 1046093 0 2024-03-22T06:15:32.345 INFO:tasks.workunit.client.1.smithi117.stdout:7/858: dwrite d8/d6c/fc2 [4194304,4194304] 0 2024-03-22T06:15:32.345 INFO:tasks.workunit.client.1.smithi117.stdout:7/859: chown d8/d6c/df7/lff 42 1 2024-03-22T06:15:32.346 INFO:tasks.workunit.client.1.smithi117.stdout:0/913: getdents d1/d10/d5c 0 2024-03-22T06:15:32.349 INFO:tasks.workunit.client.1.smithi117.stdout:9/964: symlink d4/d7/dbf/d100/dd1/dd6/d9b/dd8/d113/l145 0 2024-03-22T06:15:32.350 INFO:tasks.workunit.client.1.smithi117.stdout:5/984: dwrite dd/df/d5e/d91/d72/d35/f27 [0,4194304] 0 2024-03-22T06:15:32.351 INFO:tasks.workunit.client.1.smithi117.stdout:2/869: unlink d2/d35/c61 0 2024-03-22T06:15:32.351 INFO:tasks.workunit.client.1.smithi117.stdout:2/870: read - d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/ff9 zero size 2024-03-22T06:15:32.352 INFO:tasks.workunit.client.1.smithi117.stdout:0/914: getdents d1/d10/d18/d26 0 2024-03-22T06:15:32.354 INFO:tasks.workunit.client.1.smithi117.stdout:3/911: dwrite d9/d1a/d39/d4c/d104/f114 [0,4194304] 0 2024-03-22T06:15:32.354 INFO:tasks.workunit.client.1.smithi117.stdout:9/965: rename d4/d8/d3e/d7a/d2f/d101/c110 to d4/d7/d27/d5b/c146 0 2024-03-22T06:15:32.356 INFO:tasks.workunit.client.1.smithi117.stdout:5/985: unlink dd/df/d5e/d91/d72/d60/d6e/c12a 0 2024-03-22T06:15:32.357 INFO:tasks.workunit.client.1.smithi117.stdout:5/986: write dd/df/d5e/d91/d90/fd5 [826334,17366] 0 2024-03-22T06:15:32.358 INFO:tasks.workunit.client.1.smithi117.stdout:5/987: stat dd/df/d5e/d91/d90/dca/c142 0 2024-03-22T06:15:32.358 INFO:tasks.workunit.client.1.smithi117.stdout:5/988: fsync dd/df/d5e/d91/d72/d33/d9a/f101 0 2024-03-22T06:15:32.358 INFO:tasks.workunit.client.1.smithi117.stdout:5/989: chown dd/df/d5e/d91/d72/d35/d81/cf9 1 1 2024-03-22T06:15:32.358 INFO:tasks.workunit.client.1.smithi117.stdout:5/990: truncate dd/d16/d1a/d32/d73/db8/fc5 1391995 0 2024-03-22T06:15:32.358 INFO:tasks.workunit.client.1.smithi117.stdout:2/871: unlink d2/d5/d13/d57/dd0/d1d/d3a/d89/f93 0 2024-03-22T06:15:32.360 INFO:tasks.workunit.client.1.smithi117.stdout:0/915: creat d1/d10/d18/d26/d49/d7e/d39/d6a/f13f x:0 0 0 2024-03-22T06:15:32.360 INFO:tasks.workunit.client.1.smithi117.stdout:7/860: dread d8/d9/d37/d5f/fda [0,4194304] 0 2024-03-22T06:15:32.360 INFO:tasks.workunit.client.1.smithi117.stdout:7/861: dread - d8/d9/da/f108 zero size 2024-03-22T06:15:32.360 INFO:tasks.workunit.client.1.smithi117.stdout:8/907: sync 2024-03-22T06:15:32.360 INFO:tasks.workunit.client.1.smithi117.stdout:1/938: sync 2024-03-22T06:15:32.360 INFO:tasks.workunit.client.1.smithi117.stdout:8/908: fdatasync d1/fac 0 2024-03-22T06:15:32.361 INFO:tasks.workunit.client.1.smithi117.stdout:9/966: mkdir d4/d7/dbf/d100/dd1/dd9/d147 0 2024-03-22T06:15:32.362 INFO:tasks.workunit.client.1.smithi117.stdout:9/967: dread - d4/d47/d64/dc5/dda/d104/f108 zero size 2024-03-22T06:15:32.363 INFO:tasks.workunit.client.1.smithi117.stdout:5/991: stat dd/df/d5e/d91/d72/d35/d81/d104/lb5 0 2024-03-22T06:15:32.363 INFO:tasks.workunit.client.1.smithi117.stdout:5/992: write dd/df/d5e/d91/d72/d35/f6f [3332915,112838] 0 2024-03-22T06:15:32.364 INFO:tasks.workunit.client.1.smithi117.stdout:2/872: rename d2/d5/d13/d57/dd0/c56 to d2/d5/d13/d57/dd0/d1d/d3a/d71/c129 0 2024-03-22T06:15:32.366 INFO:tasks.workunit.client.1.smithi117.stdout:0/916: creat d1/d10/d5c/d40/d113/f140 x:0 0 0 2024-03-22T06:15:32.367 INFO:tasks.workunit.client.1.smithi117.stdout:7/862: creat d8/d50/d5e/f109 x:0 0 0 2024-03-22T06:15:32.370 INFO:tasks.workunit.client.1.smithi117.stdout:1/939: mknod d1/d1f/d32/c14b 0 2024-03-22T06:15:32.370 INFO:tasks.workunit.client.1.smithi117.stdout:1/940: fsync d1/d3c/d4d/de8/f10c 0 2024-03-22T06:15:32.373 INFO:tasks.workunit.client.1.smithi117.stdout:8/909: dwrite d1/d19/f2d [0,4194304] 0 2024-03-22T06:15:32.373 INFO:tasks.workunit.client.1.smithi117.stdout:8/910: fsync d1/d3a/d44/d11/d97/f99 0 2024-03-22T06:15:32.373 INFO:tasks.workunit.client.1.smithi117.stdout:3/912: link d9/f2a d9/d1a/d39/d7d/f130 0 2024-03-22T06:15:32.374 INFO:tasks.workunit.client.1.smithi117.stdout:9/968: creat d4/d7/d27/d5b/dae/f148 x:0 0 0 2024-03-22T06:15:32.374 INFO:tasks.workunit.client.1.smithi117.stdout:9/969: truncate d4/d7/dbf/d100/dd1/dd6/fc0 698068 0 2024-03-22T06:15:32.375 INFO:tasks.workunit.client.1.smithi117.stdout:9/970: fsync d4/d7/f12e 0 2024-03-22T06:15:32.375 INFO:tasks.workunit.client.1.smithi117.stdout:5/993: mknod dd/d16/d86/d93/c153 0 2024-03-22T06:15:32.376 INFO:tasks.workunit.client.1.smithi117.stdout:2/873: mkdir d2/d9a/d12a 0 2024-03-22T06:15:32.376 INFO:tasks.workunit.client.1.smithi117.stdout:2/874: rename d2/d5/db/d2c/d7d/dff to d2/d5/db/d2c/d7d/dff/d39/d45/d88/d124/d12b 22 2024-03-22T06:15:32.377 INFO:tasks.workunit.client.1.smithi117.stdout:0/917: creat d1/d3/d2b/d3e/de/f141 x:0 0 0 2024-03-22T06:15:32.378 INFO:tasks.workunit.client.1.smithi117.stdout:7/863: truncate d8/d9/da/f72 2156411 0 2024-03-22T06:15:32.378 INFO:tasks.workunit.client.1.smithi117.stdout:7/864: fsync d8/d9/da/d2b/dc4/dac/ddc/fe2 0 2024-03-22T06:15:32.379 INFO:tasks.workunit.client.1.smithi117.stdout:1/941: symlink d1/d30/d44/d83/l14c 0 2024-03-22T06:15:32.386 INFO:tasks.workunit.client.1.smithi117.stdout:1/942: write d1/d1f/fb1 [1637371,100727] 0 2024-03-22T06:15:32.387 INFO:tasks.workunit.client.1.smithi117.stdout:1/943: fdatasync d1/d22/d27/d4e/f102 0 2024-03-22T06:15:32.403 INFO:tasks.workunit.client.1.smithi117.stdout:8/911: unlink d1/d3a/d44/d9/la4 0 2024-03-22T06:15:32.406 INFO:tasks.workunit.client.1.smithi117.stdout:3/913: unlink d9/d1a/d39/d4c/d37/d69/fe4 0 2024-03-22T06:15:32.406 INFO:tasks.workunit.client.1.smithi117.stdout:3/914: dread - d9/d1a/d39/d8a/da1/db6/f10e zero size 2024-03-22T06:15:32.406 INFO:tasks.workunit.client.1.smithi117.stdout:3/915: dread - d9/d1a/d39/d4c/d113/fee zero size 2024-03-22T06:15:32.409 INFO:tasks.workunit.client.1.smithi117.stdout:9/971: creat d4/d46/f149 x:0 0 0 2024-03-22T06:15:32.409 INFO:tasks.workunit.client.1.smithi117.stdout:9/972: truncate d4/d7/d27/f60 4584303 0 2024-03-22T06:15:32.413 INFO:tasks.workunit.client.1.smithi117.stdout:5/994: symlink dd/df/d5e/d91/deb/l154 0 2024-03-22T06:15:32.414 INFO:tasks.workunit.client.1.smithi117.stdout:2/875: write d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/faf [996525,111027] 0 2024-03-22T06:15:32.414 INFO:tasks.workunit.client.1.smithi117.stdout:4/894: sync 2024-03-22T06:15:32.417 INFO:tasks.workunit.client.1.smithi117.stdout:0/918: mkdir d1/d3/d2b/d3e/de/d1f/d3b/d142 0 2024-03-22T06:15:32.418 INFO:tasks.workunit.client.1.smithi117.stdout:9/973: write d4/d8/d3e/d7a/d2f/f7b [2068320,69908] 0 2024-03-22T06:15:32.422 INFO:tasks.workunit.client.1.smithi117.stdout:7/865: mknod d8/d50/c10a 0 2024-03-22T06:15:32.437 INFO:tasks.workunit.client.1.smithi117.stdout:3/916: mkdir d9/d1a/dc1/d102/d131 0 2024-03-22T06:15:32.437 INFO:tasks.workunit.client.1.smithi117.stdout:3/917: readlink d9/l8d 0 2024-03-22T06:15:32.437 INFO:tasks.workunit.client.1.smithi117.stdout:3/918: fdatasync d9/d55/f7c 0 2024-03-22T06:15:32.439 INFO:tasks.workunit.client.1.smithi117.stdout:2/876: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/f12c x:0 0 0 2024-03-22T06:15:32.440 INFO:tasks.workunit.client.1.smithi117.stdout:0/919: symlink d1/d10/d18/l143 0 2024-03-22T06:15:32.440 INFO:tasks.workunit.client.1.smithi117.stdout:0/920: write d1/d3/d2b/d3e/de/d1f/d3b/d55/f92 [688068,60995] 0 2024-03-22T06:15:32.441 INFO:tasks.workunit.client.1.smithi117.stdout:5/995: dwrite dd/d16/d1a/d32/f40 [0,4194304] 0 2024-03-22T06:15:32.441 INFO:tasks.workunit.client.1.smithi117.stdout:4/895: dwrite d0/d48/da0/dd2/db6/dde/d108/d6f/d6a/f8c [0,4194304] 0 2024-03-22T06:15:32.441 INFO:tasks.workunit.client.1.smithi117.stdout:9/974: dwrite d4/d8/d3e/d7a/da9/fc7 [4194304,4194304] 0 2024-03-22T06:15:32.447 INFO:tasks.workunit.client.1.smithi117.stdout:3/919: unlink d9/d1a/d39/d4c/d37/d69/ced 0 2024-03-22T06:15:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.449+0000 7fde6e96f700 1 -- 172.21.15.6:0/3047717008 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde54000bf0 con 0x7fde5006b910 2024-03-22T06:15:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.451+0000 7fde64ff9700 1 -- 172.21.15.6:0/3047717008 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fde54000bf0 con 0x7fde5006b910 2024-03-22T06:15:32.452 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:15:32.452 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:15:32.452 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:15:32.452 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:15:32.453 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [], 2024-03-22T06:15:32.453 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "1/21 daemons upgraded", 2024-03-22T06:15:32.453 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "", 2024-03-22T06:15:32.453 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:15:32.453 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:15:32.453 INFO:tasks.workunit.client.1.smithi117.stdout:5/996: write dd/df/d5e/d91/f4f [1456618,67076] 0 2024-03-22T06:15:32.453 INFO:tasks.workunit.client.1.smithi117.stdout:2/877: creat d2/d5/d13/d57/dd0/d1d/d3a/d10f/f12d x:0 0 0 2024-03-22T06:15:32.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.454+0000 7fde6e96f700 1 -- 172.21.15.6:0/3047717008 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fde5006b910 msgr2=0x7fde5006ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:32.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.454+0000 7fde6e96f700 1 --2- 172.21.15.6:0/3047717008 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fde5006b910 0x7fde5006ddc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fde58004f90 tx=0x7fde58009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:32.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.454+0000 7fde6e96f700 1 -- 172.21.15.6:0/3047717008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde680fffb0 msgr2=0x7fde680fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:32.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.454+0000 7fde6e96f700 1 --2- 172.21.15.6:0/3047717008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde680fffb0 0x7fde680fb970 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00b790 tx=0x7fde5c009250 comp rx=0 tx=0).stop 2024-03-22T06:15:32.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.454+0000 7fde6e96f700 1 -- 172.21.15.6:0/3047717008 shutdown_connections 2024-03-22T06:15:32.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.454+0000 7fde6e96f700 1 --2- 172.21.15.6:0/3047717008 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fde5006b910 0x7fde5006ddc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:32.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.454+0000 7fde6e96f700 1 --2- 172.21.15.6:0/3047717008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde68066ef0 0x7fde680fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:32.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.454+0000 7fde6e96f700 1 --2- 172.21.15.6:0/3047717008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde680fffb0 0x7fde680fb970 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:32.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.454+0000 7fde6e96f700 1 -- 172.21.15.6:0/3047717008 >> 172.21.15.6:0/3047717008 conn(0x7fde680f7440 msgr2=0x7fde680f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:32.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.454+0000 7fde6e96f700 1 -- 172.21.15.6:0/3047717008 shutdown_connections 2024-03-22T06:15:32.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.454+0000 7fde6e96f700 1 -- 172.21.15.6:0/3047717008 wait complete. 2024-03-22T06:15:32.456 INFO:tasks.workunit.client.1.smithi117.stdout:0/921: creat d1/d3/d2b/d3e/de/d1f/d3b/f144 x:0 0 0 2024-03-22T06:15:32.456 INFO:tasks.workunit.client.1.smithi117.stdout:0/922: truncate d1/d10/d5c/d40/d4e/dbc/ddb/fea 683333 0 2024-03-22T06:15:32.456 INFO:tasks.workunit.client.1.smithi117.stdout:0/923: read - d1/d10/d5c/d40/dac/ff7 zero size 2024-03-22T06:15:32.461 INFO:tasks.workunit.client.1.smithi117.stdout:5/997: dread dd/df/d5e/d91/f5d [0,4194304] 0 2024-03-22T06:15:32.461 INFO:tasks.workunit.client.1.smithi117.stdout:4/896: dwrite d0/f12e [0,4194304] 0 2024-03-22T06:15:32.464 INFO:tasks.workunit.client.1.smithi117.stdout:5/998: write dd/df/d5e/d91/d36/f39 [2651082,123213] 0 2024-03-22T06:15:32.465 INFO:tasks.workunit.client.1.smithi117.stdout:9/975: truncate d4/d7/d27/d5b/f6a 352682 0 2024-03-22T06:15:32.470 INFO:tasks.workunit.client.1.smithi117.stdout:7/866: rename d8/d9/da/d10/d23/d27/d56/fa4 to d8/d9/da/d2b/dc4/f10b 0 2024-03-22T06:15:32.475 INFO:tasks.workunit.client.1.smithi117.stdout:3/920: rmdir d9/d1a/d39/d7d 39 2024-03-22T06:15:32.478 INFO:tasks.workunit.client.1.smithi117.stdout:8/912: dwrite d1/d3a/d44/fa [0,4194304] 0 2024-03-22T06:15:32.479 INFO:tasks.workunit.client.1.smithi117.stdout:2/878: mknod d2/d9a/d12a/c12e 0 2024-03-22T06:15:32.483 INFO:tasks.workunit.client.1.smithi117.stdout:0/924: symlink d1/d10/d5c/d40/d4e/dbc/ddb/df9/l145 0 2024-03-22T06:15:32.483 INFO:tasks.workunit.client.1.smithi117.stdout:0/925: write d1/d3/d2b/d3e/d3f/f112 [1014267,16309] 0 2024-03-22T06:15:32.498 INFO:tasks.workunit.client.1.smithi117.stdout:8/913: dwrite d1/d28/fa8 [0,4194304] 0 2024-03-22T06:15:32.502 INFO:tasks.workunit.client.1.smithi117.stdout:4/897: rmdir d0/d48/da0/dd2/db6 39 2024-03-22T06:15:32.507 INFO:tasks.workunit.client.1.smithi117.stdout:5/999: mknod dd/df/d5e/d91/d90/dca/c155 0 2024-03-22T06:15:32.511 INFO:tasks.workunit.client.1.smithi117.stdout:9/976: mkdir d4/d47/d64/dc5/d14a 0 2024-03-22T06:15:32.511 INFO:tasks.workunit.client.1.smithi117.stdout:9/977: dread - d4/d8/d3e/d7a/d3b/f130 zero size 2024-03-22T06:15:32.511 INFO:tasks.workunit.client.1.smithi117.stdout:9/978: write d4/d47/d85/dab/fb3 [371021,78025] 0 2024-03-22T06:15:32.511 INFO:tasks.workunit.client.1.smithi117.stdout:9/979: fsync d4/d8/d3f/fd7 0 2024-03-22T06:15:32.511 INFO:tasks.workunit.client.1.smithi117.stdout:9/980: creat d4/d47/d64/dc5/dda/d104/f14b x:0 0 0 2024-03-22T06:15:32.512 INFO:tasks.workunit.client.1.smithi117.stdout:8/914: dwrite d1/d3a/d44/d16/fb5 [0,4194304] 0 2024-03-22T06:15:32.512 INFO:tasks.workunit.client.1.smithi117.stdout:8/915: read d1/d28/d6b/fd0 [225940,3851] 0 2024-03-22T06:15:32.514 INFO:tasks.workunit.client.1.smithi117.stdout:7/867: rename d8/d9/da/d2b/dc4/d5c/la9 to d8/d9/dd1/l10c 0 2024-03-22T06:15:32.515 INFO:tasks.workunit.client.1.smithi117.stdout:4/898: dread d0/d19/d28/f55 [0,4194304] 0 2024-03-22T06:15:32.515 INFO:tasks.workunit.client.1.smithi117.stdout:4/899: chown d0/d48/da0/dd2 1117386 1 2024-03-22T06:15:32.515 INFO:tasks.workunit.client.1.smithi117.stdout:8/916: dread d1/d3a/d44/d4b/dae/fb8 [0,4194304] 0 2024-03-22T06:15:32.515 INFO:tasks.workunit.client.1.smithi117.stdout:8/917: write d1/d3a/d44/f3c [5099929,45904] 0 2024-03-22T06:15:32.515 INFO:tasks.workunit.client.1.smithi117.stdout:8/918: chown d1/d28/d66/d7f/d9d/lf3 12767 1 2024-03-22T06:15:32.515 INFO:tasks.workunit.client.1.smithi117.stdout:3/921: creat d9/d1a/d39/d4c/d113/f132 x:0 0 0 2024-03-22T06:15:32.516 INFO:tasks.workunit.client.1.smithi117.stdout:3/922: chown d9/d1a/d39/d8a/da1/db6/d121/c12e 6 1 2024-03-22T06:15:32.516 INFO:tasks.workunit.client.1.smithi117.stdout:1/944: sync 2024-03-22T06:15:32.522 INFO:tasks.workunit.client.1.smithi117.stdout:0/926: link d1/d10/d18/f38 d1/d10/d5c/d40/d69/f146 0 2024-03-22T06:15:32.524 INFO:tasks.workunit.client.1.smithi117.stdout:1/945: dread d1/d30/fc4 [0,4194304] 0 2024-03-22T06:15:32.524 INFO:tasks.workunit.client.1.smithi117.stdout:1/946: write d1/d3c/f122 [442159,47016] 0 2024-03-22T06:15:32.524 INFO:tasks.workunit.client.1.smithi117.stdout:1/947: dread - d1/d30/d35/fc5 zero size 2024-03-22T06:15:32.536 INFO:tasks.workunit.client.1.smithi117.stdout:8/919: dwrite d1/d3a/d44/d6a/db7/fd6 [0,4194304] 0 2024-03-22T06:15:32.536 INFO:tasks.workunit.client.1.smithi117.stdout:7/868: dwrite d8/d9/f7c [0,4194304] 0 2024-03-22T06:15:32.543 INFO:tasks.workunit.client.1.smithi117.stdout:2/879: rename d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/cfd to d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/c12f 0 2024-03-22T06:15:32.546 INFO:tasks.workunit.client.1.smithi117.stdout:4/900: creat d0/d48/da0/dd2/db6/dde/d108/d8d/f12f x:0 0 0 2024-03-22T06:15:32.547 INFO:tasks.workunit.client.1.smithi117.stdout:3/923: mknod d9/d1a/d39/d4c/d37/c133 0 2024-03-22T06:15:32.548 INFO:tasks.workunit.client.1.smithi117.stdout:4/901: dread d0/fd9 [0,4194304] 0 2024-03-22T06:15:32.549 INFO:tasks.workunit.client.1.smithi117.stdout:4/902: chown d0/d19/d28/cf8 302081 1 2024-03-22T06:15:32.550 INFO:tasks.workunit.client.1.smithi117.stdout:0/927: mknod d1/c147 0 2024-03-22T06:15:32.555 INFO:tasks.workunit.client.1.smithi117.stdout:7/869: symlink d8/d9/d37/d45/l10d 0 2024-03-22T06:15:32.555 INFO:tasks.workunit.client.1.smithi117.stdout:7/870: stat d8/d9/dd1/fa0 0 2024-03-22T06:15:32.557 INFO:tasks.workunit.client.1.smithi117.stdout:9/981: rename d4/d7/dbf/d100/dbc/l10e to d4/d47/l14c 0 2024-03-22T06:15:32.557 INFO:tasks.workunit.client.1.smithi117.stdout:9/982: truncate d4/f59 777646 0 2024-03-22T06:15:32.558 INFO:tasks.workunit.client.1.smithi117.stdout:2/880: mknod d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/c130 0 2024-03-22T06:15:32.559 INFO:tasks.workunit.client.1.smithi117.stdout:0/928: write d1/d3/f12 [4174947,129060] 0 2024-03-22T06:15:32.559 INFO:tasks.workunit.client.1.smithi117.stdout:4/903: mkdir d0/d48/da0/dd2/d130 0 2024-03-22T06:15:32.562 INFO:tasks.workunit.client.1.smithi117.stdout:7/871: truncate d8/d9/da/d2b/dc4/f83 265426 0 2024-03-22T06:15:32.562 INFO:tasks.workunit.client.1.smithi117.stdout:8/920: dwrite d1/d19/f2d [4194304,4194304] 0 2024-03-22T06:15:32.563 INFO:tasks.workunit.client.1.smithi117.stdout:1/948: dwrite d1/d22/d27/d114/f60 [4194304,4194304] 0 2024-03-22T06:15:32.563 INFO:tasks.workunit.client.1.smithi117.stdout:1/949: write d1/d22/d27/d114/f10f [111455,12565] 0 2024-03-22T06:15:32.563 INFO:tasks.workunit.client.1.smithi117.stdout:2/881: rename d2/d5/d13/f30 to d2/d5/db/d2c/d7d/dff/d39/d45/d7c/f131 0 2024-03-22T06:15:32.563 INFO:tasks.workunit.client.1.smithi117.stdout:2/882: dread - d2/d5/d13/d57/dd0/d1d/d3a/f78 zero size 2024-03-22T06:15:32.565 INFO:tasks.workunit.client.1.smithi117.stdout:3/924: dwrite d9/d1a/d39/d4c/d113/fd3 [0,4194304] 0 2024-03-22T06:15:32.565 INFO:tasks.workunit.client.1.smithi117.stdout:0/929: truncate d1/d3/d2b/d3e/d3f/f71 282993 0 2024-03-22T06:15:32.568 INFO:tasks.workunit.client.1.smithi117.stdout:4/904: mknod d0/d19/d28/c131 0 2024-03-22T06:15:32.568 INFO:tasks.workunit.client.1.smithi117.stdout:4/905: write d0/d51/fc8 [884195,99767] 0 2024-03-22T06:15:32.568 INFO:tasks.workunit.client.1.smithi117.stdout:4/906: truncate d0/d19/d28/d47/ffb 1031253 0 2024-03-22T06:15:32.571 INFO:tasks.workunit.client.1.smithi117.stdout:7/872: truncate d8/d9/d37/d45/d88/db1/dc6/de0/ff3 2336025 0 2024-03-22T06:15:32.571 INFO:tasks.workunit.client.1.smithi117.stdout:7/873: chown d8/d6c/fc2 15031462 1 2024-03-22T06:15:32.575 INFO:tasks.workunit.client.1.smithi117.stdout:9/983: dwrite d4/d46/f128 [0,4194304] 0 2024-03-22T06:15:32.578 INFO:tasks.workunit.client.1.smithi117.stdout:1/950: link d1/d30/d5d/d6f/l11f d1/d84/d77/l14d 0 2024-03-22T06:15:32.578 INFO:tasks.workunit.client.1.smithi117.stdout:1/951: fsync d1/d1f/f105 0 2024-03-22T06:15:32.579 INFO:tasks.workunit.client.1.smithi117.stdout:2/883: creat d2/d5/db/f132 x:0 0 0 2024-03-22T06:15:32.581 INFO:tasks.workunit.client.1.smithi117.stdout:8/921: rename d1/d3a/d44/d105 to d1/d12d 0 2024-03-22T06:15:32.585 INFO:tasks.workunit.client.1.smithi117.stdout:3/925: mknod d9/d1a/d39/d7d/d97/dc9/c134 0 2024-03-22T06:15:32.585 INFO:tasks.workunit.client.1.smithi117.stdout:3/926: dread - d9/d1a/d39/d7d/d97/f12a zero size 2024-03-22T06:15:32.586 INFO:tasks.workunit.client.1.smithi117.stdout:3/927: creat d9/d1a/d3a/f135 x:0 0 0 2024-03-22T06:15:32.586 INFO:tasks.workunit.client.1.smithi117.stdout:3/928: creat d9/d1a/d3a/f136 x:0 0 0 2024-03-22T06:15:32.586 INFO:tasks.workunit.client.1.smithi117.stdout:3/929: creat d9/d1a/d39/d8a/da1/db6/f137 x:0 0 0 2024-03-22T06:15:32.586 INFO:tasks.workunit.client.1.smithi117.stdout:3/930: chown d9/d1a/d39/d4c/d36/d41/l76 7289 1 2024-03-22T06:15:32.595 INFO:tasks.workunit.client.1.smithi117.stdout:9/984: stat d4/d7/l62 0 2024-03-22T06:15:32.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.598+0000 7f67aa3ac700 1 -- 172.21.15.6:0/2727439224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a40fcf90 msgr2=0x7f67a40fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:32.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.598+0000 7f67aa3ac700 1 --2- 172.21.15.6:0/2727439224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a40fcf90 0x7f67a40fd400 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6798004940 tx=0x7f6798009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:32.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.598+0000 7f67aa3ac700 1 -- 172.21.15.6:0/2727439224 shutdown_connections 2024-03-22T06:15:32.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.598+0000 7f67aa3ac700 1 --2- 172.21.15.6:0/2727439224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a40fcf90 0x7f67a40fd400 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:32.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.598+0000 7f67aa3ac700 1 --2- 172.21.15.6:0/2727439224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67a40fbce0 0x7f67a40fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:32.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.598+0000 7f67aa3ac700 1 -- 172.21.15.6:0/2727439224 >> 172.21.15.6:0/2727439224 conn(0x7f67a40f7440 msgr2=0x7f67a40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:32.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.598+0000 7f67aa3ac700 1 -- 172.21.15.6:0/2727439224 shutdown_connections 2024-03-22T06:15:32.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.598+0000 7f67aa3ac700 1 -- 172.21.15.6:0/2727439224 wait complete. 2024-03-22T06:15:32.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.599+0000 7f67aa3ac700 1 Processor -- start 2024-03-22T06:15:32.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.599+0000 7f67aa3ac700 1 -- start start 2024-03-22T06:15:32.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.599+0000 7f67aa3ac700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67a40fbce0 0x7f67a410fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:32.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.599+0000 7f67aa3ac700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a40fcf90 0x7f67a41100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:32.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.599+0000 7f67aa3ac700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a41106b0 con 0x7f67a40fcf90 2024-03-22T06:15:32.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.599+0000 7f67aa3ac700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4110f00 con 0x7f67a40fbce0 2024-03-22T06:15:32.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.599+0000 7f67a3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67a40fbce0 0x7f67a410fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:32.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.599+0000 7f67a3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67a40fbce0 0x7f67a410fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:42886/0 (socket says 172.21.15.6:42886) 2024-03-22T06:15:32.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.599+0000 7f67a37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a40fcf90 0x7f67a41100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:32.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.599+0000 7f67a3fff700 1 -- 172.21.15.6:0/180945841 learned_addr learned my addr 172.21.15.6:0/180945841 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:15:32.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.600+0000 7f67a3fff700 1 -- 172.21.15.6:0/180945841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a40fcf90 msgr2=0x7f67a41100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:32.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.600+0000 7f67a3fff700 1 --2- 172.21.15.6:0/180945841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a40fcf90 0x7f67a41100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:32.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.600+0000 7f67a3fff700 1 -- 172.21.15.6:0/180945841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798005040 con 0x7f67a40fbce0 2024-03-22T06:15:32.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.600+0000 7f67a37fe700 1 --2- 172.21.15.6:0/180945841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a40fcf90 0x7f67a41100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:15:32.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.600+0000 7f67a3fff700 1 --2- 172.21.15.6:0/180945841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67a40fbce0 0x7f67a410fb70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f679400eb40 tx=0x7f679400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:32.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.600+0000 7f67a0ff9700 1 -- 172.21.15.6:0/180945841 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6794007cc0 con 0x7f67a40fbce0 2024-03-22T06:15:32.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.600+0000 7f67a0ff9700 1 -- 172.21.15.6:0/180945841 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f679400f040 con 0x7f67a40fbce0 2024-03-22T06:15:32.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.600+0000 7f67aa3ac700 1 -- 172.21.15.6:0/180945841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a4111180 con 0x7f67a40fbce0 2024-03-22T06:15:32.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.600+0000 7f67aa3ac700 1 -- 172.21.15.6:0/180945841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a41116c0 con 0x7f67a40fbce0 2024-03-22T06:15:32.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.600+0000 7f67a0ff9700 1 -- 172.21.15.6:0/180945841 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f679400ca10 con 0x7f67a40fbce0 2024-03-22T06:15:32.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.601+0000 7f67aa3ac700 1 -- 172.21.15.6:0/180945841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a405f5f0 con 0x7f67a40fbce0 2024-03-22T06:15:32.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.602+0000 7f67a0ff9700 1 -- 172.21.15.6:0/180945841 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f679400cb70 con 0x7f67a40fbce0 2024-03-22T06:15:32.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.603+0000 7f67a0ff9700 1 --2- 172.21.15.6:0/180945841 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f678c06bb10 0x7f678c06dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:15:32.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.603+0000 7f67a0ff9700 1 -- 172.21.15.6:0/180945841 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f679408a2c0 con 0x7f67a40fbce0 2024-03-22T06:15:32.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.603+0000 7f67a37fe700 1 --2- 172.21.15.6:0/180945841 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f678c06bb10 0x7f678c06dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:15:32.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.603+0000 7f67a37fe700 1 --2- 172.21.15.6:0/180945841 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f678c06bb10 0x7f678c06dfc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f679800b0d0 tx=0x7f6798009e30 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:15:32.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:32.609+0000 7f67a0ff9700 1 -- 172.21.15.6:0/180945841 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f6794057560 con 0x7f67a40fbce0 2024-03-22T06:15:32.613 INFO:tasks.workunit.client.1.smithi117.stdout:1/952: mknod d1/d30/d44/d9c/d107/c14e 0 2024-03-22T06:15:32.613 INFO:tasks.workunit.client.1.smithi117.stdout:2/884: getdents d2/d5/d13 0 2024-03-22T06:15:32.614 INFO:tasks.workunit.client.1.smithi117.stdout:2/885: dread - d2/d5/d13/d57/dd0/d1d/d3a/d10f/f12d zero size 2024-03-22T06:15:32.615 INFO:tasks.workunit.client.1.smithi117.stdout:0/930: rename d1/d10/f10d to d1/d10/d18/d26/d49/d7e/f148 0 2024-03-22T06:15:32.616 INFO:tasks.workunit.client.1.smithi117.stdout:8/922: stat d1/d3a/d44/d16/c89 0 2024-03-22T06:15:32.616 INFO:tasks.workunit.client.1.smithi117.stdout:8/923: dread - d1/d3a/d44/d6a/dfa/ffc zero size 2024-03-22T06:15:32.632 INFO:tasks.workunit.client.1.smithi117.stdout:1/953: getdents d1/d1f/d32/d33/da4 0 2024-03-22T06:15:32.633 INFO:tasks.workunit.client.1.smithi117.stdout:1/954: truncate d1/d1f/f105 440450 0 2024-03-22T06:15:32.635 INFO:tasks.workunit.client.1.smithi117.stdout:2/886: symlink d2/d5/d13/d57/dd0/d1d/d86/l133 0 2024-03-22T06:15:32.639 INFO:tasks.workunit.client.1.smithi117.stdout:0/931: mkdir d1/d10/d18/d8c/d104/d149 0 2024-03-22T06:15:32.643 INFO:tasks.workunit.client.1.smithi117.stdout:8/924: symlink d1/d3a/d44/d11/d97/l12e 0 2024-03-22T06:15:32.643 INFO:tasks.workunit.client.1.smithi117.stdout:8/925: creat d1/d3a/d44/d6a/dfd/f12f x:0 0 0 2024-03-22T06:15:32.647 INFO:tasks.workunit.client.1.smithi117.stdout:1/955: truncate d1/d1f/f7f 418770 0 2024-03-22T06:15:32.651 INFO:tasks.workunit.client.1.smithi117.stdout:2/887: truncate d2/d35/d4c/f9f 2512298 0 2024-03-22T06:15:32.656 INFO:tasks.workunit.client.1.smithi117.stdout:0/932: creat d1/d10/d18/d26/d49/f14a x:0 0 0 2024-03-22T06:15:32.657 INFO:tasks.workunit.client.1.smithi117.stdout:0/933: write d1/d10/d5c/d40/dbb/fcd [4914304,63495] 0 2024-03-22T06:15:32.662 INFO:tasks.workunit.client.1.smithi117.stdout:1/956: write d1/d22/d27/f100 [4030505,24241] 0 2024-03-22T06:15:32.662 INFO:tasks.workunit.client.1.smithi117.stdout:1/957: readlink d1/d22/d27/le6 0 2024-03-22T06:15:32.662 INFO:tasks.workunit.client.1.smithi117.stdout:1/958: dread - d1/d1f/d32/f13d zero size 2024-03-22T06:15:32.662 INFO:tasks.workunit.client.1.smithi117.stdout:7/874: rename d8/d9/d37/d5f to d8/d9/d37/d45/d88/db1/dc6/d10e 0 2024-03-22T06:15:32.663 INFO:tasks.workunit.client.1.smithi117.stdout:2/888: dread d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d102/fb4 [0,4194304] 0 2024-03-22T06:15:32.663 INFO:tasks.workunit.client.1.smithi117.stdout:0/934: creat d1/d10/d5c/d40/d4e/dbc/ddb/df9/f14b x:0 0 0 2024-03-22T06:15:32.667 INFO:tasks.workunit.client.1.smithi117.stdout:3/931: rename d9/la4 to d9/d66/d12b/l138 0 2024-03-22T06:15:32.668 INFO:tasks.workunit.client.1.smithi117.stdout:2/889: truncate d2/f26 365951 0 2024-03-22T06:15:32.671 INFO:tasks.workunit.client.1.smithi117.stdout:0/935: rmdir d1/d9d 39 2024-03-22T06:15:32.672 INFO:tasks.workunit.client.1.smithi117.stdout:1/959: mknod d1/c14f 0 2024-03-22T06:15:32.676 INFO:tasks.workunit.client.1.smithi117.stdout:8/926: dwrite d1/d19/d38/f75 [0,4194304] 0 2024-03-22T06:15:32.677 INFO:tasks.workunit.client.1.smithi117.stdout:7/875: dwrite d8/d9/d37/fb5 [0,4194304] 0 2024-03-22T06:15:32.679 INFO:tasks.workunit.client.1.smithi117.stdout:2/890: link d2/d5/d13/d57/dd0/d1d/d86/l133 d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/dc9/l134 0 2024-03-22T06:15:32.684 INFO:tasks.workunit.client.1.smithi117.stdout:0/936: dread d1/d10/d18/d26/d53/f93 [0,4194304] 0 2024-03-22T06:15:32.684 INFO:tasks.workunit.client.1.smithi117.stdout:8/927: dread d1/d3a/d44/d4b/dc9/fda [0,4194304] 0 2024-03-22T06:15:32.685 INFO:tasks.workunit.client.1.smithi117.stdout:8/928: chown d1/d28/d104/c113 343948 1 2024-03-22T06:15:32.685 INFO:tasks.workunit.client.1.smithi117.stdout:8/929: stat d1/d19/d98/fd8 0 2024-03-22T06:15:32.690 INFO:tasks.workunit.client.1.smithi117.stdout:9/985: dwrite d4/d46/f55 [0,4194304] 0 2024-03-22T06:15:32.692 INFO:tasks.workunit.client.1.smithi117.stdout:7/876: unlink d8/d9/dd1/l3f 0 2024-03-22T06:15:32.692 INFO:tasks.workunit.client.1.smithi117.stdout:7/877: readlink d8/d9/d37/d45/l10d 0 2024-03-22T06:15:32.692 INFO:tasks.workunit.client.1.smithi117.stdout:7/878: getdents d8/d9/db0/dcf 0 2024-03-22T06:15:32.695 INFO:tasks.workunit.client.1.smithi117.stdout:2/891: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/f135 x:0 0 0 2024-03-22T06:15:32.696 INFO:tasks.workunit.client.1.smithi117.stdout:1/960: mknod d1/d22/d27/d4e/c150 0 2024-03-22T06:15:32.697 INFO:tasks.workunit.client.1.smithi117.stdout:0/937: truncate d1/d10/f27 1108270 0 2024-03-22T06:15:32.698 INFO:tasks.workunit.client.1.smithi117.stdout:8/930: truncate d1/d28/d66/d7f/d9d/ff1 1903046 0 2024-03-22T06:15:32.700 INFO:tasks.workunit.client.1.smithi117.stdout:7/879: link d8/cc d8/d9/da/d10/d23/d48/d91/c10f 0 2024-03-22T06:15:32.701 INFO:tasks.workunit.client.1.smithi117.stdout:7/880: write d8/d9/d37/d45/d88/db1/dc6/d10e/dce/f101 [957093,59965] 0 2024-03-22T06:15:32.702 INFO:tasks.workunit.client.1.smithi117.stdout:2/892: creat d2/d5/db/d2c/d7d/dff/d39/d77/f136 x:0 0 0 2024-03-22T06:15:32.702 INFO:tasks.workunit.client.1.smithi117.stdout:2/893: creat d2/d5/d13/dde/f137 x:0 0 0 2024-03-22T06:15:32.702 INFO:tasks.workunit.client.1.smithi117.stdout:2/894: readlink d2/d35/d38/l119 0 2024-03-22T06:15:32.705 INFO:tasks.workunit.client.1.smithi117.stdout:1/961: rename d1/d22/l57 to d1/d30/l151 0 2024-03-22T06:15:32.705 INFO:tasks.workunit.client.1.smithi117.stdout:1/962: truncate d1/d30/d44/d99/f133 414891 0 2024-03-22T06:15:32.705 INFO:tasks.workunit.client.1.smithi117.stdout:1/963: stat d1/d22/d27/d114/fe0 0 2024-03-22T06:15:32.705 INFO:tasks.workunit.client.1.smithi117.stdout:1/964: truncate d1/d30/d50/f63 1799256 0 2024-03-22T06:15:32.709 INFO:tasks.workunit.client.1.smithi117.stdout:0/938: truncate d1/d3/d2b/d3e/de/d1f/f3c 430902 0 2024-03-22T06:15:32.710 INFO:tasks.workunit.client.1.smithi117.stdout:9/986: getdents d4/d8/d93 0 2024-03-22T06:15:32.710 INFO:tasks.workunit.client.1.smithi117.stdout:9/987: write d4/d8/d3e/d7a/d2f/fe0 [2304386,93233] 0 2024-03-22T06:15:32.711 INFO:tasks.workunit.client.1.smithi117.stdout:7/881: truncate d8/d9/da/f28 4862342 0 2024-03-22T06:15:32.714 INFO:tasks.workunit.client.1.smithi117.stdout:2/895: creat d2/d5/db/d2c/d7d/dff/d39/d45/ddd/f138 x:0 0 0 2024-03-22T06:15:32.715 INFO:tasks.workunit.client.1.smithi117.stdout:1/965: mknod d1/d22/d27/d4e/df6/c152 0 2024-03-22T06:15:32.715 INFO:tasks.workunit.client.1.smithi117.stdout:8/931: dwrite d1/d3a/d44/d4b/dae/fb8 [0,4194304] 0 2024-03-22T06:15:32.717 INFO:tasks.workunit.client.1.smithi117.stdout:9/988: creat d4/d8/d3f/f14d x:0 0 0 2024-03-22T06:15:32.719 INFO:tasks.workunit.client.1.smithi117.stdout:7/882: creat d8/d9/d37/d45/d61/f110 x:0 0 0 2024-03-22T06:15:32.722 INFO:tasks.workunit.client.1.smithi117.stdout:2/896: unlink d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/f15 0 2024-03-22T06:15:32.723 INFO:tasks.workunit.client.1.smithi117.stdout:2/897: write d2/d5/db/d2c/d7d/dff/d39/fec [118798,76765] 0 2024-03-22T06:15:32.723 INFO:tasks.workunit.client.1.smithi117.stdout:1/966: getdents d1/d30/d44/dd3/d13f 0 2024-03-22T06:15:32.726 INFO:tasks.workunit.client.1.smithi117.stdout:0/939: dwrite d1/d10/d18/d8c/fc7 [0,4194304] 0 2024-03-22T06:15:32.732 INFO:tasks.workunit.client.1.smithi117.stdout:1/967: mkdir d1/d62/d12f/d153 0 2024-03-22T06:15:32.735 INFO:tasks.workunit.client.1.smithi117.stdout:1/968: dread d1/d30/d44/d99/dd7/f116 [0,4194304] 0 2024-03-22T06:15:32.735 INFO:tasks.workunit.client.1.smithi117.stdout:1/969: truncate d1/d22/d27/f8e 2246803 0 2024-03-22T06:15:32.736 INFO:tasks.workunit.client.1.smithi117.stdout:1/970: getdents d1/d30/d44/d99/df7 0 2024-03-22T06:15:32.737 INFO:tasks.workunit.client.1.smithi117.stdout:0/940: rmdir d1/d3/dae/dbe/d117 39 2024-03-22T06:15:32.737 INFO:tasks.workunit.client.1.smithi117.stdout:0/941: creat d1/d3/d2b/d3e/de/f14c x:0 0 0 2024-03-22T06:15:32.737 INFO:tasks.workunit.client.1.smithi117.stdout:0/942: chown d1/d3/d2b/d3e/de/d1f/d3b/d55/fec 0 1 2024-03-22T06:15:32.739 INFO:tasks.workunit.client.1.smithi117.stdout:1/971: creat d1/d30/d35/d58/d13e/f154 x:0 0 0 2024-03-22T06:15:32.739 INFO:tasks.workunit.client.1.smithi117.stdout:1/972: chown d1/d1f/d11d/c138 474 1 2024-03-22T06:15:32.739 INFO:tasks.workunit.client.1.smithi117.stdout:1/973: fdatasync d1/d30/d35/d58/fb0 0 2024-03-22T06:15:32.741 INFO:tasks.workunit.client.1.smithi117.stdout:2/898: dwrite d2/d5/d13/d57/dd0/d1d/fa4 [0,4194304] 0 2024-03-22T06:15:32.741 INFO:tasks.workunit.client.1.smithi117.stdout:9/989: dwrite d4/d8/d3e/d7a/d3b/f57 [0,4194304] 0 2024-03-22T06:15:32.741 INFO:tasks.workunit.client.1.smithi117.stdout:2/899: chown d2/d9a/ld1 64785 1 2024-03-22T06:15:32.743 INFO:tasks.workunit.client.1.smithi117.stdout:0/943: link d1/d10/f86 d1/d10/d18/d8c/d104/d149/f14d 0 2024-03-22T06:15:32.750 INFO:tasks.workunit.client.1.smithi117.stdout:2/900: dread - d2/d5/d13/d57/dd0/f97 zero size 2024-03-22T06:15:32.751 INFO:tasks.workunit.client.1.smithi117.stdout:1/974: write d1/d22/d27/d114/f96 [3439346,31731] 0 2024-03-22T06:15:32.752 INFO:tasks.workunit.client.1.smithi117.stdout:9/990: rename d4/d47/lf2 to d4/d47/d64/dc5/d14a/l14e 0 2024-03-22T06:15:32.752 INFO:tasks.workunit.client.1.smithi117.stdout:9/991: write d4/d8/d3e/d7a/f42 [1643067,27059] 0 2024-03-22T06:15:32.759 INFO:tasks.workunit.client.1.smithi117.stdout:0/944: unlink d1/d3/d2b/d3e/de/d1f/ddd/f101 0 2024-03-22T06:15:32.759 INFO:tasks.workunit.client.1.smithi117.stdout:0/945: fdatasync d1/f87 0 2024-03-22T06:15:32.759 INFO:tasks.workunit.client.1.smithi117.stdout:0/946: dread - d1/d10/d5c/d40/d4e/dbc/f121 zero size 2024-03-22T06:15:32.759 INFO:tasks.workunit.client.1.smithi117.stdout:0/947: dread - d1/d10/d5c/d40/dac/ff7 zero size 2024-03-22T06:15:32.759 INFO:tasks.workunit.client.1.smithi117.stdout:4/907: sync 2024-03-22T06:15:32.773 INFO:tasks.workunit.client.1.smithi117.stdout:2/901: mknod d2/d5/d13/d57/dd0/d1d/d3a/d71/c139 0 2024-03-22T06:15:32.779 INFO:tasks.workunit.client.1.smithi117.stdout:9/992: mknod d4/d47/d85/c14f 0 2024-03-22T06:15:32.786 INFO:tasks.workunit.client.1.smithi117.stdout:9/993: write d4/d7/f2d [3411004,32652] 0 2024-03-22T06:15:32.786 INFO:tasks.workunit.client.1.smithi117.stdout:9/994: creat d4/d8/f150 x:0 0 0 2024-03-22T06:15:32.786 INFO:tasks.workunit.client.1.smithi117.stdout:0/948: creat d1/d10/d5c/d40/dac/f14e x:0 0 0 2024-03-22T06:15:32.786 INFO:tasks.workunit.client.1.smithi117.stdout:0/949: dread - d1/d10/d5c/d40/d4e/dbc/f135 zero size 2024-03-22T06:15:32.787 INFO:tasks.workunit.client.1.smithi117.stdout:4/908: link d0/d4/d2a/d5b/fa6 d0/d4/d2a/d2f/f132 0 2024-03-22T06:15:32.799 INFO:tasks.workunit.client.1.smithi117.stdout:9/995: mkdir d4/d7/d151 0 2024-03-22T06:15:32.800 INFO:tasks.workunit.client.1.smithi117.stdout:0/950: truncate d1/d3/d2b/d3e/ffe 3738068 0 2024-03-22T06:15:32.802 INFO:tasks.workunit.client.1.smithi117.stdout:4/909: mknod d0/d51/da4/c133 0 2024-03-22T06:15:32.802 INFO:tasks.workunit.client.1.smithi117.stdout:4/910: dread - d0/d4/d2a/d2f/fd0 zero size 2024-03-22T06:15:32.802 INFO:tasks.workunit.client.1.smithi117.stdout:4/911: write d0/d19/d28/f45 [4663386,31425] 0 2024-03-22T06:15:32.803 INFO:tasks.workunit.client.1.smithi117.stdout:4/912: fsync d0/d19/d34/d43/f62 0 2024-03-22T06:15:32.804 INFO:tasks.workunit.client.1.smithi117.stdout:2/902: rename d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d102/fe6 to d2/d5/db/d2c/d7d/dff/d39/d77/f13a 0 2024-03-22T06:15:32.805 INFO:tasks.workunit.client.1.smithi117.stdout:9/996: mkdir d4/d8/d93/d152 0 2024-03-22T06:15:32.808 INFO:tasks.workunit.client.1.smithi117.stdout:2/903: mknod d2/d5/db/d2c/d7d/dff/c13b 0 2024-03-22T06:15:32.810 INFO:tasks.workunit.client.1.smithi117.stdout:9/997: rmdir d4/d7/dbf/d100/dd1/df8 0 2024-03-22T06:15:32.810 INFO:tasks.workunit.client.1.smithi117.stdout:0/951: write d1/d10/d5c/d40/dbb/fcd [4140864,111902] 0 2024-03-22T06:15:32.812 INFO:tasks.workunit.client.1.smithi117.stdout:9/998: creat d4/d7/dbf/d100/dd1/dd6/f153 x:0 0 0 2024-03-22T06:15:32.817 INFO:tasks.workunit.client.1.smithi117.stdout:2/904: write d2/d5/d13/d57/dd0/f51 [181196,17905] 0 2024-03-22T06:15:32.822 INFO:tasks.workunit.client.1.smithi117.stdout:2/905: symlink d2/d5/db/d2c/d7d/dff/l13c 0 2024-03-22T06:15:32.823 INFO:tasks.workunit.client.1.smithi117.stdout:2/906: stat d2/d5/db/d2c/d7d/dff/d39/f65 0 2024-03-22T06:15:32.825 INFO:tasks.workunit.client.1.smithi117.stdout:9/999: write d4/d8/d3e/d82/faa [2238844,43498] 0 2024-03-22T06:15:32.841 INFO:tasks.workunit.client.1.smithi117.stdout:7/883: dwrite f7 [0,4194304] 0 2024-03-22T06:15:32.842 INFO:tasks.workunit.client.1.smithi117.stdout:7/884: chown d8/d9/da/d10/d23/fbb 296112 1 2024-03-22T06:15:32.842 INFO:tasks.workunit.client.1.smithi117.stdout:0/952: dwrite d1/d10/d18/d26/db6/fcc [0,4194304] 0 2024-03-22T06:15:32.842 INFO:tasks.workunit.client.1.smithi117.stdout:0/953: write d1/d10/d5c/d40/f7f [1035824,32357] 0 2024-03-22T06:15:32.842 INFO:tasks.workunit.client.1.smithi117.stdout:8/932: dwrite d1/d19/fb9 [0,4194304] 0 2024-03-22T06:15:32.849 INFO:tasks.workunit.client.1.smithi117.stdout:7/885: getdents d8/d9/da/d2b/dc4/d26 0 2024-03-22T06:15:32.849 INFO:tasks.workunit.client.1.smithi117.stdout:7/886: chown d8/d9/da/d2b/d9b/fa8 466427 1 2024-03-22T06:15:32.859 INFO:tasks.workunit.client.1.smithi117.stdout:0/954: link d1/d3/d2b/f125 d1/d10/d5c/d40/d4e/dbc/ddb/f14f 0 2024-03-22T06:15:32.862 INFO:tasks.workunit.client.1.smithi117.stdout:8/933: getdents d1/d3a 0 2024-03-22T06:15:32.869 INFO:tasks.workunit.client.1.smithi117.stdout:3/932: sync 2024-03-22T06:15:32.870 INFO:tasks.workunit.client.1.smithi117.stdout:2/907: mknod d2/d5/d13/d57/dd0/d1d/d3a/d71/c13d 0 2024-03-22T06:15:32.872 INFO:tasks.workunit.client.1.smithi117.stdout:7/887: dwrite d8/d9/dd1/f44 [0,4194304] 0 2024-03-22T06:15:32.872 INFO:tasks.workunit.client.1.smithi117.stdout:7/888: creat d8/d9/d37/d45/d61/f111 x:0 0 0 2024-03-22T06:15:32.873 INFO:tasks.workunit.client.1.smithi117.stdout:0/955: dread d1/d10/d18/db0/f136 [0,4194304] 0 2024-03-22T06:15:32.877 INFO:tasks.workunit.client.1.smithi117.stdout:2/908: creat d2/d35/f13e x:0 0 0 2024-03-22T06:15:32.877 INFO:tasks.workunit.client.1.smithi117.stdout:8/934: write d1/d3a/d44/d16/fb5 [1815860,51566] 0 2024-03-22T06:15:32.877 INFO:tasks.workunit.client.1.smithi117.stdout:8/935: chown d1/d3a/d44/d9/f8f 7746 1 2024-03-22T06:15:32.879 INFO:tasks.workunit.client.1.smithi117.stdout:0/956: write d1/d10/d5c/d40/d69/d72/f76 [446100,64454] 0 2024-03-22T06:15:32.883 INFO:tasks.workunit.client.1.smithi117.stdout:7/889: mkdir d8/d9/d112 0 2024-03-22T06:15:32.885 INFO:tasks.workunit.client.1.smithi117.stdout:3/933: rmdir d9/d1a/d39/d4c/d36/d41 39 2024-03-22T06:15:32.888 INFO:tasks.workunit.client.1.smithi117.stdout:2/909: creat d2/d5/f13f x:0 0 0 2024-03-22T06:15:32.892 INFO:tasks.workunit.client.1.smithi117.stdout:0/957: mkdir d1/d10/d5c/d40/d113/d150 0 2024-03-22T06:15:32.896 INFO:tasks.workunit.client.1.smithi117.stdout:2/910: rename d2/d5/db/d2c/d7d/dff/d39/d45/d88 to d2/d9a/d140 0 2024-03-22T06:15:32.909 INFO:tasks.workunit.client.1.smithi117.stdout:2/911: dread d2/d5/f33 [0,4194304] 0 2024-03-22T06:15:32.910 INFO:tasks.workunit.client.1.smithi117.stdout:2/912: symlink d2/d5/db/d2c/d7d/dff/d11e/l141 0 2024-03-22T06:15:32.910 INFO:tasks.workunit.client.1.smithi117.stdout:2/913: dread - d2/d5/d13/dde/f137 zero size 2024-03-22T06:15:32.912 INFO:tasks.workunit.client.1.smithi117.stdout:2/914: symlink d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/dfe/l142 0 2024-03-22T06:15:32.912 INFO:tasks.workunit.client.1.smithi117.stdout:2/915: chown d2/d5/d13/d57/dd0/d1d/d3a/c29 11626 1 2024-03-22T06:15:32.912 INFO:tasks.workunit.client.1.smithi117.stdout:2/916: chown d2/d5/d13/d57/dd0/d1d/d3a/fea 60 1 2024-03-22T06:15:32.914 INFO:tasks.workunit.client.1.smithi117.stdout:2/917: rename d2/d5/db/d2c/d7d/dff/d39/d45/ddd/f116 to d2/d5/db/d2c/f143 0 2024-03-22T06:15:32.914 INFO:tasks.workunit.client.1.smithi117.stdout:2/918: mknod d2/d5/db/d2c/d7d/dff/d39/d45/c144 0 2024-03-22T06:15:32.915 INFO:tasks.workunit.client.1.smithi117.stdout:2/919: mknod d2/d5/d13/d57/dd0/d1d/c145 0 2024-03-22T06:15:32.915 INFO:tasks.workunit.client.1.smithi117.stdout:2/920: dread - d2/d5/d13/d57/dd0/d1d/d3a/f78 zero size 2024-03-22T06:15:32.915 INFO:tasks.workunit.client.1.smithi117.stdout:8/936: dwrite d1/d3a/d44/d16/f7a [0,4194304] 0 2024-03-22T06:15:32.917 INFO:tasks.workunit.client.1.smithi117.stdout:3/934: dwrite d9/d1a/f1e [0,4194304] 0 2024-03-22T06:15:32.917 INFO:tasks.workunit.client.1.smithi117.stdout:7/890: dwrite d8/d9/da/d2b/d9b/fd9 [0,4194304] 0 2024-03-22T06:15:32.917 INFO:tasks.workunit.client.1.smithi117.stdout:2/921: symlink d2/d5/db/d2c/l146 0 2024-03-22T06:15:32.917 INFO:tasks.workunit.client.1.smithi117.stdout:2/922: write d2/fba [5337389,20172] 0 2024-03-22T06:15:32.918 INFO:tasks.workunit.client.1.smithi117.stdout:0/958: dwrite d1/d10/d5c/d40/dac/ff7 [0,4194304] 0 2024-03-22T06:15:32.921 INFO:tasks.workunit.client.1.smithi117.stdout:8/937: unlink d1/d28/d6b/c84 0 2024-03-22T06:15:32.924 INFO:tasks.workunit.client.1.smithi117.stdout:0/959: dread d1/f4f [0,4194304] 0 2024-03-22T06:15:32.924 INFO:tasks.workunit.client.1.smithi117.stdout:0/960: truncate d1/d3/d2b/d3e/de/d1f/d59/f5f 478532 0 2024-03-22T06:15:32.925 INFO:tasks.workunit.client.1.smithi117.stdout:3/935: symlink d9/d1a/d39/d82/l139 0 2024-03-22T06:15:32.926 INFO:tasks.workunit.client.1.smithi117.stdout:2/923: write d2/d5/d13/d57/f36 [444012,128517] 0 2024-03-22T06:15:32.927 INFO:tasks.workunit.client.1.smithi117.stdout:7/891: symlink d8/d9/d37/d45/l113 0 2024-03-22T06:15:32.931 INFO:tasks.workunit.client.1.smithi117.stdout:8/938: symlink d1/d28/d66/d7f/l130 0 2024-03-22T06:15:32.933 INFO:tasks.workunit.client.1.smithi117.stdout:3/936: mkdir d9/d1a/d39/d4c/d113/def/d129/d13a 0 2024-03-22T06:15:32.935 INFO:tasks.workunit.client.1.smithi117.stdout:2/924: mknod d2/d35/d38/c147 0 2024-03-22T06:15:32.935 INFO:tasks.workunit.client.1.smithi117.stdout:2/925: stat d2/d5/db/d2c/d7d/dff/d39/l120 0 2024-03-22T06:15:32.937 INFO:tasks.workunit.client.1.smithi117.stdout:0/961: write d1/d10/d18/f95 [3733096,57436] 0 2024-03-22T06:15:32.937 INFO:tasks.workunit.client.1.smithi117.stdout:7/892: link d8/d9/da/d10/d23/d41/f75 d8/d9/da/d2b/dc4/dac/ddc/f114 0 2024-03-22T06:15:32.939 INFO:tasks.workunit.client.1.smithi117.stdout:8/939: creat d1/d3a/d44/d16/d108/f131 x:0 0 0 2024-03-22T06:15:32.939 INFO:tasks.workunit.client.1.smithi117.stdout:8/940: chown d1/d28/d66/d7f/d9d/lc0 748462961 1 2024-03-22T06:15:32.939 INFO:tasks.workunit.client.1.smithi117.stdout:7/893: write d8/d9/ff2 [66615,5190] 0 2024-03-22T06:15:32.939 INFO:tasks.workunit.client.1.smithi117.stdout:7/894: creat d8/d9/da/d2b/dc4/d5c/f115 x:0 0 0 2024-03-22T06:15:32.941 INFO:tasks.workunit.client.1.smithi117.stdout:3/937: creat d9/d1a/d39/d8a/f13b x:0 0 0 2024-03-22T06:15:32.941 INFO:tasks.workunit.client.1.smithi117.stdout:2/926: mknod d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/c148 0 2024-03-22T06:15:32.941 INFO:tasks.workunit.client.1.smithi117.stdout:2/927: write d2/d5/d13/d57/dd0/d1d/f121 [210243,20139] 0 2024-03-22T06:15:32.944 INFO:tasks.workunit.client.1.smithi117.stdout:0/962: mknod d1/d10/d5c/d40/d4e/dbc/c151 0 2024-03-22T06:15:32.948 INFO:tasks.workunit.client.1.smithi117.stdout:7/895: unlink d8/d9/da/d10/d23/l6b 0 2024-03-22T06:15:32.950 INFO:tasks.workunit.client.1.smithi117.stdout:3/938: symlink d9/d1a/l13c 0 2024-03-22T06:15:32.951 INFO:tasks.workunit.client.1.smithi117.stdout:3/939: readlink d9/d1a/dc1/dd0/lf9 0 2024-03-22T06:15:32.952 INFO:tasks.workunit.client.1.smithi117.stdout:3/940: read d9/d1a/d39/d4c/d37/fc3 [533914,24757] 0 2024-03-22T06:15:32.953 INFO:tasks.workunit.client.1.smithi117.stdout:3/941: write d9/ff2 [165733,122549] 0 2024-03-22T06:15:32.953 INFO:tasks.workunit.client.1.smithi117.stdout:3/942: creat d9/d1a/d39/d4c/d113/f13d x:0 0 0 2024-03-22T06:15:32.953 INFO:tasks.workunit.client.1.smithi117.stdout:3/943: creat d9/d1a/d39/d4c/d37/d69/d101/f13e x:0 0 0 2024-03-22T06:15:32.953 INFO:tasks.workunit.client.1.smithi117.stdout:3/944: truncate d9/d1a/d39/d4c/d37/d69/d101/f13e 891017 0 2024-03-22T06:15:32.953 INFO:tasks.workunit.client.1.smithi117.stdout:2/928: mkdir d2/d5/d13/d57/dd0/d1d/d3a/d149 0 2024-03-22T06:15:32.954 INFO:tasks.workunit.client.1.smithi117.stdout:2/929: fdatasync d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/faf 0 2024-03-22T06:15:32.955 INFO:tasks.workunit.client.1.smithi117.stdout:0/963: rmdir d1/d10/d18/d26/d53 39 2024-03-22T06:15:32.955 INFO:tasks.workunit.client.1.smithi117.stdout:0/964: creat d1/d3/d2b/d3e/de/f152 x:0 0 0 2024-03-22T06:15:32.961 INFO:tasks.workunit.client.1.smithi117.stdout:2/930: mkdir d2/d9a/d140/d124/d14a 0 2024-03-22T06:15:32.961 INFO:tasks.workunit.client.1.smithi117.stdout:2/931: fdatasync d2/d5/d13/d57/dd0/f51 0 2024-03-22T06:15:32.961 INFO:tasks.workunit.client.1.smithi117.stdout:2/932: fsync d2/d5/d13/d57/dd0/f51 0 2024-03-22T06:15:32.965 INFO:tasks.workunit.client.1.smithi117.stdout:2/933: dread d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/fc5 [0,4194304] 0 2024-03-22T06:15:32.965 INFO:tasks.workunit.client.1.smithi117.stdout:2/934: chown d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d102/f8a 726 1 2024-03-22T06:15:32.969 INFO:tasks.workunit.client.1.smithi117.stdout:2/935: rename d2/d5/db/d2c/d7d/dff/d39/d77/ced to d2/d5/d13/d57/dd0/c14b 0 2024-03-22T06:15:32.969 INFO:tasks.workunit.client.1.smithi117.stdout:2/936: fdatasync d2/d5/d13/d57/dd0/f97 0 2024-03-22T06:15:32.971 INFO:tasks.workunit.client.1.smithi117.stdout:4/913: sync 2024-03-22T06:15:32.971 INFO:tasks.workunit.client.1.smithi117.stdout:1/975: sync 2024-03-22T06:15:32.971 INFO:tasks.workunit.client.1.smithi117.stdout:1/976: read d1/d30/f136 [5851345,11195] 0 2024-03-22T06:15:32.972 INFO:tasks.workunit.client.1.smithi117.stdout:3/945: dwrite d9/d1a/f83 [0,4194304] 0 2024-03-22T06:15:32.972 INFO:tasks.workunit.client.1.smithi117.stdout:8/941: dwrite d1/d28/d6b/df4/ff5 [0,4194304] 0 2024-03-22T06:15:32.972 INFO:tasks.workunit.client.1.smithi117.stdout:1/977: mkdir d1/d30/d35/d155 0 2024-03-22T06:15:32.973 INFO:tasks.workunit.client.1.smithi117.stdout:2/937: dread d2/d5/d13/d57/ff2 [0,4194304] 0 2024-03-22T06:15:32.978 INFO:tasks.workunit.client.1.smithi117.stdout:3/946: symlink d9/d1a/d39/d82/l13f 0 2024-03-22T06:15:32.985 INFO:tasks.workunit.client.1.smithi117.stdout:4/914: write d0/d48/da0/dd2/db6/dde/d108/d6f/f91 [2141290,69901] 0 2024-03-22T06:15:32.987 INFO:tasks.workunit.client.1.smithi117.stdout:8/942: link d1/d3a/fa6 d1/d28/d6b/df4/f132 0 2024-03-22T06:15:32.988 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:32 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3772418066' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:15:32.991 INFO:tasks.workunit.client.1.smithi117.stdout:3/947: mkdir d9/d1a/d39/d8a/d140 0 2024-03-22T06:15:32.992 INFO:tasks.workunit.client.1.smithi117.stdout:1/978: dwrite d1/d30/d5d/d6f/f11a [0,4194304] 0 2024-03-22T06:15:32.992 INFO:tasks.workunit.client.1.smithi117.stdout:0/965: dwrite d1/d10/d5c/d40/d4e/dbc/fe7 [0,4194304] 0 2024-03-22T06:15:32.997 INFO:tasks.workunit.client.1.smithi117.stdout:4/915: creat d0/d48/da0/dd2/d121/de8/f134 x:0 0 0 2024-03-22T06:15:32.998 INFO:tasks.workunit.client.1.smithi117.stdout:2/938: getdents d2/d5/d13/d57/dd0/d1d/d3a/d10f 0 2024-03-22T06:15:33.001 INFO:tasks.workunit.client.1.smithi117.stdout:3/948: creat d9/d1a/d39/d4c/d37/f141 x:0 0 0 2024-03-22T06:15:33.005 INFO:tasks.workunit.client.1.smithi117.stdout:1/979: link d1/d30/d44/d99/fb7 d1/d22/d106/dc1/f156 0 2024-03-22T06:15:33.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:32 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3772418066' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:15:33.006 INFO:tasks.workunit.client.1.smithi117.stdout:8/943: dwrite d1/d3a/d44/d9/fec [0,4194304] 0 2024-03-22T06:15:33.008 INFO:tasks.workunit.client.1.smithi117.stdout:0/966: creat d1/d3/d2b/d3e/de/d1f/d59/dcb/f153 x:0 0 0 2024-03-22T06:15:33.012 INFO:tasks.workunit.client.1.smithi117.stdout:3/949: dread d9/d1a/d39/d4c/d36/fb2 [0,4194304] 0 2024-03-22T06:15:33.014 INFO:tasks.workunit.client.1.smithi117.stdout:4/916: link d0/d4/d2a/lc5 d0/d48/da0/dd2/db6/dde/d108/d6f/de5/l135 0 2024-03-22T06:15:33.016 INFO:tasks.workunit.client.1.smithi117.stdout:2/939: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d102/d83/f14c x:0 0 0 2024-03-22T06:15:33.017 INFO:tasks.workunit.client.1.smithi117.stdout:8/944: dread d1/d28/d6b/df4/ff5 [0,4194304] 0 2024-03-22T06:15:33.017 INFO:tasks.workunit.client.1.smithi117.stdout:8/945: write d1/d3a/d44/d9/fec [4761268,64528] 0 2024-03-22T06:15:33.019 INFO:tasks.workunit.client.1.smithi117.stdout:1/980: mkdir d1/d84/d77/d157 0 2024-03-22T06:15:33.021 INFO:tasks.workunit.client.1.smithi117.stdout:3/950: dread d9/d55/f59 [0,4194304] 0 2024-03-22T06:15:33.022 INFO:tasks.workunit.client.1.smithi117.stdout:0/967: link d1/d3/d2b/d3e/de/d1f/d3b/d55/fec d1/d10/f154 0 2024-03-22T06:15:33.023 INFO:tasks.workunit.client.1.smithi117.stdout:8/946: dread d1/d19/d98/fe6 [0,4194304] 0 2024-03-22T06:15:33.024 INFO:tasks.workunit.client.1.smithi117.stdout:4/917: link d0/d4/d2a/d2f/fec d0/d4/d2a/f136 0 2024-03-22T06:15:33.024 INFO:tasks.workunit.client.1.smithi117.stdout:4/918: write d0/d4/d2a/d5b/f97 [175400,91981] 0 2024-03-22T06:15:33.027 INFO:tasks.workunit.client.1.smithi117.stdout:1/981: mknod d1/d1f/d32/d33/da4/c158 0 2024-03-22T06:15:33.032 INFO:tasks.workunit.client.1.smithi117.stdout:3/951: symlink d9/d1a/dc1/d102/l142 0 2024-03-22T06:15:33.032 INFO:tasks.workunit.client.1.smithi117.stdout:3/952: read - d9/d66/fbd zero size 2024-03-22T06:15:33.032 INFO:tasks.workunit.client.1.smithi117.stdout:3/953: write d9/d1a/d39/d4c/d37/dcc/ff5 [61952,120086] 0 2024-03-22T06:15:33.032 INFO:tasks.workunit.client.1.smithi117.stdout:3/954: stat d9/d93/fcd 0 2024-03-22T06:15:33.032 INFO:tasks.workunit.client.1.smithi117.stdout:3/955: chown d9/d1a/d39/d4c/d113/fee 38817 1 2024-03-22T06:15:33.035 INFO:tasks.workunit.client.1.smithi117.stdout:0/968: link d1/d3/d2b/d3e/de/d1f/d3b/f144 d1/d10/d5c/d40/d113/d150/f155 0 2024-03-22T06:15:33.037 INFO:tasks.workunit.client.1.smithi117.stdout:8/947: getdents d1/d19/de0 0 2024-03-22T06:15:33.040 INFO:tasks.workunit.client.1.smithi117.stdout:4/919: link d0/d4/d2a/d2f/fd0 d0/f137 0 2024-03-22T06:15:33.041 INFO:tasks.workunit.client.1.smithi117.stdout:1/982: mkdir d1/d30/d35/d58/d159 0 2024-03-22T06:15:33.042 INFO:tasks.workunit.client.1.smithi117.stdout:0/969: unlink d1/d10/d18/l1e 0 2024-03-22T06:15:33.045 INFO:tasks.workunit.client.1.smithi117.stdout:4/920: link d0/d48/da0/dd2/db6/dde/d108/f74 d0/d19/d28/d47/dd6/f138 0 2024-03-22T06:15:33.046 INFO:tasks.workunit.client.1.smithi117.stdout:0/970: dread d1/d3/f12 [0,4194304] 0 2024-03-22T06:15:33.046 INFO:tasks.workunit.client.1.smithi117.stdout:0/971: dread - d1/d10/d5c/d40/d4e/dbc/ddb/df9/f14b zero size 2024-03-22T06:15:33.047 INFO:tasks.workunit.client.1.smithi117.stdout:0/972: dread - d1/d3/dae/dbe/d117/f119 zero size 2024-03-22T06:15:33.047 INFO:tasks.workunit.client.1.smithi117.stdout:0/973: dread - d1/d3/d2b/d3e/de/d1f/d3b/d55/f91 zero size 2024-03-22T06:15:33.047 INFO:tasks.workunit.client.1.smithi117.stdout:0/974: chown d1/d10/d118 14853 1 2024-03-22T06:15:33.047 INFO:tasks.workunit.client.1.smithi117.stdout:0/975: chown d1/d10/d18/d26/d49/cc1 356 1 2024-03-22T06:15:33.047 INFO:tasks.workunit.client.1.smithi117.stdout:0/976: dread - d1/d10/d5c/d40/d69/fa2 zero size 2024-03-22T06:15:33.047 INFO:tasks.workunit.client.1.smithi117.stdout:1/983: mkdir d1/d30/d44/d15a 0 2024-03-22T06:15:33.047 INFO:tasks.workunit.client.1.smithi117.stdout:4/921: truncate d0/d48/fd8 362891 0 2024-03-22T06:15:33.048 INFO:tasks.workunit.client.1.smithi117.stdout:1/984: mkdir d1/d30/d35/d155/d15b 0 2024-03-22T06:15:33.048 INFO:tasks.workunit.client.1.smithi117.stdout:1/985: creat d1/d1f/d32/f15c x:0 0 0 2024-03-22T06:15:33.049 INFO:tasks.workunit.client.1.smithi117.stdout:4/922: mkdir d0/d19/d28/d139 0 2024-03-22T06:15:33.049 INFO:tasks.workunit.client.1.smithi117.stdout:4/923: fsync d0/d3f/f95 0 2024-03-22T06:15:33.051 INFO:tasks.workunit.client.1.smithi117.stdout:3/956: dwrite d9/d1a/d39/d4c/d36/fb2 [4194304,4194304] 0 2024-03-22T06:15:33.052 INFO:tasks.workunit.client.1.smithi117.stdout:1/986: creat d1/d22/d27/d4e/f15d x:0 0 0 2024-03-22T06:15:33.057 INFO:tasks.workunit.client.1.smithi117.stdout:0/977: dread d1/d10/d5c/d40/df4/f100 [0,4194304] 0 2024-03-22T06:15:33.057 INFO:tasks.workunit.client.1.smithi117.stdout:0/978: chown d1/d10/d5c/d40/d4e/f79 6 1 2024-03-22T06:15:33.058 INFO:tasks.workunit.client.1.smithi117.stdout:1/987: getdents d1/d84/d77 0 2024-03-22T06:15:33.061 INFO:tasks.workunit.client.1.smithi117.stdout:1/988: dread d1/d30/fb4 [0,4194304] 0 2024-03-22T06:15:33.065 INFO:tasks.workunit.client.1.smithi117.stdout:0/979: read d1/d3/d2b/d3e/d3f/f6d [710433,16686] 0 2024-03-22T06:15:33.065 INFO:tasks.workunit.client.1.smithi117.stdout:0/980: write d1/d10/d18/d26/d49/d7e/f13c [873702,10577] 0 2024-03-22T06:15:33.065 INFO:tasks.workunit.client.1.smithi117.stdout:0/981: creat d1/d3/d2b/d3e/de/d1f/d3b/f156 x:0 0 0 2024-03-22T06:15:33.065 INFO:tasks.workunit.client.1.smithi117.stdout:0/982: write d1/d10/d18/d26/d53/f93 [2505813,92071] 0 2024-03-22T06:15:33.065 INFO:tasks.workunit.client.1.smithi117.stdout:1/989: mkdir d1/d15e 0 2024-03-22T06:15:33.065 INFO:tasks.workunit.client.1.smithi117.stdout:1/990: chown d1/d1f/f105 6932 1 2024-03-22T06:15:33.067 INFO:tasks.workunit.client.1.smithi117.stdout:4/924: write d0/d48/da0/dd2/db6/dde/d108/d6f/fbc [1689268,106955] 0 2024-03-22T06:15:33.067 INFO:tasks.workunit.client.1.smithi117.stdout:8/948: dwrite d1/d28/d6b/da3/f100 [0,4194304] 0 2024-03-22T06:15:33.067 INFO:tasks.workunit.client.1.smithi117.stdout:8/949: fdatasync d1/d3a/fcb 0 2024-03-22T06:15:33.067 INFO:tasks.workunit.client.1.smithi117.stdout:8/950: write d1/d19/d98/fbc [3391219,2179] 0 2024-03-22T06:15:33.067 INFO:tasks.workunit.client.1.smithi117.stdout:3/957: read d9/f21 [607999,79481] 0 2024-03-22T06:15:33.074 INFO:tasks.workunit.client.1.smithi117.stdout:0/983: symlink d1/d10/d5c/d40/dbb/dbf/l157 0 2024-03-22T06:15:33.074 INFO:tasks.workunit.client.1.smithi117.stdout:0/984: chown d1/d3/d2b 2990758 1 2024-03-22T06:15:33.078 INFO:tasks.workunit.client.1.smithi117.stdout:1/991: mknod d1/d3c/db6/c15f 0 2024-03-22T06:15:33.082 INFO:tasks.workunit.client.1.smithi117.stdout:4/925: write d0/f23 [1154739,128270] 0 2024-03-22T06:15:33.087 INFO:tasks.workunit.client.1.smithi117.stdout:3/958: truncate d9/d93/fdc 3259722 0 2024-03-22T06:15:33.088 INFO:tasks.workunit.client.1.smithi117.stdout:0/985: dwrite d1/d3/d2b/d3e/d3f/fd7 [0,4194304] 0 2024-03-22T06:15:33.090 INFO:tasks.workunit.client.1.smithi117.stdout:4/926: creat d0/d19/dcc/f13a x:0 0 0 2024-03-22T06:15:33.096 INFO:tasks.workunit.client.1.smithi117.stdout:8/951: rmdir d1/d3a/d44/d16 39 2024-03-22T06:15:33.096 INFO:tasks.workunit.client.1.smithi117.stdout:3/959: read d9/d1a/d39/d4c/d37/dcc/fe0 [2422402,8538] 0 2024-03-22T06:15:33.097 INFO:tasks.workunit.client.1.smithi117.stdout:4/927: dread d0/d19/d34/ff3 [0,4194304] 0 2024-03-22T06:15:33.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.097+0000 7f67aa3ac700 1 -- 172.21.15.6:0/180945841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f67a404bed0 con 0x7f67a40fbce0 2024-03-22T06:15:33.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.098+0000 7f67a0ff9700 1 -- 172.21.15.6:0/180945841 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f67940570f0 con 0x7f67a40fbce0 2024-03-22T06:15:33.098 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_OK 2024-03-22T06:15:33.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.101+0000 7f67aa3ac700 1 -- 172.21.15.6:0/180945841 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f678c06bb10 msgr2=0x7f678c06dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:33.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.101+0000 7f67aa3ac700 1 --2- 172.21.15.6:0/180945841 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f678c06bb10 0x7f678c06dfc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f679800b0d0 tx=0x7f6798009e30 comp rx=0 tx=0).stop 2024-03-22T06:15:33.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.102+0000 7f67aa3ac700 1 -- 172.21.15.6:0/180945841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67a40fbce0 msgr2=0x7f67a410fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:15:33.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.102+0000 7f67aa3ac700 1 --2- 172.21.15.6:0/180945841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67a40fbce0 0x7f67a410fb70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f679400eb40 tx=0x7f679400b040 comp rx=0 tx=0).stop 2024-03-22T06:15:33.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.102+0000 7f67aa3ac700 1 -- 172.21.15.6:0/180945841 shutdown_connections 2024-03-22T06:15:33.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.102+0000 7f67aa3ac700 1 --2- 172.21.15.6:0/180945841 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f678c06bb10 0x7f678c06dfc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:33.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.102+0000 7f67aa3ac700 1 --2- 172.21.15.6:0/180945841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67a40fbce0 0x7f67a410fb70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:33.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.102+0000 7f67aa3ac700 1 --2- 172.21.15.6:0/180945841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a40fcf90 0x7f67a41100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:15:33.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.102+0000 7f67aa3ac700 1 -- 172.21.15.6:0/180945841 >> 172.21.15.6:0/180945841 conn(0x7f67a40f7440 msgr2=0x7f67a40f8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:15:33.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.102+0000 7f67aa3ac700 1 -- 172.21.15.6:0/180945841 shutdown_connections 2024-03-22T06:15:33.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:15:33.102+0000 7f67aa3ac700 1 -- 172.21.15.6:0/180945841 wait complete. 2024-03-22T06:15:33.114 INFO:tasks.workunit.client.1.smithi117.stdout:0/986: symlink d1/d3/d2b/d3e/l158 0 2024-03-22T06:15:33.117 INFO:tasks.workunit.client.1.smithi117.stdout:8/952: mkdir d1/d28/d6b/df4/d133 0 2024-03-22T06:15:33.117 INFO:tasks.workunit.client.1.smithi117.stdout:2/940: sync 2024-03-22T06:15:33.117 INFO:tasks.workunit.client.1.smithi117.stdout:7/896: sync 2024-03-22T06:15:33.120 INFO:tasks.workunit.client.1.smithi117.stdout:3/960: link d9/d1a/d39/dc7/c107 d9/d1a/c143 0 2024-03-22T06:15:33.121 INFO:tasks.workunit.client.1.smithi117.stdout:4/928: mknod d0/d48/da0/dd2/d105/c13b 0 2024-03-22T06:15:33.123 INFO:tasks.workunit.client.1.smithi117.stdout:4/929: read d0/d19/dcc/fe1 [59223,26229] 0 2024-03-22T06:15:33.123 INFO:tasks.workunit.client.1.smithi117.stdout:4/930: read - d0/d48/da0/dd2/d121/f11d zero size 2024-03-22T06:15:33.123 INFO:tasks.workunit.client.1.smithi117.stdout:4/931: read - d0/d48/da0/dd2/db6/dde/d108/d6f/ffe zero size 2024-03-22T06:15:33.123 INFO:tasks.workunit.client.1.smithi117.stdout:8/953: symlink d1/d3a/d44/d11/dc7/l134 0 2024-03-22T06:15:33.126 INFO:tasks.workunit.client.1.smithi117.stdout:2/941: mkdir d2/d35/d14d 0 2024-03-22T06:15:33.126 INFO:tasks.workunit.client.1.smithi117.stdout:2/942: chown d2/d5/db/d2c/d7d/dff/d39/d45/ddd 2433 1 2024-03-22T06:15:33.127 INFO:tasks.workunit.client.1.smithi117.stdout:7/897: dread d8/d9/da/d10/d23/f38 [0,4194304] 0 2024-03-22T06:15:33.127 INFO:tasks.workunit.client.1.smithi117.stdout:7/898: chown d8/d9/d37/d45 19 1 2024-03-22T06:15:33.129 INFO:tasks.workunit.client.1.smithi117.stdout:3/961: mkdir d9/d1a/d39/d8a/da1/d144 0 2024-03-22T06:15:33.131 INFO:tasks.workunit.client.1.smithi117.stdout:4/932: symlink d0/d19/d28/d47/l13c 0 2024-03-22T06:15:33.133 INFO:tasks.workunit.client.1.smithi117.stdout:8/954: rename d1/d19/d98/d12c to d1/d3a/d44/d11/dc7/d135 0 2024-03-22T06:15:33.134 INFO:tasks.workunit.client.1.smithi117.stdout:2/943: symlink d2/d5/d13/d57/dd0/d1d/d3a/d149/l14e 0 2024-03-22T06:15:33.134 INFO:tasks.workunit.client.1.smithi117.stdout:2/944: creat d2/d5/d13/d57/dd0/d1d/f14f x:0 0 0 2024-03-22T06:15:33.138 INFO:tasks.workunit.client.1.smithi117.stdout:7/899: rename d8/d9/da/d10/d23/d48/f57 to d8/d9/da/d10/d23/f116 0 2024-03-22T06:15:33.139 INFO:tasks.workunit.client.1.smithi117.stdout:8/955: mkdir d1/d3a/d44/d6a/dfd/d136 0 2024-03-22T06:15:33.143 INFO:tasks.workunit.client.1.smithi117.stdout:2/945: mkdir d2/d9a/dd3/d150 0 2024-03-22T06:15:33.145 INFO:tasks.workunit.client.1.smithi117.stdout:4/933: rename d0/d4/d2a/d5b/c107 to d0/d48/da0/dd2/db6/dde/d108/d6f/d5f/d12d/c13d 0 2024-03-22T06:15:33.146 INFO:tasks.workunit.client.1.smithi117.stdout:8/956: symlink d1/d19/de0/l137 0 2024-03-22T06:15:33.146 INFO:tasks.workunit.client.1.smithi117.stdout:8/957: write d1/d3a/d44/d4b/dc9/fd7 [533815,57199] 0 2024-03-22T06:15:33.148 INFO:tasks.workunit.client.1.smithi117.stdout:7/900: write d8/d9/da/d2b/dc4/f10b [6216965,104378] 0 2024-03-22T06:15:33.148 INFO:tasks.workunit.client.1.smithi117.stdout:7/901: readlink d8/d9/da/d2b/dc4/d5c/l68 0 2024-03-22T06:15:33.148 INFO:tasks.workunit.client.1.smithi117.stdout:7/902: fsync d8/d50/d5e/ffb 0 2024-03-22T06:15:33.148 INFO:tasks.workunit.client.1.smithi117.stdout:2/946: dread d2/d5/db/d2c/d7d/dff/d39/f4b [0,4194304] 0 2024-03-22T06:15:33.149 INFO:tasks.workunit.client.1.smithi117.stdout:2/947: write d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/f12c [716871,116301] 0 2024-03-22T06:15:33.149 INFO:tasks.workunit.client.1.smithi117.stdout:8/958: creat d1/d28/d66/f138 x:0 0 0 2024-03-22T06:15:33.152 INFO:tasks.workunit.client.1.smithi117.stdout:3/962: dwrite d9/d1a/d39/d82/d10a/f12c [0,4194304] 0 2024-03-22T06:15:33.153 INFO:tasks.workunit.client.1.smithi117.stdout:7/903: mknod d8/d9/da/d10/d23/d41/c117 0 2024-03-22T06:15:33.154 INFO:tasks.workunit.client.1.smithi117.stdout:2/948: mkdir d2/d5/d13/d57/dd0/d1d/d3a/da6/d151 0 2024-03-22T06:15:33.154 INFO:tasks.workunit.client.1.smithi117.stdout:2/949: creat d2/d5/f152 x:0 0 0 2024-03-22T06:15:33.155 INFO:tasks.workunit.client.1.smithi117.stdout:2/950: write d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/dfe/f109 [190029,60656] 0 2024-03-22T06:15:33.155 INFO:tasks.workunit.client.1.smithi117.stdout:2/951: dread - d2/d9a/d140/fe5 zero size 2024-03-22T06:15:33.155 INFO:tasks.workunit.client.1.smithi117.stdout:2/952: creat d2/d5/db/d2c/d7d/dff/d39/d45/ddd/f153 x:0 0 0 2024-03-22T06:15:33.155 INFO:tasks.workunit.client.1.smithi117.stdout:2/953: chown d2/d5/d13/d57/dd0/d1d/d3a/d89/fab 68055 1 2024-03-22T06:15:33.156 INFO:tasks.workunit.client.1.smithi117.stdout:8/959: symlink d1/d19/d98/l139 0 2024-03-22T06:15:33.158 INFO:tasks.workunit.client.1.smithi117.stdout:4/934: dread d0/d48/da0/dd2/db6/dde/ff0 [0,4194304] 0 2024-03-22T06:15:33.163 INFO:tasks.workunit.client.1.smithi117.stdout:7/904: dread d8/d9/da/d2b/f6e [0,4194304] 0 2024-03-22T06:15:33.164 INFO:tasks.workunit.client.1.smithi117.stdout:2/954: symlink d2/d5/d13/d57/dd0/d1d/d3a/d89/l154 0 2024-03-22T06:15:33.167 INFO:tasks.workunit.client.1.smithi117.stdout:8/960: creat d1/d3a/d44/d4b/dc9/f13a x:0 0 0 2024-03-22T06:15:33.167 INFO:tasks.workunit.client.1.smithi117.stdout:8/961: chown d1/d3a/cc1 2 1 2024-03-22T06:15:33.167 INFO:tasks.workunit.client.1.smithi117.stdout:3/963: dwrite d9/f63 [4194304,4194304] 0 2024-03-22T06:15:33.169 INFO:tasks.workunit.client.1.smithi117.stdout:4/935: link d0/d19/d28/d47/l52 d0/d19/d34/d43/d127/l13e 0 2024-03-22T06:15:33.171 INFO:tasks.workunit.client.1.smithi117.stdout:7/905: rename d8/d9/da/d10/d23/f92 to d8/d50/f118 0 2024-03-22T06:15:33.171 INFO:tasks.workunit.client.1.smithi117.stdout:7/906: readlink d8/d9/db0/ld0 0 2024-03-22T06:15:33.173 INFO:tasks.workunit.client.1.smithi117.stdout:8/962: creat d1/d3a/d44/d4b/df7/f13b x:0 0 0 2024-03-22T06:15:33.177 INFO:tasks.workunit.client.1.smithi117.stdout:7/907: dread d8/d9/d37/d45/d88/db1/dc6/d10e/fda [0,4194304] 0 2024-03-22T06:15:33.187 INFO:tasks.workunit.client.1.smithi117.stdout:3/964: mkdir d9/d55/d145 0 2024-03-22T06:15:33.202 INFO:tasks.workunit.client.1.smithi117.stdout:7/908: dwrite d8/d9/da/d2b/dc4/dac/ddc/ff9 [0,4194304] 0 2024-03-22T06:15:33.202 INFO:tasks.workunit.client.1.smithi117.stdout:7/909: write d8/d9/d37/d45/d88/fea [316705,46807] 0 2024-03-22T06:15:33.209 INFO:tasks.workunit.client.1.smithi117.stdout:8/963: mknod d1/d12d/c13c 0 2024-03-22T06:15:33.212 INFO:tasks.workunit.client.1.smithi117.stdout:7/910: dread d8/d9/da/d10/d23/f38 [0,4194304] 0 2024-03-22T06:15:33.213 INFO:tasks.workunit.client.1.smithi117.stdout:7/911: chown d8/d9/da/d2b 70064 1 2024-03-22T06:15:33.213 INFO:tasks.workunit.client.1.smithi117.stdout:8/964: dread d1/d3a/d44/f3c [4194304,4194304] 0 2024-03-22T06:15:33.213 INFO:tasks.workunit.client.1.smithi117.stdout:3/965: dwrite d9/d1a/d39/d4c/d37/dd5/fd6 [0,4194304] 0 2024-03-22T06:15:33.214 INFO:tasks.workunit.client.1.smithi117.stdout:3/966: chown d9/d1a/d39/d4c/d36/d41/l10d 170 1 2024-03-22T06:15:33.215 INFO:tasks.workunit.client.1.smithi117.stdout:8/965: write d1/d3a/d44/d16/fea [594767,26054] 0 2024-03-22T06:15:33.216 INFO:tasks.workunit.client.1.smithi117.stdout:4/936: rmdir d0/d48/da0 39 2024-03-22T06:15:33.218 INFO:tasks.workunit.client.1.smithi117.stdout:2/955: rename d2/d5/db/d2c/d7d/l122 to d2/d5/d13/d57/dd0/l155 0 2024-03-22T06:15:33.221 INFO:tasks.workunit.client.1.smithi117.stdout:1/992: sync 2024-03-22T06:15:33.221 INFO:tasks.workunit.client.1.smithi117.stdout:1/993: fsync d1/d22/d27/d114/fe0 0 2024-03-22T06:15:33.222 INFO:tasks.workunit.client.1.smithi117.stdout:3/967: symlink d9/d1a/d39/d4c/d37/dcc/l146 0 2024-03-22T06:15:33.225 INFO:tasks.workunit.client.1.smithi117.stdout:8/966: mknod d1/d3a/d44/d6a/dfa/d107/c13d 0 2024-03-22T06:15:33.227 INFO:tasks.workunit.client.1.smithi117.stdout:4/937: link d0/d4/c24 d0/d19/d34/c13f 0 2024-03-22T06:15:33.229 INFO:tasks.workunit.client.1.smithi117.stdout:0/987: sync 2024-03-22T06:15:33.230 INFO:tasks.workunit.client.1.smithi117.stdout:2/956: getdents d2/d9a/dd3 0 2024-03-22T06:15:33.233 INFO:tasks.workunit.client.1.smithi117.stdout:1/994: mknod d1/d3c/d4d/de8/c160 0 2024-03-22T06:15:33.233 INFO:tasks.workunit.client.1.smithi117.stdout:1/995: chown d1/fd 139642405 1 2024-03-22T06:15:33.233 INFO:tasks.workunit.client.1.smithi117.stdout:3/968: creat d9/d1a/d39/d7d/f147 x:0 0 0 2024-03-22T06:15:33.236 INFO:tasks.workunit.client.1.smithi117.stdout:4/938: creat d0/d19/d28/d47/f140 x:0 0 0 2024-03-22T06:15:33.236 INFO:tasks.workunit.client.1.smithi117.stdout:7/912: dwrite d8/d9/da/d10/d23/d48/dc8/fe8 [0,4194304] 0 2024-03-22T06:15:33.239 INFO:tasks.workunit.client.1.smithi117.stdout:2/957: getdents d2/d5/db/d2c/d7d/dff/d39/d45/d7c 0 2024-03-22T06:15:33.239 INFO:tasks.workunit.client.1.smithi117.stdout:2/958: dread - d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d102/f8a zero size 2024-03-22T06:15:33.239 INFO:tasks.workunit.client.1.smithi117.stdout:2/959: read - d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/f135 zero size 2024-03-22T06:15:33.239 INFO:tasks.workunit.client.1.smithi117.stdout:2/960: read - d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d102/f8a zero size 2024-03-22T06:15:33.240 INFO:tasks.workunit.client.1.smithi117.stdout:1/996: link d1/d84/d77/l11c d1/d1f/d11d/l161 0 2024-03-22T06:15:33.242 INFO:tasks.workunit.client.1.smithi117.stdout:8/967: dread d1/d19/d38/f75 [0,4194304] 0 2024-03-22T06:15:33.242 INFO:tasks.workunit.client.1.smithi117.stdout:8/968: chown d1/d3a 911 1 2024-03-22T06:15:33.242 INFO:tasks.workunit.client.1.smithi117.stdout:3/969: rmdir d9/d1a/d39/d4c/d37/d69/d122 0 2024-03-22T06:15:33.245 INFO:tasks.workunit.client.1.smithi117.stdout:7/913: rmdir d8/d9/da/d10/d23/d48/de6 0 2024-03-22T06:15:33.246 INFO:tasks.workunit.client.1.smithi117.stdout:0/988: dwrite d1/d3/d2b/d3e/ffe [0,4194304] 0 2024-03-22T06:15:33.248 INFO:tasks.workunit.client.1.smithi117.stdout:2/961: rename d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/dc9/l10a to d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/d11c/l156 0 2024-03-22T06:15:33.248 INFO:tasks.workunit.client.1.smithi117.stdout:2/962: stat d2/d35/d38 0 2024-03-22T06:15:33.249 INFO:tasks.workunit.client.1.smithi117.stdout:2/963: chown d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/dc9/l134 25108 1 2024-03-22T06:15:33.249 INFO:tasks.workunit.client.1.smithi117.stdout:2/964: write d2/d5/d13/f7a [1446533,103513] 0 2024-03-22T06:15:33.249 INFO:tasks.workunit.client.1.smithi117.stdout:1/997: truncate d1/d1f/d32/d33/da4/fd8 149133 0 2024-03-22T06:15:33.253 INFO:tasks.workunit.client.1.smithi117.stdout:3/970: symlink d9/d1a/d39/d4c/l148 0 2024-03-22T06:15:33.253 INFO:tasks.workunit.client.1.smithi117.stdout:3/971: chown d9/d1a/d3a/l8f 14 1 2024-03-22T06:15:33.253 INFO:tasks.workunit.client.1.smithi117.stdout:3/972: truncate d9/d55/fa0 730595 0 2024-03-22T06:15:33.253 INFO:tasks.workunit.client.1.smithi117.stdout:0/989: creat d1/d10/d18/d26/d49/d7e/d39/f159 x:0 0 0 2024-03-22T06:15:33.261 INFO:tasks.workunit.client.1.smithi117.stdout:2/965: write d2/d5/d13/d57/dd0/fc8 [1036097,111127] 0 2024-03-22T06:15:33.267 INFO:tasks.workunit.client.1.smithi117.stdout:1/998: link d1/d14/c47 d1/d30/d35/d155/c162 0 2024-03-22T06:15:33.269 INFO:tasks.workunit.client.1.smithi117.stdout:8/969: getdents d1/d3a/d44/d4b/dc9 0 2024-03-22T06:15:33.269 INFO:tasks.workunit.client.1.smithi117.stdout:4/939: dwrite d0/d51/fc8 [0,4194304] 0 2024-03-22T06:15:33.269 INFO:tasks.workunit.client.1.smithi117.stdout:7/914: dwrite d8/d9/da/d10/f25 [0,4194304] 0 2024-03-22T06:15:33.269 INFO:tasks.workunit.client.1.smithi117.stdout:7/915: fsync d8/d9/d37/d45/d88/db1/dc6/d10e/d82/f95 0 2024-03-22T06:15:33.269 INFO:tasks.workunit.client.1.smithi117.stdout:7/916: chown d8/d9/d37/d45/d88/db1/dc6 39805593 1 2024-03-22T06:15:33.269 INFO:tasks.workunit.client.1.smithi117.stdout:7/917: chown d8/d9/d37/d45/d88/fbf 4444 1 2024-03-22T06:15:33.274 INFO:tasks.workunit.client.1.smithi117.stdout:3/973: dwrite d9/d1a/d39/d4c/d37/d69/f87 [0,4194304] 0 2024-03-22T06:15:33.274 INFO:tasks.workunit.client.1.smithi117.stdout:0/990: link d1/d3/l37 d1/d10/d5c/l15a 0 2024-03-22T06:15:33.274 INFO:tasks.workunit.client.1.smithi117.stdout:0/991: write d1/d10/d5c/d40/dbb/fcd [870995,25771] 0 2024-03-22T06:15:33.274 INFO:tasks.workunit.client.1.smithi117.stdout:0/992: truncate d1/d3/d2b/d3e/de/fa4 4359078 0 2024-03-22T06:15:33.274 INFO:tasks.workunit.client.1.smithi117.stdout:2/966: symlink d2/d5/d13/dde/l157 0 2024-03-22T06:15:33.276 INFO:tasks.workunit.client.1.smithi117.stdout:8/970: unlink d1/d3a/d44/d11/fff 0 2024-03-22T06:15:33.276 INFO:tasks.workunit.client.1.smithi117.stdout:8/971: read d1/d3a/d44/d9/f8f [237378,48774] 0 2024-03-22T06:15:33.276 INFO:tasks.workunit.client.1.smithi117.stdout:8/972: write d1/d3a/d44/d11/dc7/f128 [891776,59439] 0 2024-03-22T06:15:33.279 INFO:tasks.workunit.client.1.smithi117.stdout:4/940: rmdir d0/d51 39 2024-03-22T06:15:33.283 INFO:tasks.workunit.client.1.smithi117.stdout:7/918: unlink d8/d9/da/d2b/dc4/dac/ddc/f114 0 2024-03-22T06:15:33.287 INFO:tasks.workunit.client.1.smithi117.stdout:8/973: dread d1/fac [0,4194304] 0 2024-03-22T06:15:33.288 INFO:tasks.workunit.client.1.smithi117.stdout:3/974: mknod d9/d1a/d39/d4c/d104/c149 0 2024-03-22T06:15:33.295 INFO:tasks.workunit.client.1.smithi117.stdout:2/967: creat d2/d9a/f158 x:0 0 0 2024-03-22T06:15:33.297 INFO:tasks.workunit.client.1.smithi117.stdout:4/941: dwrite d0/f42 [0,4194304] 0 2024-03-22T06:15:33.299 INFO:tasks.workunit.client.1.smithi117.stdout:1/999: getdents d1/d3c/d4d/de8 0 2024-03-22T06:15:33.304 INFO:tasks.workunit.client.1.smithi117.stdout:7/919: rmdir d8/d9/da/d2b/dc4/dac/ddc 39 2024-03-22T06:15:33.304 INFO:tasks.workunit.client.1.smithi117.stdout:8/974: stat d1/d28/d6b/df4/c112 0 2024-03-22T06:15:33.307 INFO:tasks.workunit.client.1.smithi117.stdout:3/975: truncate d9/d1a/fc6 4168554 0 2024-03-22T06:15:33.310 INFO:tasks.workunit.client.1.smithi117.stdout:7/920: dread d8/d9/dd1/fa0 [0,4194304] 0 2024-03-22T06:15:33.314 INFO:tasks.workunit.client.1.smithi117.stdout:2/968: rename d2/d5/d13/d57/dd0/d1d/d3a/fb8 to d2/d9a/f159 0 2024-03-22T06:15:33.314 INFO:tasks.workunit.client.1.smithi117.stdout:2/969: write d2/d5/db/d2c/d7d/dff/d39/d45/d7c/f131 [600780,9738] 0 2024-03-22T06:15:33.322 INFO:tasks.workunit.client.1.smithi117.stdout:4/942: mkdir d0/d4/d2a/d2f/d141 0 2024-03-22T06:15:33.323 INFO:tasks.workunit.client.1.smithi117.stdout:4/943: read d0/d19/d34/f77 [340985,42241] 0 2024-03-22T06:15:33.325 INFO:tasks.workunit.client.1.smithi117.stdout:8/975: creat d1/d3a/d44/d6a/dfd/d136/f13e x:0 0 0 2024-03-22T06:15:33.325 INFO:tasks.workunit.client.1.smithi117.stdout:8/976: stat d1/d19/f2d 0 2024-03-22T06:15:33.325 INFO:tasks.workunit.client.1.smithi117.stdout:8/977: write d1/d3a/d44/f8d [583871,48077] 0 2024-03-22T06:15:33.325 INFO:tasks.workunit.client.1.smithi117.stdout:8/978: chown d1/d3a/d44/d16/c21 0 1 2024-03-22T06:15:33.325 INFO:tasks.workunit.client.1.smithi117.stdout:8/979: creat d1/d28/d66/d7f/d9d/f13f x:0 0 0 2024-03-22T06:15:33.325 INFO:tasks.workunit.client.1.smithi117.stdout:8/980: truncate d1/d3a/d44/d6a/dfd/d136/f13e 895954 0 2024-03-22T06:15:33.325 INFO:tasks.workunit.client.1.smithi117.stdout:3/976: mkdir d9/d1a/d39/d8a/da1/db6/d14a 0 2024-03-22T06:15:33.332 INFO:tasks.workunit.client.1.smithi117.stdout:2/970: creat d2/d5/db/d2c/d7d/dff/d39/d45/f15a x:0 0 0 2024-03-22T06:15:33.333 INFO:tasks.workunit.client.1.smithi117.stdout:4/944: rename d0/d19/d28/d47/fbb to d0/d48/da0/dd2/db6/dde/d108/d6f/de5/d113/f142 0 2024-03-22T06:15:33.333 INFO:tasks.workunit.client.1.smithi117.stdout:4/945: creat d0/dd1/f143 x:0 0 0 2024-03-22T06:15:33.337 INFO:tasks.workunit.client.1.smithi117.stdout:8/981: unlink d1/d3a/d44/d11/d97/ldf 0 2024-03-22T06:15:33.337 INFO:tasks.workunit.client.1.smithi117.stdout:3/977: creat d9/d1a/d39/d4c/d113/def/f14b x:0 0 0 2024-03-22T06:15:33.338 INFO:tasks.workunit.client.1.smithi117.stdout:7/921: mknod d8/d9/da/d2b/dc4/c119 0 2024-03-22T06:15:33.341 INFO:tasks.workunit.client.1.smithi117.stdout:0/993: rmdir d1/d3/d2b/d3e/de/d1f/ddd 0 2024-03-22T06:15:33.341 INFO:tasks.workunit.client.1.smithi117.stdout:0/994: truncate d1/d10/f11b 258103 0 2024-03-22T06:15:33.346 INFO:tasks.workunit.client.1.smithi117.stdout:4/946: truncate d0/d48/da0/dd2/db6/dde/ff0 3548587 0 2024-03-22T06:15:33.347 INFO:tasks.workunit.client.1.smithi117.stdout:7/922: write d8/d9/da/d10/d23/f116 [1633181,28226] 0 2024-03-22T06:15:33.347 INFO:tasks.workunit.client.1.smithi117.stdout:7/923: chown d8/d9/da/d2b/dc4/dac 4 1 2024-03-22T06:15:33.348 INFO:tasks.workunit.client.1.smithi117.stdout:8/982: rename d1/d19/d98/fd8 to d1/d28/d66/d7f/d9d/f140 0 2024-03-22T06:15:33.348 INFO:tasks.workunit.client.1.smithi117.stdout:8/983: readlink d1/d3a/d44/d16/l50 0 2024-03-22T06:15:33.351 INFO:tasks.workunit.client.1.smithi117.stdout:7/924: symlink d8/d9/da/d2b/d9b/da5/l11a 0 2024-03-22T06:15:33.354 INFO:tasks.workunit.client.1.smithi117.stdout:7/925: truncate d8/d9/da/d10/f25 2430132 0 2024-03-22T06:15:33.355 INFO:tasks.workunit.client.1.smithi117.stdout:7/926: read d8/d9/d37/d45/d88/db1/fd8 [903966,68035] 0 2024-03-22T06:15:33.357 INFO:tasks.workunit.client.1.smithi117.stdout:7/927: getdents d8/d9/da/d10/d23/d48/d91 0 2024-03-22T06:15:33.357 INFO:tasks.workunit.client.1.smithi117.stdout:7/928: readlink d8/d9/da/d2b/d9b/da5/l11a 0 2024-03-22T06:15:33.359 INFO:tasks.workunit.client.1.smithi117.stdout:4/947: dread d0/d4/d2a/d2f/fec [0,4194304] 0 2024-03-22T06:15:33.360 INFO:tasks.workunit.client.1.smithi117.stdout:4/948: creat d0/dc0/f144 x:0 0 0 2024-03-22T06:15:33.360 INFO:tasks.workunit.client.1.smithi117.stdout:4/949: write d0/d48/da0/dd2/db6/dde/d108/d8d/fed [649662,20019] 0 2024-03-22T06:15:33.360 INFO:tasks.workunit.client.1.smithi117.stdout:4/950: dread - d0/d19/f110 zero size 2024-03-22T06:15:33.360 INFO:tasks.workunit.client.1.smithi117.stdout:7/929: mknod d8/d9/da/d10/c11b 0 2024-03-22T06:15:33.360 INFO:tasks.workunit.client.1.smithi117.stdout:7/930: readlink d8/d9/da/d10/d23/d27/d56/lee 0 2024-03-22T06:15:33.361 INFO:tasks.workunit.client.1.smithi117.stdout:4/951: mknod d0/d19/d9b/c145 0 2024-03-22T06:15:33.362 INFO:tasks.workunit.client.1.smithi117.stdout:7/931: creat d8/d9/da/d2b/d9b/da5/f11c x:0 0 0 2024-03-22T06:15:33.362 INFO:tasks.workunit.client.1.smithi117.stdout:7/932: chown d8/d9/da/d10/d23/d27/d56/cf8 3 1 2024-03-22T06:15:33.363 INFO:tasks.workunit.client.1.smithi117.stdout:4/952: mknod d0/d4/d2a/d2f/d141/c146 0 2024-03-22T06:15:33.363 INFO:tasks.workunit.client.1.smithi117.stdout:4/953: write d0/d48/da0/dd2/db6/fbd [593636,97991] 0 2024-03-22T06:15:33.363 INFO:tasks.workunit.client.1.smithi117.stdout:4/954: truncate d0/d4/d2a/f9e 1788627 0 2024-03-22T06:15:33.363 INFO:tasks.workunit.client.1.smithi117.stdout:7/933: mkdir d8/d9/d37/d45/d102/d11d 0 2024-03-22T06:15:33.363 INFO:tasks.workunit.client.1.smithi117.stdout:4/955: creat d0/d48/da0/dd2/db6/dde/d108/d6f/de5/f147 x:0 0 0 2024-03-22T06:15:33.367 INFO:tasks.workunit.client.1.smithi117.stdout:3/978: dwrite d9/d1a/d39/d4c/d37/dd5/fe5 [0,4194304] 0 2024-03-22T06:15:33.369 INFO:tasks.workunit.client.1.smithi117.stdout:2/971: dwrite d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/d96/fd9 [0,4194304] 0 2024-03-22T06:15:33.369 INFO:tasks.workunit.client.1.smithi117.stdout:2/972: chown d2/d5/db/lc4 3 1 2024-03-22T06:15:33.369 INFO:tasks.workunit.client.1.smithi117.stdout:2/973: chown d2/d5/db/d2c/d7d/dff/d39/f4b 73290 1 2024-03-22T06:15:33.372 INFO:tasks.workunit.client.1.smithi117.stdout:2/974: link d2/d5/d13/d57/dd0/d1d/d3a/d71/d76/fb0 d2/d5/d13/f15b 0 2024-03-22T06:15:33.372 INFO:tasks.workunit.client.1.smithi117.stdout:3/979: mkdir d9/d1a/d39/d4c/d36/d41/d14c 0 2024-03-22T06:15:33.373 INFO:tasks.workunit.client.1.smithi117.stdout:0/995: dwrite d1/d3/d2b/d3e/de/d1f/d3b/fa6 [4194304,4194304] 0 2024-03-22T06:15:33.373 INFO:tasks.workunit.client.1.smithi117.stdout:8/984: dwrite d1/d3a/d44/d11/dc7/f101 [0,4194304] 0 2024-03-22T06:15:33.373 INFO:tasks.workunit.client.1.smithi117.stdout:8/985: write d1/d3a/d44/d6a/fba [5074892,78557] 0 2024-03-22T06:15:33.379 INFO:tasks.workunit.client.1.smithi117.stdout:4/956: dread d0/d51/d92/f112 [0,4194304] 0 2024-03-22T06:15:33.379 INFO:tasks.workunit.client.1.smithi117.stdout:4/957: creat d0/d48/f148 x:0 0 0 2024-03-22T06:15:33.382 INFO:tasks.workunit.client.1.smithi117.stdout:0/996: dread d1/d3/d2b/d3e/de/f23 [0,4194304] 0 2024-03-22T06:15:33.382 INFO:tasks.workunit.client.1.smithi117.stdout:7/934: dwrite d8/d50/d66/fe9 [0,4194304] 0 2024-03-22T06:15:33.386 INFO:tasks.workunit.client.1.smithi117.stdout:3/980: dread d9/d1a/d39/f7f [0,4194304] 0 2024-03-22T06:15:33.388 INFO:tasks.workunit.client.1.smithi117.stdout:0/997: creat d1/d10/d18/d26/db6/f15b x:0 0 0 2024-03-22T06:15:33.388 INFO:tasks.workunit.client.1.smithi117.stdout:0/998: readlink d1/d3/d2b/d3e/de/d1f/d59/le4 0 2024-03-22T06:15:33.389 INFO:tasks.workunit.client.1.smithi117.stdout:2/975: dread d2/d5/d13/d57/dd0/f6b [0,4194304] 0 2024-03-22T06:15:33.389 INFO:tasks.workunit.client.1.smithi117.stdout:2/976: creat d2/d5/db/d2c/d7d/dff/d39/d45/ddd/f15c x:0 0 0 2024-03-22T06:15:33.390 INFO:tasks.workunit.client.1.smithi117.stdout:8/986: dwrite d1/d3a/d44/d4b/dc9/f13a [0,4194304] 0 2024-03-22T06:15:33.391 INFO:tasks.workunit.client.1.smithi117.stdout:7/935: unlink d8/d9/da/d10/d23/d48/d91/lad 0 2024-03-22T06:15:33.391 INFO:tasks.workunit.client.1.smithi117.stdout:7/936: fsync d8/d9/da/d2b/f6e 0 2024-03-22T06:15:33.392 INFO:tasks.workunit.client.1.smithi117.stdout:7/937: readlink d8/d9/d37/d45/d88/l98 0 2024-03-22T06:15:33.392 INFO:tasks.workunit.client.1.smithi117.stdout:7/938: fdatasync d8/d9/da/d2b/dc4/f83 0 2024-03-22T06:15:33.393 INFO:tasks.workunit.client.1.smithi117.stdout:3/981: creat d9/d1a/d39/d7d/d97/dc9/f14d x:0 0 0 2024-03-22T06:15:33.395 INFO:tasks.workunit.client.1.smithi117.stdout:2/977: mknod d2/d9a/dd3/c15d 0 2024-03-22T06:15:33.395 INFO:tasks.workunit.client.1.smithi117.stdout:2/978: fsync d2/d5/d13/d57/dd0/f6b 0 2024-03-22T06:15:33.400 INFO:tasks.workunit.client.1.smithi117.stdout:7/939: creat d8/d9/da/d2b/dc4/dac/dfa/f11e x:0 0 0 2024-03-22T06:15:33.401 INFO:tasks.workunit.client.1.smithi117.stdout:4/958: dwrite d0/f83 [0,4194304] 0 2024-03-22T06:15:33.402 INFO:tasks.workunit.client.1.smithi117.stdout:0/999: dread d1/d10/d18/d26/d53/f93 [0,4194304] 0 2024-03-22T06:15:33.404 INFO:tasks.workunit.client.1.smithi117.stdout:2/979: mkdir d2/d9a/d12a/d15e 0 2024-03-22T06:15:33.405 INFO:tasks.workunit.client.1.smithi117.stdout:2/980: dread - d2/d5/db/d2c/d7d/fb2 zero size 2024-03-22T06:15:33.405 INFO:tasks.workunit.client.1.smithi117.stdout:2/981: chown d2/d5/db/d2c/ldc 6 1 2024-03-22T06:15:33.406 INFO:tasks.workunit.client.1.smithi117.stdout:7/940: creat d8/d9/da/d10/d23/d48/d91/f11f x:0 0 0 2024-03-22T06:15:33.407 INFO:tasks.workunit.client.1.smithi117.stdout:4/959: mknod d0/d4/c149 0 2024-03-22T06:15:33.408 INFO:tasks.workunit.client.1.smithi117.stdout:4/960: write d0/d48/da0/dd2/db6/dde/d108/d6f/f91 [2315132,6282] 0 2024-03-22T06:15:33.410 INFO:tasks.workunit.client.1.smithi117.stdout:3/982: rename d9/d1a/d39/d4c/d104/f108 to d9/d1a/d39/d4c/d36/d41/f14e 0 2024-03-22T06:15:33.411 INFO:tasks.workunit.client.1.smithi117.stdout:3/983: fdatasync d9/d1a/d39/dc7/f115 0 2024-03-22T06:15:33.411 INFO:tasks.workunit.client.1.smithi117.stdout:3/984: write d9/d1a/d3a/f7a [4684104,41170] 0 2024-03-22T06:15:33.415 INFO:tasks.workunit.client.1.smithi117.stdout:8/987: dwrite d1/d3a/d44/f3c [0,4194304] 0 2024-03-22T06:15:33.415 INFO:tasks.workunit.client.1.smithi117.stdout:8/988: creat d1/d28/d6b/df4/f141 x:0 0 0 2024-03-22T06:15:33.416 INFO:tasks.workunit.client.1.smithi117.stdout:2/982: creat d2/d5/d13/d57/dd0/d1d/d3a/d89/d8d/dd4/d11c/f15f x:0 0 0 2024-03-22T06:15:33.419 INFO:tasks.workunit.client.1.smithi117.stdout:7/941: truncate d8/d9/da/d2b/dc4/f5d 220091 0 2024-03-22T06:15:33.422 INFO:tasks.workunit.client.1.smithi117.stdout:4/961: creat d0/d51/d100/f14a x:0 0 0 2024-03-22T06:15:33.425 INFO:tasks.workunit.client.1.smithi117.stdout:8/989: rename d1/d28/f7d to d1/d28/d66/f142 0 2024-03-22T06:15:33.425 INFO:tasks.workunit.client.1.smithi117.stdout:8/990: creat d1/d3a/d44/d9/f143 x:0 0 0 2024-03-22T06:15:33.426 INFO:tasks.workunit.client.1.smithi117.stdout:8/991: write d1/d3a/d44/d4b/dc9/fd7 [428017,21342] 0 2024-03-22T06:15:33.427 INFO:tasks.workunit.client.1.smithi117.stdout:7/942: dread f7 [0,4194304] 0 2024-03-22T06:15:33.427 INFO:tasks.workunit.client.1.smithi117.stdout:7/943: chown d8/d9/db0/lc9 1991 1 2024-03-22T06:15:33.430 INFO:tasks.workunit.client.1.smithi117.stdout:2/983: mknod d2/d9a/d12a/c160 0 2024-03-22T06:15:33.455 INFO:tasks.workunit.client.1.smithi117.stdout:4/962: truncate d0/d19/f22 328695 0 2024-03-22T06:15:33.456 INFO:tasks.workunit.client.1.smithi117.stdout:8/992: mknod d1/d3a/d44/d16/d108/c144 0 2024-03-22T06:15:33.457 INFO:tasks.workunit.client.1.smithi117.stdout:7/944: creat d8/d9/d112/f120 x:0 0 0 2024-03-22T06:15:33.458 INFO:tasks.workunit.client.1.smithi117.stdout:7/945: creat d8/d9/f121 x:0 0 0 2024-03-22T06:15:33.458 INFO:tasks.workunit.client.1.smithi117.stdout:7/946: truncate d8/d9/da/d2b/dc4/f7b 5366493 0 2024-03-22T06:15:33.458 INFO:tasks.workunit.client.1.smithi117.stdout:7/947: creat d8/f122 x:0 0 0 2024-03-22T06:15:33.458 INFO:tasks.workunit.client.1.smithi117.stdout:2/984: symlink d2/d5/db/d2c/d7d/dff/d39/d45/l161 0 2024-03-22T06:15:33.458 INFO:tasks.workunit.client.1.smithi117.stdout:2/985: write d2/d5/d13/d57/dd0/f97 [757213,15760] 0 2024-03-22T06:15:33.460 INFO:tasks.workunit.client.1.smithi117.stdout:4/963: rename d0/d48/da0/dd2/db6/dde/d108/d6f/d6a to d0/d19/dcc/d14b 0 2024-03-22T06:15:33.464 INFO:tasks.workunit.client.1.smithi117.stdout:7/948: truncate d8/d9/d37/d45/d88/db1/dc6/d10e/dce/f101 500731 0 2024-03-22T06:15:33.466 INFO:tasks.workunit.client.1.smithi117.stdout:2/986: symlink d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/l162 0 2024-03-22T06:15:33.468 INFO:tasks.workunit.client.1.smithi117.stdout:7/949: write d8/d9/da/d10/d23/f38 [4000763,98961] 0 2024-03-22T06:15:33.472 INFO:tasks.workunit.client.1.smithi117.stdout:2/987: creat d2/d5/d13/d57/dd0/d1d/d3a/d149/f163 x:0 0 0 2024-03-22T06:15:33.472 INFO:tasks.workunit.client.1.smithi117.stdout:7/950: mknod d8/d50/d5e/c123 0 2024-03-22T06:15:33.473 INFO:tasks.workunit.client.1.smithi117.stdout:7/951: write d8/d9/d37/d45/d61/f111 [40797,130078] 0 2024-03-22T06:15:33.473 INFO:tasks.workunit.client.1.smithi117.stdout:7/952: fsync d8/d9/da/d2b/dc4/f4a 0 2024-03-22T06:15:33.473 INFO:tasks.workunit.client.1.smithi117.stdout:7/953: fdatasync d8/d9/da/d10/d23/fbb 0 2024-03-22T06:15:33.478 INFO:tasks.workunit.client.1.smithi117.stdout:2/988: dread d2/d35/d38/f104 [0,4194304] 0 2024-03-22T06:15:33.478 INFO:tasks.workunit.client.1.smithi117.stdout:2/989: truncate d2/d5/d13/d57/dd0/d1d/d3a/d89/fab 1151295 0 2024-03-22T06:15:33.485 INFO:tasks.workunit.client.1.smithi117.stdout:8/993: dwrite d1/d3a/d44/d6a/dfd/d136/f13e [0,4194304] 0 2024-03-22T06:15:33.489 INFO:tasks.workunit.client.1.smithi117.stdout:8/994: mkdir d1/d3a/d44/d6a/d129/d145 0 2024-03-22T06:15:33.497 INFO:tasks.workunit.client.1.smithi117.stdout:4/964: dwrite d0/d51/d92/f102 [0,4194304] 0 2024-03-22T06:15:33.497 INFO:tasks.workunit.client.1.smithi117.stdout:7/954: dwrite d8/d9/d37/f4e [0,4194304] 0 2024-03-22T06:15:33.502 INFO:tasks.workunit.client.1.smithi117.stdout:8/995: read d1/d28/fa8 [3039232,83513] 0 2024-03-22T06:15:33.502 INFO:tasks.workunit.client.1.smithi117.stdout:8/996: chown d1/d3a/d44/d4b/df7 40 1 2024-03-22T06:15:33.502 INFO:tasks.workunit.client.1.smithi117.stdout:2/990: dwrite d2/d35/d4c/f9f [0,4194304] 0 2024-03-22T06:15:33.502 INFO:tasks.workunit.client.1.smithi117.stdout:2/991: readlink d2/d5/d13/d57/dd0/d1d/d3a/d89/daa/d4f/l6a 0 2024-03-22T06:15:33.502 INFO:tasks.workunit.client.1.smithi117.stdout:2/992: creat d2/d5/db/d2c/d7d/dff/d39/d45/f164 x:0 0 0 2024-03-22T06:15:33.508 INFO:tasks.workunit.client.1.smithi117.stdout:4/965: dread d0/d48/da0/dd2/db6/dde/d108/f74 [0,4194304] 0 2024-03-22T06:15:33.510 INFO:tasks.workunit.client.1.smithi117.stdout:7/955: read d8/d50/d66/fe9 [383428,63546] 0 2024-03-22T06:15:33.523 INFO:tasks.workunit.client.1.smithi117.stdout:7/956: rename d8/d9/da/d10/l15 to d8/d9/da/d2b/dc4/d26/l124 0 2024-03-22T06:15:33.523 INFO:tasks.workunit.client.1.smithi117.stdout:7/957: fdatasync d8/d9/da/d2b/dc4/d26/f34 0 2024-03-22T06:15:33.523 INFO:tasks.workunit.client.1.smithi117.stdout:7/958: creat d8/d9/f125 x:0 0 0 2024-03-22T06:15:33.526 INFO:tasks.workunit.client.1.smithi117.stdout:2/993: dwrite d2/d5/d13/d57/dd0/d1d/d3a/d89/fab [0,4194304] 0 2024-03-22T06:15:33.526 INFO:tasks.workunit.client.1.smithi117.stdout:2/994: chown d2/d35/d38/f104 2668750 1 2024-03-22T06:15:33.529 INFO:tasks.workunit.client.1.smithi117.stdout:7/959: dread d8/d9/d37/d45/d88/fbf [0,4194304] 0 2024-03-22T06:15:33.531 INFO:tasks.workunit.client.1.smithi117.stdout:2/995: dread d2/d35/d38/f5e [4194304,4194304] 0 2024-03-22T06:15:33.531 INFO:tasks.workunit.client.1.smithi117.stdout:8/997: dwrite d1/d3a/d44/d16/d108/f131 [0,4194304] 0 2024-03-22T06:15:33.531 INFO:tasks.workunit.client.1.smithi117.stdout:2/996: chown d2/c53 5796640 1 2024-03-22T06:15:33.531 INFO:tasks.workunit.client.1.smithi117.stdout:4/966: dwrite d0/f27 [0,4194304] 0 2024-03-22T06:15:33.532 INFO:tasks.workunit.client.1.smithi117.stdout:8/998: dread d1/d3a/fcb [0,4194304] 0 2024-03-22T06:15:33.534 INFO:tasks.workunit.client.1.smithi117.stdout:2/997: creat d2/f165 x:0 0 0 2024-03-22T06:15:33.534 INFO:tasks.workunit.client.1.smithi117.stdout:2/998: dread - d2/d5/d13/fe1 zero size 2024-03-22T06:15:33.535 INFO:tasks.workunit.client.1.smithi117.stdout:4/967: symlink d0/d4/d115/l14c 0 2024-03-22T06:15:33.537 INFO:tasks.workunit.client.1.smithi117.stdout:8/999: write d1/d3a/d44/d4b/df7/f12b [5092368,112142] 0 2024-03-22T06:15:33.538 INFO:tasks.workunit.client.1.smithi117.stdout:2/999: mknod d2/d5/d13/d57/dd0/d1d/d3a/da6/c166 0 2024-03-22T06:15:33.540 INFO:tasks.workunit.client.1.smithi117.stdout:3/985: sync 2024-03-22T06:15:33.541 INFO:tasks.workunit.client.1.smithi117.stdout:4/968: getdents d0/d19/d34/d43 0 2024-03-22T06:15:33.541 INFO:tasks.workunit.client.1.smithi117.stdout:4/969: write d0/dd1/f143 [587065,87765] 0 2024-03-22T06:15:33.545 INFO:tasks.workunit.client.1.smithi117.stdout:3/986: rename d9/d1a/d39/d4c/l77 to d9/d1a/dc1/d102/l14f 0 2024-03-22T06:15:33.545 INFO:tasks.workunit.client.1.smithi117.stdout:3/987: truncate d9/d1a/d39/d4c/d36/f43 4701687 0 2024-03-22T06:15:33.545 INFO:tasks.workunit.client.1.smithi117.stdout:3/988: write d9/d66/f74 [459472,97635] 0 2024-03-22T06:15:33.547 INFO:tasks.workunit.client.1.smithi117.stdout:4/970: mkdir d0/d4/d115/d14d 0 2024-03-22T06:15:33.560 INFO:tasks.workunit.client.1.smithi117.stdout:4/971: dwrite d0/d4/d2a/d5b/f97 [0,4194304] 0 2024-03-22T06:15:33.564 INFO:tasks.workunit.client.1.smithi117.stdout:4/972: creat d0/d48/da0/dd2/d121/f14e x:0 0 0 2024-03-22T06:15:33.564 INFO:tasks.workunit.client.1.smithi117.stdout:4/973: fsync d0/d4/d2a/d2f/fec 0 2024-03-22T06:15:33.564 INFO:tasks.workunit.client.1.smithi117.stdout:4/974: fdatasync d0/d4/d2a/ff2 0 2024-03-22T06:15:33.564 INFO:tasks.workunit.client.1.smithi117.stdout:4/975: fsync d0/d4/d2a/f9e 0 2024-03-22T06:15:33.564 INFO:tasks.workunit.client.1.smithi117.stdout:4/976: creat d0/d19/d9b/f14f x:0 0 0 2024-03-22T06:15:33.568 INFO:tasks.workunit.client.1.smithi117.stdout:4/977: creat d0/d19/d34/d43/f150 x:0 0 0 2024-03-22T06:15:33.569 INFO:tasks.workunit.client.1.smithi117.stdout:4/978: mknod d0/d51/da4/c151 0 2024-03-22T06:15:33.571 INFO:tasks.workunit.client.1.smithi117.stdout:4/979: creat d0/d19/d34/d43/d127/f152 x:0 0 0 2024-03-22T06:15:33.572 INFO:tasks.workunit.client.1.smithi117.stdout:4/980: read d0/d51/d92/f103 [395269,22334] 0 2024-03-22T06:15:33.573 INFO:tasks.workunit.client.1.smithi117.stdout:4/981: rename d0/d48/da0/dd2/lb2 to d0/d19/dcc/l153 0 2024-03-22T06:15:33.575 INFO:tasks.workunit.client.1.smithi117.stdout:4/982: creat d0/d4/d115/d14d/f154 x:0 0 0 2024-03-22T06:15:33.577 INFO:tasks.workunit.client.1.smithi117.stdout:4/983: mknod d0/d19/c155 0 2024-03-22T06:15:33.578 INFO:tasks.workunit.client.1.smithi117.stdout:4/984: creat d0/d48/da0/dd2/db6/dde/d108/d6f/d11b/f156 x:0 0 0 2024-03-22T06:15:33.580 INFO:tasks.workunit.client.1.smithi117.stdout:4/985: dread d0/d48/fdd [0,4194304] 0 2024-03-22T06:15:33.581 INFO:tasks.workunit.client.1.smithi117.stdout:4/986: symlink d0/d48/da0/dd2/db6/dde/d108/d6f/l157 0 2024-03-22T06:15:33.581 INFO:tasks.workunit.client.1.smithi117.stdout:4/987: write d0/d19/dcc/fe1 [48774,68210] 0 2024-03-22T06:15:33.595 INFO:tasks.workunit.client.1.smithi117.stdout:4/988: dwrite d0/d48/da0/dd2/d121/fd5 [0,4194304] 0 2024-03-22T06:15:33.595 INFO:tasks.workunit.client.1.smithi117.stdout:4/989: read - d0/d48/da0/dd2/d121/de8/f134 zero size 2024-03-22T06:15:33.600 INFO:tasks.workunit.client.1.smithi117.stdout:4/990: dread d0/d19/f49 [0,4194304] 0 2024-03-22T06:15:33.600 INFO:tasks.workunit.client.1.smithi117.stdout:4/991: truncate d0/dc0/f12a 503315 0 2024-03-22T06:15:33.601 INFO:tasks.workunit.client.1.smithi117.stdout:4/992: link d0/d48/da0/dd2/db6/fbd d0/d48/da0/dd2/db6/f158 0 2024-03-22T06:15:33.603 INFO:tasks.workunit.client.1.smithi117.stdout:4/993: getdents d0/d19/d34 0 2024-03-22T06:15:33.604 INFO:tasks.workunit.client.1.smithi117.stdout:4/994: mkdir d0/d48/da0/dd2/db6/d159 0 2024-03-22T06:15:33.620 INFO:tasks.workunit.client.1.smithi117.stdout:4/995: dwrite d0/d4/d2a/f30 [0,4194304] 0 2024-03-22T06:15:33.623 INFO:tasks.workunit.client.1.smithi117.stdout:4/996: rename d0/d48/fee to d0/d48/da0/dd2/db6/dde/d108/d6f/d11b/f15a 0 2024-03-22T06:15:33.624 INFO:tasks.workunit.client.1.smithi117.stdout:4/997: write d0/d19/f49 [1640253,83185] 0 2024-03-22T06:15:33.624 INFO:tasks.workunit.client.1.smithi117.stdout:4/998: write d0/d19/f49 [458344,75087] 0 2024-03-22T06:15:33.625 INFO:tasks.workunit.client.1.smithi117.stdout:4/999: mknod d0/d48/da0/dd2/db6/d159/c15b 0 2024-03-22T06:15:33.635 INFO:tasks.workunit.client.1.smithi117.stdout:7/960: sync 2024-03-22T06:15:33.635 INFO:tasks.workunit.client.1.smithi117.stdout:7/961: rmdir d8/d9/da/d10/d23/d27 39 2024-03-22T06:15:33.638 INFO:tasks.workunit.client.1.smithi117.stdout:7/962: unlink d8/d9/da/d10/d23/f116 0 2024-03-22T06:15:33.638 INFO:tasks.workunit.client.1.smithi117.stdout:7/963: chown d8/d9/dd1/fc5 13067 1 2024-03-22T06:15:33.644 INFO:tasks.workunit.client.1.smithi117.stdout:7/964: dread d8/d9/da/f72 [0,4194304] 0 2024-03-22T06:15:33.650 INFO:tasks.workunit.client.1.smithi117.stdout:7/965: unlink d8/d9/da/f4b 0 2024-03-22T06:15:33.653 INFO:tasks.workunit.client.1.smithi117.stdout:7/966: mknod d8/d6c/df7/c126 0 2024-03-22T06:15:33.654 INFO:tasks.workunit.client.1.smithi117.stdout:3/989: sync 2024-03-22T06:15:33.655 INFO:tasks.workunit.client.1.smithi117.stdout:7/967: dread d8/d9/da/d2b/dc4/d26/f34 [0,4194304] 0 2024-03-22T06:15:33.657 INFO:tasks.workunit.client.1.smithi117.stdout:7/968: unlink d8/d9/da/d2b/dc4/d26/l80 0 2024-03-22T06:15:33.657 INFO:tasks.workunit.client.1.smithi117.stdout:7/969: creat d8/d9/da/f127 x:0 0 0 2024-03-22T06:15:33.657 INFO:tasks.workunit.client.1.smithi117.stdout:7/970: truncate d8/d9/da/d10/f8d 2126129 0 2024-03-22T06:15:33.660 INFO:tasks.workunit.client.1.smithi117.stdout:7/971: getdents d8/d9/da/d2b 0 2024-03-22T06:15:33.664 INFO:tasks.workunit.client.1.smithi117.stdout:7/972: fdatasync d8/d9/f7c 0 2024-03-22T06:15:33.664 INFO:tasks.workunit.client.1.smithi117.stdout:7/973: write d8/ff5 [1816093,76802] 0 2024-03-22T06:15:33.666 INFO:tasks.workunit.client.1.smithi117.stdout:7/974: mknod d8/d6c/df7/c128 0 2024-03-22T06:15:33.671 INFO:tasks.workunit.client.1.smithi117.stdout:7/975: creat d8/d9/dd1/f129 x:0 0 0 2024-03-22T06:15:33.673 INFO:tasks.workunit.client.1.smithi117.stdout:7/976: creat d8/d9/d37/d45/d88/db1/dc6/f12a x:0 0 0 2024-03-22T06:15:33.673 INFO:tasks.workunit.client.1.smithi117.stdout:7/977: write d8/d9/d37/d45/d88/db1/fd8 [1575186,1983] 0 2024-03-22T06:15:33.686 INFO:tasks.workunit.client.1.smithi117.stdout:7/978: dwrite d8/d9/da/d2b/db8/fe7 [0,4194304] 0 2024-03-22T06:15:33.686 INFO:tasks.workunit.client.1.smithi117.stdout:7/979: write d8/d9/da/d10/d23/d48/d91/f11f [493364,58402] 0 2024-03-22T06:15:33.699 INFO:tasks.workunit.client.1.smithi117.stdout:7/980: dwrite d8/fb [8388608,4194304] 0 2024-03-22T06:15:33.699 INFO:tasks.workunit.client.1.smithi117.stdout:7/981: write d8/d9/dd1/f129 [216509,109874] 0 2024-03-22T06:15:33.704 INFO:tasks.workunit.client.1.smithi117.stdout:7/982: rename d8/d9/da/d2b/d9b/da5 to d8/d9/dd1/d12b 0 2024-03-22T06:15:33.709 INFO:tasks.workunit.client.1.smithi117.stdout:7/983: rmdir d8/d9/db0/ddb 39 2024-03-22T06:15:33.709 INFO:tasks.workunit.client.1.smithi117.stdout:7/984: chown d8/d6c/fc2 47758726 1 2024-03-22T06:15:33.709 INFO:tasks.workunit.client.1.smithi117.stdout:7/985: fsync d8/d9/da/d2b/dc4/f5d 0 2024-03-22T06:15:33.709 INFO:tasks.workunit.client.1.smithi117.stdout:7/986: stat d8/d9/da/d2b/f6e 0 2024-03-22T06:15:33.710 INFO:tasks.workunit.client.1.smithi117.stdout:7/987: creat d8/d9/da/d2b/dc4/d26/f12c x:0 0 0 2024-03-22T06:15:33.712 INFO:tasks.workunit.client.1.smithi117.stdout:7/988: rename d8/d9/d37/d45/d61/ff0 to d8/d9/da/d10/d23/d27/d56/f12d 0 2024-03-22T06:15:33.729 INFO:tasks.workunit.client.1.smithi117.stdout:7/989: dwrite d8/d6c/f106 [0,4194304] 0 2024-03-22T06:15:33.730 INFO:tasks.workunit.client.1.smithi117.stdout:7/990: chown d8/d9/da/df1 11 1 2024-03-22T06:15:33.738 INFO:tasks.workunit.client.1.smithi117.stdout:3/990: sync 2024-03-22T06:15:33.741 INFO:tasks.workunit.client.1.smithi117.stdout:3/991: rename d9/d1a/d3a/f136 to d9/d1a/d39/d7d/d97/dc9/f150 0 2024-03-22T06:15:33.741 INFO:tasks.workunit.client.1.smithi117.stdout:3/992: chown d9/l27 857 1 2024-03-22T06:15:33.741 INFO:tasks.workunit.client.1.smithi117.stdout:3/993: dread - d9/d1a/d39/d4c/d113/f132 zero size 2024-03-22T06:15:33.741 INFO:tasks.workunit.client.1.smithi117.stdout:3/994: truncate d9/d1a/d39/d4c/d37/d69/fa8 8657097 0 2024-03-22T06:15:33.741 INFO:tasks.workunit.client.1.smithi117.stdout:3/995: write d9/d1a/d39/d4c/d36/d41/f14e [813484,20124] 0 2024-03-22T06:15:33.741 INFO:tasks.workunit.client.1.smithi117.stdout:3/996: chown d9/d1a/d3a/f100 29537 1 2024-03-22T06:15:33.741 INFO:tasks.workunit.client.1.smithi117.stdout:3/997: chown d9/d1a/d39/d8a 1 1 2024-03-22T06:15:33.750 INFO:tasks.workunit.client.1.smithi117.stdout:3/998: dread d9/d1a/d39/dc7/f115 [0,4194304] 0 2024-03-22T06:15:33.756 INFO:tasks.workunit.client.1.smithi117.stdout:3/999: write d9/d1a/d3a/fe1 [3699264,13264] 0 2024-03-22T06:15:33.813 INFO:tasks.workunit.client.1.smithi117.stdout:7/991: sync 2024-03-22T06:15:33.814 INFO:tasks.workunit.client.1.smithi117.stdout:7/992: creat d8/d9/d37/d45/f12e x:0 0 0 2024-03-22T06:15:33.814 INFO:tasks.workunit.client.1.smithi117.stdout:7/993: chown d8/d9/d37/c4f 235493 1 2024-03-22T06:15:33.814 INFO:tasks.workunit.client.1.smithi117.stdout:7/994: dread - d8/d50/d5e/fbe zero size 2024-03-22T06:15:33.815 INFO:tasks.workunit.client.1.smithi117.stdout:7/995: creat d8/d9/da/d2b/db8/d104/f12f x:0 0 0 2024-03-22T06:15:33.817 INFO:tasks.workunit.client.1.smithi117.stdout:7/996: symlink d8/d9/d37/d45/d88/db1/dc6/d10e/d82/l130 0 2024-03-22T06:15:33.824 INFO:tasks.workunit.client.1.smithi117.stdout:7/997: dread d8/d9/da/f69 [4194304,4194304] 0 2024-03-22T06:15:33.826 INFO:tasks.workunit.client.1.smithi117.stdout:7/998: symlink d8/d9/d37/d45/l131 0 2024-03-22T06:15:33.841 INFO:tasks.workunit.client.1.smithi117.stdout:7/999: dwrite d8/d9/ff2 [0,4194304] 0 2024-03-22T06:15:33.844 INFO:tasks.workunit.client.1.smithi117.stderr:+ rm -rf -- ./tmp.fgtQOVs9az 2024-03-22T06:15:33.997 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:33 smithi006 ceph-mon[29301]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:15:33.997 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:33 smithi006 ceph-mon[29301]: pgmap v30: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 191 MiB/s rd, 214 MiB/s wr, 356 op/s 2024-03-22T06:15:33.998 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:33 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/180945841' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:15:34.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:33 smithi117 ceph-mon[36514]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:15:34.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:33 smithi117 ceph-mon[36514]: pgmap v30: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 191 MiB/s rd, 214 MiB/s wr, 356 op/s 2024-03-22T06:15:34.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:33 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/180945841' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:15:36.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:35 smithi117 ceph-mon[36514]: pgmap v31: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 209 MiB/s rd, 238 MiB/s wr, 380 op/s 2024-03-22T06:15:36.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:35 smithi006 ceph-mon[29301]: pgmap v31: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 209 MiB/s rd, 238 MiB/s wr, 380 op/s 2024-03-22T06:15:38.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:37 smithi117 ceph-mon[36514]: pgmap v32: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 215 MiB/s rd, 249 MiB/s wr, 455 op/s 2024-03-22T06:15:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:37 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:37 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:37 smithi117 ceph-mon[36514]: Reconfiguring prometheus.smithi006 (dependencies changed)... 2024-03-22T06:15:38.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:37 smithi006 ceph-mon[29301]: pgmap v32: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 215 MiB/s rd, 249 MiB/s wr, 455 op/s 2024-03-22T06:15:38.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:37 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:38.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:37 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:38.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:37 smithi006 ceph-mon[29301]: Reconfiguring prometheus.smithi006 (dependencies changed)... 2024-03-22T06:15:39.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:38 smithi117 ceph-mon[36514]: Reconfiguring daemon prometheus.smithi006 on smithi006 2024-03-22T06:15:39.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:38 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:15:39.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:38 smithi006 ceph-mon[29301]: Reconfiguring daemon prometheus.smithi006 on smithi006 2024-03-22T06:15:39.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:38 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:15:40.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:39 smithi006 ceph-mon[29301]: pgmap v33: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 138 MiB/s rd, 154 MiB/s wr, 294 op/s 2024-03-22T06:15:40.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:39 smithi117 ceph-mon[36514]: pgmap v33: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 138 MiB/s rd, 154 MiB/s wr, 294 op/s 2024-03-22T06:15:42.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:41 smithi117 ceph-mon[36514]: pgmap v34: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 138 MiB/s rd, 155 MiB/s wr, 379 op/s 2024-03-22T06:15:42.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:41 smithi006 ceph-mon[29301]: pgmap v34: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 138 MiB/s rd, 155 MiB/s wr, 379 op/s 2024-03-22T06:15:44.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:43 smithi117 ceph-mon[36514]: pgmap v35: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 55 MiB/s rd, 70 MiB/s wr, 213 op/s 2024-03-22T06:15:44.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:43 smithi006 ceph-mon[29301]: pgmap v35: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 55 MiB/s rd, 70 MiB/s wr, 213 op/s 2024-03-22T06:15:45.412 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-22T06:15:45.412 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-22T06:15:46.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:45 smithi117 ceph-mon[36514]: pgmap v36: 65 pgs: 65 active+clean; 2.0 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 55 MiB/s rd, 70 MiB/s wr, 251 op/s 2024-03-22T06:15:46.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:45 smithi006 ceph-mon[29301]: pgmap v36: 65 pgs: 65 active+clean; 2.0 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 55 MiB/s rd, 70 MiB/s wr, 251 op/s 2024-03-22T06:15:48.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:47 smithi117 ceph-mon[36514]: pgmap v37: 65 pgs: 65 active+clean; 1.4 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 37 MiB/s rd, 46 MiB/s wr, 272 op/s 2024-03-22T06:15:48.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:47 smithi006 ceph-mon[29301]: pgmap v37: 65 pgs: 65 active+clean; 1.4 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 37 MiB/s rd, 46 MiB/s wr, 272 op/s 2024-03-22T06:15:50.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:49 smithi117 ceph-mon[36514]: pgmap v38: 65 pgs: 65 active+clean; 1.4 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-03-22T06:15:50.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:49 smithi006 ceph-mon[29301]: pgmap v38: 65 pgs: 65 active+clean; 1.4 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-03-22T06:15:52.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:52 smithi117 ceph-mon[36514]: pgmap v39: 65 pgs: 65 active+clean; 784 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 32 KiB/s rd, 2.0 MiB/s wr, 212 op/s 2024-03-22T06:15:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:52 smithi006 ceph-mon[29301]: pgmap v39: 65 pgs: 65 active+clean; 784 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 32 KiB/s rd, 2.0 MiB/s wr, 212 op/s 2024-03-22T06:15:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:54 smithi117 ceph-mon[36514]: pgmap v40: 65 pgs: 65 active+clean; 784 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 21 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-03-22T06:15:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:54 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:15:54.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:54 smithi006 ceph-mon[29301]: pgmap v40: 65 pgs: 65 active+clean; 784 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 21 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-03-22T06:15:54.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:54 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:15:55.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:55 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:55.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:55 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:55.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:55 smithi117 ceph-mon[36514]: Reconfiguring node-exporter.smithi117 (dependencies changed)... 2024-03-22T06:15:55.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:55 smithi117 ceph-mon[36514]: Reconfiguring daemon node-exporter.smithi117 on smithi117 2024-03-22T06:15:55.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:55 smithi117 ceph-mon[36514]: pgmap v41: 65 pgs: 65 active+clean; 763 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 21 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-03-22T06:15:55.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:55 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:55.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:55 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:55.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:55 smithi006 ceph-mon[29301]: Reconfiguring node-exporter.smithi117 (dependencies changed)... 2024-03-22T06:15:55.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:55 smithi006 ceph-mon[29301]: Reconfiguring daemon node-exporter.smithi117 on smithi117 2024-03-22T06:15:55.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:55 smithi006 ceph-mon[29301]: pgmap v41: 65 pgs: 65 active+clean; 763 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 21 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-03-22T06:15:57.331 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-22T06:15:57.331 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-22T06:15:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: pgmap v42: 65 pgs: 65 active+clean; 751 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 22 KiB/s rd, 2.0 MiB/s wr, 114 op/s 2024-03-22T06:15:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:15:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:15:58.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:15:58.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:15:58.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:15:58.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:15:58.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:15:58.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: Upgrade: Need to upgrade myself (mgr.smithi117.bqeskk) 2024-03-22T06:15:58.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:57 smithi006 ceph-mon[29301]: Upgrade: Need to upgrade myself (mgr.smithi117.bqeskk) 2024-03-22T06:15:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: pgmap v42: 65 pgs: 65 active+clean; 751 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 22 KiB/s rd, 2.0 MiB/s wr, 114 op/s 2024-03-22T06:15:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:15:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:15:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:15:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:15:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:15:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:15:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:15:58.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:15:58.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: Upgrade: Need to upgrade myself (mgr.smithi117.bqeskk) 2024-03-22T06:15:58.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:57 smithi117 ceph-mon[36514]: Upgrade: Need to upgrade myself (mgr.smithi117.bqeskk) 2024-03-22T06:16:00.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:15:59 smithi006 ceph-mon[29301]: pgmap v43: 65 pgs: 65 active+clean; 751 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 8.2 KiB/s rd, 1.7 MiB/s wr, 68 op/s 2024-03-22T06:16:00.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:15:59 smithi117 ceph-mon[36514]: pgmap v43: 65 pgs: 65 active+clean; 751 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 8.2 KiB/s rd, 1.7 MiB/s wr, 68 op/s 2024-03-22T06:16:02.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:01 smithi006 ceph-mon[29301]: pgmap v44: 65 pgs: 65 active+clean; 294 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-03-22T06:16:02.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:01 smithi117 ceph-mon[36514]: pgmap v44: 65 pgs: 65 active+clean; 294 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-03-22T06:16:03.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.256+0000 7f13d7850700 1 -- 172.21.15.6:0/3987842713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13d006f9d0 msgr2=0x7f13d006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:03.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.256+0000 7f13d7850700 1 --2- 172.21.15.6:0/3987842713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13d006f9d0 0x7f13d006e460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f13cc004940 tx=0x7f13cc009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:03.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.256+0000 7f13d7850700 1 -- 172.21.15.6:0/3987842713 shutdown_connections 2024-03-22T06:16:03.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.256+0000 7f13d7850700 1 --2- 172.21.15.6:0/3987842713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d006e9a0 0x7f13d006ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.256+0000 7f13d7850700 1 --2- 172.21.15.6:0/3987842713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13d006f9d0 0x7f13d006e460 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.256+0000 7f13d7850700 1 -- 172.21.15.6:0/3987842713 >> 172.21.15.6:0/3987842713 conn(0x7f13d0069de0 msgr2=0x7f13d006c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:03.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.257+0000 7f13d7850700 1 -- 172.21.15.6:0/3987842713 shutdown_connections 2024-03-22T06:16:03.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.257+0000 7f13d7850700 1 -- 172.21.15.6:0/3987842713 wait complete. 2024-03-22T06:16:03.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.258+0000 7f13d7850700 1 Processor -- start 2024-03-22T06:16:03.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.258+0000 7f13d7850700 1 -- start start 2024-03-22T06:16:03.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.258+0000 7f13d7850700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d006e9a0 0x7f13d0112020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:03.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.258+0000 7f13d7850700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13d006f9d0 0x7f13d0112560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:03.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.258+0000 7f13d7850700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d0113ac0 con 0x7f13d006f9d0 2024-03-22T06:16:03.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.258+0000 7f13d7850700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d0113c30 con 0x7f13d006e9a0 2024-03-22T06:16:03.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.260+0000 7f13d55ec700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d006e9a0 0x7f13d0112020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:03.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.260+0000 7f13d55ec700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d006e9a0 0x7f13d0112020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:53358/0 (socket says 172.21.15.6:53358) 2024-03-22T06:16:03.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.260+0000 7f13d55ec700 1 -- 172.21.15.6:0/2141168194 learned_addr learned my addr 172.21.15.6:0/2141168194 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:03.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.263+0000 7f13d4deb700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13d006f9d0 0x7f13d0112560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:03.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.263+0000 7f13d55ec700 1 -- 172.21.15.6:0/2141168194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13d006f9d0 msgr2=0x7f13d0112560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:03.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.263+0000 7f13d55ec700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13d006f9d0 0x7f13d0112560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.263+0000 7f13d55ec700 1 -- 172.21.15.6:0/2141168194 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc005040 con 0x7f13d006e9a0 2024-03-22T06:16:03.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.263+0000 7f13d4deb700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13d006f9d0 0x7f13d0112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:16:03.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.264+0000 7f13d55ec700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d006e9a0 0x7f13d0112020 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00b3b0 tx=0x7f13cc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:03.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.265+0000 7f13c5ffb700 1 -- 172.21.15.6:0/2141168194 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13cc00e030 con 0x7f13d006e9a0 2024-03-22T06:16:03.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.265+0000 7f13d7850700 1 -- 172.21.15.6:0/2141168194 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d0112b60 con 0x7f13d006e9a0 2024-03-22T06:16:03.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.265+0000 7f13d7850700 1 -- 172.21.15.6:0/2141168194 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d012ad70 con 0x7f13d006e9a0 2024-03-22T06:16:03.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.266+0000 7f13c5ffb700 1 -- 172.21.15.6:0/2141168194 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13cc013420 con 0x7f13d006e9a0 2024-03-22T06:16:03.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.268+0000 7f13c5ffb700 1 -- 172.21.15.6:0/2141168194 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13cc00bee0 con 0x7f13d006e9a0 2024-03-22T06:16:03.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.270+0000 7f13bb7fe700 1 -- 172.21.15.6:0/2141168194 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d004c710 con 0x7f13d006e9a0 2024-03-22T06:16:03.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.273+0000 7f13c5ffb700 1 -- 172.21.15.6:0/2141168194 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f13cc00ba10 con 0x7f13d006e9a0 2024-03-22T06:16:03.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.273+0000 7f13c5ffb700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f13bc06ba60 0x7f13bc06df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:03.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.273+0000 7f13c5ffb700 1 -- 172.21.15.6:0/2141168194 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f13cc08c850 con 0x7f13d006e9a0 2024-03-22T06:16:03.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.274+0000 7f13d4deb700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f13bc06ba60 0x7f13bc06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:03.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.277+0000 7f13d4deb700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f13bc06ba60 0x7f13bc06df10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f13c0006fd0 tx=0x7f13c0008040 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:03.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.280+0000 7f13c5ffb700 1 -- 172.21.15.6:0/2141168194 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f13cc059b60 con 0x7f13d006e9a0 2024-03-22T06:16:03.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.580+0000 7f13bb7fe700 1 -- 172.21.15.6:0/2141168194 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13d0114740 con 0x7f13bc06ba60 2024-03-22T06:16:03.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.582+0000 7f13c5ffb700 1 -- 172.21.15.6:0/2141168194 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f13d0114740 con 0x7f13bc06ba60 2024-03-22T06:16:03.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.584+0000 7f13bb7fe700 1 -- 172.21.15.6:0/2141168194 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f13bc06ba60 msgr2=0x7f13bc06df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:03.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.584+0000 7f13bb7fe700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f13bc06ba60 0x7f13bc06df10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f13c0006fd0 tx=0x7f13c0008040 comp rx=0 tx=0).stop 2024-03-22T06:16:03.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.584+0000 7f13bb7fe700 1 -- 172.21.15.6:0/2141168194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d006e9a0 msgr2=0x7f13d0112020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:03.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.584+0000 7f13bb7fe700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d006e9a0 0x7f13d0112020 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00b3b0 tx=0x7f13cc009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:03.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.584+0000 7f13bb7fe700 1 -- 172.21.15.6:0/2141168194 shutdown_connections 2024-03-22T06:16:03.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.584+0000 7f13bb7fe700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f13bc06ba60 0x7f13bc06df10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.584+0000 7f13bb7fe700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d006e9a0 0x7f13d0112020 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.584+0000 7f13bb7fe700 1 --2- 172.21.15.6:0/2141168194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13d006f9d0 0x7f13d0112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.584+0000 7f13bb7fe700 1 -- 172.21.15.6:0/2141168194 >> 172.21.15.6:0/2141168194 conn(0x7f13d0069de0 msgr2=0x7f13d006b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:03.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.585+0000 7f13bb7fe700 1 -- 172.21.15.6:0/2141168194 shutdown_connections 2024-03-22T06:16:03.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.585+0000 7f13bb7fe700 1 -- 172.21.15.6:0/2141168194 wait complete. 2024-03-22T06:16:03.598 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:16:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.717+0000 7f7d796fe700 1 -- 172.21.15.6:0/353558236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d740fcf80 msgr2=0x7f7d740fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.717+0000 7f7d796fe700 1 --2- 172.21.15.6:0/353558236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d740fcf80 0x7f7d740fd3f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c004940 tx=0x7f7d5c009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.717+0000 7f7d796fe700 1 -- 172.21.15.6:0/353558236 shutdown_connections 2024-03-22T06:16:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.717+0000 7f7d796fe700 1 --2- 172.21.15.6:0/353558236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d740fcf80 0x7f7d740fd3f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.717+0000 7f7d796fe700 1 --2- 172.21.15.6:0/353558236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740fbcd0 0x7f7d740fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.717+0000 7f7d796fe700 1 -- 172.21.15.6:0/353558236 >> 172.21.15.6:0/353558236 conn(0x7f7d740f7430 msgr2=0x7f7d740f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.717+0000 7f7d796fe700 1 -- 172.21.15.6:0/353558236 shutdown_connections 2024-03-22T06:16:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.717+0000 7f7d796fe700 1 -- 172.21.15.6:0/353558236 wait complete. 2024-03-22T06:16:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.717+0000 7f7d796fe700 1 Processor -- start 2024-03-22T06:16:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d796fe700 1 -- start start 2024-03-22T06:16:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d796fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d740fbcd0 0x7f7d7410c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d796fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740fcf80 0x7f7d7410cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d796fe700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7410d3c0 con 0x7f7d740fbcd0 2024-03-22T06:16:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d796fe700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7410dc10 con 0x7f7d740fcf80 2024-03-22T06:16:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d72ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d740fbcd0 0x7f7d7410c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d72ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d740fbcd0 0x7f7d7410c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33490/0 (socket says 172.21.15.6:33490) 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d72ffd700 1 -- 172.21.15.6:0/154366021 learned_addr learned my addr 172.21.15.6:0/154366021 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d727fc700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740fcf80 0x7f7d7410cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d72ffd700 1 -- 172.21.15.6:0/154366021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740fcf80 msgr2=0x7f7d7410cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d72ffd700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740fcf80 0x7f7d7410cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d72ffd700 1 -- 172.21.15.6:0/154366021 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c005040 con 0x7f7d740fbcd0 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d727fc700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740fcf80 0x7f7d7410cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.718+0000 7f7d72ffd700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d740fbcd0 0x7f7d7410c880 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7d64010d50 tx=0x7f7d6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.719+0000 7f7d6b7fe700 1 -- 172.21.15.6:0/154366021 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d64007ab0 con 0x7f7d740fbcd0 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.719+0000 7f7d796fe700 1 -- 172.21.15.6:0/154366021 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7410de90 con 0x7f7d740fbcd0 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.719+0000 7f7d6b7fe700 1 -- 172.21.15.6:0/154366021 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d64007c10 con 0x7f7d740fbcd0 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.719+0000 7f7d6b7fe700 1 -- 172.21.15.6:0/154366021 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d640126b0 con 0x7f7d740fbcd0 2024-03-22T06:16:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.719+0000 7f7d796fe700 1 -- 172.21.15.6:0/154366021 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7410e3d0 con 0x7f7d740fbcd0 2024-03-22T06:16:03.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.720+0000 7f7d796fe700 1 -- 172.21.15.6:0/154366021 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7405f5f0 con 0x7f7d740fbcd0 2024-03-22T06:16:03.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.721+0000 7f7d6b7fe700 1 -- 172.21.15.6:0/154366021 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f7d64018030 con 0x7f7d740fbcd0 2024-03-22T06:16:03.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.721+0000 7f7d6b7fe700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f7d6006bb10 0x7f7d6006dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:03.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.721+0000 7f7d6b7fe700 1 -- 172.21.15.6:0/154366021 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f7d64016070 con 0x7f7d740fbcd0 2024-03-22T06:16:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.721+0000 7f7d727fc700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f7d6006bb10 0x7f7d6006dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.722+0000 7f7d727fc700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f7d6006bb10 0x7f7d6006dfc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0043d0 tx=0x7f7d5c009e30 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:03.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.726+0000 7f7d6b7fe700 1 -- 172.21.15.6:0/154366021 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f7d6405a210 con 0x7f7d740fbcd0 2024-03-22T06:16:03.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.992+0000 7f7d796fe700 1 -- 172.21.15.6:0/154366021 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d7410e7b0 con 0x7f7d6006bb10 2024-03-22T06:16:03.993 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:03 smithi006 ceph-mon[29301]: pgmap v45: 65 pgs: 65 active+clean; 294 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-03-22T06:16:03.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.993+0000 7f7d6b7fe700 1 -- 172.21.15.6:0/154366021 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f7d7410e7b0 con 0x7f7d6006bb10 2024-03-22T06:16:03.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.995+0000 7f7d796fe700 1 -- 172.21.15.6:0/154366021 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f7d6006bb10 msgr2=0x7f7d6006dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:03.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.995+0000 7f7d796fe700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f7d6006bb10 0x7f7d6006dfc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0043d0 tx=0x7f7d5c009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:03.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.995+0000 7f7d796fe700 1 -- 172.21.15.6:0/154366021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d740fbcd0 msgr2=0x7f7d7410c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:03.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.995+0000 7f7d796fe700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d740fbcd0 0x7f7d7410c880 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7d64010d50 tx=0x7f7d6400b040 comp rx=0 tx=0).stop 2024-03-22T06:16:03.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.995+0000 7f7d796fe700 1 -- 172.21.15.6:0/154366021 shutdown_connections 2024-03-22T06:16:03.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.995+0000 7f7d796fe700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f7d6006bb10 0x7f7d6006dfc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.995+0000 7f7d796fe700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d740fbcd0 0x7f7d7410c880 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.995+0000 7f7d796fe700 1 --2- 172.21.15.6:0/154366021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740fcf80 0x7f7d7410cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:03.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.995+0000 7f7d796fe700 1 -- 172.21.15.6:0/154366021 >> 172.21.15.6:0/154366021 conn(0x7f7d740f7430 msgr2=0x7f7d740f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:03.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.996+0000 7f7d796fe700 1 -- 172.21.15.6:0/154366021 shutdown_connections 2024-03-22T06:16:03.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:03.996+0000 7f7d796fe700 1 -- 172.21.15.6:0/154366021 wait complete. 2024-03-22T06:16:04.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.129+0000 7f3338ef1700 1 -- 172.21.15.6:0/2756812545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340fcf80 msgr2=0x7f33340fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:04.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.129+0000 7f3338ef1700 1 --2- 172.21.15.6:0/2756812545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340fcf80 0x7f33340fd3f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3328004940 tx=0x7f3328009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:04.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.129+0000 7f3338ef1700 1 -- 172.21.15.6:0/2756812545 shutdown_connections 2024-03-22T06:16:04.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.129+0000 7f3338ef1700 1 --2- 172.21.15.6:0/2756812545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340fcf80 0x7f33340fd3f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.129+0000 7f3338ef1700 1 --2- 172.21.15.6:0/2756812545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340fbcd0 0x7f33340fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.129+0000 7f3338ef1700 1 -- 172.21.15.6:0/2756812545 >> 172.21.15.6:0/2756812545 conn(0x7f33340f7430 msgr2=0x7f33340f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:04.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.129+0000 7f3338ef1700 1 -- 172.21.15.6:0/2756812545 shutdown_connections 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.129+0000 7f3338ef1700 1 -- 172.21.15.6:0/2756812545 wait complete. 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.130+0000 7f3338ef1700 1 Processor -- start 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.130+0000 7f3338ef1700 1 -- start start 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.130+0000 7f3338ef1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340fbcd0 0x7f333410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.130+0000 7f3338ef1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340fcf80 0x7f333410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.130+0000 7f3338ef1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333410e4c0 con 0x7f33340fbcd0 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.130+0000 7f3338ef1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333410ed10 con 0x7f33340fcf80 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.130+0000 7f333259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340fbcd0 0x7f333410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.130+0000 7f333259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340fbcd0 0x7f333410d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33510/0 (socket says 172.21.15.6:33510) 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.130+0000 7f333259c700 1 -- 172.21.15.6:0/2735063631 learned_addr learned my addr 172.21.15.6:0/2735063631 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.130+0000 7f3331d9b700 1 --2- 172.21.15.6:0/2735063631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340fcf80 0x7f333410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.131+0000 7f333259c700 1 -- 172.21.15.6:0/2735063631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340fcf80 msgr2=0x7f333410dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.131+0000 7f333259c700 1 --2- 172.21.15.6:0/2735063631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340fcf80 0x7f333410dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.131+0000 7f333259c700 1 -- 172.21.15.6:0/2735063631 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3328005040 con 0x7f33340fbcd0 2024-03-22T06:16:04.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.131+0000 7f333259c700 1 --2- 172.21.15.6:0/2735063631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340fbcd0 0x7f333410d980 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3324002a80 tx=0x7f33240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:04.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.131+0000 7f3322ffd700 1 -- 172.21.15.6:0/2735063631 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3324007cc0 con 0x7f33340fbcd0 2024-03-22T06:16:04.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.131+0000 7f3338ef1700 1 -- 172.21.15.6:0/2735063631 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333410eff0 con 0x7f33340fbcd0 2024-03-22T06:16:04.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.131+0000 7f3322ffd700 1 -- 172.21.15.6:0/2735063631 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f332400f040 con 0x7f33340fbcd0 2024-03-22T06:16:04.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.131+0000 7f3338ef1700 1 -- 172.21.15.6:0/2735063631 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333410f470 con 0x7f33340fbcd0 2024-03-22T06:16:04.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.131+0000 7f3322ffd700 1 -- 172.21.15.6:0/2735063631 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f332400d840 con 0x7f33340fbcd0 2024-03-22T06:16:04.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.133+0000 7f3338ef1700 1 -- 172.21.15.6:0/2735063631 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3334101b10 con 0x7f33340fbcd0 2024-03-22T06:16:04.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.133+0000 7f3322ffd700 1 -- 172.21.15.6:0/2735063631 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f33240097a0 con 0x7f33340fbcd0 2024-03-22T06:16:04.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.134+0000 7f3322ffd700 1 --2- 172.21.15.6:0/2735063631 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f331c06bb10 0x7f331c06dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:04.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.134+0000 7f3322ffd700 1 -- 172.21.15.6:0/2735063631 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f332408a2f0 con 0x7f33340fbcd0 2024-03-22T06:16:04.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.134+0000 7f3331d9b700 1 --2- 172.21.15.6:0/2735063631 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f331c06bb10 0x7f331c06dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:04.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.134+0000 7f3331d9b700 1 --2- 172.21.15.6:0/2735063631 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f331c06bb10 0x7f331c06dfc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f332800b3b0 tx=0x7f3328009e30 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:04.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.139+0000 7f3322ffd700 1 -- 172.21.15.6:0/2735063631 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f3324057510 con 0x7f33340fbcd0 2024-03-22T06:16:04.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:03 smithi117 ceph-mon[36514]: pgmap v45: 65 pgs: 65 active+clean; 294 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-03-22T06:16:04.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.430+0000 7f3338ef1700 1 -- 172.21.15.6:0/2735063631 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f333410f740 con 0x7f331c06bb10 2024-03-22T06:16:04.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.440+0000 7f3322ffd700 1 -- 172.21.15.6:0/2735063631 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f333410f740 con 0x7f331c06bb10 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 starting - - - - 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (9m) 64s ago 9m 6656k - 17.2.7-726-g9c4e0c34 837124da804f a49ab95d42e6 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (8m) 77s ago 8m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0a36af1f8ed0 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 starting - - - - 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (3m) 64s ago 3m 137M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (3m) 64s ago 3m 15.7M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (3m) 77s ago 3m 13.0M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (3m) 77s ago 3m 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:9283 running (10m) 64s ago 10m 338M - 17.2.7-726-g9c4e0c34 837124da804f 2cea0b1a5c60 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283 running (104s) 77s ago 8m 489M - 19.0.0-2299-gea01217f aa461e7c5e7f ff1047688325 2024-03-22T06:16:04.440 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (10m) 64s ago 10m 50.5M 2048M 17.2.7-726-g9c4e0c34 837124da804f b2fe36c6b005 2024-03-22T06:16:04.441 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (8m) 77s ago 8m 39.9M 2048M 17.2.7-726-g9c4e0c34 837124da804f 0559848da9ab 2024-03-22T06:16:04.441 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 starting - - - - 2024-03-22T06:16:04.441 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 starting - - - - 2024-03-22T06:16:04.441 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (7m) 64s ago 7m 104M 1559M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:16:04.441 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (6m) 64s ago 6m 111M 1559M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:16:04.441 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (6m) 64s ago 6m 106M 1559M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:16:04.441 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (6m) 77s ago 6m 125M 2581M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:16:04.441 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (5m) 77s ago 5m 108M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:16:04.441 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (5m) 77s ago 5m 120M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:16:04.441 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 starting - - - - 2024-03-22T06:16:04.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.443+0000 7f3338ef1700 1 -- 172.21.15.6:0/2735063631 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f331c06bb10 msgr2=0x7f331c06dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:04.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.443+0000 7f3338ef1700 1 --2- 172.21.15.6:0/2735063631 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f331c06bb10 0x7f331c06dfc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f332800b3b0 tx=0x7f3328009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:04.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.443+0000 7f3338ef1700 1 -- 172.21.15.6:0/2735063631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340fbcd0 msgr2=0x7f333410d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:04.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.443+0000 7f3338ef1700 1 --2- 172.21.15.6:0/2735063631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340fbcd0 0x7f333410d980 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3324002a80 tx=0x7f33240046b0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.443+0000 7f3338ef1700 1 -- 172.21.15.6:0/2735063631 shutdown_connections 2024-03-22T06:16:04.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.443+0000 7f3338ef1700 1 --2- 172.21.15.6:0/2735063631 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f331c06bb10 0x7f331c06dfc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.443+0000 7f3338ef1700 1 --2- 172.21.15.6:0/2735063631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340fbcd0 0x7f333410d980 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.443+0000 7f3338ef1700 1 --2- 172.21.15.6:0/2735063631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340fcf80 0x7f333410dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.443+0000 7f3338ef1700 1 -- 172.21.15.6:0/2735063631 >> 172.21.15.6:0/2735063631 conn(0x7f33340f7430 msgr2=0x7f33340f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:04.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.443+0000 7f3338ef1700 1 -- 172.21.15.6:0/2735063631 shutdown_connections 2024-03-22T06:16:04.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.443+0000 7f3338ef1700 1 -- 172.21.15.6:0/2735063631 wait complete. 2024-03-22T06:16:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.572+0000 7f538785c700 1 -- 172.21.15.6:0/2101075631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53800fe040 msgr2=0x7f53800fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.572+0000 7f538785c700 1 --2- 172.21.15.6:0/2101075631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53800fe040 0x7f53800fe4b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5370004940 tx=0x7f5370009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.572+0000 7f538785c700 1 -- 172.21.15.6:0/2101075631 shutdown_connections 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.572+0000 7f538785c700 1 --2- 172.21.15.6:0/2101075631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53800fe040 0x7f53800fe4b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.572+0000 7f538785c700 1 --2- 172.21.15.6:0/2101075631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53800fce40 0x7f53800fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.572+0000 7f538785c700 1 -- 172.21.15.6:0/2101075631 >> 172.21.15.6:0/2101075631 conn(0x7f53800f8510 msgr2=0x7f53800fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.572+0000 7f538785c700 1 -- 172.21.15.6:0/2101075631 shutdown_connections 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.572+0000 7f538785c700 1 -- 172.21.15.6:0/2101075631 wait complete. 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.573+0000 7f538785c700 1 Processor -- start 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.573+0000 7f538785c700 1 -- start start 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.573+0000 7f538785c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53800fce40 0x7f538010da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.573+0000 7f538785c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53800fe040 0x7f538010df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.573+0000 7f538785c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538010e560 con 0x7f53800fce40 2024-03-22T06:16:04.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.573+0000 7f538785c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538010edb0 con 0x7f53800fe040 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.574+0000 7f53855f8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53800fce40 0x7f538010da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.574+0000 7f53855f8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53800fce40 0x7f538010da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33530/0 (socket says 172.21.15.6:33530) 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.574+0000 7f53855f8700 1 -- 172.21.15.6:0/3878847669 learned_addr learned my addr 172.21.15.6:0/3878847669 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.574+0000 7f5384df7700 1 --2- 172.21.15.6:0/3878847669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53800fe040 0x7f538010df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.574+0000 7f53855f8700 1 -- 172.21.15.6:0/3878847669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53800fe040 msgr2=0x7f538010df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.574+0000 7f53855f8700 1 --2- 172.21.15.6:0/3878847669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53800fe040 0x7f538010df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.574+0000 7f53855f8700 1 -- 172.21.15.6:0/3878847669 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5370005040 con 0x7f53800fce40 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.574+0000 7f53855f8700 1 --2- 172.21.15.6:0/3878847669 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53800fce40 0x7f538010da20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f537c002a80 tx=0x7f537c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.575+0000 7f5375ffb700 1 -- 172.21.15.6:0/3878847669 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f537c007cc0 con 0x7f53800fce40 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.575+0000 7f5375ffb700 1 -- 172.21.15.6:0/3878847669 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f537c00f040 con 0x7f53800fce40 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.575+0000 7f538785c700 1 -- 172.21.15.6:0/3878847669 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538010f090 con 0x7f53800fce40 2024-03-22T06:16:04.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.575+0000 7f5375ffb700 1 -- 172.21.15.6:0/3878847669 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f537c00dad0 con 0x7f53800fce40 2024-03-22T06:16:04.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.576+0000 7f538785c700 1 -- 172.21.15.6:0/3878847669 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538010f5d0 con 0x7f53800fce40 2024-03-22T06:16:04.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.576+0000 7f5375ffb700 1 -- 172.21.15.6:0/3878847669 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f537c00dc30 con 0x7f53800fce40 2024-03-22T06:16:04.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.577+0000 7f538785c700 1 -- 172.21.15.6:0/3878847669 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538005f5f0 con 0x7f53800fce40 2024-03-22T06:16:04.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.577+0000 7f5375ffb700 1 --2- 172.21.15.6:0/3878847669 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f536c06bb10 0x7f536c06dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:04.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.577+0000 7f5375ffb700 1 -- 172.21.15.6:0/3878847669 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f537c08e170 con 0x7f53800fce40 2024-03-22T06:16:04.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.577+0000 7f5384df7700 1 --2- 172.21.15.6:0/3878847669 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f536c06bb10 0x7f536c06dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:04.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.578+0000 7f5384df7700 1 --2- 172.21.15.6:0/3878847669 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f536c06bb10 0x7f536c06dfc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f537000b3b0 tx=0x7f5370009e30 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:04.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.582+0000 7f5375ffb700 1 -- 172.21.15.6:0/3878847669 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f537c08f060 con 0x7f53800fce40 2024-03-22T06:16:04.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.971+0000 7f538785c700 1 -- 172.21.15.6:0/3878847669 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f538004bed0 con 0x7f53800fce40 2024-03-22T06:16:04.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.972+0000 7f5375ffb700 1 -- 172.21.15.6:0/3878847669 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f537c057000 con 0x7f53800fce40 2024-03-22T06:16:04.971 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:16:04.971 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:16:04.971 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T06:16:04.971 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:16:04.972 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:16:04.972 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T06:16:04.972 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 1 2024-03-22T06:16:04.972 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:16:04.972 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:16:04.972 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T06:16:04.972 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:16:04.972 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:16:04.973 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:16:04.973 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:16:04.973 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:16:04.973 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-22T06:16:04.973 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 1 2024-03-22T06:16:04.973 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:16:04.973 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:16:04.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.974+0000 7f538785c700 1 -- 172.21.15.6:0/3878847669 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f536c06bb10 msgr2=0x7f536c06dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:04.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.974+0000 7f538785c700 1 --2- 172.21.15.6:0/3878847669 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f536c06bb10 0x7f536c06dfc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f537000b3b0 tx=0x7f5370009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:04.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.974+0000 7f538785c700 1 -- 172.21.15.6:0/3878847669 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53800fce40 msgr2=0x7f538010da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:04.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.974+0000 7f538785c700 1 --2- 172.21.15.6:0/3878847669 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53800fce40 0x7f538010da20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f537c002a80 tx=0x7f537c004600 comp rx=0 tx=0).stop 2024-03-22T06:16:04.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.974+0000 7f538785c700 1 -- 172.21.15.6:0/3878847669 shutdown_connections 2024-03-22T06:16:04.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.974+0000 7f538785c700 1 --2- 172.21.15.6:0/3878847669 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f536c06bb10 0x7f536c06dfc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.974+0000 7f538785c700 1 --2- 172.21.15.6:0/3878847669 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53800fce40 0x7f538010da20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.974+0000 7f538785c700 1 --2- 172.21.15.6:0/3878847669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53800fe040 0x7f538010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:04.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.974+0000 7f538785c700 1 -- 172.21.15.6:0/3878847669 >> 172.21.15.6:0/3878847669 conn(0x7f53800f8510 msgr2=0x7f53800f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:04.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.974+0000 7f538785c700 1 -- 172.21.15.6:0/3878847669 shutdown_connections 2024-03-22T06:16:04.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:04.974+0000 7f538785c700 1 -- 172.21.15.6:0/3878847669 wait complete. 2024-03-22T06:16:05.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:04 smithi006 ceph-mon[29301]: from='client.24471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:05.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:04 smithi006 ceph-mon[29301]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:05.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:04 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:05.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:04 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi006.sfpxdy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:16:05.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:04 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi006.sfpxdy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:16:05.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:04 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:16:05.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:04 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:16:05.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.124+0000 7f806af8b700 1 -- 172.21.15.6:0/474235139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80640fed50 msgr2=0x7f80640fae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:05.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.124+0000 7f806af8b700 1 --2- 172.21.15.6:0/474235139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80640fed50 0x7f80640fae90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8050004940 tx=0x7f8050009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:05.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.124+0000 7f806af8b700 1 -- 172.21.15.6:0/474235139 shutdown_connections 2024-03-22T06:16:05.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.124+0000 7f806af8b700 1 --2- 172.21.15.6:0/474235139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80640fed50 0x7f80640fae90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.124+0000 7f806af8b700 1 --2- 172.21.15.6:0/474235139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80640fe400 0x7f80640fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.124+0000 7f806af8b700 1 -- 172.21.15.6:0/474235139 >> 172.21.15.6:0/474235139 conn(0x7f80640f6ba0 msgr2=0x7f80640f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:05.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.124+0000 7f806af8b700 1 -- 172.21.15.6:0/474235139 shutdown_connections 2024-03-22T06:16:05.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.124+0000 7f806af8b700 1 -- 172.21.15.6:0/474235139 wait complete. 2024-03-22T06:16:05.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.124+0000 7f806af8b700 1 Processor -- start 2024-03-22T06:16:05.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.124+0000 7f806af8b700 1 -- start start 2024-03-22T06:16:05.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f806af8b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80640fe400 0x7f80640fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:05.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f806af8b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80640fed50 0x7f80640fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:05.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f806af8b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80640fc670 con 0x7f80640fe400 2024-03-22T06:16:05.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f806af8b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80640fc7e0 con 0x7f80640fed50 2024-03-22T06:16:05.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f8068d27700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80640fe400 0x7f80640fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:05.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f8063fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80640fed50 0x7f80640fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:05.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f8068d27700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80640fe400 0x7f80640fdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33544/0 (socket says 172.21.15.6:33544) 2024-03-22T06:16:05.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f8063fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80640fed50 0x7f80640fc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:33320/0 (socket says 172.21.15.6:33320) 2024-03-22T06:16:05.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f8068d27700 1 -- 172.21.15.6:0/3149648632 learned_addr learned my addr 172.21.15.6:0/3149648632 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:05.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f8063fff700 1 -- 172.21.15.6:0/3149648632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80640fe400 msgr2=0x7f80640fdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:05.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f8063fff700 1 --2- 172.21.15.6:0/3149648632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80640fe400 0x7f80640fdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.125+0000 7f8063fff700 1 -- 172.21.15.6:0/3149648632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8050005040 con 0x7f80640fed50 2024-03-22T06:16:05.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.126+0000 7f8063fff700 1 --2- 172.21.15.6:0/3149648632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80640fed50 0x7f80640fc0c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8050000c00 tx=0x7f8050009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:05.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.126+0000 7f80617fa700 1 -- 172.21.15.6:0/3149648632 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f805000e030 con 0x7f80640fed50 2024-03-22T06:16:05.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.126+0000 7f80617fa700 1 -- 172.21.15.6:0/3149648632 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8050013de0 con 0x7f80640fed50 2024-03-22T06:16:05.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.126+0000 7f80617fa700 1 -- 172.21.15.6:0/3149648632 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f805001ce50 con 0x7f80640fed50 2024-03-22T06:16:05.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.126+0000 7f806af8b700 1 -- 172.21.15.6:0/3149648632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80640fca60 con 0x7f80640fed50 2024-03-22T06:16:05.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.126+0000 7f806af8b700 1 -- 172.21.15.6:0/3149648632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80640fce90 con 0x7f80640fed50 2024-03-22T06:16:05.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.127+0000 7f806af8b700 1 -- 172.21.15.6:0/3149648632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806405f5f0 con 0x7f80640fed50 2024-03-22T06:16:05.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.127+0000 7f80617fa700 1 -- 172.21.15.6:0/3149648632 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f8050013910 con 0x7f80640fed50 2024-03-22T06:16:05.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.128+0000 7f80617fa700 1 --2- 172.21.15.6:0/3149648632 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f805406bab0 0x7f805406df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:05.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.128+0000 7f80617fa700 1 -- 172.21.15.6:0/3149648632 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f805008c190 con 0x7f80640fed50 2024-03-22T06:16:05.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.128+0000 7f8068d27700 1 --2- 172.21.15.6:0/3149648632 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f805406bab0 0x7f805406df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:05.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.128+0000 7f8068d27700 1 --2- 172.21.15.6:0/3149648632 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f805406bab0 0x7f805406df60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8058008fd0 tx=0x7f805800b040 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:05.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.133+0000 7f80617fa700 1 -- 172.21.15.6:0/3149648632 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f8050055990 con 0x7f80640fed50 2024-03-22T06:16:05.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:04 smithi117 ceph-mon[36514]: from='client.24471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:05.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:04 smithi117 ceph-mon[36514]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:05.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:04 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:05.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:04 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi006.sfpxdy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:16:05.158 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:04 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi006.sfpxdy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:16:05.158 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:04 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:16:05.158 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:04 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:16:05.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.466+0000 7f806af8b700 1 -- 172.21.15.6:0/3149648632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f80641058b0 con 0x7f80640fed50 2024-03-22T06:16:05.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.466+0000 7f80617fa700 1 -- 172.21.15.6:0/3149648632 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1805 (secure 0 0 0) 0x7f8050059050 con 0x7f80640fed50 2024-03-22T06:16:05.466 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:16:05.466 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:16:05.466 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:16:05.466 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:16:05.466 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:16:05.466 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:16:05.466 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:16:05.466 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:16:05.466 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:16:05.467 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:16:05.468 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:16:05.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.469+0000 7f806af8b700 1 -- 172.21.15.6:0/3149648632 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f805406bab0 msgr2=0x7f805406df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:05.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.469+0000 7f806af8b700 1 --2- 172.21.15.6:0/3149648632 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f805406bab0 0x7f805406df60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8058008fd0 tx=0x7f805800b040 comp rx=0 tx=0).stop 2024-03-22T06:16:05.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.469+0000 7f806af8b700 1 -- 172.21.15.6:0/3149648632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80640fed50 msgr2=0x7f80640fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:05.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.469+0000 7f806af8b700 1 --2- 172.21.15.6:0/3149648632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80640fed50 0x7f80640fc0c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8050000c00 tx=0x7f8050009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:05.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.469+0000 7f806af8b700 1 -- 172.21.15.6:0/3149648632 shutdown_connections 2024-03-22T06:16:05.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.469+0000 7f806af8b700 1 --2- 172.21.15.6:0/3149648632 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f805406bab0 0x7f805406df60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.469+0000 7f806af8b700 1 --2- 172.21.15.6:0/3149648632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80640fe400 0x7f80640fdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.469+0000 7f806af8b700 1 --2- 172.21.15.6:0/3149648632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80640fed50 0x7f80640fc0c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.469+0000 7f806af8b700 1 -- 172.21.15.6:0/3149648632 >> 172.21.15.6:0/3149648632 conn(0x7f80640f6ba0 msgr2=0x7f80640f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:05.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.469+0000 7f806af8b700 1 -- 172.21.15.6:0/3149648632 shutdown_connections 2024-03-22T06:16:05.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.469+0000 7f806af8b700 1 -- 172.21.15.6:0/3149648632 wait complete. 2024-03-22T06:16:05.470 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:16:05.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.599+0000 7f708db89700 1 -- 172.21.15.6:0/3550651610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7088066280 msgr2=0x7f70880ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:05.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.599+0000 7f708db89700 1 --2- 172.21.15.6:0/3550651610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7088066280 0x7f70880ffa20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7078004940 tx=0x7f7078009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:05.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.599+0000 7f708db89700 1 -- 172.21.15.6:0/3550651610 shutdown_connections 2024-03-22T06:16:05.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.599+0000 7f708db89700 1 --2- 172.21.15.6:0/3550651610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70880fff60 0x7f7088102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.599+0000 7f708db89700 1 --2- 172.21.15.6:0/3550651610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7088066280 0x7f70880ffa20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.599+0000 7f708db89700 1 -- 172.21.15.6:0/3550651610 >> 172.21.15.6:0/3550651610 conn(0x7f70880f7430 msgr2=0x7f70880f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:05.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.599+0000 7f708db89700 1 -- 172.21.15.6:0/3550651610 shutdown_connections 2024-03-22T06:16:05.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.599+0000 7f708db89700 1 -- 172.21.15.6:0/3550651610 wait complete. 2024-03-22T06:16:05.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.600+0000 7f708db89700 1 Processor -- start 2024-03-22T06:16:05.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.600+0000 7f708db89700 1 -- start start 2024-03-22T06:16:05.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.600+0000 7f708db89700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088066280 0x7f70880fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:05.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.600+0000 7f708db89700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70880fff60 0x7f70880fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:05.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.600+0000 7f708db89700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70880fbeb0 con 0x7f70880fff60 2024-03-22T06:16:05.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.600+0000 7f708db89700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70880fbff0 con 0x7f7088066280 2024-03-22T06:16:05.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.600+0000 7f70877fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088066280 0x7f70880fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:05.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.600+0000 7f70877fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088066280 0x7f70880fd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:33340/0 (socket says 172.21.15.6:33340) 2024-03-22T06:16:05.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.600+0000 7f70877fe700 1 -- 172.21.15.6:0/3670263435 learned_addr learned my addr 172.21.15.6:0/3670263435 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:05.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.600+0000 7f7086ffd700 1 --2- 172.21.15.6:0/3670263435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70880fff60 0x7f70880fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:05.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.601+0000 7f70877fe700 1 -- 172.21.15.6:0/3670263435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70880fff60 msgr2=0x7f70880fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:05.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.601+0000 7f70877fe700 1 --2- 172.21.15.6:0/3670263435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70880fff60 0x7f70880fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.601+0000 7f70877fe700 1 -- 172.21.15.6:0/3670263435 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7078005040 con 0x7f7088066280 2024-03-22T06:16:05.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.601+0000 7f70877fe700 1 --2- 172.21.15.6:0/3670263435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088066280 0x7f70880fd400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7078000c00 tx=0x7f7078009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:05.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.601+0000 7f706ffff700 1 -- 172.21.15.6:0/3670263435 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707800e030 con 0x7f7088066280 2024-03-22T06:16:05.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.601+0000 7f706ffff700 1 -- 172.21.15.6:0/3670263435 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7078013de0 con 0x7f7088066280 2024-03-22T06:16:05.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.601+0000 7f708db89700 1 -- 172.21.15.6:0/3670263435 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70880fc270 con 0x7f7088066280 2024-03-22T06:16:05.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.601+0000 7f706ffff700 1 -- 172.21.15.6:0/3670263435 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707801ce50 con 0x7f7088066280 2024-03-22T06:16:05.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.601+0000 7f708db89700 1 -- 172.21.15.6:0/3670263435 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7088115ab0 con 0x7f7088066280 2024-03-22T06:16:05.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.602+0000 7f708db89700 1 -- 172.21.15.6:0/3670263435 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708805f5f0 con 0x7f7088066280 2024-03-22T06:16:05.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.603+0000 7f706ffff700 1 -- 172.21.15.6:0/3670263435 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7f707801c400 con 0x7f7088066280 2024-03-22T06:16:05.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.604+0000 7f706ffff700 1 --2- 172.21.15.6:0/3670263435 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f707006bab0 0x7f707006df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:05.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.604+0000 7f706ffff700 1 -- 172.21.15.6:0/3670263435 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f707808c2a0 con 0x7f7088066280 2024-03-22T06:16:05.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.604+0000 7f7086ffd700 1 --2- 172.21.15.6:0/3670263435 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f707006bab0 0x7f707006df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:05.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.605+0000 7f7086ffd700 1 --2- 172.21.15.6:0/3670263435 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f707006bab0 0x7f707006df60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70880fd9e0 tx=0x7f707c00b040 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:05.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.608+0000 7f706ffff700 1 -- 172.21.15.6:0/3670263435 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f7078055aa0 con 0x7f7088066280 2024-03-22T06:16:05.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.927+0000 7f708db89700 1 -- 172.21.15.6:0/3670263435 --> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70880f8f70 con 0x7f707006bab0 2024-03-22T06:16:05.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.928+0000 7f706ffff700 1 -- 172.21.15.6:0/3670263435 <== mgr.14608 v2:172.21.15.117:6828/1963654099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f70880f8f70 con 0x7f707006bab0 2024-03-22T06:16:05.928 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:16:05.928 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:16:05.928 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:16:05.928 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:16:05.928 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [], 2024-03-22T06:16:05.928 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "1/21 daemons upgraded", 2024-03-22T06:16:05.928 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading mgr daemons", 2024-03-22T06:16:05.928 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:16:05.928 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:16:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.931+0000 7f708db89700 1 -- 172.21.15.6:0/3670263435 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f707006bab0 msgr2=0x7f707006df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.931+0000 7f708db89700 1 --2- 172.21.15.6:0/3670263435 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f707006bab0 0x7f707006df60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70880fd9e0 tx=0x7f707c00b040 comp rx=0 tx=0).stop 2024-03-22T06:16:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.931+0000 7f708db89700 1 -- 172.21.15.6:0/3670263435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088066280 msgr2=0x7f70880fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.931+0000 7f708db89700 1 --2- 172.21.15.6:0/3670263435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088066280 0x7f70880fd400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7078000c00 tx=0x7f7078009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.931+0000 7f708db89700 1 -- 172.21.15.6:0/3670263435 shutdown_connections 2024-03-22T06:16:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.931+0000 7f708db89700 1 --2- 172.21.15.6:0/3670263435 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7f707006bab0 0x7f707006df60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.931+0000 7f708db89700 1 --2- 172.21.15.6:0/3670263435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7088066280 0x7f70880fd400 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.931+0000 7f708db89700 1 --2- 172.21.15.6:0/3670263435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70880fff60 0x7f70880fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.931+0000 7f708db89700 1 -- 172.21.15.6:0/3670263435 >> 172.21.15.6:0/3670263435 conn(0x7f70880f7430 msgr2=0x7f70880f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.931+0000 7f708db89700 1 -- 172.21.15.6:0/3670263435 shutdown_connections 2024-03-22T06:16:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:05.931+0000 7f708db89700 1 -- 172.21.15.6:0/3670263435 wait complete. 2024-03-22T06:16:06.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:05 smithi006 ceph-mon[29301]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:06.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:05 smithi006 ceph-mon[29301]: Upgrade: Updating mgr.smithi006.sfpxdy 2024-03-22T06:16:06.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:05 smithi006 ceph-mon[29301]: Deploying daemon mgr.smithi006.sfpxdy on smithi006 2024-03-22T06:16:06.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:05 smithi006 ceph-mon[29301]: pgmap v46: 65 pgs: 65 active+clean; 299 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-03-22T06:16:06.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:05 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3878847669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:16:06.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:05 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/3149648632' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:16:06.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.077+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1920365193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2bc0fcfa0 msgr2=0x7fd2bc0fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:06.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.077+0000 7fd2c26d6700 1 --2- 172.21.15.6:0/1920365193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2bc0fcfa0 0x7fd2bc0fd410 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0004940 tx=0x7fd2b0009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:06.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.078+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1920365193 shutdown_connections 2024-03-22T06:16:06.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.078+0000 7fd2c26d6700 1 --2- 172.21.15.6:0/1920365193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2bc0fcfa0 0x7fd2bc0fd410 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:06.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.078+0000 7fd2c26d6700 1 --2- 172.21.15.6:0/1920365193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2bc0fbcf0 0x7fd2bc0fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:06.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.078+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1920365193 >> 172.21.15.6:0/1920365193 conn(0x7fd2bc0f7450 msgr2=0x7fd2bc0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:06.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.078+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1920365193 shutdown_connections 2024-03-22T06:16:06.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.078+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1920365193 wait complete. 2024-03-22T06:16:06.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.078+0000 7fd2c26d6700 1 Processor -- start 2024-03-22T06:16:06.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2c26d6700 1 -- start start 2024-03-22T06:16:06.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2c26d6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2bc0fbcf0 0x7fd2bc10c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:06.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2c26d6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2bc0fcfa0 0x7fd2bc10cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:06.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2c26d6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc10d3f0 con 0x7fd2bc0fcfa0 2024-03-22T06:16:06.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2c26d6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc10dc40 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2bb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2bc0fcfa0 0x7fd2bc10cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:06.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2bbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2bc0fbcf0 0x7fd2bc10c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:06.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2bb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2bc0fcfa0 0x7fd2bc10cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33576/0 (socket says 172.21.15.6:33576) 2024-03-22T06:16:06.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2bb7fe700 1 -- 172.21.15.6:0/1836657368 learned_addr learned my addr 172.21.15.6:0/1836657368 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:06.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2bbfff700 1 -- 172.21.15.6:0/1836657368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2bc0fcfa0 msgr2=0x7fd2bc10cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:06.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2bbfff700 1 --2- 172.21.15.6:0/1836657368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2bc0fcfa0 0x7fd2bc10cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:06.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2bbfff700 1 -- 172.21.15.6:0/1836657368 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b0005040 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.079+0000 7fd2bb7fe700 1 --2- 172.21.15.6:0/1836657368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2bc0fcfa0 0x7fd2bc10cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:16:06.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.080+0000 7fd2bbfff700 1 --2- 172.21.15.6:0/1836657368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2bc0fbcf0 0x7fd2bc10c8b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009fd0 tx=0x7fd2ac00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:06.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.080+0000 7fd2b8ff9700 1 -- 172.21.15.6:0/1836657368 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2ac004e90 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.080+0000 7fd2b8ff9700 1 -- 172.21.15.6:0/1836657368 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2ac00f040 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.080+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1836657368 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2bc10dec0 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.080+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1836657368 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2bc10e400 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.080+0000 7fd2b8ff9700 1 -- 172.21.15.6:0/1836657368 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2ac00ca50 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.081+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1836657368 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a80052e0 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.081+0000 7fd2b8ff9700 1 -- 172.21.15.6:0/1836657368 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) v1 ==== 89552+0+0 (secure 0 0 0) 0x7fd2ac00cbb0 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.082+0000 7fd2b8ff9700 1 --2- 172.21.15.6:0/1836657368 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fd2a406bb10 0x7fd2a406dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:06.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.082+0000 7fd2b8ff9700 1 -- 172.21.15.6:0/1836657368 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fd2ac08a160 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.082+0000 7fd2bb7fe700 1 --2- 172.21.15.6:0/1836657368 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fd2a406bb10 0x7fd2a406dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:06.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.082+0000 7fd2bb7fe700 1 --2- 172.21.15.6:0/1836657368 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fd2a406bb10 0x7fd2a406dfc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000b0d0 tx=0x7fd2b0009e30 comp rx=0 tx=0).ready entity=mgr.14608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:06.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.087+0000 7fd2b8ff9700 1 -- 172.21.15.6:0/1836657368 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7fd2ac0503a0 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:05 smithi117 ceph-mon[36514]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:06.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:05 smithi117 ceph-mon[36514]: Upgrade: Updating mgr.smithi006.sfpxdy 2024-03-22T06:16:06.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:05 smithi117 ceph-mon[36514]: Deploying daemon mgr.smithi006.sfpxdy on smithi006 2024-03-22T06:16:06.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:05 smithi117 ceph-mon[36514]: pgmap v46: 65 pgs: 65 active+clean; 299 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-03-22T06:16:06.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:05 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3878847669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:16:06.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:05 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3149648632' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:16:06.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.498+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1836657368 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd2a8005860 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.498+0000 7fd2b8ff9700 1 -- 172.21.15.6:0/1836657368 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd2ac057540 con 0x7fd2bc0fbcf0 2024-03-22T06:16:06.499 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_OK 2024-03-22T06:16:06.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.501+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1836657368 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fd2a406bb10 msgr2=0x7fd2a406dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:06.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.501+0000 7fd2c26d6700 1 --2- 172.21.15.6:0/1836657368 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fd2a406bb10 0x7fd2a406dfc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000b0d0 tx=0x7fd2b0009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:06.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.501+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1836657368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2bc0fbcf0 msgr2=0x7fd2bc10c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:06.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.501+0000 7fd2c26d6700 1 --2- 172.21.15.6:0/1836657368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2bc0fbcf0 0x7fd2bc10c8b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009fd0 tx=0x7fd2ac00b040 comp rx=0 tx=0).stop 2024-03-22T06:16:06.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.501+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1836657368 shutdown_connections 2024-03-22T06:16:06.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.501+0000 7fd2c26d6700 1 --2- 172.21.15.6:0/1836657368 >> [v2:172.21.15.117:6828/1963654099,v1:172.21.15.117:6829/1963654099] conn(0x7fd2a406bb10 0x7fd2a406dfc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:06.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.501+0000 7fd2c26d6700 1 --2- 172.21.15.6:0/1836657368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2bc0fbcf0 0x7fd2bc10c8b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:06.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.501+0000 7fd2c26d6700 1 --2- 172.21.15.6:0/1836657368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2bc0fcfa0 0x7fd2bc10cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:06.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.501+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1836657368 >> 172.21.15.6:0/1836657368 conn(0x7fd2bc0f7450 msgr2=0x7fd2bc0f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:06.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.501+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1836657368 shutdown_connections 2024-03-22T06:16:06.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:06.501+0000 7fd2c26d6700 1 -- 172.21.15.6:0/1836657368 wait complete. 2024-03-22T06:16:07.079 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:06 smithi006 ceph-mon[29301]: from='client.24489 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:07.079 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:06 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1836657368' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:16:07.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:06 smithi117 ceph-mon[36514]: from='client.24489 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:07.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:06 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1836657368' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:16:08.199 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:07 smithi006 ceph-mon[29301]: pgmap v47: 65 pgs: 65 active+clean; 298 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.7 MiB/s wr, 281 op/s 2024-03-22T06:16:08.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:07 smithi117 ceph-mon[36514]: pgmap v47: 65 pgs: 65 active+clean; 298 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.7 MiB/s wr, 281 op/s 2024-03-22T06:16:09.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:08 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:16:09.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:08 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:16:10.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:09 smithi117 ceph-mon[36514]: pgmap v48: 65 pgs: 65 active+clean; 298 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 270 op/s 2024-03-22T06:16:10.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:09 smithi006 ceph-mon[29301]: pgmap v48: 65 pgs: 65 active+clean; 298 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 270 op/s 2024-03-22T06:16:10.949 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:10 smithi006 ceph-mon[29301]: mgrmap e25: smithi117.bqeskk(active, since 91s), standbys: smithi006.sfpxdy 2024-03-22T06:16:11.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:10 smithi117 ceph-mon[36514]: mgrmap e25: smithi117.bqeskk(active, since 91s), standbys: smithi006.sfpxdy 2024-03-22T06:16:12.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:11 smithi006 ceph-mon[29301]: pgmap v49: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.0 MiB/s wr, 421 op/s 2024-03-22T06:16:12.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:11 smithi117 ceph-mon[36514]: pgmap v49: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.0 MiB/s wr, 421 op/s 2024-03-22T06:16:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:12 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:12 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:12 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:16:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:12 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:12 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:12 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:12 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:12 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:16:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:12 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:12 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:14.078 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:13 smithi006 ceph-mon[29301]: pgmap v50: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 294 op/s 2024-03-22T06:16:14.222 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:13 smithi117 ceph-mon[36514]: pgmap v50: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 294 op/s 2024-03-22T06:16:16.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:15 smithi117 ceph-mon[36514]: pgmap v51: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 2.1 MiB/s wr, 381 op/s 2024-03-22T06:16:16.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:15 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:16.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:15 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:16.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:15 smithi006 ceph-mon[29301]: pgmap v51: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 2.1 MiB/s wr, 381 op/s 2024-03-22T06:16:16.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:15 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:16.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:15 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:18.011 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:17 smithi006 ceph-mon[29301]: pgmap v52: 65 pgs: 65 active+clean; 309 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.5 MiB/s wr, 366 op/s 2024-03-22T06:16:18.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:17 smithi117 ceph-mon[36514]: pgmap v52: 65 pgs: 65 active+clean; 309 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.5 MiB/s wr, 366 op/s 2024-03-22T06:16:20.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:19 smithi117 ceph-mon[36514]: pgmap v53: 65 pgs: 65 active+clean; 309 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 301 op/s 2024-03-22T06:16:20.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:19 smithi006 ceph-mon[29301]: pgmap v53: 65 pgs: 65 active+clean; 309 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 301 op/s 2024-03-22T06:16:21.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:20 smithi117 ceph-mon[36514]: Standby manager daemon smithi006.sfpxdy restarted 2024-03-22T06:16:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:20 smithi117 ceph-mon[36514]: Standby manager daemon smithi006.sfpxdy started 2024-03-22T06:16:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:20 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi006.sfpxdy/crt"}]: dispatch 2024-03-22T06:16:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:20 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:16:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:20 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi006.sfpxdy/key"}]: dispatch 2024-03-22T06:16:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:20 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:16:21.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:20 smithi006 ceph-mon[29301]: Standby manager daemon smithi006.sfpxdy restarted 2024-03-22T06:16:21.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:20 smithi006 ceph-mon[29301]: Standby manager daemon smithi006.sfpxdy started 2024-03-22T06:16:21.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:20 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi006.sfpxdy/crt"}]: dispatch 2024-03-22T06:16:21.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:20 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:16:21.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:20 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi006.sfpxdy/key"}]: dispatch 2024-03-22T06:16:21.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:20 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:16:22.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:21 smithi006 ceph-mon[29301]: pgmap v54: 65 pgs: 65 active+clean; 312 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 2.3 MiB/s wr, 397 op/s 2024-03-22T06:16:22.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:21 smithi006 ceph-mon[29301]: mgrmap e26: smithi117.bqeskk(active, since 102s), standbys: smithi006.sfpxdy 2024-03-22T06:16:22.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:21 smithi117 ceph-mon[36514]: pgmap v54: 65 pgs: 65 active+clean; 312 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 2.3 MiB/s wr, 397 op/s 2024-03-22T06:16:22.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:21 smithi117 ceph-mon[36514]: mgrmap e26: smithi117.bqeskk(active, since 102s), standbys: smithi006.sfpxdy 2024-03-22T06:16:23.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:23 smithi006 ceph-mon[29301]: pgmap v55: 65 pgs: 65 active+clean; 312 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-03-22T06:16:23.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:23 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:23.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:23 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:23.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:23 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:16:24.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:23 smithi117 ceph-mon[36514]: pgmap v55: 65 pgs: 65 active+clean; 312 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-03-22T06:16:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:23 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:23 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:23 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:16:25.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:25 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:25 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:25 smithi117 ceph-mon[36514]: pgmap v56: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 314 op/s 2024-03-22T06:16:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:25 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:25 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:25 smithi006 ceph-mon[29301]: pgmap v56: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 314 op/s 2024-03-22T06:16:28.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:27 smithi117 ceph-mon[36514]: pgmap v57: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-03-22T06:16:28.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:27 smithi006 ceph-mon[29301]: pgmap v57: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-03-22T06:16:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:16:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:16:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:16:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mgr fail", "who": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:16:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mgr fail", "who": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:16:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T06:16:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd='[{"prefix": "mgr fail", "who": "smithi117.bqeskk"}]': finished 2024-03-22T06:16:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: mgrmap e27: smithi006.sfpxdy(active, starting, since 0.0066781s) 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi006.sfpxdy", "id": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T06:16:30.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T06:16:30.009 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T06:16:30.009 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: Manager daemon smithi006.sfpxdy is now available 2024-03-22T06:16:30.009 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:16:30.009 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:16:30.009 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:16:30.009 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:16:30.009 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/trash_purge_schedule"}]: dispatch 2024-03-22T06:16:30.009 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:29 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/trash_purge_schedule"}]: dispatch 2024-03-22T06:16:30.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:30.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:30.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:16:30.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:16:30.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' 2024-03-22T06:16:30.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:16:30.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14608 172.21.15.117:0/2859963546' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mgr fail", "who": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:16:30.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "mgr fail", "who": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:16:30.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T06:16:30.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14608 ' entity='mgr.smithi117.bqeskk' cmd='[{"prefix": "mgr fail", "who": "smithi117.bqeskk"}]': finished 2024-03-22T06:16:30.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: mgrmap e27: smithi006.sfpxdy(active, starting, since 0.0066781s) 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi006.sfpxdy", "id": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:16:30.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T06:16:30.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T06:16:30.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T06:16:30.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: Manager daemon smithi006.sfpxdy is now available 2024-03-22T06:16:30.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:16:30.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:16:30.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:16:30.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:16:30.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/trash_purge_schedule"}]: dispatch 2024-03-22T06:16:30.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:29 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/trash_purge_schedule"}]: dispatch 2024-03-22T06:16:31.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:30 smithi117 ceph-mon[36514]: [22/Mar/2024:06:16:29] ENGINE Bus STARTING 2024-03-22T06:16:31.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:30 smithi117 ceph-mon[36514]: [22/Mar/2024:06:16:29] ENGINE Serving on https://172.21.15.6:7150 2024-03-22T06:16:31.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:30 smithi117 ceph-mon[36514]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-22T06:16:31.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:30 smithi117 ceph-mon[36514]: mgrmap e28: smithi006.sfpxdy(active, since 1.01074s) 2024-03-22T06:16:31.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:30 smithi006 ceph-mon[29301]: [22/Mar/2024:06:16:29] ENGINE Bus STARTING 2024-03-22T06:16:31.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:30 smithi006 ceph-mon[29301]: [22/Mar/2024:06:16:29] ENGINE Serving on https://172.21.15.6:7150 2024-03-22T06:16:31.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:30 smithi006 ceph-mon[29301]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-22T06:16:31.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:30 smithi006 ceph-mon[29301]: mgrmap e28: smithi006.sfpxdy(active, since 1.01074s) 2024-03-22T06:16:32.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:31 smithi117 ceph-mon[36514]: [22/Mar/2024:06:16:29] ENGINE Serving on http://172.21.15.6:8765 2024-03-22T06:16:32.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:31 smithi117 ceph-mon[36514]: [22/Mar/2024:06:16:29] ENGINE Bus STARTED 2024-03-22T06:16:32.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:31 smithi117 ceph-mon[36514]: pgmap v3: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:16:32.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:31 smithi117 ceph-mon[36514]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-22T06:16:32.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:31 smithi117 ceph-mon[36514]: Cluster is now healthy 2024-03-22T06:16:32.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:31 smithi117 ceph-mon[36514]: mgrmap e29: smithi006.sfpxdy(active, since 2s) 2024-03-22T06:16:32.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:31 smithi006 ceph-mon[29301]: [22/Mar/2024:06:16:29] ENGINE Serving on http://172.21.15.6:8765 2024-03-22T06:16:32.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:31 smithi006 ceph-mon[29301]: [22/Mar/2024:06:16:29] ENGINE Bus STARTED 2024-03-22T06:16:32.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:31 smithi006 ceph-mon[29301]: pgmap v3: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:16:32.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:31 smithi006 ceph-mon[29301]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-22T06:16:32.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:31 smithi006 ceph-mon[29301]: Cluster is now healthy 2024-03-22T06:16:32.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:31 smithi006 ceph-mon[29301]: mgrmap e29: smithi006.sfpxdy(active, since 2s) 2024-03-22T06:16:32.979 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:32 smithi117 ceph-mon[36514]: pgmap v4: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:16:33.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:32 smithi006 ceph-mon[29301]: pgmap v4: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:16:33.924 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:33 smithi117 ceph-mon[36514]: pgmap v5: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:16:33.925 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:33 smithi117 ceph-mon[36514]: mgrmap e30: smithi006.sfpxdy(active, since 4s) 2024-03-22T06:16:33.925 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:33 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:33.925 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:33 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:34.209 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:33 smithi006 ceph-mon[29301]: pgmap v5: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:16:34.210 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:33 smithi006 ceph-mon[29301]: mgrmap e30: smithi006.sfpxdy(active, since 4s) 2024-03-22T06:16:34.210 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:33 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:34.210 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:33 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:36.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:35 smithi006 ceph-mon[29301]: pgmap v6: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:16:36.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:35 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:36.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:35 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:36.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:35 smithi117 ceph-mon[36514]: pgmap v6: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:16:36.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:35 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:36.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:35 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:36.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.627+0000 7fc4aa460700 1 -- 172.21.15.6:0/27448878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40fce40 msgr2=0x7fc4a40fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:36.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.627+0000 7fc4aa460700 1 --2- 172.21.15.6:0/27448878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40fce40 0x7fc4a40fd250 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc494004940 tx=0x7fc494009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:36.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.627+0000 7fc4aa460700 1 -- 172.21.15.6:0/27448878 shutdown_connections 2024-03-22T06:16:36.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.627+0000 7fc4aa460700 1 --2- 172.21.15.6:0/27448878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a40fe040 0x7fc4a40fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:36.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.627+0000 7fc4aa460700 1 --2- 172.21.15.6:0/27448878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40fce40 0x7fc4a40fd250 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:36.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.627+0000 7fc4aa460700 1 -- 172.21.15.6:0/27448878 >> 172.21.15.6:0/27448878 conn(0x7fc4a40f8510 msgr2=0x7fc4a40fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:36.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.627+0000 7fc4aa460700 1 -- 172.21.15.6:0/27448878 shutdown_connections 2024-03-22T06:16:36.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.627+0000 7fc4aa460700 1 -- 172.21.15.6:0/27448878 wait complete. 2024-03-22T06:16:36.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.628+0000 7fc4aa460700 1 Processor -- start 2024-03-22T06:16:36.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.628+0000 7fc4aa460700 1 -- start start 2024-03-22T06:16:36.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.628+0000 7fc4aa460700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40fce40 0x7fc4a410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:36.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.628+0000 7fc4aa460700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a40fe040 0x7fc4a410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:36.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.628+0000 7fc4a3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40fce40 0x7fc4a410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:36.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.628+0000 7fc4aa460700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a410e4e0 con 0x7fc4a40fce40 2024-03-22T06:16:36.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.629+0000 7fc4aa460700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a410ed30 con 0x7fc4a40fe040 2024-03-22T06:16:36.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.629+0000 7fc4a3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40fce40 0x7fc4a410d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46234/0 (socket says 172.21.15.6:46234) 2024-03-22T06:16:36.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.629+0000 7fc4a3fff700 1 -- 172.21.15.6:0/1035544471 learned_addr learned my addr 172.21.15.6:0/1035544471 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:36.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.629+0000 7fc4a37fe700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a40fe040 0x7fc4a410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:36.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.629+0000 7fc4a3fff700 1 -- 172.21.15.6:0/1035544471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a40fe040 msgr2=0x7fc4a410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:36.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.629+0000 7fc4a3fff700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a40fe040 0x7fc4a410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:36.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.629+0000 7fc4a3fff700 1 -- 172.21.15.6:0/1035544471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494005040 con 0x7fc4a40fce40 2024-03-22T06:16:36.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.629+0000 7fc4a37fe700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a40fe040 0x7fc4a410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:16:36.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.629+0000 7fc4a3fff700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40fce40 0x7fc4a410d9a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc494000c00 tx=0x7fc494009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:36.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.629+0000 7fc4a0ff9700 1 -- 172.21.15.6:0/1035544471 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc49400e070 con 0x7fc4a40fce40 2024-03-22T06:16:36.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.630+0000 7fc4a0ff9700 1 -- 172.21.15.6:0/1035544471 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc49401c400 con 0x7fc4a40fce40 2024-03-22T06:16:36.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.630+0000 7fc4aa460700 1 -- 172.21.15.6:0/1035544471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a410efb0 con 0x7fc4a40fce40 2024-03-22T06:16:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.630+0000 7fc4a0ff9700 1 -- 172.21.15.6:0/1035544471 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc49400b650 con 0x7fc4a40fce40 2024-03-22T06:16:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.630+0000 7fc4aa460700 1 -- 172.21.15.6:0/1035544471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a410f490 con 0x7fc4a40fce40 2024-03-22T06:16:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.631+0000 7fc4a0ff9700 1 -- 172.21.15.6:0/1035544471 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 30) v1 ==== 46464+0+0 (secure 0 0 0) 0x7fc49400b7b0 con 0x7fc4a40fce40 2024-03-22T06:16:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.631+0000 7fc4a0ff9700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc48c0392e0 0x7fc48c03b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.631+0000 7fc4a0ff9700 1 -- 172.21.15.6:0/1035544471 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fc49404f780 con 0x7fc4a40fce40 2024-03-22T06:16:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.631+0000 7fc4aa460700 1 -- 172.21.15.6:0/1035544471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a405f5f0 con 0x7fc4a40fce40 2024-03-22T06:16:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.631+0000 7fc4a37fe700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc48c0392e0 0x7fc48c03b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:36.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.632+0000 7fc4a37fe700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc48c0392e0 0x7fc48c03b790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc498009b80 tx=0x7fc49800b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:36.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:36.638+0000 7fc4a0ff9700 1 -- 172.21.15.6:0/1035544471 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7fc49400bb50 con 0x7fc4a40fce40 2024-03-22T06:16:37.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.015+0000 7fc4aa460700 1 -- 172.21.15.6:0/1035544471 --> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4a410f7a0 con 0x7fc48c0392e0 2024-03-22T06:16:37.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.020+0000 7fc4a0ff9700 1 -- 172.21.15.6:0/1035544471 <== mgr.14750 v2:172.21.15.6:6800/3502710787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fc4a410f7a0 con 0x7fc48c0392e0 2024-03-22T06:16:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.023+0000 7fc4aa460700 1 -- 172.21.15.6:0/1035544471 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc48c0392e0 msgr2=0x7fc48c03b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.023+0000 7fc4aa460700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc48c0392e0 0x7fc48c03b790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc498009b80 tx=0x7fc49800b040 comp rx=0 tx=0).stop 2024-03-22T06:16:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.024+0000 7fc4aa460700 1 -- 172.21.15.6:0/1035544471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40fce40 msgr2=0x7fc4a410d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.024+0000 7fc4aa460700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40fce40 0x7fc4a410d9a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc494000c00 tx=0x7fc494009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:37.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.027+0000 7fc4aa460700 1 -- 172.21.15.6:0/1035544471 shutdown_connections 2024-03-22T06:16:37.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.028+0000 7fc4aa460700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc48c0392e0 0x7fc48c03b790 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.028+0000 7fc4aa460700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40fce40 0x7fc4a410d9a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.028+0000 7fc4aa460700 1 --2- 172.21.15.6:0/1035544471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a40fe040 0x7fc4a410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.028+0000 7fc4aa460700 1 -- 172.21.15.6:0/1035544471 >> 172.21.15.6:0/1035544471 conn(0x7fc4a40f8510 msgr2=0x7fc4a40f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.029+0000 7fc4aa460700 1 -- 172.21.15.6:0/1035544471 shutdown_connections 2024-03-22T06:16:37.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.029+0000 7fc4aa460700 1 -- 172.21.15.6:0/1035544471 wait complete. 2024-03-22T06:16:37.045 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:16:37.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.173+0000 7f8330502700 1 -- 172.21.15.6:0/2867561001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83280fff00 msgr2=0x7f8328102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:37.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.173+0000 7f8330502700 1 --2- 172.21.15.6:0/2867561001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83280fff00 0x7f8328102320 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8318004940 tx=0x7f8318009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:37.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.173+0000 7f8330502700 1 -- 172.21.15.6:0/2867561001 shutdown_connections 2024-03-22T06:16:37.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.173+0000 7f8330502700 1 --2- 172.21.15.6:0/2867561001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83280fff00 0x7f8328102320 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.173+0000 7f8330502700 1 --2- 172.21.15.6:0/2867561001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83280662e0 0x7f83280ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.173+0000 7f8330502700 1 -- 172.21.15.6:0/2867561001 >> 172.21.15.6:0/2867561001 conn(0x7f83280f7430 msgr2=0x7f83280f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:37.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.173+0000 7f8330502700 1 -- 172.21.15.6:0/2867561001 shutdown_connections 2024-03-22T06:16:37.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.173+0000 7f8330502700 1 -- 172.21.15.6:0/2867561001 wait complete. 2024-03-22T06:16:37.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.174+0000 7f8330502700 1 Processor -- start 2024-03-22T06:16:37.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.174+0000 7f8330502700 1 -- start start 2024-03-22T06:16:37.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.174+0000 7f8330502700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83280662e0 0x7f832810da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:37.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.174+0000 7f8330502700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83280fff00 0x7f832810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:37.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.174+0000 7f8330502700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832810e580 con 0x7f83280fff00 2024-03-22T06:16:37.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.174+0000 7f8330502700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832810ed80 con 0x7f83280662e0 2024-03-22T06:16:37.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.174+0000 7f832da9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83280fff00 0x7f832810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.174+0000 7f832e29e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83280662e0 0x7f832810da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.174+0000 7f832da9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83280fff00 0x7f832810df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46252/0 (socket says 172.21.15.6:46252) 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.174+0000 7f832e29e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83280662e0 0x7f832810da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:38910/0 (socket says 172.21.15.6:38910) 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.174+0000 7f832da9d700 1 -- 172.21.15.6:0/4262393929 learned_addr learned my addr 172.21.15.6:0/4262393929 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.175+0000 7f832da9d700 1 -- 172.21.15.6:0/4262393929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83280662e0 msgr2=0x7f832810da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.175+0000 7f832da9d700 1 --2- 172.21.15.6:0/4262393929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83280662e0 0x7f832810da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.175+0000 7f832da9d700 1 -- 172.21.15.6:0/4262393929 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8318005040 con 0x7f83280fff00 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.175+0000 7f832e29e700 1 --2- 172.21.15.6:0/4262393929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83280662e0 0x7f832810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.175+0000 7f832da9d700 1 --2- 172.21.15.6:0/4262393929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83280fff00 0x7f832810df80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8318013870 tx=0x7f8318009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.175+0000 7f831effd700 1 -- 172.21.15.6:0/4262393929 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f831800e070 con 0x7f83280fff00 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.175+0000 7f831effd700 1 -- 172.21.15.6:0/4262393929 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f831801a570 con 0x7f83280fff00 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.175+0000 7f831effd700 1 -- 172.21.15.6:0/4262393929 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f831801f070 con 0x7f83280fff00 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.175+0000 7f8330502700 1 -- 172.21.15.6:0/4262393929 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832810f000 con 0x7f83280fff00 2024-03-22T06:16:37.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.176+0000 7f8330502700 1 -- 172.21.15.6:0/4262393929 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832810f430 con 0x7f83280fff00 2024-03-22T06:16:37.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.176+0000 7f831effd700 1 -- 172.21.15.6:0/4262393929 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 30) v1 ==== 46464+0+0 (secure 0 0 0) 0x7f831802e3f0 con 0x7f83280fff00 2024-03-22T06:16:37.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.177+0000 7f831effd700 1 --2- 172.21.15.6:0/4262393929 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f8314039290 0x7f831403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:37.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.177+0000 7f8330502700 1 -- 172.21.15.6:0/4262393929 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832805f5f0 con 0x7f83280fff00 2024-03-22T06:16:37.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.177+0000 7f831effd700 1 -- 172.21.15.6:0/4262393929 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f8318052790 con 0x7f83280fff00 2024-03-22T06:16:37.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.177+0000 7f832e29e700 1 --2- 172.21.15.6:0/4262393929 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f8314039290 0x7f831403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:37.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.177+0000 7f832e29e700 1 --2- 172.21.15.6:0/4262393929 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f8314039290 0x7f831403b740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8324009b80 tx=0x7f832400b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:37.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.182+0000 7f831effd700 1 -- 172.21.15.6:0/4262393929 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f83180243d0 con 0x7f83280fff00 2024-03-22T06:16:37.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.498+0000 7f8330502700 1 -- 172.21.15.6:0/4262393929 --> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f832810f7d0 con 0x7f8314039290 2024-03-22T06:16:37.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.499+0000 7f831effd700 1 -- 172.21.15.6:0/4262393929 <== mgr.14750 v2:172.21.15.6:6800/3502710787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f832810f7d0 con 0x7f8314039290 2024-03-22T06:16:37.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.502+0000 7f8330502700 1 -- 172.21.15.6:0/4262393929 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f8314039290 msgr2=0x7f831403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:37.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.502+0000 7f8330502700 1 --2- 172.21.15.6:0/4262393929 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f8314039290 0x7f831403b740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8324009b80 tx=0x7f832400b040 comp rx=0 tx=0).stop 2024-03-22T06:16:37.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.502+0000 7f8330502700 1 -- 172.21.15.6:0/4262393929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83280fff00 msgr2=0x7f832810df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:37.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.502+0000 7f8330502700 1 --2- 172.21.15.6:0/4262393929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83280fff00 0x7f832810df80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8318013870 tx=0x7f8318009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.502+0000 7f8330502700 1 -- 172.21.15.6:0/4262393929 shutdown_connections 2024-03-22T06:16:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.502+0000 7f8330502700 1 --2- 172.21.15.6:0/4262393929 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f8314039290 0x7f831403b740 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.502+0000 7f8330502700 1 --2- 172.21.15.6:0/4262393929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83280662e0 0x7f832810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.502+0000 7f8330502700 1 --2- 172.21.15.6:0/4262393929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83280fff00 0x7f832810df80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.502+0000 7f8330502700 1 -- 172.21.15.6:0/4262393929 >> 172.21.15.6:0/4262393929 conn(0x7f83280f7430 msgr2=0x7f83280f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.502+0000 7f8330502700 1 -- 172.21.15.6:0/4262393929 shutdown_connections 2024-03-22T06:16:37.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.502+0000 7f8330502700 1 -- 172.21.15.6:0/4262393929 wait complete. 2024-03-22T06:16:37.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.637+0000 7f6898260700 1 -- 172.21.15.6:0/4108178981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68901046a0 msgr2=0x7f6890104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:37.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.637+0000 7f6898260700 1 --2- 172.21.15.6:0/4108178981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68901046a0 0x7f6890104b10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6880004940 tx=0x7f6880009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:37.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.637+0000 7f6898260700 1 -- 172.21.15.6:0/4108178981 shutdown_connections 2024-03-22T06:16:37.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.637+0000 7f6898260700 1 --2- 172.21.15.6:0/4108178981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68901046a0 0x7f6890104b10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.637+0000 7f6898260700 1 --2- 172.21.15.6:0/4108178981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68901033f0 0x7f6890103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.637+0000 7f6898260700 1 -- 172.21.15.6:0/4108178981 >> 172.21.15.6:0/4108178981 conn(0x7f6890072560 msgr2=0x7f68900749b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:37.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.638+0000 7f6898260700 1 -- 172.21.15.6:0/4108178981 shutdown_connections 2024-03-22T06:16:37.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.638+0000 7f6898260700 1 -- 172.21.15.6:0/4108178981 wait complete. 2024-03-22T06:16:37.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.639+0000 7f6898260700 1 Processor -- start 2024-03-22T06:16:37.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.639+0000 7f6898260700 1 -- start start 2024-03-22T06:16:37.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.639+0000 7f6898260700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68901033f0 0x7f6890113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:37.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.639+0000 7f6898260700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68901046a0 0x7f68901144c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:37.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.639+0000 7f6898260700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6890114ac0 con 0x7f68901033f0 2024-03-22T06:16:37.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.639+0000 7f6898260700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6890115310 con 0x7f68901046a0 2024-03-22T06:16:37.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.639+0000 7f6895ffc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68901033f0 0x7f6890113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:37.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f6895ffc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68901033f0 0x7f6890113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46270/0 (socket says 172.21.15.6:46270) 2024-03-22T06:16:37.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f6895ffc700 1 -- 172.21.15.6:0/3330633319 learned_addr learned my addr 172.21.15.6:0/3330633319 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:37.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f68957fb700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68901046a0 0x7f68901144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:37.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f6895ffc700 1 -- 172.21.15.6:0/3330633319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68901046a0 msgr2=0x7f68901144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:37.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f6895ffc700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68901046a0 0x7f68901144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f6895ffc700 1 -- 172.21.15.6:0/3330633319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6880005040 con 0x7f68901033f0 2024-03-22T06:16:37.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f68957fb700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68901046a0 0x7f68901144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:16:37.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f6895ffc700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68901033f0 0x7f6890113f80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f688c00eb40 tx=0x7f688c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:37.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f68867fc700 1 -- 172.21.15.6:0/3330633319 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f688c007cc0 con 0x7f68901033f0 2024-03-22T06:16:37.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f68867fc700 1 -- 172.21.15.6:0/3330633319 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f688c00f040 con 0x7f68901033f0 2024-03-22T06:16:37.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f6898260700 1 -- 172.21.15.6:0/3330633319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6890115590 con 0x7f68901033f0 2024-03-22T06:16:37.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f68867fc700 1 -- 172.21.15.6:0/3330633319 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f688c00ca10 con 0x7f68901033f0 2024-03-22T06:16:37.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.640+0000 7f6898260700 1 -- 172.21.15.6:0/3330633319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6890115ad0 con 0x7f68901033f0 2024-03-22T06:16:37.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.642+0000 7f68867fc700 1 -- 172.21.15.6:0/3330633319 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 30) v1 ==== 46464+0+0 (secure 0 0 0) 0x7f688c00cc80 con 0x7f68901033f0 2024-03-22T06:16:37.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.642+0000 7f6898260700 1 -- 172.21.15.6:0/3330633319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689005f5f0 con 0x7f68901033f0 2024-03-22T06:16:37.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.642+0000 7f68867fc700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f687c039290 0x7f687c03b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:37.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.642+0000 7f68867fc700 1 -- 172.21.15.6:0/3330633319 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f688c04cd20 con 0x7f68901033f0 2024-03-22T06:16:37.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.642+0000 7f68957fb700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f687c039290 0x7f687c03b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:37.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.643+0000 7f68957fb700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f687c039290 0x7f687c03b740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6880009b30 tx=0x7f6880009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:37.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.650+0000 7f68867fc700 1 -- 172.21.15.6:0/3330633319 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f688c012d60 con 0x7f68901033f0 2024-03-22T06:16:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.942+0000 7f68867fc700 1 -- 172.21.15.6:0/3330633319 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7f688c012340 con 0x7f68901033f0 2024-03-22T06:16:37.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.957+0000 7f6898260700 1 -- 172.21.15.6:0/3330633319 --> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6890115e50 con 0x7f687c039290 2024-03-22T06:16:37.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.976+0000 7f68867fc700 1 -- 172.21.15.6:0/3330633319 <== mgr.14750 v2:172.21.15.6:6800/3502710787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6890115e50 con 0x7f687c039290 2024-03-22T06:16:37.976 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (75s) 1s ago 9m 25.8M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (9m) 1s ago 9m 6656k - 17.2.7-726-g9c4e0c34 837124da804f a49ab95d42e6 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (8m) 4s ago 8m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0a36af1f8ed0 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 running (6s) 1s ago 9m - - 954c08fa6188 f192e538d520 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (4m) 1s ago 4m 226M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (4m) 1s ago 4m 16.4M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (4m) 4s ago 4m 13.7M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (4m) 4s ago 4m 15.2M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (26s) 1s ago 11m 479M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283 running (2m) 4s ago 8m 80.7M - 19.0.0-2299-gea01217f aa461e7c5e7f ff1047688325 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (11m) 1s ago 11m 52.7M 2048M 17.2.7-726-g9c4e0c34 837124da804f b2fe36c6b005 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (8m) 4s ago 8m 45.7M 2048M 17.2.7-726-g9c4e0c34 837124da804f 0559848da9ab 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (91s) 1s ago 9m 16.9M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (41s) 4s ago 8m 8238k - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (7m) 1s ago 7m 211M 1559M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (7m) 1s ago 7m 226M 1559M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:16:37.977 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (7m) 1s ago 7m 187M 1559M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:16:37.978 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (6m) 4s ago 6m 248M 2581M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:16:37.978 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (6m) 4s ago 6m 217M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:16:37.978 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (5m) 4s ago 5m 208M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:16:37.978 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (47s) 1s ago 9m 47.2M - 2.43.0 a07b618ecd1d 4d36b59051cf 2024-03-22T06:16:37.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.978+0000 7f6898260700 1 -- 172.21.15.6:0/3330633319 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f687c039290 msgr2=0x7f687c03b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:37.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.978+0000 7f6898260700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f687c039290 0x7f687c03b740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6880009b30 tx=0x7f6880009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:37.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.978+0000 7f6898260700 1 -- 172.21.15.6:0/3330633319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68901033f0 msgr2=0x7f6890113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:37.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.978+0000 7f6898260700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68901033f0 0x7f6890113f80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f688c00eb40 tx=0x7f688c00b040 comp rx=0 tx=0).stop 2024-03-22T06:16:37.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.979+0000 7f6898260700 1 -- 172.21.15.6:0/3330633319 shutdown_connections 2024-03-22T06:16:37.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.979+0000 7f6898260700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f687c039290 0x7f687c03b740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.979+0000 7f6898260700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68901033f0 0x7f6890113f80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.979+0000 7f6898260700 1 --2- 172.21.15.6:0/3330633319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68901046a0 0x7f68901144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:37.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.979+0000 7f6898260700 1 -- 172.21.15.6:0/3330633319 >> 172.21.15.6:0/3330633319 conn(0x7f6890072560 msgr2=0x7f6890073bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:37.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.979+0000 7f6898260700 1 -- 172.21.15.6:0/3330633319 shutdown_connections 2024-03-22T06:16:37.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:37.979+0000 7f6898260700 1 -- 172.21.15.6:0/3330633319 wait complete. 2024-03-22T06:16:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.089+0000 7f805cbc1700 1 -- 172.21.15.6:0/3447191877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80580fe0d0 msgr2=0x7f80580fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.089+0000 7f805cbc1700 1 --2- 172.21.15.6:0/3447191877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80580fe0d0 0x7f80580fe540 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8040004940 tx=0x7f8040009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:38.091 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:37 smithi006 ceph-mon[29301]: pgmap v7: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 59 KiB/s rd, 1.4 MiB/s wr, 375 op/s 2024-03-22T06:16:38.091 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:37 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:38.091 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:37 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:38.091 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:37 smithi006 ceph-mon[29301]: Standby manager daemon smithi117.bqeskk started 2024-03-22T06:16:38.091 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:37 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/2006176365' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/crt"}]: dispatch 2024-03-22T06:16:38.091 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:37 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/2006176365' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:16:38.091 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:37 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/2006176365' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/key"}]: dispatch 2024-03-22T06:16:38.091 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:37 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/2006176365' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:16:38.091 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:37 smithi006 ceph-mon[29301]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:38.091 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:37 smithi006 ceph-mon[29301]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:38.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.090+0000 7f805cbc1700 1 -- 172.21.15.6:0/3447191877 shutdown_connections 2024-03-22T06:16:38.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.090+0000 7f805cbc1700 1 --2- 172.21.15.6:0/3447191877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80580fe0d0 0x7f80580fe540 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:38.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.090+0000 7f805cbc1700 1 --2- 172.21.15.6:0/3447191877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80580fcee0 0x7f80580fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:38.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.090+0000 7f805cbc1700 1 -- 172.21.15.6:0/3447191877 >> 172.21.15.6:0/3447191877 conn(0x7f80580f8560 msgr2=0x7f80580fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:38.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.090+0000 7f805cbc1700 1 -- 172.21.15.6:0/3447191877 shutdown_connections 2024-03-22T06:16:38.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.090+0000 7f805cbc1700 1 -- 172.21.15.6:0/3447191877 wait complete. 2024-03-22T06:16:38.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.091+0000 7f805cbc1700 1 Processor -- start 2024-03-22T06:16:38.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.091+0000 7f805cbc1700 1 -- start start 2024-03-22T06:16:38.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.091+0000 7f805cbc1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80580fcee0 0x7f8058071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:38.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.091+0000 7f805cbc1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80580fe0d0 0x7f8058070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:38.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.091+0000 7f805659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80580fcee0 0x7f8058071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.091+0000 7f805659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80580fcee0 0x7f8058071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:38946/0 (socket says 172.21.15.6:38946) 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.091+0000 7f805659c700 1 -- 172.21.15.6:0/1191008438 learned_addr learned my addr 172.21.15.6:0/1191008438 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.091+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8058111680 con 0x7f80580fe0d0 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.091+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80581117c0 con 0x7f80580fcee0 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.091+0000 7f8055d9b700 1 --2- 172.21.15.6:0/1191008438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80580fe0d0 0x7f8058070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.092+0000 7f805659c700 1 -- 172.21.15.6:0/1191008438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80580fe0d0 msgr2=0x7f8058070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.092+0000 7f805659c700 1 --2- 172.21.15.6:0/1191008438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80580fe0d0 0x7f8058070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.092+0000 7f805659c700 1 -- 172.21.15.6:0/1191008438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040005040 con 0x7f80580fcee0 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.092+0000 7f805659c700 1 --2- 172.21.15.6:0/1191008438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80580fcee0 0x7f8058071df0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f804800ecb0 tx=0x7f804800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.092+0000 7f804effd700 1 -- 172.21.15.6:0/1191008438 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f804800ee60 con 0x7f80580fcee0 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.092+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8058070910 con 0x7f80580fcee0 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.092+0000 7f804effd700 1 -- 172.21.15.6:0/1191008438 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f804800f040 con 0x7f80580fcee0 2024-03-22T06:16:38.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.092+0000 7f804effd700 1 -- 172.21.15.6:0/1191008438 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8048009680 con 0x7f80580fcee0 2024-03-22T06:16:38.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.092+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8058070e70 con 0x7f80580fcee0 2024-03-22T06:16:38.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.094+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80380052e0 con 0x7f80580fcee0 2024-03-22T06:16:38.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.094+0000 7f804effd700 1 -- 172.21.15.6:0/1191008438 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7f8048014480 con 0x7f80580fcee0 2024-03-22T06:16:38.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.095+0000 7f804effd700 1 --2- 172.21.15.6:0/1191008438 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f804406e330 0x7f80440707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:38.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.095+0000 7f804effd700 1 -- 172.21.15.6:0/1191008438 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f804808dd60 con 0x7f80580fcee0 2024-03-22T06:16:38.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.095+0000 7f8055d9b700 1 --2- 172.21.15.6:0/1191008438 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f804406e330 0x7f80440707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:38.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.096+0000 7f8055d9b700 1 --2- 172.21.15.6:0/1191008438 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f804406e330 0x7f80440707e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f804000b0d0 tx=0x7f8040009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:38.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.104+0000 7f804effd700 1 -- 172.21.15.6:0/1191008438 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f80480574e0 con 0x7f80580fcee0 2024-03-22T06:16:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:37 smithi117 ceph-mon[36514]: pgmap v7: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 59 KiB/s rd, 1.4 MiB/s wr, 375 op/s 2024-03-22T06:16:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:37 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:37 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:37 smithi117 ceph-mon[36514]: Standby manager daemon smithi117.bqeskk started 2024-03-22T06:16:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:37 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2006176365' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/crt"}]: dispatch 2024-03-22T06:16:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:37 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2006176365' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:16:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:37 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2006176365' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/key"}]: dispatch 2024-03-22T06:16:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:37 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2006176365' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:16:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:37 smithi117 ceph-mon[36514]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:38.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:37 smithi117 ceph-mon[36514]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.510+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8038005860 con 0x7f80580fcee0 2024-03-22T06:16:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.511+0000 7f804effd700 1 -- 172.21.15.6:0/1191008438 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f804805aba0 con 0x7f80580fcee0 2024-03-22T06:16:38.512 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:16:38.512 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:16:38.512 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T06:16:38.512 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:16:38.512 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:16:38.512 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:16:38.512 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:16:38.512 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:16:38.512 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T06:16:38.512 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:16:38.513 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:16:38.513 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:16:38.513 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:16:38.513 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:16:38.513 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-22T06:16:38.513 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:16:38.513 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:16:38.513 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:16:38.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.514+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f804406e330 msgr2=0x7f80440707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:38.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.514+0000 7f805cbc1700 1 --2- 172.21.15.6:0/1191008438 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f804406e330 0x7f80440707e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f804000b0d0 tx=0x7f8040009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:38.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.514+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80580fcee0 msgr2=0x7f8058071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:38.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.514+0000 7f805cbc1700 1 --2- 172.21.15.6:0/1191008438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80580fcee0 0x7f8058071df0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f804800ecb0 tx=0x7f804800b040 comp rx=0 tx=0).stop 2024-03-22T06:16:38.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.514+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 shutdown_connections 2024-03-22T06:16:38.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.514+0000 7f805cbc1700 1 --2- 172.21.15.6:0/1191008438 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f804406e330 0x7f80440707e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:38.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.514+0000 7f805cbc1700 1 --2- 172.21.15.6:0/1191008438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80580fcee0 0x7f8058071df0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:38.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.514+0000 7f805cbc1700 1 --2- 172.21.15.6:0/1191008438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80580fe0d0 0x7f8058070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:38.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.514+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 >> 172.21.15.6:0/1191008438 conn(0x7f80580f8560 msgr2=0x7f80580f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:38.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.514+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 shutdown_connections 2024-03-22T06:16:38.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.514+0000 7f805cbc1700 1 -- 172.21.15.6:0/1191008438 wait complete. 2024-03-22T06:16:38.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.649+0000 7efe79499700 1 -- 172.21.15.6:0/205849973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe74104fc0 msgr2=0x7efe741073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:38.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.649+0000 7efe79499700 1 --2- 172.21.15.6:0/205849973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe74104fc0 0x7efe741073a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efe6c004940 tx=0x7efe6c009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:38.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.649+0000 7efe79499700 1 -- 172.21.15.6:0/205849973 shutdown_connections 2024-03-22T06:16:38.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.649+0000 7efe79499700 1 --2- 172.21.15.6:0/205849973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe741078e0 0x7efe74109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:38.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.649+0000 7efe79499700 1 --2- 172.21.15.6:0/205849973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe74104fc0 0x7efe741073a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:38.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.649+0000 7efe79499700 1 -- 172.21.15.6:0/205849973 >> 172.21.15.6:0/205849973 conn(0x7efe7406a380 msgr2=0x7efe7406c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:38.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.650+0000 7efe79499700 1 -- 172.21.15.6:0/205849973 shutdown_connections 2024-03-22T06:16:38.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.650+0000 7efe79499700 1 -- 172.21.15.6:0/205849973 wait complete. 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe79499700 1 Processor -- start 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe79499700 1 -- start start 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe79499700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe74104fc0 0x7efe74114fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe79499700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe741078e0 0x7efe74115500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe79499700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe74115b00 con 0x7efe741078e0 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe79499700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe74115c70 con 0x7efe74104fc0 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe727fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe741078e0 0x7efe74115500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe727fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe741078e0 0x7efe74115500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46308/0 (socket says 172.21.15.6:46308) 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe727fc700 1 -- 172.21.15.6:0/1420116446 learned_addr learned my addr 172.21.15.6:0/1420116446 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe72ffd700 1 --2- 172.21.15.6:0/1420116446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe74104fc0 0x7efe74114fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe727fc700 1 -- 172.21.15.6:0/1420116446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe74104fc0 msgr2=0x7efe74114fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe727fc700 1 --2- 172.21.15.6:0/1420116446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe74104fc0 0x7efe74114fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.651+0000 7efe727fc700 1 -- 172.21.15.6:0/1420116446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c005040 con 0x7efe741078e0 2024-03-22T06:16:38.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.652+0000 7efe727fc700 1 --2- 172.21.15.6:0/1420116446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe741078e0 0x7efe74115500 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efe6400ec30 tx=0x7efe6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:38.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.652+0000 7efe5b7fe700 1 -- 172.21.15.6:0/1420116446 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe64007b30 con 0x7efe741078e0 2024-03-22T06:16:38.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.652+0000 7efe79499700 1 -- 172.21.15.6:0/1420116446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7406eff0 con 0x7efe741078e0 2024-03-22T06:16:38.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.652+0000 7efe79499700 1 -- 172.21.15.6:0/1420116446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7406f530 con 0x7efe741078e0 2024-03-22T06:16:38.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.652+0000 7efe5b7fe700 1 -- 172.21.15.6:0/1420116446 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe6400f040 con 0x7efe741078e0 2024-03-22T06:16:38.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.652+0000 7efe5b7fe700 1 -- 172.21.15.6:0/1420116446 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe64009520 con 0x7efe741078e0 2024-03-22T06:16:38.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.654+0000 7efe79499700 1 -- 172.21.15.6:0/1420116446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe600052e0 con 0x7efe741078e0 2024-03-22T06:16:38.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.654+0000 7efe5b7fe700 1 -- 172.21.15.6:0/1420116446 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7efe64009680 con 0x7efe741078e0 2024-03-22T06:16:38.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.655+0000 7efe5b7fe700 1 --2- 172.21.15.6:0/1420116446 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7efe5c06e270 0x7efe5c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:38.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.655+0000 7efe5b7fe700 1 -- 172.21.15.6:0/1420116446 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7efe6408e060 con 0x7efe741078e0 2024-03-22T06:16:38.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.655+0000 7efe72ffd700 1 --2- 172.21.15.6:0/1420116446 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7efe5c06e270 0x7efe5c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:38.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.656+0000 7efe72ffd700 1 --2- 172.21.15.6:0/1420116446 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7efe5c06e270 0x7efe5c070720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009b30 tx=0x7efe6c009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:38.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:38.663+0000 7efe5b7fe700 1 -- 172.21.15.6:0/1420116446 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7efe6405b210 con 0x7efe741078e0 2024-03-22T06:16:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.092+0000 7efe79499700 1 -- 172.21.15.6:0/1420116446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efe60005860 con 0x7efe741078e0 2024-03-22T06:16:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.092+0000 7efe5b7fe700 1 -- 172.21.15.6:0/1420116446 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1805 (secure 0 0 0) 0x7efe6405ada0 con 0x7efe741078e0 2024-03-22T06:16:39.093 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:16:39.093 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:16:39.093 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:16:39.093 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:16:39.094 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:16:39.094 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:16:39.094 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:16:39.094 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:16:39.094 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:16:39.094 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:16:39.094 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:16:39.094 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:16:39.094 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:16:39.094 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:16:39.094 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:16:39.095 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:16:39.096 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:16:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.095+0000 7efe79499700 1 -- 172.21.15.6:0/1420116446 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7efe5c06e270 msgr2=0x7efe5c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.095+0000 7efe79499700 1 --2- 172.21.15.6:0/1420116446 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7efe5c06e270 0x7efe5c070720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009b30 tx=0x7efe6c009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.095+0000 7efe79499700 1 -- 172.21.15.6:0/1420116446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe741078e0 msgr2=0x7efe74115500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.095+0000 7efe79499700 1 --2- 172.21.15.6:0/1420116446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe741078e0 0x7efe74115500 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efe6400ec30 tx=0x7efe6400b040 comp rx=0 tx=0).stop 2024-03-22T06:16:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.095+0000 7efe79499700 1 -- 172.21.15.6:0/1420116446 shutdown_connections 2024-03-22T06:16:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.095+0000 7efe79499700 1 --2- 172.21.15.6:0/1420116446 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7efe5c06e270 0x7efe5c070720 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.095+0000 7efe79499700 1 --2- 172.21.15.6:0/1420116446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe74104fc0 0x7efe74114fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.095+0000 7efe79499700 1 --2- 172.21.15.6:0/1420116446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe741078e0 0x7efe74115500 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.095+0000 7efe79499700 1 -- 172.21.15.6:0/1420116446 >> 172.21.15.6:0/1420116446 conn(0x7efe7406a380 msgr2=0x7efe74107ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.095+0000 7efe79499700 1 -- 172.21.15.6:0/1420116446 shutdown_connections 2024-03-22T06:16:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.095+0000 7efe79499700 1 -- 172.21.15.6:0/1420116446 wait complete. 2024-03-22T06:16:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:16:39.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.249+0000 7f783331e700 1 -- 172.21.15.6:0/2047558076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f782c0fce80 msgr2=0x7f782c0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:39.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.249+0000 7f783331e700 1 --2- 172.21.15.6:0/2047558076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f782c0fce80 0x7f782c0fd290 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7828004940 tx=0x7f7828009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:39.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.249+0000 7f783331e700 1 -- 172.21.15.6:0/2047558076 shutdown_connections 2024-03-22T06:16:39.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.249+0000 7f783331e700 1 --2- 172.21.15.6:0/2047558076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782c0fdff0 0x7f782c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.249+0000 7f783331e700 1 --2- 172.21.15.6:0/2047558076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f782c0fce80 0x7f782c0fd290 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.249+0000 7f783331e700 1 -- 172.21.15.6:0/2047558076 >> 172.21.15.6:0/2047558076 conn(0x7f782c0f8580 msgr2=0x7f782c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:39.250 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:38 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi117.bqeskk", "id": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:16:39.251 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:38 smithi006 ceph-mon[29301]: mgrmap e31: smithi006.sfpxdy(active, since 9s), standbys: smithi117.bqeskk 2024-03-22T06:16:39.251 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:38 smithi006 ceph-mon[29301]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:39.251 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:38 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:39.251 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:38 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:39.251 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:38 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1191008438' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:16:39.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.249+0000 7f783331e700 1 -- 172.21.15.6:0/2047558076 shutdown_connections 2024-03-22T06:16:39.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.249+0000 7f783331e700 1 -- 172.21.15.6:0/2047558076 wait complete. 2024-03-22T06:16:39.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.250+0000 7f783331e700 1 Processor -- start 2024-03-22T06:16:39.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.250+0000 7f783331e700 1 -- start start 2024-03-22T06:16:39.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.250+0000 7f783331e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f782c0fce80 0x7f782c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:39.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.250+0000 7f783331e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782c0fdff0 0x7f782c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.250+0000 7f783331e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0708c0 con 0x7f782c0fce80 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.250+0000 7f783331e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c070a30 con 0x7f782c0fdff0 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.250+0000 7f78310ba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f782c0fce80 0x7f782c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.250+0000 7f78310ba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f782c0fce80 0x7f782c071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46328/0 (socket says 172.21.15.6:46328) 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.250+0000 7f78310ba700 1 -- 172.21.15.6:0/285089430 learned_addr learned my addr 172.21.15.6:0/285089430 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.250+0000 7f78308b9700 1 --2- 172.21.15.6:0/285089430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782c0fdff0 0x7f782c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.251+0000 7f78310ba700 1 -- 172.21.15.6:0/285089430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782c0fdff0 msgr2=0x7f782c070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.251+0000 7f78310ba700 1 --2- 172.21.15.6:0/285089430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782c0fdff0 0x7f782c070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.251+0000 7f78310ba700 1 -- 172.21.15.6:0/285089430 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7828005040 con 0x7f782c0fce80 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.251+0000 7f78310ba700 1 --2- 172.21.15.6:0/285089430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f782c0fce80 0x7f782c071e40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f782800b0d0 tx=0x7f7828009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.251+0000 7f7821ffb700 1 -- 172.21.15.6:0/285089430 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f782800e030 con 0x7f782c0fce80 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.251+0000 7f783331e700 1 -- 172.21.15.6:0/285089430 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782c070cb0 con 0x7f782c0fce80 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.251+0000 7f7821ffb700 1 -- 172.21.15.6:0/285089430 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f782801c400 con 0x7f782c0fce80 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.251+0000 7f7821ffb700 1 -- 172.21.15.6:0/285089430 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f782800b630 con 0x7f782c0fce80 2024-03-22T06:16:39.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.251+0000 7f783331e700 1 -- 172.21.15.6:0/285089430 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c1159d0 con 0x7f782c0fce80 2024-03-22T06:16:39.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.253+0000 7f783331e700 1 -- 172.21.15.6:0/285089430 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782c05f5f0 con 0x7f782c0fce80 2024-03-22T06:16:39.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.253+0000 7f7821ffb700 1 -- 172.21.15.6:0/285089430 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7f782801c570 con 0x7f782c0fce80 2024-03-22T06:16:39.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.254+0000 7f7821ffb700 1 --2- 172.21.15.6:0/285089430 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f781806e260 0x7f7818070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:39.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.254+0000 7f7821ffb700 1 -- 172.21.15.6:0/285089430 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f782808f450 con 0x7f782c0fce80 2024-03-22T06:16:39.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.254+0000 7f78308b9700 1 --2- 172.21.15.6:0/285089430 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f781806e260 0x7f7818070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:39.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.255+0000 7f78308b9700 1 --2- 172.21.15.6:0/285089430 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f781806e260 0x7f7818070710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f781c004fd0 tx=0x7f781c00b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:39.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:38 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi117.bqeskk", "id": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:16:39.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:38 smithi117 ceph-mon[36514]: mgrmap e31: smithi006.sfpxdy(active, since 9s), standbys: smithi117.bqeskk 2024-03-22T06:16:39.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:38 smithi117 ceph-mon[36514]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:39.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:38 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:39.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:38 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:39.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:38 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1191008438' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:16:39.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.263+0000 7f7821ffb700 1 -- 172.21.15.6:0/285089430 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f7828058bf0 con 0x7f782c0fce80 2024-03-22T06:16:39.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.542+0000 7f783331e700 1 -- 172.21.15.6:0/285089430 --> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f782c102310 con 0x7f781806e260 2024-03-22T06:16:39.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.544+0000 7f7821ffb700 1 -- 172.21.15.6:0/285089430 <== mgr.14750 v2:172.21.15.6:6800/3502710787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f782c102310 con 0x7f781806e260 2024-03-22T06:16:39.544 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:16:39.544 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:16:39.544 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:16:39.544 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:16:39.544 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:16:39.544 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:16:39.545 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:16:39.545 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "2/21 daemons upgraded", 2024-03-22T06:16:39.545 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "", 2024-03-22T06:16:39.545 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:16:39.545 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:16:39.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.546+0000 7f783331e700 1 -- 172.21.15.6:0/285089430 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f781806e260 msgr2=0x7f7818070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:39.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.546+0000 7f783331e700 1 --2- 172.21.15.6:0/285089430 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f781806e260 0x7f7818070710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f781c004fd0 tx=0x7f781c00b040 comp rx=0 tx=0).stop 2024-03-22T06:16:39.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.546+0000 7f783331e700 1 -- 172.21.15.6:0/285089430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f782c0fce80 msgr2=0x7f782c071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:39.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.546+0000 7f783331e700 1 --2- 172.21.15.6:0/285089430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f782c0fce80 0x7f782c071e40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f782800b0d0 tx=0x7f7828009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:39.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.546+0000 7f783331e700 1 -- 172.21.15.6:0/285089430 shutdown_connections 2024-03-22T06:16:39.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.546+0000 7f783331e700 1 --2- 172.21.15.6:0/285089430 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f781806e260 0x7f7818070710 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.546+0000 7f783331e700 1 --2- 172.21.15.6:0/285089430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f782c0fce80 0x7f782c071e40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.546+0000 7f783331e700 1 --2- 172.21.15.6:0/285089430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782c0fdff0 0x7f782c070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.546+0000 7f783331e700 1 -- 172.21.15.6:0/285089430 >> 172.21.15.6:0/285089430 conn(0x7f782c0f8580 msgr2=0x7f782c0f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:39.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.546+0000 7f783331e700 1 -- 172.21.15.6:0/285089430 shutdown_connections 2024-03-22T06:16:39.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.546+0000 7f783331e700 1 -- 172.21.15.6:0/285089430 wait complete. 2024-03-22T06:16:39.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.683+0000 7f480be45700 1 -- 172.21.15.6:0/2436813971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48040fe060 msgr2=0x7f48040fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:39.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.683+0000 7f480be45700 1 --2- 172.21.15.6:0/2436813971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48040fe060 0x7f48040fe4d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f47f4004940 tx=0x7f47f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:39.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.683+0000 7f480be45700 1 -- 172.21.15.6:0/2436813971 shutdown_connections 2024-03-22T06:16:39.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.683+0000 7f480be45700 1 --2- 172.21.15.6:0/2436813971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48040fe060 0x7f48040fe4d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.683+0000 7f480be45700 1 --2- 172.21.15.6:0/2436813971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48040fcef0 0x7f48040fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.683+0000 7f480be45700 1 -- 172.21.15.6:0/2436813971 >> 172.21.15.6:0/2436813971 conn(0x7f48040f85b0 msgr2=0x7f48040fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:39.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.683+0000 7f480be45700 1 -- 172.21.15.6:0/2436813971 shutdown_connections 2024-03-22T06:16:39.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.683+0000 7f480be45700 1 -- 172.21.15.6:0/2436813971 wait complete. 2024-03-22T06:16:39.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f480be45700 1 Processor -- start 2024-03-22T06:16:39.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f480be45700 1 -- start start 2024-03-22T06:16:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f480be45700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48040fcef0 0x7f480410fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f480be45700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48040fe060 0x7f4804110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f480be45700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4804110760 con 0x7f48040fe060 2024-03-22T06:16:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f480be45700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4804110fb0 con 0x7f48040fcef0 2024-03-22T06:16:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f48093e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48040fe060 0x7f4804110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f48093e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48040fe060 0x7f4804110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46334/0 (socket says 172.21.15.6:46334) 2024-03-22T06:16:39.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f48093e0700 1 -- 172.21.15.6:0/377103954 learned_addr learned my addr 172.21.15.6:0/377103954 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:16:39.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f4809be1700 1 --2- 172.21.15.6:0/377103954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48040fcef0 0x7f480410fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:39.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f48093e0700 1 -- 172.21.15.6:0/377103954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48040fcef0 msgr2=0x7f480410fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:39.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f48093e0700 1 --2- 172.21.15.6:0/377103954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48040fcef0 0x7f480410fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:39.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.684+0000 7f48093e0700 1 -- 172.21.15.6:0/377103954 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f4005040 con 0x7f48040fe060 2024-03-22T06:16:39.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.685+0000 7f48093e0700 1 --2- 172.21.15.6:0/377103954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48040fe060 0x7f4804110160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f47f400b970 tx=0x7f47f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:39.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.685+0000 7f47fa7fc700 1 -- 172.21.15.6:0/377103954 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f400e040 con 0x7f48040fe060 2024-03-22T06:16:39.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.685+0000 7f480be45700 1 -- 172.21.15.6:0/377103954 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4804111230 con 0x7f48040fe060 2024-03-22T06:16:39.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.685+0000 7f47fa7fc700 1 -- 172.21.15.6:0/377103954 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47f400bbf0 con 0x7f48040fe060 2024-03-22T06:16:39.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.685+0000 7f47fa7fc700 1 -- 172.21.15.6:0/377103954 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f401c630 con 0x7f48040fe060 2024-03-22T06:16:39.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.685+0000 7f480be45700 1 -- 172.21.15.6:0/377103954 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4804111770 con 0x7f48040fe060 2024-03-22T06:16:39.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.686+0000 7f480be45700 1 -- 172.21.15.6:0/377103954 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480405f5f0 con 0x7f48040fe060 2024-03-22T06:16:39.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.687+0000 7f47fa7fc700 1 -- 172.21.15.6:0/377103954 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7f47f400bd60 con 0x7f48040fe060 2024-03-22T06:16:39.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.687+0000 7f47fa7fc700 1 --2- 172.21.15.6:0/377103954 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f47f006e260 0x7f47f0070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:16:39.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.688+0000 7f47fa7fc700 1 -- 172.21.15.6:0/377103954 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f47f408eea0 con 0x7f48040fe060 2024-03-22T06:16:39.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.688+0000 7f4809be1700 1 --2- 172.21.15.6:0/377103954 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f47f006e260 0x7f47f0070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:16:39.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.688+0000 7f4809be1700 1 --2- 172.21.15.6:0/377103954 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f47f006e260 0x7f47f0070710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4800004f40 tx=0x7f480000b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:16:39.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:39.695+0000 7f47fa7fc700 1 -- 172.21.15.6:0/377103954 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f47f405c170 con 0x7f48040fe060 2024-03-22T06:16:40.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.252+0000 7f480be45700 1 -- 172.21.15.6:0/377103954 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4804070f60 con 0x7f48040fe060 2024-03-22T06:16:40.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.254+0000 7f47fa7fc700 1 -- 172.21.15.6:0/377103954 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f47f405bd00 con 0x7f48040fe060 2024-03-22T06:16:40.255 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:39 smithi006 ceph-mon[29301]: pgmap v8: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.1 MiB/s wr, 291 op/s 2024-03-22T06:16:40.255 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:39 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1420116446' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:16:40.255 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:39 smithi006 ceph-mon[29301]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:40.255 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_OK 2024-03-22T06:16:40.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:39 smithi117 ceph-mon[36514]: pgmap v8: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.1 MiB/s wr, 291 op/s 2024-03-22T06:16:40.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:39 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1420116446' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:16:40.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:39 smithi117 ceph-mon[36514]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:16:40.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.258+0000 7f480be45700 1 -- 172.21.15.6:0/377103954 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f47f006e260 msgr2=0x7f47f0070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:40.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.258+0000 7f480be45700 1 --2- 172.21.15.6:0/377103954 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f47f006e260 0x7f47f0070710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4800004f40 tx=0x7f480000b040 comp rx=0 tx=0).stop 2024-03-22T06:16:40.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.258+0000 7f480be45700 1 -- 172.21.15.6:0/377103954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48040fe060 msgr2=0x7f4804110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:16:40.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.258+0000 7f480be45700 1 --2- 172.21.15.6:0/377103954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48040fe060 0x7f4804110160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f47f400b970 tx=0x7f47f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:16:40.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.258+0000 7f480be45700 1 -- 172.21.15.6:0/377103954 shutdown_connections 2024-03-22T06:16:40.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.258+0000 7f480be45700 1 --2- 172.21.15.6:0/377103954 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f47f006e260 0x7f47f0070710 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:40.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.258+0000 7f480be45700 1 --2- 172.21.15.6:0/377103954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48040fcef0 0x7f480410fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:40.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.258+0000 7f480be45700 1 --2- 172.21.15.6:0/377103954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48040fe060 0x7f4804110160 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:16:40.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.258+0000 7f480be45700 1 -- 172.21.15.6:0/377103954 >> 172.21.15.6:0/377103954 conn(0x7f48040f85b0 msgr2=0x7f48040f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:16:40.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.258+0000 7f480be45700 1 -- 172.21.15.6:0/377103954 shutdown_connections 2024-03-22T06:16:40.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:16:40.258+0000 7f480be45700 1 -- 172.21.15.6:0/377103954 wait complete. 2024-03-22T06:16:41.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:40 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/377103954' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:16:41.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:40 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/377103954' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:16:42.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:41 smithi117 ceph-mon[36514]: pgmap v9: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 1.7 MiB/s wr, 924 op/s 2024-03-22T06:16:42.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:41 smithi006 ceph-mon[29301]: pgmap v9: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 1.7 MiB/s wr, 924 op/s 2024-03-22T06:16:44.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:43 smithi117 ceph-mon[36514]: pgmap v10: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.5 MiB/s wr, 839 op/s 2024-03-22T06:16:44.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:43 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:16:44.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:43 smithi006 ceph-mon[29301]: pgmap v10: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.5 MiB/s wr, 839 op/s 2024-03-22T06:16:44.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:43 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:16:46.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:45 smithi117 ceph-mon[36514]: pgmap v11: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.5 MiB/s wr, 839 op/s 2024-03-22T06:16:46.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:45 smithi006 ceph-mon[29301]: pgmap v11: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.5 MiB/s wr, 839 op/s 2024-03-22T06:16:48.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:47 smithi117 ceph-mon[36514]: pgmap v12: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 2.2 MiB/s wr, 1.37k op/s 2024-03-22T06:16:48.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:47 smithi006 ceph-mon[29301]: pgmap v12: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 2.2 MiB/s wr, 1.37k op/s 2024-03-22T06:16:50.183 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:49 smithi006 ceph-mon[29301]: pgmap v13: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 1.15k op/s 2024-03-22T06:16:50.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:49 smithi117 ceph-mon[36514]: pgmap v13: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 1.15k op/s 2024-03-22T06:16:52.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:51 smithi117 ceph-mon[36514]: pgmap v14: 65 pgs: 65 active+clean; 262 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 2.0 MiB/s wr, 1.29k op/s 2024-03-22T06:16:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:51 smithi006 ceph-mon[29301]: pgmap v14: 65 pgs: 65 active+clean; 262 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 2.0 MiB/s wr, 1.29k op/s 2024-03-22T06:16:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:53 smithi117 ceph-mon[36514]: pgmap v15: 65 pgs: 65 active+clean; 262 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 663 op/s 2024-03-22T06:16:54.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:53 smithi006 ceph-mon[29301]: pgmap v15: 65 pgs: 65 active+clean; 262 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 663 op/s 2024-03-22T06:16:56.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:55 smithi117 ceph-mon[36514]: pgmap v16: 65 pgs: 65 active+clean; 262 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 663 op/s 2024-03-22T06:16:56.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:55 smithi006 ceph-mon[29301]: pgmap v16: 65 pgs: 65 active+clean; 262 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 663 op/s 2024-03-22T06:16:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:57 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:57 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:57 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:16:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:57 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:16:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:57 smithi117 ceph-mon[36514]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-22T06:16:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:57 smithi117 ceph-mon[36514]: Updating smithi117:/etc/ceph/ceph.conf 2024-03-22T06:16:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:57 smithi117 ceph-mon[36514]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:16:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:57 smithi117 ceph-mon[36514]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:16:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:57 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:57 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:57 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:16:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:57 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:16:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:57 smithi006 ceph-mon[29301]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-22T06:16:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:57 smithi006 ceph-mon[29301]: Updating smithi117:/etc/ceph/ceph.conf 2024-03-22T06:16:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:57 smithi006 ceph-mon[29301]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:16:57.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:57 smithi006 ceph-mon[29301]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:16:58.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:58 smithi006 ceph-mon[29301]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:16:58.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:58 smithi006 ceph-mon[29301]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:16:58.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:58 smithi006 ceph-mon[29301]: pgmap v17: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 1.9 MiB/s wr, 812 op/s 2024-03-22T06:16:58.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:58 smithi006 ceph-mon[29301]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:16:58.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:58 smithi006 ceph-mon[29301]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:16:58.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:58 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:58.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:58 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:58.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:58 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:58.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:58 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:58.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:58 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:58.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:58 smithi006 ceph-mon[29301]: Reconfiguring prometheus.smithi006 (dependencies changed)... 2024-03-22T06:16:58.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:58 smithi117 ceph-mon[36514]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:16:58.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:58 smithi117 ceph-mon[36514]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:16:58.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:58 smithi117 ceph-mon[36514]: pgmap v17: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 1.9 MiB/s wr, 812 op/s 2024-03-22T06:16:58.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:58 smithi117 ceph-mon[36514]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:16:58.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:58 smithi117 ceph-mon[36514]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:16:58.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:58 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:58.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:58 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:58.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:58 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:58.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:58 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:58.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:58 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:16:58.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:58 smithi117 ceph-mon[36514]: Reconfiguring prometheus.smithi006 (dependencies changed)... 2024-03-22T06:16:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:59 smithi006 ceph-mon[29301]: Reconfiguring daemon prometheus.smithi006 on smithi006 2024-03-22T06:16:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:16:59 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:16:59.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:59 smithi117 ceph-mon[36514]: Reconfiguring daemon prometheus.smithi006 on smithi006 2024-03-22T06:16:59.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:16:59 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:17:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:00 smithi006 ceph-mon[29301]: pgmap v18: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 286 op/s 2024-03-22T06:17:00.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:00 smithi117 ceph-mon[36514]: pgmap v18: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 286 op/s 2024-03-22T06:17:02.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:02 smithi006 ceph-mon[29301]: pgmap v19: 65 pgs: 65 active+clean; 243 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.9 MiB/s wr, 548 op/s 2024-03-22T06:17:02.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:02 smithi117 ceph-mon[36514]: pgmap v19: 65 pgs: 65 active+clean; 243 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.9 MiB/s wr, 548 op/s 2024-03-22T06:17:04.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:04 smithi006 ceph-mon[29301]: pgmap v20: 65 pgs: 65 active+clean; 243 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.4 MiB/s wr, 411 op/s 2024-03-22T06:17:04.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:04 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:04.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:04 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:04.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:04 smithi117 ceph-mon[36514]: pgmap v20: 65 pgs: 65 active+clean; 243 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.4 MiB/s wr, 411 op/s 2024-03-22T06:17:04.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:04 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:04.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:04 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:05.500 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:05 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:17:05.500 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:05 smithi117 ceph-mon[36514]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:17:05.500 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:05 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:05.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:05 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:17:05.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:05 smithi006 ceph-mon[29301]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:17:05.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:05 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:06.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:06 smithi117 ceph-mon[36514]: pgmap v21: 65 pgs: 65 active+clean; 243 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.4 MiB/s wr, 411 op/s 2024-03-22T06:17:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:06 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:17:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:06 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:17:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:06 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:17:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:06 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:17:06.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:06 smithi006 ceph-mon[29301]: pgmap v21: 65 pgs: 65 active+clean; 243 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.4 MiB/s wr, 411 op/s 2024-03-22T06:17:06.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:06 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:17:06.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:06 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:17:06.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:06 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:17:06.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:06 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:17:07.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:07 smithi117 ceph-mon[36514]: Upgrade: Updating mgr.smithi117.bqeskk 2024-03-22T06:17:07.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:07 smithi117 ceph-mon[36514]: Deploying daemon mgr.smithi117.bqeskk on smithi117 2024-03-22T06:17:07.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:07 smithi006 ceph-mon[29301]: Upgrade: Updating mgr.smithi117.bqeskk 2024-03-22T06:17:07.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:07 smithi006 ceph-mon[29301]: Deploying daemon mgr.smithi117.bqeskk on smithi117 2024-03-22T06:17:08.469 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:08 smithi117 ceph-mon[36514]: pgmap v22: 65 pgs: 65 active+clean; 232 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 2.1 MiB/s wr, 742 op/s 2024-03-22T06:17:08.592 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:08 smithi006 ceph-mon[29301]: pgmap v22: 65 pgs: 65 active+clean; 232 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 2.1 MiB/s wr, 742 op/s 2024-03-22T06:17:08.865 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-22T06:17:08.865 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-22T06:17:09.220 DEBUG:teuthology.parallel:result is None 2024-03-22T06:17:10.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.430+0000 7f7c05cba700 1 -- 172.21.15.6:0/3387209468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c000fd410 msgr2=0x7f7c000fd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:10.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.430+0000 7f7c05cba700 1 --2- 172.21.15.6:0/3387209468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c000fd410 0x7f7c000fd820 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0004940 tx=0x7f7bf0009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:10.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.430+0000 7f7c05cba700 1 -- 172.21.15.6:0/3387209468 shutdown_connections 2024-03-22T06:17:10.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.430+0000 7f7c05cba700 1 --2- 172.21.15.6:0/3387209468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c000fdd60 0x7f7c001046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:10.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.430+0000 7f7c05cba700 1 --2- 172.21.15.6:0/3387209468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c000fd410 0x7f7c000fd820 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:10.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.430+0000 7f7c05cba700 1 -- 172.21.15.6:0/3387209468 >> 172.21.15.6:0/3387209468 conn(0x7f7c000f6ba0 msgr2=0x7f7c000f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:10.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.431+0000 7f7c05cba700 1 -- 172.21.15.6:0/3387209468 shutdown_connections 2024-03-22T06:17:10.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.436+0000 7f7c05cba700 1 -- 172.21.15.6:0/3387209468 wait complete. 2024-03-22T06:17:10.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.436+0000 7f7c05cba700 1 Processor -- start 2024-03-22T06:17:10.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.437+0000 7f7c05cba700 1 -- start start 2024-03-22T06:17:10.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.437+0000 7f7c05cba700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c000fd410 0x7f7c0010fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:10.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.438+0000 7f7c05cba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c000fdd60 0x7f7c001101d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:10.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.438+0000 7f7c05cba700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c00110860 con 0x7f7c000fdd60 2024-03-22T06:17:10.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.438+0000 7f7bff7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c000fd410 0x7f7c0010fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:10.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.438+0000 7f7bff7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c000fd410 0x7f7c0010fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:37582/0 (socket says 172.21.15.6:37582) 2024-03-22T06:17:10.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.438+0000 7f7bff7fe700 1 -- 172.21.15.6:0/1786132115 learned_addr learned my addr 172.21.15.6:0/1786132115 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:10.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.438+0000 7f7bf7fff700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c000fdd60 0x7f7c001101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:10.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.438+0000 7f7c05cba700 1 -- 172.21.15.6:0/1786132115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c00111000 con 0x7f7c000fd410 2024-03-22T06:17:10.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.439+0000 7f7bff7fe700 1 -- 172.21.15.6:0/1786132115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c000fdd60 msgr2=0x7f7c001101d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:10.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.439+0000 7f7bff7fe700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c000fdd60 0x7f7c001101d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:10.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.439+0000 7f7bff7fe700 1 -- 172.21.15.6:0/1786132115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf0005040 con 0x7f7c000fd410 2024-03-22T06:17:10.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.439+0000 7f7bf7fff700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c000fdd60 0x7f7c001101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:17:10.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.441+0000 7f7bff7fe700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c000fd410 0x7f7c0010fc90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000b970 tx=0x7f7bf0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:10.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.442+0000 7f7bfcff9700 1 -- 172.21.15.6:0/1786132115 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf000e030 con 0x7f7c000fd410 2024-03-22T06:17:10.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.443+0000 7f7bfcff9700 1 -- 172.21.15.6:0/1786132115 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bf0009430 con 0x7f7c000fd410 2024-03-22T06:17:10.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.443+0000 7f7bfcff9700 1 -- 172.21.15.6:0/1786132115 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf001c400 con 0x7f7c000fd410 2024-03-22T06:17:10.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.443+0000 7f7c05cba700 1 -- 172.21.15.6:0/1786132115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c00111280 con 0x7f7c000fd410 2024-03-22T06:17:10.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.444+0000 7f7c05cba700 1 -- 172.21.15.6:0/1786132115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c00111720 con 0x7f7c000fd410 2024-03-22T06:17:10.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.445+0000 7f7c05cba700 1 -- 172.21.15.6:0/1786132115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0005f5f0 con 0x7f7c000fd410 2024-03-22T06:17:10.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.446+0000 7f7bfcff9700 1 -- 172.21.15.6:0/1786132115 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7f7bf0013420 con 0x7f7c000fd410 2024-03-22T06:17:10.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.447+0000 7f7bfcff9700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f7bec06e440 0x7f7bec0708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:10.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.447+0000 7f7bfcff9700 1 -- 172.21.15.6:0/1786132115 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f7bf008edc0 con 0x7f7c000fd410 2024-03-22T06:17:10.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.447+0000 7f7bf7fff700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f7bec06e440 0x7f7bec0708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:10.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.449+0000 7f7bf7fff700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f7bec06e440 0x7f7bec0708f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009fd0 tx=0x7f7be80091a0 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:10.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.455+0000 7f7bfcff9700 1 -- 172.21.15.6:0/1786132115 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f7bf005bfd0 con 0x7f7c000fd410 2024-03-22T06:17:10.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:10 smithi117 ceph-mon[36514]: pgmap v23: 65 pgs: 65 active+clean; 232 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 592 op/s 2024-03-22T06:17:10.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:10 smithi006 ceph-mon[29301]: pgmap v23: 65 pgs: 65 active+clean; 232 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 592 op/s 2024-03-22T06:17:10.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.792+0000 7f7c05cba700 1 -- 172.21.15.6:0/1786132115 --> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c00111ad0 con 0x7f7bec06e440 2024-03-22T06:17:10.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.793+0000 7f7bfcff9700 1 -- 172.21.15.6:0/1786132115 <== mgr.14750 v2:172.21.15.6:6800/3502710787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f7c00111ad0 con 0x7f7bec06e440 2024-03-22T06:17:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.796+0000 7f7c05cba700 1 -- 172.21.15.6:0/1786132115 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f7bec06e440 msgr2=0x7f7bec0708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.796+0000 7f7c05cba700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f7bec06e440 0x7f7bec0708f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009fd0 tx=0x7f7be80091a0 comp rx=0 tx=0).stop 2024-03-22T06:17:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.796+0000 7f7c05cba700 1 -- 172.21.15.6:0/1786132115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c000fd410 msgr2=0x7f7c0010fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.796+0000 7f7c05cba700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c000fd410 0x7f7c0010fc90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000b970 tx=0x7f7bf0009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.796+0000 7f7c05cba700 1 -- 172.21.15.6:0/1786132115 shutdown_connections 2024-03-22T06:17:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.796+0000 7f7c05cba700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f7bec06e440 0x7f7bec0708f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.796+0000 7f7c05cba700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c000fd410 0x7f7c0010fc90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.796+0000 7f7c05cba700 1 --2- 172.21.15.6:0/1786132115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c000fdd60 0x7f7c001101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.796+0000 7f7c05cba700 1 -- 172.21.15.6:0/1786132115 >> 172.21.15.6:0/1786132115 conn(0x7f7c000f6ba0 msgr2=0x7f7c000f6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.796+0000 7f7c05cba700 1 -- 172.21.15.6:0/1786132115 shutdown_connections 2024-03-22T06:17:10.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.796+0000 7f7c05cba700 1 -- 172.21.15.6:0/1786132115 wait complete. 2024-03-22T06:17:10.807 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:17:10.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.911+0000 7f0cb59c6700 1 -- 172.21.15.6:0/3460473452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cb0066b10 msgr2=0x7f0cb00ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:10.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.911+0000 7f0cb59c6700 1 --2- 172.21.15.6:0/3460473452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cb0066b10 0x7f0cb00ffa30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0004940 tx=0x7f0ca0009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:10.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.912+0000 7f0cb59c6700 1 -- 172.21.15.6:0/3460473452 shutdown_connections 2024-03-22T06:17:10.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.912+0000 7f0cb59c6700 1 --2- 172.21.15.6:0/3460473452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb00fff70 0x7f0cb0102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:10.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.912+0000 7f0cb59c6700 1 --2- 172.21.15.6:0/3460473452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cb0066b10 0x7f0cb00ffa30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:10.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.912+0000 7f0cb59c6700 1 -- 172.21.15.6:0/3460473452 >> 172.21.15.6:0/3460473452 conn(0x7f0cb00f7480 msgr2=0x7f0cb00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:10.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.912+0000 7f0cb59c6700 1 -- 172.21.15.6:0/3460473452 shutdown_connections 2024-03-22T06:17:10.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.912+0000 7f0cb59c6700 1 -- 172.21.15.6:0/3460473452 wait complete. 2024-03-22T06:17:10.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.912+0000 7f0cb59c6700 1 Processor -- start 2024-03-22T06:17:10.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.912+0000 7f0cb59c6700 1 -- start start 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0cb59c6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cb0066b10 0x7f0cb010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0cb59c6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb00fff70 0x7f0cb010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0cb59c6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb010e570 con 0x7f0cb0066b10 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0cb59c6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb010ed70 con 0x7f0cb00fff70 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0caeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cb0066b10 0x7f0cb010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0caeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cb0066b10 0x7f0cb010da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52834/0 (socket says 172.21.15.6:52834) 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0caeffd700 1 -- 172.21.15.6:0/2865087440 learned_addr learned my addr 172.21.15.6:0/2865087440 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0cae7fc700 1 --2- 172.21.15.6:0/2865087440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb00fff70 0x7f0cb010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0caeffd700 1 -- 172.21.15.6:0/2865087440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb00fff70 msgr2=0x7f0cb010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0caeffd700 1 --2- 172.21.15.6:0/2865087440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb00fff70 0x7f0cb010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0caeffd700 1 -- 172.21.15.6:0/2865087440 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca0005040 con 0x7f0cb0066b10 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0caeffd700 1 --2- 172.21.15.6:0/2865087440 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cb0066b10 0x7f0cb010da30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0000c00 tx=0x7f0ca0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.913+0000 7f0c977fe700 1 -- 172.21.15.6:0/2865087440 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ca000e030 con 0x7f0cb0066b10 2024-03-22T06:17:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.914+0000 7f0cb59c6700 1 -- 172.21.15.6:0/2865087440 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb010eff0 con 0x7f0cb0066b10 2024-03-22T06:17:10.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.914+0000 7f0c977fe700 1 -- 172.21.15.6:0/2865087440 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ca0013de0 con 0x7f0cb0066b10 2024-03-22T06:17:10.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.914+0000 7f0c977fe700 1 -- 172.21.15.6:0/2865087440 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ca001ce50 con 0x7f0cb0066b10 2024-03-22T06:17:10.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.914+0000 7f0cb59c6700 1 -- 172.21.15.6:0/2865087440 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb010f530 con 0x7f0cb0066b10 2024-03-22T06:17:10.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.915+0000 7f0cb59c6700 1 -- 172.21.15.6:0/2865087440 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb005f5f0 con 0x7f0cb0066b10 2024-03-22T06:17:10.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.916+0000 7f0c977fe700 1 -- 172.21.15.6:0/2865087440 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7f0ca001c400 con 0x7f0cb0066b10 2024-03-22T06:17:10.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.916+0000 7f0c977fe700 1 --2- 172.21.15.6:0/2865087440 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f0c9806e260 0x7f0c98070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:10.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.917+0000 7f0c977fe700 1 -- 172.21.15.6:0/2865087440 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f0ca008f4f0 con 0x7f0cb0066b10 2024-03-22T06:17:10.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.917+0000 7f0cae7fc700 1 --2- 172.21.15.6:0/2865087440 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f0c9806e260 0x7f0c98070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:10.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.917+0000 7f0cae7fc700 1 --2- 172.21.15.6:0/2865087440 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f0c9806e260 0x7f0c98070710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009b80 tx=0x7f0ca400b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:10.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:10.925+0000 7f0c977fe700 1 -- 172.21.15.6:0/2865087440 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f0ca005c720 con 0x7f0cb0066b10 2024-03-22T06:17:11.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.191+0000 7f0cb59c6700 1 -- 172.21.15.6:0/2865087440 --> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cb010f860 con 0x7f0c9806e260 2024-03-22T06:17:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.193+0000 7f0c977fe700 1 -- 172.21.15.6:0/2865087440 <== mgr.14750 v2:172.21.15.6:6800/3502710787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f0cb010f860 con 0x7f0c9806e260 2024-03-22T06:17:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.196+0000 7f0cb59c6700 1 -- 172.21.15.6:0/2865087440 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f0c9806e260 msgr2=0x7f0c98070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.196+0000 7f0cb59c6700 1 --2- 172.21.15.6:0/2865087440 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f0c9806e260 0x7f0c98070710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009b80 tx=0x7f0ca400b040 comp rx=0 tx=0).stop 2024-03-22T06:17:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.196+0000 7f0cb59c6700 1 -- 172.21.15.6:0/2865087440 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cb0066b10 msgr2=0x7f0cb010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.196+0000 7f0cb59c6700 1 --2- 172.21.15.6:0/2865087440 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cb0066b10 0x7f0cb010da30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0000c00 tx=0x7f0ca0009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.196+0000 7f0cb59c6700 1 -- 172.21.15.6:0/2865087440 shutdown_connections 2024-03-22T06:17:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.196+0000 7f0cb59c6700 1 --2- 172.21.15.6:0/2865087440 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f0c9806e260 0x7f0c98070710 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.196+0000 7f0cb59c6700 1 --2- 172.21.15.6:0/2865087440 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cb0066b10 0x7f0cb010da30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.196+0000 7f0cb59c6700 1 --2- 172.21.15.6:0/2865087440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb00fff70 0x7f0cb010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.196+0000 7f0cb59c6700 1 -- 172.21.15.6:0/2865087440 >> 172.21.15.6:0/2865087440 conn(0x7f0cb00f7480 msgr2=0x7f0cb00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.196+0000 7f0cb59c6700 1 -- 172.21.15.6:0/2865087440 shutdown_connections 2024-03-22T06:17:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.196+0000 7f0cb59c6700 1 -- 172.21.15.6:0/2865087440 wait complete. 2024-03-22T06:17:11.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.350+0000 7f3553873700 1 -- 172.21.15.6:0/1363405807 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f354c0fb440 msgr2=0x7f354c0fb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:11.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.350+0000 7f3553873700 1 --2- 172.21.15.6:0/1363405807 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f354c0fb440 0x7f354c0fb850 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3548004940 tx=0x7f3548009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:11.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.350+0000 7f3553873700 1 -- 172.21.15.6:0/1363405807 shutdown_connections 2024-03-22T06:17:11.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.350+0000 7f3553873700 1 --2- 172.21.15.6:0/1363405807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c0fc6f0 0x7f354c0fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.350+0000 7f3553873700 1 --2- 172.21.15.6:0/1363405807 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f354c0fb440 0x7f354c0fb850 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.350+0000 7f3553873700 1 -- 172.21.15.6:0/1363405807 >> 172.21.15.6:0/1363405807 conn(0x7f354c0f6ba0 msgr2=0x7f354c0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:11.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.350+0000 7f3553873700 1 -- 172.21.15.6:0/1363405807 shutdown_connections 2024-03-22T06:17:11.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.350+0000 7f3553873700 1 -- 172.21.15.6:0/1363405807 wait complete. 2024-03-22T06:17:11.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.351+0000 7f3553873700 1 Processor -- start 2024-03-22T06:17:11.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.351+0000 7f3553873700 1 -- start start 2024-03-22T06:17:11.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.351+0000 7f3553873700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c0fb440 0x7f354c10d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:11.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.351+0000 7f3553873700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f354c0fc6f0 0x7f354c10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:11.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.351+0000 7f3553873700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c10e4f0 con 0x7f354c0fc6f0 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.351+0000 7f3553873700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c10ed40 con 0x7f354c0fb440 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.352+0000 7f3550e0e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f354c0fc6f0 0x7f354c10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.352+0000 7f355160f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c0fb440 0x7f354c10d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.352+0000 7f3550e0e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f354c0fc6f0 0x7f354c10def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52842/0 (socket says 172.21.15.6:52842) 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.352+0000 7f3550e0e700 1 -- 172.21.15.6:0/3370698165 learned_addr learned my addr 172.21.15.6:0/3370698165 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.352+0000 7f355160f700 1 -- 172.21.15.6:0/3370698165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f354c0fc6f0 msgr2=0x7f354c10def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.352+0000 7f355160f700 1 --2- 172.21.15.6:0/3370698165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f354c0fc6f0 0x7f354c10def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.352+0000 7f355160f700 1 -- 172.21.15.6:0/3370698165 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548005040 con 0x7f354c0fb440 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.352+0000 7f355160f700 1 --2- 172.21.15.6:0/3370698165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c0fb440 0x7f354c10d9b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3548000c00 tx=0x7f3548009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.352+0000 7f3541ffb700 1 -- 172.21.15.6:0/3370698165 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f354800e070 con 0x7f354c0fb440 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.353+0000 7f3541ffb700 1 -- 172.21.15.6:0/3370698165 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35480093f0 con 0x7f354c0fb440 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.353+0000 7f3553873700 1 -- 172.21.15.6:0/3370698165 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c10efc0 con 0x7f354c0fb440 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.353+0000 7f3541ffb700 1 -- 172.21.15.6:0/3370698165 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f354800b410 con 0x7f354c0fb440 2024-03-22T06:17:11.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.353+0000 7f3553873700 1 -- 172.21.15.6:0/3370698165 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c10f4a0 con 0x7f354c0fb440 2024-03-22T06:17:11.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.354+0000 7f3553873700 1 -- 172.21.15.6:0/3370698165 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c05f5f0 con 0x7f354c0fb440 2024-03-22T06:17:11.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.355+0000 7f3541ffb700 1 -- 172.21.15.6:0/3370698165 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7f354801c400 con 0x7f354c0fb440 2024-03-22T06:17:11.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.355+0000 7f3541ffb700 1 --2- 172.21.15.6:0/3370698165 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f353806e260 0x7f3538070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:11.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.355+0000 7f3541ffb700 1 -- 172.21.15.6:0/3370698165 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f354808ee90 con 0x7f354c0fb440 2024-03-22T06:17:11.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.355+0000 7f3550e0e700 1 --2- 172.21.15.6:0/3370698165 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f353806e260 0x7f3538070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:11.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.356+0000 7f3550e0e700 1 --2- 172.21.15.6:0/3370698165 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f353806e260 0x7f3538070710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f353c0041d0 tx=0x7f353c00b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:11.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.361+0000 7f3541ffb700 1 -- 172.21.15.6:0/3370698165 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f354805c160 con 0x7f354c0fb440 2024-03-22T06:17:11.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.691+0000 7f3553873700 1 -- 172.21.15.6:0/3370698165 --> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f354c10f750 con 0x7f353806e260 2024-03-22T06:17:11.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.705+0000 7f3541ffb700 1 -- 172.21.15.6:0/3370698165 <== mgr.14750 v2:172.21.15.6:6800/3502710787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f354c10f750 con 0x7f353806e260 2024-03-22T06:17:11.706 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:17:11.706 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (109s) 34s ago 10m 25.8M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (10m) 34s ago 10m 6656k - 17.2.7-726-g9c4e0c34 837124da804f a49ab95d42e6 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (9m) 38s ago 9m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0a36af1f8ed0 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 running (39s) 34s ago 9m - - 954c08fa6188 f192e538d520 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (4m) 34s ago 4m 226M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (4m) 34s ago 4m 16.4M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (4m) 38s ago 4m 13.7M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (4m) 38s ago 4m 15.2M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (60s) 34s ago 11m 479M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 starting - - - - 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (11m) 34s ago 11m 52.7M 2048M 17.2.7-726-g9c4e0c34 837124da804f b2fe36c6b005 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (9m) 38s ago 9m 45.7M 2048M 17.2.7-726-g9c4e0c34 837124da804f 0559848da9ab 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (2m) 34s ago 10m 16.9M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (75s) 38s ago 9m 8238k - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (8m) 34s ago 8m 211M 1559M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:17:11.707 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (8m) 34s ago 8m 226M 1559M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:17:11.708 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (7m) 34s ago 7m 187M 1559M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:17:11.708 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (7m) 38s ago 7m 248M 2581M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:17:11.708 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (6m) 38s ago 6m 217M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:17:11.708 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (6m) 38s ago 6m 208M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:17:11.708 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 starting - - - - 2024-03-22T06:17:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.708+0000 7f3553873700 1 -- 172.21.15.6:0/3370698165 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f353806e260 msgr2=0x7f3538070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.708+0000 7f3553873700 1 --2- 172.21.15.6:0/3370698165 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f353806e260 0x7f3538070710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f353c0041d0 tx=0x7f353c00b040 comp rx=0 tx=0).stop 2024-03-22T06:17:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.708+0000 7f3553873700 1 -- 172.21.15.6:0/3370698165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c0fb440 msgr2=0x7f354c10d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.708+0000 7f3553873700 1 --2- 172.21.15.6:0/3370698165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c0fb440 0x7f354c10d9b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3548000c00 tx=0x7f3548009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.708+0000 7f3553873700 1 -- 172.21.15.6:0/3370698165 shutdown_connections 2024-03-22T06:17:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.708+0000 7f3553873700 1 --2- 172.21.15.6:0/3370698165 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f353806e260 0x7f3538070710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.708+0000 7f3553873700 1 --2- 172.21.15.6:0/3370698165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c0fb440 0x7f354c10d9b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.708+0000 7f3553873700 1 --2- 172.21.15.6:0/3370698165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f354c0fc6f0 0x7f354c10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.708+0000 7f3553873700 1 -- 172.21.15.6:0/3370698165 >> 172.21.15.6:0/3370698165 conn(0x7f354c0f6ba0 msgr2=0x7f354c0f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.708+0000 7f3553873700 1 -- 172.21.15.6:0/3370698165 shutdown_connections 2024-03-22T06:17:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.708+0000 7f3553873700 1 -- 172.21.15.6:0/3370698165 wait complete. 2024-03-22T06:17:11.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.864+0000 7fbc2efa7700 1 -- 172.21.15.6:0/1380743456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc280fcf80 msgr2=0x7fbc280fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:11.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.864+0000 7fbc2efa7700 1 --2- 172.21.15.6:0/1380743456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc280fcf80 0x7fbc280fd3f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbc14004940 tx=0x7fbc14009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:11.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.864+0000 7fbc2efa7700 1 -- 172.21.15.6:0/1380743456 shutdown_connections 2024-03-22T06:17:11.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.864+0000 7fbc2efa7700 1 --2- 172.21.15.6:0/1380743456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc280fcf80 0x7fbc280fd3f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.864+0000 7fbc2efa7700 1 --2- 172.21.15.6:0/1380743456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc280fbcd0 0x7fbc280fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.864+0000 7fbc2efa7700 1 -- 172.21.15.6:0/1380743456 >> 172.21.15.6:0/1380743456 conn(0x7fbc280f7430 msgr2=0x7fbc280f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:11.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.864+0000 7fbc2efa7700 1 -- 172.21.15.6:0/1380743456 shutdown_connections 2024-03-22T06:17:11.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.864+0000 7fbc2efa7700 1 -- 172.21.15.6:0/1380743456 wait complete. 2024-03-22T06:17:11.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.865+0000 7fbc2efa7700 1 Processor -- start 2024-03-22T06:17:11.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.865+0000 7fbc2efa7700 1 -- start start 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.865+0000 7fbc2efa7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc280fbcd0 0x7fbc280ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.865+0000 7fbc2efa7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc280fcf80 0x7fbc280fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.865+0000 7fbc2efa7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc28100210 con 0x7fbc280fbcd0 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.865+0000 7fbc2efa7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc281115a0 con 0x7fbc280fcf80 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.865+0000 7fbc27fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc280fcf80 0x7fbc280fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.865+0000 7fbc27fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc280fcf80 0x7fbc280fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:37632/0 (socket says 172.21.15.6:37632) 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.865+0000 7fbc27fff700 1 -- 172.21.15.6:0/2922916757 learned_addr learned my addr 172.21.15.6:0/2922916757 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.865+0000 7fbc2cd43700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc280fbcd0 0x7fbc280ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.866+0000 7fbc27fff700 1 -- 172.21.15.6:0/2922916757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc280fbcd0 msgr2=0x7fbc280ffc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.866+0000 7fbc27fff700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc280fbcd0 0x7fbc280ffc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.866+0000 7fbc27fff700 1 -- 172.21.15.6:0/2922916757 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc14005040 con 0x7fbc280fcf80 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.866+0000 7fbc2cd43700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc280fbcd0 0x7fbc280ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:17:11.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.866+0000 7fbc27fff700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc280fcf80 0x7fbc280fe2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc14013870 tx=0x7fbc14009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:11.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.867+0000 7fbc257fa700 1 -- 172.21.15.6:0/2922916757 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc1400e070 con 0x7fbc280fcf80 2024-03-22T06:17:11.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.867+0000 7fbc257fa700 1 -- 172.21.15.6:0/2922916757 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc14013de0 con 0x7fbc280fcf80 2024-03-22T06:17:11.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.867+0000 7fbc257fa700 1 -- 172.21.15.6:0/2922916757 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc1401cea0 con 0x7fbc280fcf80 2024-03-22T06:17:11.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.867+0000 7fbc2efa7700 1 -- 172.21.15.6:0/2922916757 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc281117f0 con 0x7fbc280fcf80 2024-03-22T06:17:11.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.868+0000 7fbc2efa7700 1 -- 172.21.15.6:0/2922916757 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc280feb50 con 0x7fbc280fcf80 2024-03-22T06:17:11.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.868+0000 7fbc2efa7700 1 -- 172.21.15.6:0/2922916757 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc28101b10 con 0x7fbc280fcf80 2024-03-22T06:17:11.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.869+0000 7fbc257fa700 1 -- 172.21.15.6:0/2922916757 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7fbc1401c690 con 0x7fbc280fcf80 2024-03-22T06:17:11.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.869+0000 7fbc257fa700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fbc1806e320 0x7fbc180707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:11.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.869+0000 7fbc257fa700 1 -- 172.21.15.6:0/2922916757 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fbc1408eb90 con 0x7fbc280fcf80 2024-03-22T06:17:11.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.869+0000 7fbc2cd43700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fbc1806e320 0x7fbc180707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:11.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.870+0000 7fbc2cd43700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fbc1806e320 0x7fbc180707d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009b80 tx=0x7fbc1c00b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:11.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:11.877+0000 7fbc257fa700 1 -- 172.21.15.6:0/2922916757 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7fbc1405be60 con 0x7fbc280fcf80 2024-03-22T06:17:12.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.256+0000 7fbc2efa7700 1 -- 172.21.15.6:0/2922916757 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbc2804c710 con 0x7fbc280fcf80 2024-03-22T06:17:12.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.257+0000 7fbc257fa700 1 -- 172.21.15.6:0/2922916757 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fbc14004490 con 0x7fbc280fcf80 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:17:12.258 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:17:12.259 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:17:12.259 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-22T06:17:12.259 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:17:12.259 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:17:12.259 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:17:12.259 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:12 smithi006 ceph-mon[29301]: from='client.24553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:12.259 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:12 smithi006 ceph-mon[29301]: pgmap v24: 65 pgs: 65 active+clean; 215 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 2.1 MiB/s wr, 829 op/s 2024-03-22T06:17:12.259 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:12 smithi006 ceph-mon[29301]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:12.259 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:12 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:12.259 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:12 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:12.260 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:12 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:12.260 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:12 smithi006 ceph-mon[29301]: from='client.24565 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:12.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.259+0000 7fbc2efa7700 1 -- 172.21.15.6:0/2922916757 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fbc1806e320 msgr2=0x7fbc180707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:12.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.259+0000 7fbc2efa7700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fbc1806e320 0x7fbc180707d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009b80 tx=0x7fbc1c00b040 comp rx=0 tx=0).stop 2024-03-22T06:17:12.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.259+0000 7fbc2efa7700 1 -- 172.21.15.6:0/2922916757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc280fcf80 msgr2=0x7fbc280fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:12.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.259+0000 7fbc2efa7700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc280fcf80 0x7fbc280fe2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc14013870 tx=0x7fbc14009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:12.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.259+0000 7fbc2efa7700 1 -- 172.21.15.6:0/2922916757 shutdown_connections 2024-03-22T06:17:12.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.259+0000 7fbc2efa7700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fbc1806e320 0x7fbc180707d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.259+0000 7fbc2efa7700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc280fbcd0 0x7fbc280ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.259+0000 7fbc2efa7700 1 --2- 172.21.15.6:0/2922916757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc280fcf80 0x7fbc280fe2c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.259+0000 7fbc2efa7700 1 -- 172.21.15.6:0/2922916757 >> 172.21.15.6:0/2922916757 conn(0x7fbc280f7430 msgr2=0x7fbc280f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:12.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.260+0000 7fbc2efa7700 1 -- 172.21.15.6:0/2922916757 shutdown_connections 2024-03-22T06:17:12.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.260+0000 7fbc2efa7700 1 -- 172.21.15.6:0/2922916757 wait complete. 2024-03-22T06:17:12.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.438+0000 7fc1d9f87700 1 -- 172.21.15.6:0/4066209009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1d4104fc0 msgr2=0x7fc1d41073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:12.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.438+0000 7fc1d9f87700 1 --2- 172.21.15.6:0/4066209009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1d4104fc0 0x7fc1d41073a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4004940 tx=0x7fc1c4009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:12.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.439+0000 7fc1d9f87700 1 -- 172.21.15.6:0/4066209009 shutdown_connections 2024-03-22T06:17:12.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.439+0000 7fc1d9f87700 1 --2- 172.21.15.6:0/4066209009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1d41078e0 0x7fc1d4109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.439+0000 7fc1d9f87700 1 --2- 172.21.15.6:0/4066209009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1d4104fc0 0x7fc1d41073a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.439+0000 7fc1d9f87700 1 -- 172.21.15.6:0/4066209009 >> 172.21.15.6:0/4066209009 conn(0x7fc1d406a380 msgr2=0x7fc1d406c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:12.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.440+0000 7fc1d9f87700 1 -- 172.21.15.6:0/4066209009 shutdown_connections 2024-03-22T06:17:12.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.440+0000 7fc1d9f87700 1 -- 172.21.15.6:0/4066209009 wait complete. 2024-03-22T06:17:12.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.440+0000 7fc1d9f87700 1 Processor -- start 2024-03-22T06:17:12.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.441+0000 7fc1d9f87700 1 -- start start 2024-03-22T06:17:12.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.441+0000 7fc1d9f87700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1d4104fc0 0x7fc1d4113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:12.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.441+0000 7fc1d9f87700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1d41078e0 0x7fc1d4114460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:12.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.441+0000 7fc1d9f87700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d4114af0 con 0x7fc1d41078e0 2024-03-22T06:17:12.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.441+0000 7fc1d9f87700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d412a4b0 con 0x7fc1d4104fc0 2024-03-22T06:17:12.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.442+0000 7fc1d2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1d41078e0 0x7fc1d4114460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:12.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.443+0000 7fc1d2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1d41078e0 0x7fc1d4114460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52870/0 (socket says 172.21.15.6:52870) 2024-03-22T06:17:12.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.443+0000 7fc1d2ffd700 1 -- 172.21.15.6:0/623903913 learned_addr learned my addr 172.21.15.6:0/623903913 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:12.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.443+0000 7fc1d2ffd700 1 -- 172.21.15.6:0/623903913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1d4104fc0 msgr2=0x7fc1d4113f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T06:17:12.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.443+0000 7fc1d2ffd700 1 --2- 172.21.15.6:0/623903913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1d4104fc0 0x7fc1d4113f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.443+0000 7fc1d2ffd700 1 -- 172.21.15.6:0/623903913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c4005040 con 0x7fc1d41078e0 2024-03-22T06:17:12.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.444+0000 7fc1d2ffd700 1 --2- 172.21.15.6:0/623903913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1d41078e0 0x7fc1d4114460 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800b790 tx=0x7fc1c8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:12.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.444+0000 7fc1bbfff700 1 -- 172.21.15.6:0/623903913 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1c800f6f0 con 0x7fc1d41078e0 2024-03-22T06:17:12.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.444+0000 7fc1bbfff700 1 -- 172.21.15.6:0/623903913 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1c800fd20 con 0x7fc1d41078e0 2024-03-22T06:17:12.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.444+0000 7fc1d9f87700 1 -- 172.21.15.6:0/623903913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d412a710 con 0x7fc1d41078e0 2024-03-22T06:17:12.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.444+0000 7fc1d9f87700 1 -- 172.21.15.6:0/623903913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d412ac00 con 0x7fc1d41078e0 2024-03-22T06:17:12.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.444+0000 7fc1bbfff700 1 -- 172.21.15.6:0/623903913 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1c80103a0 con 0x7fc1d41078e0 2024-03-22T06:17:12.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.446+0000 7fc1d9f87700 1 -- 172.21.15.6:0/623903913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c00052e0 con 0x7fc1d41078e0 2024-03-22T06:17:12.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.447+0000 7fc1bbfff700 1 -- 172.21.15.6:0/623903913 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7fc1c8011400 con 0x7fc1d41078e0 2024-03-22T06:17:12.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.448+0000 7fc1bbfff700 1 --2- 172.21.15.6:0/623903913 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc1bc06e370 0x7fc1bc070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:12.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.448+0000 7fc1bbfff700 1 -- 172.21.15.6:0/623903913 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fc1c808d770 con 0x7fc1d41078e0 2024-03-22T06:17:12.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.448+0000 7fc1d37fe700 1 --2- 172.21.15.6:0/623903913 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc1bc06e370 0x7fc1bc070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:12.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.449+0000 7fc1d37fe700 1 --2- 172.21.15.6:0/623903913 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc1bc06e370 0x7fc1bc070820 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400b970 tx=0x7fc1c4009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:12.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.456+0000 7fc1bbfff700 1 -- 172.21.15.6:0/623903913 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7fc1c805a8c0 con 0x7fc1d41078e0 2024-03-22T06:17:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:12 smithi117 ceph-mon[36514]: from='client.24553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:12 smithi117 ceph-mon[36514]: pgmap v24: 65 pgs: 65 active+clean; 215 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 2.1 MiB/s wr, 829 op/s 2024-03-22T06:17:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:12 smithi117 ceph-mon[36514]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:12 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:12 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:12 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:12 smithi117 ceph-mon[36514]: from='client.24565 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:12.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.859+0000 7fc1d9f87700 1 -- 172.21.15.6:0/623903913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc1c0005860 con 0x7fc1d41078e0 2024-03-22T06:17:12.860 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:17:12.861 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:17:12.862 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:12.863 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:12.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.859+0000 7fc1bbfff700 1 -- 172.21.15.6:0/623903913 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1805 (secure 0 0 0) 0x7fc1c805a6e0 con 0x7fc1d41078e0 2024-03-22T06:17:12.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.861+0000 7fc1d9f87700 1 -- 172.21.15.6:0/623903913 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc1bc06e370 msgr2=0x7fc1bc070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:12.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.861+0000 7fc1d9f87700 1 --2- 172.21.15.6:0/623903913 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc1bc06e370 0x7fc1bc070820 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400b970 tx=0x7fc1c4009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:12.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.861+0000 7fc1d9f87700 1 -- 172.21.15.6:0/623903913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1d41078e0 msgr2=0x7fc1d4114460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:12.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.861+0000 7fc1d9f87700 1 --2- 172.21.15.6:0/623903913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1d41078e0 0x7fc1d4114460 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800b790 tx=0x7fc1c8009220 comp rx=0 tx=0).stop 2024-03-22T06:17:12.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.861+0000 7fc1d9f87700 1 -- 172.21.15.6:0/623903913 shutdown_connections 2024-03-22T06:17:12.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.861+0000 7fc1d9f87700 1 --2- 172.21.15.6:0/623903913 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7fc1bc06e370 0x7fc1bc070820 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.861+0000 7fc1d9f87700 1 --2- 172.21.15.6:0/623903913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1d4104fc0 0x7fc1d4113f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.861+0000 7fc1d9f87700 1 --2- 172.21.15.6:0/623903913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1d41078e0 0x7fc1d4114460 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.862+0000 7fc1d9f87700 1 -- 172.21.15.6:0/623903913 >> 172.21.15.6:0/623903913 conn(0x7fc1d406a380 msgr2=0x7fc1d4107ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:12.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.862+0000 7fc1d9f87700 1 -- 172.21.15.6:0/623903913 shutdown_connections 2024-03-22T06:17:12.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.862+0000 7fc1d9f87700 1 -- 172.21.15.6:0/623903913 wait complete. 2024-03-22T06:17:12.865 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:17:12.904 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-22T06:17:12.904 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-22T06:17:12.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.979+0000 7f4c77285700 1 -- 172.21.15.6:0/1263683865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c700fbcd0 msgr2=0x7f4c700fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:12.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.979+0000 7f4c77285700 1 --2- 172.21.15.6:0/1263683865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c700fbcd0 0x7f4c700fc0e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c004940 tx=0x7f4c6c009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:12.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.979+0000 7f4c77285700 1 -- 172.21.15.6:0/1263683865 shutdown_connections 2024-03-22T06:17:12.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.979+0000 7f4c77285700 1 --2- 172.21.15.6:0/1263683865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c700fcf80 0x7f4c700fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.979+0000 7f4c77285700 1 --2- 172.21.15.6:0/1263683865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c700fbcd0 0x7f4c700fc0e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.979+0000 7f4c77285700 1 -- 172.21.15.6:0/1263683865 >> 172.21.15.6:0/1263683865 conn(0x7f4c700f7430 msgr2=0x7f4c700f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:12.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.979+0000 7f4c77285700 1 -- 172.21.15.6:0/1263683865 shutdown_connections 2024-03-22T06:17:12.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.979+0000 7f4c77285700 1 -- 172.21.15.6:0/1263683865 wait complete. 2024-03-22T06:17:12.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.979+0000 7f4c77285700 1 Processor -- start 2024-03-22T06:17:12.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c77285700 1 -- start start 2024-03-22T06:17:12.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c77285700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c700fbcd0 0x7f4c7010c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:12.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c77285700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c700fcf80 0x7f4c7010ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:12.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c77285700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7010d4e0 con 0x7f4c700fcf80 2024-03-22T06:17:12.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c77285700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7010dcd0 con 0x7f4c700fbcd0 2024-03-22T06:17:12.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c74820700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c700fcf80 0x7f4c7010ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:12.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c74820700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c700fcf80 0x7f4c7010ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52886/0 (socket says 172.21.15.6:52886) 2024-03-22T06:17:12.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c74820700 1 -- 172.21.15.6:0/3504500127 learned_addr learned my addr 172.21.15.6:0/3504500127 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:12.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c75021700 1 --2- 172.21.15.6:0/3504500127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c700fbcd0 0x7f4c7010c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:12.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c74820700 1 -- 172.21.15.6:0/3504500127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c700fbcd0 msgr2=0x7f4c7010c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:12.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c74820700 1 --2- 172.21.15.6:0/3504500127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c700fbcd0 0x7f4c7010c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:12.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c74820700 1 -- 172.21.15.6:0/3504500127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c005040 con 0x7f4c700fcf80 2024-03-22T06:17:12.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c74820700 1 --2- 172.21.15.6:0/3504500127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c700fcf80 0x7f4c7010ce50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000daf0 tx=0x7f4c6000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:12.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c65ffb700 1 -- 172.21.15.6:0/3504500127 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c6000ecc0 con 0x7f4c700fcf80 2024-03-22T06:17:12.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c65ffb700 1 -- 172.21.15.6:0/3504500127 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c6000ee20 con 0x7f4c700fcf80 2024-03-22T06:17:12.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.980+0000 7f4c77285700 1 -- 172.21.15.6:0/3504500127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7010dfb0 con 0x7f4c700fcf80 2024-03-22T06:17:12.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.981+0000 7f4c65ffb700 1 -- 172.21.15.6:0/3504500127 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c60016940 con 0x7f4c700fcf80 2024-03-22T06:17:12.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.981+0000 7f4c77285700 1 -- 172.21.15.6:0/3504500127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7010e570 con 0x7f4c700fcf80 2024-03-22T06:17:12.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.982+0000 7f4c77285700 1 -- 172.21.15.6:0/3504500127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7005f5f0 con 0x7f4c700fcf80 2024-03-22T06:17:12.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.982+0000 7f4c65ffb700 1 -- 172.21.15.6:0/3504500127 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7f4c60014070 con 0x7f4c700fcf80 2024-03-22T06:17:12.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.982+0000 7f4c65ffb700 1 --2- 172.21.15.6:0/3504500127 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f4c5c06e260 0x7f4c5c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:12.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.983+0000 7f4c65ffb700 1 -- 172.21.15.6:0/3504500127 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f4c6008fb60 con 0x7f4c700fcf80 2024-03-22T06:17:12.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.983+0000 7f4c75021700 1 --2- 172.21.15.6:0/3504500127 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f4c5c06e260 0x7f4c5c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:12.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.983+0000 7f4c75021700 1 --2- 172.21.15.6:0/3504500127 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f4c5c06e260 0x7f4c5c070710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00b970 tx=0x7f4c6c009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:12.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:12.987+0000 7f4c65ffb700 1 -- 172.21.15.6:0/3504500127 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f4c6005cff0 con 0x7f4c700fcf80 2024-03-22T06:17:13.201 DEBUG:teuthology.parallel:result is None 2024-03-22T06:17:13.201 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-22T06:17:13.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.227+0000 7f4c77285700 1 -- 172.21.15.6:0/3504500127 --> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c7010e950 con 0x7f4c5c06e260 2024-03-22T06:17:13.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.230+0000 7f4c65ffb700 1 -- 172.21.15.6:0/3504500127 <== mgr.14750 v2:172.21.15.6:6800/3502710787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f4c7010e950 con 0x7f4c5c06e260 2024-03-22T06:17:13.231 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:17:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:17:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:17:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:17:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [], 2024-03-22T06:17:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "", 2024-03-22T06:17:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading mgr daemons", 2024-03-22T06:17:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:17:13.231 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:17:13.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.234+0000 7f4c77285700 1 -- 172.21.15.6:0/3504500127 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f4c5c06e260 msgr2=0x7f4c5c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:13.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.234+0000 7f4c77285700 1 --2- 172.21.15.6:0/3504500127 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f4c5c06e260 0x7f4c5c070710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00b970 tx=0x7f4c6c009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:13.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.234+0000 7f4c77285700 1 -- 172.21.15.6:0/3504500127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c700fcf80 msgr2=0x7f4c7010ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:13.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.234+0000 7f4c77285700 1 --2- 172.21.15.6:0/3504500127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c700fcf80 0x7f4c7010ce50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000daf0 tx=0x7f4c6000b4f0 comp rx=0 tx=0).stop 2024-03-22T06:17:13.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.234+0000 7f4c77285700 1 -- 172.21.15.6:0/3504500127 shutdown_connections 2024-03-22T06:17:13.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.234+0000 7f4c77285700 1 --2- 172.21.15.6:0/3504500127 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f4c5c06e260 0x7f4c5c070710 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:13.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.234+0000 7f4c77285700 1 --2- 172.21.15.6:0/3504500127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c700fbcd0 0x7f4c7010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:13.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.234+0000 7f4c77285700 1 --2- 172.21.15.6:0/3504500127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c700fcf80 0x7f4c7010ce50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:13.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.234+0000 7f4c77285700 1 -- 172.21.15.6:0/3504500127 >> 172.21.15.6:0/3504500127 conn(0x7f4c700f7430 msgr2=0x7f4c700f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:13.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.234+0000 7f4c77285700 1 -- 172.21.15.6:0/3504500127 shutdown_connections 2024-03-22T06:17:13.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.234+0000 7f4c77285700 1 -- 172.21.15.6:0/3504500127 wait complete. 2024-03-22T06:17:13.256 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-22T06:17:13.256 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-22T06:17:13.312 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-22T06:17:13.312 DEBUG:teuthology.parallel:result is None 2024-03-22T06:17:13.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.369+0000 7f5e15302700 1 -- 172.21.15.6:0/2643724547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e100fff40 msgr2=0x7f5e10102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:13.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.369+0000 7f5e15302700 1 --2- 172.21.15.6:0/2643724547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e100fff40 0x7f5e10102320 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5e04004940 tx=0x7f5e04009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:13.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.369+0000 7f5e15302700 1 -- 172.21.15.6:0/2643724547 shutdown_connections 2024-03-22T06:17:13.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.369+0000 7f5e15302700 1 --2- 172.21.15.6:0/2643724547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e100fff40 0x7f5e10102320 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:13.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.369+0000 7f5e15302700 1 --2- 172.21.15.6:0/2643724547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e100662a0 0x7f5e100ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:13.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.369+0000 7f5e15302700 1 -- 172.21.15.6:0/2643724547 >> 172.21.15.6:0/2643724547 conn(0x7f5e100f7430 msgr2=0x7f5e100f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:13.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.369+0000 7f5e15302700 1 -- 172.21.15.6:0/2643724547 shutdown_connections 2024-03-22T06:17:13.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.369+0000 7f5e15302700 1 -- 172.21.15.6:0/2643724547 wait complete. 2024-03-22T06:17:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e15302700 1 Processor -- start 2024-03-22T06:17:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e15302700 1 -- start start 2024-03-22T06:17:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e15302700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e100662a0 0x7f5e1010da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e15302700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e100fff40 0x7f5e1010dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e15302700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1010e5b0 con 0x7f5e100fff40 2024-03-22T06:17:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e15302700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1010edb0 con 0x7f5e100662a0 2024-03-22T06:17:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e0e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e100fff40 0x7f5e1010dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e0e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e100fff40 0x7f5e1010dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52892/0 (socket says 172.21.15.6:52892) 2024-03-22T06:17:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e0e7fc700 1 -- 172.21.15.6:0/1759207267 learned_addr learned my addr 172.21.15.6:0/1759207267 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e0effd700 1 --2- 172.21.15.6:0/1759207267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e100662a0 0x7f5e1010da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e0e7fc700 1 -- 172.21.15.6:0/1759207267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e100662a0 msgr2=0x7f5e1010da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e0e7fc700 1 --2- 172.21.15.6:0/1759207267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e100662a0 0x7f5e1010da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.370+0000 7f5e0e7fc700 1 -- 172.21.15.6:0/1759207267 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e04005040 con 0x7f5e100fff40 2024-03-22T06:17:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.371+0000 7f5e0e7fc700 1 --2- 172.21.15.6:0/1759207267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e100fff40 0x7f5e1010dfb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400b3b0 tx=0x7f5e04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.371+0000 7f5df77fe700 1 -- 172.21.15.6:0/1759207267 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e0400e040 con 0x7f5e100fff40 2024-03-22T06:17:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.371+0000 7f5df77fe700 1 -- 172.21.15.6:0/1759207267 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e0400bbf0 con 0x7f5e100fff40 2024-03-22T06:17:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.371+0000 7f5e15302700 1 -- 172.21.15.6:0/1759207267 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1010f030 con 0x7f5e100fff40 2024-03-22T06:17:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.371+0000 7f5df77fe700 1 -- 172.21.15.6:0/1759207267 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e0401c630 con 0x7f5e100fff40 2024-03-22T06:17:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.371+0000 7f5e15302700 1 -- 172.21.15.6:0/1759207267 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1010f570 con 0x7f5e100fff40 2024-03-22T06:17:13.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.372+0000 7f5e15302700 1 -- 172.21.15.6:0/1759207267 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1005f5f0 con 0x7f5e100fff40 2024-03-22T06:17:13.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.372+0000 7f5df77fe700 1 -- 172.21.15.6:0/1759207267 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 31) v1 ==== 92375+0+0 (secure 0 0 0) 0x7f5e0400bd60 con 0x7f5e100fff40 2024-03-22T06:17:13.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.373+0000 7f5df77fe700 1 --2- 172.21.15.6:0/1759207267 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f5df806e260 0x7f5df8070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:13.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.373+0000 7f5df77fe700 1 -- 172.21.15.6:0/1759207267 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f5e0408e810 con 0x7f5e100fff40 2024-03-22T06:17:13.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.373+0000 7f5e0effd700 1 --2- 172.21.15.6:0/1759207267 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f5df806e260 0x7f5df8070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:13.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.373+0000 7f5e0effd700 1 --2- 172.21.15.6:0/1759207267 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f5df806e260 0x7f5df8070710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5e10065fe0 tx=0x7f5e0000b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:13.376 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:13 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2922916757' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:17:13.376 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:13 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/623903913' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:17:13.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.377+0000 7f5df77fe700 1 -- 172.21.15.6:0/1759207267 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185228 (secure 0 0 0) 0x7f5e0405ba40 con 0x7f5e100fff40 2024-03-22T06:17:13.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:13 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/2922916757' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:17:13.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:13 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/623903913' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:17:13.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.691+0000 7f5e15302700 1 -- 172.21.15.6:0/1759207267 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5e1010f790 con 0x7f5e100fff40 2024-03-22T06:17:13.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.692+0000 7f5df77fe700 1 -- 172.21.15.6:0/1759207267 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5e0405b5d0 con 0x7f5e100fff40 2024-03-22T06:17:13.693 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_OK 2024-03-22T06:17:13.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.694+0000 7f5e15302700 1 -- 172.21.15.6:0/1759207267 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f5df806e260 msgr2=0x7f5df8070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:13.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.694+0000 7f5e15302700 1 --2- 172.21.15.6:0/1759207267 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f5df806e260 0x7f5df8070710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5e10065fe0 tx=0x7f5e0000b040 comp rx=0 tx=0).stop 2024-03-22T06:17:13.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.694+0000 7f5e15302700 1 -- 172.21.15.6:0/1759207267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e100fff40 msgr2=0x7f5e1010dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:13.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.694+0000 7f5e15302700 1 --2- 172.21.15.6:0/1759207267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e100fff40 0x7f5e1010dfb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400b3b0 tx=0x7f5e04009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:13.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.694+0000 7f5e15302700 1 -- 172.21.15.6:0/1759207267 shutdown_connections 2024-03-22T06:17:13.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.694+0000 7f5e15302700 1 --2- 172.21.15.6:0/1759207267 >> [v2:172.21.15.6:6800/3502710787,v1:172.21.15.6:6801/3502710787] conn(0x7f5df806e260 0x7f5df8070710 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:13.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.694+0000 7f5e15302700 1 --2- 172.21.15.6:0/1759207267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e100662a0 0x7f5e1010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:13.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.694+0000 7f5e15302700 1 --2- 172.21.15.6:0/1759207267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e100fff40 0x7f5e1010dfb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:13.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.694+0000 7f5e15302700 1 -- 172.21.15.6:0/1759207267 >> 172.21.15.6:0/1759207267 conn(0x7f5e100f7430 msgr2=0x7f5e100f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:13.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.694+0000 7f5e15302700 1 -- 172.21.15.6:0/1759207267 shutdown_connections 2024-03-22T06:17:13.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:13.694+0000 7f5e15302700 1 -- 172.21.15.6:0/1759207267 wait complete. 2024-03-22T06:17:14.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:14 smithi117 ceph-mon[36514]: pgmap v25: 65 pgs: 65 active+clean; 215 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.4 MiB/s wr, 567 op/s 2024-03-22T06:17:14.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:14 smithi117 ceph-mon[36514]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:14.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:14 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1759207267' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:17:14.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:14 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:17:14.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:14 smithi006 ceph-mon[29301]: pgmap v25: 65 pgs: 65 active+clean; 215 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.4 MiB/s wr, 567 op/s 2024-03-22T06:17:14.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:14 smithi006 ceph-mon[29301]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:14.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:14 smithi006 ceph-mon[29301]: from='client.? 172.21.15.6:0/1759207267' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:17:14.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:14 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:17:15.858 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:15 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:15.859 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:15 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:16.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:15 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:16.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:15 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:17.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:16 smithi117 ceph-mon[36514]: pgmap v26: 65 pgs: 65 active+clean; 215 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.4 MiB/s wr, 567 op/s 2024-03-22T06:17:17.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:16 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:17.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:16 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:17.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:16 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:17.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:16 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:17.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:16 smithi006 ceph-mon[29301]: pgmap v26: 65 pgs: 65 active+clean; 215 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.4 MiB/s wr, 567 op/s 2024-03-22T06:17:17.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:16 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:17.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:16 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:17.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:16 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:17.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:16 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:19.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:18 smithi117 ceph-mon[36514]: pgmap v27: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.7 MiB/s wr, 685 op/s 2024-03-22T06:17:19.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:18 smithi117 ceph-mon[36514]: Standby manager daemon smithi117.bqeskk restarted 2024-03-22T06:17:19.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:18 smithi117 ceph-mon[36514]: Standby manager daemon smithi117.bqeskk started 2024-03-22T06:17:19.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:18 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/274464561' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/crt"}]: dispatch 2024-03-22T06:17:19.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:18 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/274464561' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:17:19.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:18 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/274464561' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/key"}]: dispatch 2024-03-22T06:17:19.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:18 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/274464561' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:17:19.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:18 smithi006 ceph-mon[29301]: pgmap v27: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.7 MiB/s wr, 685 op/s 2024-03-22T06:17:19.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:18 smithi006 ceph-mon[29301]: Standby manager daemon smithi117.bqeskk restarted 2024-03-22T06:17:19.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:18 smithi006 ceph-mon[29301]: Standby manager daemon smithi117.bqeskk started 2024-03-22T06:17:19.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:18 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/274464561' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/crt"}]: dispatch 2024-03-22T06:17:19.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:18 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/274464561' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:17:19.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:18 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/274464561' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/key"}]: dispatch 2024-03-22T06:17:19.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:18 smithi006 ceph-mon[29301]: from='mgr.? 172.21.15.117:0/274464561' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:17:20.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: mgrmap e32: smithi006.sfpxdy(active, since 49s), standbys: smithi117.bqeskk 2024-03-22T06:17:20.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:20.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:20.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:17:20.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:17:20.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:20.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:20.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:17:20.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:20.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi006.sfpxdy"}]: dispatch 2024-03-22T06:17:20.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi006.sfpxdy"}]: dispatch 2024-03-22T06:17:20.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi006.sfpxdy"}]': finished 2024-03-22T06:17:20.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.bqeskk"}]: dispatch 2024-03-22T06:17:20.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.bqeskk"}]: dispatch 2024-03-22T06:17:20.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.bqeskk"}]': finished 2024-03-22T06:17:20.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:19 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T06:17:20.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: mgrmap e32: smithi006.sfpxdy(active, since 49s), standbys: smithi117.bqeskk 2024-03-22T06:17:20.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:20.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:20.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:17:20.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:17:20.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:20.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:20.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:17:20.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:20.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi006.sfpxdy"}]: dispatch 2024-03-22T06:17:20.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi006.sfpxdy"}]: dispatch 2024-03-22T06:17:20.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi006.sfpxdy"}]': finished 2024-03-22T06:17:20.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.bqeskk"}]: dispatch 2024-03-22T06:17:20.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.bqeskk"}]: dispatch 2024-03-22T06:17:20.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.bqeskk"}]': finished 2024-03-22T06:17:20.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:19 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T06:17:20.969 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:20 smithi006 ceph-mon[29301]: pgmap v28: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 1.1 MiB/s wr, 354 op/s 2024-03-22T06:17:20.969 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:20 smithi006 ceph-mon[29301]: pgmap v29: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 408 op/s 2024-03-22T06:17:20.969 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:20 smithi006 ceph-mon[29301]: Upgrade: Setting container_image for all mgr 2024-03-22T06:17:20.969 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:20 smithi006 ceph-mon[29301]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-22T06:17:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:20 smithi117 ceph-mon[36514]: pgmap v28: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 1.1 MiB/s wr, 354 op/s 2024-03-22T06:17:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:20 smithi117 ceph-mon[36514]: pgmap v29: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 408 op/s 2024-03-22T06:17:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:20 smithi117 ceph-mon[36514]: Upgrade: Setting container_image for all mgr 2024-03-22T06:17:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:20 smithi117 ceph-mon[36514]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-22T06:17:23.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:22 smithi006 ceph-mon[29301]: pgmap v30: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 14 KiB/s rd, 459 KiB/s wr, 148 op/s 2024-03-22T06:17:23.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:22 smithi117 ceph-mon[36514]: pgmap v30: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 14 KiB/s rd, 459 KiB/s wr, 148 op/s 2024-03-22T06:17:25.030 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:24 smithi006 ceph-mon[29301]: pgmap v31: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 14 KiB/s rd, 459 KiB/s wr, 148 op/s 2024-03-22T06:17:25.030 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:24 smithi006 ceph-mon[29301]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:25.030 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:24 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:17:25.030 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:24 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:17:25.030 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:24 smithi006 ceph-mon[29301]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:17:25.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:24 smithi117 ceph-mon[36514]: pgmap v31: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 14 KiB/s rd, 459 KiB/s wr, 148 op/s 2024-03-22T06:17:25.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:24 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:25.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:24 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:17:25.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:24 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:17:25.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:24 smithi117 ceph-mon[36514]: from='mgr.14750 172.21.15.6:0/3194688378' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:17:26.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:25 smithi006 ceph-mon[29301]: Upgrade: Updating mon.smithi006 2024-03-22T06:17:26.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:25 smithi006 ceph-mon[29301]: Deploying daemon mon.smithi006 on smithi006 2024-03-22T06:17:26.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:25 smithi117 ceph-mon[36514]: Upgrade: Updating mon.smithi006 2024-03-22T06:17:26.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:25 smithi117 ceph-mon[36514]: Deploying daemon mon.smithi006 on smithi006 2024-03-22T06:17:27.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:26 smithi006 ceph-mon[29301]: pgmap v32: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 14 KiB/s rd, 459 KiB/s wr, 148 op/s 2024-03-22T06:17:27.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:26 smithi006 systemd[1]: Stopping Ceph mon.smithi006 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:17:27.074 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:26 smithi117 ceph-mon[36514]: pgmap v32: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 14 KiB/s rd, 459 KiB/s wr, 148 op/s 2024-03-22T06:17:27.490 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:27 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006[29297]: 2024-03-22T06:17:27.071+0000 7fa8ef12f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi006 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:17:27.490 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:27 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006[29297]: 2024-03-22T06:17:27.071+0000 7fa8ef12f700 -1 mon.smithi006@0(leader) e2 *** Got Signal Terminated *** 2024-03-22T06:17:27.490 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:27 smithi006 podman[94313]: 2024-03-22 06:17:27.225448875 +0000 UTC m=+0.270279904 container died b2fe36c6b00594a1110ecb7ecfc952a6e7f98479fe63b4c71cf0fb610bd77eb8 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-22T06:17:27.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:27 smithi006 podman[94313]: 2024-03-22 06:17:27.589220623 +0000 UTC m=+0.634051639 container cleanup b2fe36c6b00594a1110ecb7ecfc952a6e7f98479fe63b4c71cf0fb610bd77eb8 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-22T06:17:27.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:27 smithi006 bash[94313]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006 2024-03-22T06:17:28.272 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:27 smithi006 podman[94327]: 2024-03-22 06:17:27.90741211 +0000 UTC m=+0.678867661 container remove b2fe36c6b00594a1110ecb7ecfc952a6e7f98479fe63b4c71cf0fb610bd77eb8 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=quincy-9c4e0c3, org.label-schema.build-date=20240319, ceph=True) 2024-03-22T06:17:28.272 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006.service: Deactivated successfully. 2024-03-22T06:17:28.272 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 systemd[1]: Stopped Ceph mon.smithi006 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:17:28.272 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006.service: Consumed 11.418s CPU time. 2024-03-22T06:17:28.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 systemd[1]: Starting Ceph mon.smithi006 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:17:28.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 podman[94463]: 2024-03-22 06:17:28.387034524 +0000 UTC m=+0.017978374 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:17:28.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 podman[94463]: 2024-03-22 06:17:28.491924307 +0000 UTC m=+0.122868149 container create 7eafc6be6dcac266a1811663c7b778d83ea283998c5ca32058f9de56ed048877 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:17:29.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 podman[94463]: 2024-03-22 06:17:28.694263827 +0000 UTC m=+0.325207679 container init 7eafc6be6dcac266a1811663c7b778d83ea283998c5ca32058f9de56ed048877 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-22T06:17:29.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 podman[94463]: 2024-03-22 06:17:28.697711448 +0000 UTC m=+0.328655290 container start 7eafc6be6dcac266a1811663c7b778d83ea283998c5ca32058f9de56ed048877 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:17:29.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T06:17:29.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev), process ceph-mon, pid 2 2024-03-22T06:17:29.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: pidfile_write: ignore empty --pid-file 2024-03-22T06:17:29.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: load: jerasure load: lrc 2024-03-22T06:17:29.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: RocksDB version: 7.9.2 2024-03-22T06:17:29.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Git sha 0 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Compile date 2024-03-21 23:09:57 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: DB SUMMARY 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: DB Session ID: TYB7U1OZ9JNTFAGZMII0 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: CURRENT file: CURRENT 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: MANIFEST file: MANIFEST-000015 size: 972 Bytes 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi006/store.db dir, Total Num: 1, files: 000030.sst 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi006/store.db: 000028.log size: 2175580 ; 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.error_if_exists: 0 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.create_if_missing: 0 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.paranoid_checks: 1 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-22T06:17:29.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.env: 0x556f9016eb20 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.fs: PosixFileSystem 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.info_log: 0x556f928b57e0 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.statistics: (nil) 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.use_fsync: 0 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_log_file_size: 0 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.allow_fallocate: 1 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T06:17:29.035 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.use_direct_reads: 0 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.db_log_dir: 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.wal_dir: 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T06:17:29.036 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T06:17:29.037 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.write_buffer_manager: 0x556f928861e0 2024-03-22T06:17:29.037 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T06:17:29.037 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T06:17:29.037 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T06:17:29.037 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T06:17:29.037 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T06:17:29.037 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T06:17:29.037 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T06:17:29.037 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T06:17:29.037 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.unordered_write: 0 2024-03-22T06:17:29.037 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T06:17:29.038 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T06:17:29.038 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T06:17:29.038 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T06:17:29.038 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.row_cache: None 2024-03-22T06:17:29.038 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.wal_filter: None 2024-03-22T06:17:29.039 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T06:17:29.039 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.two_write_queues: 0 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.wal_compression: 0 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.atomic_flush: 0 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.log_readahead_size: 0 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T06:17:29.040 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_background_jobs: 2 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_background_compactions: -1 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_subcompactions: 1 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_open_files: -1 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T06:17:29.041 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_background_flushes: -1 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Compression algorithms supported: 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: kZSTD supported: 0 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: kXpressCompression supported: 0 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: kZlibCompression supported: 1 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: kSnappyCompression supported: 1 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: kLZ4Compression supported: 1 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: kBZip2Compression supported: 0 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-22T06:17:29.042 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi006/store.db/MANIFEST-000015 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.merge_operator: 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_filter: None 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556f928b5900) 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: cache_index_and_filter_blocks: 1 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: pin_top_level_index_and_filter: 1 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: index_type: 0 2024-03-22T06:17:29.043 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: data_block_index_type: 0 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: index_shortening: 1 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: checksum: 4 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: no_block_cache: 0 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_cache: 0x556f928b9090 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_cache_name: BinnedLRUCache 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_cache_options: 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: capacity : 536870912 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: num_shard_bits : 4 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: strict_capacity_limit : 0 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: high_pri_pool_ratio: 0.000 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_cache_compressed: (nil) 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: persistent_cache: (nil) 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_size: 4096 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_size_deviation: 10 2024-03-22T06:17:29.044 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_restart_interval: 16 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: index_block_restart_interval: 1 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: metadata_block_size: 4096 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: partition_filters: 0 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: use_delta_encoding: 1 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: filter_policy: bloomfilter 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: whole_key_filtering: 1 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: verify_compression: 0 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: read_amp_bytes_per_bit: 0 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: format_version: 5 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: enable_index_compression: 1 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: block_align: 0 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: max_auto_readahead_size: 262144 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: prepopulate_block_cache: 0 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: initial_auto_readahead_size: 8192 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: num_file_reads_for_auto_readahead: 2 2024-03-22T06:17:29.045 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compression: NoCompression 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.num_levels: 7 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T06:17:29.046 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T06:17:29.047 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-22T06:17:29.048 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.arena_block_size: 1048576 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.table_properties_collectors: 2024-03-22T06:17:29.049 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.inplace_update_support: 0 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.bloom_locality: 0 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.max_successive_merges: 0 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.ttl: 2592000 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-22T06:17:29.050 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.enable_blob_files: false 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.min_blob_size: 0 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.blob_file_starting_level: 0 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 30.sst 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi006/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 32, last_sequence is 10527, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 80009ed1-8913-4099-a8af-470819289974 2024-03-22T06:17:29.051 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711088248726776, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711088248733124, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2008028, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10528, "largest_seqno": 11495, "table_properties": {"data_size": 2002849, "index_size": 3012, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1157, "raw_key_size": 10301, "raw_average_key_size": 23, "raw_value_size": 1993236, "raw_average_value_size": 4499, "num_data_blocks": 138, "num_entries": 443, "num_filter_entries": 443, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711088248, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "80009ed1-8913-4099-a8af-470819289974", "db_session_id": "TYB7U1OZ9JNTFAGZMII0", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711088248733220, "job": 1, "event": "recovery_finished"} 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi006/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556f929a6000 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: DB pointer 0x556f92992000 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ** DB Stats ** 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T06:17:29.052 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ** Compaction Stats [default] ** 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: L0 1/0 1.92 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 595.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: L6 1/0 5.60 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Sum 2/0 7.51 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 595.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 595.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ** Compaction Stats [default] ** 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T06:17:29.053 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 595.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Cumulative compaction: 0.00 GB write, 225.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Interval compaction: 0.00 GB write, 225.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Block cache BinnedLRUCache@0x556f928b9090#2 capacity: 512.00 MB usage: 4.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.28 KB,0.000244379%) IndexBlock(1,3.08 KB,0.000587106%) Misc(1,0.00 KB,0%) 2024-03-22T06:17:29.054 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: starting mon.smithi006 rank 0 at public addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] at bind addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi006 fsid fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: mon.smithi006@-1(???) e2 preinit fsid fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: mon.smithi006@-1(???).mds e14 new map 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: mon.smithi006@-1(???).mds e14 print_map 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: e14 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: legacy client fscid: 1 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Filesystem 'cephfs' (1) 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: fs_name cephfs 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: epoch 14 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:17:29.055 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: tableserver 0 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: root 0 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: session_timeout 60 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: session_autoclose 300 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: max_file_size 1099511627776 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: max_xattr_size 65536 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: required_client_features {} 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: last_failure 0 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: last_failure_osd_epoch 0 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: max_mds 1 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: in 0 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: up {0=14466} 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: failed 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: damaged 2024-03-22T06:17:29.056 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: stopped 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: data_pools [3] 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: metadata_pool 2 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: inline_data disabled 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: balancer 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: bal_rank_mask -1 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: standby_count_wanted 1 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: qdb_cluster leader: 0 members: 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: [mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: Standby daemons: 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: [mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: [mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:29.057 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout: [mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:29.058 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: mon.smithi006@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-22T06:17:29.058 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: mon.smithi006@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T06:17:29.058 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: mon.smithi006@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T06:17:29.058 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: mon.smithi006@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T06:17:29.058 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 ceph-mon[94482]: mon.smithi006@-1(???).paxosservice(auth 1..20) refresh upgraded, format 0 -> 3 2024-03-22T06:17:29.058 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 bash[94463]: 7eafc6be6dcac266a1811663c7b778d83ea283998c5ca32058f9de56ed048877 2024-03-22T06:17:29.058 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:28 smithi006 systemd[1]: Started Ceph mon.smithi006 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:17:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: pgmap v33: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-03-22T06:17:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: pgmap v34: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-03-22T06:17:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: mon.smithi006 calling monitor election 2024-03-22T06:17:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: mon.smithi006 is new leader, mons smithi006,smithi117 in quorum (ranks 0,1) 2024-03-22T06:17:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: monmap e2: 2 mons at {smithi006=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T06:17:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:17:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T06:17:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: mgrmap e32: smithi006.sfpxdy(active, since 61s), standbys: smithi117.bqeskk 2024-03-22T06:17:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:17:30.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:17:30.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:17:30.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='' 2024-03-22T06:17:30.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: mgrmap e33: smithi006.sfpxdy(active, since 61s), standbys: smithi117.bqeskk 2024-03-22T06:17:30.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:30 smithi117 ceph-mon[36514]: from='mgr.14750 ' entity='' 2024-03-22T06:17:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: pgmap v33: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-03-22T06:17:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: pgmap v34: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-03-22T06:17:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: mon.smithi006 calling monitor election 2024-03-22T06:17:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: mon.smithi006 is new leader, mons smithi006,smithi117 in quorum (ranks 0,1) 2024-03-22T06:17:30.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: monmap e2: 2 mons at {smithi006=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T06:17:30.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:17:30.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T06:17:30.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: mgrmap e32: smithi006.sfpxdy(active, since 61s), standbys: smithi117.bqeskk 2024-03-22T06:17:30.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:17:30.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:17:30.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:17:30.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: from='mgr.14750 ' entity='' 2024-03-22T06:17:30.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: mgrmap e33: smithi006.sfpxdy(active, since 61s), standbys: smithi117.bqeskk 2024-03-22T06:17:30.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:30 smithi006 ceph-mon[94482]: from='mgr.14750 ' entity='' 2024-03-22T06:17:37.234 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:37 smithi006 ceph-mon[94482]: Standby manager daemon smithi117.bqeskk restarted 2024-03-22T06:17:37.234 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:37 smithi006 ceph-mon[94482]: Standby manager daemon smithi117.bqeskk started 2024-03-22T06:17:37.234 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:37 smithi006 ceph-mon[94482]: from='mgr.? 172.21.15.117:0/2318745692' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/crt"}]: dispatch 2024-03-22T06:17:37.234 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:37 smithi006 ceph-mon[94482]: from='mgr.? 172.21.15.117:0/2318745692' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:17:37.234 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:37 smithi006 ceph-mon[94482]: from='mgr.? 172.21.15.117:0/2318745692' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/key"}]: dispatch 2024-03-22T06:17:37.234 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:37 smithi006 ceph-mon[94482]: from='mgr.? 172.21.15.117:0/2318745692' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:17:37.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:37 smithi117 ceph-mon[36514]: Standby manager daemon smithi117.bqeskk restarted 2024-03-22T06:17:37.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:37 smithi117 ceph-mon[36514]: Standby manager daemon smithi117.bqeskk started 2024-03-22T06:17:37.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:37 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2318745692' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/crt"}]: dispatch 2024-03-22T06:17:37.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:37 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2318745692' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T06:17:37.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:37 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2318745692' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.bqeskk/key"}]: dispatch 2024-03-22T06:17:37.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:37 smithi117 ceph-mon[36514]: from='mgr.? 172.21.15.117:0/2318745692' entity='mgr.smithi117.bqeskk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T06:17:38.445 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: mgrmap e34: smithi006.sfpxdy(active, since 68s), standbys: smithi117.bqeskk 2024-03-22T06:17:38.445 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: Active manager daemon smithi006.sfpxdy restarted 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: Activating manager daemon smithi006.sfpxdy 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: mgrmap e35: smithi006.sfpxdy(active, starting, since 0.00958649s), standbys: smithi117.bqeskk 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi006.sfpxdy", "id": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi117.bqeskk", "id": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:17:38.446 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:17:38.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:17:38.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:17:38.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:17:38.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T06:17:38.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T06:17:38.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T06:17:38.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: Manager daemon smithi006.sfpxdy is now available 2024-03-22T06:17:38.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:38.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:17:38.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:17:38.447 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/trash_purge_schedule"}]: dispatch 2024-03-22T06:17:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: mgrmap e34: smithi006.sfpxdy(active, since 68s), standbys: smithi117.bqeskk 2024-03-22T06:17:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: Active manager daemon smithi006.sfpxdy restarted 2024-03-22T06:17:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: Activating manager daemon smithi006.sfpxdy 2024-03-22T06:17:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T06:17:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: mgrmap e35: smithi006.sfpxdy(active, starting, since 0.00958649s), standbys: smithi117.bqeskk 2024-03-22T06:17:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:17:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:17:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi006.sfpxdy", "id": "smithi006.sfpxdy"}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr metadata", "who": "smithi117.bqeskk", "id": "smithi117.bqeskk"}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T06:17:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: Manager daemon smithi006.sfpxdy is now available 2024-03-22T06:17:38.508 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:38.508 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:17:38.508 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:17:38.508 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:38 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi006.sfpxdy/trash_purge_schedule"}]: dispatch 2024-03-22T06:17:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:39 smithi117 ceph-mon[36514]: [22/Mar/2024:06:17:37] ENGINE Bus STARTING 2024-03-22T06:17:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:39 smithi117 ceph-mon[36514]: [22/Mar/2024:06:17:37] ENGINE Serving on http://172.21.15.6:8765 2024-03-22T06:17:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:39 smithi117 ceph-mon[36514]: [22/Mar/2024:06:17:38] ENGINE Serving on https://172.21.15.6:7150 2024-03-22T06:17:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:39 smithi117 ceph-mon[36514]: [22/Mar/2024:06:17:38] ENGINE Bus STARTED 2024-03-22T06:17:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:39 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:39 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:39 smithi117 ceph-mon[36514]: mgrmap e36: smithi006.sfpxdy(active, since 1.01027s), standbys: smithi117.bqeskk 2024-03-22T06:17:39.526 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:39 smithi006 ceph-mon[94482]: [22/Mar/2024:06:17:37] ENGINE Bus STARTING 2024-03-22T06:17:39.526 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:39 smithi006 ceph-mon[94482]: [22/Mar/2024:06:17:37] ENGINE Serving on http://172.21.15.6:8765 2024-03-22T06:17:39.526 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:39 smithi006 ceph-mon[94482]: [22/Mar/2024:06:17:38] ENGINE Serving on https://172.21.15.6:7150 2024-03-22T06:17:39.526 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:39 smithi006 ceph-mon[94482]: [22/Mar/2024:06:17:38] ENGINE Bus STARTED 2024-03-22T06:17:39.526 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:39.526 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:39.526 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:39 smithi006 ceph-mon[94482]: mgrmap e36: smithi006.sfpxdy(active, since 1.01027s), standbys: smithi117.bqeskk 2024-03-22T06:17:40.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:40 smithi117 ceph-mon[36514]: pgmap v3: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:17:40.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:40 smithi117 ceph-mon[36514]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-22T06:17:40.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:40 smithi117 ceph-mon[36514]: Cluster is now healthy 2024-03-22T06:17:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:40 smithi006 ceph-mon[94482]: pgmap v3: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:17:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:40 smithi006 ceph-mon[94482]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-22T06:17:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:40 smithi006 ceph-mon[94482]: Cluster is now healthy 2024-03-22T06:17:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:41 smithi117 ceph-mon[36514]: pgmap v4: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:17:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:41 smithi117 ceph-mon[36514]: mgrmap e37: smithi006.sfpxdy(active, since 3s), standbys: smithi117.bqeskk 2024-03-22T06:17:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:41 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:41 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:41.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:41 smithi006 ceph-mon[94482]: pgmap v4: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:17:41.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:41 smithi006 ceph-mon[94482]: mgrmap e37: smithi006.sfpxdy(active, since 3s), standbys: smithi117.bqeskk 2024-03-22T06:17:41.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:41 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:41.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:41 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:43.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:43 smithi006 ceph-mon[94482]: pgmap v5: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:17:43.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:43 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:43.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:43 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:43.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:43 smithi006 ceph-mon[94482]: mgrmap e38: smithi006.sfpxdy(active, since 4s), standbys: smithi117.bqeskk 2024-03-22T06:17:43.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:43 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:43.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:43 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:43.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:43 smithi117 ceph-mon[36514]: pgmap v5: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:17:43.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:43 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:43.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:43 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:43.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:43 smithi117 ceph-mon[36514]: mgrmap e38: smithi006.sfpxdy(active, since 4s), standbys: smithi117.bqeskk 2024-03-22T06:17:43.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:43 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:43.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:43 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:43.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.801+0000 7fab3dfe8700 1 -- 172.21.15.6:0/4090944931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab380fed90 msgr2=0x7fab380faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:43.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.801+0000 7fab3dfe8700 1 --2- 172.21.15.6:0/4090944931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab380fed90 0x7fab380faed0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab2c004940 tx=0x7fab2c009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:43.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.801+0000 7fab3dfe8700 1 -- 172.21.15.6:0/4090944931 shutdown_connections 2024-03-22T06:17:43.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.801+0000 7fab3dfe8700 1 --2- 172.21.15.6:0/4090944931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab380fed90 0x7fab380faed0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:43.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.801+0000 7fab3dfe8700 1 --2- 172.21.15.6:0/4090944931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab380fe440 0x7fab380fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:43.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.801+0000 7fab3dfe8700 1 -- 172.21.15.6:0/4090944931 >> 172.21.15.6:0/4090944931 conn(0x7fab380f6be0 msgr2=0x7fab380f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:43.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.801+0000 7fab3dfe8700 1 -- 172.21.15.6:0/4090944931 shutdown_connections 2024-03-22T06:17:43.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.801+0000 7fab3dfe8700 1 -- 172.21.15.6:0/4090944931 wait complete. 2024-03-22T06:17:43.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.802+0000 7fab3dfe8700 1 Processor -- start 2024-03-22T06:17:43.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.802+0000 7fab3dfe8700 1 -- start start 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.802+0000 7fab3dfe8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab380fe440 0x7fab380fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.802+0000 7fab3dfe8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab380fed90 0x7fab380fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.802+0000 7fab3dfe8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab380fe050 con 0x7fab380fed90 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.802+0000 7fab3dfe8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab380fc640 con 0x7fab380fe440 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.802+0000 7fab36ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab380fed90 0x7fab380fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.802+0000 7fab36ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab380fed90 0x7fab380fc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37140/0 (socket says 172.21.15.6:37140) 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.803+0000 7fab36ffd700 1 -- 172.21.15.6:0/1212934359 learned_addr learned my addr 172.21.15.6:0/1212934359 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.803+0000 7fab377fe700 1 --2- 172.21.15.6:0/1212934359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab380fe440 0x7fab380fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.803+0000 7fab36ffd700 1 -- 172.21.15.6:0/1212934359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab380fe440 msgr2=0x7fab380fda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.803+0000 7fab36ffd700 1 --2- 172.21.15.6:0/1212934359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab380fe440 0x7fab380fda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.803+0000 7fab36ffd700 1 -- 172.21.15.6:0/1212934359 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c005040 con 0x7fab380fed90 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.803+0000 7fab36ffd700 1 --2- 172.21.15.6:0/1212934359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab380fed90 0x7fab380fc100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00b3b0 tx=0x7fab2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:43.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.803+0000 7fab1ffff700 1 -- 172.21.15.6:0/1212934359 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2c00e040 con 0x7fab380fed90 2024-03-22T06:17:43.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.803+0000 7fab1ffff700 1 -- 172.21.15.6:0/1212934359 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab2c00bbf0 con 0x7fab380fed90 2024-03-22T06:17:43.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.803+0000 7fab3dfe8700 1 -- 172.21.15.6:0/1212934359 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab380fc890 con 0x7fab380fed90 2024-03-22T06:17:43.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.804+0000 7fab1ffff700 1 -- 172.21.15.6:0/1212934359 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2c01c630 con 0x7fab380fed90 2024-03-22T06:17:43.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.804+0000 7fab3dfe8700 1 -- 172.21.15.6:0/1212934359 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab380fce50 con 0x7fab380fed90 2024-03-22T06:17:43.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.805+0000 7fab3dfe8700 1 -- 172.21.15.6:0/1212934359 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3804c710 con 0x7fab380fed90 2024-03-22T06:17:43.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.805+0000 7fab1ffff700 1 -- 172.21.15.6:0/1212934359 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7fab2c00bd60 con 0x7fab380fed90 2024-03-22T06:17:43.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.805+0000 7fab1ffff700 1 --2- 172.21.15.6:0/1212934359 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab2006e760 0x7fab20070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:43.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.805+0000 7fab377fe700 1 --2- 172.21.15.6:0/1212934359 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab2006e760 0x7fab20070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:43.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.805+0000 7fab1ffff700 1 -- 172.21.15.6:0/1212934359 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fab2c08f0f0 con 0x7fab380fed90 2024-03-22T06:17:43.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.806+0000 7fab377fe700 1 --2- 172.21.15.6:0/1212934359 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab2006e760 0x7fab20070c10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab28004f60 tx=0x7fab2800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:43.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:43.810+0000 7fab1ffff700 1 -- 172.21.15.6:0/1212934359 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fab2c05bac0 con 0x7fab380fed90 2024-03-22T06:17:44.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.035+0000 7fab3dfe8700 1 -- 172.21.15.6:0/1212934359 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fab38102560 con 0x7fab2006e760 2024-03-22T06:17:44.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.037+0000 7fab1ffff700 1 -- 172.21.15.6:0/1212934359 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fab38102560 con 0x7fab2006e760 2024-03-22T06:17:44.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.039+0000 7fab3dfe8700 1 -- 172.21.15.6:0/1212934359 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab2006e760 msgr2=0x7fab20070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.039+0000 7fab3dfe8700 1 --2- 172.21.15.6:0/1212934359 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab2006e760 0x7fab20070c10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab28004f60 tx=0x7fab2800b040 comp rx=0 tx=0).stop 2024-03-22T06:17:44.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.039+0000 7fab3dfe8700 1 -- 172.21.15.6:0/1212934359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab380fed90 msgr2=0x7fab380fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.039+0000 7fab3dfe8700 1 --2- 172.21.15.6:0/1212934359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab380fed90 0x7fab380fc100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00b3b0 tx=0x7fab2c009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:44.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.039+0000 7fab3dfe8700 1 -- 172.21.15.6:0/1212934359 shutdown_connections 2024-03-22T06:17:44.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.039+0000 7fab3dfe8700 1 --2- 172.21.15.6:0/1212934359 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab2006e760 0x7fab20070c10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.039+0000 7fab3dfe8700 1 --2- 172.21.15.6:0/1212934359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab380fe440 0x7fab380fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.039+0000 7fab3dfe8700 1 --2- 172.21.15.6:0/1212934359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab380fed90 0x7fab380fc100 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.039+0000 7fab3dfe8700 1 -- 172.21.15.6:0/1212934359 >> 172.21.15.6:0/1212934359 conn(0x7fab380f6be0 msgr2=0x7fab380f82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:44.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.039+0000 7fab3dfe8700 1 -- 172.21.15.6:0/1212934359 shutdown_connections 2024-03-22T06:17:44.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.039+0000 7fab3dfe8700 1 -- 172.21.15.6:0/1212934359 wait complete. 2024-03-22T06:17:44.049 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:17:44.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.142+0000 7fbfbff15700 1 -- 172.21.15.6:0/1548785586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb80fed80 msgr2=0x7fbfb80faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.142+0000 7fbfbff15700 1 --2- 172.21.15.6:0/1548785586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb80fed80 0x7fbfb80faec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8004940 tx=0x7fbfa8009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:44.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.142+0000 7fbfbff15700 1 -- 172.21.15.6:0/1548785586 shutdown_connections 2024-03-22T06:17:44.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.142+0000 7fbfbff15700 1 --2- 172.21.15.6:0/1548785586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb80fed80 0x7fbfb80faec0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.142+0000 7fbfbff15700 1 --2- 172.21.15.6:0/1548785586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb80fe430 0x7fbfb80fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.142+0000 7fbfbff15700 1 -- 172.21.15.6:0/1548785586 >> 172.21.15.6:0/1548785586 conn(0x7fbfb80f6bd0 msgr2=0x7fbfb80f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:44.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.143+0000 7fbfbff15700 1 -- 172.21.15.6:0/1548785586 shutdown_connections 2024-03-22T06:17:44.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.143+0000 7fbfbff15700 1 -- 172.21.15.6:0/1548785586 wait complete. 2024-03-22T06:17:44.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.143+0000 7fbfbff15700 1 Processor -- start 2024-03-22T06:17:44.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.143+0000 7fbfbff15700 1 -- start start 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.143+0000 7fbfbff15700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb80fe430 0x7fbfb80fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.143+0000 7fbfbff15700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb80fed80 0x7fbfb80fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.143+0000 7fbfbff15700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80fdff0 con 0x7fbfb80fe430 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.143+0000 7fbfbff15700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80fc630 con 0x7fbfb80fed80 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.143+0000 7fbfbdcb1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb80fe430 0x7fbfb80fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.143+0000 7fbfbdcb1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb80fe430 0x7fbfb80fd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37158/0 (socket says 172.21.15.6:37158) 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.144+0000 7fbfbdcb1700 1 -- 172.21.15.6:0/4219693745 learned_addr learned my addr 172.21.15.6:0/4219693745 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.144+0000 7fbfbd4b0700 1 --2- 172.21.15.6:0/4219693745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb80fed80 0x7fbfb80fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.144+0000 7fbfbdcb1700 1 -- 172.21.15.6:0/4219693745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb80fed80 msgr2=0x7fbfb80fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.144+0000 7fbfbdcb1700 1 --2- 172.21.15.6:0/4219693745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb80fed80 0x7fbfb80fc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.144+0000 7fbfbdcb1700 1 -- 172.21.15.6:0/4219693745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa8005040 con 0x7fbfb80fe430 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.144+0000 7fbfbdcb1700 1 --2- 172.21.15.6:0/4219693745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb80fe430 0x7fbfb80fd9f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4002a50 tx=0x7fbfb40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.144+0000 7fbfae7fc700 1 -- 172.21.15.6:0/4219693745 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfb4007cc0 con 0x7fbfb80fe430 2024-03-22T06:17:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.144+0000 7fbfae7fc700 1 -- 172.21.15.6:0/4219693745 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfb400f040 con 0x7fbfb80fe430 2024-03-22T06:17:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.144+0000 7fbfbff15700 1 -- 172.21.15.6:0/4219693745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb80fc910 con 0x7fbfb80fe430 2024-03-22T06:17:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.144+0000 7fbfae7fc700 1 -- 172.21.15.6:0/4219693745 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfb400da70 con 0x7fbfb80fe430 2024-03-22T06:17:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.144+0000 7fbfbff15700 1 -- 172.21.15.6:0/4219693745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb80fce50 con 0x7fbfb80fe430 2024-03-22T06:17:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.145+0000 7fbfbff15700 1 -- 172.21.15.6:0/4219693745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb804c710 con 0x7fbfb80fe430 2024-03-22T06:17:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.146+0000 7fbfae7fc700 1 -- 172.21.15.6:0/4219693745 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7fbfb400dc10 con 0x7fbfb80fe430 2024-03-22T06:17:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.146+0000 7fbfae7fc700 1 --2- 172.21.15.6:0/4219693745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fbfa406e970 0x7fbfa4070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.146+0000 7fbfae7fc700 1 -- 172.21.15.6:0/4219693745 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbfb408dc20 con 0x7fbfb80fe430 2024-03-22T06:17:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.146+0000 7fbfbd4b0700 1 --2- 172.21.15.6:0/4219693745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fbfa406e970 0x7fbfa4070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.146+0000 7fbfbd4b0700 1 --2- 172.21.15.6:0/4219693745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fbfa406e970 0x7fbfa4070e20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800b3b0 tx=0x7fbfa8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.151+0000 7fbfae7fc700 1 -- 172.21.15.6:0/4219693745 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fbfb405a570 con 0x7fbfb80fe430 2024-03-22T06:17:44.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.377+0000 7fbfbff15700 1 -- 172.21.15.6:0/4219693745 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfb8102510 con 0x7fbfa406e970 2024-03-22T06:17:44.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.378+0000 7fbfae7fc700 1 -- 172.21.15.6:0/4219693745 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fbfb8102510 con 0x7fbfa406e970 2024-03-22T06:17:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.380+0000 7fbfbff15700 1 -- 172.21.15.6:0/4219693745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fbfa406e970 msgr2=0x7fbfa4070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.380+0000 7fbfbff15700 1 --2- 172.21.15.6:0/4219693745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fbfa406e970 0x7fbfa4070e20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800b3b0 tx=0x7fbfa8009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.380+0000 7fbfbff15700 1 -- 172.21.15.6:0/4219693745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb80fe430 msgr2=0x7fbfb80fd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.380+0000 7fbfbff15700 1 --2- 172.21.15.6:0/4219693745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb80fe430 0x7fbfb80fd9f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4002a50 tx=0x7fbfb40046b0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.380+0000 7fbfbff15700 1 -- 172.21.15.6:0/4219693745 shutdown_connections 2024-03-22T06:17:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.380+0000 7fbfbff15700 1 --2- 172.21.15.6:0/4219693745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fbfa406e970 0x7fbfa4070e20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.380+0000 7fbfbff15700 1 --2- 172.21.15.6:0/4219693745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb80fe430 0x7fbfb80fd9f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.380+0000 7fbfbff15700 1 --2- 172.21.15.6:0/4219693745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfb80fed80 0x7fbfb80fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.380+0000 7fbfbff15700 1 -- 172.21.15.6:0/4219693745 >> 172.21.15.6:0/4219693745 conn(0x7fbfb80f6bd0 msgr2=0x7fbfb80f8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.380+0000 7fbfbff15700 1 -- 172.21.15.6:0/4219693745 shutdown_connections 2024-03-22T06:17:44.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.380+0000 7fbfbff15700 1 -- 172.21.15.6:0/4219693745 wait complete. 2024-03-22T06:17:44.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.483+0000 7f84b45d3700 1 -- 172.21.15.6:0/1293129427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84ac0fbcd0 msgr2=0x7f84ac0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.483+0000 7f84b45d3700 1 --2- 172.21.15.6:0/1293129427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84ac0fbcd0 0x7f84ac0fc0e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f84a8004940 tx=0x7f84a8009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:44.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.483+0000 7f84b45d3700 1 -- 172.21.15.6:0/1293129427 shutdown_connections 2024-03-22T06:17:44.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.483+0000 7f84b45d3700 1 --2- 172.21.15.6:0/1293129427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84ac0fcf80 0x7f84ac0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.483+0000 7f84b45d3700 1 --2- 172.21.15.6:0/1293129427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84ac0fbcd0 0x7f84ac0fc0e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.483+0000 7f84b45d3700 1 -- 172.21.15.6:0/1293129427 >> 172.21.15.6:0/1293129427 conn(0x7f84ac0f7430 msgr2=0x7f84ac0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.483+0000 7f84b45d3700 1 -- 172.21.15.6:0/1293129427 shutdown_connections 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.483+0000 7f84b45d3700 1 -- 172.21.15.6:0/1293129427 wait complete. 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.483+0000 7f84b45d3700 1 Processor -- start 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.483+0000 7f84b45d3700 1 -- start start 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.483+0000 7f84b45d3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84ac0fbcd0 0x7f84ac10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b45d3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84ac0fcf80 0x7f84ac10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b45d3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac10e540 con 0x7f84ac0fbcd0 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b45d3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac10e6b0 con 0x7f84ac0fcf80 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b236f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84ac0fbcd0 0x7f84ac10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b236f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84ac0fbcd0 0x7f84ac10da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37180/0 (socket says 172.21.15.6:37180) 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b236f700 1 -- 172.21.15.6:0/1102082727 learned_addr learned my addr 172.21.15.6:0/1102082727 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b1b6e700 1 --2- 172.21.15.6:0/1102082727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84ac0fcf80 0x7f84ac10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b236f700 1 -- 172.21.15.6:0/1102082727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84ac0fcf80 msgr2=0x7f84ac10df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b236f700 1 --2- 172.21.15.6:0/1102082727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84ac0fcf80 0x7f84ac10df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b236f700 1 -- 172.21.15.6:0/1102082727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a8005040 con 0x7f84ac0fbcd0 2024-03-22T06:17:44.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b236f700 1 --2- 172.21.15.6:0/1102082727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84ac0fbcd0 0x7f84ac10da00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b3b0 tx=0x7f84a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:44.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84a2ffd700 1 -- 172.21.15.6:0/1102082727 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a800e040 con 0x7f84ac0fbcd0 2024-03-22T06:17:44.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84a2ffd700 1 -- 172.21.15.6:0/1102082727 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84a800bbf0 con 0x7f84ac0fbcd0 2024-03-22T06:17:44.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b45d3700 1 -- 172.21.15.6:0/1102082727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ac113010 con 0x7f84ac0fbcd0 2024-03-22T06:17:44.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84a2ffd700 1 -- 172.21.15.6:0/1102082727 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a801c630 con 0x7f84ac0fbcd0 2024-03-22T06:17:44.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.484+0000 7f84b45d3700 1 -- 172.21.15.6:0/1102082727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac1135d0 con 0x7f84ac0fbcd0 2024-03-22T06:17:44.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.485+0000 7f84b45d3700 1 -- 172.21.15.6:0/1102082727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ac04c710 con 0x7f84ac0fbcd0 2024-03-22T06:17:44.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.486+0000 7f84a2ffd700 1 -- 172.21.15.6:0/1102082727 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7f84a800bd60 con 0x7f84ac0fbcd0 2024-03-22T06:17:44.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.487+0000 7f84a2ffd700 1 --2- 172.21.15.6:0/1102082727 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f849806e970 0x7f8498070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:44.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.487+0000 7f84b1b6e700 1 --2- 172.21.15.6:0/1102082727 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f849806e970 0x7f8498070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:44.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.487+0000 7f84a2ffd700 1 -- 172.21.15.6:0/1102082727 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f84a808f200 con 0x7f84ac0fbcd0 2024-03-22T06:17:44.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.487+0000 7f84b1b6e700 1 --2- 172.21.15.6:0/1102082727 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f849806e970 0x7f8498070e20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f849c004f40 tx=0x7f849c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:44.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.491+0000 7f84a2ffd700 1 -- 172.21.15.6:0/1102082727 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f84a805bbd0 con 0x7f84ac0fbcd0 2024-03-22T06:17:44.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.708+0000 7f84b45d3700 1 -- 172.21.15.6:0/1102082727 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f84ac05e400 con 0x7f849806e970 2024-03-22T06:17:44.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.718+0000 7f84a2ffd700 1 -- 172.21.15.6:0/1102082727 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f84ac05e400 con 0x7f849806e970 2024-03-22T06:17:44.718 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:17:44.718 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (2m) 2s ago 10m 27.9M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:17:44.718 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (11m) 2s ago 11m 6656k - 17.2.7-726-g9c4e0c34 837124da804f a49ab95d42e6 2024-03-22T06:17:44.718 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (10m) 3s ago 10m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0a36af1f8ed0 2024-03-22T06:17:44.718 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 unknown 2s ago 10m - - 2024-03-22T06:17:44.718 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (5m) 2s ago 5m 158M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (5m) 2s ago 5m 17.1M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (5m) 3s ago 5m 14.2M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (5m) 3s ago 5m 15.7M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (93s) 2s ago 12m 478M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (34s) 3s ago 9m 447M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (16s) 2s ago 12m 30.0M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (9m) 3s ago 9m 50.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f 0559848da9ab 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (2m) 2s ago 10m 17.4M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (108s) 3s ago 9m 14.9M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (9m) 2s ago 9m 248M 1559M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (8m) 2s ago 8m 258M 1559M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (8m) 2s ago 8m 213M 1559M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (7m) 3s ago 7m 282M 2581M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (7m) 3s ago 7m 249M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:17:44.719 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (6m) 3s ago 6m 236M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:17:44.720 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (40s) 2s ago 10m 53.0M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:17:44.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.720+0000 7f84b45d3700 1 -- 172.21.15.6:0/1102082727 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f849806e970 msgr2=0x7f8498070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.720+0000 7f84b45d3700 1 --2- 172.21.15.6:0/1102082727 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f849806e970 0x7f8498070e20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f849c004f40 tx=0x7f849c00b040 comp rx=0 tx=0).stop 2024-03-22T06:17:44.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.720+0000 7f84b45d3700 1 -- 172.21.15.6:0/1102082727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84ac0fbcd0 msgr2=0x7f84ac10da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.720+0000 7f84b45d3700 1 --2- 172.21.15.6:0/1102082727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84ac0fbcd0 0x7f84ac10da00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b3b0 tx=0x7f84a8009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:44.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.720+0000 7f84b45d3700 1 -- 172.21.15.6:0/1102082727 shutdown_connections 2024-03-22T06:17:44.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.720+0000 7f84b45d3700 1 --2- 172.21.15.6:0/1102082727 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f849806e970 0x7f8498070e20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.720+0000 7f84b45d3700 1 --2- 172.21.15.6:0/1102082727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84ac0fbcd0 0x7f84ac10da00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.720+0000 7f84b45d3700 1 --2- 172.21.15.6:0/1102082727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84ac0fcf80 0x7f84ac10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.720+0000 7f84b45d3700 1 -- 172.21.15.6:0/1102082727 >> 172.21.15.6:0/1102082727 conn(0x7f84ac0f7430 msgr2=0x7f84ac0668c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:44.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.721+0000 7f84b45d3700 1 -- 172.21.15.6:0/1102082727 shutdown_connections 2024-03-22T06:17:44.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.721+0000 7f84b45d3700 1 -- 172.21.15.6:0/1102082727 wait complete. 2024-03-22T06:17:44.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:44 smithi006 ceph-mon[94482]: pgmap v6: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-03-22T06:17:44.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:44 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:44.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:44 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:44.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:44 smithi006 ceph-mon[94482]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:44.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.822+0000 7f605561f700 1 -- 172.21.15.6:0/3062763716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60500fdff0 msgr2=0x7f60500fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.822+0000 7f605561f700 1 --2- 172.21.15.6:0/3062763716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60500fdff0 0x7f60500fe460 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6038004940 tx=0x7f6038009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:44.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.822+0000 7f605561f700 1 -- 172.21.15.6:0/3062763716 shutdown_connections 2024-03-22T06:17:44.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.822+0000 7f605561f700 1 --2- 172.21.15.6:0/3062763716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60500fdff0 0x7f60500fe460 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.822+0000 7f605561f700 1 --2- 172.21.15.6:0/3062763716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60500fce80 0x7f60500fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.822+0000 7f605561f700 1 -- 172.21.15.6:0/3062763716 >> 172.21.15.6:0/3062763716 conn(0x7f60500f8580 msgr2=0x7f60500fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:44.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.822+0000 7f605561f700 1 -- 172.21.15.6:0/3062763716 shutdown_connections 2024-03-22T06:17:44.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.822+0000 7f605561f700 1 -- 172.21.15.6:0/3062763716 wait complete. 2024-03-22T06:17:44.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.822+0000 7f605561f700 1 Processor -- start 2024-03-22T06:17:44.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.822+0000 7f605561f700 1 -- start start 2024-03-22T06:17:44.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f605561f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60500fce80 0x7f605010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:44.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f605561f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60500fdff0 0x7f605010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:44.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f605561f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605010e4c0 con 0x7f60500fce80 2024-03-22T06:17:44.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f605561f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605010ed10 con 0x7f60500fdff0 2024-03-22T06:17:44.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f604effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60500fce80 0x7f605010d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:44.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f604effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60500fce80 0x7f605010d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37194/0 (socket says 172.21.15.6:37194) 2024-03-22T06:17:44.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f604effd700 1 -- 172.21.15.6:0/2461445745 learned_addr learned my addr 172.21.15.6:0/2461445745 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:44.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f604e7fc700 1 --2- 172.21.15.6:0/2461445745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60500fdff0 0x7f605010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:44.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f604effd700 1 -- 172.21.15.6:0/2461445745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60500fdff0 msgr2=0x7f605010dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:44.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f604effd700 1 --2- 172.21.15.6:0/2461445745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60500fdff0 0x7f605010dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:44.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f604effd700 1 -- 172.21.15.6:0/2461445745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6038005040 con 0x7f60500fce80 2024-03-22T06:17:44.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f604effd700 1 --2- 172.21.15.6:0/2461445745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60500fce80 0x7f605010d980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6040001db0 tx=0x7f6040009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:44.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f60477fe700 1 -- 172.21.15.6:0/2461445745 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f604000ed50 con 0x7f60500fce80 2024-03-22T06:17:44.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f60477fe700 1 -- 172.21.15.6:0/2461445745 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6040007df0 con 0x7f60500fce80 2024-03-22T06:17:44.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f605561f700 1 -- 172.21.15.6:0/2461445745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605010eff0 con 0x7f60500fce80 2024-03-22T06:17:44.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f60477fe700 1 -- 172.21.15.6:0/2461445745 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6040014640 con 0x7f60500fce80 2024-03-22T06:17:44.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.823+0000 7f605561f700 1 -- 172.21.15.6:0/2461445745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605010f530 con 0x7f60500fce80 2024-03-22T06:17:44.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.824+0000 7f605561f700 1 -- 172.21.15.6:0/2461445745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f605005f5f0 con 0x7f60500fce80 2024-03-22T06:17:44.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.825+0000 7f60477fe700 1 -- 172.21.15.6:0/2461445745 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7f6040007920 con 0x7f60500fce80 2024-03-22T06:17:44.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.825+0000 7f60477fe700 1 --2- 172.21.15.6:0/2461445745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f603c06e960 0x7f603c070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:44.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.825+0000 7f60477fe700 1 -- 172.21.15.6:0/2461445745 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f604008e330 con 0x7f60500fce80 2024-03-22T06:17:44.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.825+0000 7f604e7fc700 1 --2- 172.21.15.6:0/2461445745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f603c06e960 0x7f603c070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:44.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.826+0000 7f604e7fc700 1 --2- 172.21.15.6:0/2461445745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f603c06e960 0x7f603c070e10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f603800b3b0 tx=0x7f6038009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:44.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:44.830+0000 7f60477fe700 1 -- 172.21.15.6:0/2461445745 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f604005ada0 con 0x7f60500fce80 2024-03-22T06:17:44.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:44 smithi117 ceph-mon[36514]: pgmap v6: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-03-22T06:17:44.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:44 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:44.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:44 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:44.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:44 smithi117 ceph-mon[36514]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:45.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.120+0000 7f605561f700 1 -- 172.21.15.6:0/2461445745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f605004bed0 con 0x7f60500fce80 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.121+0000 7f60477fe700 1 -- 172.21.15.6:0/2461445745 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7f6040079e60 con 0x7f60500fce80 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 1 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:17:45.121 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:17:45.122 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:17:45.122 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:17:45.122 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:17:45.122 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 11, 2024-03-22T06:17:45.122 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 3 2024-03-22T06:17:45.122 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:17:45.122 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:17:45.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.123+0000 7f605561f700 1 -- 172.21.15.6:0/2461445745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f603c06e960 msgr2=0x7f603c070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.123+0000 7f605561f700 1 --2- 172.21.15.6:0/2461445745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f603c06e960 0x7f603c070e10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f603800b3b0 tx=0x7f6038009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:45.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.123+0000 7f605561f700 1 -- 172.21.15.6:0/2461445745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60500fce80 msgr2=0x7f605010d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.123+0000 7f605561f700 1 --2- 172.21.15.6:0/2461445745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60500fce80 0x7f605010d980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6040001db0 tx=0x7f6040009ee0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.123+0000 7f605561f700 1 -- 172.21.15.6:0/2461445745 shutdown_connections 2024-03-22T06:17:45.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.123+0000 7f605561f700 1 --2- 172.21.15.6:0/2461445745 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f603c06e960 0x7f603c070e10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.123+0000 7f605561f700 1 --2- 172.21.15.6:0/2461445745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60500fce80 0x7f605010d980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.123+0000 7f605561f700 1 --2- 172.21.15.6:0/2461445745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60500fdff0 0x7f605010dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.123+0000 7f605561f700 1 -- 172.21.15.6:0/2461445745 >> 172.21.15.6:0/2461445745 conn(0x7f60500f8580 msgr2=0x7f60500f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:45.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.123+0000 7f605561f700 1 -- 172.21.15.6:0/2461445745 shutdown_connections 2024-03-22T06:17:45.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.123+0000 7f605561f700 1 -- 172.21.15.6:0/2461445745 wait complete. 2024-03-22T06:17:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.225+0000 7fc581a8e700 1 -- 172.21.15.6:0/2031614597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc57c0fcf80 msgr2=0x7fc57c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.225+0000 7fc581a8e700 1 --2- 172.21.15.6:0/2031614597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc57c0fcf80 0x7fc57c0fd3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc570004940 tx=0x7fc570009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.225+0000 7fc581a8e700 1 -- 172.21.15.6:0/2031614597 shutdown_connections 2024-03-22T06:17:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.225+0000 7fc581a8e700 1 --2- 172.21.15.6:0/2031614597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc57c0fcf80 0x7fc57c0fd3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.225+0000 7fc581a8e700 1 --2- 172.21.15.6:0/2031614597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc57c0fbcd0 0x7fc57c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.225+0000 7fc581a8e700 1 -- 172.21.15.6:0/2031614597 >> 172.21.15.6:0/2031614597 conn(0x7fc57c0f7430 msgr2=0x7fc57c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.225+0000 7fc581a8e700 1 -- 172.21.15.6:0/2031614597 shutdown_connections 2024-03-22T06:17:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.225+0000 7fc581a8e700 1 -- 172.21.15.6:0/2031614597 wait complete. 2024-03-22T06:17:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.225+0000 7fc581a8e700 1 Processor -- start 2024-03-22T06:17:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc581a8e700 1 -- start start 2024-03-22T06:17:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc581a8e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc57c0fbcd0 0x7fc57c066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc581a8e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc57c0fcf80 0x7fc57c1126d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc581a8e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c112c10 con 0x7fc57c0fcf80 2024-03-22T06:17:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc581a8e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c112d50 con 0x7fc57c0fbcd0 2024-03-22T06:17:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc57affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc57c0fcf80 0x7fc57c1126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc57b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc57c0fbcd0 0x7fc57c066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc57affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc57c0fcf80 0x7fc57c1126d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37200/0 (socket says 172.21.15.6:37200) 2024-03-22T06:17:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc57affd700 1 -- 172.21.15.6:0/1210843785 learned_addr learned my addr 172.21.15.6:0/1210843785 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc57affd700 1 -- 172.21.15.6:0/1210843785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc57c0fbcd0 msgr2=0x7fc57c066d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc57affd700 1 --2- 172.21.15.6:0/1210843785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc57c0fbcd0 0x7fc57c066d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc57affd700 1 -- 172.21.15.6:0/1210843785 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc570005040 con 0x7fc57c0fcf80 2024-03-22T06:17:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.226+0000 7fc57affd700 1 --2- 172.21.15.6:0/1210843785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc57c0fcf80 0x7fc57c1126d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b3b0 tx=0x7fc570009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.227+0000 7fc563fff700 1 -- 172.21.15.6:0/1210843785 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc57000e040 con 0x7fc57c0fcf80 2024-03-22T06:17:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.227+0000 7fc563fff700 1 -- 172.21.15.6:0/1210843785 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc57000bbf0 con 0x7fc57c0fcf80 2024-03-22T06:17:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.227+0000 7fc581a8e700 1 -- 172.21.15.6:0/1210843785 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c112fd0 con 0x7fc57c0fcf80 2024-03-22T06:17:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.227+0000 7fc563fff700 1 -- 172.21.15.6:0/1210843785 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc57001c630 con 0x7fc57c0fcf80 2024-03-22T06:17:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.227+0000 7fc581a8e700 1 -- 172.21.15.6:0/1210843785 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57c113560 con 0x7fc57c0fcf80 2024-03-22T06:17:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.228+0000 7fc581a8e700 1 -- 172.21.15.6:0/1210843785 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57c04c710 con 0x7fc57c0fcf80 2024-03-22T06:17:45.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.228+0000 7fc563fff700 1 -- 172.21.15.6:0/1210843785 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7fc57000bd60 con 0x7fc57c0fcf80 2024-03-22T06:17:45.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.228+0000 7fc563fff700 1 --2- 172.21.15.6:0/1210843785 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc56406e970 0x7fc564070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:45.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.229+0000 7fc563fff700 1 -- 172.21.15.6:0/1210843785 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc57008f150 con 0x7fc57c0fcf80 2024-03-22T06:17:45.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.229+0000 7fc57b7fe700 1 --2- 172.21.15.6:0/1210843785 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc56406e970 0x7fc564070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:45.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.229+0000 7fc57b7fe700 1 --2- 172.21.15.6:0/1210843785 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc56406e970 0x7fc564070e20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc56c004f40 tx=0x7fc56c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:45.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.233+0000 7fc563fff700 1 -- 172.21.15.6:0/1210843785 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fc57005bb20 con 0x7fc57c0fcf80 2024-03-22T06:17:45.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.484+0000 7fc581a8e700 1 -- 172.21.15.6:0/1210843785 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc57c1047c0 con 0x7fc57c0fcf80 2024-03-22T06:17:45.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.485+0000 7fc563fff700 1 -- 172.21.15.6:0/1210843785 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fc57005b6b0 con 0x7fc57c0fcf80 2024-03-22T06:17:45.485 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:17:45.485 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:17:45.485 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:17:45.486 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:17:45.487 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:17:45.488 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:17:45.488 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:17:45.488 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 0 members: 2024-03-22T06:17:45.488 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:45.488 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:17:45.488 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:17:45.488 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:17:45.488 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:17:45.488 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:45.488 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:45.488 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:45.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.487+0000 7fc581a8e700 1 -- 172.21.15.6:0/1210843785 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc56406e970 msgr2=0x7fc564070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.487+0000 7fc581a8e700 1 --2- 172.21.15.6:0/1210843785 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc56406e970 0x7fc564070e20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc56c004f40 tx=0x7fc56c00b040 comp rx=0 tx=0).stop 2024-03-22T06:17:45.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.487+0000 7fc581a8e700 1 -- 172.21.15.6:0/1210843785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc57c0fcf80 msgr2=0x7fc57c1126d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.487+0000 7fc581a8e700 1 --2- 172.21.15.6:0/1210843785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc57c0fcf80 0x7fc57c1126d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b3b0 tx=0x7fc570009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:45.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.487+0000 7fc581a8e700 1 -- 172.21.15.6:0/1210843785 shutdown_connections 2024-03-22T06:17:45.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.487+0000 7fc581a8e700 1 --2- 172.21.15.6:0/1210843785 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc56406e970 0x7fc564070e20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.487+0000 7fc581a8e700 1 --2- 172.21.15.6:0/1210843785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc57c0fbcd0 0x7fc57c066d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.487+0000 7fc581a8e700 1 --2- 172.21.15.6:0/1210843785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc57c0fcf80 0x7fc57c1126d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.487+0000 7fc581a8e700 1 -- 172.21.15.6:0/1210843785 >> 172.21.15.6:0/1210843785 conn(0x7fc57c0f7430 msgr2=0x7fc57c0f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:45.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.487+0000 7fc581a8e700 1 -- 172.21.15.6:0/1210843785 shutdown_connections 2024-03-22T06:17:45.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.487+0000 7fc581a8e700 1 -- 172.21.15.6:0/1210843785 wait complete. 2024-03-22T06:17:45.490 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:17:45.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.592+0000 7f1db041d700 1 -- 172.21.15.6:0/966755638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1da80fed80 msgr2=0x7f1da80faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.592+0000 7f1db041d700 1 --2- 172.21.15.6:0/966755638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1da80fed80 0x7f1da80faec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c004940 tx=0x7f1d9c009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:45.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.592+0000 7f1db041d700 1 -- 172.21.15.6:0/966755638 shutdown_connections 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.592+0000 7f1db041d700 1 --2- 172.21.15.6:0/966755638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1da80fed80 0x7f1da80faec0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.592+0000 7f1db041d700 1 --2- 172.21.15.6:0/966755638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1da80fe430 0x7f1da80fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.592+0000 7f1db041d700 1 -- 172.21.15.6:0/966755638 >> 172.21.15.6:0/966755638 conn(0x7f1da80f6bd0 msgr2=0x7f1da80f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.592+0000 7f1db041d700 1 -- 172.21.15.6:0/966755638 shutdown_connections 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.592+0000 7f1db041d700 1 -- 172.21.15.6:0/966755638 wait complete. 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.592+0000 7f1db041d700 1 Processor -- start 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.592+0000 7f1db041d700 1 -- start start 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1db041d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1da80fe430 0x7f1da80fda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1db041d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1da80fed80 0x7f1da80fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1db041d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80fe040 con 0x7f1da80fed80 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1db041d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80fc630 con 0x7f1da80fe430 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1dad9b8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1da80fed80 0x7f1da80fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1dad9b8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1da80fed80 0x7f1da80fc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37218/0 (socket says 172.21.15.6:37218) 2024-03-22T06:17:45.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1dad9b8700 1 -- 172.21.15.6:0/1099480963 learned_addr learned my addr 172.21.15.6:0/1099480963 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:45.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1dae1b9700 1 --2- 172.21.15.6:0/1099480963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1da80fe430 0x7f1da80fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:45.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1dad9b8700 1 -- 172.21.15.6:0/1099480963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1da80fe430 msgr2=0x7f1da80fda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1dad9b8700 1 --2- 172.21.15.6:0/1099480963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1da80fe430 0x7f1da80fda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1dad9b8700 1 -- 172.21.15.6:0/1099480963 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c005040 con 0x7f1da80fed80 2024-03-22T06:17:45.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1dad9b8700 1 --2- 172.21.15.6:0/1099480963 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1da80fed80 0x7f1da80fc0f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c013870 tx=0x7f1d9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:45.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1da6ffd700 1 -- 172.21.15.6:0/1099480963 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9c021070 con 0x7f1da80fed80 2024-03-22T06:17:45.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1da6ffd700 1 -- 172.21.15.6:0/1099480963 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d9c009430 con 0x7f1da80fed80 2024-03-22T06:17:45.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1da6ffd700 1 -- 172.21.15.6:0/1099480963 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9c00e070 con 0x7f1da80fed80 2024-03-22T06:17:45.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1db041d700 1 -- 172.21.15.6:0/1099480963 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da80fc880 con 0x7f1da80fed80 2024-03-22T06:17:45.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.593+0000 7f1db041d700 1 -- 172.21.15.6:0/1099480963 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da80fcdc0 con 0x7f1da80fed80 2024-03-22T06:17:45.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.594+0000 7f1db041d700 1 -- 172.21.15.6:0/1099480963 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da804c710 con 0x7f1da80fed80 2024-03-22T06:17:45.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.595+0000 7f1da6ffd700 1 -- 172.21.15.6:0/1099480963 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7f1d9c01c750 con 0x7f1da80fed80 2024-03-22T06:17:45.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.595+0000 7f1da6ffd700 1 --2- 172.21.15.6:0/1099480963 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1d9006e900 0x7f1d90070db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:45.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.595+0000 7f1da6ffd700 1 -- 172.21.15.6:0/1099480963 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1d9c08f710 con 0x7f1da80fed80 2024-03-22T06:17:45.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.595+0000 7f1dae1b9700 1 --2- 172.21.15.6:0/1099480963 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1d9006e900 0x7f1d90070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:45.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.596+0000 7f1dae1b9700 1 --2- 172.21.15.6:0/1099480963 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1d9006e900 0x7f1d90070db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1d98001470 tx=0x7f1d9800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:45.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.600+0000 7f1da6ffd700 1 -- 172.21.15.6:0/1099480963 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f1d9c05c0e0 con 0x7f1da80fed80 2024-03-22T06:17:45.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:45 smithi006 ceph-mon[94482]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:45.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:45 smithi006 ceph-mon[94482]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:45.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:45 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2461445745' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:17:45.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:45 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1210843785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:17:45.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.877+0000 7f1db041d700 1 -- 172.21.15.6:0/1099480963 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1da8102420 con 0x7f1d9006e900 2024-03-22T06:17:45.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.879+0000 7f1da6ffd700 1 -- 172.21.15.6:0/1099480963 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f1da8102420 con 0x7f1d9006e900 2024-03-22T06:17:45.879 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:17:45.879 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:17:45.879 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:17:45.880 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:17:45.880 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:17:45.880 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:17:45.880 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:17:45.880 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "3/21 daemons upgraded", 2024-03-22T06:17:45.880 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "", 2024-03-22T06:17:45.880 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:17:45.880 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:17:45.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.881+0000 7f1db041d700 1 -- 172.21.15.6:0/1099480963 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1d9006e900 msgr2=0x7f1d90070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.881+0000 7f1db041d700 1 --2- 172.21.15.6:0/1099480963 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1d9006e900 0x7f1d90070db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1d98001470 tx=0x7f1d9800b040 comp rx=0 tx=0).stop 2024-03-22T06:17:45.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.881+0000 7f1db041d700 1 -- 172.21.15.6:0/1099480963 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1da80fed80 msgr2=0x7f1da80fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.881+0000 7f1db041d700 1 --2- 172.21.15.6:0/1099480963 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1da80fed80 0x7f1da80fc0f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c013870 tx=0x7f1d9c009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:45.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.881+0000 7f1db041d700 1 -- 172.21.15.6:0/1099480963 shutdown_connections 2024-03-22T06:17:45.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.881+0000 7f1db041d700 1 --2- 172.21.15.6:0/1099480963 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1d9006e900 0x7f1d90070db0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.881+0000 7f1db041d700 1 --2- 172.21.15.6:0/1099480963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1da80fe430 0x7f1da80fda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.881+0000 7f1db041d700 1 --2- 172.21.15.6:0/1099480963 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1da80fed80 0x7f1da80fc0f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.881+0000 7f1db041d700 1 -- 172.21.15.6:0/1099480963 >> 172.21.15.6:0/1099480963 conn(0x7f1da80f6bd0 msgr2=0x7f1da80f8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:45.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.881+0000 7f1db041d700 1 -- 172.21.15.6:0/1099480963 shutdown_connections 2024-03-22T06:17:45.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.881+0000 7f1db041d700 1 -- 172.21.15.6:0/1099480963 wait complete. 2024-03-22T06:17:45.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.993+0000 7fceff814700 1 -- 172.21.15.6:0/2976262714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef80fce70 msgr2=0x7fcef80fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.993+0000 7fceff814700 1 --2- 172.21.15.6:0/2976262714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef80fce70 0x7fcef80fd280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcef4004940 tx=0x7fcef4009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.993+0000 7fceff814700 1 -- 172.21.15.6:0/2976262714 shutdown_connections 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.993+0000 7fceff814700 1 --2- 172.21.15.6:0/2976262714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef80fdfe0 0x7fcef80fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.993+0000 7fceff814700 1 --2- 172.21.15.6:0/2976262714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef80fce70 0x7fcef80fd280 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.993+0000 7fceff814700 1 -- 172.21.15.6:0/2976262714 >> 172.21.15.6:0/2976262714 conn(0x7fcef80f8550 msgr2=0x7fcef80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.993+0000 7fceff814700 1 -- 172.21.15.6:0/2976262714 shutdown_connections 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.993+0000 7fceff814700 1 -- 172.21.15.6:0/2976262714 wait complete. 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.994+0000 7fceff814700 1 Processor -- start 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.994+0000 7fceff814700 1 -- start start 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.994+0000 7fceff814700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef80fce70 0x7fcef810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.994+0000 7fceff814700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef80fdfe0 0x7fcef810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.994+0000 7fceff814700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef810e490 con 0x7fcef80fdfe0 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.994+0000 7fceff814700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef810e600 con 0x7fcef80fce70 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.994+0000 7fcefcdaf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef80fdfe0 0x7fcef810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.994+0000 7fcefcdaf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef80fdfe0 0x7fcef810de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37242/0 (socket says 172.21.15.6:37242) 2024-03-22T06:17:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.994+0000 7fcefcdaf700 1 -- 172.21.15.6:0/3564386446 learned_addr learned my addr 172.21.15.6:0/3564386446 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:17:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.994+0000 7fcefd5b0700 1 --2- 172.21.15.6:0/3564386446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef80fce70 0x7fcef810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.995+0000 7fcefcdaf700 1 -- 172.21.15.6:0/3564386446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef80fce70 msgr2=0x7fcef810d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.995+0000 7fcefcdaf700 1 --2- 172.21.15.6:0/3564386446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef80fce70 0x7fcef810d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.995+0000 7fcefcdaf700 1 -- 172.21.15.6:0/3564386446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4005040 con 0x7fcef80fdfe0 2024-03-22T06:17:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.995+0000 7fcefcdaf700 1 --2- 172.21.15.6:0/3564386446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef80fdfe0 0x7fcef810de90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcee8002a50 tx=0x7fcee80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.995+0000 7fceedffb700 1 -- 172.21.15.6:0/3564386446 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcee8007cc0 con 0x7fcef80fdfe0 2024-03-22T06:17:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.995+0000 7fceedffb700 1 -- 172.21.15.6:0/3564386446 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcee800f040 con 0x7fcef80fdfe0 2024-03-22T06:17:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.995+0000 7fceff814700 1 -- 172.21.15.6:0/3564386446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef8113060 con 0x7fcef80fdfe0 2024-03-22T06:17:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.995+0000 7fceedffb700 1 -- 172.21.15.6:0/3564386446 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcee800da30 con 0x7fcef80fdfe0 2024-03-22T06:17:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.995+0000 7fceff814700 1 -- 172.21.15.6:0/3564386446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef8113620 con 0x7fcef80fdfe0 2024-03-22T06:17:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.996+0000 7fceff814700 1 -- 172.21.15.6:0/3564386446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef804c710 con 0x7fcef80fdfe0 2024-03-22T06:17:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.996+0000 7fceedffb700 1 -- 172.21.15.6:0/3564386446 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7fcee800db90 con 0x7fcef80fdfe0 2024-03-22T06:17:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.997+0000 7fceedffb700 1 --2- 172.21.15.6:0/3564386446 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcee406e970 0x7fcee4070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:17:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.997+0000 7fceedffb700 1 -- 172.21.15.6:0/3564386446 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcee808e3f0 con 0x7fcef80fdfe0 2024-03-22T06:17:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.997+0000 7fcefd5b0700 1 --2- 172.21.15.6:0/3564386446 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcee406e970 0x7fcee4070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:17:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:45.997+0000 7fcefd5b0700 1 --2- 172.21.15.6:0/3564386446 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcee406e970 0x7fcee4070e20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcef400b3b0 tx=0x7fcef4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:17:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.001+0000 7fceedffb700 1 -- 172.21.15.6:0/3564386446 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fcee805aca0 con 0x7fcef80fdfe0 2024-03-22T06:17:46.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:45 smithi117 ceph-mon[36514]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:46.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:45 smithi117 ceph-mon[36514]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:46.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:45 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/2461445745' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:17:46.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:45 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/1210843785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:17:46.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.333+0000 7fceff814700 1 -- 172.21.15.6:0/3564386446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcef8113a50 con 0x7fcef80fdfe0 2024-03-22T06:17:46.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.333+0000 7fceedffb700 1 -- 172.21.15.6:0/3564386446 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fcee805a830 con 0x7fcef80fdfe0 2024-03-22T06:17:46.335 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_OK 2024-03-22T06:17:46.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.336+0000 7fceff814700 1 -- 172.21.15.6:0/3564386446 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcee406e970 msgr2=0x7fcee4070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:46.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.336+0000 7fceff814700 1 --2- 172.21.15.6:0/3564386446 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcee406e970 0x7fcee4070e20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcef400b3b0 tx=0x7fcef4009e30 comp rx=0 tx=0).stop 2024-03-22T06:17:46.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.336+0000 7fceff814700 1 -- 172.21.15.6:0/3564386446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef80fdfe0 msgr2=0x7fcef810de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:17:46.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.336+0000 7fceff814700 1 --2- 172.21.15.6:0/3564386446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef80fdfe0 0x7fcef810de90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcee8002a50 tx=0x7fcee80046b0 comp rx=0 tx=0).stop 2024-03-22T06:17:46.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.336+0000 7fceff814700 1 -- 172.21.15.6:0/3564386446 shutdown_connections 2024-03-22T06:17:46.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.336+0000 7fceff814700 1 --2- 172.21.15.6:0/3564386446 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcee406e970 0x7fcee4070e20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:46.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.336+0000 7fceff814700 1 --2- 172.21.15.6:0/3564386446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef80fce70 0x7fcef810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:46.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.336+0000 7fceff814700 1 --2- 172.21.15.6:0/3564386446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef80fdfe0 0x7fcef810de90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:17:46.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.336+0000 7fceff814700 1 -- 172.21.15.6:0/3564386446 >> 172.21.15.6:0/3564386446 conn(0x7fcef80f8550 msgr2=0x7fcef80f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:17:46.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.336+0000 7fceff814700 1 -- 172.21.15.6:0/3564386446 shutdown_connections 2024-03-22T06:17:46.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:17:46.336+0000 7fceff814700 1 -- 172.21.15.6:0/3564386446 wait complete. 2024-03-22T06:17:47.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:46 smithi117 ceph-mon[36514]: pgmap v7: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-22T06:17:47.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:46 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:47.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:46 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:47.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:46 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:17:47.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:46 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:17:47.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:46 smithi117 ceph-mon[36514]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-22T06:17:47.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:46 smithi117 ceph-mon[36514]: Updating smithi117:/etc/ceph/ceph.conf 2024-03-22T06:17:47.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:46 smithi117 ceph-mon[36514]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:47.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:46 smithi117 ceph-mon[36514]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:17:47.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:46 smithi117 ceph-mon[36514]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:17:47.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:46 smithi117 ceph-mon[36514]: from='client.? 172.21.15.6:0/3564386446' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:17:47.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:46 smithi006 ceph-mon[94482]: pgmap v7: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-22T06:17:47.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:46 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:47.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:46 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:47.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:46 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:17:47.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:46 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:17:47.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:46 smithi006 ceph-mon[94482]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-22T06:17:47.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:46 smithi006 ceph-mon[94482]: Updating smithi117:/etc/ceph/ceph.conf 2024-03-22T06:17:47.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:46 smithi006 ceph-mon[94482]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:17:47.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:46 smithi006 ceph-mon[94482]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:17:47.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:46 smithi006 ceph-mon[94482]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T06:17:47.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:46 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3564386446' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:17:48.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:17:48.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:17:48.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:17:48.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:17:48.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:48.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:48.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:48.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:48.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: pgmap v8: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-22T06:17:48.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:48.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:48.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:17:48.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:47 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T06:17:48.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:17:48.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-03-22T06:17:48.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: Updating smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:17:48.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: Updating smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T06:17:48.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:48.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:48.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:48.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:48.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: pgmap v8: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-22T06:17:48.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:48.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:48.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:17:48.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:47 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T06:17:49.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:48 smithi006 ceph-mon[94482]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-22T06:17:49.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:49.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:17:49.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:17:49.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:17:49.054 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:48 smithi117 ceph-mon[36514]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-22T06:17:49.054 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:48 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:49.054 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:48 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:17:49.054 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:48 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:17:49.054 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:48 smithi117 ceph-mon[36514]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:17:49.874 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:49 smithi117 ceph-mon[36514]: Upgrade: Updating mon.smithi117 2024-03-22T06:17:49.874 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:49 smithi117 ceph-mon[36514]: Deploying daemon mon.smithi117 on smithi117 2024-03-22T06:17:49.874 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:49 smithi117 ceph-mon[36514]: pgmap v9: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-22T06:17:50.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:50 smithi117 systemd[1]: Stopping Ceph mon.smithi117 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:17:50.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:49 smithi006 ceph-mon[94482]: Upgrade: Updating mon.smithi117 2024-03-22T06:17:50.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:49 smithi006 ceph-mon[94482]: Deploying daemon mon.smithi117 on smithi117 2024-03-22T06:17:50.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:49 smithi006 ceph-mon[94482]: pgmap v9: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-22T06:17:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:50 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117[36496]: 2024-03-22T06:17:50.312+0000 7f133bf31700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi117 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:17:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:50 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117[36496]: 2024-03-22T06:17:50.312+0000 7f133bf31700 -1 mon.smithi117@1(peon) e2 *** Got Signal Terminated *** 2024-03-22T06:17:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:50 smithi117 podman[81961]: 2024-03-22 06:17:50.353280251 +0000 UTC m=+0.147506869 container died 0559848da9abed16b599fa184d50b6a4b7896e5e2a06dc321bb94070e6e74d4f (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=quincy-9c4e0c3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:17:51.080 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:50 smithi117 podman[81961]: 2024-03-22 06:17:50.86274048 +0000 UTC m=+0.656967097 container cleanup 0559848da9abed16b599fa184d50b6a4b7896e5e2a06dc321bb94070e6e74d4f (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=quincy-9c4e0c3, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-22T06:17:51.081 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:50 smithi117 bash[81961]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117 2024-03-22T06:17:51.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 podman[81975]: 2024-03-22 06:17:51.080885427 +0000 UTC m=+0.726373311 container remove 0559848da9abed16b599fa184d50b6a4b7896e5e2a06dc321bb94070e6e74d4f (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-22T06:17:51.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi117.service: Deactivated successfully. 2024-03-22T06:17:51.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 systemd[1]: Stopped Ceph mon.smithi117 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:17:51.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi117.service: Consumed 6.045s CPU time. 2024-03-22T06:17:51.737 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 systemd[1]: Starting Ceph mon.smithi117 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:17:51.737 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 podman[82107]: 2024-03-22 06:17:51.519186959 +0000 UTC m=+0.018659553 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:17:51.737 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 podman[82107]: 2024-03-22 06:17:51.623564298 +0000 UTC m=+0.123036918 container create 4766be58b3b28999ca9d77f6e2f72e6a05ff8e14dc03f1aae47d21c9050743f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-22T06:17:52.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 podman[82107]: 2024-03-22 06:17:51.842563566 +0000 UTC m=+0.342036154 container init 4766be58b3b28999ca9d77f6e2f72e6a05ff8e14dc03f1aae47d21c9050743f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-22T06:17:52.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 podman[82107]: 2024-03-22 06:17:51.845409689 +0000 UTC m=+0.344882266 container start 4766be58b3b28999ca9d77f6e2f72e6a05ff8e14dc03f1aae47d21c9050743f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, io.buildah.version=1.33.5) 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev), process ceph-mon, pid 2 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: pidfile_write: ignore empty --pid-file 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: load: jerasure load: lrc 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: RocksDB version: 7.9.2 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Git sha 0 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Compile date 2024-03-21 23:09:57 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: DB SUMMARY 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: DB Session ID: 59K71GGT7ER5WIZZ0XUE 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: CURRENT file: CURRENT 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1084 Bytes 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi117/store.db dir, Total Num: 1, files: 000027.sst 2024-03-22T06:17:52.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi117/store.db: 000025.log size: 522635 ; 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.error_if_exists: 0 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.create_if_missing: 0 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.paranoid_checks: 1 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.env: 0x55d199b08b20 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.fs: PosixFileSystem 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.info_log: 0x55d19b0277e0 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.statistics: (nil) 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.use_fsync: 0 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_log_file_size: 0 2024-03-22T06:17:52.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.allow_fallocate: 1 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.use_direct_reads: 0 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.db_log_dir: 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.wal_dir: 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T06:17:52.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.write_buffer_manager: 0x55d19aff81e0 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T06:17:52.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T06:17:52.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.unordered_write: 0 2024-03-22T06:17:52.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T06:17:52.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T06:17:52.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T06:17:52.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T06:17:52.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.row_cache: None 2024-03-22T06:17:52.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.wal_filter: None 2024-03-22T06:17:52.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T06:17:52.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.two_write_queues: 0 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.wal_compression: 0 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.atomic_flush: 0 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.log_readahead_size: 0 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T06:17:52.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_background_jobs: 2 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_background_compactions: -1 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_subcompactions: 1 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_open_files: -1 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T06:17:52.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_background_flushes: -1 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Compression algorithms supported: 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: kZSTD supported: 0 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: kXpressCompression supported: 0 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: kZlibCompression supported: 1 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: kSnappyCompression supported: 1 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: kLZ4Compression supported: 1 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: kBZip2Compression supported: 0 2024-03-22T06:17:52.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi117/store.db/MANIFEST-000009 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.merge_operator: 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_filter: None 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d19b027900) 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: cache_index_and_filter_blocks: 1 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T06:17:52.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: pin_top_level_index_and_filter: 1 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_type: 0 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_block_index_type: 0 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_shortening: 1 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: checksum: 4 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: no_block_cache: 0 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache: 0x55d19b02b090 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_name: BinnedLRUCache 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_options: 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: capacity : 536870912 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: num_shard_bits : 4 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: strict_capacity_limit : 0 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: high_pri_pool_ratio: 0.000 2024-03-22T06:17:52.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_compressed: (nil) 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: persistent_cache: (nil) 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_size: 4096 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_size_deviation: 10 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_restart_interval: 16 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_block_restart_interval: 1 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: metadata_block_size: 4096 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: partition_filters: 0 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: use_delta_encoding: 1 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: filter_policy: bloomfilter 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: whole_key_filtering: 1 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: verify_compression: 0 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: read_amp_bytes_per_bit: 0 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: format_version: 5 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: enable_index_compression: 1 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_align: 0 2024-03-22T06:17:52.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_auto_readahead_size: 262144 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: prepopulate_block_cache: 0 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: initial_auto_readahead_size: 8192 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: num_file_reads_for_auto_readahead: 2 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compression: NoCompression 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.num_levels: 7 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T06:17:52.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-22T06:17:52.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T06:17:52.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.arena_block_size: 1048576 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T06:17:52.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.table_properties_collectors: 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.inplace_update_support: 0 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.bloom_locality: 0 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.max_successive_merges: 0 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T06:17:52.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.ttl: 2592000 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.enable_blob_files: false 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.min_blob_size: 0 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-22T06:17:52.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.blob_file_starting_level: 0 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi117/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12246, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b975ce51-7eb8-4c9e-b78e-bbfa26646168 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711088271874995, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711088271876912, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 307710, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12251, "largest_seqno": 12473, "table_properties": {"data_size": 305634, "index_size": 744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 325, "raw_key_size": 2727, "raw_average_key_size": 25, "raw_value_size": 303116, "raw_average_value_size": 2780, "num_data_blocks": 32, "num_entries": 109, "num_filter_entries": 109, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711088271, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b975ce51-7eb8-4c9e-b78e-bbfa26646168", "db_session_id": "59K71GGT7ER5WIZZ0XUE", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711088271876995, "job": 1, "event": "recovery_finished"} 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi117/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T06:17:52.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d19b118000 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: DB pointer 0x55d19b104000 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** DB Stats ** 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-03-22T06:17:52.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** Compaction Stats [default] ** 2024-03-22T06:17:52.277 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T06:17:52.277 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-22T06:17:52.277 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: L0 1/0 300.50 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 264.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: L6 1/0 7.95 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Sum 2/0 8.24 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 264.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 264.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** Compaction Stats [default] ** 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 264.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-22T06:17:52.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative compaction: 0.00 GB write, 73.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval compaction: 0.00 GB write, 73.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Block cache BinnedLRUCache@0x55d19b02b090#2 capacity: 512.00 MB usage: 1.20 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.38 KB,7.15256e-05%) IndexBlock(1,0.83 KB,0.000157952%) Misc(1,0.00 KB,0%) 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: starting mon.smithi117 rank 1 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi117 fsid fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: mon.smithi117@-1(???) e2 preinit fsid fa0e13c2-e811-11ee-95cd-87774f69a715 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: mon.smithi117@-1(???).mds e14 new map 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: mon.smithi117@-1(???).mds e14 print_map 2024-03-22T06:17:52.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: e14 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: legacy client fscid: 1 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Filesystem 'cephfs' (1) 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: fs_name cephfs 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: epoch 14 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: tableserver 0 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: root 0 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: session_timeout 60 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: session_autoclose 300 2024-03-22T06:17:52.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_file_size 1099511627776 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_xattr_size 65536 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: required_client_features {} 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: last_failure 0 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: last_failure_osd_epoch 0 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_mds 1 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: in 0 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: up {0=14466} 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: failed 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: damaged 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: stopped 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_pools [3] 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: metadata_pool 2 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: inline_data disabled 2024-03-22T06:17:52.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: balancer 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: bal_rank_mask -1 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: standby_count_wanted 1 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: qdb_cluster leader: 0 members: 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Standby daemons: 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: mon.smithi117@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-22T06:17:52.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: mon.smithi117@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T06:17:52.283 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: mon.smithi117@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T06:17:52.283 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: mon.smithi117@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T06:17:52.283 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 ceph-mon[82126]: mon.smithi117@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-03-22T06:17:52.283 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 bash[82107]: 4766be58b3b28999ca9d77f6e2f72e6a05ff8e14dc03f1aae47d21c9050743f1 2024-03-22T06:17:52.283 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:51 smithi117 systemd[1]: Started Ceph mon.smithi117 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:17:53.420 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: mon.smithi006 calling monitor election 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: mon.smithi117 calling monitor election 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: mon.smithi006 is new leader, mons smithi006,smithi117 in quorum (ranks 0,1) 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: monmap e3: 2 mons at {smithi006=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: mgrmap e38: smithi006.sfpxdy(active, since 15s), standbys: smithi117.bqeskk 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:53.421 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:53.422 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:53 smithi117 ceph-mon[82126]: pgmap v11: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-22T06:17:53.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi006"}]: dispatch 2024-03-22T06:17:53.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: mon.smithi006 calling monitor election 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: mon.smithi117 calling monitor election 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: mon.smithi006 is new leader, mons smithi006,smithi117 in quorum (ranks 0,1) 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: monmap e3: 2 mons at {smithi006=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: mgrmap e38: smithi006.sfpxdy(active, since 15s), standbys: smithi117.bqeskk 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:53.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:17:53.507 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:53 smithi006 ceph-mon[94482]: pgmap v11: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-22T06:17:56.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:55 smithi117 ceph-mon[82126]: pgmap v12: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:17:56.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:55 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:56.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:55 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:56.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:55 smithi006 ceph-mon[94482]: pgmap v12: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:17:56.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:55 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:56.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:55 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:57.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:57.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:57.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:57 smithi117 ceph-mon[82126]: pgmap v13: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:17:57.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:57.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:17:57.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:57 smithi006 ceph-mon[94482]: pgmap v13: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:00.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:59 smithi117 ceph-mon[82126]: pgmap v14: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:00.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:00.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:00.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:00.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:18:00.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:00.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:18:00.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:18:00.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:17:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:59 smithi006 ceph-mon[94482]: pgmap v14: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:00.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:18:00.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:00.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:18:00.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:18:00.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:17:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:01.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:00 smithi006 ceph-mon[94482]: pgmap v15: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:01.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:00 smithi006 ceph-mon[94482]: pgmap v16: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:01.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:00 smithi006 ceph-mon[94482]: Reconfiguring mon.smithi006 (monmap changed)... 2024-03-22T06:18:01.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:00 smithi006 ceph-mon[94482]: Reconfiguring daemon mon.smithi006 on smithi006 2024-03-22T06:18:01.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:00 smithi117 ceph-mon[82126]: pgmap v15: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:01.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:00 smithi117 ceph-mon[82126]: pgmap v16: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:01.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:00 smithi117 ceph-mon[82126]: Reconfiguring mon.smithi006 (monmap changed)... 2024-03-22T06:18:01.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:00 smithi117 ceph-mon[82126]: Reconfiguring daemon mon.smithi006 on smithi006 2024-03-22T06:18:02.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:02.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:02.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:02 smithi006 ceph-mon[94482]: Reconfiguring mgr.smithi006.sfpxdy (monmap changed)... 2024-03-22T06:18:02.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi006.sfpxdy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:18:02.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:18:02.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:02.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:02 smithi006 ceph-mon[94482]: Reconfiguring daemon mgr.smithi006.sfpxdy on smithi006 2024-03-22T06:18:02.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:02 smithi006 ceph-mon[94482]: pgmap v17: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:03.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:03.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:03.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:02 smithi117 ceph-mon[82126]: Reconfiguring mgr.smithi006.sfpxdy (monmap changed)... 2024-03-22T06:18:03.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi006.sfpxdy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:18:03.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:18:03.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:03.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:02 smithi117 ceph-mon[82126]: Reconfiguring daemon mgr.smithi006.sfpxdy on smithi006 2024-03-22T06:18:03.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:02 smithi117 ceph-mon[82126]: pgmap v17: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:05.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:05 smithi006 ceph-mon[94482]: pgmap v18: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:05.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:05 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:05.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:05 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:05.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:05 smithi006 ceph-mon[94482]: Reconfiguring crash.smithi006 (monmap changed)... 2024-03-22T06:18:05.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:05 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi006", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:18:05.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:05 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:05.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:05 smithi006 ceph-mon[94482]: Reconfiguring daemon crash.smithi006 on smithi006 2024-03-22T06:18:05.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:05 smithi117 ceph-mon[82126]: pgmap v18: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:05.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:05 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:05.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:05 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:05.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:05 smithi117 ceph-mon[82126]: Reconfiguring crash.smithi006 (monmap changed)... 2024-03-22T06:18:05.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:05 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi006", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:18:05.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:05 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:05.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:05 smithi117 ceph-mon[82126]: Reconfiguring daemon crash.smithi006 on smithi006 2024-03-22T06:18:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:06 smithi006 ceph-mon[94482]: pgmap v19: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:06 smithi006 ceph-mon[94482]: Reconfiguring osd.0 (monmap changed)... 2024-03-22T06:18:07.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T06:18:07.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:07.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:06 smithi006 ceph-mon[94482]: Reconfiguring daemon osd.0 on smithi006 2024-03-22T06:18:07.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:06 smithi117 ceph-mon[82126]: pgmap v19: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:07.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:07.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:07.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:06 smithi117 ceph-mon[82126]: Reconfiguring osd.0 (monmap changed)... 2024-03-22T06:18:07.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T06:18:07.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:07.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:06 smithi117 ceph-mon[82126]: Reconfiguring daemon osd.0 on smithi006 2024-03-22T06:18:08.198 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:18:08.198 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:08.198 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:08.198 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T06:18:08.198 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:08.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:18:08.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:08.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:08.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T06:18:08.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:09.206 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:08 smithi006 ceph-mon[94482]: Reconfiguring osd.1 (monmap changed)... 2024-03-22T06:18:09.206 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:08 smithi006 ceph-mon[94482]: Reconfiguring daemon osd.1 on smithi006 2024-03-22T06:18:09.206 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:08 smithi006 ceph-mon[94482]: pgmap v20: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:09.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:08 smithi117 ceph-mon[82126]: Reconfiguring osd.1 (monmap changed)... 2024-03-22T06:18:09.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:08 smithi117 ceph-mon[82126]: Reconfiguring daemon osd.1 on smithi006 2024-03-22T06:18:09.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:08 smithi117 ceph-mon[82126]: pgmap v20: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:10.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:10 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:10.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:10 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:10.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:10 smithi006 ceph-mon[94482]: Reconfiguring osd.2 (monmap changed)... 2024-03-22T06:18:10.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:10 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T06:18:10.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:10 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:10.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:10 smithi006 ceph-mon[94482]: Reconfiguring daemon osd.2 on smithi006 2024-03-22T06:18:10.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:10 smithi006 ceph-mon[94482]: pgmap v21: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:10.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:10 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:10.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:10 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:10.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:10 smithi117 ceph-mon[82126]: Reconfiguring osd.2 (monmap changed)... 2024-03-22T06:18:10.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:10 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T06:18:10.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:10 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:10.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:10 smithi117 ceph-mon[82126]: Reconfiguring daemon osd.2 on smithi006 2024-03-22T06:18:10.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:10 smithi117 ceph-mon[82126]: pgmap v21: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:12.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:12 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:12.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:12 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:12.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:12 smithi006 ceph-mon[94482]: Reconfiguring mds.cephfs.smithi006.bztopc (monmap changed)... 2024-03-22T06:18:12.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:12 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.bztopc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:18:12.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:12 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:12.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:12 smithi006 ceph-mon[94482]: Reconfiguring daemon mds.cephfs.smithi006.bztopc on smithi006 2024-03-22T06:18:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:12 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:12 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:12 smithi117 ceph-mon[82126]: Reconfiguring mds.cephfs.smithi006.bztopc (monmap changed)... 2024-03-22T06:18:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:12 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.bztopc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:18:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:12 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:12.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:12 smithi117 ceph-mon[82126]: Reconfiguring daemon mds.cephfs.smithi006.bztopc on smithi006 2024-03-22T06:18:13.456 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:13 smithi006 ceph-mon[94482]: pgmap v22: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:13.456 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:13.457 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:13.457 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.dcjpla", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:18:13.457 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:13.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:13 smithi117 ceph-mon[82126]: pgmap v22: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:13.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:13.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:13.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.dcjpla", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:18:13.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:14.147 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:14 smithi006 ceph-mon[94482]: Reconfiguring mds.cephfs.smithi006.dcjpla (monmap changed)... 2024-03-22T06:18:14.148 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:14 smithi006 ceph-mon[94482]: Reconfiguring daemon mds.cephfs.smithi006.dcjpla on smithi006 2024-03-22T06:18:14.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:14 smithi117 ceph-mon[82126]: Reconfiguring mds.cephfs.smithi006.dcjpla (monmap changed)... 2024-03-22T06:18:14.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:14 smithi117 ceph-mon[82126]: Reconfiguring daemon mds.cephfs.smithi006.dcjpla on smithi006 2024-03-22T06:18:15.143 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:15 smithi006 ceph-mon[94482]: pgmap v23: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:15.143 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:15 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:15.143 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:15 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:15.143 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:15 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:18:15.143 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:15 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:15.428 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:15 smithi117 ceph-mon[82126]: pgmap v23: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:15.428 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:15 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:15.428 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:15 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:15.428 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:15 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:18:15.428 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:15 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:16.137 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:16 smithi006 ceph-mon[94482]: Reconfiguring crash.smithi117 (monmap changed)... 2024-03-22T06:18:16.138 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:16 smithi006 ceph-mon[94482]: Reconfiguring daemon crash.smithi117 on smithi117 2024-03-22T06:18:16.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.444+0000 7f79c8049700 1 -- 172.21.15.6:0/2539507905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79c00fcf80 msgr2=0x7f79c00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:16.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.444+0000 7f79c8049700 1 --2- 172.21.15.6:0/2539507905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79c00fcf80 0x7f79c00fd3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f79bc004940 tx=0x7f79bc009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:16.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.444+0000 7f79c8049700 1 -- 172.21.15.6:0/2539507905 shutdown_connections 2024-03-22T06:18:16.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.444+0000 7f79c8049700 1 --2- 172.21.15.6:0/2539507905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79c00fcf80 0x7f79c00fd3f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:16.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.444+0000 7f79c8049700 1 --2- 172.21.15.6:0/2539507905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79c00fbcd0 0x7f79c00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:16.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.444+0000 7f79c8049700 1 -- 172.21.15.6:0/2539507905 >> 172.21.15.6:0/2539507905 conn(0x7f79c00f7430 msgr2=0x7f79c00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:16.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.444+0000 7f79c8049700 1 -- 172.21.15.6:0/2539507905 shutdown_connections 2024-03-22T06:18:16.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.444+0000 7f79c8049700 1 -- 172.21.15.6:0/2539507905 wait complete. 2024-03-22T06:18:16.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.445+0000 7f79c8049700 1 Processor -- start 2024-03-22T06:18:16.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.445+0000 7f79c8049700 1 -- start start 2024-03-22T06:18:16.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.445+0000 7f79c8049700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79c00fbcd0 0x7f79c0066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:16.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.445+0000 7f79c8049700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79c00fcf80 0x7f79c0112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.445+0000 7f79c8049700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0112c70 con 0x7f79c00fcf80 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.445+0000 7f79c8049700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0112db0 con 0x7f79c00fbcd0 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.445+0000 7f79c55e4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79c00fcf80 0x7f79c0112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.445+0000 7f79c55e4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79c00fcf80 0x7f79c0112730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38404/0 (socket says 172.21.15.6:38404) 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.445+0000 7f79c55e4700 1 -- 172.21.15.6:0/1041790659 learned_addr learned my addr 172.21.15.6:0/1041790659 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.445+0000 7f79c5de5700 1 --2- 172.21.15.6:0/1041790659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79c00fbcd0 0x7f79c0066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.446+0000 7f79c55e4700 1 -- 172.21.15.6:0/1041790659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79c00fbcd0 msgr2=0x7f79c0066cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.446+0000 7f79c55e4700 1 --2- 172.21.15.6:0/1041790659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79c00fbcd0 0x7f79c0066cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.446+0000 7f79c55e4700 1 -- 172.21.15.6:0/1041790659 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc005040 con 0x7f79c00fcf80 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.446+0000 7f79c55e4700 1 --2- 172.21.15.6:0/1041790659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79c00fcf80 0x7f79c0112730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00b3b0 tx=0x7f79bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.446+0000 7f79b67fc700 1 -- 172.21.15.6:0/1041790659 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79bc00e040 con 0x7f79c00fcf80 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.446+0000 7f79b67fc700 1 -- 172.21.15.6:0/1041790659 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79bc00bbf0 con 0x7f79c00fcf80 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.446+0000 7f79b67fc700 1 -- 172.21.15.6:0/1041790659 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79bc01c630 con 0x7f79c00fcf80 2024-03-22T06:18:16.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.446+0000 7f79c8049700 1 -- 172.21.15.6:0/1041790659 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c0112fb0 con 0x7f79c00fcf80 2024-03-22T06:18:16.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.446+0000 7f79c8049700 1 -- 172.21.15.6:0/1041790659 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c01134f0 con 0x7f79c00fcf80 2024-03-22T06:18:16.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.447+0000 7f79c8049700 1 -- 172.21.15.6:0/1041790659 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c004c710 con 0x7f79c00fcf80 2024-03-22T06:18:16.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.448+0000 7f79b67fc700 1 -- 172.21.15.6:0/1041790659 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7f79bc00bd60 con 0x7f79c00fcf80 2024-03-22T06:18:16.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.448+0000 7f79b67fc700 1 --2- 172.21.15.6:0/1041790659 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79ac06e970 0x7f79ac070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:16.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.448+0000 7f79b67fc700 1 -- 172.21.15.6:0/1041790659 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f79bc08f1b0 con 0x7f79c00fcf80 2024-03-22T06:18:16.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.448+0000 7f79c5de5700 1 --2- 172.21.15.6:0/1041790659 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79ac06e970 0x7f79ac070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:16.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.448+0000 7f79c5de5700 1 --2- 172.21.15.6:0/1041790659 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79ac06e970 0x7f79ac070e20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f79b0004f40 tx=0x7f79b000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:16.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.453+0000 7f79b67fc700 1 -- 172.21.15.6:0/1041790659 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f79bc05bb20 con 0x7f79c00fcf80 2024-03-22T06:18:16.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:16 smithi117 ceph-mon[82126]: Reconfiguring crash.smithi117 (monmap changed)... 2024-03-22T06:18:16.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:16 smithi117 ceph-mon[82126]: Reconfiguring daemon crash.smithi117 on smithi117 2024-03-22T06:18:16.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.678+0000 7f79c8049700 1 -- 172.21.15.6:0/1041790659 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79c0101340 con 0x7f79ac06e970 2024-03-22T06:18:16.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.679+0000 7f79b67fc700 1 -- 172.21.15.6:0/1041790659 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f79c0101340 con 0x7f79ac06e970 2024-03-22T06:18:16.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.681+0000 7f79c8049700 1 -- 172.21.15.6:0/1041790659 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79ac06e970 msgr2=0x7f79ac070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:16.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.681+0000 7f79c8049700 1 --2- 172.21.15.6:0/1041790659 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79ac06e970 0x7f79ac070e20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f79b0004f40 tx=0x7f79b000b040 comp rx=0 tx=0).stop 2024-03-22T06:18:16.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.681+0000 7f79c8049700 1 -- 172.21.15.6:0/1041790659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79c00fcf80 msgr2=0x7f79c0112730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:16.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.681+0000 7f79c8049700 1 --2- 172.21.15.6:0/1041790659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79c00fcf80 0x7f79c0112730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00b3b0 tx=0x7f79bc009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:16.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.681+0000 7f79c8049700 1 -- 172.21.15.6:0/1041790659 shutdown_connections 2024-03-22T06:18:16.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.681+0000 7f79c8049700 1 --2- 172.21.15.6:0/1041790659 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79ac06e970 0x7f79ac070e20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:16.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.681+0000 7f79c8049700 1 --2- 172.21.15.6:0/1041790659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79c00fbcd0 0x7f79c0066cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:16.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.681+0000 7f79c8049700 1 --2- 172.21.15.6:0/1041790659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79c00fcf80 0x7f79c0112730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:16.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.681+0000 7f79c8049700 1 -- 172.21.15.6:0/1041790659 >> 172.21.15.6:0/1041790659 conn(0x7f79c00f7430 msgr2=0x7f79c00f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.681+0000 7f79c8049700 1 -- 172.21.15.6:0/1041790659 shutdown_connections 2024-03-22T06:18:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.681+0000 7f79c8049700 1 -- 172.21.15.6:0/1041790659 wait complete. 2024-03-22T06:18:16.692 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:18:16.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.787+0000 7fed29551700 1 -- 172.21.15.6:0/1125227286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed240fcf80 msgr2=0x7fed240fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:16.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.787+0000 7fed29551700 1 --2- 172.21.15.6:0/1125227286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed240fcf80 0x7fed240fd3f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fed0c004940 tx=0x7fed0c009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:16.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.787+0000 7fed29551700 1 -- 172.21.15.6:0/1125227286 shutdown_connections 2024-03-22T06:18:16.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.787+0000 7fed29551700 1 --2- 172.21.15.6:0/1125227286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed240fcf80 0x7fed240fd3f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:16.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.787+0000 7fed29551700 1 --2- 172.21.15.6:0/1125227286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed240fbcd0 0x7fed240fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.787+0000 7fed29551700 1 -- 172.21.15.6:0/1125227286 >> 172.21.15.6:0/1125227286 conn(0x7fed240f7430 msgr2=0x7fed240f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.787+0000 7fed29551700 1 -- 172.21.15.6:0/1125227286 shutdown_connections 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.787+0000 7fed29551700 1 -- 172.21.15.6:0/1125227286 wait complete. 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed29551700 1 Processor -- start 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed29551700 1 -- start start 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed29551700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed240fbcd0 0x7fed24110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed29551700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed240fcf80 0x7fed24112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed29551700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed24066370 con 0x7fed240fcf80 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed29551700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed240664b0 con 0x7fed240fbcd0 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed227fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed240fcf80 0x7fed24112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed227fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed240fcf80 0x7fed24112c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38430/0 (socket says 172.21.15.6:38430) 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed227fc700 1 -- 172.21.15.6:0/2059816178 learned_addr learned my addr 172.21.15.6:0/2059816178 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:16.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed22ffd700 1 --2- 172.21.15.6:0/2059816178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed240fbcd0 0x7fed24110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:16.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed227fc700 1 -- 172.21.15.6:0/2059816178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed240fbcd0 msgr2=0x7fed24110700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:16.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed227fc700 1 --2- 172.21.15.6:0/2059816178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed240fbcd0 0x7fed24110700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:16.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed227fc700 1 -- 172.21.15.6:0/2059816178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c005040 con 0x7fed240fcf80 2024-03-22T06:18:16.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.788+0000 7fed227fc700 1 --2- 172.21.15.6:0/2059816178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed240fcf80 0x7fed24112c50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009150 tx=0x7fed0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:16.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.789+0000 7fed1b7fe700 1 -- 172.21.15.6:0/2059816178 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed0c00e040 con 0x7fed240fcf80 2024-03-22T06:18:16.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.789+0000 7fed1b7fe700 1 -- 172.21.15.6:0/2059816178 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed0c004290 con 0x7fed240fcf80 2024-03-22T06:18:16.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.789+0000 7fed29551700 1 -- 172.21.15.6:0/2059816178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed241131f0 con 0x7fed240fcf80 2024-03-22T06:18:16.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.789+0000 7fed1b7fe700 1 -- 172.21.15.6:0/2059816178 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed0c01c630 con 0x7fed240fcf80 2024-03-22T06:18:16.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.789+0000 7fed29551700 1 -- 172.21.15.6:0/2059816178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed24113780 con 0x7fed240fcf80 2024-03-22T06:18:16.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.790+0000 7fed29551700 1 -- 172.21.15.6:0/2059816178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2405f5f0 con 0x7fed240fcf80 2024-03-22T06:18:16.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.790+0000 7fed1b7fe700 1 -- 172.21.15.6:0/2059816178 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7fed0c00bbf0 con 0x7fed240fcf80 2024-03-22T06:18:16.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.791+0000 7fed1b7fe700 1 --2- 172.21.15.6:0/2059816178 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fed1006e960 0x7fed10070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:16.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.791+0000 7fed1b7fe700 1 -- 172.21.15.6:0/2059816178 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fed0c08f190 con 0x7fed240fcf80 2024-03-22T06:18:16.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.791+0000 7fed22ffd700 1 --2- 172.21.15.6:0/2059816178 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fed1006e960 0x7fed10070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:16.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.791+0000 7fed22ffd700 1 --2- 172.21.15.6:0/2059816178 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fed1006e960 0x7fed10070e10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed24065ff0 tx=0x7fed1400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:16.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:16.795+0000 7fed1b7fe700 1 -- 172.21.15.6:0/2059816178 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fed0c058030 con 0x7fed240fcf80 2024-03-22T06:18:17.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.021+0000 7fed29551700 1 -- 172.21.15.6:0/2059816178 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed24101360 con 0x7fed1006e960 2024-03-22T06:18:17.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.023+0000 7fed1b7fe700 1 -- 172.21.15.6:0/2059816178 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fed24101360 con 0x7fed1006e960 2024-03-22T06:18:17.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.025+0000 7fed29551700 1 -- 172.21.15.6:0/2059816178 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fed1006e960 msgr2=0x7fed10070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.025+0000 7fed29551700 1 --2- 172.21.15.6:0/2059816178 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fed1006e960 0x7fed10070e10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed24065ff0 tx=0x7fed1400b040 comp rx=0 tx=0).stop 2024-03-22T06:18:17.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.025+0000 7fed29551700 1 -- 172.21.15.6:0/2059816178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed240fcf80 msgr2=0x7fed24112c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.025+0000 7fed29551700 1 --2- 172.21.15.6:0/2059816178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed240fcf80 0x7fed24112c50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009150 tx=0x7fed0c009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:17.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.025+0000 7fed29551700 1 -- 172.21.15.6:0/2059816178 shutdown_connections 2024-03-22T06:18:17.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.025+0000 7fed29551700 1 --2- 172.21.15.6:0/2059816178 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fed1006e960 0x7fed10070e10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.025+0000 7fed29551700 1 --2- 172.21.15.6:0/2059816178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed240fbcd0 0x7fed24110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.025+0000 7fed29551700 1 --2- 172.21.15.6:0/2059816178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed240fcf80 0x7fed24112c50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.025+0000 7fed29551700 1 -- 172.21.15.6:0/2059816178 >> 172.21.15.6:0/2059816178 conn(0x7fed240f7430 msgr2=0x7fed240f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:17.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.025+0000 7fed29551700 1 -- 172.21.15.6:0/2059816178 shutdown_connections 2024-03-22T06:18:17.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.025+0000 7fed29551700 1 -- 172.21.15.6:0/2059816178 wait complete. 2024-03-22T06:18:17.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.127+0000 7f97538c8700 1 -- 172.21.15.6:0/2537070544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fdfe0 msgr2=0x7f974c0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.127+0000 7f97538c8700 1 --2- 172.21.15.6:0/2537070544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fdfe0 0x7f974c0fe450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f973c004940 tx=0x7f973c009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:17.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.127+0000 7f97538c8700 1 -- 172.21.15.6:0/2537070544 shutdown_connections 2024-03-22T06:18:17.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.127+0000 7f97538c8700 1 --2- 172.21.15.6:0/2537070544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fdfe0 0x7f974c0fe450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.127+0000 7f97538c8700 1 --2- 172.21.15.6:0/2537070544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f974c0fce70 0x7f974c0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.127+0000 7f97538c8700 1 -- 172.21.15.6:0/2537070544 >> 172.21.15.6:0/2537070544 conn(0x7f974c0f8550 msgr2=0x7f974c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:17.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.127+0000 7f97538c8700 1 -- 172.21.15.6:0/2537070544 shutdown_connections 2024-03-22T06:18:17.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.127+0000 7f97538c8700 1 -- 172.21.15.6:0/2537070544 wait complete. 2024-03-22T06:18:17.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f97538c8700 1 Processor -- start 2024-03-22T06:18:17.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f97538c8700 1 -- start start 2024-03-22T06:18:17.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f97538c8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f974c0fce70 0x7f974c071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f97538c8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fdfe0 0x7f974c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f97538c8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c072260 con 0x7f974c0fdfe0 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f97538c8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c070850 con 0x7f974c0fce70 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f9750e63700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fdfe0 0x7f974c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f9750e63700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fdfe0 0x7f974c070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38448/0 (socket says 172.21.15.6:38448) 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f9750e63700 1 -- 172.21.15.6:0/3262005504 learned_addr learned my addr 172.21.15.6:0/3262005504 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f9751664700 1 --2- 172.21.15.6:0/3262005504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f974c0fce70 0x7f974c071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f9750e63700 1 -- 172.21.15.6:0/3262005504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f974c0fce70 msgr2=0x7f974c071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f9750e63700 1 --2- 172.21.15.6:0/3262005504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f974c0fce70 0x7f974c071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f9750e63700 1 -- 172.21.15.6:0/3262005504 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c005040 con 0x7f974c0fdfe0 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.128+0000 7f9750e63700 1 --2- 172.21.15.6:0/3262005504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fdfe0 0x7f974c070310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f973c009150 tx=0x7f973c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.129+0000 7f9741ffb700 1 -- 172.21.15.6:0/3262005504 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f973c00e040 con 0x7f974c0fdfe0 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.129+0000 7f9741ffb700 1 -- 172.21.15.6:0/3262005504 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f973c00bbf0 con 0x7f974c0fdfe0 2024-03-22T06:18:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.129+0000 7f97538c8700 1 -- 172.21.15.6:0/3262005504 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c070aa0 con 0x7f974c0fdfe0 2024-03-22T06:18:17.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.129+0000 7f9741ffb700 1 -- 172.21.15.6:0/3262005504 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f973c01c630 con 0x7f974c0fdfe0 2024-03-22T06:18:17.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.129+0000 7f97538c8700 1 -- 172.21.15.6:0/3262005504 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c071060 con 0x7f974c0fdfe0 2024-03-22T06:18:17.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.130+0000 7f97538c8700 1 -- 172.21.15.6:0/3262005504 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c04c710 con 0x7f974c0fdfe0 2024-03-22T06:18:17.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.131+0000 7f9741ffb700 1 -- 172.21.15.6:0/3262005504 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7f973c00bd60 con 0x7f974c0fdfe0 2024-03-22T06:18:17.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.131+0000 7f9741ffb700 1 --2- 172.21.15.6:0/3262005504 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f973806e970 0x7f9738070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:17.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.131+0000 7f9741ffb700 1 -- 172.21.15.6:0/3262005504 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f973c08f860 con 0x7f974c0fdfe0 2024-03-22T06:18:17.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.131+0000 7f9751664700 1 --2- 172.21.15.6:0/3262005504 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f973806e970 0x7f9738070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:17.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.132+0000 7f9751664700 1 --2- 172.21.15.6:0/3262005504 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f973806e970 0x7f9738070e20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9748004f40 tx=0x7f974800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:17.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.136+0000 7f9741ffb700 1 -- 172.21.15.6:0/3262005504 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f973c05c2d0 con 0x7f974c0fdfe0 2024-03-22T06:18:17.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.355+0000 7f97538c8700 1 -- 172.21.15.6:0/3262005504 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f974c1023a0 con 0x7f973806e970 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.365+0000 7f9741ffb700 1 -- 172.21.15.6:0/3262005504 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f974c1023a0 con 0x7f973806e970 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (2m) 34s ago 11m 27.9M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (11m) 34s ago 11m 6656k - 17.2.7-726-g9c4e0c34 837124da804f a49ab95d42e6 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (10m) 22s ago 10m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0a36af1f8ed0 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 unknown 34s ago 11m - - 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (5m) 34s ago 5m 158M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (5m) 34s ago 5m 17.1M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (5m) 22s ago 5m 14.4M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (5m) 22s ago 5m 16.1M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (2m) 34s ago 12m 478M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:18:17.366 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (66s) 22s ago 10m 449M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:18:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (48s) 34s ago 12m 30.0M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:18:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (25s) 22s ago 10m 20.8M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:18:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (3m) 34s ago 11m 17.4M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:18:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (2m) 22s ago 10m 14.9M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:18:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (9m) 34s ago 9m 248M 1559M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:18:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (9m) 34s ago 9m 258M 1559M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:18:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (8m) 34s ago 8m 213M 1559M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:18:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (8m) 22s ago 8m 282M 2581M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:18:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (7m) 22s ago 7m 249M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:18:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (7m) 22s ago 7m 237M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:18:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (73s) 34s ago 10m 53.0M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:18:17.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.367+0000 7f97538c8700 1 -- 172.21.15.6:0/3262005504 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f973806e970 msgr2=0x7f9738070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.367+0000 7f97538c8700 1 --2- 172.21.15.6:0/3262005504 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f973806e970 0x7f9738070e20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9748004f40 tx=0x7f974800b040 comp rx=0 tx=0).stop 2024-03-22T06:18:17.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.367+0000 7f97538c8700 1 -- 172.21.15.6:0/3262005504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fdfe0 msgr2=0x7f974c070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.367+0000 7f97538c8700 1 --2- 172.21.15.6:0/3262005504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fdfe0 0x7f974c070310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f973c009150 tx=0x7f973c009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:17.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.367+0000 7f97538c8700 1 -- 172.21.15.6:0/3262005504 shutdown_connections 2024-03-22T06:18:17.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.367+0000 7f97538c8700 1 --2- 172.21.15.6:0/3262005504 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f973806e970 0x7f9738070e20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.367+0000 7f97538c8700 1 --2- 172.21.15.6:0/3262005504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f974c0fce70 0x7f974c071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.367+0000 7f97538c8700 1 --2- 172.21.15.6:0/3262005504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c0fdfe0 0x7f974c070310 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.367+0000 7f97538c8700 1 -- 172.21.15.6:0/3262005504 >> 172.21.15.6:0/3262005504 conn(0x7f974c0f8550 msgr2=0x7f974c0f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:17.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.367+0000 7f97538c8700 1 -- 172.21.15.6:0/3262005504 shutdown_connections 2024-03-22T06:18:17.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.367+0000 7f97538c8700 1 -- 172.21.15.6:0/3262005504 wait complete. 2024-03-22T06:18:17.436 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:17 smithi117 ceph-mon[82126]: pgmap v24: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:17.436 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:17 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:17.436 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:17 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:17.436 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:17 smithi117 ceph-mon[82126]: Reconfiguring mgr.smithi117.bqeskk (monmap changed)... 2024-03-22T06:18:17.437 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:17 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:18:17.437 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:17 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:18:17.437 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:17 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:17.437 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:17 smithi117 ceph-mon[82126]: Reconfiguring daemon mgr.smithi117.bqeskk on smithi117 2024-03-22T06:18:17.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.468+0000 7f704c4cc700 1 -- 172.21.15.6:0/1330324045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70440fce70 msgr2=0x7f70440fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.468+0000 7f704c4cc700 1 --2- 172.21.15.6:0/1330324045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70440fce70 0x7f70440fd280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7040004940 tx=0x7f7040009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:17.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.468+0000 7f704c4cc700 1 -- 172.21.15.6:0/1330324045 shutdown_connections 2024-03-22T06:18:17.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.468+0000 7f704c4cc700 1 --2- 172.21.15.6:0/1330324045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70440fdfe0 0x7f70440fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.468+0000 7f704c4cc700 1 --2- 172.21.15.6:0/1330324045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70440fce70 0x7f70440fd280 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.468+0000 7f704c4cc700 1 -- 172.21.15.6:0/1330324045 >> 172.21.15.6:0/1330324045 conn(0x7f70440f8550 msgr2=0x7f70440fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:17.470 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:17 smithi006 ceph-mon[94482]: pgmap v24: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:17.470 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:17 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:17.470 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:17 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:17.470 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:17 smithi006 ceph-mon[94482]: Reconfiguring mgr.smithi117.bqeskk (monmap changed)... 2024-03-22T06:18:17.470 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:17 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.bqeskk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T06:18:17.470 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:17 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T06:18:17.470 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:17 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:17.470 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:17 smithi006 ceph-mon[94482]: Reconfiguring daemon mgr.smithi117.bqeskk on smithi117 2024-03-22T06:18:17.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.469+0000 7f704c4cc700 1 -- 172.21.15.6:0/1330324045 shutdown_connections 2024-03-22T06:18:17.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.469+0000 7f704c4cc700 1 -- 172.21.15.6:0/1330324045 wait complete. 2024-03-22T06:18:17.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.469+0000 7f704c4cc700 1 Processor -- start 2024-03-22T06:18:17.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.469+0000 7f704c4cc700 1 -- start start 2024-03-22T06:18:17.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.469+0000 7f704c4cc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70440fce70 0x7f704410d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:17.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.469+0000 7f704c4cc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70440fdfe0 0x7f704410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:17.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.469+0000 7f704c4cc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704410e490 con 0x7f70440fce70 2024-03-22T06:18:17.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.469+0000 7f704c4cc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704410e600 con 0x7f70440fdfe0 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.469+0000 7f704a268700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70440fce70 0x7f704410d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.469+0000 7f704a268700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70440fce70 0x7f704410d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38456/0 (socket says 172.21.15.6:38456) 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.470+0000 7f704a268700 1 -- 172.21.15.6:0/193212674 learned_addr learned my addr 172.21.15.6:0/193212674 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.470+0000 7f7049a67700 1 --2- 172.21.15.6:0/193212674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70440fdfe0 0x7f704410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.470+0000 7f704a268700 1 -- 172.21.15.6:0/193212674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70440fdfe0 msgr2=0x7f704410de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.470+0000 7f704a268700 1 --2- 172.21.15.6:0/193212674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70440fdfe0 0x7f704410de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.470+0000 7f704a268700 1 -- 172.21.15.6:0/193212674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040005040 con 0x7f70440fce70 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.470+0000 7f704a268700 1 --2- 172.21.15.6:0/193212674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70440fce70 0x7f704410d950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f704000b3b0 tx=0x7f7040009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.470+0000 7f703affd700 1 -- 172.21.15.6:0/193212674 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f704000e040 con 0x7f70440fce70 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.470+0000 7f703affd700 1 -- 172.21.15.6:0/193212674 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7040004290 con 0x7f70440fce70 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.470+0000 7f704c4cc700 1 -- 172.21.15.6:0/193212674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7044113000 con 0x7f70440fce70 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.470+0000 7f704c4cc700 1 -- 172.21.15.6:0/193212674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70441135c0 con 0x7f70440fce70 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.470+0000 7f703affd700 1 -- 172.21.15.6:0/193212674 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f704001c630 con 0x7f70440fce70 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.471+0000 7f704c4cc700 1 -- 172.21.15.6:0/193212674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704404c710 con 0x7f70440fce70 2024-03-22T06:18:17.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.472+0000 7f703affd700 1 -- 172.21.15.6:0/193212674 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7f704000bbf0 con 0x7f70440fce70 2024-03-22T06:18:17.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.472+0000 7f703affd700 1 --2- 172.21.15.6:0/193212674 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f703006e970 0x7f7030070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:17.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.472+0000 7f703affd700 1 -- 172.21.15.6:0/193212674 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f704008fa10 con 0x7f70440fce70 2024-03-22T06:18:17.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.472+0000 7f7049a67700 1 --2- 172.21.15.6:0/193212674 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f703006e970 0x7f7030070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:17.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.473+0000 7f7049a67700 1 --2- 172.21.15.6:0/193212674 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f703006e970 0x7f7030070e20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7034004990 tx=0x7f7034009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:17.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.477+0000 7f703affd700 1 -- 172.21.15.6:0/193212674 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f704005c3e0 con 0x7f70440fce70 2024-03-22T06:18:17.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.766+0000 7f704c4cc700 1 -- 172.21.15.6:0/193212674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7044113950 con 0x7f70440fce70 2024-03-22T06:18:17.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.767+0000 7f703affd700 1 -- 172.21.15.6:0/193212674 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f704005bf70 con 0x7f70440fce70 2024-03-22T06:18:17.768 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:18:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:18:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:18:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:18:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:18:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:18:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:18:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:18:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T06:18:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:18:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:18:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:18:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:18:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:18:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 10, 2024-03-22T06:18:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 4 2024-03-22T06:18:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:18:17.769 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.769+0000 7f704c4cc700 1 -- 172.21.15.6:0/193212674 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f703006e970 msgr2=0x7f7030070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.769+0000 7f704c4cc700 1 --2- 172.21.15.6:0/193212674 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f703006e970 0x7f7030070e20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7034004990 tx=0x7f7034009ee0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.769+0000 7f704c4cc700 1 -- 172.21.15.6:0/193212674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70440fce70 msgr2=0x7f704410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.769+0000 7f704c4cc700 1 --2- 172.21.15.6:0/193212674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70440fce70 0x7f704410d950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f704000b3b0 tx=0x7f7040009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.769+0000 7f704c4cc700 1 -- 172.21.15.6:0/193212674 shutdown_connections 2024-03-22T06:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.769+0000 7f704c4cc700 1 --2- 172.21.15.6:0/193212674 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f703006e970 0x7f7030070e20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.769+0000 7f704c4cc700 1 --2- 172.21.15.6:0/193212674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70440fce70 0x7f704410d950 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.769+0000 7f704c4cc700 1 --2- 172.21.15.6:0/193212674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70440fdfe0 0x7f704410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.769+0000 7f704c4cc700 1 -- 172.21.15.6:0/193212674 >> 172.21.15.6:0/193212674 conn(0x7f70440f8550 msgr2=0x7f70440f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.769+0000 7f704c4cc700 1 -- 172.21.15.6:0/193212674 shutdown_connections 2024-03-22T06:18:17.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.769+0000 7f704c4cc700 1 -- 172.21.15.6:0/193212674 wait complete. 2024-03-22T06:18:17.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.872+0000 7fec7525c700 1 -- 172.21.15.6:0/3164826002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec700fce70 msgr2=0x7fec700fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.872+0000 7fec7525c700 1 --2- 172.21.15.6:0/3164826002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec700fce70 0x7fec700fd280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fec60004940 tx=0x7fec60009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:17.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.872+0000 7fec7525c700 1 -- 172.21.15.6:0/3164826002 shutdown_connections 2024-03-22T06:18:17.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.872+0000 7fec7525c700 1 --2- 172.21.15.6:0/3164826002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec700fdfe0 0x7fec700fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.872+0000 7fec7525c700 1 --2- 172.21.15.6:0/3164826002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec700fce70 0x7fec700fd280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.872+0000 7fec7525c700 1 -- 172.21.15.6:0/3164826002 >> 172.21.15.6:0/3164826002 conn(0x7fec700f8550 msgr2=0x7fec700fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:17.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.872+0000 7fec7525c700 1 -- 172.21.15.6:0/3164826002 shutdown_connections 2024-03-22T06:18:17.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.872+0000 7fec7525c700 1 -- 172.21.15.6:0/3164826002 wait complete. 2024-03-22T06:18:17.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.873+0000 7fec7525c700 1 Processor -- start 2024-03-22T06:18:17.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.873+0000 7fec7525c700 1 -- start start 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.873+0000 7fec7525c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec700fce70 0x7fec70071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.873+0000 7fec7525c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec700fdfe0 0x7fec70070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.873+0000 7fec7525c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70072260 con 0x7fec700fce70 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.873+0000 7fec7525c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70070850 con 0x7fec700fdfe0 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.873+0000 7fec6ed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec700fce70 0x7fec70071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.873+0000 7fec6ed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec700fce70 0x7fec70071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38482/0 (socket says 172.21.15.6:38482) 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.873+0000 7fec6ed9d700 1 -- 172.21.15.6:0/1857927714 learned_addr learned my addr 172.21.15.6:0/1857927714 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.873+0000 7fec6e59c700 1 --2- 172.21.15.6:0/1857927714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec700fdfe0 0x7fec70070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.874+0000 7fec6ed9d700 1 -- 172.21.15.6:0/1857927714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec700fdfe0 msgr2=0x7fec70070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.874+0000 7fec6ed9d700 1 --2- 172.21.15.6:0/1857927714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec700fdfe0 0x7fec70070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.874+0000 7fec6ed9d700 1 -- 172.21.15.6:0/1857927714 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60005040 con 0x7fec700fce70 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.874+0000 7fec6ed9d700 1 --2- 172.21.15.6:0/1857927714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec700fce70 0x7fec70071c60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec60013870 tx=0x7fec60009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:17.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.874+0000 7fec577fe700 1 -- 172.21.15.6:0/1857927714 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec60021070 con 0x7fec700fce70 2024-03-22T06:18:17.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.874+0000 7fec577fe700 1 -- 172.21.15.6:0/1857927714 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec60009430 con 0x7fec700fce70 2024-03-22T06:18:17.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.874+0000 7fec577fe700 1 -- 172.21.15.6:0/1857927714 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec6000e070 con 0x7fec700fce70 2024-03-22T06:18:17.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.874+0000 7fec7525c700 1 -- 172.21.15.6:0/1857927714 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec70070aa0 con 0x7fec700fce70 2024-03-22T06:18:17.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.874+0000 7fec7525c700 1 -- 172.21.15.6:0/1857927714 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec70070fe0 con 0x7fec700fce70 2024-03-22T06:18:17.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.875+0000 7fec7525c700 1 -- 172.21.15.6:0/1857927714 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7004c710 con 0x7fec700fce70 2024-03-22T06:18:17.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.876+0000 7fec577fe700 1 -- 172.21.15.6:0/1857927714 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7fec6001c750 con 0x7fec700fce70 2024-03-22T06:18:17.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.876+0000 7fec577fe700 1 --2- 172.21.15.6:0/1857927714 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec5806e900 0x7fec58070db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:17.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.876+0000 7fec577fe700 1 -- 172.21.15.6:0/1857927714 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fec6008f7a0 con 0x7fec700fce70 2024-03-22T06:18:17.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.876+0000 7fec6e59c700 1 --2- 172.21.15.6:0/1857927714 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec5806e900 0x7fec58070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:17.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.876+0000 7fec6e59c700 1 --2- 172.21.15.6:0/1857927714 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec5806e900 0x7fec58070db0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fec64009b80 tx=0x7fec6400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:17.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:17.881+0000 7fec577fe700 1 -- 172.21.15.6:0/1857927714 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fec6005c170 con 0x7fec700fce70 2024-03-22T06:18:18.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.135+0000 7fec7525c700 1 -- 172.21.15.6:0/1857927714 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fec70115b60 con 0x7fec700fce70 2024-03-22T06:18:18.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.135+0000 7fec577fe700 1 -- 172.21.15.6:0/1857927714 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fec6005bd00 con 0x7fec700fce70 2024-03-22T06:18:18.136 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:18:18.136 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:18:18.136 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:18:18.136 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:18:18.136 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:18:18.136 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:18:18.136 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:18:18.137 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:18:18.138 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:18:18.139 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 0 members: 2024-03-22T06:18:18.139 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:18:18.139 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:18:18.139 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:18:18.139 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:18:18.139 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:18:18.139 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:18:18.140 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:18:18.140 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:18:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.137+0000 7fec7525c700 1 -- 172.21.15.6:0/1857927714 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec5806e900 msgr2=0x7fec58070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.137+0000 7fec7525c700 1 --2- 172.21.15.6:0/1857927714 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec5806e900 0x7fec58070db0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fec64009b80 tx=0x7fec6400b040 comp rx=0 tx=0).stop 2024-03-22T06:18:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.137+0000 7fec7525c700 1 -- 172.21.15.6:0/1857927714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec700fce70 msgr2=0x7fec70071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.137+0000 7fec7525c700 1 --2- 172.21.15.6:0/1857927714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec700fce70 0x7fec70071c60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec60013870 tx=0x7fec60009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.137+0000 7fec7525c700 1 -- 172.21.15.6:0/1857927714 shutdown_connections 2024-03-22T06:18:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.137+0000 7fec7525c700 1 --2- 172.21.15.6:0/1857927714 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec5806e900 0x7fec58070db0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.137+0000 7fec7525c700 1 --2- 172.21.15.6:0/1857927714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec700fce70 0x7fec70071c60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.137+0000 7fec7525c700 1 --2- 172.21.15.6:0/1857927714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec700fdfe0 0x7fec70070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.137+0000 7fec7525c700 1 -- 172.21.15.6:0/1857927714 >> 172.21.15.6:0/1857927714 conn(0x7fec700f8550 msgr2=0x7fec700f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:18.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.137+0000 7fec7525c700 1 -- 172.21.15.6:0/1857927714 shutdown_connections 2024-03-22T06:18:18.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.137+0000 7fec7525c700 1 -- 172.21.15.6:0/1857927714 wait complete. 2024-03-22T06:18:18.142 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:18:18.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.241+0000 7f0b1bc74700 1 -- 172.21.15.6:0/2707592727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b140fe4f0 msgr2=0x7f0b140fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:18.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.241+0000 7f0b1bc74700 1 --2- 172.21.15.6:0/2707592727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b140fe4f0 0x7f0b140fe900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b10004940 tx=0x7f0b10009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:18.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.241+0000 7f0b1bc74700 1 -- 172.21.15.6:0/2707592727 shutdown_connections 2024-03-22T06:18:18.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.241+0000 7f0b1bc74700 1 --2- 172.21.15.6:0/2707592727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b140fee40 0x7f0b140faf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.241+0000 7f0b1bc74700 1 --2- 172.21.15.6:0/2707592727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b140fe4f0 0x7f0b140fe900 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.241+0000 7f0b1bc74700 1 -- 172.21.15.6:0/2707592727 >> 172.21.15.6:0/2707592727 conn(0x7f0b140f6bd0 msgr2=0x7f0b140f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:18.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.241+0000 7f0b1bc74700 1 -- 172.21.15.6:0/2707592727 shutdown_connections 2024-03-22T06:18:18.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.242+0000 7f0b1bc74700 1 -- 172.21.15.6:0/2707592727 wait complete. 2024-03-22T06:18:18.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.242+0000 7f0b1bc74700 1 Processor -- start 2024-03-22T06:18:18.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.242+0000 7f0b1bc74700 1 -- start start 2024-03-22T06:18:18.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.242+0000 7f0b1bc74700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b140fe4f0 0x7f0b1410d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.242+0000 7f0b1bc74700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b140fee40 0x7f0b1410de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.242+0000 7f0b1bc74700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1410e420 con 0x7f0b140fe4f0 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.242+0000 7f0b1bc74700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1410e590 con 0x7f0b140fee40 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.242+0000 7f0b19a10700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b140fe4f0 0x7f0b1410d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.242+0000 7f0b19a10700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b140fe4f0 0x7f0b1410d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38504/0 (socket says 172.21.15.6:38504) 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.242+0000 7f0b19a10700 1 -- 172.21.15.6:0/3416534126 learned_addr learned my addr 172.21.15.6:0/3416534126 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.243+0000 7f0b1920f700 1 --2- 172.21.15.6:0/3416534126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b140fee40 0x7f0b1410de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.243+0000 7f0b19a10700 1 -- 172.21.15.6:0/3416534126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b140fee40 msgr2=0x7f0b1410de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.243+0000 7f0b19a10700 1 --2- 172.21.15.6:0/3416534126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b140fee40 0x7f0b1410de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.243+0000 7f0b19a10700 1 -- 172.21.15.6:0/3416534126 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b10005040 con 0x7f0b140fe4f0 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.243+0000 7f0b19a10700 1 --2- 172.21.15.6:0/3416534126 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b140fe4f0 0x7f0b1410d8e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000b0d0 tx=0x7f0b10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.243+0000 7f0b067fc700 1 -- 172.21.15.6:0/3416534126 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b1000e070 con 0x7f0b140fe4f0 2024-03-22T06:18:18.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.243+0000 7f0b067fc700 1 -- 172.21.15.6:0/3416534126 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b1000b410 con 0x7f0b140fe4f0 2024-03-22T06:18:18.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.243+0000 7f0b1bc74700 1 -- 172.21.15.6:0/3416534126 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b14112f90 con 0x7f0b140fe4f0 2024-03-22T06:18:18.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.243+0000 7f0b067fc700 1 -- 172.21.15.6:0/3416534126 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b100125e0 con 0x7f0b140fe4f0 2024-03-22T06:18:18.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.243+0000 7f0b1bc74700 1 -- 172.21.15.6:0/3416534126 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b141134c0 con 0x7f0b140fe4f0 2024-03-22T06:18:18.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.244+0000 7f0b1bc74700 1 -- 172.21.15.6:0/3416534126 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1404c710 con 0x7f0b140fe4f0 2024-03-22T06:18:18.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.244+0000 7f0b067fc700 1 -- 172.21.15.6:0/3416534126 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7f0b1000b580 con 0x7f0b140fe4f0 2024-03-22T06:18:18.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.245+0000 7f0b067fc700 1 --2- 172.21.15.6:0/3416534126 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0b0006e970 0x7f0b00070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:18.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.245+0000 7f0b067fc700 1 -- 172.21.15.6:0/3416534126 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0b1008f3b0 con 0x7f0b140fe4f0 2024-03-22T06:18:18.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.245+0000 7f0b1920f700 1 --2- 172.21.15.6:0/3416534126 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0b0006e970 0x7f0b00070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:18.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.245+0000 7f0b1920f700 1 --2- 172.21.15.6:0/3416534126 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0b0006e970 0x7f0b00070e20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009b80 tx=0x7f0b0800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:18.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.250+0000 7f0b067fc700 1 -- 172.21.15.6:0/3416534126 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f0b1005bd80 con 0x7f0b140fe4f0 2024-03-22T06:18:18.443 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:18 smithi117 ceph-mon[82126]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:18.444 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:18 smithi117 ceph-mon[82126]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:18.444 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:18 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/193212674' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:18.444 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:18 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:18.444 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:18 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:18.444 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:18 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:18:18.444 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:18 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:18:18.444 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:18 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:18.444 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:18 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1857927714' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:18:18.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.475+0000 7f0b1bc74700 1 -- 172.21.15.6:0/3416534126 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b14102540 con 0x7f0b0006e970 2024-03-22T06:18:18.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:18 smithi006 ceph-mon[94482]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:18.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:18 smithi006 ceph-mon[94482]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:18.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:18 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/193212674' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:18.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:18 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:18.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:18 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:18.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:18 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T06:18:18.476 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:18 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T06:18:18.477 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:18 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:18.477 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:18 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1857927714' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:18:18.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.477+0000 7f0b067fc700 1 -- 172.21.15.6:0/3416534126 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f0b14102540 con 0x7f0b0006e970 2024-03-22T06:18:18.477 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:18:18.477 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:18:18.477 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:18:18.477 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:18:18.478 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:18:18.478 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:18:18.478 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:18:18.478 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:18:18.478 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "4/21 daemons upgraded", 2024-03-22T06:18:18.478 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading mon daemons", 2024-03-22T06:18:18.478 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:18:18.478 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:18:18.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.479+0000 7f0b1bc74700 1 -- 172.21.15.6:0/3416534126 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0b0006e970 msgr2=0x7f0b00070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:18.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.479+0000 7f0b1bc74700 1 --2- 172.21.15.6:0/3416534126 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0b0006e970 0x7f0b00070e20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009b80 tx=0x7f0b0800b040 comp rx=0 tx=0).stop 2024-03-22T06:18:18.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.479+0000 7f0b1bc74700 1 -- 172.21.15.6:0/3416534126 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b140fe4f0 msgr2=0x7f0b1410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:18.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.479+0000 7f0b1bc74700 1 --2- 172.21.15.6:0/3416534126 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b140fe4f0 0x7f0b1410d8e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000b0d0 tx=0x7f0b10009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:18.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.479+0000 7f0b1bc74700 1 -- 172.21.15.6:0/3416534126 shutdown_connections 2024-03-22T06:18:18.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.479+0000 7f0b1bc74700 1 --2- 172.21.15.6:0/3416534126 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0b0006e970 0x7f0b00070e20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.479+0000 7f0b1bc74700 1 --2- 172.21.15.6:0/3416534126 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b140fe4f0 0x7f0b1410d8e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.479+0000 7f0b1bc74700 1 --2- 172.21.15.6:0/3416534126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b140fee40 0x7f0b1410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.479+0000 7f0b1bc74700 1 -- 172.21.15.6:0/3416534126 >> 172.21.15.6:0/3416534126 conn(0x7f0b140f6bd0 msgr2=0x7f0b140f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:18.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.479+0000 7f0b1bc74700 1 -- 172.21.15.6:0/3416534126 shutdown_connections 2024-03-22T06:18:18.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.479+0000 7f0b1bc74700 1 -- 172.21.15.6:0/3416534126 wait complete. 2024-03-22T06:18:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.580+0000 7fe4a1c65700 1 -- 172.21.15.6:0/310001452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe49c0fce70 msgr2=0x7fe49c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.580+0000 7fe4a1c65700 1 --2- 172.21.15.6:0/310001452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe49c0fce70 0x7fe49c0fd280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe48c004940 tx=0x7fe48c009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.580+0000 7fe4a1c65700 1 -- 172.21.15.6:0/310001452 shutdown_connections 2024-03-22T06:18:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.580+0000 7fe4a1c65700 1 --2- 172.21.15.6:0/310001452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe49c0fdfe0 0x7fe49c0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.580+0000 7fe4a1c65700 1 --2- 172.21.15.6:0/310001452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe49c0fce70 0x7fe49c0fd280 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.580+0000 7fe4a1c65700 1 -- 172.21.15.6:0/310001452 >> 172.21.15.6:0/310001452 conn(0x7fe49c0f8550 msgr2=0x7fe49c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.580+0000 7fe4a1c65700 1 -- 172.21.15.6:0/310001452 shutdown_connections 2024-03-22T06:18:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.580+0000 7fe4a1c65700 1 -- 172.21.15.6:0/310001452 wait complete. 2024-03-22T06:18:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.581+0000 7fe4a1c65700 1 Processor -- start 2024-03-22T06:18:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.581+0000 7fe4a1c65700 1 -- start start 2024-03-22T06:18:18.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.581+0000 7fe4a1c65700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe49c0fce70 0x7fe49c10d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.581+0000 7fe4a1c65700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe49c0fdfe0 0x7fe49c10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.581+0000 7fe4a1c65700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c10e490 con 0x7fe49c0fce70 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.581+0000 7fe4a1c65700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c10e600 con 0x7fe49c0fdfe0 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.581+0000 7fe49b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe49c0fce70 0x7fe49c10d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.581+0000 7fe49b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe49c0fce70 0x7fe49c10d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38528/0 (socket says 172.21.15.6:38528) 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.581+0000 7fe49b7fe700 1 -- 172.21.15.6:0/3142487795 learned_addr learned my addr 172.21.15.6:0/3142487795 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.582+0000 7fe49affd700 1 --2- 172.21.15.6:0/3142487795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe49c0fdfe0 0x7fe49c10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.582+0000 7fe49b7fe700 1 -- 172.21.15.6:0/3142487795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe49c0fdfe0 msgr2=0x7fe49c10de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.582+0000 7fe49b7fe700 1 --2- 172.21.15.6:0/3142487795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe49c0fdfe0 0x7fe49c10de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.582+0000 7fe49b7fe700 1 -- 172.21.15.6:0/3142487795 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48c005040 con 0x7fe49c0fce70 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.582+0000 7fe49b7fe700 1 --2- 172.21.15.6:0/3142487795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe49c0fce70 0x7fe49c10d950 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00b3b0 tx=0x7fe48c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.582+0000 7fe483fff700 1 -- 172.21.15.6:0/3142487795 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe48c00e040 con 0x7fe49c0fce70 2024-03-22T06:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.582+0000 7fe483fff700 1 -- 172.21.15.6:0/3142487795 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe48c00bbf0 con 0x7fe49c0fce70 2024-03-22T06:18:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.582+0000 7fe4a1c65700 1 -- 172.21.15.6:0/3142487795 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c113000 con 0x7fe49c0fce70 2024-03-22T06:18:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.582+0000 7fe483fff700 1 -- 172.21.15.6:0/3142487795 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe48c01c630 con 0x7fe49c0fce70 2024-03-22T06:18:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.582+0000 7fe4a1c65700 1 -- 172.21.15.6:0/3142487795 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c1135c0 con 0x7fe49c0fce70 2024-03-22T06:18:18.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.583+0000 7fe4a1c65700 1 -- 172.21.15.6:0/3142487795 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c04c710 con 0x7fe49c0fce70 2024-03-22T06:18:18.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.584+0000 7fe483fff700 1 -- 172.21.15.6:0/3142487795 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7fe48c00bd60 con 0x7fe49c0fce70 2024-03-22T06:18:18.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.584+0000 7fe483fff700 1 --2- 172.21.15.6:0/3142487795 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe48406e970 0x7fe484070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:18.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.584+0000 7fe483fff700 1 -- 172.21.15.6:0/3142487795 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe48c08f190 con 0x7fe49c0fce70 2024-03-22T06:18:18.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.584+0000 7fe49affd700 1 --2- 172.21.15.6:0/3142487795 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe48406e970 0x7fe484070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:18.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.585+0000 7fe49affd700 1 --2- 172.21.15.6:0/3142487795 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe48406e970 0x7fe484070e20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe490004f40 tx=0x7fe49000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:18.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.589+0000 7fe483fff700 1 -- 172.21.15.6:0/3142487795 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fe48c05bb60 con 0x7fe49c0fce70 2024-03-22T06:18:18.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.884+0000 7fe4a1c65700 1 -- 172.21.15.6:0/3142487795 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe49c1139c0 con 0x7fe49c0fce70 2024-03-22T06:18:18.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.884+0000 7fe483fff700 1 -- 172.21.15.6:0/3142487795 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+157 (secure 0 0 0) 0x7fe48c05b6f0 con 0x7fe49c0fce70 2024-03-22T06:18:18.885 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:18:18.885 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:18:18.885 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:18:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.886+0000 7fe4a1c65700 1 -- 172.21.15.6:0/3142487795 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe48406e970 msgr2=0x7fe484070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.886+0000 7fe4a1c65700 1 --2- 172.21.15.6:0/3142487795 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe48406e970 0x7fe484070e20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe490004f40 tx=0x7fe49000b040 comp rx=0 tx=0).stop 2024-03-22T06:18:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.886+0000 7fe4a1c65700 1 -- 172.21.15.6:0/3142487795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe49c0fce70 msgr2=0x7fe49c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.886+0000 7fe4a1c65700 1 --2- 172.21.15.6:0/3142487795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe49c0fce70 0x7fe49c10d950 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00b3b0 tx=0x7fe48c009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.887+0000 7fe4a1c65700 1 -- 172.21.15.6:0/3142487795 shutdown_connections 2024-03-22T06:18:18.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.887+0000 7fe4a1c65700 1 --2- 172.21.15.6:0/3142487795 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe48406e970 0x7fe484070e20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.887+0000 7fe4a1c65700 1 --2- 172.21.15.6:0/3142487795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe49c0fce70 0x7fe49c10d950 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.887+0000 7fe4a1c65700 1 --2- 172.21.15.6:0/3142487795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe49c0fdfe0 0x7fe49c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:18.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.887+0000 7fe4a1c65700 1 -- 172.21.15.6:0/3142487795 >> 172.21.15.6:0/3142487795 conn(0x7fe49c0f8550 msgr2=0x7fe49c0f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:18.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.887+0000 7fe4a1c65700 1 -- 172.21.15.6:0/3142487795 shutdown_connections 2024-03-22T06:18:18.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:18.887+0000 7fe4a1c65700 1 -- 172.21.15.6:0/3142487795 wait complete. 2024-03-22T06:18:19.173 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:19 smithi117 ceph-mon[82126]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:19.173 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:19 smithi117 ceph-mon[82126]: pgmap v25: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:19.173 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:19 smithi117 ceph-mon[82126]: Reconfiguring mon.smithi117 (monmap changed)... 2024-03-22T06:18:19.465 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:19 smithi117 ceph-mon[82126]: Reconfiguring daemon mon.smithi117 on smithi117 2024-03-22T06:18:19.466 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:19 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3142487795' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:18:19.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:19 smithi006 ceph-mon[94482]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:19.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:19 smithi006 ceph-mon[94482]: pgmap v25: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:19.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:19 smithi006 ceph-mon[94482]: Reconfiguring mon.smithi117 (monmap changed)... 2024-03-22T06:18:19.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:19 smithi006 ceph-mon[94482]: Reconfiguring daemon mon.smithi117 on smithi117 2024-03-22T06:18:19.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:19 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3142487795' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:18:20.433 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:20 smithi006 ceph-mon[94482]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:20.433 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:20 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:20.433 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:20 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:20.433 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:20 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T06:18:20.433 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:20 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:20.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:20 smithi117 ceph-mon[82126]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:20.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:20 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:20.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:20 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:20.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:20 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T06:18:20.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:20 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:21.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:21 smithi117 ceph-mon[82126]: Reconfiguring osd.3 (monmap changed)... 2024-03-22T06:18:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:21 smithi117 ceph-mon[82126]: Reconfiguring daemon osd.3 on smithi117 2024-03-22T06:18:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:21 smithi117 ceph-mon[82126]: pgmap v26: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:21 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:21 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:21 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T06:18:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:21 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:21.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:21 smithi006 ceph-mon[94482]: Reconfiguring osd.3 (monmap changed)... 2024-03-22T06:18:21.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:21 smithi006 ceph-mon[94482]: Reconfiguring daemon osd.3 on smithi117 2024-03-22T06:18:21.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:21 smithi006 ceph-mon[94482]: pgmap v26: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:21.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:21 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:21.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:21 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:21.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:21 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T06:18:21.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:21 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:22.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:22 smithi117 ceph-mon[82126]: Reconfiguring osd.4 (monmap changed)... 2024-03-22T06:18:22.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:22 smithi117 ceph-mon[82126]: Reconfiguring daemon osd.4 on smithi117 2024-03-22T06:18:22.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:22 smithi006 ceph-mon[94482]: Reconfiguring osd.4 (monmap changed)... 2024-03-22T06:18:22.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:22 smithi006 ceph-mon[94482]: Reconfiguring daemon osd.4 on smithi117 2024-03-22T06:18:23.442 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:23 smithi117 ceph-mon[82126]: pgmap v27: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:23.442 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:18:23.442 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:23.443 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:23.443 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T06:18:23.443 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:23.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:23 smithi006 ceph-mon[94482]: pgmap v27: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:23.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:18:23.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:23.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:23.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T06:18:23.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:24.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:24 smithi117 ceph-mon[82126]: Reconfiguring osd.5 (monmap changed)... 2024-03-22T06:18:24.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:24 smithi117 ceph-mon[82126]: Reconfiguring daemon osd.5 on smithi117 2024-03-22T06:18:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:24 smithi006 ceph-mon[94482]: Reconfiguring osd.5 (monmap changed)... 2024-03-22T06:18:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:24 smithi006 ceph-mon[94482]: Reconfiguring daemon osd.5 on smithi117 2024-03-22T06:18:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:25 smithi117 ceph-mon[82126]: pgmap v28: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.fwkech", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:18:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:25 smithi006 ceph-mon[94482]: pgmap v28: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.fwkech", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:18:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:26.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:26 smithi117 ceph-mon[82126]: Reconfiguring mds.cephfs.smithi117.fwkech (monmap changed)... 2024-03-22T06:18:26.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:26 smithi117 ceph-mon[82126]: Reconfiguring daemon mds.cephfs.smithi117.fwkech on smithi117 2024-03-22T06:18:26.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:26 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:26.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:26 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:26.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:26 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.zqxyql", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:18:26.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:26 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:26.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:26 smithi006 ceph-mon[94482]: Reconfiguring mds.cephfs.smithi117.fwkech (monmap changed)... 2024-03-22T06:18:26.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:26 smithi006 ceph-mon[94482]: Reconfiguring daemon mds.cephfs.smithi117.fwkech on smithi117 2024-03-22T06:18:26.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:26 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:26.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:26 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:26.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:26 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.zqxyql", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:18:26.522 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:26 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:27.460 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:27 smithi117 ceph-mon[82126]: pgmap v29: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:27.460 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:27 smithi117 ceph-mon[82126]: Reconfiguring mds.cephfs.smithi117.zqxyql (monmap changed)... 2024-03-22T06:18:27.460 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:27 smithi117 ceph-mon[82126]: Reconfiguring daemon mds.cephfs.smithi117.zqxyql on smithi117 2024-03-22T06:18:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:27 smithi006 ceph-mon[94482]: pgmap v29: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:27 smithi006 ceph-mon[94482]: Reconfiguring mds.cephfs.smithi117.zqxyql (monmap changed)... 2024-03-22T06:18:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:27 smithi006 ceph-mon[94482]: Reconfiguring daemon mds.cephfs.smithi117.zqxyql on smithi117 2024-03-22T06:18:28.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:28.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:28.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:18:28.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:28.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:28.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: Upgrade: Setting container_image for all mon 2024-03-22T06:18:28.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:28.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi006"}]: dispatch 2024-03-22T06:18:28.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi006"}]': finished 2024-03-22T06:18:28.520 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]: dispatch 2024-03-22T06:18:28.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:28.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:28.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:18:28.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:28.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:28.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: Upgrade: Setting container_image for all mon 2024-03-22T06:18:28.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:28.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi006"}]: dispatch 2024-03-22T06:18:28.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi006"}]': finished 2024-03-22T06:18:28.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]: dispatch 2024-03-22T06:18:28.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]': finished 2024-03-22T06:18:28.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:28 smithi006 ceph-mon[94482]: pgmap v30: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:29.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]': finished 2024-03-22T06:18:29.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:28 smithi117 ceph-mon[82126]: pgmap v30: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:30 smithi117 ceph-mon[82126]: Upgrade: Updating crash.smithi006 (1/2) 2024-03-22T06:18:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:30 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:30 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi006", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:18:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:30 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:30 smithi117 ceph-mon[82126]: Deploying daemon crash.smithi006 on smithi006 2024-03-22T06:18:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:30 smithi006 ceph-mon[94482]: Upgrade: Updating crash.smithi006 (1/2) 2024-03-22T06:18:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:30 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:30 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi006", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:18:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:30 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:30 smithi006 ceph-mon[94482]: Deploying daemon crash.smithi006 on smithi006 2024-03-22T06:18:31.350 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:31 smithi006 ceph-mon[94482]: pgmap v31: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:31.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:31 smithi117 ceph-mon[82126]: pgmap v31: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:33.504 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:33 smithi006 ceph-mon[94482]: pgmap v32: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:33.504 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:33 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:33.504 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:33 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:33.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:33 smithi117 ceph-mon[82126]: pgmap v32: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:33.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:33 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:33.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:33 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:35.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:35 smithi117 ceph-mon[82126]: pgmap v33: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:35 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:35 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:18:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:35 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:35.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:35 smithi006 ceph-mon[94482]: pgmap v33: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:35.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:35 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:35.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:35 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T06:18:35.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:35 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:36.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:36 smithi006 ceph-mon[94482]: Upgrade: Updating crash.smithi117 (2/2) 2024-03-22T06:18:36.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:36 smithi006 ceph-mon[94482]: Deploying daemon crash.smithi117 on smithi117 2024-03-22T06:18:36.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:36 smithi006 ceph-mon[94482]: pgmap v34: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:36.709 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:36 smithi117 ceph-mon[82126]: Upgrade: Updating crash.smithi117 (2/2) 2024-03-22T06:18:36.709 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:36 smithi117 ceph-mon[82126]: Deploying daemon crash.smithi117 on smithi117 2024-03-22T06:18:36.709 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:36 smithi117 ceph-mon[82126]: pgmap v34: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:37.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:37 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:18:37.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:37 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:18:38.429 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:38 smithi117 ceph-mon[82126]: pgmap v35: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:38.429 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:38.429 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:38.429 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:18:38.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:38 smithi006 ceph-mon[94482]: pgmap v35: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:38.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:38.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:38.667 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:18:41.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:40 smithi117 ceph-mon[82126]: pgmap v36: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:41.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:40 smithi006 ceph-mon[94482]: pgmap v36: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:42.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:42 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:42.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:42 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:42.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:42 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:42.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:42 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:43.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:43 smithi006 ceph-mon[94482]: pgmap v37: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:43.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:43 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:43.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:43 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:43.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:43 smithi117 ceph-mon[82126]: pgmap v37: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:43.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:43 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:43.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:43 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:44.932 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:44 smithi006 ceph-mon[94482]: pgmap v38: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:44.932 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:44 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:44.932 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:44 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:45.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:44 smithi117 ceph-mon[82126]: pgmap v38: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:45.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:44 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:45.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:44 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:46.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:45 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:46.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:45 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:46.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:45 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:46.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:45 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:46.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:45 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:46.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:45 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:46.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:45 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:46.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:45 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:47.102 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:46 smithi006 ceph-mon[94482]: pgmap v39: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:47.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:46 smithi117 ceph-mon[82126]: pgmap v39: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:48.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.994+0000 7f774ffe6700 1 -- 172.21.15.6:0/321544593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77480fff00 msgr2=0x7f7748102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:48.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.994+0000 7f774ffe6700 1 --2- 172.21.15.6:0/321544593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77480fff00 0x7f7748102320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7738004940 tx=0x7f7738009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:48.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.994+0000 7f774ffe6700 1 -- 172.21.15.6:0/321544593 shutdown_connections 2024-03-22T06:18:48.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.994+0000 7f774ffe6700 1 --2- 172.21.15.6:0/321544593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77480fff00 0x7f7748102320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.994+0000 7f774ffe6700 1 --2- 172.21.15.6:0/321544593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77480662e0 0x7f77480ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.994+0000 7f774ffe6700 1 -- 172.21.15.6:0/321544593 >> 172.21.15.6:0/321544593 conn(0x7f77480f7430 msgr2=0x7f77480f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.994+0000 7f774ffe6700 1 -- 172.21.15.6:0/321544593 shutdown_connections 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.994+0000 7f774ffe6700 1 -- 172.21.15.6:0/321544593 wait complete. 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.994+0000 7f774ffe6700 1 Processor -- start 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.994+0000 7f774ffe6700 1 -- start start 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774ffe6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77480662e0 0x7f774810dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774ffe6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77480fff00 0x7f774810e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774ffe6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774810e690 con 0x7f77480fff00 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774ffe6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774810ee30 con 0x7f77480662e0 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774d581700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77480fff00 0x7f774810e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774d581700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77480fff00 0x7f774810e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55624/0 (socket says 172.21.15.6:55624) 2024-03-22T06:18:48.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774d581700 1 -- 172.21.15.6:0/2609968551 learned_addr learned my addr 172.21.15.6:0/2609968551 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:48.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774dd82700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77480662e0 0x7f774810dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:48.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774d581700 1 -- 172.21.15.6:0/2609968551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77480662e0 msgr2=0x7f774810dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:48.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774d581700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77480662e0 0x7f774810dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:48.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774d581700 1 -- 172.21.15.6:0/2609968551 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7738005040 con 0x7f77480fff00 2024-03-22T06:18:48.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774dd82700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77480662e0 0x7f774810dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:18:48.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774d581700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77480fff00 0x7f774810e000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f773800b3b0 tx=0x7f7738009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:48.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f773e7fc700 1 -- 172.21.15.6:0/2609968551 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f773800e040 con 0x7f77480fff00 2024-03-22T06:18:48.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f773e7fc700 1 -- 172.21.15.6:0/2609968551 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f773800bcc0 con 0x7f77480fff00 2024-03-22T06:18:48.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f774ffe6700 1 -- 172.21.15.6:0/2609968551 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774810f0b0 con 0x7f77480fff00 2024-03-22T06:18:48.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.995+0000 7f773e7fc700 1 -- 172.21.15.6:0/2609968551 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f773801c630 con 0x7f77480fff00 2024-03-22T06:18:48.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.996+0000 7f774ffe6700 1 -- 172.21.15.6:0/2609968551 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774810f670 con 0x7f77480fff00 2024-03-22T06:18:48.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.997+0000 7f774ffe6700 1 -- 172.21.15.6:0/2609968551 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f774805f5f0 con 0x7f77480fff00 2024-03-22T06:18:49.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.997+0000 7f773e7fc700 1 -- 172.21.15.6:0/2609968551 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7f7738013980 con 0x7f77480fff00 2024-03-22T06:18:49.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.997+0000 7f773e7fc700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f773406e960 0x7f7734070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:49.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.998+0000 7f773e7fc700 1 -- 172.21.15.6:0/2609968551 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f773808f220 con 0x7f77480fff00 2024-03-22T06:18:49.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.998+0000 7f774dd82700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f773406e960 0x7f7734070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:49.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:48.998+0000 7f774dd82700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f773406e960 0x7f7734070e10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7744006fd0 tx=0x7f7744008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:49.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.003+0000 7f773e7fc700 1 -- 172.21.15.6:0/2609968551 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f773805bbf0 con 0x7f77480fff00 2024-03-22T06:18:49.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: pgmap v40: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:49.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:49.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:49.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:49.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:18:49.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:49.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:18:49.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:49.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:49.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:49.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:49.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi006"}]: dispatch 2024-03-22T06:18:49.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi006"}]': finished 2024-03-22T06:18:49.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]: dispatch 2024-03-22T06:18:49.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]': finished 2024-03-22T06:18:49.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T06:18:49.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: pgmap v40: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi006"}]: dispatch 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi006"}]': finished 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]: dispatch 2024-03-22T06:18:49.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]': finished 2024-03-22T06:18:49.284 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T06:18:49.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.301+0000 7f774ffe6700 1 -- 172.21.15.6:0/2609968551 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f774810f980 con 0x7f773406e960 2024-03-22T06:18:49.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.302+0000 7f773e7fc700 1 -- 172.21.15.6:0/2609968551 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f774810f980 con 0x7f773406e960 2024-03-22T06:18:49.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.304+0000 7f774ffe6700 1 -- 172.21.15.6:0/2609968551 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f773406e960 msgr2=0x7f7734070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:49.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.304+0000 7f774ffe6700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f773406e960 0x7f7734070e10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7744006fd0 tx=0x7f7744008040 comp rx=0 tx=0).stop 2024-03-22T06:18:49.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.304+0000 7f774ffe6700 1 -- 172.21.15.6:0/2609968551 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77480fff00 msgr2=0x7f774810e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:49.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.304+0000 7f774ffe6700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77480fff00 0x7f774810e000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f773800b3b0 tx=0x7f7738009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:49.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.304+0000 7f774ffe6700 1 -- 172.21.15.6:0/2609968551 shutdown_connections 2024-03-22T06:18:49.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.304+0000 7f774ffe6700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f773406e960 0x7f7734070e10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.304+0000 7f774ffe6700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77480662e0 0x7f774810dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.304+0000 7f774ffe6700 1 --2- 172.21.15.6:0/2609968551 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77480fff00 0x7f774810e000 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.304+0000 7f774ffe6700 1 -- 172.21.15.6:0/2609968551 >> 172.21.15.6:0/2609968551 conn(0x7f77480f7430 msgr2=0x7f77480f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:49.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.304+0000 7f774ffe6700 1 -- 172.21.15.6:0/2609968551 shutdown_connections 2024-03-22T06:18:49.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.304+0000 7f774ffe6700 1 -- 172.21.15.6:0/2609968551 wait complete. 2024-03-22T06:18:49.316 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:18:49.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.412+0000 7fb135aad700 1 -- 172.21.15.6:0/1366810918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1300feda0 msgr2=0x7fb1300faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:49.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.412+0000 7fb135aad700 1 --2- 172.21.15.6:0/1366810918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1300feda0 0x7fb1300faee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb124004940 tx=0x7fb124009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:49.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.412+0000 7fb135aad700 1 -- 172.21.15.6:0/1366810918 shutdown_connections 2024-03-22T06:18:49.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.412+0000 7fb135aad700 1 --2- 172.21.15.6:0/1366810918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1300feda0 0x7fb1300faee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.412+0000 7fb135aad700 1 --2- 172.21.15.6:0/1366810918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1300fe450 0x7fb1300fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.412+0000 7fb135aad700 1 -- 172.21.15.6:0/1366810918 >> 172.21.15.6:0/1366810918 conn(0x7fb1300f6c10 msgr2=0x7fb1300f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:49.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.412+0000 7fb135aad700 1 -- 172.21.15.6:0/1366810918 shutdown_connections 2024-03-22T06:18:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.412+0000 7fb135aad700 1 -- 172.21.15.6:0/1366810918 wait complete. 2024-03-22T06:18:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.412+0000 7fb135aad700 1 Processor -- start 2024-03-22T06:18:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.412+0000 7fb135aad700 1 -- start start 2024-03-22T06:18:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb135aad700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1300fe450 0x7fb13010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb135aad700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1300feda0 0x7fb13010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb135aad700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13010e470 con 0x7fb1300fe450 2024-03-22T06:18:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb135aad700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13010e5e0 con 0x7fb1300feda0 2024-03-22T06:18:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb12f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1300fe450 0x7fb13010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb12f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1300fe450 0x7fb13010d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55644/0 (socket says 172.21.15.6:55644) 2024-03-22T06:18:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb12f7fe700 1 -- 172.21.15.6:0/1066450335 learned_addr learned my addr 172.21.15.6:0/1066450335 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:49.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb12f7fe700 1 -- 172.21.15.6:0/1066450335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1300feda0 msgr2=0x7fb13010de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:49.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb12f7fe700 1 --2- 172.21.15.6:0/1066450335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1300feda0 0x7fb13010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb12f7fe700 1 -- 172.21.15.6:0/1066450335 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124005040 con 0x7fb1300fe450 2024-03-22T06:18:49.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb12f7fe700 1 --2- 172.21.15.6:0/1066450335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1300fe450 0x7fb13010d930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb120002a50 tx=0x7fb1200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:49.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb117fff700 1 -- 172.21.15.6:0/1066450335 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb120007cc0 con 0x7fb1300fe450 2024-03-22T06:18:49.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb135aad700 1 -- 172.21.15.6:0/1066450335 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb130112fe0 con 0x7fb1300fe450 2024-03-22T06:18:49.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb117fff700 1 -- 172.21.15.6:0/1066450335 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb12000f040 con 0x7fb1300fe450 2024-03-22T06:18:49.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb117fff700 1 -- 172.21.15.6:0/1066450335 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb120009640 con 0x7fb1300fe450 2024-03-22T06:18:49.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.413+0000 7fb135aad700 1 -- 172.21.15.6:0/1066450335 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1301135a0 con 0x7fb1300fe450 2024-03-22T06:18:49.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.414+0000 7fb135aad700 1 -- 172.21.15.6:0/1066450335 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13004c710 con 0x7fb1300fe450 2024-03-22T06:18:49.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.415+0000 7fb117fff700 1 -- 172.21.15.6:0/1066450335 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7fb1200097a0 con 0x7fb1300fe450 2024-03-22T06:18:49.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.415+0000 7fb117fff700 1 --2- 172.21.15.6:0/1066450335 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb11806e970 0x7fb118070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:49.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.415+0000 7fb117fff700 1 -- 172.21.15.6:0/1066450335 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb12008e3d0 con 0x7fb1300fe450 2024-03-22T06:18:49.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.415+0000 7fb12effd700 1 --2- 172.21.15.6:0/1066450335 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb11806e970 0x7fb118070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:49.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.416+0000 7fb12effd700 1 --2- 172.21.15.6:0/1066450335 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb11806e970 0x7fb118070e20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb124004f90 tx=0x7fb124009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:49.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.420+0000 7fb117fff700 1 -- 172.21.15.6:0/1066450335 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fb12005ad20 con 0x7fb1300fe450 2024-03-22T06:18:49.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.647+0000 7fb135aad700 1 -- 172.21.15.6:0/1066450335 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1301024f0 con 0x7fb11806e970 2024-03-22T06:18:49.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.648+0000 7fb117fff700 1 -- 172.21.15.6:0/1066450335 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb1301024f0 con 0x7fb11806e970 2024-03-22T06:18:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.650+0000 7fb135aad700 1 -- 172.21.15.6:0/1066450335 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb11806e970 msgr2=0x7fb118070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.650+0000 7fb135aad700 1 --2- 172.21.15.6:0/1066450335 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb11806e970 0x7fb118070e20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb124004f90 tx=0x7fb124009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.650+0000 7fb135aad700 1 -- 172.21.15.6:0/1066450335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1300fe450 msgr2=0x7fb13010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.650+0000 7fb135aad700 1 --2- 172.21.15.6:0/1066450335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1300fe450 0x7fb13010d930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb120002a50 tx=0x7fb1200046b0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.650+0000 7fb135aad700 1 -- 172.21.15.6:0/1066450335 shutdown_connections 2024-03-22T06:18:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.650+0000 7fb135aad700 1 --2- 172.21.15.6:0/1066450335 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb11806e970 0x7fb118070e20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.650+0000 7fb135aad700 1 --2- 172.21.15.6:0/1066450335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1300fe450 0x7fb13010d930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.650+0000 7fb135aad700 1 --2- 172.21.15.6:0/1066450335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1300feda0 0x7fb13010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.650+0000 7fb135aad700 1 -- 172.21.15.6:0/1066450335 >> 172.21.15.6:0/1066450335 conn(0x7fb1300f6c10 msgr2=0x7fb1300f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.650+0000 7fb135aad700 1 -- 172.21.15.6:0/1066450335 shutdown_connections 2024-03-22T06:18:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.650+0000 7fb135aad700 1 -- 172.21.15.6:0/1066450335 wait complete. 2024-03-22T06:18:49.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.753+0000 7fa2a8c46700 1 -- 172.21.15.6:0/1498302818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2a40662e0 msgr2=0x7fa2a40ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.753+0000 7fa2a8c46700 1 --2- 172.21.15.6:0/1498302818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2a40662e0 0x7fa2a40ff9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa294004940 tx=0x7fa294009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.753+0000 7fa2a8c46700 1 -- 172.21.15.6:0/1498302818 shutdown_connections 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.753+0000 7fa2a8c46700 1 --2- 172.21.15.6:0/1498302818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2a40fff00 0x7fa2a4102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.753+0000 7fa2a8c46700 1 --2- 172.21.15.6:0/1498302818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2a40662e0 0x7fa2a40ff9c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.753+0000 7fa2a8c46700 1 -- 172.21.15.6:0/1498302818 >> 172.21.15.6:0/1498302818 conn(0x7fa2a40f7430 msgr2=0x7fa2a40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.753+0000 7fa2a8c46700 1 -- 172.21.15.6:0/1498302818 shutdown_connections 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.753+0000 7fa2a8c46700 1 -- 172.21.15.6:0/1498302818 wait complete. 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.754+0000 7fa2a8c46700 1 Processor -- start 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.754+0000 7fa2a8c46700 1 -- start start 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.754+0000 7fa2a8c46700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2a40662e0 0x7fa2a410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.754+0000 7fa2a8c46700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2a40fff00 0x7fa2a410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.754+0000 7fa2a8c46700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a410e590 con 0x7fa2a40fff00 2024-03-22T06:18:49.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.754+0000 7fa2a8c46700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a410ede0 con 0x7fa2a40662e0 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.754+0000 7fa2a1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2a40fff00 0x7fa2a410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.754+0000 7fa2a1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2a40fff00 0x7fa2a410df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55668/0 (socket says 172.21.15.6:55668) 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.754+0000 7fa2a1d9b700 1 -- 172.21.15.6:0/2493700124 learned_addr learned my addr 172.21.15.6:0/2493700124 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.754+0000 7fa2a259c700 1 --2- 172.21.15.6:0/2493700124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2a40662e0 0x7fa2a410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.755+0000 7fa2a1d9b700 1 -- 172.21.15.6:0/2493700124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2a40662e0 msgr2=0x7fa2a410da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.755+0000 7fa2a1d9b700 1 --2- 172.21.15.6:0/2493700124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2a40662e0 0x7fa2a410da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.755+0000 7fa2a1d9b700 1 -- 172.21.15.6:0/2493700124 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa294005040 con 0x7fa2a40fff00 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.755+0000 7fa2a1d9b700 1 --2- 172.21.15.6:0/2493700124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2a40fff00 0x7fa2a410df90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa298002a50 tx=0x7fa2980046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.755+0000 7fa292ffd700 1 -- 172.21.15.6:0/2493700124 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa298007cc0 con 0x7fa2a40fff00 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.755+0000 7fa2a8c46700 1 -- 172.21.15.6:0/2493700124 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a410f0c0 con 0x7fa2a40fff00 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.755+0000 7fa292ffd700 1 -- 172.21.15.6:0/2493700124 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa29800f040 con 0x7fa2a40fff00 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.755+0000 7fa292ffd700 1 -- 172.21.15.6:0/2493700124 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa29800da70 con 0x7fa2a40fff00 2024-03-22T06:18:49.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.755+0000 7fa2a8c46700 1 -- 172.21.15.6:0/2493700124 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a410f600 con 0x7fa2a40fff00 2024-03-22T06:18:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.756+0000 7fa2a8c46700 1 -- 172.21.15.6:0/2493700124 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a405f5f0 con 0x7fa2a40fff00 2024-03-22T06:18:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.757+0000 7fa292ffd700 1 -- 172.21.15.6:0/2493700124 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7fa29800dc10 con 0x7fa2a40fff00 2024-03-22T06:18:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.757+0000 7fa292ffd700 1 --2- 172.21.15.6:0/2493700124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa28c06e960 0x7fa28c070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.757+0000 7fa2a259c700 1 --2- 172.21.15.6:0/2493700124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa28c06e960 0x7fa28c070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:49.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.757+0000 7fa292ffd700 1 -- 172.21.15.6:0/2493700124 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa29808dc20 con 0x7fa2a40fff00 2024-03-22T06:18:49.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.758+0000 7fa2a259c700 1 --2- 172.21.15.6:0/2493700124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa28c06e960 0x7fa28c070e10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa294004890 tx=0x7fa294009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:49.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.762+0000 7fa292ffd700 1 -- 172.21.15.6:0/2493700124 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fa29805a570 con 0x7fa2a40fff00 2024-03-22T06:18:49.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.984+0000 7fa2a8c46700 1 -- 172.21.15.6:0/2493700124 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2a410f960 con 0x7fa28c06e960 2024-03-22T06:18:49.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.994+0000 7fa292ffd700 1 -- 172.21.15.6:0/2493700124 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa2a410f960 con 0x7fa28c06e960 2024-03-22T06:18:49.995 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:18:49.995 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (3m) 6s ago 11m 26.2M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:18:49.995 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (17s) 6s ago 12m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:18:49.995 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (12s) 8s ago 11m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:18:49.995 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 unknown 6s ago 11m - - 2024-03-22T06:18:49.995 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (6m) 6s ago 6m 158M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:18:49.995 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (6m) 6s ago 6m 17.6M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:18:49.995 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (6m) 8s ago 6m 14.6M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (6m) 8s ago 6m 16.4M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (2m) 6s ago 13m 499M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (99s) 8s ago 10m 450M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (81s) 6s ago 13m 41.8M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (58s) 8s ago 10m 33.5M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (3m) 6s ago 12m 17.2M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (2m) 8s ago 11m 14.9M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (10m) 6s ago 10m 249M 1559M 17.2.7-726-g9c4e0c34 837124da804f 997b2b80a744 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (9m) 6s ago 9m 259M 1559M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (9m) 6s ago 9m 214M 1559M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (8m) 8s ago 8m 284M 2581M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (8m) 8s ago 8m 250M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (7m) 8s ago 7m 237M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:18:49.996 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (106s) 6s ago 11m 61.9M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:18:49.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.996+0000 7fa2a8c46700 1 -- 172.21.15.6:0/2493700124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa28c06e960 msgr2=0x7fa28c070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:49.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.996+0000 7fa2a8c46700 1 --2- 172.21.15.6:0/2493700124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa28c06e960 0x7fa28c070e10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa294004890 tx=0x7fa294009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:49.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.996+0000 7fa2a8c46700 1 -- 172.21.15.6:0/2493700124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2a40fff00 msgr2=0x7fa2a410df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:49.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.996+0000 7fa2a8c46700 1 --2- 172.21.15.6:0/2493700124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2a40fff00 0x7fa2a410df90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa298002a50 tx=0x7fa2980046b0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.996+0000 7fa2a8c46700 1 -- 172.21.15.6:0/2493700124 shutdown_connections 2024-03-22T06:18:49.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.996+0000 7fa2a8c46700 1 --2- 172.21.15.6:0/2493700124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa28c06e960 0x7fa28c070e10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.997+0000 7fa2a8c46700 1 --2- 172.21.15.6:0/2493700124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2a40662e0 0x7fa2a410da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.997+0000 7fa2a8c46700 1 --2- 172.21.15.6:0/2493700124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2a40fff00 0x7fa2a410df90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:49.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.997+0000 7fa2a8c46700 1 -- 172.21.15.6:0/2493700124 >> 172.21.15.6:0/2493700124 conn(0x7fa2a40f7430 msgr2=0x7fa2a40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:49.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.997+0000 7fa2a8c46700 1 -- 172.21.15.6:0/2493700124 shutdown_connections 2024-03-22T06:18:49.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:49.997+0000 7fa2a8c46700 1 -- 172.21.15.6:0/2493700124 wait complete. 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.097+0000 7fefb601a700 1 -- 172.21.15.6:0/2514899806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefb00fed50 msgr2=0x7fefb00fae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.097+0000 7fefb601a700 1 --2- 172.21.15.6:0/2514899806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefb00fed50 0x7fefb00fae90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fefa4004940 tx=0x7fefa4009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.097+0000 7fefb601a700 1 -- 172.21.15.6:0/2514899806 shutdown_connections 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.097+0000 7fefb601a700 1 --2- 172.21.15.6:0/2514899806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefb00fed50 0x7fefb00fae90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.097+0000 7fefb601a700 1 --2- 172.21.15.6:0/2514899806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb00fe400 0x7fefb00fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.097+0000 7fefb601a700 1 -- 172.21.15.6:0/2514899806 >> 172.21.15.6:0/2514899806 conn(0x7fefb00f6ba0 msgr2=0x7fefb00f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.097+0000 7fefb601a700 1 -- 172.21.15.6:0/2514899806 shutdown_connections 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.097+0000 7fefb601a700 1 -- 172.21.15.6:0/2514899806 wait complete. 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefb601a700 1 Processor -- start 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefb601a700 1 -- start start 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefb601a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefb00fe400 0x7fefb010d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefb601a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb00fed50 0x7fefb010df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:50.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefb601a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb010e500 con 0x7fefb00fe400 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefb601a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb010ed50 con 0x7fefb00fed50 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefaf7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefb00fe400 0x7fefb010d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefaf7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefb00fe400 0x7fefb010d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55684/0 (socket says 172.21.15.6:55684) 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefaf7fe700 1 -- 172.21.15.6:0/1909538198 learned_addr learned my addr 172.21.15.6:0/1909538198 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefaeffd700 1 --2- 172.21.15.6:0/1909538198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb00fed50 0x7fefb010df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefaf7fe700 1 -- 172.21.15.6:0/1909538198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb00fed50 msgr2=0x7fefb010df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefaf7fe700 1 --2- 172.21.15.6:0/1909538198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb00fed50 0x7fefb010df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.098+0000 7fefaf7fe700 1 -- 172.21.15.6:0/1909538198 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa4005040 con 0x7fefb00fe400 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.099+0000 7fefaf7fe700 1 --2- 172.21.15.6:0/1909538198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefb00fe400 0x7fefb010d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fefa000c830 tx=0x7fefa0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.099+0000 7fef97fff700 1 -- 172.21.15.6:0/1909538198 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefa00106f0 con 0x7fefb00fe400 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.099+0000 7fef97fff700 1 -- 172.21.15.6:0/1909538198 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefa0010d20 con 0x7fefb00fe400 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.099+0000 7fefb601a700 1 -- 172.21.15.6:0/1909538198 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb010f030 con 0x7fefb00fe400 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.099+0000 7fef97fff700 1 -- 172.21.15.6:0/1909538198 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefa00113a0 con 0x7fefb00fe400 2024-03-22T06:18:50.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.099+0000 7fefb601a700 1 -- 172.21.15.6:0/1909538198 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb010f570 con 0x7fefb00fe400 2024-03-22T06:18:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.100+0000 7fefb601a700 1 -- 172.21.15.6:0/1909538198 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb005f5f0 con 0x7fefb00fe400 2024-03-22T06:18:50.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.100+0000 7fef97fff700 1 -- 172.21.15.6:0/1909538198 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7fefa0010850 con 0x7fefb00fe400 2024-03-22T06:18:50.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.101+0000 7fef97fff700 1 --2- 172.21.15.6:0/1909538198 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fef9806e960 0x7fef98070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:50.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.101+0000 7fef97fff700 1 -- 172.21.15.6:0/1909538198 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fefa008e380 con 0x7fefb00fe400 2024-03-22T06:18:50.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.101+0000 7fefaeffd700 1 --2- 172.21.15.6:0/1909538198 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fef9806e960 0x7fef98070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:50.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.101+0000 7fefaeffd700 1 --2- 172.21.15.6:0/1909538198 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fef9806e960 0x7fef98070e10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fefa400b970 tx=0x7fefa4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:50.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.106+0000 7fef97fff700 1 -- 172.21.15.6:0/1909538198 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fefa005af00 con 0x7fefb00fe400 2024-03-22T06:18:50.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:49 smithi117 ceph-mon[82126]: pgmap v41: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:50.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:49 smithi117 ceph-mon[82126]: pgmap v42: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:50.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:49 smithi117 ceph-mon[82126]: Upgrade: Setting container_image for all crash 2024-03-22T06:18:50.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:49 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T06:18:50.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:49 smithi117 ceph-mon[82126]: Upgrade: osd.0 is safe to restart 2024-03-22T06:18:50.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:49 smithi006 ceph-mon[94482]: pgmap v41: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:50.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:49 smithi006 ceph-mon[94482]: pgmap v42: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:50.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:49 smithi006 ceph-mon[94482]: Upgrade: Setting container_image for all crash 2024-03-22T06:18:50.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:49 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T06:18:50.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:49 smithi006 ceph-mon[94482]: Upgrade: osd.0 is safe to restart 2024-03-22T06:18:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.398+0000 7fefb601a700 1 -- 172.21.15.6:0/1909538198 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fefb01058b0 con 0x7fefb00fe400 2024-03-22T06:18:50.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.399+0000 7fef97fff700 1 -- 172.21.15.6:0/1909538198 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fefa005aa90 con 0x7fefb00fe400 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:18:50.400 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:18:50.401 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:18:50.401 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 10, 2024-03-22T06:18:50.401 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 4 2024-03-22T06:18:50.401 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:18:50.401 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:18:50.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.401+0000 7fefb601a700 1 -- 172.21.15.6:0/1909538198 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fef9806e960 msgr2=0x7fef98070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:50.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.401+0000 7fefb601a700 1 --2- 172.21.15.6:0/1909538198 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fef9806e960 0x7fef98070e10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fefa400b970 tx=0x7fefa4009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:50.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.401+0000 7fefb601a700 1 -- 172.21.15.6:0/1909538198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefb00fe400 msgr2=0x7fefb010d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:50.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.401+0000 7fefb601a700 1 --2- 172.21.15.6:0/1909538198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefb00fe400 0x7fefb010d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fefa000c830 tx=0x7fefa0004470 comp rx=0 tx=0).stop 2024-03-22T06:18:50.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.401+0000 7fefb601a700 1 -- 172.21.15.6:0/1909538198 shutdown_connections 2024-03-22T06:18:50.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.401+0000 7fefb601a700 1 --2- 172.21.15.6:0/1909538198 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fef9806e960 0x7fef98070e10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.401+0000 7fefb601a700 1 --2- 172.21.15.6:0/1909538198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefb00fe400 0x7fefb010d9c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.401+0000 7fefb601a700 1 --2- 172.21.15.6:0/1909538198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb00fed50 0x7fefb010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.401+0000 7fefb601a700 1 -- 172.21.15.6:0/1909538198 >> 172.21.15.6:0/1909538198 conn(0x7fefb00f6ba0 msgr2=0x7fefb00f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:50.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.401+0000 7fefb601a700 1 -- 172.21.15.6:0/1909538198 shutdown_connections 2024-03-22T06:18:50.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.401+0000 7fefb601a700 1 -- 172.21.15.6:0/1909538198 wait complete. 2024-03-22T06:18:50.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.503+0000 7fdaff4d9700 1 -- 172.21.15.6:0/2025782425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaf80fed50 msgr2=0x7fdaf80fae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:50.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.503+0000 7fdaff4d9700 1 --2- 172.21.15.6:0/2025782425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaf80fed50 0x7fdaf80fae90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdae8004940 tx=0x7fdae8009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:50.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.503+0000 7fdaff4d9700 1 -- 172.21.15.6:0/2025782425 shutdown_connections 2024-03-22T06:18:50.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.503+0000 7fdaff4d9700 1 --2- 172.21.15.6:0/2025782425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaf80fed50 0x7fdaf80fae90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.503+0000 7fdaff4d9700 1 --2- 172.21.15.6:0/2025782425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf80fe400 0x7fdaf80fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.503+0000 7fdaff4d9700 1 -- 172.21.15.6:0/2025782425 >> 172.21.15.6:0/2025782425 conn(0x7fdaf80f6ba0 msgr2=0x7fdaf80f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:50.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.503+0000 7fdaff4d9700 1 -- 172.21.15.6:0/2025782425 shutdown_connections 2024-03-22T06:18:50.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.503+0000 7fdaff4d9700 1 -- 172.21.15.6:0/2025782425 wait complete. 2024-03-22T06:18:50.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.503+0000 7fdaff4d9700 1 Processor -- start 2024-03-22T06:18:50.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdaff4d9700 1 -- start start 2024-03-22T06:18:50.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdaff4d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaf80fe400 0x7fdaf80fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:50.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdaff4d9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf80fed50 0x7fdaf80fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:50.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdaff4d9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80fc670 con 0x7fdaf80fe400 2024-03-22T06:18:50.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdaff4d9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80fc7e0 con 0x7fdaf80fed50 2024-03-22T06:18:50.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdafd275700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaf80fe400 0x7fdaf80fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:50.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdafd275700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaf80fe400 0x7fdaf80fdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55694/0 (socket says 172.21.15.6:55694) 2024-03-22T06:18:50.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdafd275700 1 -- 172.21.15.6:0/870866474 learned_addr learned my addr 172.21.15.6:0/870866474 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:50.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdafca74700 1 --2- 172.21.15.6:0/870866474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf80fed50 0x7fdaf80fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:50.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdafd275700 1 -- 172.21.15.6:0/870866474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf80fed50 msgr2=0x7fdaf80fc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:50.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdafd275700 1 --2- 172.21.15.6:0/870866474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf80fed50 0x7fdaf80fc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdafd275700 1 -- 172.21.15.6:0/870866474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae8005040 con 0x7fdaf80fe400 2024-03-22T06:18:50.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdafd275700 1 --2- 172.21.15.6:0/870866474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaf80fe400 0x7fdaf80fdbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400c830 tx=0x7fdaf4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:50.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.504+0000 7fdaedffb700 1 -- 172.21.15.6:0/870866474 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaf40106f0 con 0x7fdaf80fe400 2024-03-22T06:18:50.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.505+0000 7fdaedffb700 1 -- 172.21.15.6:0/870866474 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdaf4010d20 con 0x7fdaf80fe400 2024-03-22T06:18:50.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.505+0000 7fdaff4d9700 1 -- 172.21.15.6:0/870866474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf80fcac0 con 0x7fdaf80fe400 2024-03-22T06:18:50.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.505+0000 7fdaedffb700 1 -- 172.21.15.6:0/870866474 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaf40113a0 con 0x7fdaf80fe400 2024-03-22T06:18:50.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.505+0000 7fdaff4d9700 1 -- 172.21.15.6:0/870866474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf81159f0 con 0x7fdaf80fe400 2024-03-22T06:18:50.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.506+0000 7fdaff4d9700 1 -- 172.21.15.6:0/870866474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf805f5f0 con 0x7fdaf80fe400 2024-03-22T06:18:50.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.507+0000 7fdaedffb700 1 -- 172.21.15.6:0/870866474 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7fdaf4012400 con 0x7fdaf80fe400 2024-03-22T06:18:50.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.507+0000 7fdaedffb700 1 --2- 172.21.15.6:0/870866474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fdae406e960 0x7fdae4070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:50.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.507+0000 7fdafca74700 1 --2- 172.21.15.6:0/870866474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fdae406e960 0x7fdae4070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:50.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.507+0000 7fdaedffb700 1 -- 172.21.15.6:0/870866474 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdaf408e440 con 0x7fdaf80fe400 2024-03-22T06:18:50.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.508+0000 7fdafca74700 1 --2- 172.21.15.6:0/870866474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fdae406e960 0x7fdae4070e10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009fd0 tx=0x7fdae8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:50.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.512+0000 7fdaedffb700 1 -- 172.21.15.6:0/870866474 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fdaf40573f0 con 0x7fdaf80fe400 2024-03-22T06:18:50.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.799+0000 7fdaff4d9700 1 -- 172.21.15.6:0/870866474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdaf81058b0 con 0x7fdaf80fe400 2024-03-22T06:18:50.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.800+0000 7fdaedffb700 1 -- 172.21.15.6:0/870866474 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fdaf405aab0 con 0x7fdaf80fe400 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:18:50.801 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:18:50.802 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 0 members: 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:18:50.803 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:18:50.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.803+0000 7fdaff4d9700 1 -- 172.21.15.6:0/870866474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fdae406e960 msgr2=0x7fdae4070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:50.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.803+0000 7fdaff4d9700 1 --2- 172.21.15.6:0/870866474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fdae406e960 0x7fdae4070e10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009fd0 tx=0x7fdae8009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:50.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.803+0000 7fdaff4d9700 1 -- 172.21.15.6:0/870866474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaf80fe400 msgr2=0x7fdaf80fdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:50.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.803+0000 7fdaff4d9700 1 --2- 172.21.15.6:0/870866474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaf80fe400 0x7fdaf80fdbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400c830 tx=0x7fdaf4004470 comp rx=0 tx=0).stop 2024-03-22T06:18:50.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.803+0000 7fdaff4d9700 1 -- 172.21.15.6:0/870866474 shutdown_connections 2024-03-22T06:18:50.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.803+0000 7fdaff4d9700 1 --2- 172.21.15.6:0/870866474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fdae406e960 0x7fdae4070e10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.803+0000 7fdaff4d9700 1 --2- 172.21.15.6:0/870866474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdaf80fe400 0x7fdaf80fdbf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.803+0000 7fdaff4d9700 1 --2- 172.21.15.6:0/870866474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf80fed50 0x7fdaf80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.803+0000 7fdaff4d9700 1 -- 172.21.15.6:0/870866474 >> 172.21.15.6:0/870866474 conn(0x7fdaf80f6ba0 msgr2=0x7fdaf80f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:50.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.803+0000 7fdaff4d9700 1 -- 172.21.15.6:0/870866474 shutdown_connections 2024-03-22T06:18:50.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.803+0000 7fdaff4d9700 1 -- 172.21.15.6:0/870866474 wait complete. 2024-03-22T06:18:50.805 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:18:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.916+0000 7f5f001a5700 1 -- 172.21.15.6:0/1679142900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ef80fed90 msgr2=0x7f5ef80faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.916+0000 7f5f001a5700 1 --2- 172.21.15.6:0/1679142900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ef80fed90 0x7f5ef80faed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4004940 tx=0x7f5ef4009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.916+0000 7f5f001a5700 1 -- 172.21.15.6:0/1679142900 shutdown_connections 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.916+0000 7f5f001a5700 1 --2- 172.21.15.6:0/1679142900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ef80fed90 0x7f5ef80faed0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.916+0000 7f5f001a5700 1 --2- 172.21.15.6:0/1679142900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ef80fe440 0x7f5ef80fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.916+0000 7f5f001a5700 1 -- 172.21.15.6:0/1679142900 >> 172.21.15.6:0/1679142900 conn(0x7f5ef80f6bc0 msgr2=0x7f5ef80f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.916+0000 7f5f001a5700 1 -- 172.21.15.6:0/1679142900 shutdown_connections 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.916+0000 7f5f001a5700 1 -- 172.21.15.6:0/1679142900 wait complete. 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.916+0000 7f5f001a5700 1 Processor -- start 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5f001a5700 1 -- start start 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5f001a5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ef80fe440 0x7f5ef810d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5f001a5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ef80fed90 0x7f5ef810de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5f001a5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef810e4d0 con 0x7f5ef80fe440 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5f001a5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8112e70 con 0x7f5ef80fed90 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5efdf41700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ef80fe440 0x7f5ef810d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5efdf41700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ef80fe440 0x7f5ef810d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55720/0 (socket says 172.21.15.6:55720) 2024-03-22T06:18:50.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5efdf41700 1 -- 172.21.15.6:0/3075992295 learned_addr learned my addr 172.21.15.6:0/3075992295 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5efdf41700 1 -- 172.21.15.6:0/3075992295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ef80fed90 msgr2=0x7f5ef810de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5efdf41700 1 --2- 172.21.15.6:0/3075992295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ef80fed90 0x7f5ef810de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5efdf41700 1 -- 172.21.15.6:0/3075992295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef4005040 con 0x7f5ef80fe440 2024-03-22T06:18:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5efdf41700 1 --2- 172.21.15.6:0/3075992295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ef80fe440 0x7f5ef810d900 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b790 tx=0x7f5ee8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5eee7fc700 1 -- 172.21.15.6:0/3075992295 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee800f770 con 0x7f5ef80fe440 2024-03-22T06:18:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5eee7fc700 1 -- 172.21.15.6:0/3075992295 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ee800fda0 con 0x7f5ef80fe440 2024-03-22T06:18:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.917+0000 7f5f001a5700 1 -- 172.21.15.6:0/3075992295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef81130d0 con 0x7f5ef80fe440 2024-03-22T06:18:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.918+0000 7f5eee7fc700 1 -- 172.21.15.6:0/3075992295 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee800d310 con 0x7f5ef80fe440 2024-03-22T06:18:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.918+0000 7f5f001a5700 1 -- 172.21.15.6:0/3075992295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef8113640 con 0x7f5ef80fe440 2024-03-22T06:18:50.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.919+0000 7f5f001a5700 1 -- 172.21.15.6:0/3075992295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef804c710 con 0x7f5ef80fe440 2024-03-22T06:18:50.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.919+0000 7f5eee7fc700 1 -- 172.21.15.6:0/3075992295 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7f5ee8010420 con 0x7f5ef80fe440 2024-03-22T06:18:50.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.919+0000 7f5eee7fc700 1 --2- 172.21.15.6:0/3075992295 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5ee406e970 0x7f5ee4070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:50.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.920+0000 7f5eee7fc700 1 -- 172.21.15.6:0/3075992295 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5ee808e4f0 con 0x7f5ef80fe440 2024-03-22T06:18:50.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.920+0000 7f5efd740700 1 --2- 172.21.15.6:0/3075992295 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5ee406e970 0x7f5ee4070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:50.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.920+0000 7f5efd740700 1 --2- 172.21.15.6:0/3075992295 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5ee406e970 0x7f5ee4070e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4004f90 tx=0x7f5ef4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:50.924+0000 7f5eee7fc700 1 -- 172.21.15.6:0/3075992295 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f5ee805afd0 con 0x7f5ef80fe440 2024-03-22T06:18:50.977 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:50 smithi006 ceph-mon[94482]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:50.978 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:50 smithi006 ceph-mon[94482]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:50.978 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:50 smithi006 ceph-mon[94482]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:50.978 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:50 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1909538198' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:50.978 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:50.978 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T06:18:50.978 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:50.978 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:50 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/870866474' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:18:51.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.164+0000 7f5f001a5700 1 -- 172.21.15.6:0/3075992295 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ef8102430 con 0x7f5ee406e970 2024-03-22T06:18:51.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.165+0000 7f5eee7fc700 1 -- 172.21.15.6:0/3075992295 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5ef8102430 con 0x7f5ee406e970 2024-03-22T06:18:51.166 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:18:51.166 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:18:51.166 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:18:51.166 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:18:51.166 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:18:51.166 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:18:51.167 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:18:51.167 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:18:51.167 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:18:51.167 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "6/21 daemons upgraded", 2024-03-22T06:18:51.167 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T06:18:51.167 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:18:51.167 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:18:51.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.168+0000 7f5f001a5700 1 -- 172.21.15.6:0/3075992295 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5ee406e970 msgr2=0x7f5ee4070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:51.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.168+0000 7f5f001a5700 1 --2- 172.21.15.6:0/3075992295 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5ee406e970 0x7f5ee4070e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4004f90 tx=0x7f5ef4009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:51.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.168+0000 7f5f001a5700 1 -- 172.21.15.6:0/3075992295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ef80fe440 msgr2=0x7f5ef810d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:51.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.168+0000 7f5f001a5700 1 --2- 172.21.15.6:0/3075992295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ef80fe440 0x7f5ef810d900 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b790 tx=0x7f5ee8009250 comp rx=0 tx=0).stop 2024-03-22T06:18:51.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.168+0000 7f5f001a5700 1 -- 172.21.15.6:0/3075992295 shutdown_connections 2024-03-22T06:18:51.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.168+0000 7f5f001a5700 1 --2- 172.21.15.6:0/3075992295 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5ee406e970 0x7f5ee4070e20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:51.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.168+0000 7f5f001a5700 1 --2- 172.21.15.6:0/3075992295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ef80fe440 0x7f5ef810d900 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:51.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.168+0000 7f5f001a5700 1 --2- 172.21.15.6:0/3075992295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ef80fed90 0x7f5ef810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:51.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.168+0000 7f5f001a5700 1 -- 172.21.15.6:0/3075992295 >> 172.21.15.6:0/3075992295 conn(0x7f5ef80f6bc0 msgr2=0x7f5ef80f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:51.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.168+0000 7f5f001a5700 1 -- 172.21.15.6:0/3075992295 shutdown_connections 2024-03-22T06:18:51.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.168+0000 7f5f001a5700 1 -- 172.21.15.6:0/3075992295 wait complete. 2024-03-22T06:18:51.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:50 smithi117 ceph-mon[82126]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:51.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:50 smithi117 ceph-mon[82126]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:51.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:50 smithi117 ceph-mon[82126]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:51.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:50 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1909538198' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:18:51.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:18:51.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T06:18:51.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:18:51.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:50 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/870866474' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:18:51.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.281+0000 7f0f3d34b700 1 -- 172.21.15.6:0/4213699861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f380fff10 msgr2=0x7f0f38102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:51.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.281+0000 7f0f3d34b700 1 --2- 172.21.15.6:0/4213699861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f380fff10 0x7f0f38102330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c004940 tx=0x7f0f2c009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:51.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.281+0000 7f0f3d34b700 1 -- 172.21.15.6:0/4213699861 shutdown_connections 2024-03-22T06:18:51.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.281+0000 7f0f3d34b700 1 --2- 172.21.15.6:0/4213699861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f380fff10 0x7f0f38102330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:51.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.281+0000 7f0f3d34b700 1 --2- 172.21.15.6:0/4213699861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f38066b70 0x7f0f380ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:51.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.281+0000 7f0f3d34b700 1 -- 172.21.15.6:0/4213699861 >> 172.21.15.6:0/4213699861 conn(0x7f0f380f7440 msgr2=0x7f0f380f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:51.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.281+0000 7f0f3d34b700 1 -- 172.21.15.6:0/4213699861 shutdown_connections 2024-03-22T06:18:51.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.281+0000 7f0f3d34b700 1 -- 172.21.15.6:0/4213699861 wait complete. 2024-03-22T06:18:51.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.282+0000 7f0f3d34b700 1 Processor -- start 2024-03-22T06:18:51.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.282+0000 7f0f3d34b700 1 -- start start 2024-03-22T06:18:51.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.282+0000 7f0f3d34b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f38066b70 0x7f0f3810da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:51.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.282+0000 7f0f3d34b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f380fff10 0x7f0f3810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:51.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.282+0000 7f0f3d34b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3810e5b0 con 0x7f0f38066b70 2024-03-22T06:18:51.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.282+0000 7f0f3d34b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3810ee00 con 0x7f0f380fff10 2024-03-22T06:18:51.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.282+0000 7f0f36ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f38066b70 0x7f0f3810da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.282+0000 7f0f367fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f380fff10 0x7f0f3810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.282+0000 7f0f36ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f38066b70 0x7f0f3810da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55736/0 (socket says 172.21.15.6:55736) 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.282+0000 7f0f367fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f380fff10 0x7f0f3810dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:38576/0 (socket says 172.21.15.6:38576) 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.282+0000 7f0f36ffd700 1 -- 172.21.15.6:0/30894928 learned_addr learned my addr 172.21.15.6:0/30894928 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.283+0000 7f0f36ffd700 1 -- 172.21.15.6:0/30894928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f380fff10 msgr2=0x7f0f3810dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.283+0000 7f0f36ffd700 1 --2- 172.21.15.6:0/30894928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f380fff10 0x7f0f3810dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.283+0000 7f0f36ffd700 1 -- 172.21.15.6:0/30894928 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c005040 con 0x7f0f38066b70 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.283+0000 7f0f36ffd700 1 --2- 172.21.15.6:0/30894928 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f38066b70 0x7f0f3810da70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f28002a50 tx=0x7f0f28004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.283+0000 7f0f1f7fe700 1 -- 172.21.15.6:0/30894928 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f28007cc0 con 0x7f0f38066b70 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.283+0000 7f0f1f7fe700 1 -- 172.21.15.6:0/30894928 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f2800f040 con 0x7f0f38066b70 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.283+0000 7f0f3d34b700 1 -- 172.21.15.6:0/30894928 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3810f0e0 con 0x7f0f38066b70 2024-03-22T06:18:51.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.283+0000 7f0f1f7fe700 1 -- 172.21.15.6:0/30894928 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f2800d9c0 con 0x7f0f38066b70 2024-03-22T06:18:51.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.284+0000 7f0f3d34b700 1 -- 172.21.15.6:0/30894928 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3810f620 con 0x7f0f38066b70 2024-03-22T06:18:51.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.285+0000 7f0f3d34b700 1 -- 172.21.15.6:0/30894928 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3805f5f0 con 0x7f0f38066b70 2024-03-22T06:18:51.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.286+0000 7f0f1f7fe700 1 -- 172.21.15.6:0/30894928 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 38) v1 ==== 92661+0+0 (secure 0 0 0) 0x7f0f28009730 con 0x7f0f38066b70 2024-03-22T06:18:51.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.286+0000 7f0f1f7fe700 1 --2- 172.21.15.6:0/30894928 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0f2006e960 0x7f0f20070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:18:51.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.286+0000 7f0f1f7fe700 1 -- 172.21.15.6:0/30894928 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0f2808e250 con 0x7f0f38066b70 2024-03-22T06:18:51.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.286+0000 7f0f367fc700 1 --2- 172.21.15.6:0/30894928 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0f2006e960 0x7f0f20070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:18:51.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.287+0000 7f0f367fc700 1 --2- 172.21.15.6:0/30894928 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0f2006e960 0x7f0f20070e10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00b970 tx=0x7f0f2c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:18:51.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.292+0000 7f0f1f7fe700 1 -- 172.21.15.6:0/30894928 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f0f2805ac20 con 0x7f0f38066b70 2024-03-22T06:18:51.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.588+0000 7f0f3d34b700 1 -- 172.21.15.6:0/30894928 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0f381058e0 con 0x7f0f38066b70 2024-03-22T06:18:51.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.588+0000 7f0f1f7fe700 1 -- 172.21.15.6:0/30894928 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+157 (secure 0 0 0) 0x7f0f2805a7b0 con 0x7f0f38066b70 2024-03-22T06:18:51.590 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:18:51.590 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:18:51.590 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:18:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.591+0000 7f0f3d34b700 1 -- 172.21.15.6:0/30894928 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0f2006e960 msgr2=0x7f0f20070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.591+0000 7f0f3d34b700 1 --2- 172.21.15.6:0/30894928 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0f2006e960 0x7f0f20070e10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00b970 tx=0x7f0f2c009e30 comp rx=0 tx=0).stop 2024-03-22T06:18:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.591+0000 7f0f3d34b700 1 -- 172.21.15.6:0/30894928 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f38066b70 msgr2=0x7f0f3810da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:18:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.591+0000 7f0f3d34b700 1 --2- 172.21.15.6:0/30894928 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f38066b70 0x7f0f3810da70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f28002a50 tx=0x7f0f28004600 comp rx=0 tx=0).stop 2024-03-22T06:18:51.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.591+0000 7f0f3d34b700 1 -- 172.21.15.6:0/30894928 shutdown_connections 2024-03-22T06:18:51.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.591+0000 7f0f3d34b700 1 --2- 172.21.15.6:0/30894928 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0f2006e960 0x7f0f20070e10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:51.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.591+0000 7f0f3d34b700 1 --2- 172.21.15.6:0/30894928 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f38066b70 0x7f0f3810da70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:51.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.591+0000 7f0f3d34b700 1 --2- 172.21.15.6:0/30894928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f380fff10 0x7f0f3810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:18:51.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.591+0000 7f0f3d34b700 1 -- 172.21.15.6:0/30894928 >> 172.21.15.6:0/30894928 conn(0x7f0f380f7440 msgr2=0x7f0f380f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:18:51.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.591+0000 7f0f3d34b700 1 -- 172.21.15.6:0/30894928 shutdown_connections 2024-03-22T06:18:51.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:18:51.591+0000 7f0f3d34b700 1 -- 172.21.15.6:0/30894928 wait complete. 2024-03-22T06:18:52.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:51 smithi117 ceph-mon[82126]: Upgrade: Updating osd.0 2024-03-22T06:18:52.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:51 smithi117 ceph-mon[82126]: Deploying daemon osd.0 on smithi006 2024-03-22T06:18:52.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:51 smithi117 ceph-mon[82126]: pgmap v43: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:52.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:51 smithi117 ceph-mon[82126]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:52.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:51 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/30894928' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:18:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:51 smithi006 ceph-mon[94482]: Upgrade: Updating osd.0 2024-03-22T06:18:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:51 smithi006 ceph-mon[94482]: Deploying daemon osd.0 on smithi006 2024-03-22T06:18:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:51 smithi006 ceph-mon[94482]: pgmap v43: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:51 smithi006 ceph-mon[94482]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:18:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:51 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/30894928' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:18:52.967 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:18:53.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:18:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:53 smithi117 ceph-mon[82126]: pgmap v44: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:54.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:53 smithi006 ceph-mon[94482]: pgmap v44: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:54.939 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:54 smithi006 systemd[1]: Stopping Ceph osd.0 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:18:55.282 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:54 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[47983]: 2024-03-22T06:18:54.938+0000 7fe0be6c0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:18:55.282 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:54 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[47983]: 2024-03-22T06:18:54.938+0000 7fe0be6c0700 -1 osd.0 49 *** Got signal Terminated *** 2024-03-22T06:18:55.282 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:54 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[47983]: 2024-03-22T06:18:54.938+0000 7fe0be6c0700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:18:56.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:55 smithi117 ceph-mon[82126]: pgmap v45: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:56.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:55 smithi117 ceph-mon[82126]: osd.0 marked itself down and dead 2024-03-22T06:18:56.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:55 smithi006 ceph-mon[94482]: pgmap v45: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:56.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:55 smithi006 ceph-mon[94482]: osd.0 marked itself down and dead 2024-03-22T06:18:56.715 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:56 smithi006 podman[103508]: 2024-03-22 06:18:56.441235779 +0000 UTC m=+1.613721981 container died 997b2b80a7443cc6d76c0307cddb243d434f1304f7444b87765d5dd991af047c (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=quincy-9c4e0c3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:18:56.998 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:56 smithi006 ceph-mon[94482]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:18:56.998 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:56 smithi006 ceph-mon[94482]: osdmap e50: 6 total, 5 up, 6 in 2024-03-22T06:18:56.998 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:56 smithi006 podman[103508]: 2024-03-22 06:18:56.780575421 +0000 UTC m=+1.953061630 container cleanup 997b2b80a7443cc6d76c0307cddb243d434f1304f7444b87765d5dd991af047c (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=quincy-9c4e0c3) 2024-03-22T06:18:56.998 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:56 smithi006 bash[103508]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0 2024-03-22T06:18:57.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:56 smithi117 ceph-mon[82126]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:18:57.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:56 smithi117 ceph-mon[82126]: osdmap e50: 6 total, 5 up, 6 in 2024-03-22T06:18:57.282 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:56 smithi006 podman[103526]: 2024-03-22 06:18:56.998519936 +0000 UTC m=+0.558855423 container remove 997b2b80a7443cc6d76c0307cddb243d434f1304f7444b87765d5dd991af047c (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0, CEPH_POINT_RELEASE=, RELEASE=quincy-9c4e0c3, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-22T06:18:57.685 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:57 smithi006 podman[103619]: 2024-03-22 06:18:57.298747912 +0000 UTC m=+0.017963767 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:18:57.686 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:57 smithi006 podman[103619]: 2024-03-22 06:18:57.441487602 +0000 UTC m=+0.160703492 container create b1d164090c53eb3ff5c38b261d27f54080c68b1f962246a2a0442ee299df64f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:18:57.686 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:57 smithi006 podman[103619]: 2024-03-22 06:18:57.685736343 +0000 UTC m=+0.404952201 container init b1d164090c53eb3ff5c38b261d27f54080c68b1f962246a2a0442ee299df64f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-deactivate, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-22T06:18:57.989 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:57 smithi006 ceph-mon[94482]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:57.989 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:57 smithi006 ceph-mon[94482]: osdmap e51: 6 total, 5 up, 6 in 2024-03-22T06:18:57.989 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:57 smithi006 podman[103619]: 2024-03-22 06:18:57.688952031 +0000 UTC m=+0.408167888 container start b1d164090c53eb3ff5c38b261d27f54080c68b1f962246a2a0442ee299df64f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-22T06:18:57.989 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:57 smithi006 podman[103619]: 2024-03-22 06:18:57.802628049 +0000 UTC m=+0.521843903 container attach b1d164090c53eb3ff5c38b261d27f54080c68b1f962246a2a0442ee299df64f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:18:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:57 smithi117 ceph-mon[82126]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:18:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:57 smithi117 ceph-mon[82126]: osdmap e51: 6 total, 5 up, 6 in 2024-03-22T06:18:58.282 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:57 smithi006 podman[103619]: 2024-03-22 06:18:57.990148236 +0000 UTC m=+0.709364097 container died b1d164090c53eb3ff5c38b261d27f54080c68b1f962246a2a0442ee299df64f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:18:58.936 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:58 smithi006 podman[103645]: 2024-03-22 06:18:58.565007785 +0000 UTC m=+0.566447166 container remove b1d164090c53eb3ff5c38b261d27f54080c68b1f962246a2a0442ee299df64f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:18:58.936 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:58 smithi006 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-22T06:18:58.936 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:58 smithi006 systemd[1]: Stopped Ceph osd.0 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:18:58.937 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:58 smithi006 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.0.service: Consumed 1min 282ms CPU time. 2024-03-22T06:18:58.937 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:58 smithi006 systemd[1]: Starting Ceph osd.0 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:18:59.187 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:58 smithi006 ceph-mon[94482]: Health check failed: Degraded data redundancy: 43/219 objects degraded (19.635%), 17 pgs degraded (PG_DEGRADED) 2024-03-22T06:18:59.187 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 podman[103746]: 2024-03-22 06:18:58.911149327 +0000 UTC m=+0.017912963 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:18:59.187 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 podman[103746]: 2024-03-22 06:18:59.058453887 +0000 UTC m=+0.165217506 container create 4fb8a6d417d1706aebcc717844fc6540129f63c350c1d1122da8c219b24b3ffd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-22T06:18:59.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:58 smithi117 ceph-mon[82126]: Health check failed: Degraded data redundancy: 43/219 objects degraded (19.635%), 17 pgs degraded (PG_DEGRADED) 2024-03-22T06:18:59.735 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 podman[103746]: 2024-03-22 06:18:59.286570628 +0000 UTC m=+0.393334252 container init 4fb8a6d417d1706aebcc717844fc6540129f63c350c1d1122da8c219b24b3ffd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:18:59.735 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 podman[103746]: 2024-03-22 06:18:59.289972371 +0000 UTC m=+0.396736006 container start 4fb8a6d417d1706aebcc717844fc6540129f63c350c1d1122da8c219b24b3ffd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-22T06:18:59.735 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 podman[103746]: 2024-03-22 06:18:59.369576711 +0000 UTC m=+0.476340331 container attach 4fb8a6d417d1706aebcc717844fc6540129f63c350c1d1122da8c219b24b3ffd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_BRANCH=HEAD) 2024-03-22T06:19:00.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:18:59 smithi006 ceph-mon[94482]: pgmap v49: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 43/219 objects degraded (19.635%) 2024-03-22T06:19:00.032 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate[103761]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T06:19:00.032 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate[103761]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 bash[103746]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 bash[103746]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 bash[103746]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 bash[103746]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 bash[103746]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 bash[103746]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 bash[103746]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate[103761]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate[103761]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate[103761]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate[103761]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate[103761]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-22T06:19:00.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:18:59 smithi006 podman[103746]: 2024-03-22 06:18:59.759842272 +0000 UTC m=+0.866605892 container died 4fb8a6d417d1706aebcc717844fc6540129f63c350c1d1122da8c219b24b3ffd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:19:00.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:18:59 smithi117 ceph-mon[82126]: pgmap v49: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 43/219 objects degraded (19.635%) 2024-03-22T06:19:00.650 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:00 smithi006 podman[103883]: 2024-03-22 06:19:00.382182341 +0000 UTC m=+0.617280639 container remove 4fb8a6d417d1706aebcc717844fc6540129f63c350c1d1122da8c219b24b3ffd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:19:00.650 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:00 smithi006 podman[103948]: 2024-03-22 06:19:00.502605192 +0000 UTC m=+0.017871793 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:19:00.650 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:00 smithi006 podman[103948]: 2024-03-22 06:19:00.650525084 +0000 UTC m=+0.165791684 container create a106f4647f4e1fab1b8d3dea8ff76bb4365e5f4909644deb55bc81b8d3e73b6c (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-22T06:19:00.960 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:00 smithi006 podman[103948]: 2024-03-22 06:19:00.869121074 +0000 UTC m=+0.384387684 container init a106f4647f4e1fab1b8d3dea8ff76bb4365e5f4909644deb55bc81b8d3e73b6c (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-22T06:19:00.960 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:00 smithi006 podman[103948]: 2024-03-22 06:19:00.871881783 +0000 UTC m=+0.387148389 container start a106f4647f4e1fab1b8d3dea8ff76bb4365e5f4909644deb55bc81b8d3e73b6c (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:19:00.960 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:00 smithi006 ceph-osd[103967]: -- 172.21.15.6:0/19807970 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fc437bf680 con 0x55fc43789000 2024-03-22T06:19:01.282 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:00 smithi006 bash[103948]: a106f4647f4e1fab1b8d3dea8ff76bb4365e5f4909644deb55bc81b8d3e73b6c 2024-03-22T06:19:01.282 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:00 smithi006 systemd[1]: Started Ceph osd.0 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:19:01.782 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:01 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[103963]: 2024-03-22T06:19:01.409+0000 7f902cc187c0 -1 Falling back to public interface 2024-03-22T06:19:02.145 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:02 smithi006 ceph-mon[94482]: pgmap v50: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 43/219 objects degraded (19.635%) 2024-03-22T06:19:02.146 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:02.146 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:02.146 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:19:02.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:02 smithi117 ceph-mon[82126]: pgmap v50: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 43/219 objects degraded (19.635%) 2024-03-22T06:19:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:19:04.217 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:04 smithi006 ceph-mon[94482]: pgmap v51: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 43/219 objects degraded (19.635%) 2024-03-22T06:19:04.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:04 smithi117 ceph-mon[82126]: pgmap v51: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 43/219 objects degraded (19.635%) 2024-03-22T06:19:06.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:06 smithi006 ceph-mon[94482]: pgmap v52: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 43/219 objects degraded (19.635%) 2024-03-22T06:19:06.033 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:05 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[103963]: 2024-03-22T06:19:05.960+0000 7f902cc187c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-22T06:19:06.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:06 smithi117 ceph-mon[82126]: pgmap v52: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 43/219 objects degraded (19.635%) 2024-03-22T06:19:06.782 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:06 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[103963]: 2024-03-22T06:19:06.475+0000 7f902cc187c0 -1 osd.0 49 log_to_monitors true 2024-03-22T06:19:07.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:07 smithi006 ceph-mon[94482]: from='osd.0 [v2:172.21.15.6:6802/1666131505,v1:172.21.15.6:6803/1666131505]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T06:19:07.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:07 smithi117 ceph-mon[82126]: from='osd.0 [v2:172.21.15.6:6802/1666131505,v1:172.21.15.6:6803/1666131505]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T06:19:08.282 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:19:07 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[103963]: 2024-03-22T06:19:07.958+0000 7f901a068700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:19:08.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:08 smithi006 ceph-mon[94482]: pgmap v53: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 43/219 objects degraded (19.635%) 2024-03-22T06:19:08.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:08 smithi006 ceph-mon[94482]: from='osd.0 [v2:172.21.15.6:6802/1666131505,v1:172.21.15.6:6803/1666131505]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T06:19:08.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:08 smithi006 ceph-mon[94482]: osdmap e52: 6 total, 5 up, 6 in 2024-03-22T06:19:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:08 smithi006 ceph-mon[94482]: from='osd.0 [v2:172.21.15.6:6802/1666131505,v1:172.21.15.6:6803/1666131505]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:19:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:19:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:08 smithi006 ceph-mon[94482]: from='osd.0 [v2:172.21.15.6:6802/1666131505,v1:172.21.15.6:6803/1666131505]' entity='osd.0' 2024-03-22T06:19:08.431 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:08 smithi117 ceph-mon[82126]: pgmap v53: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 43/219 objects degraded (19.635%) 2024-03-22T06:19:08.431 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:08 smithi117 ceph-mon[82126]: from='osd.0 [v2:172.21.15.6:6802/1666131505,v1:172.21.15.6:6803/1666131505]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T06:19:08.431 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:08 smithi117 ceph-mon[82126]: osdmap e52: 6 total, 5 up, 6 in 2024-03-22T06:19:08.431 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:08 smithi117 ceph-mon[82126]: from='osd.0 [v2:172.21.15.6:6802/1666131505,v1:172.21.15.6:6803/1666131505]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:19:08.431 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:08.431 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:08.431 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:08.432 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:19:08.432 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:08 smithi117 ceph-mon[82126]: from='osd.0 [v2:172.21.15.6:6802/1666131505,v1:172.21.15.6:6803/1666131505]' entity='osd.0' 2024-03-22T06:19:09.493 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:09 smithi006 ceph-mon[94482]: mgrmap e39: smithi006.sfpxdy(active, since 91s), standbys: smithi117.bqeskk 2024-03-22T06:19:09.493 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:09 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:09.493 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:09 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:09.493 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:09 smithi006 ceph-mon[94482]: pgmap v55: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 43/219 objects degraded (19.635%) 2024-03-22T06:19:09.493 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:09 smithi006 ceph-mon[94482]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:19:09.494 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:09 smithi006 ceph-mon[94482]: osd.0 [v2:172.21.15.6:6802/1666131505,v1:172.21.15.6:6803/1666131505] boot 2024-03-22T06:19:09.494 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:09 smithi006 ceph-mon[94482]: osdmap e53: 6 total, 6 up, 6 in 2024-03-22T06:19:09.494 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:09 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:19:09.631 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:09 smithi117 ceph-mon[82126]: mgrmap e39: smithi006.sfpxdy(active, since 91s), standbys: smithi117.bqeskk 2024-03-22T06:19:09.631 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:09 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:09.631 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:09 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:09.632 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:09 smithi117 ceph-mon[82126]: pgmap v55: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 43/219 objects degraded (19.635%) 2024-03-22T06:19:09.632 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:09 smithi117 ceph-mon[82126]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:19:09.632 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:09 smithi117 ceph-mon[82126]: osd.0 [v2:172.21.15.6:6802/1666131505,v1:172.21.15.6:6803/1666131505] boot 2024-03-22T06:19:09.632 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:09 smithi117 ceph-mon[82126]: osdmap e53: 6 total, 6 up, 6 in 2024-03-22T06:19:09.632 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:09 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T06:19:11.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:10 smithi117 ceph-mon[82126]: osdmap e54: 6 total, 6 up, 6 in 2024-03-22T06:19:11.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:10 smithi006 ceph-mon[94482]: osdmap e54: 6 total, 6 up, 6 in 2024-03-22T06:19:12.220 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:11 smithi006 ceph-mon[94482]: pgmap v58: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 43/219 objects degraded (19.635%) 2024-03-22T06:19:12.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:11 smithi117 ceph-mon[82126]: pgmap v58: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 43/219 objects degraded (19.635%) 2024-03-22T06:19:13.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:13.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:13.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:19:13.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:19:13.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: pgmap v59: 65 pgs: 15 active+undersized+degraded, 10 active+undersized, 40 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 565 B/s wr, 0 op/s; 38/219 objects degraded (17.352%) 2024-03-22T06:19:13.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: pgmap v60: 65 pgs: 15 active+undersized+degraded, 10 active+undersized, 40 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 608 B/s wr, 0 op/s; 38/219 objects degraded (17.352%) 2024-03-22T06:19:13.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:13.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:19:13.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:13.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:13.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:13.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T06:19:13.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T06:19:13.506 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:13 smithi006 ceph-mon[94482]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-03-22T06:19:13.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: pgmap v59: 65 pgs: 15 active+undersized+degraded, 10 active+undersized, 40 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 565 B/s wr, 0 op/s; 38/219 objects degraded (17.352%) 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: pgmap v60: 65 pgs: 15 active+undersized+degraded, 10 active+undersized, 40 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 608 B/s wr, 0 op/s; 38/219 objects degraded (17.352%) 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T06:19:13.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:13 smithi117 ceph-mon[82126]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-03-22T06:19:14.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:14 smithi117 ceph-mon[82126]: Health check update: Degraded data redundancy: 38/219 objects degraded (17.352%), 15 pgs degraded (PG_DEGRADED) 2024-03-22T06:19:14.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:14 smithi006 ceph-mon[94482]: Health check update: Degraded data redundancy: 38/219 objects degraded (17.352%), 15 pgs degraded (PG_DEGRADED) 2024-03-22T06:19:15.371 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:15 smithi006 ceph-mon[94482]: pgmap v61: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 51 B/s, 0 objects/s recovering 2024-03-22T06:19:15.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:15 smithi117 ceph-mon[82126]: pgmap v61: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 51 B/s, 0 objects/s recovering 2024-03-22T06:19:16.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:16 smithi006 ceph-mon[94482]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 38/219 objects degraded (17.352%), 15 pgs degraded) 2024-03-22T06:19:16.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:16 smithi117 ceph-mon[82126]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 38/219 objects degraded (17.352%), 15 pgs degraded) 2024-03-22T06:19:17.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:17 smithi006 ceph-mon[94482]: pgmap v62: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 39 B/s, 0 objects/s recovering 2024-03-22T06:19:17.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:17 smithi117 ceph-mon[82126]: pgmap v62: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 39 B/s, 0 objects/s recovering 2024-03-22T06:19:19.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:19 smithi117 ceph-mon[82126]: pgmap v63: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 34 B/s, 0 objects/s recovering 2024-03-22T06:19:19.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:19 smithi006 ceph-mon[94482]: pgmap v63: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 34 B/s, 0 objects/s recovering 2024-03-22T06:19:21.335 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:21 smithi006 ceph-mon[94482]: pgmap v64: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 29 B/s, 0 objects/s recovering 2024-03-22T06:19:21.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:21 smithi117 ceph-mon[82126]: pgmap v64: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 29 B/s, 0 objects/s recovering 2024-03-22T06:19:21.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.698+0000 7fece2fc2700 1 -- 172.21.15.6:0/2764601581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecdc0fcfb0 msgr2=0x7fecdc0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:21.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.698+0000 7fece2fc2700 1 --2- 172.21.15.6:0/2764601581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecdc0fcfb0 0x7fecdc0fd420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fecc8004940 tx=0x7fecc8009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:21.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 -- 172.21.15.6:0/2764601581 shutdown_connections 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 --2- 172.21.15.6:0/2764601581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecdc0fcfb0 0x7fecdc0fd420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 --2- 172.21.15.6:0/2764601581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc0fbd00 0x7fecdc0fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 -- 172.21.15.6:0/2764601581 >> 172.21.15.6:0/2764601581 conn(0x7fecdc0f7480 msgr2=0x7fecdc0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 -- 172.21.15.6:0/2764601581 shutdown_connections 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 -- 172.21.15.6:0/2764601581 wait complete. 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 Processor -- start 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 -- start start 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecdc0fbd00 0x7fecdc10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc0fcfb0 0x7fecdc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc10e4b0 con 0x7fecdc0fbd00 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.699+0000 7fece2fc2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc10e620 con 0x7fecdc0fcfb0 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fece0d5e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecdc0fbd00 0x7fecdc10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fece0d5e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecdc0fbd00 0x7fecdc10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35858/0 (socket says 172.21.15.6:35858) 2024-03-22T06:19:21.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fece0d5e700 1 -- 172.21.15.6:0/87498604 learned_addr learned my addr 172.21.15.6:0/87498604 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:21.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fecdbfff700 1 --2- 172.21.15.6:0/87498604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc0fcfb0 0x7fecdc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:21.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fece0d5e700 1 -- 172.21.15.6:0/87498604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc0fcfb0 msgr2=0x7fecdc10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:21.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fece0d5e700 1 --2- 172.21.15.6:0/87498604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc0fcfb0 0x7fecdc10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:21.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fece0d5e700 1 -- 172.21.15.6:0/87498604 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc8005040 con 0x7fecdc0fbd00 2024-03-22T06:19:21.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fece0d5e700 1 --2- 172.21.15.6:0/87498604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecdc0fbd00 0x7fecdc10d970 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fecd000c830 tx=0x7fecd0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:21.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fecd97fa700 1 -- 172.21.15.6:0/87498604 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd000a760 con 0x7fecdc0fbd00 2024-03-22T06:19:21.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fecd97fa700 1 -- 172.21.15.6:0/87498604 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecd000ad90 con 0x7fecdc0fbd00 2024-03-22T06:19:21.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fece2fc2700 1 -- 172.21.15.6:0/87498604 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc112fd0 con 0x7fecdc0fbd00 2024-03-22T06:19:21.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fecd97fa700 1 -- 172.21.15.6:0/87498604 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd00113a0 con 0x7fecdc0fbd00 2024-03-22T06:19:21.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.700+0000 7fece2fc2700 1 -- 172.21.15.6:0/87498604 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc113560 con 0x7fecdc0fbd00 2024-03-22T06:19:21.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.701+0000 7fece2fc2700 1 -- 172.21.15.6:0/87498604 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecdc101b40 con 0x7fecdc0fbd00 2024-03-22T06:19:21.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.703+0000 7fecd97fa700 1 -- 172.21.15.6:0/87498604 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fecd0012400 con 0x7fecdc0fbd00 2024-03-22T06:19:21.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.703+0000 7fecd97fa700 1 --2- 172.21.15.6:0/87498604 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7feccc06e840 0x7feccc070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:21.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.703+0000 7fecd97fa700 1 -- 172.21.15.6:0/87498604 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fecd008ef50 con 0x7fecdc0fbd00 2024-03-22T06:19:21.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.703+0000 7fecdbfff700 1 --2- 172.21.15.6:0/87498604 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7feccc06e840 0x7feccc070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:21.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.704+0000 7fecdbfff700 1 --2- 172.21.15.6:0/87498604 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7feccc06e840 0x7feccc070cf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fecc8000c00 tx=0x7fecc8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:21.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.707+0000 7fecd97fa700 1 -- 172.21.15.6:0/87498604 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fecd005bbe0 con 0x7fecdc0fbd00 2024-03-22T06:19:21.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.934+0000 7fece2fc2700 1 -- 172.21.15.6:0/87498604 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fecdc02b4f0 con 0x7feccc06e840 2024-03-22T06:19:21.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.936+0000 7fecd97fa700 1 -- 172.21.15.6:0/87498604 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fecdc02b4f0 con 0x7feccc06e840 2024-03-22T06:19:21.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.939+0000 7fece2fc2700 1 -- 172.21.15.6:0/87498604 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7feccc06e840 msgr2=0x7feccc070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:21.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.939+0000 7fece2fc2700 1 --2- 172.21.15.6:0/87498604 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7feccc06e840 0x7feccc070cf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fecc8000c00 tx=0x7fecc8009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:21.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.939+0000 7fece2fc2700 1 -- 172.21.15.6:0/87498604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecdc0fbd00 msgr2=0x7fecdc10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:21.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.939+0000 7fece2fc2700 1 --2- 172.21.15.6:0/87498604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecdc0fbd00 0x7fecdc10d970 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fecd000c830 tx=0x7fecd0004470 comp rx=0 tx=0).stop 2024-03-22T06:19:21.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.939+0000 7fece2fc2700 1 -- 172.21.15.6:0/87498604 shutdown_connections 2024-03-22T06:19:21.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.939+0000 7fece2fc2700 1 --2- 172.21.15.6:0/87498604 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7feccc06e840 0x7feccc070cf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:21.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.939+0000 7fece2fc2700 1 --2- 172.21.15.6:0/87498604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecdc0fbd00 0x7fecdc10d970 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:21.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.939+0000 7fece2fc2700 1 --2- 172.21.15.6:0/87498604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc0fcfb0 0x7fecdc10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:21.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.939+0000 7fece2fc2700 1 -- 172.21.15.6:0/87498604 >> 172.21.15.6:0/87498604 conn(0x7fecdc0f7480 msgr2=0x7fecdc066830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:21.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.940+0000 7fece2fc2700 1 -- 172.21.15.6:0/87498604 shutdown_connections 2024-03-22T06:19:21.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:21.940+0000 7fece2fc2700 1 -- 172.21.15.6:0/87498604 wait complete. 2024-03-22T06:19:21.951 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:19:22.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.046+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/1570765119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f84066b70 msgr2=0x7f9f840ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:22.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.046+0000 7f9f8b5a4700 1 --2- 172.21.15.6:0/1570765119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f84066b70 0x7f9f840ff9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f80004940 tx=0x7f9f80009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:22.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.046+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/1570765119 shutdown_connections 2024-03-22T06:19:22.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.046+0000 7f9f8b5a4700 1 --2- 172.21.15.6:0/1570765119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f840fff20 0x7f9f84102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.046+0000 7f9f8b5a4700 1 --2- 172.21.15.6:0/1570765119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f84066b70 0x7f9f840ff9e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.046+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/1570765119 >> 172.21.15.6:0/1570765119 conn(0x7f9f840f7450 msgr2=0x7f9f840f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:22.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.046+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/1570765119 shutdown_connections 2024-03-22T06:19:22.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.046+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/1570765119 wait complete. 2024-03-22T06:19:22.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.046+0000 7f9f8b5a4700 1 Processor -- start 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f8b5a4700 1 -- start start 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f8b5a4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f84066b70 0x7f9f840fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f8b5a4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f840fff20 0x7f9f840fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f8b5a4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840fbe90 con 0x7f9f840fff20 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f8b5a4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840fc000 con 0x7f9f84066b70 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f88b3f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f840fff20 0x7f9f840fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f88b3f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f840fff20 0x7f9f840fb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35870/0 (socket says 172.21.15.6:35870) 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f88b3f700 1 -- 172.21.15.6:0/2172921710 learned_addr learned my addr 172.21.15.6:0/2172921710 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f89340700 1 --2- 172.21.15.6:0/2172921710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f84066b70 0x7f9f840fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f88b3f700 1 -- 172.21.15.6:0/2172921710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f84066b70 msgr2=0x7f9f840fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f88b3f700 1 --2- 172.21.15.6:0/2172921710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f84066b70 0x7f9f840fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f88b3f700 1 -- 172.21.15.6:0/2172921710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f80005040 con 0x7f9f840fff20 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.047+0000 7f9f88b3f700 1 --2- 172.21.15.6:0/2172921710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f840fff20 0x7f9f840fb8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f74002a50 tx=0x7f9f740046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.048+0000 7f9f79ffb700 1 -- 172.21.15.6:0/2172921710 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f74007cc0 con 0x7f9f840fff20 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.048+0000 7f9f79ffb700 1 -- 172.21.15.6:0/2172921710 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f7400f040 con 0x7f9f840fff20 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.048+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/2172921710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f840fc2e0 con 0x7f9f840fff20 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.048+0000 7f9f79ffb700 1 -- 172.21.15.6:0/2172921710 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f7400da70 con 0x7f9f840fff20 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.048+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/2172921710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f84115ab0 con 0x7f9f840fff20 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.049+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/2172921710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8405f5f0 con 0x7f9f840fff20 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.049+0000 7f9f79ffb700 1 -- 172.21.15.6:0/2172921710 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f9f7400dc10 con 0x7f9f840fff20 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.050+0000 7f9f79ffb700 1 --2- 172.21.15.6:0/2172921710 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9f7006e840 0x7f9f70070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.050+0000 7f9f79ffb700 1 -- 172.21.15.6:0/2172921710 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9f7408d910 con 0x7f9f840fff20 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.050+0000 7f9f89340700 1 --2- 172.21.15.6:0/2172921710 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9f7006e840 0x7f9f70070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:22.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.050+0000 7f9f89340700 1 --2- 172.21.15.6:0/2172921710 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9f7006e840 0x7f9f70070cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9f840fd1a0 tx=0x7f9f80009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:22.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.055+0000 7f9f79ffb700 1 -- 172.21.15.6:0/2172921710 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f9f740569d0 con 0x7f9f840fff20 2024-03-22T06:19:22.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.282+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/2172921710 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f840f8f90 con 0x7f9f7006e840 2024-03-22T06:19:22.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.283+0000 7f9f79ffb700 1 -- 172.21.15.6:0/2172921710 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9f840f8f90 con 0x7f9f7006e840 2024-03-22T06:19:22.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.285+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/2172921710 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9f7006e840 msgr2=0x7f9f70070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:22.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.285+0000 7f9f8b5a4700 1 --2- 172.21.15.6:0/2172921710 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9f7006e840 0x7f9f70070cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9f840fd1a0 tx=0x7f9f80009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:22.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.285+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/2172921710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f840fff20 msgr2=0x7f9f840fb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:22.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.285+0000 7f9f8b5a4700 1 --2- 172.21.15.6:0/2172921710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f840fff20 0x7f9f840fb8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f74002a50 tx=0x7f9f740046b0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.285+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/2172921710 shutdown_connections 2024-03-22T06:19:22.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.285+0000 7f9f8b5a4700 1 --2- 172.21.15.6:0/2172921710 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9f7006e840 0x7f9f70070cf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.285+0000 7f9f8b5a4700 1 --2- 172.21.15.6:0/2172921710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f84066b70 0x7f9f840fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.285+0000 7f9f8b5a4700 1 --2- 172.21.15.6:0/2172921710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f840fff20 0x7f9f840fb8e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.285+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/2172921710 >> 172.21.15.6:0/2172921710 conn(0x7f9f840f7450 msgr2=0x7f9f840f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:22.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.286+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/2172921710 shutdown_connections 2024-03-22T06:19:22.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.286+0000 7f9f8b5a4700 1 -- 172.21.15.6:0/2172921710 wait complete. 2024-03-22T06:19:22.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.391+0000 7fd7870f4700 1 -- 172.21.15.6:0/706001649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7800fbcd0 msgr2=0x7fd7800fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:22.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.391+0000 7fd7870f4700 1 --2- 172.21.15.6:0/706001649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7800fbcd0 0x7fd7800fc0e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd774004940 tx=0x7fd774009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:22.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.392+0000 7fd7870f4700 1 -- 172.21.15.6:0/706001649 shutdown_connections 2024-03-22T06:19:22.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.392+0000 7fd7870f4700 1 --2- 172.21.15.6:0/706001649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7800fcf80 0x7fd7800fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.392+0000 7fd7870f4700 1 --2- 172.21.15.6:0/706001649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7800fbcd0 0x7fd7800fc0e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.392+0000 7fd7870f4700 1 -- 172.21.15.6:0/706001649 >> 172.21.15.6:0/706001649 conn(0x7fd7800f7430 msgr2=0x7fd7800f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:22.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.392+0000 7fd7870f4700 1 -- 172.21.15.6:0/706001649 shutdown_connections 2024-03-22T06:19:22.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.392+0000 7fd7870f4700 1 -- 172.21.15.6:0/706001649 wait complete. 2024-03-22T06:19:22.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.392+0000 7fd7870f4700 1 Processor -- start 2024-03-22T06:19:22.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.392+0000 7fd7870f4700 1 -- start start 2024-03-22T06:19:22.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd7870f4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7800fbcd0 0x7fd78010c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:22.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd7870f4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7800fcf80 0x7fd78010ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:22.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd7870f4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78010d4e0 con 0x7fd7800fbcd0 2024-03-22T06:19:22.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd7870f4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78010dcd0 con 0x7fd7800fcf80 2024-03-22T06:19:22.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd784e90700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7800fbcd0 0x7fd78010c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:22.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd784e90700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7800fbcd0 0x7fd78010c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35882/0 (socket says 172.21.15.6:35882) 2024-03-22T06:19:22.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd784e90700 1 -- 172.21.15.6:0/1995523039 learned_addr learned my addr 172.21.15.6:0/1995523039 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:22.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd77ffff700 1 --2- 172.21.15.6:0/1995523039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7800fcf80 0x7fd78010ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd784e90700 1 -- 172.21.15.6:0/1995523039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7800fcf80 msgr2=0x7fd78010ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd784e90700 1 --2- 172.21.15.6:0/1995523039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7800fcf80 0x7fd78010ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd784e90700 1 -- 172.21.15.6:0/1995523039 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774005040 con 0x7fd7800fbcd0 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd784e90700 1 --2- 172.21.15.6:0/1995523039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7800fbcd0 0x7fd78010c910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd774009fd0 tx=0x7fd774009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd77d7fa700 1 -- 172.21.15.6:0/1995523039 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd77400e040 con 0x7fd7800fbcd0 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd77d7fa700 1 -- 172.21.15.6:0/1995523039 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd77400bed0 con 0x7fd7800fbcd0 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd7870f4700 1 -- 172.21.15.6:0/1995523039 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78010df50 con 0x7fd7800fbcd0 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd77d7fa700 1 -- 172.21.15.6:0/1995523039 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7740137c0 con 0x7fd7800fbcd0 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.393+0000 7fd7870f4700 1 -- 172.21.15.6:0/1995523039 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78010e510 con 0x7fd7800fbcd0 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.395+0000 7fd7870f4700 1 -- 172.21.15.6:0/1995523039 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78005f5f0 con 0x7fd7800fbcd0 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.395+0000 7fd77d7fa700 1 -- 172.21.15.6:0/1995523039 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fd77401fbc0 con 0x7fd7800fbcd0 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.395+0000 7fd77d7fa700 1 --2- 172.21.15.6:0/1995523039 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd77006e840 0x7fd770070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.396+0000 7fd77d7fa700 1 -- 172.21.15.6:0/1995523039 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd774091c30 con 0x7fd7800fbcd0 2024-03-22T06:19:22.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.396+0000 7fd77ffff700 1 --2- 172.21.15.6:0/1995523039 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd77006e840 0x7fd770070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:22.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.396+0000 7fd77ffff700 1 --2- 172.21.15.6:0/1995523039 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd77006e840 0x7fd770070cf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006fd0 tx=0x7fd76c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:22.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.400+0000 7fd77d7fa700 1 -- 172.21.15.6:0/1995523039 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fd77405e850 con 0x7fd7800fbcd0 2024-03-22T06:19:22.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.616+0000 7fd7870f4700 1 -- 172.21.15.6:0/1995523039 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd78010e8b0 con 0x7fd77006e840 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.626+0000 7fd77d7fa700 1 -- 172.21.15.6:0/1995523039 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fd78010e8b0 con 0x7fd77006e840 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (4m) 15s ago 12m 26.2M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (50s) 15s ago 12m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (44s) 41s ago 11m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 unknown 15s ago 12m - - 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (6m) 15s ago 6m 158M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (6m) 15s ago 6m 17.7M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (6m) 41s ago 6m 14.6M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (6m) 41s ago 6m 16.4M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (3m) 15s ago 13m 500M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (2m) 41s ago 11m 450M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:19:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (114s) 15s ago 13m 43.6M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:19:22.628 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (91s) 41s ago 11m 33.5M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:19:22.628 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (4m) 15s ago 12m 17.0M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:19:22.628 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (3m) 41s ago 11m 14.9M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:19:22.628 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (22s) 15s ago 10m 14.7M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:19:22.628 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (10m) 15s ago 10m 259M 1559M 17.2.7-726-g9c4e0c34 837124da804f 1223447bec6b 2024-03-22T06:19:22.628 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (9m) 15s ago 9m 213M 1559M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:19:22.628 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (9m) 41s ago 9m 284M 2581M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:19:22.628 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (8m) 41s ago 8m 250M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:19:22.628 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (8m) 41s ago 8m 237M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:19:22.628 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (2m) 15s ago 11m 64.2M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:19:22.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.628+0000 7fd7870f4700 1 -- 172.21.15.6:0/1995523039 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd77006e840 msgr2=0x7fd770070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:22.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.628+0000 7fd7870f4700 1 --2- 172.21.15.6:0/1995523039 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd77006e840 0x7fd770070cf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006fd0 tx=0x7fd76c008040 comp rx=0 tx=0).stop 2024-03-22T06:19:22.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.628+0000 7fd7870f4700 1 -- 172.21.15.6:0/1995523039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7800fbcd0 msgr2=0x7fd78010c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:22.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.628+0000 7fd7870f4700 1 --2- 172.21.15.6:0/1995523039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7800fbcd0 0x7fd78010c910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd774009fd0 tx=0x7fd774009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:22.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.629+0000 7fd7870f4700 1 -- 172.21.15.6:0/1995523039 shutdown_connections 2024-03-22T06:19:22.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.629+0000 7fd7870f4700 1 --2- 172.21.15.6:0/1995523039 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd77006e840 0x7fd770070cf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.629+0000 7fd7870f4700 1 --2- 172.21.15.6:0/1995523039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7800fbcd0 0x7fd78010c910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.629+0000 7fd7870f4700 1 --2- 172.21.15.6:0/1995523039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7800fcf80 0x7fd78010ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.629+0000 7fd7870f4700 1 -- 172.21.15.6:0/1995523039 >> 172.21.15.6:0/1995523039 conn(0x7fd7800f7430 msgr2=0x7fd7800f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:22.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.629+0000 7fd7870f4700 1 -- 172.21.15.6:0/1995523039 shutdown_connections 2024-03-22T06:19:22.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.629+0000 7fd7870f4700 1 -- 172.21.15.6:0/1995523039 wait complete. 2024-03-22T06:19:22.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.730+0000 7f88561aa700 1 -- 172.21.15.6:0/3033916310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88500fdfe0 msgr2=0x7f88500fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:22.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.730+0000 7f88561aa700 1 --2- 172.21.15.6:0/3033916310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88500fdfe0 0x7f88500fe450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8838004940 tx=0x7f8838009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:22.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.730+0000 7f88561aa700 1 -- 172.21.15.6:0/3033916310 shutdown_connections 2024-03-22T06:19:22.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.730+0000 7f88561aa700 1 --2- 172.21.15.6:0/3033916310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88500fdfe0 0x7f88500fe450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.730+0000 7f88561aa700 1 --2- 172.21.15.6:0/3033916310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88500fce70 0x7f88500fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.730+0000 7f88561aa700 1 -- 172.21.15.6:0/3033916310 >> 172.21.15.6:0/3033916310 conn(0x7f88500f8550 msgr2=0x7f88500fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:22.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.730+0000 7f88561aa700 1 -- 172.21.15.6:0/3033916310 shutdown_connections 2024-03-22T06:19:22.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.730+0000 7f88561aa700 1 -- 172.21.15.6:0/3033916310 wait complete. 2024-03-22T06:19:22.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f88561aa700 1 Processor -- start 2024-03-22T06:19:22.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f88561aa700 1 -- start start 2024-03-22T06:19:22.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f88561aa700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88500fce70 0x7f885010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:22.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f88561aa700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88500fdfe0 0x7f885010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:22.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f88561aa700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885010e460 con 0x7f88500fce70 2024-03-22T06:19:22.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f88561aa700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885010e5d0 con 0x7f88500fdfe0 2024-03-22T06:19:22.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f884f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88500fce70 0x7f885010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:22.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f884f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88500fce70 0x7f885010d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35902/0 (socket says 172.21.15.6:35902) 2024-03-22T06:19:22.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f884f7fe700 1 -- 172.21.15.6:0/3047506212 learned_addr learned my addr 172.21.15.6:0/3047506212 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:22.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f884effd700 1 --2- 172.21.15.6:0/3047506212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88500fdfe0 0x7f885010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:22.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f884f7fe700 1 -- 172.21.15.6:0/3047506212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88500fdfe0 msgr2=0x7f885010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:22.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f884f7fe700 1 --2- 172.21.15.6:0/3047506212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88500fdfe0 0x7f885010de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:22.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.731+0000 7f884f7fe700 1 -- 172.21.15.6:0/3047506212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8838005040 con 0x7f88500fce70 2024-03-22T06:19:22.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.732+0000 7f884f7fe700 1 --2- 172.21.15.6:0/3047506212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88500fce70 0x7f885010d920 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8840001d80 tx=0x7f8840009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:22.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.732+0000 7f8847fff700 1 -- 172.21.15.6:0/3047506212 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8840007d10 con 0x7f88500fce70 2024-03-22T06:19:22.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.732+0000 7f8847fff700 1 -- 172.21.15.6:0/3047506212 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8840001e80 con 0x7f88500fce70 2024-03-22T06:19:22.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.732+0000 7f8847fff700 1 -- 172.21.15.6:0/3047506212 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8840014640 con 0x7f88500fce70 2024-03-22T06:19:22.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.732+0000 7f88561aa700 1 -- 172.21.15.6:0/3047506212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8850112fd0 con 0x7f88500fce70 2024-03-22T06:19:22.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.732+0000 7f88561aa700 1 -- 172.21.15.6:0/3047506212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8850113560 con 0x7f88500fce70 2024-03-22T06:19:22.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.733+0000 7f88561aa700 1 -- 172.21.15.6:0/3047506212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885004c710 con 0x7f88500fce70 2024-03-22T06:19:22.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.733+0000 7f8847fff700 1 -- 172.21.15.6:0/3047506212 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f884000e960 con 0x7f88500fce70 2024-03-22T06:19:22.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.734+0000 7f8847fff700 1 --2- 172.21.15.6:0/3047506212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f883c06e840 0x7f883c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:22.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.734+0000 7f8847fff700 1 -- 172.21.15.6:0/3047506212 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f884008e0c0 con 0x7f88500fce70 2024-03-22T06:19:22.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.734+0000 7f884effd700 1 --2- 172.21.15.6:0/3047506212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f883c06e840 0x7f883c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:22.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.734+0000 7f884effd700 1 --2- 172.21.15.6:0/3047506212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f883c06e840 0x7f883c070cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f883800b970 tx=0x7f8838009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:22.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:22.738+0000 7f8847fff700 1 -- 172.21.15.6:0/3047506212 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f884005ace0 con 0x7f88500fce70 2024-03-22T06:19:23.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.034+0000 7f88561aa700 1 -- 172.21.15.6:0/3047506212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8850113b00 con 0x7f88500fce70 2024-03-22T06:19:23.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.034+0000 7f8847fff700 1 -- 172.21.15.6:0/3047506212 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f88400097e0 con 0x7f88500fce70 2024-03-22T06:19:23.035 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:19:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:19:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:19:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:19:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:19:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:19:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 5, 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 1 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 9, 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 5 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:19:23.036 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:19:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.036+0000 7f88561aa700 1 -- 172.21.15.6:0/3047506212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f883c06e840 msgr2=0x7f883c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.036+0000 7f88561aa700 1 --2- 172.21.15.6:0/3047506212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f883c06e840 0x7f883c070cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f883800b970 tx=0x7f8838009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.036+0000 7f88561aa700 1 -- 172.21.15.6:0/3047506212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88500fce70 msgr2=0x7f885010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.036+0000 7f88561aa700 1 --2- 172.21.15.6:0/3047506212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88500fce70 0x7f885010d920 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8840001d80 tx=0x7f8840009ec0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.036+0000 7f88561aa700 1 -- 172.21.15.6:0/3047506212 shutdown_connections 2024-03-22T06:19:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.036+0000 7f88561aa700 1 --2- 172.21.15.6:0/3047506212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f883c06e840 0x7f883c070cf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.036+0000 7f88561aa700 1 --2- 172.21.15.6:0/3047506212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88500fce70 0x7f885010d920 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.036+0000 7f88561aa700 1 --2- 172.21.15.6:0/3047506212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88500fdfe0 0x7f885010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.036+0000 7f88561aa700 1 -- 172.21.15.6:0/3047506212 >> 172.21.15.6:0/3047506212 conn(0x7f88500f8550 msgr2=0x7f88500f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.036+0000 7f88561aa700 1 -- 172.21.15.6:0/3047506212 shutdown_connections 2024-03-22T06:19:23.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.036+0000 7f88561aa700 1 -- 172.21.15.6:0/3047506212 wait complete. 2024-03-22T06:19:23.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.139+0000 7f66cd19b700 1 -- 172.21.15.6:0/2448981419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66c80fce70 msgr2=0x7f66c80fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.139+0000 7f66cd19b700 1 --2- 172.21.15.6:0/2448981419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66c80fce70 0x7f66c80fd280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66bc004940 tx=0x7f66bc009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:23.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.139+0000 7f66cd19b700 1 -- 172.21.15.6:0/2448981419 shutdown_connections 2024-03-22T06:19:23.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.139+0000 7f66cd19b700 1 --2- 172.21.15.6:0/2448981419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c80fdfe0 0x7f66c80fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.139+0000 7f66cd19b700 1 --2- 172.21.15.6:0/2448981419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66c80fce70 0x7f66c80fd280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.139+0000 7f66cd19b700 1 -- 172.21.15.6:0/2448981419 >> 172.21.15.6:0/2448981419 conn(0x7f66c80f8550 msgr2=0x7f66c80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:23.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.139+0000 7f66cd19b700 1 -- 172.21.15.6:0/2448981419 shutdown_connections 2024-03-22T06:19:23.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.139+0000 7f66cd19b700 1 -- 172.21.15.6:0/2448981419 wait complete. 2024-03-22T06:19:23.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.139+0000 7f66cd19b700 1 Processor -- start 2024-03-22T06:19:23.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66cd19b700 1 -- start start 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66cd19b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66c80fce70 0x7f66c810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66cd19b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c80fdfe0 0x7f66c810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66cd19b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c810e490 con 0x7f66c80fce70 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66cd19b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c810e600 con 0x7f66c80fdfe0 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66c6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66c80fce70 0x7f66c810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66c6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66c80fce70 0x7f66c810d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35920/0 (socket says 172.21.15.6:35920) 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66c6d9d700 1 -- 172.21.15.6:0/3954561348 learned_addr learned my addr 172.21.15.6:0/3954561348 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66c659c700 1 --2- 172.21.15.6:0/3954561348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c80fdfe0 0x7f66c810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66c6d9d700 1 -- 172.21.15.6:0/3954561348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c80fdfe0 msgr2=0x7f66c810de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66c6d9d700 1 --2- 172.21.15.6:0/3954561348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c80fdfe0 0x7f66c810de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66c6d9d700 1 -- 172.21.15.6:0/3954561348 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc005040 con 0x7f66c80fce70 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66c6d9d700 1 --2- 172.21.15.6:0/3954561348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66c80fce70 0x7f66c810d950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009150 tx=0x7f66bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:23.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66af7fe700 1 -- 172.21.15.6:0/3954561348 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66bc00e040 con 0x7f66c80fce70 2024-03-22T06:19:23.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.140+0000 7f66af7fe700 1 -- 172.21.15.6:0/3954561348 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66bc00bbf0 con 0x7f66c80fce70 2024-03-22T06:19:23.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.141+0000 7f66cd19b700 1 -- 172.21.15.6:0/3954561348 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c8113000 con 0x7f66c80fce70 2024-03-22T06:19:23.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.141+0000 7f66af7fe700 1 -- 172.21.15.6:0/3954561348 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66bc01c630 con 0x7f66c80fce70 2024-03-22T06:19:23.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.141+0000 7f66cd19b700 1 -- 172.21.15.6:0/3954561348 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c81135c0 con 0x7f66c80fce70 2024-03-22T06:19:23.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.142+0000 7f66cd19b700 1 -- 172.21.15.6:0/3954561348 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c804c710 con 0x7f66c80fce70 2024-03-22T06:19:23.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.142+0000 7f66af7fe700 1 -- 172.21.15.6:0/3954561348 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f66bc00bd60 con 0x7f66c80fce70 2024-03-22T06:19:23.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.142+0000 7f66af7fe700 1 --2- 172.21.15.6:0/3954561348 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f66b006e840 0x7f66b0070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:23.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.143+0000 7f66af7fe700 1 -- 172.21.15.6:0/3954561348 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f66bc08f5d0 con 0x7f66c80fce70 2024-03-22T06:19:23.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.143+0000 7f66c659c700 1 --2- 172.21.15.6:0/3954561348 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f66b006e840 0x7f66b0070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:23.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.143+0000 7f66c659c700 1 --2- 172.21.15.6:0/3954561348 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f66b006e840 0x7f66b0070cf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f66b8004f40 tx=0x7f66b800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:23.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.147+0000 7f66af7fe700 1 -- 172.21.15.6:0/3954561348 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f66bc05c260 con 0x7f66c80fce70 2024-03-22T06:19:23.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.397+0000 7f66cd19b700 1 -- 172.21.15.6:0/3954561348 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f66c8113a10 con 0x7f66c80fce70 2024-03-22T06:19:23.398 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:23 smithi006 ceph-mon[94482]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:23.398 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:23 smithi006 ceph-mon[94482]: pgmap v65: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28 B/s, 0 objects/s recovering 2024-03-22T06:19:23.398 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:23.398 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:19:23.398 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:23.398 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:23 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3047506212' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:23.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.397+0000 7f66af7fe700 1 -- 172.21.15.6:0/3954561348 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1875 (secure 0 0 0) 0x7f66bc013e60 con 0x7f66c80fce70 2024-03-22T06:19:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:19:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:19:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:19:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:19:23.399 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:19:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:19:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:19:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:19:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 0 members: 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:19:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:19:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:19:23.402 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:19:23.402 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:19:23.402 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:19:23.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.402+0000 7f66cd19b700 1 -- 172.21.15.6:0/3954561348 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f66b006e840 msgr2=0x7f66b0070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.402+0000 7f66cd19b700 1 --2- 172.21.15.6:0/3954561348 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f66b006e840 0x7f66b0070cf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f66b8004f40 tx=0x7f66b800b040 comp rx=0 tx=0).stop 2024-03-22T06:19:23.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.402+0000 7f66cd19b700 1 -- 172.21.15.6:0/3954561348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66c80fce70 msgr2=0x7f66c810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.402+0000 7f66cd19b700 1 --2- 172.21.15.6:0/3954561348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66c80fce70 0x7f66c810d950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009150 tx=0x7f66bc009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:23.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.402+0000 7f66cd19b700 1 -- 172.21.15.6:0/3954561348 shutdown_connections 2024-03-22T06:19:23.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.402+0000 7f66cd19b700 1 --2- 172.21.15.6:0/3954561348 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f66b006e840 0x7f66b0070cf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.402+0000 7f66cd19b700 1 --2- 172.21.15.6:0/3954561348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66c80fce70 0x7f66c810d950 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.402+0000 7f66cd19b700 1 --2- 172.21.15.6:0/3954561348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c80fdfe0 0x7f66c810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.402+0000 7f66cd19b700 1 -- 172.21.15.6:0/3954561348 >> 172.21.15.6:0/3954561348 conn(0x7f66c80f8550 msgr2=0x7f66c80f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:23.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.402+0000 7f66cd19b700 1 -- 172.21.15.6:0/3954561348 shutdown_connections 2024-03-22T06:19:23.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.402+0000 7f66cd19b700 1 -- 172.21.15.6:0/3954561348 wait complete. 2024-03-22T06:19:23.403 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:19:23.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.504+0000 7fc1610c2700 1 -- 172.21.15.6:0/1853884270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc15c0fbcd0 msgr2=0x7fc15c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.504+0000 7fc1610c2700 1 --2- 172.21.15.6:0/1853884270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc15c0fbcd0 0x7fc15c0fc0e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc14c004940 tx=0x7fc14c009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:23.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.504+0000 7fc1610c2700 1 -- 172.21.15.6:0/1853884270 shutdown_connections 2024-03-22T06:19:23.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.504+0000 7fc1610c2700 1 --2- 172.21.15.6:0/1853884270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c0fcf80 0x7fc15c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.504+0000 7fc1610c2700 1 --2- 172.21.15.6:0/1853884270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc15c0fbcd0 0x7fc15c0fc0e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.504+0000 7fc1610c2700 1 -- 172.21.15.6:0/1853884270 >> 172.21.15.6:0/1853884270 conn(0x7fc15c0f7430 msgr2=0x7fc15c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:23.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.504+0000 7fc1610c2700 1 -- 172.21.15.6:0/1853884270 shutdown_connections 2024-03-22T06:19:23.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.504+0000 7fc1610c2700 1 -- 172.21.15.6:0/1853884270 wait complete. 2024-03-22T06:19:23.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.505+0000 7fc1610c2700 1 Processor -- start 2024-03-22T06:19:23.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.505+0000 7fc1610c2700 1 -- start start 2024-03-22T06:19:23.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.505+0000 7fc1610c2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c0fbcd0 0x7fc15c110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:23.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.505+0000 7fc1610c2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc15c0fcf80 0x7fc15c112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:23.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.505+0000 7fc1610c2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c066350 con 0x7fc15c0fcf80 2024-03-22T06:19:23.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.505+0000 7fc1610c2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c066490 con 0x7fc15c0fbcd0 2024-03-22T06:19:23.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.505+0000 7fc15a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc15c0fcf80 0x7fc15c112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:23.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.505+0000 7fc15a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc15c0fcf80 0x7fc15c112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35940/0 (socket says 172.21.15.6:35940) 2024-03-22T06:19:23.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.505+0000 7fc15a59c700 1 -- 172.21.15.6:0/883634363 learned_addr learned my addr 172.21.15.6:0/883634363 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:23.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.505+0000 7fc15ad9d700 1 --2- 172.21.15.6:0/883634363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c0fbcd0 0x7fc15c110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:23.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.506+0000 7fc15a59c700 1 -- 172.21.15.6:0/883634363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c0fbcd0 msgr2=0x7fc15c110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.506+0000 7fc15a59c700 1 --2- 172.21.15.6:0/883634363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c0fbcd0 0x7fc15c110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.506+0000 7fc15a59c700 1 -- 172.21.15.6:0/883634363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c005040 con 0x7fc15c0fcf80 2024-03-22T06:19:23.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.506+0000 7fc15a59c700 1 --2- 172.21.15.6:0/883634363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc15c0fcf80 0x7fc15c112c70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc150002a50 tx=0x7fc1500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:23.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.506+0000 7fc1437fe700 1 -- 172.21.15.6:0/883634363 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc150007cc0 con 0x7fc15c0fcf80 2024-03-22T06:19:23.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.506+0000 7fc1437fe700 1 -- 172.21.15.6:0/883634363 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc15000f040 con 0x7fc15c0fcf80 2024-03-22T06:19:23.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.506+0000 7fc1610c2700 1 -- 172.21.15.6:0/883634363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c113210 con 0x7fc15c0fcf80 2024-03-22T06:19:23.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.506+0000 7fc1437fe700 1 -- 172.21.15.6:0/883634363 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc15000da30 con 0x7fc15c0fcf80 2024-03-22T06:19:23.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.506+0000 7fc1610c2700 1 -- 172.21.15.6:0/883634363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c1137d0 con 0x7fc15c0fcf80 2024-03-22T06:19:23.508 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:23 smithi117 ceph-mon[82126]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:23.508 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:23 smithi117 ceph-mon[82126]: pgmap v65: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28 B/s, 0 objects/s recovering 2024-03-22T06:19:23.508 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:23.508 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:19:23.508 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:23.508 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:23 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3047506212' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:23.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.507+0000 7fc1610c2700 1 -- 172.21.15.6:0/883634363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c05f5f0 con 0x7fc15c0fcf80 2024-03-22T06:19:23.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.508+0000 7fc1437fe700 1 -- 172.21.15.6:0/883634363 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fc15000dc70 con 0x7fc15c0fcf80 2024-03-22T06:19:23.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.508+0000 7fc1437fe700 1 --2- 172.21.15.6:0/883634363 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc14406e840 0x7fc144070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:23.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.508+0000 7fc1437fe700 1 -- 172.21.15.6:0/883634363 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc15008dff0 con 0x7fc15c0fcf80 2024-03-22T06:19:23.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.508+0000 7fc15ad9d700 1 --2- 172.21.15.6:0/883634363 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc14406e840 0x7fc144070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:23.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.509+0000 7fc15ad9d700 1 --2- 172.21.15.6:0/883634363 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc14406e840 0x7fc144070cf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc14c004890 tx=0x7fc14c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:23.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.513+0000 7fc1437fe700 1 -- 172.21.15.6:0/883634363 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fc1500570b0 con 0x7fc15c0fcf80 2024-03-22T06:19:23.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.742+0000 7fc1610c2700 1 -- 172.21.15.6:0/883634363 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc15c101290 con 0x7fc14406e840 2024-03-22T06:19:23.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.744+0000 7fc1437fe700 1 -- 172.21.15.6:0/883634363 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc15c101290 con 0x7fc14406e840 2024-03-22T06:19:23.743 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:19:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:19:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:19:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:19:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:19:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:19:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:19:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:19:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:19:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "7/21 daemons upgraded", 2024-03-22T06:19:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T06:19:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:19:23.744 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:19:23.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.746+0000 7fc1610c2700 1 -- 172.21.15.6:0/883634363 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc14406e840 msgr2=0x7fc144070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.746+0000 7fc1610c2700 1 --2- 172.21.15.6:0/883634363 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc14406e840 0x7fc144070cf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc14c004890 tx=0x7fc14c009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:23.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.746+0000 7fc1610c2700 1 -- 172.21.15.6:0/883634363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc15c0fcf80 msgr2=0x7fc15c112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.746+0000 7fc1610c2700 1 --2- 172.21.15.6:0/883634363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc15c0fcf80 0x7fc15c112c70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc150002a50 tx=0x7fc1500046b0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.746+0000 7fc1610c2700 1 -- 172.21.15.6:0/883634363 shutdown_connections 2024-03-22T06:19:23.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.746+0000 7fc1610c2700 1 --2- 172.21.15.6:0/883634363 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc14406e840 0x7fc144070cf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.746+0000 7fc1610c2700 1 --2- 172.21.15.6:0/883634363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c0fbcd0 0x7fc15c110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.746+0000 7fc1610c2700 1 --2- 172.21.15.6:0/883634363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc15c0fcf80 0x7fc15c112c70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.746+0000 7fc1610c2700 1 -- 172.21.15.6:0/883634363 >> 172.21.15.6:0/883634363 conn(0x7fc15c0f7430 msgr2=0x7fc15c0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:23.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.746+0000 7fc1610c2700 1 -- 172.21.15.6:0/883634363 shutdown_connections 2024-03-22T06:19:23.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.746+0000 7fc1610c2700 1 -- 172.21.15.6:0/883634363 wait complete. 2024-03-22T06:19:23.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.848+0000 7f9430c29700 1 -- 172.21.15.6:0/3861252915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c0fdfe0 msgr2=0x7f942c0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.848+0000 7f9430c29700 1 --2- 172.21.15.6:0/3861252915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c0fdfe0 0x7f942c0fe450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9414004940 tx=0x7f9414009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:23.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.848+0000 7f9430c29700 1 -- 172.21.15.6:0/3861252915 shutdown_connections 2024-03-22T06:19:23.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.848+0000 7f9430c29700 1 --2- 172.21.15.6:0/3861252915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c0fdfe0 0x7f942c0fe450 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.848+0000 7f9430c29700 1 --2- 172.21.15.6:0/3861252915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f942c0fce70 0x7f942c0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.848+0000 7f9430c29700 1 -- 172.21.15.6:0/3861252915 >> 172.21.15.6:0/3861252915 conn(0x7f942c0f8550 msgr2=0x7f942c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:23.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.848+0000 7f9430c29700 1 -- 172.21.15.6:0/3861252915 shutdown_connections 2024-03-22T06:19:23.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.848+0000 7f9430c29700 1 -- 172.21.15.6:0/3861252915 wait complete. 2024-03-22T06:19:23.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f9430c29700 1 Processor -- start 2024-03-22T06:19:23.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f9430c29700 1 -- start start 2024-03-22T06:19:23.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f9430c29700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c0fce70 0x7f942c10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f9430c29700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f942c0fdfe0 0x7f942c10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f9430c29700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c10e460 con 0x7f942c0fce70 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f9430c29700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c10e5d0 con 0x7f942c0fdfe0 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f942a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c0fce70 0x7f942c10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f942a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c0fce70 0x7f942c10d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50294/0 (socket says 172.21.15.6:50294) 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f942a59c700 1 -- 172.21.15.6:0/1318116552 learned_addr learned my addr 172.21.15.6:0/1318116552 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f9429d9b700 1 --2- 172.21.15.6:0/1318116552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f942c0fdfe0 0x7f942c10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f942a59c700 1 -- 172.21.15.6:0/1318116552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f942c0fdfe0 msgr2=0x7f942c10de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f942a59c700 1 --2- 172.21.15.6:0/1318116552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f942c0fdfe0 0x7f942c10de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.849+0000 7f942a59c700 1 -- 172.21.15.6:0/1318116552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9414005040 con 0x7f942c0fce70 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.850+0000 7f942a59c700 1 --2- 172.21.15.6:0/1318116552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c0fce70 0x7f942c10d920 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f941c002a50 tx=0x7f941c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.850+0000 7f9422ffd700 1 -- 172.21.15.6:0/1318116552 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f941c007cc0 con 0x7f942c0fce70 2024-03-22T06:19:23.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.850+0000 7f9422ffd700 1 -- 172.21.15.6:0/1318116552 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f941c00f040 con 0x7f942c0fce70 2024-03-22T06:19:23.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.850+0000 7f9430c29700 1 -- 172.21.15.6:0/1318116552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c112fd0 con 0x7f942c0fce70 2024-03-22T06:19:23.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.850+0000 7f9422ffd700 1 -- 172.21.15.6:0/1318116552 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f941c00da30 con 0x7f942c0fce70 2024-03-22T06:19:23.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.850+0000 7f9430c29700 1 -- 172.21.15.6:0/1318116552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c113560 con 0x7f942c0fce70 2024-03-22T06:19:23.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.851+0000 7f9430c29700 1 -- 172.21.15.6:0/1318116552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c04c710 con 0x7f942c0fce70 2024-03-22T06:19:23.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.851+0000 7f9422ffd700 1 -- 172.21.15.6:0/1318116552 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f941c00db90 con 0x7f942c0fce70 2024-03-22T06:19:23.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.852+0000 7f9422ffd700 1 --2- 172.21.15.6:0/1318116552 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f941806e840 0x7f9418070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:23.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.852+0000 7f9422ffd700 1 -- 172.21.15.6:0/1318116552 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f941c08e020 con 0x7f942c0fce70 2024-03-22T06:19:23.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.852+0000 7f9429d9b700 1 --2- 172.21.15.6:0/1318116552 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f941806e840 0x7f9418070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:23.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.852+0000 7f9429d9b700 1 --2- 172.21.15.6:0/1318116552 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f941806e840 0x7f9418070cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f941400b970 tx=0x7f9414009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:23.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:23.856+0000 7f9422ffd700 1 -- 172.21.15.6:0/1318116552 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f941c05ac10 con 0x7f942c0fce70 2024-03-22T06:19:24.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.146+0000 7f9430c29700 1 -- 172.21.15.6:0/1318116552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f942c1137a0 con 0x7f942c0fce70 2024-03-22T06:19:24.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.146+0000 7f9422ffd700 1 -- 172.21.15.6:0/1318116552 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+157 (secure 0 0 0) 0x7f941c05a7a0 con 0x7f942c0fce70 2024-03-22T06:19:24.146 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:19:24.146 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:19:24.146 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:19:24.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.148+0000 7f9430c29700 1 -- 172.21.15.6:0/1318116552 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f941806e840 msgr2=0x7f9418070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:24.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.148+0000 7f9430c29700 1 --2- 172.21.15.6:0/1318116552 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f941806e840 0x7f9418070cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f941400b970 tx=0x7f9414009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:24.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.148+0000 7f9430c29700 1 -- 172.21.15.6:0/1318116552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c0fce70 msgr2=0x7f942c10d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:24.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.148+0000 7f9430c29700 1 --2- 172.21.15.6:0/1318116552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c0fce70 0x7f942c10d920 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f941c002a50 tx=0x7f941c0046b0 comp rx=0 tx=0).stop 2024-03-22T06:19:24.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.148+0000 7f9430c29700 1 -- 172.21.15.6:0/1318116552 shutdown_connections 2024-03-22T06:19:24.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.148+0000 7f9430c29700 1 --2- 172.21.15.6:0/1318116552 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f941806e840 0x7f9418070cf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:24.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.149+0000 7f9430c29700 1 --2- 172.21.15.6:0/1318116552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c0fce70 0x7f942c10d920 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:24.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.149+0000 7f9430c29700 1 --2- 172.21.15.6:0/1318116552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f942c0fdfe0 0x7f942c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:24.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.149+0000 7f9430c29700 1 -- 172.21.15.6:0/1318116552 >> 172.21.15.6:0/1318116552 conn(0x7f942c0f8550 msgr2=0x7f942c0f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:24.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.149+0000 7f9430c29700 1 -- 172.21.15.6:0/1318116552 shutdown_connections 2024-03-22T06:19:24.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:24.149+0000 7f9430c29700 1 -- 172.21.15.6:0/1318116552 wait complete. 2024-03-22T06:19:24.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:24 smithi117 ceph-mon[82126]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:24.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:24 smithi117 ceph-mon[82126]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:24.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:24 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3954561348' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:19:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:24 smithi006 ceph-mon[94482]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:24 smithi006 ceph-mon[94482]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:24 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3954561348' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:19:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:25 smithi117 ceph-mon[82126]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:25 smithi117 ceph-mon[82126]: pgmap v66: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-03-22T06:19:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:25 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1318116552' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:19:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:25 smithi006 ceph-mon[94482]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:25 smithi006 ceph-mon[94482]: pgmap v66: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-03-22T06:19:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:25 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1318116552' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:19:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:27 smithi117 ceph-mon[82126]: pgmap v67: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:27 smithi006 ceph-mon[94482]: pgmap v67: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:28.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T06:19:28.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:28 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T06:19:28.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:28 smithi117 ceph-mon[82126]: Upgrade: osd.1 is safe to restart 2024-03-22T06:19:28.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T06:19:28.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:28 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T06:19:28.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:28 smithi006 ceph-mon[94482]: Upgrade: osd.1 is safe to restart 2024-03-22T06:19:29.241 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:29 smithi006 ceph-mon[94482]: pgmap v68: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:29.242 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:29 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:29.242 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:29 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T06:19:29.242 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:29 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:19:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:29 smithi117 ceph-mon[82126]: pgmap v68: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:29 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:29 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T06:19:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:29 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:19:30.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:30 smithi117 ceph-mon[82126]: Upgrade: Updating osd.1 2024-03-22T06:19:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:30 smithi117 ceph-mon[82126]: Deploying daemon osd.1 on smithi006 2024-03-22T06:19:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:30 smithi006 ceph-mon[94482]: Upgrade: Updating osd.1 2024-03-22T06:19:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:30 smithi006 ceph-mon[94482]: Deploying daemon osd.1 on smithi006 2024-03-22T06:19:31.345 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:31 smithi006 ceph-mon[94482]: pgmap v69: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:31.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:31 smithi117 ceph-mon[82126]: pgmap v69: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:33.106 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:32 smithi006 systemd[1]: Stopping Ceph osd.1 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:19:33.505 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:33 smithi006 ceph-mon[94482]: pgmap v70: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:33.505 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:33 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[55399]: 2024-03-22T06:19:33.240+0000 7f90f5786700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:19:33.505 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:33 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[55399]: 2024-03-22T06:19:33.240+0000 7f90f5786700 -1 osd.1 54 *** Got signal Terminated *** 2024-03-22T06:19:33.505 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:33 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[55399]: 2024-03-22T06:19:33.240+0000 7f90f5786700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:19:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:33 smithi117 ceph-mon[82126]: pgmap v70: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:34.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:34 smithi117 ceph-mon[82126]: osd.1 marked itself down and dead 2024-03-22T06:19:34.531 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:34 smithi006 ceph-mon[94482]: osd.1 marked itself down and dead 2024-03-22T06:19:34.782 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:34 smithi006 podman[109611]: 2024-03-22 06:19:34.531271546 +0000 UTC m=+1.402125318 container died 1223447bec6b09112dd82ae6b092d23836d755292f3c674de43b90bc17da5947 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-22T06:19:35.112 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:34 smithi006 podman[109611]: 2024-03-22 06:19:34.882815925 +0000 UTC m=+1.753669696 container cleanup 1223447bec6b09112dd82ae6b092d23836d755292f3c674de43b90bc17da5947 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1, GIT_CLEAN=True, RELEASE=quincy-9c4e0c3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:19:35.113 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:34 smithi006 bash[109611]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1 2024-03-22T06:19:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:35 smithi117 ceph-mon[82126]: pgmap v71: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:35 smithi117 ceph-mon[82126]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:19:35.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:35 smithi117 ceph-mon[82126]: osdmap e55: 6 total, 5 up, 6 in 2024-03-22T06:19:35.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:35 smithi006 ceph-mon[94482]: pgmap v71: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:35.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:35 smithi006 ceph-mon[94482]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:19:35.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:35 smithi006 ceph-mon[94482]: osdmap e55: 6 total, 5 up, 6 in 2024-03-22T06:19:35.533 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:35 smithi006 podman[109631]: 2024-03-22 06:19:35.201128583 +0000 UTC m=+0.672931108 container remove 1223447bec6b09112dd82ae6b092d23836d755292f3c674de43b90bc17da5947 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T06:19:35.855 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:35 smithi006 podman[109723]: 2024-03-22 06:19:35.454060395 +0000 UTC m=+0.017936781 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:19:35.856 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:35 smithi006 podman[109723]: 2024-03-22 06:19:35.577085416 +0000 UTC m=+0.140961768 container create 13c1252c8e7f85c9954662c45afa0e1ad2ea6628c4ae4e9155f833c053e47cce (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319) 2024-03-22T06:19:35.856 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:35 smithi006 podman[109723]: 2024-03-22 06:19:35.855502073 +0000 UTC m=+0.419378442 container init 13c1252c8e7f85c9954662c45afa0e1ad2ea6628c4ae4e9155f833c053e47cce (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-22T06:19:36.122 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:35 smithi006 podman[109723]: 2024-03-22 06:19:35.859257605 +0000 UTC m=+0.423133959 container start 13c1252c8e7f85c9954662c45afa0e1ad2ea6628c4ae4e9155f833c053e47cce (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:19:36.122 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:35 smithi006 podman[109723]: 2024-03-22 06:19:35.988664687 +0000 UTC m=+0.552541049 container attach 13c1252c8e7f85c9954662c45afa0e1ad2ea6628c4ae4e9155f833c053e47cce (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-22T06:19:36.431 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:36 smithi006 ceph-mon[94482]: osdmap e56: 6 total, 5 up, 6 in 2024-03-22T06:19:36.431 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:36 smithi006 podman[109723]: 2024-03-22 06:19:36.150101042 +0000 UTC m=+0.713977395 container died 13c1252c8e7f85c9954662c45afa0e1ad2ea6628c4ae4e9155f833c053e47cce (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-22T06:19:36.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:36 smithi117 ceph-mon[82126]: osdmap e56: 6 total, 5 up, 6 in 2024-03-22T06:19:37.032 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:36 smithi006 podman[109748]: 2024-03-22 06:19:36.759527535 +0000 UTC m=+0.603955236 container remove 13c1252c8e7f85c9954662c45afa0e1ad2ea6628c4ae4e9155f833c053e47cce (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:19:37.032 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:36 smithi006 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-22T06:19:37.032 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:36 smithi006 systemd[1]: Stopped Ceph osd.1 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:19:37.033 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:36 smithi006 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.1.service: Consumed 1min 6.986s CPU time. 2024-03-22T06:19:37.033 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 systemd[1]: Starting Ceph osd.1 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:19:37.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:37 smithi117 ceph-mon[82126]: pgmap v74: 65 pgs: 12 stale+active+clean, 53 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:37.432 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:37 smithi006 ceph-mon[94482]: pgmap v74: 65 pgs: 12 stale+active+clean, 53 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-22T06:19:37.432 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 podman[109865]: 2024-03-22 06:19:37.124066499 +0000 UTC m=+0.018651322 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:19:37.432 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 podman[109865]: 2024-03-22 06:19:37.31947033 +0000 UTC m=+0.214055129 container create 996795b89ee3bc5e67f2ce8a0c7f77fe87d33b9ab87aba0ad349515f57d91792 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-22T06:19:37.782 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 podman[109865]: 2024-03-22 06:19:37.547810044 +0000 UTC m=+0.442394854 container init 996795b89ee3bc5e67f2ce8a0c7f77fe87d33b9ab87aba0ad349515f57d91792 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-22T06:19:37.782 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 podman[109865]: 2024-03-22 06:19:37.551138994 +0000 UTC m=+0.445723790 container start 996795b89ee3bc5e67f2ce8a0c7f77fe87d33b9ab87aba0ad349515f57d91792 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-22T06:19:37.782 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 podman[109865]: 2024-03-22 06:19:37.639199529 +0000 UTC m=+0.533784338 container attach 996795b89ee3bc5e67f2ce8a0c7f77fe87d33b9ab87aba0ad349515f57d91792 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-03-22T06:19:38.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [0, 2]}]: dispatch 2024-03-22T06:19:38.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 1]}]: dispatch 2024-03-22T06:19:38.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [3, 1]}]: dispatch 2024-03-22T06:19:38.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]: dispatch 2024-03-22T06:19:38.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [0, 2]}]: dispatch 2024-03-22T06:19:38.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 1]}]: dispatch 2024-03-22T06:19:38.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 2]}]: dispatch 2024-03-22T06:19:38.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]: dispatch 2024-03-22T06:19:38.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 0]}]: dispatch 2024-03-22T06:19:38.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1f", "id": [3, 1]}]: dispatch 2024-03-22T06:19:38.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:38.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:19:38.283 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate[109880]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T06:19:38.283 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate[109880]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 bash[109865]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 bash[109865]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 bash[109865]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 bash[109865]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 bash[109865]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 bash[109865]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 bash[109865]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate[109880]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate[109880]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate[109880]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate[109880]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:37 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate[109880]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-22T06:19:38.284 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:38 smithi006 podman[109865]: 2024-03-22 06:19:38.022118885 +0000 UTC m=+0.916703676 container died 996795b89ee3bc5e67f2ce8a0c7f77fe87d33b9ab87aba0ad349515f57d91792 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-22T06:19:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [0, 2]}]: dispatch 2024-03-22T06:19:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 1]}]: dispatch 2024-03-22T06:19:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [3, 1]}]: dispatch 2024-03-22T06:19:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]: dispatch 2024-03-22T06:19:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [0, 2]}]: dispatch 2024-03-22T06:19:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 1]}]: dispatch 2024-03-22T06:19:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 2]}]: dispatch 2024-03-22T06:19:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]: dispatch 2024-03-22T06:19:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 0]}]: dispatch 2024-03-22T06:19:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1f", "id": [3, 1]}]: dispatch 2024-03-22T06:19:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:19:39.133 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:38 smithi006 podman[110004]: 2024-03-22 06:19:38.877287494 +0000 UTC m=+0.850012437 container remove 996795b89ee3bc5e67f2ce8a0c7f77fe87d33b9ab87aba0ad349515f57d91792 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-22T06:19:39.134 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:39 smithi006 podman[110067]: 2024-03-22 06:19:39.086911158 +0000 UTC m=+0.110897586 container create 2b0a2f2a205f37f4bd4acdb700ca6a794f6a963416e95f98cbefda9c237a848d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-22T06:19:39.134 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:39 smithi006 podman[110067]: 2024-03-22 06:19:38.993861363 +0000 UTC m=+0.017847790 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:19:39.420 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:39 smithi006 podman[110067]: 2024-03-22 06:19:39.30596791 +0000 UTC m=+0.329954338 container init 2b0a2f2a205f37f4bd4acdb700ca6a794f6a963416e95f98cbefda9c237a848d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:19:39.420 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:39 smithi006 podman[110067]: 2024-03-22 06:19:39.308848103 +0000 UTC m=+0.332834532 container start 2b0a2f2a205f37f4bd4acdb700ca6a794f6a963416e95f98cbefda9c237a848d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-22T06:19:39.420 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-osd[110085]: -- 172.21.15.6:0/1365750824 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ca1bd6f680 con 0x55ca1bd39000 2024-03-22T06:19:39.420 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: pgmap v75: 65 pgs: 2 active+undersized+degraded, 9 stale+active+clean, 3 active+undersized, 51 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2/219 objects degraded (0.913%) 2024-03-22T06:19:39.420 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [0, 2]}]': finished 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 1]}]': finished 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [3, 1]}]': finished 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]': finished 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [0, 2]}]': finished 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 1]}]': finished 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 2]}]': finished 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]': finished 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 0]}]': finished 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1f", "id": [3, 1]}]': finished 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: osdmap e57: 6 total, 5 up, 6 in 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: Health check failed: Degraded data redundancy: 2/219 objects degraded (0.913%), 2 pgs degraded (PG_DEGRADED) 2024-03-22T06:19:39.421 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-mon[94482]: osdmap e58: 6 total, 5 up, 6 in 2024-03-22T06:19:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: pgmap v75: 65 pgs: 2 active+undersized+degraded, 9 stale+active+clean, 3 active+undersized, 51 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2/219 objects degraded (0.913%) 2024-03-22T06:19:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [0, 2]}]': finished 2024-03-22T06:19:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 1]}]': finished 2024-03-22T06:19:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [3, 1]}]': finished 2024-03-22T06:19:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]': finished 2024-03-22T06:19:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [0, 2]}]': finished 2024-03-22T06:19:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 1]}]': finished 2024-03-22T06:19:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 2]}]': finished 2024-03-22T06:19:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]': finished 2024-03-22T06:19:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 0]}]': finished 2024-03-22T06:19:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1f", "id": [3, 1]}]': finished 2024-03-22T06:19:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: osdmap e57: 6 total, 5 up, 6 in 2024-03-22T06:19:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: Health check failed: Degraded data redundancy: 2/219 objects degraded (0.913%), 2 pgs degraded (PG_DEGRADED) 2024-03-22T06:19:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:39 smithi117 ceph-mon[82126]: osdmap e58: 6 total, 5 up, 6 in 2024-03-22T06:19:39.782 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:39 smithi006 bash[110067]: 2b0a2f2a205f37f4bd4acdb700ca6a794f6a963416e95f98cbefda9c237a848d 2024-03-22T06:19:39.782 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:39 smithi006 systemd[1]: Started Ceph osd.1 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:19:40.282 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:39 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[110081]: 2024-03-22T06:19:39.849+0000 7fe25b5ed7c0 -1 Falling back to public interface 2024-03-22T06:19:40.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:40 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:40.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:40 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:40.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:40 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:19:40.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:40 smithi117 ceph-mon[82126]: osdmap e59: 6 total, 5 up, 6 in 2024-03-22T06:19:40.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:40 smithi117 ceph-mon[82126]: pgmap v79: 65 pgs: 9 peering, 13 active+undersized+degraded, 17 active+undersized, 26 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:19:40.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:40 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:40.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:40 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:40.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:40 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:19:40.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:40 smithi006 ceph-mon[94482]: osdmap e59: 6 total, 5 up, 6 in 2024-03-22T06:19:40.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:40 smithi006 ceph-mon[94482]: pgmap v79: 65 pgs: 9 peering, 13 active+undersized+degraded, 17 active+undersized, 26 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:19:41.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:41 smithi117 ceph-mon[82126]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-22T06:19:41.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:41 smithi117 ceph-mon[82126]: osdmap e60: 6 total, 5 up, 6 in 2024-03-22T06:19:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:41 smithi006 ceph-mon[94482]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-22T06:19:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:41 smithi006 ceph-mon[94482]: osdmap e60: 6 total, 5 up, 6 in 2024-03-22T06:19:42.957 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:42 smithi006 ceph-mon[94482]: osdmap e61: 6 total, 5 up, 6 in 2024-03-22T06:19:42.957 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:42 smithi006 ceph-mon[94482]: pgmap v82: 65 pgs: 1 active+clean+remapped, 9 peering, 13 active+undersized+degraded, 17 active+undersized, 25 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:19:43.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:42 smithi117 ceph-mon[82126]: osdmap e61: 6 total, 5 up, 6 in 2024-03-22T06:19:43.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:42 smithi117 ceph-mon[82126]: pgmap v82: 65 pgs: 1 active+clean+remapped, 9 peering, 13 active+undersized+degraded, 17 active+undersized, 25 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:19:44.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:43 smithi006 ceph-mon[94482]: osdmap e62: 6 total, 5 up, 6 in 2024-03-22T06:19:44.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:43 smithi006 ceph-mon[94482]: Health check update: Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T06:19:44.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:43 smithi006 ceph-mon[94482]: osdmap e63: 6 total, 5 up, 6 in 2024-03-22T06:19:44.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:43 smithi117 ceph-mon[82126]: osdmap e62: 6 total, 5 up, 6 in 2024-03-22T06:19:44.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:43 smithi117 ceph-mon[82126]: Health check update: Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T06:19:44.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:43 smithi117 ceph-mon[82126]: osdmap e63: 6 total, 5 up, 6 in 2024-03-22T06:19:45.032 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:44 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[110081]: 2024-03-22T06:19:44.666+0000 7fe25b5ed7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-22T06:19:45.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:44 smithi006 ceph-mon[94482]: pgmap v85: 65 pgs: 1 active+recovering+undersized+remapped, 1 remapped+peering, 7 active+clean+remapped, 13 active+undersized+degraded, 17 active+undersized, 26 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 31/219 objects degraded (14.155%); 12/219 objects misplaced (5.479%); 4.0 MiB/s, 504 objects/s recovering 2024-03-22T06:19:45.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:44 smithi006 ceph-mon[94482]: osdmap e64: 6 total, 5 up, 6 in 2024-03-22T06:19:45.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:44 smithi117 ceph-mon[82126]: pgmap v85: 65 pgs: 1 active+recovering+undersized+remapped, 1 remapped+peering, 7 active+clean+remapped, 13 active+undersized+degraded, 17 active+undersized, 26 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 31/219 objects degraded (14.155%); 12/219 objects misplaced (5.479%); 4.0 MiB/s, 504 objects/s recovering 2024-03-22T06:19:45.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:44 smithi117 ceph-mon[82126]: osdmap e64: 6 total, 5 up, 6 in 2024-03-22T06:19:45.532 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:45 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[110081]: 2024-03-22T06:19:45.268+0000 7fe25b5ed7c0 -1 osd.1 54 log_to_monitors true 2024-03-22T06:19:46.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:45 smithi117 ceph-mon[82126]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-22T06:19:46.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:45 smithi117 ceph-mon[82126]: from='osd.1 [v2:172.21.15.6:6810/1922945925,v1:172.21.15.6:6811/1922945925]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T06:19:46.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:45 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:46.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:45 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:46.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:45 smithi006 ceph-mon[94482]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-22T06:19:46.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:45 smithi006 ceph-mon[94482]: from='osd.1 [v2:172.21.15.6:6810/1922945925,v1:172.21.15.6:6811/1922945925]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T06:19:46.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:45 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:46.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:45 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:47.032 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:19:46 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[110081]: 2024-03-22T06:19:46.693+0000 7fe248a3d700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:19:47.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:46 smithi006 ceph-mon[94482]: from='osd.1 [v2:172.21.15.6:6810/1922945925,v1:172.21.15.6:6811/1922945925]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T06:19:47.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:46 smithi006 ceph-mon[94482]: osdmap e65: 6 total, 5 up, 6 in 2024-03-22T06:19:47.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:46 smithi006 ceph-mon[94482]: from='osd.1 [v2:172.21.15.6:6810/1922945925,v1:172.21.15.6:6811/1922945925]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:19:47.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:46 smithi006 ceph-mon[94482]: pgmap v88: 65 pgs: 1 active+recovering+undersized+remapped, 1 remapped+peering, 7 active+clean+remapped, 13 active+undersized+degraded, 17 active+undersized, 26 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 31/219 objects degraded (14.155%); 12/219 objects misplaced (5.479%); 4.0 MiB/s, 504 objects/s recovering 2024-03-22T06:19:47.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:46 smithi006 ceph-mon[94482]: from='osd.1 [v2:172.21.15.6:6810/1922945925,v1:172.21.15.6:6811/1922945925]' entity='osd.1' 2024-03-22T06:19:47.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:46 smithi117 ceph-mon[82126]: from='osd.1 [v2:172.21.15.6:6810/1922945925,v1:172.21.15.6:6811/1922945925]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T06:19:47.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:46 smithi117 ceph-mon[82126]: osdmap e65: 6 total, 5 up, 6 in 2024-03-22T06:19:47.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:46 smithi117 ceph-mon[82126]: from='osd.1 [v2:172.21.15.6:6810/1922945925,v1:172.21.15.6:6811/1922945925]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:19:47.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:46 smithi117 ceph-mon[82126]: pgmap v88: 65 pgs: 1 active+recovering+undersized+remapped, 1 remapped+peering, 7 active+clean+remapped, 13 active+undersized+degraded, 17 active+undersized, 26 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 31/219 objects degraded (14.155%); 12/219 objects misplaced (5.479%); 4.0 MiB/s, 504 objects/s recovering 2024-03-22T06:19:47.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:46 smithi117 ceph-mon[82126]: from='osd.1 [v2:172.21.15.6:6810/1922945925,v1:172.21.15.6:6811/1922945925]' entity='osd.1' 2024-03-22T06:19:48.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:47 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:48.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:47 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:48.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:47 smithi006 ceph-mon[94482]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:19:48.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:47 smithi006 ceph-mon[94482]: osd.1 [v2:172.21.15.6:6810/1922945925,v1:172.21.15.6:6811/1922945925] boot 2024-03-22T06:19:48.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:47 smithi006 ceph-mon[94482]: osdmap e66: 6 total, 6 up, 6 in 2024-03-22T06:19:48.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:47 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:19:48.099 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:47 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:48.099 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:47 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:48.099 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:47 smithi117 ceph-mon[82126]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:19:48.099 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:47 smithi117 ceph-mon[82126]: osd.1 [v2:172.21.15.6:6810/1922945925,v1:172.21.15.6:6811/1922945925] boot 2024-03-22T06:19:48.099 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:47 smithi117 ceph-mon[82126]: osdmap e66: 6 total, 6 up, 6 in 2024-03-22T06:19:48.099 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:47 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T06:19:49.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:48 smithi117 ceph-mon[82126]: pgmap v90: 65 pgs: 1 active+recovering+undersized+remapped, 1 remapped+peering, 7 active+clean+remapped, 13 active+undersized+degraded, 17 active+undersized, 26 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 31/219 objects degraded (14.155%); 12/219 objects misplaced (5.479%) 2024-03-22T06:19:49.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:48 smithi117 ceph-mon[82126]: osdmap e67: 6 total, 6 up, 6 in 2024-03-22T06:19:49.176 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:48 smithi006 ceph-mon[94482]: pgmap v90: 65 pgs: 1 active+recovering+undersized+remapped, 1 remapped+peering, 7 active+clean+remapped, 13 active+undersized+degraded, 17 active+undersized, 26 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 31/219 objects degraded (14.155%); 12/219 objects misplaced (5.479%) 2024-03-22T06:19:49.176 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:48 smithi006 ceph-mon[94482]: osdmap e67: 6 total, 6 up, 6 in 2024-03-22T06:19:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: osdmap e68: 6 total, 6 up, 6 in 2024-03-22T06:19:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: pgmap v93: 65 pgs: 17 peering, 26 active+clean, 7 active+undersized+degraded, 1 active+clean+remapped, 8 remapped+peering, 6 active+undersized; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 16/219 objects degraded (7.306%); 0 B/s, 41 objects/s recovering 2024-03-22T06:19:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:19:51.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:19:51.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:51.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:19:51.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:51.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:51.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:51.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:50 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T06:19:51.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: osdmap e68: 6 total, 6 up, 6 in 2024-03-22T06:19:51.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: pgmap v93: 65 pgs: 17 peering, 26 active+clean, 7 active+undersized+degraded, 1 active+clean+remapped, 8 remapped+peering, 6 active+undersized; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 16/219 objects degraded (7.306%); 0 B/s, 41 objects/s recovering 2024-03-22T06:19:51.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:51.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:51.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:19:51.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:19:51.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:51.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:19:51.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:51.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:51.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:51.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:50 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T06:19:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:51 smithi117 ceph-mon[82126]: pgmap v94: 65 pgs: 17 peering, 26 active+clean, 7 active+undersized+degraded, 1 active+clean+remapped, 8 remapped+peering, 6 active+undersized; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 16/219 objects degraded (7.306%); 0 B/s, 40 objects/s recovering 2024-03-22T06:19:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:51 smithi117 ceph-mon[82126]: pgmap v95: 65 pgs: 17 peering, 26 active+clean, 7 active+undersized+degraded, 1 active+clean+remapped, 8 remapped+peering, 6 active+undersized; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 16/219 objects degraded (7.306%); 0 B/s, 63 objects/s recovering 2024-03-22T06:19:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:51 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T06:19:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:51 smithi117 ceph-mon[82126]: Upgrade: unsafe to stop osd(s) at this time (17 PGs are or would become offline) 2024-03-22T06:19:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:51 smithi117 ceph-mon[82126]: Health check update: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded (PG_DEGRADED) 2024-03-22T06:19:52.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:51 smithi117 ceph-mon[82126]: osdmap e69: 6 total, 6 up, 6 in 2024-03-22T06:19:52.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:51 smithi006 ceph-mon[94482]: pgmap v94: 65 pgs: 17 peering, 26 active+clean, 7 active+undersized+degraded, 1 active+clean+remapped, 8 remapped+peering, 6 active+undersized; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 16/219 objects degraded (7.306%); 0 B/s, 40 objects/s recovering 2024-03-22T06:19:52.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:51 smithi006 ceph-mon[94482]: pgmap v95: 65 pgs: 17 peering, 26 active+clean, 7 active+undersized+degraded, 1 active+clean+remapped, 8 remapped+peering, 6 active+undersized; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 16/219 objects degraded (7.306%); 0 B/s, 63 objects/s recovering 2024-03-22T06:19:52.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:51 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T06:19:52.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:51 smithi006 ceph-mon[94482]: Upgrade: unsafe to stop osd(s) at this time (17 PGs are or would become offline) 2024-03-22T06:19:52.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:51 smithi006 ceph-mon[94482]: Health check update: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded (PG_DEGRADED) 2024-03-22T06:19:52.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:51 smithi006 ceph-mon[94482]: osdmap e69: 6 total, 6 up, 6 in 2024-03-22T06:19:53.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:52 smithi117 ceph-mon[82126]: osdmap e70: 6 total, 6 up, 6 in 2024-03-22T06:19:53.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:53.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:19:53.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:52 smithi006 ceph-mon[94482]: osdmap e70: 6 total, 6 up, 6 in 2024-03-22T06:19:53.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:19:53.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:19:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:53 smithi006 ceph-mon[94482]: pgmap v98: 65 pgs: 20 peering, 40 active+clean, 5 remapped+peering; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 4.3 MiB/s, 0 keys/s, 6 objects/s recovering 2024-03-22T06:19:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:53 smithi006 ceph-mon[94482]: osdmap e71: 6 total, 6 up, 6 in 2024-03-22T06:19:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:53 smithi006 ceph-mon[94482]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-22T06:19:53.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:53 smithi006 ceph-mon[94482]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded) 2024-03-22T06:19:54.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:53 smithi117 ceph-mon[82126]: pgmap v98: 65 pgs: 20 peering, 40 active+clean, 5 remapped+peering; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 4.3 MiB/s, 0 keys/s, 6 objects/s recovering 2024-03-22T06:19:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:53 smithi117 ceph-mon[82126]: osdmap e71: 6 total, 6 up, 6 in 2024-03-22T06:19:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:53 smithi117 ceph-mon[82126]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-22T06:19:54.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:53 smithi117 ceph-mon[82126]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded) 2024-03-22T06:19:54.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 -- 172.21.15.6:0/4000035146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff740fce70 msgr2=0x7eff740fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:54.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 --2- 172.21.15.6:0/4000035146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff740fce70 0x7eff740fd280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff70004940 tx=0x7eff70009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:54.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 -- 172.21.15.6:0/4000035146 shutdown_connections 2024-03-22T06:19:54.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 --2- 172.21.15.6:0/4000035146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff740fdfe0 0x7eff740fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 --2- 172.21.15.6:0/4000035146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff740fce70 0x7eff740fd280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 -- 172.21.15.6:0/4000035146 >> 172.21.15.6:0/4000035146 conn(0x7eff740f8550 msgr2=0x7eff740fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 -- 172.21.15.6:0/4000035146 shutdown_connections 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 -- 172.21.15.6:0/4000035146 wait complete. 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 Processor -- start 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 -- start start 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff740fce70 0x7eff7410fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff740fdfe0 0x7eff74110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff74110670 con 0x7eff740fdfe0 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.265+0000 7eff7ba9a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff741107e0 con 0x7eff740fce70 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff79035700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff740fdfe0 0x7eff74110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff79035700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff740fdfe0 0x7eff74110070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35436/0 (socket says 172.21.15.6:35436) 2024-03-22T06:19:54.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff79035700 1 -- 172.21.15.6:0/563425593 learned_addr learned my addr 172.21.15.6:0/563425593 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:54.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff79836700 1 --2- 172.21.15.6:0/563425593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff740fce70 0x7eff7410fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:54.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff79035700 1 -- 172.21.15.6:0/563425593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff740fce70 msgr2=0x7eff7410fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:54.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff79035700 1 --2- 172.21.15.6:0/563425593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff740fce70 0x7eff7410fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff79035700 1 -- 172.21.15.6:0/563425593 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff70005040 con 0x7eff740fdfe0 2024-03-22T06:19:54.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff79035700 1 --2- 172.21.15.6:0/563425593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff740fdfe0 0x7eff74110070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c8c0 tx=0x7eff640044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:54.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff6a7fc700 1 -- 172.21.15.6:0/563425593 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff6400a760 con 0x7eff740fdfe0 2024-03-22T06:19:54.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff6a7fc700 1 -- 172.21.15.6:0/563425593 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff6400ad90 con 0x7eff740fdfe0 2024-03-22T06:19:54.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff7ba9a700 1 -- 172.21.15.6:0/563425593 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff74115240 con 0x7eff740fdfe0 2024-03-22T06:19:54.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff6a7fc700 1 -- 172.21.15.6:0/563425593 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff640113a0 con 0x7eff740fdfe0 2024-03-22T06:19:54.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.266+0000 7eff7ba9a700 1 -- 172.21.15.6:0/563425593 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff74115800 con 0x7eff740fdfe0 2024-03-22T06:19:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.268+0000 7eff7ba9a700 1 -- 172.21.15.6:0/563425593 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff74101970 con 0x7eff740fdfe0 2024-03-22T06:19:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.268+0000 7eff6a7fc700 1 -- 172.21.15.6:0/563425593 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7eff64012400 con 0x7eff740fdfe0 2024-03-22T06:19:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.269+0000 7eff6a7fc700 1 --2- 172.21.15.6:0/563425593 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff6006e840 0x7eff60070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.269+0000 7eff6a7fc700 1 -- 172.21.15.6:0/563425593 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6614+0+0 (secure 0 0 0) 0x7eff6408e480 con 0x7eff740fdfe0 2024-03-22T06:19:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.269+0000 7eff79836700 1 --2- 172.21.15.6:0/563425593 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff6006e840 0x7eff60070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:54.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.269+0000 7eff79836700 1 --2- 172.21.15.6:0/563425593 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff6006e840 0x7eff60070cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7eff7000b970 tx=0x7eff70009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:54.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.274+0000 7eff6a7fc700 1 -- 172.21.15.6:0/563425593 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7eff6405af50 con 0x7eff740fdfe0 2024-03-22T06:19:54.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.509+0000 7eff7ba9a700 1 -- 172.21.15.6:0/563425593 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff7402b310 con 0x7eff6006e840 2024-03-22T06:19:54.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.511+0000 7eff6a7fc700 1 -- 172.21.15.6:0/563425593 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7eff7402b310 con 0x7eff6006e840 2024-03-22T06:19:54.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.513+0000 7eff7ba9a700 1 -- 172.21.15.6:0/563425593 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff6006e840 msgr2=0x7eff60070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:54.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.513+0000 7eff7ba9a700 1 --2- 172.21.15.6:0/563425593 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff6006e840 0x7eff60070cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7eff7000b970 tx=0x7eff70009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:54.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.513+0000 7eff7ba9a700 1 -- 172.21.15.6:0/563425593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff740fdfe0 msgr2=0x7eff74110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:54.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.513+0000 7eff7ba9a700 1 --2- 172.21.15.6:0/563425593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff740fdfe0 0x7eff74110070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c8c0 tx=0x7eff640044c0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.513+0000 7eff7ba9a700 1 -- 172.21.15.6:0/563425593 shutdown_connections 2024-03-22T06:19:54.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.513+0000 7eff7ba9a700 1 --2- 172.21.15.6:0/563425593 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff6006e840 0x7eff60070cf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.513+0000 7eff7ba9a700 1 --2- 172.21.15.6:0/563425593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff740fce70 0x7eff7410fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.513+0000 7eff7ba9a700 1 --2- 172.21.15.6:0/563425593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff740fdfe0 0x7eff74110070 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.513+0000 7eff7ba9a700 1 -- 172.21.15.6:0/563425593 >> 172.21.15.6:0/563425593 conn(0x7eff740f8550 msgr2=0x7eff740f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:54.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.513+0000 7eff7ba9a700 1 -- 172.21.15.6:0/563425593 shutdown_connections 2024-03-22T06:19:54.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.513+0000 7eff7ba9a700 1 -- 172.21.15.6:0/563425593 wait complete. 2024-03-22T06:19:54.524 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:19:54.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.619+0000 7f2874dce700 1 -- 172.21.15.6:0/279598400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28700fdfe0 msgr2=0x7f28700fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:54.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.619+0000 7f2874dce700 1 --2- 172.21.15.6:0/279598400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28700fdfe0 0x7f28700fe450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2864004940 tx=0x7f2864009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:54.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.619+0000 7f2874dce700 1 -- 172.21.15.6:0/279598400 shutdown_connections 2024-03-22T06:19:54.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.619+0000 7f2874dce700 1 --2- 172.21.15.6:0/279598400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28700fdfe0 0x7f28700fe450 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.619+0000 7f2874dce700 1 --2- 172.21.15.6:0/279598400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28700fce70 0x7f28700fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.619+0000 7f2874dce700 1 -- 172.21.15.6:0/279598400 >> 172.21.15.6:0/279598400 conn(0x7f28700f8550 msgr2=0x7f28700fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:54.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f2874dce700 1 -- 172.21.15.6:0/279598400 shutdown_connections 2024-03-22T06:19:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f2874dce700 1 -- 172.21.15.6:0/279598400 wait complete. 2024-03-22T06:19:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f2874dce700 1 Processor -- start 2024-03-22T06:19:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f2874dce700 1 -- start start 2024-03-22T06:19:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f2874dce700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28700fce70 0x7f287010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f2874dce700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28700fdfe0 0x7f287010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f2874dce700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287010e460 con 0x7f28700fdfe0 2024-03-22T06:19:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f2874dce700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287010e5d0 con 0x7f28700fce70 2024-03-22T06:19:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f286dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28700fdfe0 0x7f287010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f286dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28700fdfe0 0x7f287010de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35444/0 (socket says 172.21.15.6:35444) 2024-03-22T06:19:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f286dd9b700 1 -- 172.21.15.6:0/632104497 learned_addr learned my addr 172.21.15.6:0/632104497 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.620+0000 7f286e59c700 1 --2- 172.21.15.6:0/632104497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28700fce70 0x7f287010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.621+0000 7f286dd9b700 1 -- 172.21.15.6:0/632104497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28700fce70 msgr2=0x7f287010d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.621+0000 7f286dd9b700 1 --2- 172.21.15.6:0/632104497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28700fce70 0x7f287010d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.621+0000 7f286dd9b700 1 -- 172.21.15.6:0/632104497 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864005040 con 0x7f28700fdfe0 2024-03-22T06:19:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.621+0000 7f286dd9b700 1 --2- 172.21.15.6:0/632104497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28700fdfe0 0x7f287010de60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f286400b3b0 tx=0x7f2864009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.621+0000 7f285effd700 1 -- 172.21.15.6:0/632104497 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f286400e040 con 0x7f28700fdfe0 2024-03-22T06:19:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.621+0000 7f285effd700 1 -- 172.21.15.6:0/632104497 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f286400bcc0 con 0x7f28700fdfe0 2024-03-22T06:19:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.621+0000 7f2874dce700 1 -- 172.21.15.6:0/632104497 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2870112fd0 con 0x7f28700fdfe0 2024-03-22T06:19:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.621+0000 7f285effd700 1 -- 172.21.15.6:0/632104497 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f286401c630 con 0x7f28700fdfe0 2024-03-22T06:19:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.621+0000 7f2874dce700 1 -- 172.21.15.6:0/632104497 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2870113560 con 0x7f28700fdfe0 2024-03-22T06:19:54.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.622+0000 7f2874dce700 1 -- 172.21.15.6:0/632104497 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287004c710 con 0x7f28700fdfe0 2024-03-22T06:19:54.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.622+0000 7f285effd700 1 -- 172.21.15.6:0/632104497 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f2864013980 con 0x7f28700fdfe0 2024-03-22T06:19:54.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.623+0000 7f285effd700 1 --2- 172.21.15.6:0/632104497 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f285806e840 0x7f2858070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:54.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.623+0000 7f285effd700 1 -- 172.21.15.6:0/632104497 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f286408f140 con 0x7f28700fdfe0 2024-03-22T06:19:54.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.623+0000 7f286e59c700 1 --2- 172.21.15.6:0/632104497 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f285806e840 0x7f2858070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:54.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.623+0000 7f286e59c700 1 --2- 172.21.15.6:0/632104497 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f285806e840 0x7f2858070cf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2860007fd0 tx=0x7f2860009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:54.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.627+0000 7f285effd700 1 -- 172.21.15.6:0/632104497 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f286405bb90 con 0x7f28700fdfe0 2024-03-22T06:19:54.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.857+0000 7f2874dce700 1 -- 172.21.15.6:0/632104497 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28701023f0 con 0x7f285806e840 2024-03-22T06:19:54.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.858+0000 7f285effd700 1 -- 172.21.15.6:0/632104497 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f28701023f0 con 0x7f285806e840 2024-03-22T06:19:54.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.860+0000 7f2874dce700 1 -- 172.21.15.6:0/632104497 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f285806e840 msgr2=0x7f2858070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:54.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.860+0000 7f2874dce700 1 --2- 172.21.15.6:0/632104497 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f285806e840 0x7f2858070cf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2860007fd0 tx=0x7f2860009040 comp rx=0 tx=0).stop 2024-03-22T06:19:54.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.860+0000 7f2874dce700 1 -- 172.21.15.6:0/632104497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28700fdfe0 msgr2=0x7f287010de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:54.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.860+0000 7f2874dce700 1 --2- 172.21.15.6:0/632104497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28700fdfe0 0x7f287010de60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f286400b3b0 tx=0x7f2864009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:54.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.860+0000 7f2874dce700 1 -- 172.21.15.6:0/632104497 shutdown_connections 2024-03-22T06:19:54.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.860+0000 7f2874dce700 1 --2- 172.21.15.6:0/632104497 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f285806e840 0x7f2858070cf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.860+0000 7f2874dce700 1 --2- 172.21.15.6:0/632104497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28700fce70 0x7f287010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.860+0000 7f2874dce700 1 --2- 172.21.15.6:0/632104497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28700fdfe0 0x7f287010de60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.860+0000 7f2874dce700 1 -- 172.21.15.6:0/632104497 >> 172.21.15.6:0/632104497 conn(0x7f28700f8550 msgr2=0x7f28700f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:54.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.861+0000 7f2874dce700 1 -- 172.21.15.6:0/632104497 shutdown_connections 2024-03-22T06:19:54.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.861+0000 7f2874dce700 1 -- 172.21.15.6:0/632104497 wait complete. 2024-03-22T06:19:54.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.965+0000 7fd41107b700 1 -- 172.21.15.6:0/2953946511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd40c0fbce0 msgr2=0x7fd40c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:54.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.965+0000 7fd41107b700 1 --2- 172.21.15.6:0/2953946511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd40c0fbce0 0x7fd40c0fc0f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4004940 tx=0x7fd3f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:54.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.965+0000 7fd41107b700 1 -- 172.21.15.6:0/2953946511 shutdown_connections 2024-03-22T06:19:54.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.965+0000 7fd41107b700 1 --2- 172.21.15.6:0/2953946511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0fcf90 0x7fd40c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.965+0000 7fd41107b700 1 --2- 172.21.15.6:0/2953946511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd40c0fbce0 0x7fd40c0fc0f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.965+0000 7fd41107b700 1 -- 172.21.15.6:0/2953946511 >> 172.21.15.6:0/2953946511 conn(0x7fd40c0f7480 msgr2=0x7fd40c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:54.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.965+0000 7fd41107b700 1 -- 172.21.15.6:0/2953946511 shutdown_connections 2024-03-22T06:19:54.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.965+0000 7fd41107b700 1 -- 172.21.15.6:0/2953946511 wait complete. 2024-03-22T06:19:54.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.965+0000 7fd41107b700 1 Processor -- start 2024-03-22T06:19:54.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd41107b700 1 -- start start 2024-03-22T06:19:54.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd41107b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd40c0fbce0 0x7fd40c0ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:54.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd41107b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0fcf90 0x7fd40c0fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:54.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd41107b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0fe880 con 0x7fd40c0fbce0 2024-03-22T06:19:54.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd41107b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0fe9f0 con 0x7fd40c0fcf90 2024-03-22T06:19:54.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd40ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd40c0fbce0 0x7fd40c0ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:54.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd40ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd40c0fbce0 0x7fd40c0ffe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35456/0 (socket says 172.21.15.6:35456) 2024-03-22T06:19:54.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd40ad9d700 1 -- 172.21.15.6:0/1984775893 learned_addr learned my addr 172.21.15.6:0/1984775893 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:54.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd40a59c700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0fcf90 0x7fd40c0fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:54.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd40ad9d700 1 -- 172.21.15.6:0/1984775893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0fcf90 msgr2=0x7fd40c0fe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:54.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd40ad9d700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0fcf90 0x7fd40c0fe2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:54.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd40ad9d700 1 -- 172.21.15.6:0/1984775893 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f4005040 con 0x7fd40c0fbce0 2024-03-22T06:19:54.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd40a59c700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0fcf90 0x7fd40c0fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:19:54.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.966+0000 7fd40ad9d700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd40c0fbce0 0x7fd40c0ffe00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009150 tx=0x7fd3f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:54.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.967+0000 7fd4037fe700 1 -- 172.21.15.6:0/1984775893 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3f400e040 con 0x7fd40c0fbce0 2024-03-22T06:19:54.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.967+0000 7fd4037fe700 1 -- 172.21.15.6:0/1984775893 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3f400bcc0 con 0x7fd40c0fbce0 2024-03-22T06:19:54.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.967+0000 7fd41107b700 1 -- 172.21.15.6:0/1984775893 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c0fec70 con 0x7fd40c0fbce0 2024-03-22T06:19:54.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.967+0000 7fd4037fe700 1 -- 172.21.15.6:0/1984775893 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3f401c630 con 0x7fd40c0fbce0 2024-03-22T06:19:54.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.967+0000 7fd41107b700 1 -- 172.21.15.6:0/1984775893 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c115990 con 0x7fd40c0fbce0 2024-03-22T06:19:54.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.968+0000 7fd41107b700 1 -- 172.21.15.6:0/1984775893 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c05f5f0 con 0x7fd40c0fbce0 2024-03-22T06:19:54.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.968+0000 7fd4037fe700 1 -- 172.21.15.6:0/1984775893 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fd3f4013420 con 0x7fd40c0fbce0 2024-03-22T06:19:54.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.969+0000 7fd4037fe700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd3f806e840 0x7fd3f8070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:54.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.969+0000 7fd4037fe700 1 -- 172.21.15.6:0/1984775893 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6614+0+0 (secure 0 0 0) 0x7fd3f408f020 con 0x7fd40c0fbce0 2024-03-22T06:19:54.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.969+0000 7fd40a59c700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd3f806e840 0x7fd3f8070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:54.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.969+0000 7fd40a59c700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd3f806e840 0x7fd3f8070cf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0095c0 tx=0x7fd3fc007ab0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:54.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:54.974+0000 7fd4037fe700 1 -- 172.21.15.6:0/1984775893 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fd3f4058060 con 0x7fd40c0fbce0 2024-03-22T06:19:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.202+0000 7fd41107b700 1 -- 172.21.15.6:0/1984775893 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd40c104980 con 0x7fd3f806e840 2024-03-22T06:19:55.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.213+0000 7fd4037fe700 1 -- 172.21.15.6:0/1984775893 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd3fc009200 con 0x7fd3f806e840 2024-03-22T06:19:55.213 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:19:55.213 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (4m) 9s ago 12m 26.2M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:19:55.213 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (82s) 9s ago 13m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (77s) 74s ago 12m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 unknown 9s ago 12m - - 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (7m) 9s ago 7m 157M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (7m) 9s ago 7m 17.9M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (7m) 74s ago 7m 14.6M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (7m) 74s ago 7m 16.4M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (3m) 9s ago 14m 503M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (2m) 74s ago 12m 450M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (2m) 9s ago 14m 46.4M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (2m) 74s ago 11m 33.5M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (4m) 9s ago 13m 17.0M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (3m) 74s ago 12m 14.9M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (54s) 9s ago 11m 149M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:19:55.214 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (16s) 9s ago 10m 12.6M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:19:55.215 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (10m) 9s ago 10m 238M 1559M 17.2.7-726-g9c4e0c34 837124da804f 7bdf9fdb6319 2024-03-22T06:19:55.215 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (9m) 74s ago 9m 284M 2581M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:19:55.215 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (9m) 74s ago 9m 250M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:19:55.215 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (9m) 74s ago 9m 237M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:19:55.215 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (2m) 9s ago 12m 64.3M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:19:55.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.215+0000 7fd41107b700 1 -- 172.21.15.6:0/1984775893 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd3f806e840 msgr2=0x7fd3f8070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:55.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.215+0000 7fd41107b700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd3f806e840 0x7fd3f8070cf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0095c0 tx=0x7fd3fc007ab0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.215+0000 7fd41107b700 1 -- 172.21.15.6:0/1984775893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd40c0fbce0 msgr2=0x7fd40c0ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:55.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.215+0000 7fd41107b700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd40c0fbce0 0x7fd40c0ffe00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009150 tx=0x7fd3f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:55.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.215+0000 7fd41107b700 1 -- 172.21.15.6:0/1984775893 shutdown_connections 2024-03-22T06:19:55.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.215+0000 7fd41107b700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd3f806e840 0x7fd3f8070cf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.215+0000 7fd41107b700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd40c0fbce0 0x7fd40c0ffe00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.215+0000 7fd41107b700 1 --2- 172.21.15.6:0/1984775893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0fcf90 0x7fd40c0fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.215+0000 7fd41107b700 1 -- 172.21.15.6:0/1984775893 >> 172.21.15.6:0/1984775893 conn(0x7fd40c0f7480 msgr2=0x7fd40c0663a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:55.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.215+0000 7fd41107b700 1 -- 172.21.15.6:0/1984775893 shutdown_connections 2024-03-22T06:19:55.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.215+0000 7fd41107b700 1 -- 172.21.15.6:0/1984775893 wait complete. 2024-03-22T06:19:55.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.317+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3234553240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3140fbcd0 msgr2=0x7fa3140fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:55.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.317+0000 7fa31ae0e700 1 --2- 172.21.15.6:0/3234553240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3140fbcd0 0x7fa3140fc0e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa308004940 tx=0x7fa308009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:55.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.317+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3234553240 shutdown_connections 2024-03-22T06:19:55.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.317+0000 7fa31ae0e700 1 --2- 172.21.15.6:0/3234553240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3140fcf80 0x7fa3140fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.317+0000 7fa31ae0e700 1 --2- 172.21.15.6:0/3234553240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3140fbcd0 0x7fa3140fc0e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.317+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3234553240 >> 172.21.15.6:0/3234553240 conn(0x7fa3140f7430 msgr2=0x7fa3140f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:55.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.317+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3234553240 shutdown_connections 2024-03-22T06:19:55.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.317+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3234553240 wait complete. 2024-03-22T06:19:55.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.317+0000 7fa31ae0e700 1 Processor -- start 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.317+0000 7fa31ae0e700 1 -- start start 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa31ae0e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3140fbcd0 0x7fa31410c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa31ae0e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3140fcf80 0x7fa31410ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa31ae0e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31410d4e0 con 0x7fa3140fcf80 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa31ae0e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31410dcd0 con 0x7fa3140fbcd0 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa313fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3140fcf80 0x7fa31410ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa313fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3140fcf80 0x7fa31410ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35474/0 (socket says 172.21.15.6:35474) 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa313fff700 1 -- 172.21.15.6:0/3721822830 learned_addr learned my addr 172.21.15.6:0/3721822830 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa318baa700 1 --2- 172.21.15.6:0/3721822830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3140fbcd0 0x7fa31410c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa313fff700 1 -- 172.21.15.6:0/3721822830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3140fbcd0 msgr2=0x7fa31410c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa313fff700 1 --2- 172.21.15.6:0/3721822830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3140fbcd0 0x7fa31410c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa313fff700 1 -- 172.21.15.6:0/3721822830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa308005040 con 0x7fa3140fcf80 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa313fff700 1 --2- 172.21.15.6:0/3721822830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3140fcf80 0x7fa31410ce50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa30000b790 tx=0x7fa300009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:55.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa3117fa700 1 -- 172.21.15.6:0/3721822830 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa30000f770 con 0x7fa3140fcf80 2024-03-22T06:19:55.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3721822830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31410dfb0 con 0x7fa3140fcf80 2024-03-22T06:19:55.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.318+0000 7fa3117fa700 1 -- 172.21.15.6:0/3721822830 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa30000fda0 con 0x7fa3140fcf80 2024-03-22T06:19:55.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.319+0000 7fa3117fa700 1 -- 172.21.15.6:0/3721822830 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa30000d310 con 0x7fa3140fcf80 2024-03-22T06:19:55.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.319+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3721822830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31410e570 con 0x7fa3140fcf80 2024-03-22T06:19:55.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.320+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3721822830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31405f5f0 con 0x7fa3140fcf80 2024-03-22T06:19:55.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.320+0000 7fa3117fa700 1 -- 172.21.15.6:0/3721822830 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fa30000f8d0 con 0x7fa3140fcf80 2024-03-22T06:19:55.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.320+0000 7fa3117fa700 1 --2- 172.21.15.6:0/3721822830 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa30406e840 0x7fa304070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:55.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.320+0000 7fa3117fa700 1 -- 172.21.15.6:0/3721822830 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6614+0+0 (secure 0 0 0) 0x7fa30008e4b0 con 0x7fa3140fcf80 2024-03-22T06:19:55.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.320+0000 7fa318baa700 1 --2- 172.21.15.6:0/3721822830 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa30406e840 0x7fa304070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:55.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.321+0000 7fa318baa700 1 --2- 172.21.15.6:0/3721822830 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa30406e840 0x7fa304070cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa30800b3b0 tx=0x7fa308009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:55.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.325+0000 7fa3117fa700 1 -- 172.21.15.6:0/3721822830 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fa30005af80 con 0x7fa3140fcf80 2024-03-22T06:19:55.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.620+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3721822830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa31410e7f0 con 0x7fa3140fcf80 2024-03-22T06:19:55.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.621+0000 7fa3117fa700 1 -- 172.21.15.6:0/3721822830 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fa300020250 con 0x7fa3140fcf80 2024-03-22T06:19:55.621 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:19:55.621 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4, 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:19:55.622 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 8, 2024-03-22T06:19:55.623 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 6 2024-03-22T06:19:55.623 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:19:55.623 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:19:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.623+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3721822830 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa30406e840 msgr2=0x7fa304070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.623+0000 7fa31ae0e700 1 --2- 172.21.15.6:0/3721822830 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa30406e840 0x7fa304070cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa30800b3b0 tx=0x7fa308009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.623+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3721822830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3140fcf80 msgr2=0x7fa31410ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.623+0000 7fa31ae0e700 1 --2- 172.21.15.6:0/3721822830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3140fcf80 0x7fa31410ce50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa30000b790 tx=0x7fa300009250 comp rx=0 tx=0).stop 2024-03-22T06:19:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.623+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3721822830 shutdown_connections 2024-03-22T06:19:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.623+0000 7fa31ae0e700 1 --2- 172.21.15.6:0/3721822830 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa30406e840 0x7fa304070cf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.623+0000 7fa31ae0e700 1 --2- 172.21.15.6:0/3721822830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3140fbcd0 0x7fa31410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.623+0000 7fa31ae0e700 1 --2- 172.21.15.6:0/3721822830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3140fcf80 0x7fa31410ce50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.623+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3721822830 >> 172.21.15.6:0/3721822830 conn(0x7fa3140f7430 msgr2=0x7fa3140f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.623+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3721822830 shutdown_connections 2024-03-22T06:19:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.623+0000 7fa31ae0e700 1 -- 172.21.15.6:0/3721822830 wait complete. 2024-03-22T06:19:55.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.730+0000 7fce35d49700 1 -- 172.21.15.6:0/3874203695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce300fcf90 msgr2=0x7fce300fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:55.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.730+0000 7fce35d49700 1 --2- 172.21.15.6:0/3874203695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce300fcf90 0x7fce300fd400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fce24004940 tx=0x7fce24009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:55.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.730+0000 7fce35d49700 1 -- 172.21.15.6:0/3874203695 shutdown_connections 2024-03-22T06:19:55.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.730+0000 7fce35d49700 1 --2- 172.21.15.6:0/3874203695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce300fcf90 0x7fce300fd400 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.730+0000 7fce35d49700 1 --2- 172.21.15.6:0/3874203695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce300fbce0 0x7fce300fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.730+0000 7fce35d49700 1 -- 172.21.15.6:0/3874203695 >> 172.21.15.6:0/3874203695 conn(0x7fce300f7480 msgr2=0x7fce300f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.730+0000 7fce35d49700 1 -- 172.21.15.6:0/3874203695 shutdown_connections 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.730+0000 7fce35d49700 1 -- 172.21.15.6:0/3874203695 wait complete. 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.730+0000 7fce35d49700 1 Processor -- start 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.730+0000 7fce35d49700 1 -- start start 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce35d49700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce300fbce0 0x7fce300ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce35d49700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce300fcf90 0x7fce300fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce35d49700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300fe860 con 0x7fce300fcf90 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce35d49700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300fe9a0 con 0x7fce300fbce0 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce2effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce300fcf90 0x7fce300fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce2effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce300fcf90 0x7fce300fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35496/0 (socket says 172.21.15.6:35496) 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce2effd700 1 -- 172.21.15.6:0/2549582680 learned_addr learned my addr 172.21.15.6:0/2549582680 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce2f7fe700 1 --2- 172.21.15.6:0/2549582680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce300fbce0 0x7fce300ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce2effd700 1 -- 172.21.15.6:0/2549582680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce300fbce0 msgr2=0x7fce300ffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce2effd700 1 --2- 172.21.15.6:0/2549582680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce300fbce0 0x7fce300ffdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce2effd700 1 -- 172.21.15.6:0/2549582680 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce24005040 con 0x7fce300fcf90 2024-03-22T06:19:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce2effd700 1 --2- 172.21.15.6:0/2549582680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce300fcf90 0x7fce300fe2d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce2400b3b0 tx=0x7fce24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce17fff700 1 -- 172.21.15.6:0/2549582680 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce2400e040 con 0x7fce300fcf90 2024-03-22T06:19:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce17fff700 1 -- 172.21.15.6:0/2549582680 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce24004290 con 0x7fce300fcf90 2024-03-22T06:19:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce35d49700 1 -- 172.21.15.6:0/2549582680 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce300fec20 con 0x7fce300fcf90 2024-03-22T06:19:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce17fff700 1 -- 172.21.15.6:0/2549582680 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce2401c630 con 0x7fce300fcf90 2024-03-22T06:19:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.731+0000 7fce35d49700 1 -- 172.21.15.6:0/2549582680 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce30115ab0 con 0x7fce300fcf90 2024-03-22T06:19:55.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.732+0000 7fce35d49700 1 -- 172.21.15.6:0/2549582680 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce30101b20 con 0x7fce300fcf90 2024-03-22T06:19:55.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.733+0000 7fce17fff700 1 -- 172.21.15.6:0/2549582680 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fce2400bbf0 con 0x7fce300fcf90 2024-03-22T06:19:55.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.733+0000 7fce17fff700 1 --2- 172.21.15.6:0/2549582680 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fce1806e850 0x7fce18070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:55.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.733+0000 7fce17fff700 1 -- 172.21.15.6:0/2549582680 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6614+0+0 (secure 0 0 0) 0x7fce2408f8a0 con 0x7fce300fcf90 2024-03-22T06:19:55.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.733+0000 7fce2f7fe700 1 --2- 172.21.15.6:0/2549582680 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fce1806e850 0x7fce18070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:55.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.734+0000 7fce2f7fe700 1 --2- 172.21.15.6:0/2549582680 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fce1806e850 0x7fce18070d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce300ffb40 tx=0x7fce2000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:55.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:55.738+0000 7fce17fff700 1 -- 172.21.15.6:0/2549582680 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fce24058840 con 0x7fce300fcf90 2024-03-22T06:19:56.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:55 smithi117 ceph-mon[82126]: pgmap v100: 65 pgs: 3 peering, 62 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 8.0 MiB/s, 68 keys/s, 24 objects/s recovering 2024-03-22T06:19:56.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:55 smithi117 ceph-mon[82126]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:56.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:55 smithi117 ceph-mon[82126]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:56.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:55 smithi117 ceph-mon[82126]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:56.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:55 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3721822830' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:56.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.022+0000 7fce35d49700 1 -- 172.21.15.6:0/2549582680 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce30101540 con 0x7fce300fcf90 2024-03-22T06:19:56.023 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:55 smithi006 ceph-mon[94482]: pgmap v100: 65 pgs: 3 peering, 62 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 8.0 MiB/s, 68 keys/s, 24 objects/s recovering 2024-03-22T06:19:56.023 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:55 smithi006 ceph-mon[94482]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:56.024 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:55 smithi006 ceph-mon[94482]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:56.024 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:55 smithi006 ceph-mon[94482]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:56.024 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:55 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3721822830' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:19:56.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.023+0000 7fce17fff700 1 -- 172.21.15.6:0/2549582680 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fce2405bf00 con 0x7fce300fcf90 2024-03-22T06:19:56.024 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:19:56.024 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:19:56.024 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:19:56.025 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:19:56.026 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 0 members: 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:19:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:19:56.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.025+0000 7fce35d49700 1 -- 172.21.15.6:0/2549582680 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fce1806e850 msgr2=0x7fce18070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:56.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.025+0000 7fce35d49700 1 --2- 172.21.15.6:0/2549582680 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fce1806e850 0x7fce18070d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce300ffb40 tx=0x7fce2000b040 comp rx=0 tx=0).stop 2024-03-22T06:19:56.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.025+0000 7fce35d49700 1 -- 172.21.15.6:0/2549582680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce300fcf90 msgr2=0x7fce300fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:56.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.025+0000 7fce35d49700 1 --2- 172.21.15.6:0/2549582680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce300fcf90 0x7fce300fe2d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce2400b3b0 tx=0x7fce24009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:56.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.025+0000 7fce35d49700 1 -- 172.21.15.6:0/2549582680 shutdown_connections 2024-03-22T06:19:56.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.025+0000 7fce35d49700 1 --2- 172.21.15.6:0/2549582680 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fce1806e850 0x7fce18070d00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.025+0000 7fce35d49700 1 --2- 172.21.15.6:0/2549582680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce300fbce0 0x7fce300ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.025+0000 7fce35d49700 1 --2- 172.21.15.6:0/2549582680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce300fcf90 0x7fce300fe2d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.025+0000 7fce35d49700 1 -- 172.21.15.6:0/2549582680 >> 172.21.15.6:0/2549582680 conn(0x7fce300f7480 msgr2=0x7fce300664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:56.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.025+0000 7fce35d49700 1 -- 172.21.15.6:0/2549582680 shutdown_connections 2024-03-22T06:19:56.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.025+0000 7fce35d49700 1 -- 172.21.15.6:0/2549582680 wait complete. 2024-03-22T06:19:56.029 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:19:56.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.130+0000 7f2e623e9700 1 -- 172.21.15.6:0/2272678643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e5c0fbcd0 msgr2=0x7f2e5c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:56.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.130+0000 7f2e623e9700 1 --2- 172.21.15.6:0/2272678643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e5c0fbcd0 0x7f2e5c0fc0e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c004940 tx=0x7f2e4c009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:56.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.130+0000 7f2e623e9700 1 -- 172.21.15.6:0/2272678643 shutdown_connections 2024-03-22T06:19:56.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.130+0000 7f2e623e9700 1 --2- 172.21.15.6:0/2272678643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c0fcf80 0x7f2e5c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.130+0000 7f2e623e9700 1 --2- 172.21.15.6:0/2272678643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e5c0fbcd0 0x7f2e5c0fc0e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.130+0000 7f2e623e9700 1 -- 172.21.15.6:0/2272678643 >> 172.21.15.6:0/2272678643 conn(0x7f2e5c0f7450 msgr2=0x7f2e5c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:56.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.130+0000 7f2e623e9700 1 -- 172.21.15.6:0/2272678643 shutdown_connections 2024-03-22T06:19:56.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.130+0000 7f2e623e9700 1 -- 172.21.15.6:0/2272678643 wait complete. 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.130+0000 7f2e623e9700 1 Processor -- start 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e623e9700 1 -- start start 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e623e9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c0fbcd0 0x7f2e5c066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e623e9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e5c0fcf80 0x7f2e5c112710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e623e9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c112c50 con 0x7f2e5c0fcf80 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e623e9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c112d90 con 0x7f2e5c0fbcd0 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e5b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e5c0fcf80 0x7f2e5c112710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e5b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e5c0fcf80 0x7f2e5c112710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35504/0 (socket says 172.21.15.6:35504) 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e5b7fe700 1 -- 172.21.15.6:0/2148661854 learned_addr learned my addr 172.21.15.6:0/2148661854 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e5bfff700 1 --2- 172.21.15.6:0/2148661854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c0fbcd0 0x7f2e5c066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e5b7fe700 1 -- 172.21.15.6:0/2148661854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c0fbcd0 msgr2=0x7f2e5c066cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e5b7fe700 1 --2- 172.21.15.6:0/2148661854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c0fbcd0 0x7f2e5c066cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e5b7fe700 1 -- 172.21.15.6:0/2148661854 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c005040 con 0x7f2e5c0fcf80 2024-03-22T06:19:56.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e5b7fe700 1 --2- 172.21.15.6:0/2148661854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e5c0fcf80 0x7f2e5c112710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000c830 tx=0x7f2e50004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:56.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.131+0000 7f2e58ff9700 1 -- 172.21.15.6:0/2148661854 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e500106f0 con 0x7f2e5c0fcf80 2024-03-22T06:19:56.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.132+0000 7f2e623e9700 1 -- 172.21.15.6:0/2148661854 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5c113020 con 0x7f2e5c0fcf80 2024-03-22T06:19:56.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.132+0000 7f2e58ff9700 1 -- 172.21.15.6:0/2148661854 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e50010d20 con 0x7f2e5c0fcf80 2024-03-22T06:19:56.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.132+0000 7f2e58ff9700 1 -- 172.21.15.6:0/2148661854 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e500113a0 con 0x7f2e5c0fcf80 2024-03-22T06:19:56.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.132+0000 7f2e623e9700 1 -- 172.21.15.6:0/2148661854 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5c1135b0 con 0x7f2e5c0fcf80 2024-03-22T06:19:56.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.133+0000 7f2e623e9700 1 -- 172.21.15.6:0/2148661854 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e5c04c710 con 0x7f2e5c0fcf80 2024-03-22T06:19:56.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.133+0000 7f2e58ff9700 1 -- 172.21.15.6:0/2148661854 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f2e50010850 con 0x7f2e5c0fcf80 2024-03-22T06:19:56.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.133+0000 7f2e58ff9700 1 --2- 172.21.15.6:0/2148661854 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2e4406e840 0x7f2e44070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:56.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.134+0000 7f2e5bfff700 1 --2- 172.21.15.6:0/2148661854 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2e4406e840 0x7f2e44070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:56.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.134+0000 7f2e58ff9700 1 -- 172.21.15.6:0/2148661854 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f2e5008e400 con 0x7f2e5c0fcf80 2024-03-22T06:19:56.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.134+0000 7f2e5bfff700 1 --2- 172.21.15.6:0/2148661854 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2e4406e840 0x7f2e44070cf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c004890 tx=0x7f2e4c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:56.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.138+0000 7f2e58ff9700 1 -- 172.21.15.6:0/2148661854 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f2e5005aed0 con 0x7f2e5c0fcf80 2024-03-22T06:19:56.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.363+0000 7f2e623e9700 1 -- 172.21.15.6:0/2148661854 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e5c101290 con 0x7f2e4406e840 2024-03-22T06:19:56.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.364+0000 7f2e58ff9700 1 -- 172.21.15.6:0/2148661854 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2e5c101290 con 0x7f2e4406e840 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "8/21 daemons upgraded", 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:19:56.365 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:19:56.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.367+0000 7f2e623e9700 1 -- 172.21.15.6:0/2148661854 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2e4406e840 msgr2=0x7f2e44070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:56.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.367+0000 7f2e623e9700 1 --2- 172.21.15.6:0/2148661854 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2e4406e840 0x7f2e44070cf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c004890 tx=0x7f2e4c009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:56.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.367+0000 7f2e623e9700 1 -- 172.21.15.6:0/2148661854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e5c0fcf80 msgr2=0x7f2e5c112710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:56.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.367+0000 7f2e623e9700 1 --2- 172.21.15.6:0/2148661854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e5c0fcf80 0x7f2e5c112710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000c830 tx=0x7f2e50004470 comp rx=0 tx=0).stop 2024-03-22T06:19:56.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.367+0000 7f2e623e9700 1 -- 172.21.15.6:0/2148661854 shutdown_connections 2024-03-22T06:19:56.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.367+0000 7f2e623e9700 1 --2- 172.21.15.6:0/2148661854 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2e4406e840 0x7f2e44070cf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.367+0000 7f2e623e9700 1 --2- 172.21.15.6:0/2148661854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c0fbcd0 0x7f2e5c066cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.367+0000 7f2e623e9700 1 --2- 172.21.15.6:0/2148661854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e5c0fcf80 0x7f2e5c112710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.367+0000 7f2e623e9700 1 -- 172.21.15.6:0/2148661854 >> 172.21.15.6:0/2148661854 conn(0x7f2e5c0f7450 msgr2=0x7f2e5c0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:56.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.367+0000 7f2e623e9700 1 -- 172.21.15.6:0/2148661854 shutdown_connections 2024-03-22T06:19:56.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.367+0000 7f2e623e9700 1 -- 172.21.15.6:0/2148661854 wait complete. 2024-03-22T06:19:56.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.469+0000 7f77d9edb700 1 -- 172.21.15.6:0/1352612642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77d40fce70 msgr2=0x7f77d40fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:56.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.469+0000 7f77d9edb700 1 --2- 172.21.15.6:0/1352612642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77d40fce70 0x7f77d40fd280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f77c4004940 tx=0x7f77c4009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:56.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.469+0000 7f77d9edb700 1 -- 172.21.15.6:0/1352612642 shutdown_connections 2024-03-22T06:19:56.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.469+0000 7f77d9edb700 1 --2- 172.21.15.6:0/1352612642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d40fdfe0 0x7f77d40fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.469+0000 7f77d9edb700 1 --2- 172.21.15.6:0/1352612642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77d40fce70 0x7f77d40fd280 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.469+0000 7f77d9edb700 1 -- 172.21.15.6:0/1352612642 >> 172.21.15.6:0/1352612642 conn(0x7f77d40f8550 msgr2=0x7f77d40fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:56.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.469+0000 7f77d9edb700 1 -- 172.21.15.6:0/1352612642 shutdown_connections 2024-03-22T06:19:56.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.469+0000 7f77d9edb700 1 -- 172.21.15.6:0/1352612642 wait complete. 2024-03-22T06:19:56.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77d9edb700 1 Processor -- start 2024-03-22T06:19:56.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77d9edb700 1 -- start start 2024-03-22T06:19:56.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77d9edb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d40fce70 0x7f77d410fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:56.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77d9edb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77d40fdfe0 0x7f77d4110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:56.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77d9edb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d4110680 con 0x7f77d40fdfe0 2024-03-22T06:19:56.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77d9edb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d41107f0 con 0x7f77d40fce70 2024-03-22T06:19:56.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77caffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77d40fdfe0 0x7f77d4110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77caffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77d40fdfe0 0x7f77d4110080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35526/0 (socket says 172.21.15.6:35526) 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77caffd700 1 -- 172.21.15.6:0/2380215331 learned_addr learned my addr 172.21.15.6:0/2380215331 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77d37fe700 1 --2- 172.21.15.6:0/2380215331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d40fce70 0x7f77d410fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77caffd700 1 -- 172.21.15.6:0/2380215331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d40fce70 msgr2=0x7f77d410fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77caffd700 1 --2- 172.21.15.6:0/2380215331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d40fce70 0x7f77d410fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.470+0000 7f77caffd700 1 -- 172.21.15.6:0/2380215331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c4005040 con 0x7f77d40fdfe0 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.471+0000 7f77caffd700 1 --2- 172.21.15.6:0/2380215331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77d40fdfe0 0x7f77d4110080 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f77bc002a50 tx=0x7f77bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.471+0000 7f77d0ff9700 1 -- 172.21.15.6:0/2380215331 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77bc007cc0 con 0x7f77d40fdfe0 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.471+0000 7f77d9edb700 1 -- 172.21.15.6:0/2380215331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d4115250 con 0x7f77d40fdfe0 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.471+0000 7f77d0ff9700 1 -- 172.21.15.6:0/2380215331 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77bc00f040 con 0x7f77d40fdfe0 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.471+0000 7f77d0ff9700 1 -- 172.21.15.6:0/2380215331 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77bc009640 con 0x7f77d40fdfe0 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.471+0000 7f77d9edb700 1 -- 172.21.15.6:0/2380215331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d4115810 con 0x7f77d40fdfe0 2024-03-22T06:19:56.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.472+0000 7f77d9edb700 1 -- 172.21.15.6:0/2380215331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d4101970 con 0x7f77d40fdfe0 2024-03-22T06:19:56.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.472+0000 7f77d0ff9700 1 -- 172.21.15.6:0/2380215331 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f77bc0097a0 con 0x7f77d40fdfe0 2024-03-22T06:19:56.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.473+0000 7f77d0ff9700 1 --2- 172.21.15.6:0/2380215331 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f77c006e840 0x7f77c0070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:19:56.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.473+0000 7f77d37fe700 1 --2- 172.21.15.6:0/2380215331 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f77c006e840 0x7f77c0070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:19:56.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.473+0000 7f77d0ff9700 1 -- 172.21.15.6:0/2380215331 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f77bc08e260 con 0x7f77d40fdfe0 2024-03-22T06:19:56.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.473+0000 7f77d37fe700 1 --2- 172.21.15.6:0/2380215331 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f77c006e840 0x7f77c0070cf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77c400b3b0 tx=0x7f77c4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:19:56.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.478+0000 7f77d0ff9700 1 -- 172.21.15.6:0/2380215331 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f77bc05acb0 con 0x7f77d40fdfe0 2024-03-22T06:19:56.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.773+0000 7f77d9edb700 1 -- 172.21.15.6:0/2380215331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f77d41058d0 con 0x7f77d40fdfe0 2024-03-22T06:19:56.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.773+0000 7f77d0ff9700 1 -- 172.21.15.6:0/2380215331 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+157 (secure 0 0 0) 0x7f77bc05a840 con 0x7f77d40fdfe0 2024-03-22T06:19:56.774 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:19:56.774 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:19:56.774 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:19:56.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.776+0000 7f77d9edb700 1 -- 172.21.15.6:0/2380215331 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f77c006e840 msgr2=0x7f77c0070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:56.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.776+0000 7f77d9edb700 1 --2- 172.21.15.6:0/2380215331 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f77c006e840 0x7f77c0070cf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77c400b3b0 tx=0x7f77c4009e30 comp rx=0 tx=0).stop 2024-03-22T06:19:56.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.776+0000 7f77d9edb700 1 -- 172.21.15.6:0/2380215331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77d40fdfe0 msgr2=0x7f77d4110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:19:56.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.776+0000 7f77d9edb700 1 --2- 172.21.15.6:0/2380215331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77d40fdfe0 0x7f77d4110080 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f77bc002a50 tx=0x7f77bc0046b0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.776+0000 7f77d9edb700 1 -- 172.21.15.6:0/2380215331 shutdown_connections 2024-03-22T06:19:56.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.776+0000 7f77d9edb700 1 --2- 172.21.15.6:0/2380215331 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f77c006e840 0x7f77c0070cf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.776+0000 7f77d9edb700 1 --2- 172.21.15.6:0/2380215331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d40fce70 0x7f77d410fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.776+0000 7f77d9edb700 1 --2- 172.21.15.6:0/2380215331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77d40fdfe0 0x7f77d4110080 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:19:56.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.776+0000 7f77d9edb700 1 -- 172.21.15.6:0/2380215331 >> 172.21.15.6:0/2380215331 conn(0x7f77d40f8550 msgr2=0x7f77d40f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:19:56.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.776+0000 7f77d9edb700 1 -- 172.21.15.6:0/2380215331 shutdown_connections 2024-03-22T06:19:56.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:19:56.776+0000 7f77d9edb700 1 -- 172.21.15.6:0/2380215331 wait complete. 2024-03-22T06:19:57.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:56 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2549582680' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:19:57.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:56 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2549582680' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:19:58.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:57 smithi117 ceph-mon[82126]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:58.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:57 smithi117 ceph-mon[82126]: pgmap v101: 65 pgs: 1 peering, 64 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 5.3 MiB/s, 45 keys/s, 16 objects/s recovering 2024-03-22T06:19:58.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:57 smithi117 ceph-mon[82126]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-22T06:19:58.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:57 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2380215331' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:19:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:57 smithi006 ceph-mon[94482]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:19:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:57 smithi006 ceph-mon[94482]: pgmap v101: 65 pgs: 1 peering, 64 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 5.3 MiB/s, 45 keys/s, 16 objects/s recovering 2024-03-22T06:19:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:57 smithi006 ceph-mon[94482]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-22T06:19:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:57 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2380215331' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:20:00.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:19:59 smithi117 ceph-mon[82126]: pgmap v102: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 4.1 MiB/s, 35 keys/s, 12 objects/s recovering 2024-03-22T06:20:00.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:19:59 smithi006 ceph-mon[94482]: pgmap v102: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 4.1 MiB/s, 35 keys/s, 12 objects/s recovering 2024-03-22T06:20:01.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:00 smithi117 ceph-mon[82126]: overall HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:20:01.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:00 smithi006 ceph-mon[94482]: overall HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:20:02.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:01 smithi117 ceph-mon[82126]: pgmap v103: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s, 30 keys/s, 85 objects/s recovering 2024-03-22T06:20:02.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:01 smithi006 ceph-mon[94482]: pgmap v103: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s, 30 keys/s, 85 objects/s recovering 2024-03-22T06:20:03.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:03 smithi006 ceph-mon[94482]: pgmap v104: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s, 27 keys/s, 8 objects/s recovering 2024-03-22T06:20:04.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:03 smithi117 ceph-mon[82126]: pgmap v104: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s, 27 keys/s, 8 objects/s recovering 2024-03-22T06:20:06.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:05 smithi117 ceph-mon[82126]: pgmap v105: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s, 22 keys/s, 6 objects/s recovering 2024-03-22T06:20:06.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:05 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T06:20:06.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:05 smithi006 ceph-mon[94482]: pgmap v105: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s, 22 keys/s, 6 objects/s recovering 2024-03-22T06:20:06.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:05 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T06:20:07.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:06 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T06:20:07.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:06 smithi117 ceph-mon[82126]: Upgrade: osd.2 is safe to restart 2024-03-22T06:20:07.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:06 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T06:20:07.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:06 smithi006 ceph-mon[94482]: Upgrade: osd.2 is safe to restart 2024-03-22T06:20:07.853 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:07 smithi006 ceph-mon[94482]: pgmap v106: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:07.854 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:07.854 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:20:07.854 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:07.854 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:07.854 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T06:20:07.854 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:20:08.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:07 smithi117 ceph-mon[82126]: pgmap v106: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:08.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:08.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:20:08.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:08.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:08.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T06:20:08.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:20:09.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:08 smithi006 ceph-mon[94482]: Upgrade: Updating osd.2 2024-03-22T06:20:09.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:08 smithi006 ceph-mon[94482]: Deploying daemon osd.2 on smithi006 2024-03-22T06:20:09.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:08 smithi117 ceph-mon[82126]: Upgrade: Updating osd.2 2024-03-22T06:20:09.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:08 smithi117 ceph-mon[82126]: Deploying daemon osd.2 on smithi006 2024-03-22T06:20:09.966 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:09 smithi006 ceph-mon[94482]: pgmap v107: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:10.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:09 smithi117 ceph-mon[82126]: pgmap v107: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:11.728 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:11 smithi006 systemd[1]: Stopping Ceph osd.2 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:20:12.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:11 smithi006 ceph-mon[94482]: pgmap v108: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:12.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:11 smithi006 ceph-mon[94482]: osd.2 marked itself down and dead 2024-03-22T06:20:12.032 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:11 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[63182]: 2024-03-22T06:20:11.727+0000 7fce18d35700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:20:12.032 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:11 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[63182]: 2024-03-22T06:20:11.727+0000 7fce18d35700 -1 osd.2 71 *** Got signal Terminated *** 2024-03-22T06:20:12.033 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:11 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[63182]: 2024-03-22T06:20:11.727+0000 7fce18d35700 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:20:12.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:11 smithi117 ceph-mon[82126]: pgmap v108: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:12.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:11 smithi117 ceph-mon[82126]: osd.2 marked itself down and dead 2024-03-22T06:20:12.496 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:12 smithi006 podman[115727]: 2024-03-22 06:20:12.167599602 +0000 UTC m=+0.580315451 container died 7bdf9fdb63190692eb15a943cb366aefb34e67488b0172b7e6127bc42e3d24b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=quincy-9c4e0c3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-03-22T06:20:12.766 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:12 smithi006 podman[115727]: 2024-03-22 06:20:12.661601676 +0000 UTC m=+1.074317524 container cleanup 7bdf9fdb63190692eb15a943cb366aefb34e67488b0172b7e6127bc42e3d24b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T06:20:12.766 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:12 smithi006 bash[115727]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2 2024-03-22T06:20:13.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:12 smithi006 ceph-mon[94482]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:20:13.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:12 smithi006 ceph-mon[94482]: osdmap e72: 6 total, 5 up, 6 in 2024-03-22T06:20:13.032 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:12 smithi006 podman[115745]: 2024-03-22 06:20:12.947071649 +0000 UTC m=+0.772333613 container remove 7bdf9fdb63190692eb15a943cb366aefb34e67488b0172b7e6127bc42e3d24b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=quincy-9c4e0c3, org.label-schema.vendor=CentOS) 2024-03-22T06:20:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:12 smithi117 ceph-mon[82126]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:20:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:12 smithi117 ceph-mon[82126]: osdmap e72: 6 total, 5 up, 6 in 2024-03-22T06:20:13.576 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:13 smithi006 podman[115836]: 2024-03-22 06:20:13.202169976 +0000 UTC m=+0.018974602 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:20:13.576 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:13 smithi006 podman[115836]: 2024-03-22 06:20:13.306332901 +0000 UTC m=+0.123137523 container create eae880dfc1e3f71d76f25a2b78f9daf4c017cb631651d1e5df8f8a3d34f62efd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-22T06:20:13.883 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:13 smithi006 ceph-mon[94482]: pgmap v110: 65 pgs: 7 peering, 7 stale+active+clean, 51 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:13.883 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:13 smithi006 ceph-mon[94482]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-22T06:20:13.883 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:13 smithi006 ceph-mon[94482]: osdmap e73: 6 total, 5 up, 6 in 2024-03-22T06:20:13.883 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:13 smithi006 podman[115836]: 2024-03-22 06:20:13.576347944 +0000 UTC m=+0.393152573 container init eae880dfc1e3f71d76f25a2b78f9daf4c017cb631651d1e5df8f8a3d34f62efd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:20:13.883 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:13 smithi006 podman[115836]: 2024-03-22 06:20:13.579882047 +0000 UTC m=+0.396686670 container start eae880dfc1e3f71d76f25a2b78f9daf4c017cb631651d1e5df8f8a3d34f62efd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-22T06:20:13.884 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:13 smithi006 podman[115836]: 2024-03-22 06:20:13.650927756 +0000 UTC m=+0.467732376 container attach eae880dfc1e3f71d76f25a2b78f9daf4c017cb631651d1e5df8f8a3d34f62efd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-22T06:20:14.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:13 smithi117 ceph-mon[82126]: pgmap v110: 65 pgs: 7 peering, 7 stale+active+clean, 51 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:13 smithi117 ceph-mon[82126]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-22T06:20:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:13 smithi117 ceph-mon[82126]: osdmap e73: 6 total, 5 up, 6 in 2024-03-22T06:20:14.282 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:13 smithi006 podman[115836]: 2024-03-22 06:20:13.883840817 +0000 UTC m=+0.700645449 container died eae880dfc1e3f71d76f25a2b78f9daf4c017cb631651d1e5df8f8a3d34f62efd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-22T06:20:14.561 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:14 smithi006 podman[115862]: 2024-03-22 06:20:14.545282303 +0000 UTC m=+0.656144840 container remove eae880dfc1e3f71d76f25a2b78f9daf4c017cb631651d1e5df8f8a3d34f62efd (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:20:14.820 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:14 smithi006 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-22T06:20:14.820 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:14 smithi006 systemd[1]: Stopped Ceph osd.2 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:20:14.820 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:14 smithi006 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.2.service: Consumed 55.136s CPU time. 2024-03-22T06:20:14.820 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:14 smithi006 systemd[1]: Starting Ceph osd.2 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:20:15.105 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:14 smithi006 ceph-mon[94482]: Health check failed: Degraded data redundancy: 11/219 objects degraded (5.023%), 5 pgs degraded (PG_DEGRADED) 2024-03-22T06:20:15.105 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:14 smithi006 podman[115963]: 2024-03-22 06:20:14.998197854 +0000 UTC m=+0.115347610 container create 1ac18554c2634524f887880dbad287bc219328f0f463db6d26dab92d3208df7a (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-22T06:20:15.105 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:14 smithi006 podman[115963]: 2024-03-22 06:20:14.901489483 +0000 UTC m=+0.018639196 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:20:15.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:14 smithi117 ceph-mon[82126]: Health check failed: Degraded data redundancy: 11/219 objects degraded (5.023%), 5 pgs degraded (PG_DEGRADED) 2024-03-22T06:20:15.532 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 podman[115963]: 2024-03-22 06:20:15.21785663 +0000 UTC m=+0.335006339 container init 1ac18554c2634524f887880dbad287bc219328f0f463db6d26dab92d3208df7a (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True) 2024-03-22T06:20:15.532 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 podman[115963]: 2024-03-22 06:20:15.221406085 +0000 UTC m=+0.338555788 container start 1ac18554c2634524f887880dbad287bc219328f0f463db6d26dab92d3208df7a (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:20:15.532 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 podman[115963]: 2024-03-22 06:20:15.292433118 +0000 UTC m=+0.409582818 container attach 1ac18554c2634524f887880dbad287bc219328f0f463db6d26dab92d3208df7a (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:20:16.013 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:15 smithi006 ceph-mon[94482]: pgmap v112: 65 pgs: 8 active+undersized, 5 active+undersized+degraded, 7 peering, 5 stale+active+clean, 40 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 11/219 objects degraded (5.023%) 2024-03-22T06:20:16.013 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate[115977]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T06:20:16.013 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate[115977]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T06:20:16.013 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 bash[115963]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T06:20:16.013 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 bash[115963]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T06:20:16.013 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 bash[115963]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T06:20:16.013 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 bash[115963]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T06:20:16.013 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 bash[115963]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-22T06:20:16.013 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 bash[115963]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T06:20:16.014 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 bash[115963]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-22T06:20:16.014 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate[115977]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T06:20:16.014 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate[115977]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T06:20:16.014 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate[115977]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-22T06:20:16.014 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate[115977]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T06:20:16.014 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate[115977]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-22T06:20:16.014 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:15 smithi006 podman[115963]: 2024-03-22 06:20:15.691453784 +0000 UTC m=+0.808603483 container died 1ac18554c2634524f887880dbad287bc219328f0f463db6d26dab92d3208df7a (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:20:16.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:15 smithi117 ceph-mon[82126]: pgmap v112: 65 pgs: 8 active+undersized, 5 active+undersized+degraded, 7 peering, 5 stale+active+clean, 40 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 11/219 objects degraded (5.023%) 2024-03-22T06:20:16.665 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:16 smithi006 podman[116100]: 2024-03-22 06:20:16.413976601 +0000 UTC m=+0.712818752 container remove 1ac18554c2634524f887880dbad287bc219328f0f463db6d26dab92d3208df7a (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-22T06:20:16.665 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:16 smithi006 podman[116163]: 2024-03-22 06:20:16.537531872 +0000 UTC m=+0.018450520 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:20:16.665 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:16 smithi006 podman[116163]: 2024-03-22 06:20:16.665187865 +0000 UTC m=+0.146106528 container create 3ec1f219c19c1d692b154426a32ffc08a0b0336fba6196e9348a023264d3307b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0) 2024-03-22T06:20:17.282 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:16 smithi006 podman[116163]: 2024-03-22 06:20:16.91808833 +0000 UTC m=+0.399006968 container init 3ec1f219c19c1d692b154426a32ffc08a0b0336fba6196e9348a023264d3307b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:20:17.282 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:16 smithi006 podman[116163]: 2024-03-22 06:20:16.921390326 +0000 UTC m=+0.402308947 container start 3ec1f219c19c1d692b154426a32ffc08a0b0336fba6196e9348a023264d3307b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-03-22T06:20:17.282 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:16 smithi006 ceph-osd[116182]: -- 172.21.15.6:0/3551398281 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55da7ca57680 con 0x55da7ca20c00 2024-03-22T06:20:17.282 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:17 smithi006 bash[116163]: 3ec1f219c19c1d692b154426a32ffc08a0b0336fba6196e9348a023264d3307b 2024-03-22T06:20:17.282 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:17 smithi006 systemd[1]: Started Ceph osd.2 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:20:17.782 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:17 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[116178]: 2024-03-22T06:20:17.462+0000 7f1ca979e7c0 -1 Falling back to public interface 2024-03-22T06:20:18.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:18 smithi006 ceph-mon[94482]: pgmap v113: 65 pgs: 13 active+undersized, 10 active+undersized+degraded, 7 peering, 35 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 28/219 objects degraded (12.785%) 2024-03-22T06:20:18.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:18 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:18.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:18 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:18.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:18 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:20:18.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:18 smithi117 ceph-mon[82126]: pgmap v113: 65 pgs: 13 active+undersized, 10 active+undersized+degraded, 7 peering, 35 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 28/219 objects degraded (12.785%) 2024-03-22T06:20:18.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:18 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:18.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:18 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:18.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:18 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:20:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:19 smithi006 ceph-mon[94482]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-22T06:20:19.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:19 smithi117 ceph-mon[82126]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-22T06:20:20.424 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:20 smithi006 ceph-mon[94482]: pgmap v114: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 33/219 objects degraded (15.068%) 2024-03-22T06:20:20.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:20 smithi117 ceph-mon[82126]: pgmap v114: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 33/219 objects degraded (15.068%) 2024-03-22T06:20:22.008 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:21 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[116178]: 2024-03-22T06:20:21.695+0000 7f1ca979e7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-22T06:20:22.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:22 smithi006 ceph-mon[94482]: pgmap v115: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 33/219 objects degraded (15.068%) 2024-03-22T06:20:22.282 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:22 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[116178]: 2024-03-22T06:20:22.249+0000 7f1ca979e7c0 -1 osd.2 71 log_to_monitors true 2024-03-22T06:20:22.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:22 smithi117 ceph-mon[82126]: pgmap v115: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 33/219 objects degraded (15.068%) 2024-03-22T06:20:23.190 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:20:23 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[116178]: 2024-03-22T06:20:23.146+0000 7f1c9abf6700 -1 osd.2 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:20:23.190 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:23 smithi006 ceph-mon[94482]: from='osd.2 [v2:172.21.15.6:6818/195584623,v1:172.21.15.6:6819/195584623]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T06:20:23.190 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:23.190 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:20:23.190 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:23 smithi117 ceph-mon[82126]: from='osd.2 [v2:172.21.15.6:6818/195584623,v1:172.21.15.6:6819/195584623]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T06:20:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:20:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:24.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:24 smithi117 ceph-mon[82126]: pgmap v116: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 33/219 objects degraded (15.068%) 2024-03-22T06:20:24.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:24 smithi117 ceph-mon[82126]: from='osd.2 [v2:172.21.15.6:6818/195584623,v1:172.21.15.6:6819/195584623]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T06:20:24.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:24 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:24.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:24 smithi117 ceph-mon[82126]: osdmap e74: 6 total, 5 up, 6 in 2024-03-22T06:20:24.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:24 smithi117 ceph-mon[82126]: from='osd.2 [v2:172.21.15.6:6818/195584623,v1:172.21.15.6:6819/195584623]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:20:24.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:24 smithi117 ceph-mon[82126]: Health check update: Degraded data redundancy: 33/219 objects degraded (15.068%), 12 pgs degraded (PG_DEGRADED) 2024-03-22T06:20:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:24 smithi006 ceph-mon[94482]: pgmap v116: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 33/219 objects degraded (15.068%) 2024-03-22T06:20:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:24 smithi006 ceph-mon[94482]: from='osd.2 [v2:172.21.15.6:6818/195584623,v1:172.21.15.6:6819/195584623]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T06:20:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:24 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:24 smithi006 ceph-mon[94482]: osdmap e74: 6 total, 5 up, 6 in 2024-03-22T06:20:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:24 smithi006 ceph-mon[94482]: from='osd.2 [v2:172.21.15.6:6818/195584623,v1:172.21.15.6:6819/195584623]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-22T06:20:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:24 smithi006 ceph-mon[94482]: Health check update: Degraded data redundancy: 33/219 objects degraded (15.068%), 12 pgs degraded (PG_DEGRADED) 2024-03-22T06:20:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:25 smithi006 ceph-mon[94482]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:20:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:25 smithi006 ceph-mon[94482]: osd.2 [v2:172.21.15.6:6818/195584623,v1:172.21.15.6:6819/195584623] boot 2024-03-22T06:20:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:25 smithi006 ceph-mon[94482]: osdmap e75: 6 total, 6 up, 6 in 2024-03-22T06:20:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:20:25.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:25.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:25.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:25 smithi006 ceph-mon[94482]: osdmap e76: 6 total, 6 up, 6 in 2024-03-22T06:20:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:25 smithi117 ceph-mon[82126]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:20:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:25 smithi117 ceph-mon[82126]: osd.2 [v2:172.21.15.6:6818/195584623,v1:172.21.15.6:6819/195584623] boot 2024-03-22T06:20:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:25 smithi117 ceph-mon[82126]: osdmap e75: 6 total, 6 up, 6 in 2024-03-22T06:20:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T06:20:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:25.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:25 smithi117 ceph-mon[82126]: osdmap e76: 6 total, 6 up, 6 in 2024-03-22T06:20:26.274 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:26 smithi006 ceph-mon[94482]: pgmap v119: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 33/219 objects degraded (15.068%) 2024-03-22T06:20:26.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:26 smithi117 ceph-mon[82126]: pgmap v119: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 33/219 objects degraded (15.068%) 2024-03-22T06:20:26.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.882+0000 7fecb8be0700 1 -- 172.21.15.6:0/2970026067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecb40fb480 msgr2=0x7fecb40fb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:26.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.882+0000 7fecb8be0700 1 --2- 172.21.15.6:0/2970026067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecb40fb480 0x7fecb40fb890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feca8004a50 tx=0x7feca800af40 comp rx=0 tx=0).stop 2024-03-22T06:20:26.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.882+0000 7fecb8be0700 1 -- 172.21.15.6:0/2970026067 shutdown_connections 2024-03-22T06:20:26.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.882+0000 7fecb8be0700 1 --2- 172.21.15.6:0/2970026067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecb40fc730 0x7fecb40fcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:26.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.882+0000 7fecb8be0700 1 --2- 172.21.15.6:0/2970026067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecb40fb480 0x7fecb40fb890 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:26.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.882+0000 7fecb8be0700 1 -- 172.21.15.6:0/2970026067 >> 172.21.15.6:0/2970026067 conn(0x7fecb40f6be0 msgr2=0x7fecb40f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:26.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.882+0000 7fecb8be0700 1 -- 172.21.15.6:0/2970026067 shutdown_connections 2024-03-22T06:20:26.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.882+0000 7fecb8be0700 1 -- 172.21.15.6:0/2970026067 wait complete. 2024-03-22T06:20:26.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.883+0000 7fecb8be0700 1 Processor -- start 2024-03-22T06:20:26.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.883+0000 7fecb8be0700 1 -- start start 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.883+0000 7fecb8be0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecb40fb480 0x7fecb410b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.883+0000 7fecb8be0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecb40fc730 0x7fecb410bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.883+0000 7fecb8be0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb410c2f0 con 0x7fecb40fc730 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.883+0000 7fecb8be0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb4110c90 con 0x7fecb40fb480 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.883+0000 7fecb1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecb40fc730 0x7fecb410bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.883+0000 7fecb1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecb40fc730 0x7fecb410bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37396/0 (socket says 172.21.15.6:37396) 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.883+0000 7fecb1d9b700 1 -- 172.21.15.6:0/35146526 learned_addr learned my addr 172.21.15.6:0/35146526 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.884+0000 7fecb259c700 1 --2- 172.21.15.6:0/35146526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecb40fb480 0x7fecb410b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.884+0000 7fecb1d9b700 1 -- 172.21.15.6:0/35146526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecb40fb480 msgr2=0x7fecb410b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.884+0000 7fecb1d9b700 1 --2- 172.21.15.6:0/35146526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecb40fb480 0x7fecb410b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.884+0000 7fecb1d9b700 1 -- 172.21.15.6:0/35146526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca8005040 con 0x7fecb40fc730 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.884+0000 7fecb1d9b700 1 --2- 172.21.15.6:0/35146526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecb40fc730 0x7fecb410bc60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feca400b790 tx=0x7feca4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:26.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.884+0000 7feca2ffd700 1 -- 172.21.15.6:0/35146526 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feca400f6f0 con 0x7fecb40fc730 2024-03-22T06:20:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.884+0000 7fecb8be0700 1 -- 172.21.15.6:0/35146526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb4113100 con 0x7fecb40fc730 2024-03-22T06:20:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.884+0000 7feca2ffd700 1 -- 172.21.15.6:0/35146526 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feca400fd20 con 0x7fecb40fc730 2024-03-22T06:20:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.884+0000 7fecb8be0700 1 -- 172.21.15.6:0/35146526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb4113640 con 0x7fecb40fc730 2024-03-22T06:20:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.884+0000 7feca2ffd700 1 -- 172.21.15.6:0/35146526 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feca40103a0 con 0x7fecb40fc730 2024-03-22T06:20:26.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.885+0000 7fecb8be0700 1 -- 172.21.15.6:0/35146526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb404c710 con 0x7fecb40fc730 2024-03-22T06:20:26.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.886+0000 7feca2ffd700 1 -- 172.21.15.6:0/35146526 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7feca4011400 con 0x7fecb40fc730 2024-03-22T06:20:26.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.887+0000 7feca2ffd700 1 --2- 172.21.15.6:0/35146526 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec9c06e840 0x7fec9c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:26.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.887+0000 7feca2ffd700 1 -- 172.21.15.6:0/35146526 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6614+0+0 (secure 0 0 0) 0x7feca408e150 con 0x7fecb40fc730 2024-03-22T06:20:26.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.887+0000 7fecb259c700 1 --2- 172.21.15.6:0/35146526 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec9c06e840 0x7fec9c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:26.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.887+0000 7fecb259c700 1 --2- 172.21.15.6:0/35146526 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec9c06e840 0x7fec9c070cf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feca800c3b0 tx=0x7feca800af40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:26.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:26.892+0000 7feca2ffd700 1 -- 172.21.15.6:0/35146526 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7feca405acc0 con 0x7fecb40fc730 2024-03-22T06:20:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.117+0000 7fecb8be0700 1 -- 172.21.15.6:0/35146526 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fecb41009d0 con 0x7fec9c06e840 2024-03-22T06:20:27.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.119+0000 7feca2ffd700 1 -- 172.21.15.6:0/35146526 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fecb41009d0 con 0x7fec9c06e840 2024-03-22T06:20:27.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.121+0000 7fecb8be0700 1 -- 172.21.15.6:0/35146526 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec9c06e840 msgr2=0x7fec9c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.121+0000 7fecb8be0700 1 --2- 172.21.15.6:0/35146526 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec9c06e840 0x7fec9c070cf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feca800c3b0 tx=0x7feca800af40 comp rx=0 tx=0).stop 2024-03-22T06:20:27.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.121+0000 7fecb8be0700 1 -- 172.21.15.6:0/35146526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecb40fc730 msgr2=0x7fecb410bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.121+0000 7fecb8be0700 1 --2- 172.21.15.6:0/35146526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecb40fc730 0x7fecb410bc60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feca400b790 tx=0x7feca4009250 comp rx=0 tx=0).stop 2024-03-22T06:20:27.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.121+0000 7fecb8be0700 1 -- 172.21.15.6:0/35146526 shutdown_connections 2024-03-22T06:20:27.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.121+0000 7fecb8be0700 1 --2- 172.21.15.6:0/35146526 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fec9c06e840 0x7fec9c070cf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.121+0000 7fecb8be0700 1 --2- 172.21.15.6:0/35146526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecb40fb480 0x7fecb410b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.121+0000 7fecb8be0700 1 --2- 172.21.15.6:0/35146526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fecb40fc730 0x7fecb410bc60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.121+0000 7fecb8be0700 1 -- 172.21.15.6:0/35146526 >> 172.21.15.6:0/35146526 conn(0x7fecb40f6be0 msgr2=0x7fecb40f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:27.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.121+0000 7fecb8be0700 1 -- 172.21.15.6:0/35146526 shutdown_connections 2024-03-22T06:20:27.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.121+0000 7fecb8be0700 1 -- 172.21.15.6:0/35146526 wait complete. 2024-03-22T06:20:27.132 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:20:27.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.251+0000 7f3021a55700 1 -- 172.21.15.6:0/3851188059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f301c0fff00 msgr2=0x7f301c102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.251+0000 7f3021a55700 1 --2- 172.21.15.6:0/3851188059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f301c0fff00 0x7f301c102320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3010004940 tx=0x7f3010009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:27.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.251+0000 7f3021a55700 1 -- 172.21.15.6:0/3851188059 shutdown_connections 2024-03-22T06:20:27.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.251+0000 7f3021a55700 1 --2- 172.21.15.6:0/3851188059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f301c0fff00 0x7f301c102320 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.251+0000 7f3021a55700 1 --2- 172.21.15.6:0/3851188059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f301c0662e0 0x7f301c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.251+0000 7f3021a55700 1 -- 172.21.15.6:0/3851188059 >> 172.21.15.6:0/3851188059 conn(0x7f301c0f7430 msgr2=0x7f301c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.252+0000 7f3021a55700 1 -- 172.21.15.6:0/3851188059 shutdown_connections 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.252+0000 7f3021a55700 1 -- 172.21.15.6:0/3851188059 wait complete. 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.252+0000 7f3021a55700 1 Processor -- start 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.252+0000 7f3021a55700 1 -- start start 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.253+0000 7f3021a55700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f301c0662e0 0x7f301c10da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.253+0000 7f3021a55700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f301c0fff00 0x7f301c10dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.253+0000 7f3021a55700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c10e5b0 con 0x7f301c0662e0 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.253+0000 7f3021a55700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c10ee00 con 0x7f301c0fff00 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.253+0000 7f301affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f301c0662e0 0x7f301c10da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.253+0000 7f301affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f301c0662e0 0x7f301c10da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37418/0 (socket says 172.21.15.6:37418) 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.253+0000 7f301affd700 1 -- 172.21.15.6:0/138887395 learned_addr learned my addr 172.21.15.6:0/138887395 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:20:27.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.253+0000 7f301a7fc700 1 --2- 172.21.15.6:0/138887395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f301c0fff00 0x7f301c10dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:27.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.253+0000 7f301affd700 1 -- 172.21.15.6:0/138887395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f301c0fff00 msgr2=0x7f301c10dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.253+0000 7f301affd700 1 --2- 172.21.15.6:0/138887395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f301c0fff00 0x7f301c10dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.253+0000 7f301affd700 1 -- 172.21.15.6:0/138887395 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3010005040 con 0x7f301c0662e0 2024-03-22T06:20:27.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.254+0000 7f301affd700 1 --2- 172.21.15.6:0/138887395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f301c0662e0 0x7f301c10da70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f300c00c7e0 tx=0x7f300c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:27.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.254+0000 7f30037fe700 1 -- 172.21.15.6:0/138887395 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f300c012810 con 0x7f301c0662e0 2024-03-22T06:20:27.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.254+0000 7f30037fe700 1 -- 172.21.15.6:0/138887395 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f300c012e40 con 0x7f301c0662e0 2024-03-22T06:20:27.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.254+0000 7f3021a55700 1 -- 172.21.15.6:0/138887395 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301c10f0e0 con 0x7f301c0662e0 2024-03-22T06:20:27.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.254+0000 7f30037fe700 1 -- 172.21.15.6:0/138887395 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f300c00a550 con 0x7f301c0662e0 2024-03-22T06:20:27.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.254+0000 7f3021a55700 1 -- 172.21.15.6:0/138887395 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301c10f620 con 0x7f301c0662e0 2024-03-22T06:20:27.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.255+0000 7f3021a55700 1 -- 172.21.15.6:0/138887395 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301c05f5f0 con 0x7f301c0662e0 2024-03-22T06:20:27.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.255+0000 7f30037fe700 1 -- 172.21.15.6:0/138887395 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f300c011380 con 0x7f301c0662e0 2024-03-22T06:20:27.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.256+0000 7f30037fe700 1 --2- 172.21.15.6:0/138887395 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f300406e850 0x7f3004070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:27.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.256+0000 7f30037fe700 1 -- 172.21.15.6:0/138887395 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f300c08e780 con 0x7f301c0662e0 2024-03-22T06:20:27.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.256+0000 7f301a7fc700 1 --2- 172.21.15.6:0/138887395 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f300406e850 0x7f3004070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:27.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.257+0000 7f301a7fc700 1 --2- 172.21.15.6:0/138887395 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f300406e850 0x7f3004070d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f301000b3b0 tx=0x7f3010009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:27.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.261+0000 7f30037fe700 1 -- 172.21.15.6:0/138887395 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f300c05b250 con 0x7f301c0662e0 2024-03-22T06:20:27.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.500+0000 7f3021a55700 1 -- 172.21.15.6:0/138887395 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f301c10fa00 con 0x7f300406e850 2024-03-22T06:20:27.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.501+0000 7f30037fe700 1 -- 172.21.15.6:0/138887395 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f301c10fa00 con 0x7f300406e850 2024-03-22T06:20:27.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.503+0000 7f3021a55700 1 -- 172.21.15.6:0/138887395 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f300406e850 msgr2=0x7f3004070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.503+0000 7f3021a55700 1 --2- 172.21.15.6:0/138887395 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f300406e850 0x7f3004070d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f301000b3b0 tx=0x7f3010009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:27.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.503+0000 7f3021a55700 1 -- 172.21.15.6:0/138887395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f301c0662e0 msgr2=0x7f301c10da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.503+0000 7f3021a55700 1 --2- 172.21.15.6:0/138887395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f301c0662e0 0x7f301c10da70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f300c00c7e0 tx=0x7f300c007f90 comp rx=0 tx=0).stop 2024-03-22T06:20:27.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.503+0000 7f3021a55700 1 -- 172.21.15.6:0/138887395 shutdown_connections 2024-03-22T06:20:27.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.503+0000 7f3021a55700 1 --2- 172.21.15.6:0/138887395 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f300406e850 0x7f3004070d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.503+0000 7f3021a55700 1 --2- 172.21.15.6:0/138887395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f301c0662e0 0x7f301c10da70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.503+0000 7f3021a55700 1 --2- 172.21.15.6:0/138887395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f301c0fff00 0x7f301c10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.503+0000 7f3021a55700 1 -- 172.21.15.6:0/138887395 >> 172.21.15.6:0/138887395 conn(0x7f301c0f7430 msgr2=0x7f301c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:27.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.504+0000 7f3021a55700 1 -- 172.21.15.6:0/138887395 shutdown_connections 2024-03-22T06:20:27.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.504+0000 7f3021a55700 1 -- 172.21.15.6:0/138887395 wait complete. 2024-03-22T06:20:27.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.607+0000 7f7fc9138700 1 -- 172.21.15.6:0/1925705596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc40fed80 msgr2=0x7f7fc40faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.607+0000 7f7fc9138700 1 --2- 172.21.15.6:0/1925705596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc40fed80 0x7f7fc40faec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7fac004940 tx=0x7f7fac009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:27.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.607+0000 7f7fc9138700 1 -- 172.21.15.6:0/1925705596 shutdown_connections 2024-03-22T06:20:27.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.607+0000 7f7fc9138700 1 --2- 172.21.15.6:0/1925705596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc40fed80 0x7f7fc40faec0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.607+0000 7f7fc9138700 1 --2- 172.21.15.6:0/1925705596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc40fe430 0x7f7fc40fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.607+0000 7f7fc9138700 1 -- 172.21.15.6:0/1925705596 >> 172.21.15.6:0/1925705596 conn(0x7f7fc40f6bd0 msgr2=0x7f7fc40f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:27.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.607+0000 7f7fc9138700 1 -- 172.21.15.6:0/1925705596 shutdown_connections 2024-03-22T06:20:27.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.607+0000 7f7fc9138700 1 -- 172.21.15.6:0/1925705596 wait complete. 2024-03-22T06:20:27.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc9138700 1 Processor -- start 2024-03-22T06:20:27.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc9138700 1 -- start start 2024-03-22T06:20:27.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc9138700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc40fe430 0x7f7fc410fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:27.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc9138700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc40fed80 0x7f7fc4110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:27.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc9138700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc4110680 con 0x7f7fc40fe430 2024-03-22T06:20:27.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc9138700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc41107f0 con 0x7f7fc40fed80 2024-03-22T06:20:27.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc40fe430 0x7f7fc410fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:27.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc40fe430 0x7f7fc410fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37436/0 (socket says 172.21.15.6:37436) 2024-03-22T06:20:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc2d9d700 1 -- 172.21.15.6:0/1685187452 learned_addr learned my addr 172.21.15.6:0/1685187452 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:20:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc259c700 1 --2- 172.21.15.6:0/1685187452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc40fed80 0x7f7fc4110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc2d9d700 1 -- 172.21.15.6:0/1685187452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc40fed80 msgr2=0x7f7fc4110080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc2d9d700 1 --2- 172.21.15.6:0/1685187452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc40fed80 0x7f7fc4110080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.608+0000 7f7fc2d9d700 1 -- 172.21.15.6:0/1685187452 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac005040 con 0x7f7fc40fe430 2024-03-22T06:20:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.609+0000 7f7fc2d9d700 1 --2- 172.21.15.6:0/1685187452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc40fe430 0x7f7fc410fb40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc002a50 tx=0x7f7fbc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.609+0000 7f7fbb7fe700 1 -- 172.21.15.6:0/1685187452 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fbc007cc0 con 0x7f7fc40fe430 2024-03-22T06:20:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.609+0000 7f7fbb7fe700 1 -- 172.21.15.6:0/1685187452 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fbc00f040 con 0x7f7fc40fe430 2024-03-22T06:20:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.609+0000 7f7fc9138700 1 -- 172.21.15.6:0/1685187452 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc4115250 con 0x7f7fc40fe430 2024-03-22T06:20:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.609+0000 7f7fbb7fe700 1 -- 172.21.15.6:0/1685187452 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fbc00da30 con 0x7f7fc40fe430 2024-03-22T06:20:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.609+0000 7f7fc9138700 1 -- 172.21.15.6:0/1685187452 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc4115810 con 0x7f7fc40fe430 2024-03-22T06:20:27.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.610+0000 7f7fc9138700 1 -- 172.21.15.6:0/1685187452 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc4101b70 con 0x7f7fc40fe430 2024-03-22T06:20:27.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.610+0000 7f7fbb7fe700 1 -- 172.21.15.6:0/1685187452 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f7fbc00dbb0 con 0x7f7fc40fe430 2024-03-22T06:20:27.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.611+0000 7f7fbb7fe700 1 --2- 172.21.15.6:0/1685187452 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7fb006e840 0x7f7fb0070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:27.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.611+0000 7f7fbb7fe700 1 -- 172.21.15.6:0/1685187452 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f7fbc08e1e0 con 0x7f7fc40fe430 2024-03-22T06:20:27.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.611+0000 7f7fc259c700 1 --2- 172.21.15.6:0/1685187452 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7fb006e840 0x7f7fb0070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:27.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.611+0000 7f7fc259c700 1 --2- 172.21.15.6:0/1685187452 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7fb006e840 0x7f7fb0070cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009fd0 tx=0x7f7fac009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:27.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.616+0000 7f7fbb7fe700 1 -- 172.21.15.6:0/1685187452 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f7fbc05ac30 con 0x7f7fc40fe430 2024-03-22T06:20:27.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.831+0000 7f7fc9138700 1 -- 172.21.15.6:0/1685187452 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fc402b340 con 0x7f7fb006e840 2024-03-22T06:20:27.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.842+0000 7f7fbb7fe700 1 -- 172.21.15.6:0/1685187452 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f7fc402b340 con 0x7f7fb006e840 2024-03-22T06:20:27.842 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:20:27.842 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (5m) 4s ago 13m 26.2M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:20:27.842 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (115s) 4s ago 13m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:20:27.842 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (109s) 106s ago 12m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:20:27.842 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 unknown 4s ago 13m - - 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (8m) 4s ago 8m 157M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (7m) 4s ago 7m 18.0M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (8m) 106s ago 8m 14.6M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (7m) 106s ago 7m 16.4M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (4m) 4s ago 14m 504M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (3m) 106s ago 12m 450M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (2m) 4s ago 15m 48.0M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (2m) 106s ago 12m 33.5M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (5m) 4s ago 13m 17.6M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (4m) 106s ago 12m 14.9M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (87s) 4s ago 11m 148M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (48s) 4s ago 11m 147M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (11s) 4s ago 10m 12.5M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 3ec1f219c19c 2024-03-22T06:20:27.843 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (10m) 106s ago 10m 284M 2581M 17.2.7-726-g9c4e0c34 837124da804f 3dad19e5cae1 2024-03-22T06:20:27.844 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (10m) 106s ago 10m 250M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:20:27.844 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (9m) 106s ago 9m 237M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:20:27.844 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (3m) 4s ago 13m 64.4M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:20:27.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.844+0000 7f7fc9138700 1 -- 172.21.15.6:0/1685187452 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7fb006e840 msgr2=0x7f7fb0070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.844+0000 7f7fc9138700 1 --2- 172.21.15.6:0/1685187452 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7fb006e840 0x7f7fb0070cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009fd0 tx=0x7f7fac009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:27.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.844+0000 7f7fc9138700 1 -- 172.21.15.6:0/1685187452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc40fe430 msgr2=0x7f7fc410fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.844+0000 7f7fc9138700 1 --2- 172.21.15.6:0/1685187452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc40fe430 0x7f7fc410fb40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc002a50 tx=0x7f7fbc0046b0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.844+0000 7f7fc9138700 1 -- 172.21.15.6:0/1685187452 shutdown_connections 2024-03-22T06:20:27.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.844+0000 7f7fc9138700 1 --2- 172.21.15.6:0/1685187452 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7fb006e840 0x7f7fb0070cf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.844+0000 7f7fc9138700 1 --2- 172.21.15.6:0/1685187452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc40fe430 0x7f7fc410fb40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.844+0000 7f7fc9138700 1 --2- 172.21.15.6:0/1685187452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc40fed80 0x7f7fc4110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.844+0000 7f7fc9138700 1 -- 172.21.15.6:0/1685187452 >> 172.21.15.6:0/1685187452 conn(0x7f7fc40f6bd0 msgr2=0x7f7fc40f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:27.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.844+0000 7f7fc9138700 1 -- 172.21.15.6:0/1685187452 shutdown_connections 2024-03-22T06:20:27.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.844+0000 7f7fc9138700 1 -- 172.21.15.6:0/1685187452 wait complete. 2024-03-22T06:20:27.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.946+0000 7f36004a1700 1 -- 172.21.15.6:0/2752795646 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35f80fbcd0 msgr2=0x7f35f80fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.946+0000 7f36004a1700 1 --2- 172.21.15.6:0/2752795646 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35f80fbcd0 0x7f35f80fc0e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f35f4004940 tx=0x7f35f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:27.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.947+0000 7f36004a1700 1 -- 172.21.15.6:0/2752795646 shutdown_connections 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.947+0000 7f36004a1700 1 --2- 172.21.15.6:0/2752795646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f80fcf80 0x7f35f80fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.947+0000 7f36004a1700 1 --2- 172.21.15.6:0/2752795646 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35f80fbcd0 0x7f35f80fc0e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.947+0000 7f36004a1700 1 -- 172.21.15.6:0/2752795646 >> 172.21.15.6:0/2752795646 conn(0x7f35f80f7450 msgr2=0x7f35f80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.947+0000 7f36004a1700 1 -- 172.21.15.6:0/2752795646 shutdown_connections 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.947+0000 7f36004a1700 1 -- 172.21.15.6:0/2752795646 wait complete. 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.947+0000 7f36004a1700 1 Processor -- start 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.947+0000 7f36004a1700 1 -- start start 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f36004a1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35f80fbcd0 0x7f35f8066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f36004a1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f80fcf80 0x7f35f81126a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f36004a1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8112be0 con 0x7f35f80fbcd0 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f36004a1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8112d20 con 0x7f35f80fcf80 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f35fe23d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35f80fbcd0 0x7f35f8066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f35fe23d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35f80fbcd0 0x7f35f8066cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37450/0 (socket says 172.21.15.6:37450) 2024-03-22T06:20:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f35fe23d700 1 -- 172.21.15.6:0/2614625433 learned_addr learned my addr 172.21.15.6:0/2614625433 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:20:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f35fda3c700 1 --2- 172.21.15.6:0/2614625433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f80fcf80 0x7f35f81126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f35fe23d700 1 -- 172.21.15.6:0/2614625433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f80fcf80 msgr2=0x7f35f81126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f35fe23d700 1 --2- 172.21.15.6:0/2614625433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f80fcf80 0x7f35f81126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f35fe23d700 1 -- 172.21.15.6:0/2614625433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f4005040 con 0x7f35f80fbcd0 2024-03-22T06:20:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f35fe23d700 1 --2- 172.21.15.6:0/2614625433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35f80fbcd0 0x7f35f8066cf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f35f4013870 tx=0x7f35f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.948+0000 7f35eaffd700 1 -- 172.21.15.6:0/2614625433 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35f4021070 con 0x7f35f80fbcd0 2024-03-22T06:20:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.949+0000 7f35eaffd700 1 -- 172.21.15.6:0/2614625433 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35f4009430 con 0x7f35f80fbcd0 2024-03-22T06:20:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.949+0000 7f36004a1700 1 -- 172.21.15.6:0/2614625433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f8112f50 con 0x7f35f80fbcd0 2024-03-22T06:20:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.949+0000 7f35eaffd700 1 -- 172.21.15.6:0/2614625433 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35f400e070 con 0x7f35f80fbcd0 2024-03-22T06:20:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.949+0000 7f36004a1700 1 -- 172.21.15.6:0/2614625433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f81133b0 con 0x7f35f80fbcd0 2024-03-22T06:20:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.950+0000 7f35eaffd700 1 -- 172.21.15.6:0/2614625433 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f35f4026450 con 0x7f35f80fbcd0 2024-03-22T06:20:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.950+0000 7f36004a1700 1 -- 172.21.15.6:0/2614625433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f804c710 con 0x7f35f80fbcd0 2024-03-22T06:20:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.950+0000 7f35eaffd700 1 --2- 172.21.15.6:0/2614625433 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35e406e7e0 0x7f35e4070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.951+0000 7f35eaffd700 1 -- 172.21.15.6:0/2614625433 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f35f408ef00 con 0x7f35f80fbcd0 2024-03-22T06:20:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.951+0000 7f35fda3c700 1 --2- 172.21.15.6:0/2614625433 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35e406e7e0 0x7f35e4070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.951+0000 7f35fda3c700 1 --2- 172.21.15.6:0/2614625433 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35e406e7e0 0x7f35e4070c90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009b80 tx=0x7f35ec00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:27.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:27.956+0000 7f35eaffd700 1 -- 172.21.15.6:0/2614625433 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f35f4094060 con 0x7f35f80fbcd0 2024-03-22T06:20:28.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.264+0000 7f36004a1700 1 -- 172.21.15.6:0/2614625433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f35f8113620 con 0x7f35f80fbcd0 2024-03-22T06:20:28.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.264+0000 7f35eaffd700 1 -- 172.21.15.6:0/2614625433 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f35f405b940 con 0x7f35f80fbcd0 2024-03-22T06:20:28.265 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:20:28.265 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:20:28.265 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:20:28.265 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 3 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 7, 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 7 2024-03-22T06:20:28.266 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:20:28.267 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:20:28.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.267+0000 7f36004a1700 1 -- 172.21.15.6:0/2614625433 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35e406e7e0 msgr2=0x7f35e4070c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:28.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.267+0000 7f36004a1700 1 --2- 172.21.15.6:0/2614625433 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35e406e7e0 0x7f35e4070c90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009b80 tx=0x7f35ec00b040 comp rx=0 tx=0).stop 2024-03-22T06:20:28.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.267+0000 7f36004a1700 1 -- 172.21.15.6:0/2614625433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35f80fbcd0 msgr2=0x7f35f8066cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:28.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.267+0000 7f36004a1700 1 --2- 172.21.15.6:0/2614625433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35f80fbcd0 0x7f35f8066cf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f35f4013870 tx=0x7f35f4009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:28.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.267+0000 7f36004a1700 1 -- 172.21.15.6:0/2614625433 shutdown_connections 2024-03-22T06:20:28.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.267+0000 7f36004a1700 1 --2- 172.21.15.6:0/2614625433 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35e406e7e0 0x7f35e4070c90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.267+0000 7f36004a1700 1 --2- 172.21.15.6:0/2614625433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35f80fbcd0 0x7f35f8066cf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.267+0000 7f36004a1700 1 --2- 172.21.15.6:0/2614625433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f80fcf80 0x7f35f81126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.267+0000 7f36004a1700 1 -- 172.21.15.6:0/2614625433 >> 172.21.15.6:0/2614625433 conn(0x7f35f80f7450 msgr2=0x7f35f80f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:28.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.267+0000 7f36004a1700 1 -- 172.21.15.6:0/2614625433 shutdown_connections 2024-03-22T06:20:28.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.267+0000 7f36004a1700 1 -- 172.21.15.6:0/2614625433 wait complete. 2024-03-22T06:20:28.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.370+0000 7fd575ed3700 1 -- 172.21.15.6:0/366000411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5700fff80 msgr2=0x7fd570102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:28.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.370+0000 7fd575ed3700 1 --2- 172.21.15.6:0/366000411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5700fff80 0x7fd570102360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd564004940 tx=0x7fd564009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:28.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.370+0000 7fd575ed3700 1 -- 172.21.15.6:0/366000411 shutdown_connections 2024-03-22T06:20:28.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.370+0000 7fd575ed3700 1 --2- 172.21.15.6:0/366000411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5700fff80 0x7fd570102360 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.370+0000 7fd575ed3700 1 --2- 172.21.15.6:0/366000411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd570066260 0x7fd5700ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.370+0000 7fd575ed3700 1 -- 172.21.15.6:0/366000411 >> 172.21.15.6:0/366000411 conn(0x7fd5700f7430 msgr2=0x7fd5700f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:28.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.370+0000 7fd575ed3700 1 -- 172.21.15.6:0/366000411 shutdown_connections 2024-03-22T06:20:28.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.370+0000 7fd575ed3700 1 -- 172.21.15.6:0/366000411 wait complete. 2024-03-22T06:20:28.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.370+0000 7fd575ed3700 1 Processor -- start 2024-03-22T06:20:28.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.370+0000 7fd575ed3700 1 -- start start 2024-03-22T06:20:28.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd575ed3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd570066260 0x7fd57010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:28.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd575ed3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5700fff80 0x7fd57010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:28.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd575ed3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57010e570 con 0x7fd570066260 2024-03-22T06:20:28.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd575ed3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57010edc0 con 0x7fd5700fff80 2024-03-22T06:20:28.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd56f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd570066260 0x7fd57010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:28.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd56f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd570066260 0x7fd57010da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37458/0 (socket says 172.21.15.6:37458) 2024-03-22T06:20:28.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd56f7fe700 1 -- 172.21.15.6:0/1501398103 learned_addr learned my addr 172.21.15.6:0/1501398103 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:20:28.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd56effd700 1 --2- 172.21.15.6:0/1501398103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5700fff80 0x7fd57010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:28.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd56f7fe700 1 -- 172.21.15.6:0/1501398103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5700fff80 msgr2=0x7fd57010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:28.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd56f7fe700 1 --2- 172.21.15.6:0/1501398103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5700fff80 0x7fd57010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd56f7fe700 1 -- 172.21.15.6:0/1501398103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd564005040 con 0x7fd570066260 2024-03-22T06:20:28.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd56f7fe700 1 --2- 172.21.15.6:0/1501398103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd570066260 0x7fd57010da30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd560002a80 tx=0x7fd560004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:28.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd557fff700 1 -- 172.21.15.6:0/1501398103 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd560007cc0 con 0x7fd570066260 2024-03-22T06:20:28.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd557fff700 1 -- 172.21.15.6:0/1501398103 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd56000f040 con 0x7fd570066260 2024-03-22T06:20:28.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd575ed3700 1 -- 172.21.15.6:0/1501398103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57010f0a0 con 0x7fd570066260 2024-03-22T06:20:28.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd557fff700 1 -- 172.21.15.6:0/1501398103 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd56000da30 con 0x7fd570066260 2024-03-22T06:20:28.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.371+0000 7fd575ed3700 1 -- 172.21.15.6:0/1501398103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57010f5e0 con 0x7fd570066260 2024-03-22T06:20:28.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.373+0000 7fd575ed3700 1 -- 172.21.15.6:0/1501398103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57005f5f0 con 0x7fd570066260 2024-03-22T06:20:28.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.373+0000 7fd557fff700 1 -- 172.21.15.6:0/1501398103 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fd56000db90 con 0x7fd570066260 2024-03-22T06:20:28.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.373+0000 7fd557fff700 1 --2- 172.21.15.6:0/1501398103 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd55806e840 0x7fd558070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:28.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.373+0000 7fd557fff700 1 -- 172.21.15.6:0/1501398103 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6614+0+0 (secure 0 0 0) 0x7fd56008e220 con 0x7fd570066260 2024-03-22T06:20:28.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.373+0000 7fd56effd700 1 --2- 172.21.15.6:0/1501398103 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd55806e840 0x7fd558070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:28.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.374+0000 7fd56effd700 1 --2- 172.21.15.6:0/1501398103 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd55806e840 0x7fd558070cf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd56400b970 tx=0x7fd564009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.378+0000 7fd557fff700 1 -- 172.21.15.6:0/1501398103 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fd56005ac70 con 0x7fd570066260 2024-03-22T06:20:28.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:28 smithi117 ceph-mon[82126]: pgmap v121: 65 pgs: 15 active+undersized, 10 active+undersized+degraded, 40 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28/219 objects degraded (12.785%) 2024-03-22T06:20:28.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:28 smithi117 ceph-mon[82126]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:20:28.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:28.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:28.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:20:28.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:20:28.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:28 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:28.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:28 smithi006 ceph-mon[94482]: pgmap v121: 65 pgs: 15 active+undersized, 10 active+undersized+degraded, 40 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28/219 objects degraded (12.785%) 2024-03-22T06:20:28.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:28 smithi006 ceph-mon[94482]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:20:28.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:28.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:28.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:20:28.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:20:28.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:28 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:28.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.631+0000 7fd575ed3700 1 -- 172.21.15.6:0/1501398103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd57010f900 con 0x7fd570066260 2024-03-22T06:20:28.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.632+0000 7fd557fff700 1 -- 172.21.15.6:0/1501398103 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fd56005a800 con 0x7fd570066260 2024-03-22T06:20:28.632 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:20:28.633 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:20:28.633 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:20:28.633 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:20:28.633 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:20:28.633 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:20:28.633 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:20:28.633 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:20:28.633 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:20:28.633 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:20:28.633 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:20:28.633 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:20:28.634 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:20:28.634 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:20:28.634 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:20:28.634 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:20:28.634 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:20:28.634 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:20:28.634 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:20:28.634 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:20:28.634 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:20:28.634 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:20:28.634 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 0 members: 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:20:28.635 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:20:28.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:20:28.636 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:20:28.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:20:28.636 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:20:28.636 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:20:28.636 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:20:28.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.634+0000 7fd575ed3700 1 -- 172.21.15.6:0/1501398103 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd55806e840 msgr2=0x7fd558070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:28.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.634+0000 7fd575ed3700 1 --2- 172.21.15.6:0/1501398103 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd55806e840 0x7fd558070cf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd56400b970 tx=0x7fd564009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:28.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.634+0000 7fd575ed3700 1 -- 172.21.15.6:0/1501398103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd570066260 msgr2=0x7fd57010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:28.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.634+0000 7fd575ed3700 1 --2- 172.21.15.6:0/1501398103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd570066260 0x7fd57010da30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd560002a80 tx=0x7fd560004600 comp rx=0 tx=0).stop 2024-03-22T06:20:28.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.634+0000 7fd575ed3700 1 -- 172.21.15.6:0/1501398103 shutdown_connections 2024-03-22T06:20:28.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.634+0000 7fd575ed3700 1 --2- 172.21.15.6:0/1501398103 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd55806e840 0x7fd558070cf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.634+0000 7fd575ed3700 1 --2- 172.21.15.6:0/1501398103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd570066260 0x7fd57010da30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.634+0000 7fd575ed3700 1 --2- 172.21.15.6:0/1501398103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5700fff80 0x7fd57010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.634+0000 7fd575ed3700 1 -- 172.21.15.6:0/1501398103 >> 172.21.15.6:0/1501398103 conn(0x7fd5700f7430 msgr2=0x7fd5700f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:28.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.634+0000 7fd575ed3700 1 -- 172.21.15.6:0/1501398103 shutdown_connections 2024-03-22T06:20:28.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.634+0000 7fd575ed3700 1 -- 172.21.15.6:0/1501398103 wait complete. 2024-03-22T06:20:28.638 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:20:28.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.738+0000 7f846578b700 1 -- 172.21.15.6:0/792695045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8460066b10 msgr2=0x7f84600ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:28.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.738+0000 7f846578b700 1 --2- 172.21.15.6:0/792695045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8460066b10 0x7f84600ffab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8458004940 tx=0x7f8458009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:28.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.739+0000 7f846578b700 1 -- 172.21.15.6:0/792695045 shutdown_connections 2024-03-22T06:20:28.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.739+0000 7f846578b700 1 --2- 172.21.15.6:0/792695045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84600ffff0 0x7f84601023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.739+0000 7f846578b700 1 --2- 172.21.15.6:0/792695045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8460066b10 0x7f84600ffab0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.739+0000 7f846578b700 1 -- 172.21.15.6:0/792695045 >> 172.21.15.6:0/792695045 conn(0x7f84600f74e0 msgr2=0x7f84600f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:28.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.739+0000 7f846578b700 1 -- 172.21.15.6:0/792695045 shutdown_connections 2024-03-22T06:20:28.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.739+0000 7f846578b700 1 -- 172.21.15.6:0/792695045 wait complete. 2024-03-22T06:20:28.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.739+0000 7f846578b700 1 Processor -- start 2024-03-22T06:20:28.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.739+0000 7f846578b700 1 -- start start 2024-03-22T06:20:28.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f846578b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8460066b10 0x7f846010da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:28.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f846578b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84600ffff0 0x7f846010dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:28.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f846578b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846010e5c0 con 0x7f84600ffff0 2024-03-22T06:20:28.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f846578b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846010ee10 con 0x7f8460066b10 2024-03-22T06:20:28.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f845e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84600ffff0 0x7f846010dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:28.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f845e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84600ffff0 0x7f846010dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37480/0 (socket says 172.21.15.6:37480) 2024-03-22T06:20:28.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f845e7fc700 1 -- 172.21.15.6:0/646729421 learned_addr learned my addr 172.21.15.6:0/646729421 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:20:28.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f845effd700 1 --2- 172.21.15.6:0/646729421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8460066b10 0x7f846010da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:28.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f845e7fc700 1 -- 172.21.15.6:0/646729421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8460066b10 msgr2=0x7f846010da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:28.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f845e7fc700 1 --2- 172.21.15.6:0/646729421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8460066b10 0x7f846010da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f845e7fc700 1 -- 172.21.15.6:0/646729421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458005040 con 0x7f84600ffff0 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f845e7fc700 1 --2- 172.21.15.6:0/646729421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84600ffff0 0x7f846010dfc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f844800c830 tx=0x7f8448004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f84577fe700 1 -- 172.21.15.6:0/646729421 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f844800a760 con 0x7f84600ffff0 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f84577fe700 1 -- 172.21.15.6:0/646729421 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f844800ad90 con 0x7f84600ffff0 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f846578b700 1 -- 172.21.15.6:0/646729421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846010f0f0 con 0x7f84600ffff0 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f84577fe700 1 -- 172.21.15.6:0/646729421 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84480113a0 con 0x7f84600ffff0 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.740+0000 7f846578b700 1 -- 172.21.15.6:0/646729421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846010f630 con 0x7f84600ffff0 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.741+0000 7f846578b700 1 -- 172.21.15.6:0/646729421 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846005f5f0 con 0x7f84600ffff0 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.742+0000 7f84577fe700 1 -- 172.21.15.6:0/646729421 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f8448012400 con 0x7f84600ffff0 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.742+0000 7f84577fe700 1 --2- 172.21.15.6:0/646729421 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f844c06e840 0x7f844c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.742+0000 7f84577fe700 1 -- 172.21.15.6:0/646729421 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f844808e4b0 con 0x7f84600ffff0 2024-03-22T06:20:28.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.742+0000 7f845effd700 1 --2- 172.21.15.6:0/646729421 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f844c06e840 0x7f844c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:28.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.743+0000 7f845effd700 1 --2- 172.21.15.6:0/646729421 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f844c06e840 0x7f844c070cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f845800b970 tx=0x7f8458009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:28.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.747+0000 7f84577fe700 1 -- 172.21.15.6:0/646729421 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f844805af80 con 0x7f84600ffff0 2024-03-22T06:20:28.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.972+0000 7f846578b700 1 -- 172.21.15.6:0/646729421 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f846010fa30 con 0x7f844c06e840 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.973+0000 7f84577fe700 1 -- 172.21.15.6:0/646729421 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f846010fa30 con 0x7f844c06e840 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "9/21 daemons upgraded", 2024-03-22T06:20:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T06:20:28.975 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:20:28.975 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:20:28.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.975+0000 7f846578b700 1 -- 172.21.15.6:0/646729421 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f844c06e840 msgr2=0x7f844c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:28.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.975+0000 7f846578b700 1 --2- 172.21.15.6:0/646729421 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f844c06e840 0x7f844c070cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f845800b970 tx=0x7f8458009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:28.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.975+0000 7f846578b700 1 -- 172.21.15.6:0/646729421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84600ffff0 msgr2=0x7f846010dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:28.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.975+0000 7f846578b700 1 --2- 172.21.15.6:0/646729421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84600ffff0 0x7f846010dfc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f844800c830 tx=0x7f8448004470 comp rx=0 tx=0).stop 2024-03-22T06:20:28.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.975+0000 7f846578b700 1 -- 172.21.15.6:0/646729421 shutdown_connections 2024-03-22T06:20:28.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.975+0000 7f846578b700 1 --2- 172.21.15.6:0/646729421 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f844c06e840 0x7f844c070cf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.975+0000 7f846578b700 1 --2- 172.21.15.6:0/646729421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8460066b10 0x7f846010da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.975+0000 7f846578b700 1 --2- 172.21.15.6:0/646729421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84600ffff0 0x7f846010dfc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:28.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.975+0000 7f846578b700 1 -- 172.21.15.6:0/646729421 >> 172.21.15.6:0/646729421 conn(0x7f84600f74e0 msgr2=0x7f84600f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:28.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.975+0000 7f846578b700 1 -- 172.21.15.6:0/646729421 shutdown_connections 2024-03-22T06:20:28.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:28.975+0000 7f846578b700 1 -- 172.21.15.6:0/646729421 wait complete. 2024-03-22T06:20:29.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.078+0000 7f8e56490700 1 -- 172.21.15.6:0/3391530058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500fce10 msgr2=0x7f8e500fd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:29.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.078+0000 7f8e56490700 1 --2- 172.21.15.6:0/3391530058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500fce10 0x7f8e500fd220 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e44004940 tx=0x7f8e44009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:29.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.078+0000 7f8e56490700 1 -- 172.21.15.6:0/3391530058 shutdown_connections 2024-03-22T06:20:29.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.078+0000 7f8e56490700 1 --2- 172.21.15.6:0/3391530058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e500fdf80 0x7f8e500fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:29.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.078+0000 7f8e56490700 1 --2- 172.21.15.6:0/3391530058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500fce10 0x7f8e500fd220 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:29.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.078+0000 7f8e56490700 1 -- 172.21.15.6:0/3391530058 >> 172.21.15.6:0/3391530058 conn(0x7f8e500f84d0 msgr2=0x7f8e500fa900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:29.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.078+0000 7f8e56490700 1 -- 172.21.15.6:0/3391530058 shutdown_connections 2024-03-22T06:20:29.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.078+0000 7f8e56490700 1 -- 172.21.15.6:0/3391530058 wait complete. 2024-03-22T06:20:29.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e56490700 1 Processor -- start 2024-03-22T06:20:29.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e56490700 1 -- start start 2024-03-22T06:20:29.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e56490700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500fce10 0x7f8e5010d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:29.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e56490700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e500fdf80 0x7f8e5010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:29.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e56490700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5010e480 con 0x7f8e500fce10 2024-03-22T06:20:29.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e56490700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5010ecd0 con 0x7f8e500fdf80 2024-03-22T06:20:29.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e4ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500fce10 0x7f8e5010d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e4ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500fce10 0x7f8e5010d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37502/0 (socket says 172.21.15.6:37502) 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e4ffff700 1 -- 172.21.15.6:0/4123152449 learned_addr learned my addr 172.21.15.6:0/4123152449 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e4f7fe700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e500fdf80 0x7f8e5010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e4ffff700 1 -- 172.21.15.6:0/4123152449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e500fdf80 msgr2=0x7f8e5010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e4ffff700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e500fdf80 0x7f8e5010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e4ffff700 1 -- 172.21.15.6:0/4123152449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e44005040 con 0x7f8e500fce10 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.079+0000 7f8e4f7fe700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e500fdf80 0x7f8e5010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.080+0000 7f8e4ffff700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500fce10 0x7f8e5010d940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8e44013870 tx=0x7f8e44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.080+0000 7f8e4cff9700 1 -- 172.21.15.6:0/4123152449 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e44021070 con 0x7f8e500fce10 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.080+0000 7f8e4cff9700 1 -- 172.21.15.6:0/4123152449 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e4401c400 con 0x7f8e500fce10 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.080+0000 7f8e56490700 1 -- 172.21.15.6:0/4123152449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5010ef50 con 0x7f8e500fce10 2024-03-22T06:20:29.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.080+0000 7f8e4cff9700 1 -- 172.21.15.6:0/4123152449 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e4400e070 con 0x7f8e500fce10 2024-03-22T06:20:29.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.080+0000 7f8e56490700 1 -- 172.21.15.6:0/4123152449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5010f490 con 0x7f8e500fce10 2024-03-22T06:20:29.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.081+0000 7f8e56490700 1 -- 172.21.15.6:0/4123152449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e500fc170 con 0x7f8e500fce10 2024-03-22T06:20:29.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.081+0000 7f8e4cff9700 1 -- 172.21.15.6:0/4123152449 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f8e4401ca20 con 0x7f8e500fce10 2024-03-22T06:20:29.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.082+0000 7f8e4cff9700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8e3806e840 0x7f8e38070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:29.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.082+0000 7f8e4cff9700 1 -- 172.21.15.6:0/4123152449 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f8e4408f5a0 con 0x7f8e500fce10 2024-03-22T06:20:29.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.082+0000 7f8e4f7fe700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8e3806e840 0x7f8e38070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:29.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.082+0000 7f8e4f7fe700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8e3806e840 0x7f8e38070cf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e40004600 tx=0x7f8e4000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:29.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.086+0000 7f8e4cff9700 1 -- 172.21.15.6:0/4123152449 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f8e4405c110 con 0x7f8e500fce10 2024-03-22T06:20:29.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:20:29.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:20:29.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: pgmap v122: 65 pgs: 11 active+undersized, 9 active+undersized+degraded, 45 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26/219 objects degraded (11.872%) 2024-03-22T06:20:29.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: pgmap v123: 65 pgs: 11 active+undersized, 9 active+undersized+degraded, 45 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26/219 objects degraded (11.872%) 2024-03-22T06:20:29.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:20:29.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:20:29.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:20:29.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:20:29.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T06:20:29.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T06:20:29.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-03-22T06:20:29.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2614625433' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:20:29.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1501398103' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:20:29.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:29 smithi006 ceph-mon[94482]: Health check update: Degraded data redundancy: 28/219 objects degraded (12.785%), 10 pgs degraded (PG_DEGRADED) 2024-03-22T06:20:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.383+0000 7f8e56490700 1 -- 172.21.15.6:0/4123152449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8e5004bed0 con 0x7f8e500fce10 2024-03-22T06:20:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.384+0000 7f8e4cff9700 1 -- 172.21.15.6:0/4123152449 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+828 (secure 0 0 0) 0x7f8e44004490 con 0x7f8e500fce10 2024-03-22T06:20:29.384 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s); Degraded data redundancy: 26/219 objects degraded (11.872%), 9 pgs degraded 2024-03-22T06:20:29.384 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:20:29.385 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:20:29.385 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 26/219 objects degraded (11.872%), 9 pgs degraded 2024-03-22T06:20:29.385 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-03-22T06:20:29.385 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-03-22T06:20:29.385 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-03-22T06:20:29.385 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-03-22T06:20:29.385 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.e is active+undersized+degraded, acting [1,3] 2024-03-22T06:20:29.385 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-03-22T06:20:29.385 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.16 is active+undersized+degraded, acting [5,1] 2024-03-22T06:20:29.385 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-03-22T06:20:29.385 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-03-22T06:20:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.386+0000 7f8e56490700 1 -- 172.21.15.6:0/4123152449 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8e3806e840 msgr2=0x7f8e38070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.386+0000 7f8e56490700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8e3806e840 0x7f8e38070cf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e40004600 tx=0x7f8e4000b040 comp rx=0 tx=0).stop 2024-03-22T06:20:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.386+0000 7f8e56490700 1 -- 172.21.15.6:0/4123152449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500fce10 msgr2=0x7f8e5010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.386+0000 7f8e56490700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500fce10 0x7f8e5010d940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8e44013870 tx=0x7f8e44009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.386+0000 7f8e56490700 1 -- 172.21.15.6:0/4123152449 shutdown_connections 2024-03-22T06:20:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.386+0000 7f8e56490700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8e3806e840 0x7f8e38070cf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.386+0000 7f8e56490700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500fce10 0x7f8e5010d940 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.386+0000 7f8e56490700 1 --2- 172.21.15.6:0/4123152449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e500fdf80 0x7f8e5010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.386+0000 7f8e56490700 1 -- 172.21.15.6:0/4123152449 >> 172.21.15.6:0/4123152449 conn(0x7f8e500f84d0 msgr2=0x7f8e500f9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.386+0000 7f8e56490700 1 -- 172.21.15.6:0/4123152449 shutdown_connections 2024-03-22T06:20:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:29.386+0000 7f8e56490700 1 -- 172.21.15.6:0/4123152449 wait complete. 2024-03-22T06:20:29.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: pgmap v122: 65 pgs: 11 active+undersized, 9 active+undersized+degraded, 45 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26/219 objects degraded (11.872%) 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: pgmap v123: 65 pgs: 11 active+undersized, 9 active+undersized+degraded, 45 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26/219 objects degraded (11.872%) 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2614625433' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:20:29.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1501398103' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:20:29.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:29 smithi117 ceph-mon[82126]: Health check update: Degraded data redundancy: 28/219 objects degraded (12.785%), 10 pgs degraded (PG_DEGRADED) 2024-03-22T06:20:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:30 smithi117 ceph-mon[82126]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:20:30.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:30 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/4123152449' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:20:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:30 smithi006 ceph-mon[94482]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:20:30.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:30 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/4123152449' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:20:31.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:31 smithi117 ceph-mon[82126]: pgmap v124: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:31.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:31 smithi117 ceph-mon[82126]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 26/219 objects degraded (11.872%), 9 pgs degraded) 2024-03-22T06:20:31.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:31 smithi006 ceph-mon[94482]: pgmap v124: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:31.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:31 smithi006 ceph-mon[94482]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 26/219 objects degraded (11.872%), 9 pgs degraded) 2024-03-22T06:20:33.413 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:33 smithi006 ceph-mon[94482]: pgmap v125: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:33.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:33 smithi117 ceph-mon[82126]: pgmap v125: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:35.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:35 smithi117 ceph-mon[82126]: pgmap v126: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:35.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:35 smithi006 ceph-mon[94482]: pgmap v126: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:37.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:37 smithi117 ceph-mon[82126]: pgmap v127: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:37.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:37 smithi006 ceph-mon[94482]: pgmap v127: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:38.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:38.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:20:38.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:38.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:38 smithi117 ceph-mon[82126]: pgmap v128: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:38.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:38.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:20:38.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:38.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:38 smithi006 ceph-mon[94482]: pgmap v128: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:41 smithi117 ceph-mon[82126]: pgmap v129: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:41.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:41 smithi006 ceph-mon[94482]: pgmap v129: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:43.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:43 smithi117 ceph-mon[82126]: pgmap v130: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:43.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:43 smithi006 ceph-mon[94482]: pgmap v130: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:44.383 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:44 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T06:20:44.383 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:44 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T06:20:44.383 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:44 smithi006 ceph-mon[94482]: Upgrade: osd.3 is safe to restart 2024-03-22T06:20:44.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:44 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T06:20:44.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:44 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T06:20:44.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:44 smithi117 ceph-mon[82126]: Upgrade: osd.3 is safe to restart 2024-03-22T06:20:45.359 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:45 smithi117 ceph-mon[82126]: pgmap v131: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:45.359 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:45 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:45.360 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:45 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T06:20:45.360 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:45 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:20:45.401 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:45 smithi006 ceph-mon[94482]: pgmap v131: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:45.402 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:45 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:45.402 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:45 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T06:20:45.402 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:45 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:20:46.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:46 smithi117 ceph-mon[82126]: Upgrade: Updating osd.3 2024-03-22T06:20:46.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:46 smithi117 ceph-mon[82126]: Deploying daemon osd.3 on smithi117 2024-03-22T06:20:46.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:46 smithi006 ceph-mon[94482]: Upgrade: Updating osd.3 2024-03-22T06:20:46.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:46 smithi006 ceph-mon[94482]: Deploying daemon osd.3 on smithi117 2024-03-22T06:20:47.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:47 smithi117 ceph-mon[82126]: pgmap v132: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:47.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:47 smithi006 ceph-mon[94482]: pgmap v132: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:48.142 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:47 smithi117 systemd[1]: Stopping Ceph osd.3 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:20:48.506 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:48 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[42097]: 2024-03-22T06:20:48.164+0000 7fb07a1d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:20:48.506 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:48 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[42097]: 2024-03-22T06:20:48.164+0000 7fb07a1d9700 -1 osd.3 76 *** Got signal Terminated *** 2024-03-22T06:20:48.506 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:48 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[42097]: 2024-03-22T06:20:48.164+0000 7fb07a1d9700 -1 osd.3 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:20:49.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:49 smithi117 ceph-mon[82126]: pgmap v133: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:49.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:49 smithi117 ceph-mon[82126]: osd.3 marked itself down and dead 2024-03-22T06:20:49.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:49 smithi006 ceph-mon[94482]: pgmap v133: 65 pgs: 65 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:49.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:49 smithi006 ceph-mon[94482]: osd.3 marked itself down and dead 2024-03-22T06:20:49.906 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:49 smithi117 podman[91314]: 2024-03-22 06:20:49.63051408 +0000 UTC m=+1.593231657 container died 3dad19e5cae197b55b8b2dd3233e6d0301a212058e80ed91ea37a8288fbf628b (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=quincy-9c4e0c3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T06:20:50.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:50 smithi117 ceph-mon[82126]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:20:50.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:50 smithi117 ceph-mon[82126]: osdmap e77: 6 total, 5 up, 6 in 2024-03-22T06:20:50.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:50 smithi117 ceph-mon[82126]: osdmap e78: 6 total, 5 up, 6 in 2024-03-22T06:20:50.256 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:49 smithi117 podman[91314]: 2024-03-22 06:20:49.981088187 +0000 UTC m=+1.943805761 container cleanup 3dad19e5cae197b55b8b2dd3233e6d0301a212058e80ed91ea37a8288fbf628b (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=quincy-9c4e0c3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:20:50.256 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:49 smithi117 bash[91314]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3 2024-03-22T06:20:50.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:50 smithi006 ceph-mon[94482]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:20:50.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:50 smithi006 ceph-mon[94482]: osdmap e77: 6 total, 5 up, 6 in 2024-03-22T06:20:50.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:50 smithi006 ceph-mon[94482]: osdmap e78: 6 total, 5 up, 6 in 2024-03-22T06:20:50.613 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:50 smithi117 podman[91344]: 2024-03-22 06:20:50.257368006 +0000 UTC m=+0.628566116 container remove 3dad19e5cae197b55b8b2dd3233e6d0301a212058e80ed91ea37a8288fbf628b (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=quincy-9c4e0c3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-22T06:20:50.894 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:50 smithi117 podman[91433]: 2024-03-22 06:20:50.515195317 +0000 UTC m=+0.017635930 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:20:50.894 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:50 smithi117 podman[91433]: 2024-03-22 06:20:50.616695716 +0000 UTC m=+0.119136314 container create 9e5dc9e142f04e542b54fc60b53eb6eab17ce149e494f47ff695fcc948d600f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-22T06:20:50.894 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:50 smithi117 podman[91433]: 2024-03-22 06:20:50.81922443 +0000 UTC m=+0.321665027 container init 9e5dc9e142f04e542b54fc60b53eb6eab17ce149e494f47ff695fcc948d600f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-03-22T06:20:50.894 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:50 smithi117 podman[91433]: 2024-03-22 06:20:50.823243631 +0000 UTC m=+0.325684234 container start 9e5dc9e142f04e542b54fc60b53eb6eab17ce149e494f47ff695fcc948d600f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-22T06:20:50.894 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:50 smithi117 podman[91433]: 2024-03-22 06:20:50.8940775 +0000 UTC m=+0.396518108 container attach 9e5dc9e142f04e542b54fc60b53eb6eab17ce149e494f47ff695fcc948d600f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:20:51.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:51 smithi117 ceph-mon[82126]: pgmap v135: 65 pgs: 14 stale+active+clean, 51 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:51.256 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:51 smithi117 podman[91433]: 2024-03-22 06:20:51.118776802 +0000 UTC m=+0.621217395 container died 9e5dc9e142f04e542b54fc60b53eb6eab17ce149e494f47ff695fcc948d600f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-22T06:20:51.433 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:51 smithi006 ceph-mon[94482]: pgmap v135: 65 pgs: 14 stale+active+clean, 51 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:20:52.006 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:51 smithi117 podman[91459]: 2024-03-22 06:20:51.689435951 +0000 UTC m=+0.565554376 container remove 9e5dc9e142f04e542b54fc60b53eb6eab17ce149e494f47ff695fcc948d600f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-22T06:20:52.006 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:51 smithi117 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-22T06:20:52.006 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:51 smithi117 systemd[1]: Stopped Ceph osd.3 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:20:52.006 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:51 smithi117 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.3.service: Consumed 1min 12.596s CPU time. 2024-03-22T06:20:52.006 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:51 smithi117 systemd[1]: Starting Ceph osd.3 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:20:52.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:52 smithi117 ceph-mon[82126]: Health check failed: Degraded data redundancy: 13/219 objects degraded (5.936%), 5 pgs degraded (PG_DEGRADED) 2024-03-22T06:20:52.506 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 podman[91563]: 2024-03-22 06:20:52.131953905 +0000 UTC m=+0.113090549 container create e15398406bd7d88d6bc9793ccd43a73ff179592c61aa7e848bc91c58758c2c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-22T06:20:52.506 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 podman[91563]: 2024-03-22 06:20:52.037016966 +0000 UTC m=+0.018153619 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:20:52.506 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 podman[91563]: 2024-03-22 06:20:52.309491863 +0000 UTC m=+0.290628507 container init e15398406bd7d88d6bc9793ccd43a73ff179592c61aa7e848bc91c58758c2c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-22T06:20:52.506 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 podman[91563]: 2024-03-22 06:20:52.312743395 +0000 UTC m=+0.293880043 container start e15398406bd7d88d6bc9793ccd43a73ff179592c61aa7e848bc91c58758c2c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319) 2024-03-22T06:20:52.506 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 podman[91563]: 2024-03-22 06:20:52.367677465 +0000 UTC m=+0.348814113 container attach e15398406bd7d88d6bc9793ccd43a73ff179592c61aa7e848bc91c58758c2c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-22T06:20:52.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:52 smithi006 ceph-mon[94482]: Health check failed: Degraded data redundancy: 13/219 objects degraded (5.936%), 5 pgs degraded (PG_DEGRADED) 2024-03-22T06:20:53.134 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate[91578]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate[91578]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 bash[91563]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 bash[91563]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 bash[91563]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 bash[91563]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 bash[91563]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 bash[91563]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 bash[91563]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate[91578]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate[91578]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate[91578]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-22T06:20:53.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate[91578]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T06:20:53.136 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate[91578]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-22T06:20:53.136 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:52 smithi117 podman[91563]: 2024-03-22 06:20:52.781266487 +0000 UTC m=+0.762403122 container died e15398406bd7d88d6bc9793ccd43a73ff179592c61aa7e848bc91c58758c2c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-22T06:20:53.506 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:53 smithi117 podman[91708]: 2024-03-22 06:20:53.354813376 +0000 UTC m=+0.564558797 container remove e15398406bd7d88d6bc9793ccd43a73ff179592c61aa7e848bc91c58758c2c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-22T06:20:53.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:53 smithi117 ceph-mon[82126]: pgmap v137: 65 pgs: 3 active+undersized, 5 active+undersized+degraded, 12 stale+active+clean, 45 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 13/219 objects degraded (5.936%) 2024-03-22T06:20:53.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:53.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:20:53.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:53 smithi006 ceph-mon[94482]: pgmap v137: 65 pgs: 3 active+undersized, 5 active+undersized+degraded, 12 stale+active+clean, 45 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 13/219 objects degraded (5.936%) 2024-03-22T06:20:53.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:53.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:20:53.814 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:53 smithi117 podman[91770]: 2024-03-22 06:20:53.4678319 +0000 UTC m=+0.017540161 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:20:53.815 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:53 smithi117 podman[91770]: 2024-03-22 06:20:53.572595955 +0000 UTC m=+0.122304210 container create 9cba39b6f32c7854368a20769dca6ec44ecf646b0335f2cdc314f8db750f398b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-22T06:20:53.815 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:53 smithi117 podman[91770]: 2024-03-22 06:20:53.791607694 +0000 UTC m=+0.341315961 container init 9cba39b6f32c7854368a20769dca6ec44ecf646b0335f2cdc314f8db750f398b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-22T06:20:53.815 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:53 smithi117 podman[91770]: 2024-03-22 06:20:53.794645219 +0000 UTC m=+0.344353472 container start 9cba39b6f32c7854368a20769dca6ec44ecf646b0335f2cdc314f8db750f398b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-22T06:20:54.256 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:53 smithi117 ceph-osd[91789]: -- 172.21.15.117:0/41543743 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dce8209680 con 0x55dce81d3000 2024-03-22T06:20:54.256 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:53 smithi117 bash[91770]: 9cba39b6f32c7854368a20769dca6ec44ecf646b0335f2cdc314f8db750f398b 2024-03-22T06:20:54.256 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:53 smithi117 systemd[1]: Started Ceph osd.3 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:20:54.754 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:54 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:20:54.349+0000 7f74cf4dc7c0 -1 Falling back to public interface 2024-03-22T06:20:55.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:54 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:55.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:54 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:55.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:54 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:20:55.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:54 smithi117 ceph-mon[82126]: pgmap v138: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:20:55.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:54 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:55.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:54 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:55.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:54 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:20:55.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:54 smithi006 ceph-mon[94482]: pgmap v138: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:20:57.109 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:57 smithi006 ceph-mon[94482]: pgmap v139: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:20:57.109 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:57.109 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:57.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:57 smithi117 ceph-mon[82126]: pgmap v139: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:20:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:57.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:59.135 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:58 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:20:58.989+0000 7f74cf4dc7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-22T06:20:59.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:59 smithi117 ceph-mon[82126]: pgmap v140: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:20:59.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:59.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:59.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:20:59 smithi117 ceph-mon[82126]: Health check update: Degraded data redundancy: 39/219 objects degraded (17.808%), 14 pgs degraded (PG_DEGRADED) 2024-03-22T06:20:59.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:59 smithi006 ceph-mon[94482]: pgmap v140: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:20:59.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:59.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:20:59.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:20:59 smithi006 ceph-mon[94482]: Health check update: Degraded data redundancy: 39/219 objects degraded (17.808%), 14 pgs degraded (PG_DEGRADED) 2024-03-22T06:20:59.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.504+0000 7faa561a0700 1 -- 172.21.15.6:0/2150443260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa500fc120 msgr2=0x7faa500fc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:59.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.504+0000 7faa561a0700 1 --2- 172.21.15.6:0/2150443260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa500fc120 0x7faa500fc530 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa4c004940 tx=0x7faa4c009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:59.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 -- 172.21.15.6:0/2150443260 shutdown_connections 2024-03-22T06:20:59.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 --2- 172.21.15.6:0/2150443260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa500fca70 0x7faa500fcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 --2- 172.21.15.6:0/2150443260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa500fc120 0x7faa500fc530 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 -- 172.21.15.6:0/2150443260 >> 172.21.15.6:0/2150443260 conn(0x7faa500f69f0 msgr2=0x7faa500f8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 -- 172.21.15.6:0/2150443260 shutdown_connections 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 -- 172.21.15.6:0/2150443260 wait complete. 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 Processor -- start 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 -- start start 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa500fc120 0x7faa50110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa500fca70 0x7faa50112a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa500fbb30 con 0x7faa500fca70 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa561a0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa500fbc70 con 0x7faa500fc120 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.505+0000 7faa5499d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa500fca70 0x7faa50112a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.506+0000 7faa5499d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa500fca70 0x7faa50112a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53258/0 (socket says 172.21.15.6:53258) 2024-03-22T06:20:59.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.506+0000 7faa5499d700 1 -- 172.21.15.6:0/1623418393 learned_addr learned my addr 172.21.15.6:0/1623418393 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:20:59.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.506+0000 7faa5519e700 1 --2- 172.21.15.6:0/1623418393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa500fc120 0x7faa50110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:59.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.506+0000 7faa5499d700 1 -- 172.21.15.6:0/1623418393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa500fc120 msgr2=0x7faa50110510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:59.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.506+0000 7faa5499d700 1 --2- 172.21.15.6:0/1623418393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa500fc120 0x7faa50110510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:59.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.506+0000 7faa5499d700 1 -- 172.21.15.6:0/1623418393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c005040 con 0x7faa500fca70 2024-03-22T06:20:59.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.506+0000 7faa5499d700 1 --2- 172.21.15.6:0/1623418393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa500fca70 0x7faa50112a60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faa4000eb40 tx=0x7faa4000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:59.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.506+0000 7faa45ffb700 1 -- 172.21.15.6:0/1623418393 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa40007cc0 con 0x7faa500fca70 2024-03-22T06:20:59.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.506+0000 7faa45ffb700 1 -- 172.21.15.6:0/1623418393 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa4000f040 con 0x7faa500fca70 2024-03-22T06:20:59.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.506+0000 7faa561a0700 1 -- 172.21.15.6:0/1623418393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa50113000 con 0x7faa500fca70 2024-03-22T06:20:59.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.506+0000 7faa45ffb700 1 -- 172.21.15.6:0/1623418393 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa4000ca10 con 0x7faa500fca70 2024-03-22T06:20:59.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.507+0000 7faa561a0700 1 -- 172.21.15.6:0/1623418393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa50113590 con 0x7faa500fca70 2024-03-22T06:20:59.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.508+0000 7faa561a0700 1 -- 172.21.15.6:0/1623418393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa500fa6b0 con 0x7faa500fca70 2024-03-22T06:20:59.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.508+0000 7faa45ffb700 1 -- 172.21.15.6:0/1623418393 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7faa4000cc80 con 0x7faa500fca70 2024-03-22T06:20:59.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.508+0000 7faa45ffb700 1 --2- 172.21.15.6:0/1623418393 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faa3c06e840 0x7faa3c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:59.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.508+0000 7faa45ffb700 1 -- 172.21.15.6:0/1623418393 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6614+0+0 (secure 0 0 0) 0x7faa4008e020 con 0x7faa500fca70 2024-03-22T06:20:59.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.509+0000 7faa5519e700 1 --2- 172.21.15.6:0/1623418393 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faa3c06e840 0x7faa3c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:59.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.509+0000 7faa5519e700 1 --2- 172.21.15.6:0/1623418393 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faa3c06e840 0x7faa3c070cf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa500fb7b0 tx=0x7faa4c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:59.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.513+0000 7faa45ffb700 1 -- 172.21.15.6:0/1623418393 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7faa40056ff0 con 0x7faa500fca70 2024-03-22T06:20:59.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.739+0000 7faa561a0700 1 -- 172.21.15.6:0/1623418393 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa501011c0 con 0x7faa3c06e840 2024-03-22T06:20:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.740+0000 7faa45ffb700 1 -- 172.21.15.6:0/1623418393 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7faa501011c0 con 0x7faa3c06e840 2024-03-22T06:20:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.742+0000 7faa561a0700 1 -- 172.21.15.6:0/1623418393 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faa3c06e840 msgr2=0x7faa3c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.742+0000 7faa561a0700 1 --2- 172.21.15.6:0/1623418393 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faa3c06e840 0x7faa3c070cf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa500fb7b0 tx=0x7faa4c009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.742+0000 7faa561a0700 1 -- 172.21.15.6:0/1623418393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa500fca70 msgr2=0x7faa50112a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.742+0000 7faa561a0700 1 --2- 172.21.15.6:0/1623418393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa500fca70 0x7faa50112a60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faa4000eb40 tx=0x7faa4000b040 comp rx=0 tx=0).stop 2024-03-22T06:20:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.743+0000 7faa561a0700 1 -- 172.21.15.6:0/1623418393 shutdown_connections 2024-03-22T06:20:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.743+0000 7faa561a0700 1 --2- 172.21.15.6:0/1623418393 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faa3c06e840 0x7faa3c070cf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.743+0000 7faa561a0700 1 --2- 172.21.15.6:0/1623418393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa500fc120 0x7faa50110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.743+0000 7faa561a0700 1 --2- 172.21.15.6:0/1623418393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa500fca70 0x7faa50112a60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:59.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.743+0000 7faa561a0700 1 -- 172.21.15.6:0/1623418393 >> 172.21.15.6:0/1623418393 conn(0x7faa500f69f0 msgr2=0x7faa500f80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:59.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.743+0000 7faa561a0700 1 -- 172.21.15.6:0/1623418393 shutdown_connections 2024-03-22T06:20:59.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.743+0000 7faa561a0700 1 -- 172.21.15.6:0/1623418393 wait complete. 2024-03-22T06:20:59.753 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:20:59.755 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:20:59 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:20:59.500+0000 7f74cf4dc7c0 -1 osd.3 76 log_to_monitors true 2024-03-22T06:20:59.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.846+0000 7f9e506c5700 1 -- 172.21.15.6:0/444289724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e480fcf90 msgr2=0x7f9e480fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:59.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.846+0000 7f9e506c5700 1 --2- 172.21.15.6:0/444289724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e480fcf90 0x7f9e480fd400 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c004940 tx=0x7f9e3c009e30 comp rx=0 tx=0).stop 2024-03-22T06:20:59.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.846+0000 7f9e506c5700 1 -- 172.21.15.6:0/444289724 shutdown_connections 2024-03-22T06:20:59.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.846+0000 7f9e506c5700 1 --2- 172.21.15.6:0/444289724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e480fcf90 0x7f9e480fd400 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:59.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.846+0000 7f9e506c5700 1 --2- 172.21.15.6:0/444289724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e480fbce0 0x7f9e480fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:59.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.846+0000 7f9e506c5700 1 -- 172.21.15.6:0/444289724 >> 172.21.15.6:0/444289724 conn(0x7f9e480f7440 msgr2=0x7f9e480f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:20:59.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.846+0000 7f9e506c5700 1 -- 172.21.15.6:0/444289724 shutdown_connections 2024-03-22T06:20:59.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.846+0000 7f9e506c5700 1 -- 172.21.15.6:0/444289724 wait complete. 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.846+0000 7f9e506c5700 1 Processor -- start 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.846+0000 7f9e506c5700 1 -- start start 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e506c5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e480fbce0 0x7f9e4810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e506c5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e480fcf90 0x7f9e4810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e506c5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4810e520 con 0x7f9e480fbce0 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e506c5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e48112ec0 con 0x7f9e480fcf90 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e4e461700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e480fbce0 0x7f9e4810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e4e461700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e480fbce0 0x7f9e4810d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53272/0 (socket says 172.21.15.6:53272) 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e4e461700 1 -- 172.21.15.6:0/178614385 learned_addr learned my addr 172.21.15.6:0/178614385 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e4dc60700 1 --2- 172.21.15.6:0/178614385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e480fcf90 0x7f9e4810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e4e461700 1 -- 172.21.15.6:0/178614385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e480fcf90 msgr2=0x7f9e4810de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e4e461700 1 --2- 172.21.15.6:0/178614385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e480fcf90 0x7f9e4810de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e4e461700 1 -- 172.21.15.6:0/178614385 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c005040 con 0x7f9e480fbce0 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e4e461700 1 --2- 172.21.15.6:0/178614385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e480fbce0 0x7f9e4810d950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800bbb0 tx=0x7f9e38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:59.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e46ffd700 1 -- 172.21.15.6:0/178614385 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e380092e0 con 0x7f9e480fbce0 2024-03-22T06:20:59.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e46ffd700 1 -- 172.21.15.6:0/178614385 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e3800cbd0 con 0x7f9e480fbce0 2024-03-22T06:20:59.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e506c5700 1 -- 172.21.15.6:0/178614385 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e48113120 con 0x7f9e480fbce0 2024-03-22T06:20:59.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e46ffd700 1 -- 172.21.15.6:0/178614385 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e38007400 con 0x7f9e480fbce0 2024-03-22T06:20:59.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.847+0000 7f9e506c5700 1 -- 172.21.15.6:0/178614385 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e48113690 con 0x7f9e480fbce0 2024-03-22T06:20:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.848+0000 7f9e506c5700 1 -- 172.21.15.6:0/178614385 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4804c710 con 0x7f9e480fbce0 2024-03-22T06:20:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.849+0000 7f9e46ffd700 1 -- 172.21.15.6:0/178614385 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f9e38012070 con 0x7f9e480fbce0 2024-03-22T06:20:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.849+0000 7f9e46ffd700 1 --2- 172.21.15.6:0/178614385 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9e3006e840 0x7f9e30070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:20:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.849+0000 7f9e46ffd700 1 -- 172.21.15.6:0/178614385 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f9e3808e130 con 0x7f9e480fbce0 2024-03-22T06:20:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.849+0000 7f9e4dc60700 1 --2- 172.21.15.6:0/178614385 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9e3006e840 0x7f9e30070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:20:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.850+0000 7f9e4dc60700 1 --2- 172.21.15.6:0/178614385 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9e3006e840 0x7f9e30070cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00b3b0 tx=0x7f9e3c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:20:59.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:20:59.854+0000 7f9e46ffd700 1 -- 172.21.15.6:0/178614385 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f9e3805ab80 con 0x7f9e480fbce0 2024-03-22T06:21:00.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.086+0000 7f9e506c5700 1 -- 172.21.15.6:0/178614385 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e48101270 con 0x7f9e3006e840 2024-03-22T06:21:00.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.087+0000 7f9e46ffd700 1 -- 172.21.15.6:0/178614385 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9e48101270 con 0x7f9e3006e840 2024-03-22T06:21:00.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.089+0000 7f9e506c5700 1 -- 172.21.15.6:0/178614385 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9e3006e840 msgr2=0x7f9e30070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.089+0000 7f9e506c5700 1 --2- 172.21.15.6:0/178614385 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9e3006e840 0x7f9e30070cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00b3b0 tx=0x7f9e3c009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:00.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.089+0000 7f9e506c5700 1 -- 172.21.15.6:0/178614385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e480fbce0 msgr2=0x7f9e4810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.089+0000 7f9e506c5700 1 --2- 172.21.15.6:0/178614385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e480fbce0 0x7f9e4810d950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800bbb0 tx=0x7f9e38009840 comp rx=0 tx=0).stop 2024-03-22T06:21:00.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.089+0000 7f9e506c5700 1 -- 172.21.15.6:0/178614385 shutdown_connections 2024-03-22T06:21:00.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.089+0000 7f9e506c5700 1 --2- 172.21.15.6:0/178614385 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9e3006e840 0x7f9e30070cf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.089+0000 7f9e506c5700 1 --2- 172.21.15.6:0/178614385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e480fbce0 0x7f9e4810d950 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.089+0000 7f9e506c5700 1 --2- 172.21.15.6:0/178614385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e480fcf90 0x7f9e4810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.089+0000 7f9e506c5700 1 -- 172.21.15.6:0/178614385 >> 172.21.15.6:0/178614385 conn(0x7f9e480f7440 msgr2=0x7f9e480f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:00.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.089+0000 7f9e506c5700 1 -- 172.21.15.6:0/178614385 shutdown_connections 2024-03-22T06:21:00.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.089+0000 7f9e506c5700 1 -- 172.21.15.6:0/178614385 wait complete. 2024-03-22T06:21:00.156 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:00 smithi117 ceph-mon[82126]: from='osd.3 [v2:172.21.15.117:6800/1104353694,v1:172.21.15.117:6801/1104353694]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T06:21:00.156 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:00 smithi117 ceph-mon[82126]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T06:21:00.156 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:21:00 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:21:00.039+0000 7f74c0934700 -1 osd.3 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:21:00.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.192+0000 7f88c8496700 1 -- 172.21.15.6:0/281131835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c00fcf80 msgr2=0x7f88c00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.192+0000 7f88c8496700 1 --2- 172.21.15.6:0/281131835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c00fcf80 0x7f88c00fd3f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88b4004940 tx=0x7f88b4009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:00.193 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:00 smithi006 ceph-mon[94482]: from='osd.3 [v2:172.21.15.117:6800/1104353694,v1:172.21.15.117:6801/1104353694]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T06:21:00.193 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:00 smithi006 ceph-mon[94482]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T06:21:00.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.192+0000 7f88c8496700 1 -- 172.21.15.6:0/281131835 shutdown_connections 2024-03-22T06:21:00.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.192+0000 7f88c8496700 1 --2- 172.21.15.6:0/281131835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c00fcf80 0x7f88c00fd3f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.192+0000 7f88c8496700 1 --2- 172.21.15.6:0/281131835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00fbcd0 0x7f88c00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.192+0000 7f88c8496700 1 -- 172.21.15.6:0/281131835 >> 172.21.15.6:0/281131835 conn(0x7f88c00f7430 msgr2=0x7f88c00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.192+0000 7f88c8496700 1 -- 172.21.15.6:0/281131835 shutdown_connections 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.192+0000 7f88c8496700 1 -- 172.21.15.6:0/281131835 wait complete. 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.192+0000 7f88c8496700 1 Processor -- start 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c8496700 1 -- start start 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c8496700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00fbcd0 0x7f88c010da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c8496700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c00fcf80 0x7f88c010dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c8496700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c010e5a0 con 0x7f88c00fcf80 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c8496700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c010edf0 con 0x7f88c00fbcd0 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c5a31700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c00fcf80 0x7f88c010dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c5a31700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c00fcf80 0x7f88c010dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53288/0 (socket says 172.21.15.6:53288) 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c5a31700 1 -- 172.21.15.6:0/924225613 learned_addr learned my addr 172.21.15.6:0/924225613 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c6232700 1 --2- 172.21.15.6:0/924225613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00fbcd0 0x7f88c010da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:00.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c5a31700 1 -- 172.21.15.6:0/924225613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00fbcd0 msgr2=0x7f88c010da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c5a31700 1 --2- 172.21.15.6:0/924225613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00fbcd0 0x7f88c010da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c5a31700 1 -- 172.21.15.6:0/924225613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b4005040 con 0x7f88c00fcf80 2024-03-22T06:21:00.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.193+0000 7f88c5a31700 1 --2- 172.21.15.6:0/924225613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c00fcf80 0x7f88c010dfa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f88b400b3b0 tx=0x7f88b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:00.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.194+0000 7f88b2ffd700 1 -- 172.21.15.6:0/924225613 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88b400e040 con 0x7f88c00fcf80 2024-03-22T06:21:00.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.194+0000 7f88b2ffd700 1 -- 172.21.15.6:0/924225613 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88b400bbf0 con 0x7f88c00fcf80 2024-03-22T06:21:00.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.194+0000 7f88b2ffd700 1 -- 172.21.15.6:0/924225613 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88b401c630 con 0x7f88c00fcf80 2024-03-22T06:21:00.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.194+0000 7f88c8496700 1 -- 172.21.15.6:0/924225613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c010f070 con 0x7f88c00fcf80 2024-03-22T06:21:00.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.194+0000 7f88c8496700 1 -- 172.21.15.6:0/924225613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c010f560 con 0x7f88c00fcf80 2024-03-22T06:21:00.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.195+0000 7f88c8496700 1 -- 172.21.15.6:0/924225613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c0101b10 con 0x7f88c00fcf80 2024-03-22T06:21:00.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.196+0000 7f88b2ffd700 1 -- 172.21.15.6:0/924225613 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f88b400bd60 con 0x7f88c00fcf80 2024-03-22T06:21:00.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.196+0000 7f88b2ffd700 1 --2- 172.21.15.6:0/924225613 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f88ac06e840 0x7f88ac070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:00.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.196+0000 7f88b2ffd700 1 -- 172.21.15.6:0/924225613 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f88b408f090 con 0x7f88c00fcf80 2024-03-22T06:21:00.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.196+0000 7f88c6232700 1 --2- 172.21.15.6:0/924225613 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f88ac06e840 0x7f88ac070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:00.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.197+0000 7f88c6232700 1 --2- 172.21.15.6:0/924225613 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f88ac06e840 0x7f88ac070cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f88bc004f60 tx=0x7f88bc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:00.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.201+0000 7f88b2ffd700 1 -- 172.21.15.6:0/924225613 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f88b405bb60 con 0x7f88c00fcf80 2024-03-22T06:21:00.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.422+0000 7f88c8496700 1 -- 172.21.15.6:0/924225613 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f88c01049a0 con 0x7f88ac06e840 2024-03-22T06:21:00.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.432+0000 7f88b2ffd700 1 -- 172.21.15.6:0/924225613 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f88c01049a0 con 0x7f88ac06e840 2024-03-22T06:21:00.433 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:21:00.433 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (5m) 37s ago 14m 26.2M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:21:00.433 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (2m) 37s ago 14m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:21:00.433 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (2m) 3s ago 13m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:21:00.433 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 unknown 37s ago 13m - - 2024-03-22T06:21:00.433 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (8m) 37s ago 8m 157M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:21:00.433 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (8m) 37s ago 8m 18.0M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:21:00.433 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (8m) 3s ago 8m 15.1M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:21:00.433 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (8m) 3s ago 8m 16.9M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:21:00.433 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (4m) 37s ago 15m 504M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (3m) 3s ago 13m 451M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (3m) 37s ago 15m 48.0M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (3m) 3s ago 13m 39.3M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (5m) 37s ago 14m 17.6M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (5m) 3s ago 13m 14.7M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (119s) 37s ago 12m 148M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (81s) 37s ago 11m 147M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (43s) 37s ago 11m 12.5M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 3ec1f219c19c 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (6s) 3s ago 10m 15.0M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 9cba39b6f32c 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (10m) 3s ago 10m 254M 2581M 17.2.7-726-g9c4e0c34 837124da804f e2021dbf5ac6 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (10m) 3s ago 10m 308M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:21:00.434 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (3m) 37s ago 13m 64.4M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:21:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.434+0000 7f88c8496700 1 -- 172.21.15.6:0/924225613 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f88ac06e840 msgr2=0x7f88ac070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.434+0000 7f88c8496700 1 --2- 172.21.15.6:0/924225613 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f88ac06e840 0x7f88ac070cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f88bc004f60 tx=0x7f88bc00b040 comp rx=0 tx=0).stop 2024-03-22T06:21:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.434+0000 7f88c8496700 1 -- 172.21.15.6:0/924225613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c00fcf80 msgr2=0x7f88c010dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.434+0000 7f88c8496700 1 --2- 172.21.15.6:0/924225613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c00fcf80 0x7f88c010dfa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f88b400b3b0 tx=0x7f88b4009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.434+0000 7f88c8496700 1 -- 172.21.15.6:0/924225613 shutdown_connections 2024-03-22T06:21:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.434+0000 7f88c8496700 1 --2- 172.21.15.6:0/924225613 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f88ac06e840 0x7f88ac070cf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.434+0000 7f88c8496700 1 --2- 172.21.15.6:0/924225613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00fbcd0 0x7f88c010da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.434+0000 7f88c8496700 1 --2- 172.21.15.6:0/924225613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c00fcf80 0x7f88c010dfa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.434+0000 7f88c8496700 1 -- 172.21.15.6:0/924225613 >> 172.21.15.6:0/924225613 conn(0x7f88c00f7430 msgr2=0x7f88c0065ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:00.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.435+0000 7f88c8496700 1 -- 172.21.15.6:0/924225613 shutdown_connections 2024-03-22T06:21:00.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.435+0000 7f88c8496700 1 -- 172.21.15.6:0/924225613 wait complete. 2024-03-22T06:21:00.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.535+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/2634716343 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2ec0fbcd0 msgr2=0x7ff2ec0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.535+0000 7ff2f1c7d700 1 --2- 172.21.15.6:0/2634716343 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2ec0fbcd0 0x7ff2ec0fc0e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc004940 tx=0x7ff2dc009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:00.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.535+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/2634716343 shutdown_connections 2024-03-22T06:21:00.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.535+0000 7ff2f1c7d700 1 --2- 172.21.15.6:0/2634716343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ec0fcf80 0x7ff2ec0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.535+0000 7ff2f1c7d700 1 --2- 172.21.15.6:0/2634716343 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2ec0fbcd0 0x7ff2ec0fc0e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.535+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/2634716343 >> 172.21.15.6:0/2634716343 conn(0x7ff2ec0f7430 msgr2=0x7ff2ec0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:00.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.536+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/2634716343 shutdown_connections 2024-03-22T06:21:00.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.536+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/2634716343 wait complete. 2024-03-22T06:21:00.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.536+0000 7ff2f1c7d700 1 Processor -- start 2024-03-22T06:21:00.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.536+0000 7ff2f1c7d700 1 -- start start 2024-03-22T06:21:00.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2f1c7d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ec0fbcd0 0x7ff2ec110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:00.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2f1c7d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2ec0fcf80 0x7ff2ec112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:00.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2f1c7d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec066350 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2f1c7d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec066490 con 0x7ff2ec0fbcd0 2024-03-22T06:21:00.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2eaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2ec0fcf80 0x7ff2ec112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2eaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2ec0fcf80 0x7ff2ec112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53304/0 (socket says 172.21.15.6:53304) 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2eaffd700 1 -- 172.21.15.6:0/3675209414 learned_addr learned my addr 172.21.15.6:0/3675209414 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2eb7fe700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ec0fbcd0 0x7ff2ec110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2eaffd700 1 -- 172.21.15.6:0/3675209414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ec0fbcd0 msgr2=0x7ff2ec110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2eaffd700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ec0fbcd0 0x7ff2ec110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2eaffd700 1 -- 172.21.15.6:0/3675209414 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc005040 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2eb7fe700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ec0fbcd0 0x7ff2ec110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2eaffd700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2ec0fcf80 0x7ff2ec112c70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000c8c0 tx=0x7ff2e00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2d3fff700 1 -- 172.21.15.6:0/3675209414 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e000a980 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/3675209414 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ec113210 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2d3fff700 1 -- 172.21.15.6:0/3675209414 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2e000ce60 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2d3fff700 1 -- 172.21.15.6:0/3675209414 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e00115e0 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.537+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/3675209414 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ec1137d0 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.538+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/3675209414 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ec05f5f0 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.539+0000 7ff2d3fff700 1 -- 172.21.15.6:0/3675209414 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7ff2e0012400 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.539+0000 7ff2d3fff700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d406e840 0x7ff2d4070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:00.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.539+0000 7ff2d3fff700 1 -- 172.21.15.6:0/3675209414 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6614+0+0 (secure 0 0 0) 0x7ff2e008e520 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.539+0000 7ff2eb7fe700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d406e840 0x7ff2d4070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:00.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.540+0000 7ff2eb7fe700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d406e840 0x7ff2d4070cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc004890 tx=0x7ff2dc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:00.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.544+0000 7ff2d3fff700 1 -- 172.21.15.6:0/3675209414 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7ff2e00574c0 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.838+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/3675209414 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff2ec113b60 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.839+0000 7ff2d3fff700 1 -- 172.21.15.6:0/3675209414 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7ff2e00126b0 con 0x7ff2ec0fcf80 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2, 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 3 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:21:00.840 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:21:00.841 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:21:00.841 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:21:00.841 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6, 2024-03-22T06:21:00.841 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 7 2024-03-22T06:21:00.841 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:21:00.841 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:21:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.841+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/3675209414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d406e840 msgr2=0x7ff2d4070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.841+0000 7ff2f1c7d700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d406e840 0x7ff2d4070cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc004890 tx=0x7ff2dc009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.841+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/3675209414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2ec0fcf80 msgr2=0x7ff2ec112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.841+0000 7ff2f1c7d700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2ec0fcf80 0x7ff2ec112c70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000c8c0 tx=0x7ff2e00044c0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.841+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/3675209414 shutdown_connections 2024-03-22T06:21:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.841+0000 7ff2f1c7d700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d406e840 0x7ff2d4070cf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.841+0000 7ff2f1c7d700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ec0fbcd0 0x7ff2ec110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.841+0000 7ff2f1c7d700 1 --2- 172.21.15.6:0/3675209414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2ec0fcf80 0x7ff2ec112c70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.841+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/3675209414 >> 172.21.15.6:0/3675209414 conn(0x7ff2ec0f7430 msgr2=0x7ff2ec0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.841+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/3675209414 shutdown_connections 2024-03-22T06:21:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.841+0000 7ff2f1c7d700 1 -- 172.21.15.6:0/3675209414 wait complete. 2024-03-22T06:21:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.943+0000 7f86b62c2700 1 -- 172.21.15.6:0/3290131198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86b0066ef0 msgr2=0x7f86b00ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.943+0000 7f86b62c2700 1 --2- 172.21.15.6:0/3290131198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86b0066ef0 0x7f86b00ffa20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f86a0004940 tx=0x7f86a0009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.943+0000 7f86b62c2700 1 -- 172.21.15.6:0/3290131198 shutdown_connections 2024-03-22T06:21:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.943+0000 7f86b62c2700 1 --2- 172.21.15.6:0/3290131198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b00fff60 0x7f86b0102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.943+0000 7f86b62c2700 1 --2- 172.21.15.6:0/3290131198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86b0066ef0 0x7f86b00ffa20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.943+0000 7f86b62c2700 1 -- 172.21.15.6:0/3290131198 >> 172.21.15.6:0/3290131198 conn(0x7f86b00f7450 msgr2=0x7f86b00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.943+0000 7f86b62c2700 1 -- 172.21.15.6:0/3290131198 shutdown_connections 2024-03-22T06:21:00.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.943+0000 7f86b62c2700 1 -- 172.21.15.6:0/3290131198 wait complete. 2024-03-22T06:21:00.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.944+0000 7f86b62c2700 1 Processor -- start 2024-03-22T06:21:00.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.944+0000 7f86b62c2700 1 -- start start 2024-03-22T06:21:00.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.944+0000 7f86b62c2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0066ef0 0x7f86b00fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:00.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.944+0000 7f86b62c2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86b00fff60 0x7f86b00fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.944+0000 7f86b62c2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00fbe60 con 0x7f86b00fff60 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.944+0000 7f86b62c2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00fbfd0 con 0x7f86b0066ef0 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.944+0000 7f86af7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86b00fff60 0x7f86b00fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.944+0000 7f86af7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86b00fff60 0x7f86b00fb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53320/0 (socket says 172.21.15.6:53320) 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86af7fe700 1 -- 172.21.15.6:0/848446586 learned_addr learned my addr 172.21.15.6:0/848446586 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86affff700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0066ef0 0x7f86b00fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86af7fe700 1 -- 172.21.15.6:0/848446586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0066ef0 msgr2=0x7f86b00fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86af7fe700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0066ef0 0x7f86b00fd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86af7fe700 1 -- 172.21.15.6:0/848446586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a0005040 con 0x7f86b00fff60 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86affff700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0066ef0 0x7f86b00fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86af7fe700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86b00fff60 0x7f86b00fb920 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f86a400b790 tx=0x7f86a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86acff9700 1 -- 172.21.15.6:0/848446586 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86a400f770 con 0x7f86b00fff60 2024-03-22T06:21:00.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86acff9700 1 -- 172.21.15.6:0/848446586 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86a400fda0 con 0x7f86b00fff60 2024-03-22T06:21:00.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86b62c2700 1 -- 172.21.15.6:0/848446586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b00fc2b0 con 0x7f86b00fff60 2024-03-22T06:21:00.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86acff9700 1 -- 172.21.15.6:0/848446586 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86a400d310 con 0x7f86b00fff60 2024-03-22T06:21:00.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.945+0000 7f86b62c2700 1 -- 172.21.15.6:0/848446586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b0115a50 con 0x7f86b00fff60 2024-03-22T06:21:00.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.946+0000 7f86b62c2700 1 -- 172.21.15.6:0/848446586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b004c710 con 0x7f86b00fff60 2024-03-22T06:21:00.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.947+0000 7f86acff9700 1 -- 172.21.15.6:0/848446586 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f86a4010420 con 0x7f86b00fff60 2024-03-22T06:21:00.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.947+0000 7f86acff9700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f869806e840 0x7f8698070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:00.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.947+0000 7f86acff9700 1 -- 172.21.15.6:0/848446586 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f86a408e3d0 con 0x7f86b00fff60 2024-03-22T06:21:00.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.947+0000 7f86affff700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f869806e840 0x7f8698070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:00.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.947+0000 7f86affff700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f869806e840 0x7f8698070cf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86a000b3b0 tx=0x7f86a0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:00.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:00.952+0000 7f86acff9700 1 -- 172.21.15.6:0/848446586 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f86a405af40 con 0x7f86b00fff60 2024-03-22T06:21:01.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.214+0000 7f86b62c2700 1 -- 172.21.15.6:0/848446586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f86b0105930 con 0x7f86b00fff60 2024-03-22T06:21:01.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:01 smithi006 ceph-mon[94482]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:01.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:01 smithi006 ceph-mon[94482]: pgmap v141: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:21:01.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:01 smithi006 ceph-mon[94482]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T06:21:01.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:01 smithi006 ceph-mon[94482]: osdmap e79: 6 total, 5 up, 6 in 2024-03-22T06:21:01.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:01 smithi006 ceph-mon[94482]: from='osd.3 [v2:172.21.15.117:6800/1104353694,v1:172.21.15.117:6801/1104353694]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:21:01.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:01 smithi006 ceph-mon[94482]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:21:01.215 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:01 smithi006 ceph-mon[94482]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:01.216 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:01 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3675209414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:01.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.214+0000 7f86acff9700 1 -- 172.21.15.6:0/848446586 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1875 (secure 0 0 0) 0x7f86a405aad0 con 0x7f86b00fff60 2024-03-22T06:21:01.216 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:21:01.216 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:21:01.216 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:21:01.216 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:21:01.216 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:21:01.216 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:21:01.217 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:21:01.218 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:21:01.219 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:21:01.219 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 0 members: 2024-03-22T06:21:01.219 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:21:01.219 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:21:01.219 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:21:01.219 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:21:01.219 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:21:01.219 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:21:01.219 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:21:01.219 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:21:01.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.216+0000 7f86b62c2700 1 -- 172.21.15.6:0/848446586 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f869806e840 msgr2=0x7f8698070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:01.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.216+0000 7f86b62c2700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f869806e840 0x7f8698070cf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86a000b3b0 tx=0x7f86a0009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:01.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.216+0000 7f86b62c2700 1 -- 172.21.15.6:0/848446586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86b00fff60 msgr2=0x7f86b00fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:01.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.216+0000 7f86b62c2700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86b00fff60 0x7f86b00fb920 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f86a400b790 tx=0x7f86a4009250 comp rx=0 tx=0).stop 2024-03-22T06:21:01.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.217+0000 7f86b62c2700 1 -- 172.21.15.6:0/848446586 shutdown_connections 2024-03-22T06:21:01.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.217+0000 7f86b62c2700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f869806e840 0x7f8698070cf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.217+0000 7f86b62c2700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0066ef0 0x7f86b00fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.217+0000 7f86b62c2700 1 --2- 172.21.15.6:0/848446586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86b00fff60 0x7f86b00fb920 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.217+0000 7f86b62c2700 1 -- 172.21.15.6:0/848446586 >> 172.21.15.6:0/848446586 conn(0x7f86b00f7450 msgr2=0x7f86b00f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:01.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.217+0000 7f86b62c2700 1 -- 172.21.15.6:0/848446586 shutdown_connections 2024-03-22T06:21:01.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.217+0000 7f86b62c2700 1 -- 172.21.15.6:0/848446586 wait complete. 2024-03-22T06:21:01.221 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:21:01.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:01 smithi117 ceph-mon[82126]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:01.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:01 smithi117 ceph-mon[82126]: pgmap v141: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:21:01.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:01 smithi117 ceph-mon[82126]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T06:21:01.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:01 smithi117 ceph-mon[82126]: osdmap e79: 6 total, 5 up, 6 in 2024-03-22T06:21:01.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:01 smithi117 ceph-mon[82126]: from='osd.3 [v2:172.21.15.117:6800/1104353694,v1:172.21.15.117:6801/1104353694]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:21:01.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:01 smithi117 ceph-mon[82126]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:21:01.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:01 smithi117 ceph-mon[82126]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:01.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:01 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3675209414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:01.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.321+0000 7f912f353700 1 -- 172.21.15.6:0/78352459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91280fce70 msgr2=0x7f91280fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:01.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.321+0000 7f912f353700 1 --2- 172.21.15.6:0/78352459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91280fce70 0x7f91280fd280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9120004940 tx=0x7f9120009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:01.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.321+0000 7f912f353700 1 -- 172.21.15.6:0/78352459 shutdown_connections 2024-03-22T06:21:01.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.321+0000 7f912f353700 1 --2- 172.21.15.6:0/78352459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91280fdfe0 0x7f91280fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.321+0000 7f912f353700 1 --2- 172.21.15.6:0/78352459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91280fce70 0x7f91280fd280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.321+0000 7f912f353700 1 -- 172.21.15.6:0/78352459 >> 172.21.15.6:0/78352459 conn(0x7f91280f8550 msgr2=0x7f91280fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:01.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.321+0000 7f912f353700 1 -- 172.21.15.6:0/78352459 shutdown_connections 2024-03-22T06:21:01.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.321+0000 7f912f353700 1 -- 172.21.15.6:0/78352459 wait complete. 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912f353700 1 Processor -- start 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912f353700 1 -- start start 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912f353700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91280fce70 0x7f912810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912f353700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91280fdfe0 0x7f912810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912f353700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912810e490 con 0x7f91280fdfe0 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912f353700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912810e600 con 0x7f91280fce70 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912c8ee700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91280fdfe0 0x7f912810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912c8ee700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91280fdfe0 0x7f912810de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53350/0 (socket says 172.21.15.6:53350) 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912c8ee700 1 -- 172.21.15.6:0/964206472 learned_addr learned my addr 172.21.15.6:0/964206472 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912d0ef700 1 --2- 172.21.15.6:0/964206472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91280fce70 0x7f912810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912c8ee700 1 -- 172.21.15.6:0/964206472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91280fce70 msgr2=0x7f912810d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912c8ee700 1 --2- 172.21.15.6:0/964206472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91280fce70 0x7f912810d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.322+0000 7f912c8ee700 1 -- 172.21.15.6:0/964206472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9120005040 con 0x7f91280fdfe0 2024-03-22T06:21:01.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.323+0000 7f912c8ee700 1 --2- 172.21.15.6:0/964206472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91280fdfe0 0x7f912810de90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f911c002a50 tx=0x7f911c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:01.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.323+0000 7f9125ffb700 1 -- 172.21.15.6:0/964206472 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f911c007cc0 con 0x7f91280fdfe0 2024-03-22T06:21:01.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.323+0000 7f9125ffb700 1 -- 172.21.15.6:0/964206472 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f911c00f040 con 0x7f91280fdfe0 2024-03-22T06:21:01.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.323+0000 7f912f353700 1 -- 172.21.15.6:0/964206472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9128113060 con 0x7f91280fdfe0 2024-03-22T06:21:01.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.323+0000 7f9125ffb700 1 -- 172.21.15.6:0/964206472 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f911c00da30 con 0x7f91280fdfe0 2024-03-22T06:21:01.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.323+0000 7f912f353700 1 -- 172.21.15.6:0/964206472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9128113620 con 0x7f91280fdfe0 2024-03-22T06:21:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.324+0000 7f912f353700 1 -- 172.21.15.6:0/964206472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912804c710 con 0x7f91280fdfe0 2024-03-22T06:21:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.324+0000 7f9125ffb700 1 -- 172.21.15.6:0/964206472 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f911c00db90 con 0x7f91280fdfe0 2024-03-22T06:21:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.325+0000 7f9125ffb700 1 --2- 172.21.15.6:0/964206472 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f911006e840 0x7f9110070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.325+0000 7f9125ffb700 1 -- 172.21.15.6:0/964206472 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f911c08e1e0 con 0x7f91280fdfe0 2024-03-22T06:21:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.325+0000 7f912d0ef700 1 --2- 172.21.15.6:0/964206472 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f911006e840 0x7f9110070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:01.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.325+0000 7f912d0ef700 1 --2- 172.21.15.6:0/964206472 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f911006e840 0x7f9110070cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f912000b970 tx=0x7f9120009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:01.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.329+0000 7f9125ffb700 1 -- 172.21.15.6:0/964206472 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f911c05ac30 con 0x7f91280fdfe0 2024-03-22T06:21:01.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.561+0000 7f912f353700 1 -- 172.21.15.6:0/964206472 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9128102340 con 0x7f911006e840 2024-03-22T06:21:01.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.562+0000 7f9125ffb700 1 -- 172.21.15.6:0/964206472 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9128102340 con 0x7f911006e840 2024-03-22T06:21:01.562 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "10/21 daemons upgraded", 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:21:01.563 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:21:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.564+0000 7f912f353700 1 -- 172.21.15.6:0/964206472 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f911006e840 msgr2=0x7f9110070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.564+0000 7f912f353700 1 --2- 172.21.15.6:0/964206472 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f911006e840 0x7f9110070cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f912000b970 tx=0x7f9120009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.564+0000 7f912f353700 1 -- 172.21.15.6:0/964206472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91280fdfe0 msgr2=0x7f912810de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.564+0000 7f912f353700 1 --2- 172.21.15.6:0/964206472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91280fdfe0 0x7f912810de90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f911c002a50 tx=0x7f911c0046b0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.564+0000 7f912f353700 1 -- 172.21.15.6:0/964206472 shutdown_connections 2024-03-22T06:21:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.564+0000 7f912f353700 1 --2- 172.21.15.6:0/964206472 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f911006e840 0x7f9110070cf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.564+0000 7f912f353700 1 --2- 172.21.15.6:0/964206472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91280fce70 0x7f912810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.564+0000 7f912f353700 1 --2- 172.21.15.6:0/964206472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91280fdfe0 0x7f912810de90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.564+0000 7f912f353700 1 -- 172.21.15.6:0/964206472 >> 172.21.15.6:0/964206472 conn(0x7f91280f8550 msgr2=0x7f91280f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.564+0000 7f912f353700 1 -- 172.21.15.6:0/964206472 shutdown_connections 2024-03-22T06:21:01.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.564+0000 7f912f353700 1 -- 172.21.15.6:0/964206472 wait complete. 2024-03-22T06:21:01.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.666+0000 7f69ea59e700 1 -- 172.21.15.6:0/544822571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e40fff90 msgr2=0x7f69e4102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:01.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.666+0000 7f69ea59e700 1 --2- 172.21.15.6:0/544822571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e40fff90 0x7f69e4102370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f69cc004940 tx=0x7f69cc009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:01.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 -- 172.21.15.6:0/544822571 shutdown_connections 2024-03-22T06:21:01.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 --2- 172.21.15.6:0/544822571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e40fff90 0x7f69e4102370 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 --2- 172.21.15.6:0/544822571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e4066f10 0x7f69e40ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 -- 172.21.15.6:0/544822571 >> 172.21.15.6:0/544822571 conn(0x7f69e40f7480 msgr2=0x7f69e40f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:01.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 -- 172.21.15.6:0/544822571 shutdown_connections 2024-03-22T06:21:01.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 -- 172.21.15.6:0/544822571 wait complete. 2024-03-22T06:21:01.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 Processor -- start 2024-03-22T06:21:01.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 -- start start 2024-03-22T06:21:01.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e4066f10 0x7f69e410d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:01.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e40fff90 0x7f69e410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:01.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e410e560 con 0x7f69e40fff90 2024-03-22T06:21:01.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.667+0000 7f69ea59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e4112eb0 con 0x7f69e4066f10 2024-03-22T06:21:01.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e40fff90 0x7f69e410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:01.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e40fff90 0x7f69e410ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53372/0 (socket says 172.21.15.6:53372) 2024-03-22T06:21:01.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e37fe700 1 -- 172.21.15.6:0/1596714537 learned_addr learned my addr 172.21.15.6:0/1596714537 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e3fff700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e4066f10 0x7f69e410d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e37fe700 1 -- 172.21.15.6:0/1596714537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e4066f10 msgr2=0x7f69e410d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e37fe700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e4066f10 0x7f69e410d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e37fe700 1 -- 172.21.15.6:0/1596714537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc005040 con 0x7f69e40fff90 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e3fff700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e4066f10 0x7f69e410d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e37fe700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e40fff90 0x7f69e410ded0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00b3b0 tx=0x7f69cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e0ff9700 1 -- 172.21.15.6:0/1596714537 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69cc00e040 con 0x7f69e40fff90 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e0ff9700 1 -- 172.21.15.6:0/1596714537 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69cc00bcc0 con 0x7f69e40fff90 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69ea59e700 1 -- 172.21.15.6:0/1596714537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e41130b0 con 0x7f69e40fff90 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69e0ff9700 1 -- 172.21.15.6:0/1596714537 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69cc01c630 con 0x7f69e40fff90 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.668+0000 7f69ea59e700 1 -- 172.21.15.6:0/1596714537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e4113620 con 0x7f69e40fff90 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.669+0000 7f69ea59e700 1 -- 172.21.15.6:0/1596714537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e404c710 con 0x7f69e40fff90 2024-03-22T06:21:01.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.670+0000 7f69e0ff9700 1 -- 172.21.15.6:0/1596714537 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f69cc013980 con 0x7f69e40fff90 2024-03-22T06:21:01.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.670+0000 7f69e0ff9700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f69d006e840 0x7f69d0070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:01.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.671+0000 7f69e3fff700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f69d006e840 0x7f69d0070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:01.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.671+0000 7f69e0ff9700 1 -- 172.21.15.6:0/1596714537 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f69cc08f0a0 con 0x7f69e40fff90 2024-03-22T06:21:01.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.671+0000 7f69e3fff700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f69d006e840 0x7f69d0070cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f69d4006fd0 tx=0x7f69d4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:01.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.675+0000 7f69e0ff9700 1 -- 172.21.15.6:0/1596714537 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f69cc05bb70 con 0x7f69e40fff90 2024-03-22T06:21:01.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.967+0000 7f69ea59e700 1 -- 172.21.15.6:0/1596714537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f69e4105980 con 0x7f69e40fff90 2024-03-22T06:21:01.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.967+0000 7f69e0ff9700 1 -- 172.21.15.6:0/1596714537 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1107 (secure 0 0 0) 0x7f69cc05b700 con 0x7f69e40fff90 2024-03-22T06:21:01.968 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s); Degraded data redundancy: 39/219 objects degraded (17.808%), 14 pgs degraded 2024-03-22T06:21:01.968 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:21:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:21:01.968 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 39/219 objects degraded (17.808%), 14 pgs degraded 2024-03-22T06:21:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-03-22T06:21:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-03-22T06:21:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.5 is active+undersized+degraded, acting [2,4] 2024-03-22T06:21:01.968 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-03-22T06:21:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-03-22T06:21:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-03-22T06:21:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-03-22T06:21:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.e is active+undersized+degraded, acting [2,1] 2024-03-22T06:21:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-03-22T06:21:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-03-22T06:21:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-03-22T06:21:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-03-22T06:21:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-03-22T06:21:01.969 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-03-22T06:21:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.969+0000 7f69ea59e700 1 -- 172.21.15.6:0/1596714537 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f69d006e840 msgr2=0x7f69d0070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.969+0000 7f69ea59e700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f69d006e840 0x7f69d0070cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f69d4006fd0 tx=0x7f69d4008040 comp rx=0 tx=0).stop 2024-03-22T06:21:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.969+0000 7f69ea59e700 1 -- 172.21.15.6:0/1596714537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e40fff90 msgr2=0x7f69e410ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.969+0000 7f69ea59e700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e40fff90 0x7f69e410ded0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00b3b0 tx=0x7f69cc009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.969+0000 7f69ea59e700 1 -- 172.21.15.6:0/1596714537 shutdown_connections 2024-03-22T06:21:01.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.969+0000 7f69ea59e700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f69d006e840 0x7f69d0070cf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.969+0000 7f69ea59e700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e4066f10 0x7f69e410d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.969+0000 7f69ea59e700 1 --2- 172.21.15.6:0/1596714537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e40fff90 0x7f69e410ded0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.969+0000 7f69ea59e700 1 -- 172.21.15.6:0/1596714537 >> 172.21.15.6:0/1596714537 conn(0x7f69e40f7480 msgr2=0x7f69e40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.969+0000 7f69ea59e700 1 -- 172.21.15.6:0/1596714537 shutdown_connections 2024-03-22T06:21:01.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:01.969+0000 7f69ea59e700 1 -- 172.21.15.6:0/1596714537 wait complete. 2024-03-22T06:21:02.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:02.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: osd.3 [v2:172.21.15.117:6800/1104353694,v1:172.21.15.117:6801/1104353694] boot 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: osdmap e80: 6 total, 6 up, 6 in 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/848446586' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T06:21:02.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:02 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1596714537' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:21:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:21:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: osd.3 [v2:172.21.15.117:6800/1104353694,v1:172.21.15.117:6801/1104353694] boot 2024-03-22T06:21:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: osdmap e80: 6 total, 6 up, 6 in 2024-03-22T06:21:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T06:21:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/848446586' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:21:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:02.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:21:02.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:21:02.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:02.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:21:02.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:02.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:02.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:02.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T06:21:02.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:02 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1596714537' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:21:03.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:03 smithi006 ceph-mon[94482]: pgmap v144: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:21:03.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:03 smithi006 ceph-mon[94482]: pgmap v145: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:21:03.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:03 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T06:21:03.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:03 smithi006 ceph-mon[94482]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-03-22T06:21:03.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:03 smithi006 ceph-mon[94482]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:03.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:03 smithi006 ceph-mon[94482]: osdmap e81: 6 total, 6 up, 6 in 2024-03-22T06:21:03.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:03 smithi117 ceph-mon[82126]: pgmap v144: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:21:03.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:03 smithi117 ceph-mon[82126]: pgmap v145: 65 pgs: 19 active+undersized, 14 active+undersized+degraded, 32 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-03-22T06:21:03.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:03 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T06:21:03.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:03 smithi117 ceph-mon[82126]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-03-22T06:21:03.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:03 smithi117 ceph-mon[82126]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:03.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:03 smithi117 ceph-mon[82126]: osdmap e81: 6 total, 6 up, 6 in 2024-03-22T06:21:04.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:04 smithi117 ceph-mon[82126]: Health check update: Degraded data redundancy: 29/219 objects degraded (13.242%), 10 pgs degraded (PG_DEGRADED) 2024-03-22T06:21:04.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:04 smithi006 ceph-mon[94482]: Health check update: Degraded data redundancy: 29/219 objects degraded (13.242%), 10 pgs degraded (PG_DEGRADED) 2024-03-22T06:21:05.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:05 smithi117 ceph-mon[82126]: pgmap v147: 65 pgs: 10 active+undersized, 10 active+undersized+degraded, 45 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-03-22T06:21:05.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:05 smithi006 ceph-mon[94482]: pgmap v147: 65 pgs: 10 active+undersized, 10 active+undersized+degraded, 45 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-03-22T06:21:07.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:07 smithi117 ceph-mon[82126]: pgmap v148: 65 pgs: 2 active+undersized, 4 active+undersized+degraded, 59 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 12/219 objects degraded (5.479%) 2024-03-22T06:21:07.527 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:07 smithi006 ceph-mon[94482]: pgmap v148: 65 pgs: 2 active+undersized, 4 active+undersized+degraded, 59 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 12/219 objects degraded (5.479%) 2024-03-22T06:21:08.754 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:08.754 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:21:08.754 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:08 smithi006 ceph-mon[94482]: pgmap v149: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:08.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:08.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:21:08.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:08 smithi117 ceph-mon[82126]: pgmap v149: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:09.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:09 smithi117 ceph-mon[82126]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/219 objects degraded (5.479%), 4 pgs degraded) 2024-03-22T06:21:09.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:09 smithi006 ceph-mon[94482]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/219 objects degraded (5.479%), 4 pgs degraded) 2024-03-22T06:21:10.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:10 smithi117 ceph-mon[82126]: pgmap v150: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:10.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:10 smithi006 ceph-mon[94482]: pgmap v150: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:12.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:12 smithi117 ceph-mon[82126]: pgmap v151: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:12.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:12 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:12.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:12 smithi006 ceph-mon[94482]: pgmap v151: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:12.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:12 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:14.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:14 smithi117 ceph-mon[82126]: pgmap v152: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:14.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:14 smithi006 ceph-mon[94482]: pgmap v152: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:16.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:16 smithi117 ceph-mon[82126]: pgmap v153: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:16.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:16 smithi006 ceph-mon[94482]: pgmap v153: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:17.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:17 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T06:21:17.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:17 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T06:21:17.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:17 smithi117 ceph-mon[82126]: Upgrade: osd.4 is safe to restart 2024-03-22T06:21:17.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:17 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T06:21:17.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:17 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T06:21:17.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:17 smithi006 ceph-mon[94482]: Upgrade: osd.4 is safe to restart 2024-03-22T06:21:19.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:19 smithi117 ceph-mon[82126]: pgmap v154: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:19.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:19 smithi117 ceph-mon[82126]: Upgrade: Updating osd.4 2024-03-22T06:21:19.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:19 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:19.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:19 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T06:21:19.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:19 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:21:19.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:19 smithi117 ceph-mon[82126]: Deploying daemon osd.4 on smithi117 2024-03-22T06:21:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:19 smithi006 ceph-mon[94482]: pgmap v154: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:19 smithi006 ceph-mon[94482]: Upgrade: Updating osd.4 2024-03-22T06:21:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:19 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:19 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T06:21:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:19 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:21:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:19 smithi006 ceph-mon[94482]: Deploying daemon osd.4 on smithi117 2024-03-22T06:21:21.234 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:21 smithi117 ceph-mon[82126]: pgmap v155: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:21.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:21 smithi006 ceph-mon[94482]: pgmap v155: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:21.488 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:21 smithi117 systemd[1]: Stopping Ceph osd.4 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:21:21.755 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:21 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[48526]: 2024-03-22T06:21:21.514+0000 7f766f3eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:21:21.756 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:21 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[48526]: 2024-03-22T06:21:21.514+0000 7f766f3eb700 -1 osd.4 81 *** Got signal Terminated *** 2024-03-22T06:21:21.756 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:21 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[48526]: 2024-03-22T06:21:21.514+0000 7f766f3eb700 -1 osd.4 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:21:22.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:22 smithi006 ceph-mon[94482]: osd.4 marked itself down and dead 2024-03-22T06:21:22.477 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:22 smithi117 ceph-mon[82126]: osd.4 marked itself down and dead 2024-03-22T06:21:22.756 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:22 smithi117 podman[96185]: 2024-03-22 06:21:22.476885353 +0000 UTC m=+1.072473036 container died e2021dbf5ac65ab974d528e57f4a2098f90422b7862203c140b7446a8505d237 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=quincy-9c4e0c3, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-22T06:21:23.256 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:22 smithi117 podman[96185]: 2024-03-22 06:21:22.790392323 +0000 UTC m=+1.385980003 container cleanup e2021dbf5ac65ab974d528e57f4a2098f90422b7862203c140b7446a8505d237 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T06:21:23.256 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:22 smithi117 bash[96185]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4 2024-03-22T06:21:23.256 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:22 smithi117 podman[96204]: 2024-03-22 06:21:22.991700372 +0000 UTC m=+0.518572387 container remove e2021dbf5ac65ab974d528e57f4a2098f90422b7862203c140b7446a8505d237 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:21:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:23 smithi117 ceph-mon[82126]: pgmap v156: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:23 smithi117 ceph-mon[82126]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:21:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:23 smithi117 ceph-mon[82126]: osdmap e82: 6 total, 5 up, 6 in 2024-03-22T06:21:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:21:23.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:23 smithi006 ceph-mon[94482]: pgmap v156: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:23.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:23 smithi006 ceph-mon[94482]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:21:23.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:23 smithi006 ceph-mon[94482]: osdmap e82: 6 total, 5 up, 6 in 2024-03-22T06:21:23.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:23.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:21:23.736 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:23 smithi117 podman[96296]: 2024-03-22 06:21:23.325592526 +0000 UTC m=+0.017775596 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:21:23.736 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:23 smithi117 podman[96296]: 2024-03-22 06:21:23.451308723 +0000 UTC m=+0.143491815 container create 929c362cf95f4d46cda0d5a632588158f7998bb64db923b7c929ac9fc66f17b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-22T06:21:23.737 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:23 smithi117 podman[96296]: 2024-03-22 06:21:23.662016442 +0000 UTC m=+0.354199492 container init 929c362cf95f4d46cda0d5a632588158f7998bb64db923b7c929ac9fc66f17b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-22T06:21:23.737 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:23 smithi117 podman[96296]: 2024-03-22 06:21:23.665037548 +0000 UTC m=+0.357220608 container start 929c362cf95f4d46cda0d5a632588158f7998bb64db923b7c929ac9fc66f17b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-22T06:21:23.737 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:23 smithi117 podman[96296]: 2024-03-22 06:21:23.73690069 +0000 UTC m=+0.429083741 container attach 929c362cf95f4d46cda0d5a632588158f7998bb64db923b7c929ac9fc66f17b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-22T06:21:24.006 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:23 smithi117 podman[96296]: 2024-03-22 06:21:23.951814623 +0000 UTC m=+0.643997678 container died 929c362cf95f4d46cda0d5a632588158f7998bb64db923b7c929ac9fc66f17b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:21:24.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:24 smithi117 ceph-mon[82126]: osdmap e83: 6 total, 5 up, 6 in 2024-03-22T06:21:24.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:24 smithi006 ceph-mon[94482]: osdmap e83: 6 total, 5 up, 6 in 2024-03-22T06:21:24.932 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:24 smithi117 podman[96322]: 2024-03-22 06:21:24.506885627 +0000 UTC m=+0.545993137 container remove 929c362cf95f4d46cda0d5a632588158f7998bb64db923b7c929ac9fc66f17b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_BRANCH=HEAD) 2024-03-22T06:21:24.932 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:24 smithi117 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-22T06:21:24.932 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:24 smithi117 systemd[1]: Stopped Ceph osd.4 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:21:24.932 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:24 smithi117 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.4.service: Consumed 1min 642ms CPU time. 2024-03-22T06:21:24.933 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:24 smithi117 systemd[1]: Starting Ceph osd.4 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:21:25.243 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:25 smithi117 ceph-mon[82126]: pgmap v159: 65 pgs: 3 active+undersized, 2 active+undersized+degraded, 8 peering, 2 stale+active+clean, 50 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 5/219 objects degraded (2.283%) 2024-03-22T06:21:25.243 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:25 smithi117 ceph-mon[82126]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-22T06:21:25.243 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:25 smithi117 ceph-mon[82126]: Health check failed: Degraded data redundancy: 5/219 objects degraded (2.283%), 2 pgs degraded (PG_DEGRADED) 2024-03-22T06:21:25.244 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:24 smithi117 podman[96420]: 2024-03-22 06:21:24.834659655 +0000 UTC m=+0.017307413 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:21:25.244 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:24 smithi117 podman[96420]: 2024-03-22 06:21:24.949671838 +0000 UTC m=+0.132319615 container create 2b0758f6775790a70fad369a65e72a7fab93ce6e67d17040184118ebf00a942d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:21:25.244 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 podman[96420]: 2024-03-22 06:21:25.169054029 +0000 UTC m=+0.351701760 container init 2b0758f6775790a70fad369a65e72a7fab93ce6e67d17040184118ebf00a942d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-22T06:21:25.244 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 podman[96420]: 2024-03-22 06:21:25.172214383 +0000 UTC m=+0.354862114 container start 2b0758f6775790a70fad369a65e72a7fab93ce6e67d17040184118ebf00a942d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-22T06:21:25.244 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 podman[96420]: 2024-03-22 06:21:25.243838389 +0000 UTC m=+0.426486132 container attach 2b0758f6775790a70fad369a65e72a7fab93ce6e67d17040184118ebf00a942d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-22T06:21:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:25 smithi006 ceph-mon[94482]: pgmap v159: 65 pgs: 3 active+undersized, 2 active+undersized+degraded, 8 peering, 2 stale+active+clean, 50 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 5/219 objects degraded (2.283%) 2024-03-22T06:21:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:25 smithi006 ceph-mon[94482]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-22T06:21:25.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:25 smithi006 ceph-mon[94482]: Health check failed: Degraded data redundancy: 5/219 objects degraded (2.283%), 2 pgs degraded (PG_DEGRADED) 2024-03-22T06:21:25.952 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate[96435]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T06:21:25.952 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate[96435]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T06:21:25.952 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 bash[96420]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T06:21:25.952 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 bash[96420]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T06:21:25.952 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 bash[96420]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T06:21:25.952 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 bash[96420]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T06:21:25.952 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 bash[96420]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-22T06:21:25.952 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 bash[96420]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T06:21:25.952 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 bash[96420]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-22T06:21:25.953 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate[96435]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T06:21:25.953 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate[96435]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T06:21:25.953 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate[96435]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-22T06:21:25.953 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate[96435]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T06:21:25.953 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate[96435]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-22T06:21:25.953 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:25 smithi117 podman[96420]: 2024-03-22 06:21:25.633528582 +0000 UTC m=+0.816176314 container died 2b0758f6775790a70fad369a65e72a7fab93ce6e67d17040184118ebf00a942d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-22T06:21:26.255 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:26 smithi117 podman[96560]: 2024-03-22 06:21:26.164143537 +0000 UTC m=+0.523401144 container remove 2b0758f6775790a70fad369a65e72a7fab93ce6e67d17040184118ebf00a942d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:21:26.634 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:26 smithi117 podman[96621]: 2024-03-22 06:21:26.276639102 +0000 UTC m=+0.018322316 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:21:26.634 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:26 smithi117 podman[96621]: 2024-03-22 06:21:26.381734278 +0000 UTC m=+0.123417484 container create 1950fc21bd2e9f45ba5fb986f27f31def11ba958a84b6a1d9a23dbdaf3d61c2d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-22T06:21:26.634 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:26 smithi117 podman[96621]: 2024-03-22 06:21:26.611366416 +0000 UTC m=+0.353049632 container init 1950fc21bd2e9f45ba5fb986f27f31def11ba958a84b6a1d9a23dbdaf3d61c2d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-22T06:21:26.634 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:26 smithi117 podman[96621]: 2024-03-22 06:21:26.614325673 +0000 UTC m=+0.356008881 container start 1950fc21bd2e9f45ba5fb986f27f31def11ba958a84b6a1d9a23dbdaf3d61c2d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-22T06:21:27.005 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:26 smithi117 ceph-osd[96640]: -- 172.21.15.117:0/1565250860 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559e59b05680 con 0x559e59acf000 2024-03-22T06:21:27.006 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:26 smithi117 bash[96621]: 1950fc21bd2e9f45ba5fb986f27f31def11ba958a84b6a1d9a23dbdaf3d61c2d 2024-03-22T06:21:27.006 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:26 smithi117 systemd[1]: Started Ceph osd.4 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:21:27.506 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:27 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:21:27.161+0000 7fd974a9d7c0 -1 Falling back to public interface 2024-03-22T06:21:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:27 smithi117 ceph-mon[82126]: pgmap v160: 65 pgs: 9 active+undersized, 5 active+undersized+degraded, 8 peering, 1 stale+active+clean, 42 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17/219 objects degraded (7.763%) 2024-03-22T06:21:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:27 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:27 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:27.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:27 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:21:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:27 smithi006 ceph-mon[94482]: pgmap v160: 65 pgs: 9 active+undersized, 5 active+undersized+degraded, 8 peering, 1 stale+active+clean, 42 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17/219 objects degraded (7.763%) 2024-03-22T06:21:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:27 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:27 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:27.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:27 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:21:28.354 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:28 smithi117 ceph-mon[82126]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-22T06:21:28.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:28 smithi006 ceph-mon[94482]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-22T06:21:29.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:29 smithi117 ceph-mon[82126]: pgmap v161: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:29.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:29 smithi006 ceph-mon[94482]: pgmap v161: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:30.921 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:30 smithi117 ceph-mon[82126]: pgmap v162: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:30.921 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:30 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:30.921 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:30 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:31.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:30 smithi006 ceph-mon[94482]: pgmap v162: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:31.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:30 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:31.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:30 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:32.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:31 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:32.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:31 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:32.006 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:31 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:21:31.725+0000 7fd974a9d7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-22T06:21:32.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:31 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:32.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:31 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:32.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.086+0000 7f241ad8e700 1 -- 172.21.15.6:0/1390300086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24140fcf80 msgr2=0x7f24140fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:32.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.086+0000 7f241ad8e700 1 --2- 172.21.15.6:0/1390300086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24140fcf80 0x7f24140fd3f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2400004940 tx=0x7f2400009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:32.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.086+0000 7f241ad8e700 1 -- 172.21.15.6:0/1390300086 shutdown_connections 2024-03-22T06:21:32.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.086+0000 7f241ad8e700 1 --2- 172.21.15.6:0/1390300086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24140fcf80 0x7f24140fd3f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.086+0000 7f241ad8e700 1 --2- 172.21.15.6:0/1390300086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24140fbcd0 0x7f24140fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.086+0000 7f241ad8e700 1 -- 172.21.15.6:0/1390300086 >> 172.21.15.6:0/1390300086 conn(0x7f24140f7430 msgr2=0x7f24140f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:32.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.087+0000 7f241ad8e700 1 -- 172.21.15.6:0/1390300086 shutdown_connections 2024-03-22T06:21:32.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.087+0000 7f241ad8e700 1 -- 172.21.15.6:0/1390300086 wait complete. 2024-03-22T06:21:32.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.087+0000 7f241ad8e700 1 Processor -- start 2024-03-22T06:21:32.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.087+0000 7f241ad8e700 1 -- start start 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.087+0000 7f241ad8e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24140fbcd0 0x7f241410d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.087+0000 7f241ad8e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24140fcf80 0x7f241410de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.087+0000 7f241ad8e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241410e470 con 0x7f24140fbcd0 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.087+0000 7f241ad8e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241410e5e0 con 0x7f24140fcf80 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f2418b2a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24140fbcd0 0x7f241410d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f2418b2a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24140fbcd0 0x7f241410d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42862/0 (socket says 172.21.15.6:42862) 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f2418b2a700 1 -- 172.21.15.6:0/2449366496 learned_addr learned my addr 172.21.15.6:0/2449366496 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f2413fff700 1 --2- 172.21.15.6:0/2449366496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24140fcf80 0x7f241410de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f2418b2a700 1 -- 172.21.15.6:0/2449366496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24140fcf80 msgr2=0x7f241410de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f2418b2a700 1 --2- 172.21.15.6:0/2449366496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24140fcf80 0x7f241410de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f2418b2a700 1 -- 172.21.15.6:0/2449366496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2400005040 con 0x7f24140fbcd0 2024-03-22T06:21:32.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f2418b2a700 1 --2- 172.21.15.6:0/2449366496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24140fbcd0 0x7f241410d930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2408002a50 tx=0x7f24080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:32.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f24117fa700 1 -- 172.21.15.6:0/2449366496 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2408007cc0 con 0x7f24140fbcd0 2024-03-22T06:21:32.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f241ad8e700 1 -- 172.21.15.6:0/2449366496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2414112fe0 con 0x7f24140fbcd0 2024-03-22T06:21:32.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f24117fa700 1 -- 172.21.15.6:0/2449366496 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f240800f040 con 0x7f24140fbcd0 2024-03-22T06:21:32.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f24117fa700 1 -- 172.21.15.6:0/2449366496 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f240800da30 con 0x7f24140fbcd0 2024-03-22T06:21:32.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.088+0000 7f241ad8e700 1 -- 172.21.15.6:0/2449366496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2414113570 con 0x7f24140fbcd0 2024-03-22T06:21:32.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.090+0000 7f241ad8e700 1 -- 172.21.15.6:0/2449366496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241404c710 con 0x7f24140fbcd0 2024-03-22T06:21:32.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.090+0000 7f24117fa700 1 -- 172.21.15.6:0/2449366496 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f240800db90 con 0x7f24140fbcd0 2024-03-22T06:21:32.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.091+0000 7f24117fa700 1 --2- 172.21.15.6:0/2449366496 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f240406e840 0x7f2404070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:32.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.091+0000 7f2413fff700 1 --2- 172.21.15.6:0/2449366496 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f240406e840 0x7f2404070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:32.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.091+0000 7f24117fa700 1 -- 172.21.15.6:0/2449366496 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f240808e1f0 con 0x7f24140fbcd0 2024-03-22T06:21:32.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.091+0000 7f2413fff700 1 --2- 172.21.15.6:0/2449366496 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f240406e840 0x7f2404070cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2400009fd0 tx=0x7f2400009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:32.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.096+0000 7f24117fa700 1 -- 172.21.15.6:0/2449366496 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f240805ac40 con 0x7f24140fbcd0 2024-03-22T06:21:32.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.323+0000 7f241ad8e700 1 -- 172.21.15.6:0/2449366496 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f241405e400 con 0x7f240406e840 2024-03-22T06:21:32.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.324+0000 7f24117fa700 1 -- 172.21.15.6:0/2449366496 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f241405e400 con 0x7f240406e840 2024-03-22T06:21:32.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.326+0000 7f241ad8e700 1 -- 172.21.15.6:0/2449366496 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f240406e840 msgr2=0x7f2404070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:32.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.326+0000 7f241ad8e700 1 --2- 172.21.15.6:0/2449366496 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f240406e840 0x7f2404070cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2400009fd0 tx=0x7f2400009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:32.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.326+0000 7f241ad8e700 1 -- 172.21.15.6:0/2449366496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24140fbcd0 msgr2=0x7f241410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:32.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.326+0000 7f241ad8e700 1 --2- 172.21.15.6:0/2449366496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24140fbcd0 0x7f241410d930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2408002a50 tx=0x7f24080046b0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.326+0000 7f241ad8e700 1 -- 172.21.15.6:0/2449366496 shutdown_connections 2024-03-22T06:21:32.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.326+0000 7f241ad8e700 1 --2- 172.21.15.6:0/2449366496 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f240406e840 0x7f2404070cf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.326+0000 7f241ad8e700 1 --2- 172.21.15.6:0/2449366496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24140fbcd0 0x7f241410d930 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.326+0000 7f241ad8e700 1 --2- 172.21.15.6:0/2449366496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24140fcf80 0x7f241410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.326+0000 7f241ad8e700 1 -- 172.21.15.6:0/2449366496 >> 172.21.15.6:0/2449366496 conn(0x7f24140f7430 msgr2=0x7f24140667f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:32.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.326+0000 7f241ad8e700 1 -- 172.21.15.6:0/2449366496 shutdown_connections 2024-03-22T06:21:32.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.326+0000 7f241ad8e700 1 -- 172.21.15.6:0/2449366496 wait complete. 2024-03-22T06:21:32.337 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:21:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.434+0000 7f818177b700 1 -- 172.21.15.6:0/1570793173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f817c0feda0 msgr2=0x7f817c0faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.434+0000 7f818177b700 1 --2- 172.21.15.6:0/1570793173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f817c0feda0 0x7f817c0faee0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8170004940 tx=0x7f8170009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.434+0000 7f818177b700 1 -- 172.21.15.6:0/1570793173 shutdown_connections 2024-03-22T06:21:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.434+0000 7f818177b700 1 --2- 172.21.15.6:0/1570793173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f817c0feda0 0x7f817c0faee0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.434+0000 7f818177b700 1 --2- 172.21.15.6:0/1570793173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c0fe450 0x7f817c0fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.434+0000 7f818177b700 1 -- 172.21.15.6:0/1570793173 >> 172.21.15.6:0/1570793173 conn(0x7f817c0f6c10 msgr2=0x7f817c0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.434+0000 7f818177b700 1 -- 172.21.15.6:0/1570793173 shutdown_connections 2024-03-22T06:21:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.434+0000 7f818177b700 1 -- 172.21.15.6:0/1570793173 wait complete. 2024-03-22T06:21:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f818177b700 1 Processor -- start 2024-03-22T06:21:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f818177b700 1 -- start start 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f818177b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f817c0fe450 0x7f817c10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f818177b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c0feda0 0x7f817c10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f818177b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c10e470 con 0x7f817c0fe450 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f818177b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c10e5e0 con 0x7f817c0feda0 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f817affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f817c0fe450 0x7f817c10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f817affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f817c0fe450 0x7f817c10d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42870/0 (socket says 172.21.15.6:42870) 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f817affd700 1 -- 172.21.15.6:0/1784168602 learned_addr learned my addr 172.21.15.6:0/1784168602 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f817a7fc700 1 --2- 172.21.15.6:0/1784168602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c0feda0 0x7f817c10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f817affd700 1 -- 172.21.15.6:0/1784168602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c0feda0 msgr2=0x7f817c10de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f817affd700 1 --2- 172.21.15.6:0/1784168602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c0feda0 0x7f817c10de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f817affd700 1 -- 172.21.15.6:0/1784168602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170005040 con 0x7f817c0fe450 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.435+0000 7f817affd700 1 --2- 172.21.15.6:0/1784168602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f817c0fe450 0x7f817c10d930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f816c002a50 tx=0x7f816c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.436+0000 7f81637fe700 1 -- 172.21.15.6:0/1784168602 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f816c007cc0 con 0x7f817c0fe450 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.436+0000 7f81637fe700 1 -- 172.21.15.6:0/1784168602 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f816c00f040 con 0x7f817c0fe450 2024-03-22T06:21:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.436+0000 7f818177b700 1 -- 172.21.15.6:0/1784168602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c112fe0 con 0x7f817c0fe450 2024-03-22T06:21:32.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.436+0000 7f81637fe700 1 -- 172.21.15.6:0/1784168602 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f816c00da30 con 0x7f817c0fe450 2024-03-22T06:21:32.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.436+0000 7f818177b700 1 -- 172.21.15.6:0/1784168602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817c113570 con 0x7f817c0fe450 2024-03-22T06:21:32.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.437+0000 7f818177b700 1 -- 172.21.15.6:0/1784168602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817c04c710 con 0x7f817c0fe450 2024-03-22T06:21:32.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.437+0000 7f81637fe700 1 -- 172.21.15.6:0/1784168602 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f816c00dbb0 con 0x7f817c0fe450 2024-03-22T06:21:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.438+0000 7f81637fe700 1 --2- 172.21.15.6:0/1784168602 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f816406e840 0x7f8164070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.438+0000 7f817a7fc700 1 --2- 172.21.15.6:0/1784168602 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f816406e840 0x7f8164070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.438+0000 7f81637fe700 1 -- 172.21.15.6:0/1784168602 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f816c08e100 con 0x7f817c0fe450 2024-03-22T06:21:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.438+0000 7f817a7fc700 1 --2- 172.21.15.6:0/1784168602 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f816406e840 0x7f8164070cf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f817000b970 tx=0x7f8170009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:32.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.442+0000 7f81637fe700 1 -- 172.21.15.6:0/1784168602 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f816c05abd0 con 0x7f817c0fe450 2024-03-22T06:21:32.530 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:32 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:21:32.269+0000 7fd974a9d7c0 -1 osd.4 81 log_to_monitors true 2024-03-22T06:21:32.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.668+0000 7f818177b700 1 -- 172.21.15.6:0/1784168602 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f817c1024f0 con 0x7f816406e840 2024-03-22T06:21:32.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.669+0000 7f81637fe700 1 -- 172.21.15.6:0/1784168602 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f817c1024f0 con 0x7f816406e840 2024-03-22T06:21:32.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.671+0000 7f818177b700 1 -- 172.21.15.6:0/1784168602 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f816406e840 msgr2=0x7f8164070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:32.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.671+0000 7f818177b700 1 --2- 172.21.15.6:0/1784168602 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f816406e840 0x7f8164070cf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f817000b970 tx=0x7f8170009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:32.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.671+0000 7f818177b700 1 -- 172.21.15.6:0/1784168602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f817c0fe450 msgr2=0x7f817c10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:32.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.671+0000 7f818177b700 1 --2- 172.21.15.6:0/1784168602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f817c0fe450 0x7f817c10d930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f816c002a50 tx=0x7f816c0046b0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.671+0000 7f818177b700 1 -- 172.21.15.6:0/1784168602 shutdown_connections 2024-03-22T06:21:32.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.671+0000 7f818177b700 1 --2- 172.21.15.6:0/1784168602 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f816406e840 0x7f8164070cf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.671+0000 7f818177b700 1 --2- 172.21.15.6:0/1784168602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f817c0fe450 0x7f817c10d930 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.671+0000 7f818177b700 1 --2- 172.21.15.6:0/1784168602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f817c0feda0 0x7f817c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.671+0000 7f818177b700 1 -- 172.21.15.6:0/1784168602 >> 172.21.15.6:0/1784168602 conn(0x7f817c0f6c10 msgr2=0x7f817c0f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:32.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.672+0000 7f818177b700 1 -- 172.21.15.6:0/1784168602 shutdown_connections 2024-03-22T06:21:32.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.672+0000 7f818177b700 1 -- 172.21.15.6:0/1784168602 wait complete. 2024-03-22T06:21:32.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.776+0000 7fd535007700 1 -- 172.21.15.6:0/3416227159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300fbd00 msgr2=0x7fd5300fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:32.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.776+0000 7fd535007700 1 --2- 172.21.15.6:0/3416227159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300fbd00 0x7fd5300fc110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd528004940 tx=0x7fd528009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:32.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.776+0000 7fd535007700 1 -- 172.21.15.6:0/3416227159 shutdown_connections 2024-03-22T06:21:32.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.776+0000 7fd535007700 1 --2- 172.21.15.6:0/3416227159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5300fcfb0 0x7fd5300fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.776+0000 7fd535007700 1 --2- 172.21.15.6:0/3416227159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300fbd00 0x7fd5300fc110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.776+0000 7fd535007700 1 -- 172.21.15.6:0/3416227159 >> 172.21.15.6:0/3416227159 conn(0x7fd5300f7480 msgr2=0x7fd5300f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:32.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.776+0000 7fd535007700 1 -- 172.21.15.6:0/3416227159 shutdown_connections 2024-03-22T06:21:32.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.776+0000 7fd535007700 1 -- 172.21.15.6:0/3416227159 wait complete. 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.776+0000 7fd535007700 1 Processor -- start 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd535007700 1 -- start start 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd535007700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5300fbd00 0x7fd5300ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd535007700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300fcfb0 0x7fd5300fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd535007700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd530100240 con 0x7fd5300fcfb0 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd535007700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300fe830 con 0x7fd5300fbd00 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd52e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300fcfb0 0x7fd5300fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd52e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300fcfb0 0x7fd5300fe2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42894/0 (socket says 172.21.15.6:42894) 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd52e59c700 1 -- 172.21.15.6:0/167359536 learned_addr learned my addr 172.21.15.6:0/167359536 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd52ed9d700 1 --2- 172.21.15.6:0/167359536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5300fbd00 0x7fd5300ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd52e59c700 1 -- 172.21.15.6:0/167359536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5300fbd00 msgr2=0x7fd5300ffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd52e59c700 1 --2- 172.21.15.6:0/167359536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5300fbd00 0x7fd5300ffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd52e59c700 1 -- 172.21.15.6:0/167359536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd528005040 con 0x7fd5300fcfb0 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd52e59c700 1 --2- 172.21.15.6:0/167359536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300fcfb0 0x7fd5300fe2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd518002a50 tx=0x7fd5180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd5277fe700 1 -- 172.21.15.6:0/167359536 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd518007cc0 con 0x7fd5300fcfb0 2024-03-22T06:21:32.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.777+0000 7fd5277fe700 1 -- 172.21.15.6:0/167359536 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd51800f040 con 0x7fd5300fcfb0 2024-03-22T06:21:32.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.778+0000 7fd535007700 1 -- 172.21.15.6:0/167359536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5300feae0 con 0x7fd5300fcfb0 2024-03-22T06:21:32.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.778+0000 7fd5277fe700 1 -- 172.21.15.6:0/167359536 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd51800da30 con 0x7fd5300fcfb0 2024-03-22T06:21:32.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.778+0000 7fd535007700 1 -- 172.21.15.6:0/167359536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5300ff0a0 con 0x7fd5300fcfb0 2024-03-22T06:21:32.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.779+0000 7fd535007700 1 -- 172.21.15.6:0/167359536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd530101b40 con 0x7fd5300fcfb0 2024-03-22T06:21:32.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.779+0000 7fd5277fe700 1 -- 172.21.15.6:0/167359536 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fd51800db90 con 0x7fd5300fcfb0 2024-03-22T06:21:32.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.779+0000 7fd5277fe700 1 --2- 172.21.15.6:0/167359536 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd51c06e840 0x7fd51c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:32.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.780+0000 7fd5277fe700 1 -- 172.21.15.6:0/167359536 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6614+0+0 (secure 0 0 0) 0x7fd51808e1e0 con 0x7fd5300fcfb0 2024-03-22T06:21:32.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.780+0000 7fd52ed9d700 1 --2- 172.21.15.6:0/167359536 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd51c06e840 0x7fd51c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:32.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.780+0000 7fd52ed9d700 1 --2- 172.21.15.6:0/167359536 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd51c06e840 0x7fd51c070cf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd52800b3b0 tx=0x7fd528009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:32.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:32.786+0000 7fd5277fe700 1 -- 172.21.15.6:0/167359536 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fd51805ac30 con 0x7fd5300fcfb0 2024-03-22T06:21:32.812 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:32 smithi117 ceph-mon[82126]: pgmap v163: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:32.812 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:32 smithi117 ceph-mon[82126]: from='osd.4 [v2:172.21.15.117:6808/208554150,v1:172.21.15.117:6809/208554150]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T06:21:32.812 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:32 smithi117 ceph-mon[82126]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T06:21:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.001+0000 7fd535007700 1 -- 172.21.15.6:0/167359536 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd53002b390 con 0x7fd51c06e840 2024-03-22T06:21:33.001 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:32 smithi006 ceph-mon[94482]: pgmap v163: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:33.001 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:32 smithi006 ceph-mon[94482]: from='osd.4 [v2:172.21.15.117:6808/208554150,v1:172.21.15.117:6809/208554150]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T06:21:33.002 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:32 smithi006 ceph-mon[94482]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T06:21:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.013+0000 7fd5277fe700 1 -- 172.21.15.6:0/167359536 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fd53002b390 con 0x7fd51c06e840 2024-03-22T06:21:33.013 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:21:33.013 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (6m) 69s ago 14m 26.2M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:21:33.013 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (3m) 69s ago 14m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:21:33.013 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (2m) 3s ago 13m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:21:33.013 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 unknown 69s ago 14m - - 2024-03-22T06:21:33.013 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (9m) 69s ago 9m 157M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:21:33.013 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (9m) 69s ago 9m 18.0M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (9m) 3s ago 9m 15.3M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (9m) 3s ago 9m 17.0M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (5m) 69s ago 15m 504M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (4m) 3s ago 13m 451M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (4m) 69s ago 16m 48.0M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (3m) 3s ago 13m 40.6M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (6m) 69s ago 14m 17.6M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (5m) 3s ago 13m 14.7M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (2m) 69s ago 12m 148M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (114s) 69s ago 12m 147M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (76s) 69s ago 11m 12.5M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 3ec1f219c19c 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (39s) 3s ago 11m 98.7M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 9cba39b6f32c 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (6s) 3s ago 11m 11.7M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1950fc21bd2e 2024-03-22T06:21:33.014 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (10m) 3s ago 10m 308M 2581M 17.2.7-726-g9c4e0c34 837124da804f ac90669ae5d2 2024-03-22T06:21:33.015 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (4m) 69s ago 14m 64.4M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:21:33.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.015+0000 7fd535007700 1 -- 172.21.15.6:0/167359536 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd51c06e840 msgr2=0x7fd51c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.015+0000 7fd535007700 1 --2- 172.21.15.6:0/167359536 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd51c06e840 0x7fd51c070cf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd52800b3b0 tx=0x7fd528009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:33.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.015+0000 7fd535007700 1 -- 172.21.15.6:0/167359536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300fcfb0 msgr2=0x7fd5300fe2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.015+0000 7fd535007700 1 --2- 172.21.15.6:0/167359536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300fcfb0 0x7fd5300fe2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd518002a50 tx=0x7fd5180046b0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.015+0000 7fd535007700 1 -- 172.21.15.6:0/167359536 shutdown_connections 2024-03-22T06:21:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.015+0000 7fd535007700 1 --2- 172.21.15.6:0/167359536 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd51c06e840 0x7fd51c070cf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.015+0000 7fd535007700 1 --2- 172.21.15.6:0/167359536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5300fbd00 0x7fd5300ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.015+0000 7fd535007700 1 --2- 172.21.15.6:0/167359536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300fcfb0 0x7fd5300fe2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.015+0000 7fd535007700 1 -- 172.21.15.6:0/167359536 >> 172.21.15.6:0/167359536 conn(0x7fd5300f7480 msgr2=0x7fd530066820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.015+0000 7fd535007700 1 -- 172.21.15.6:0/167359536 shutdown_connections 2024-03-22T06:21:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.015+0000 7fd535007700 1 -- 172.21.15.6:0/167359536 wait complete. 2024-03-22T06:21:33.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.117+0000 7f8386687700 1 -- 172.21.15.6:0/335521767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8380066ed0 msgr2=0x7f83800ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.117+0000 7f8386687700 1 --2- 172.21.15.6:0/335521767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8380066ed0 0x7f83800ffa40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8370004940 tx=0x7f8370009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:33.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.117+0000 7f8386687700 1 -- 172.21.15.6:0/335521767 shutdown_connections 2024-03-22T06:21:33.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.117+0000 7f8386687700 1 --2- 172.21.15.6:0/335521767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83800fff80 0x7f8380102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.117+0000 7f8386687700 1 --2- 172.21.15.6:0/335521767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8380066ed0 0x7f83800ffa40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.117+0000 7f8386687700 1 -- 172.21.15.6:0/335521767 >> 172.21.15.6:0/335521767 conn(0x7f83800f7450 msgr2=0x7f83800f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:33.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.117+0000 7f8386687700 1 -- 172.21.15.6:0/335521767 shutdown_connections 2024-03-22T06:21:33.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.117+0000 7f8386687700 1 -- 172.21.15.6:0/335521767 wait complete. 2024-03-22T06:21:33.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.117+0000 7f8386687700 1 Processor -- start 2024-03-22T06:21:33.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.117+0000 7f8386687700 1 -- start start 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f8386687700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8380066ed0 0x7f83800fd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f8386687700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83800fff80 0x7f83800fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f8386687700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83800fd840 con 0x7f83800fff80 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f8386687700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83800fbe80 con 0x7f8380066ed0 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f837f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83800fff80 0x7f83800fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f837f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83800fff80 0x7f83800fb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42916/0 (socket says 172.21.15.6:42916) 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f837f7fe700 1 -- 172.21.15.6:0/122479522 learned_addr learned my addr 172.21.15.6:0/122479522 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f837ffff700 1 --2- 172.21.15.6:0/122479522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8380066ed0 0x7f83800fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f837f7fe700 1 -- 172.21.15.6:0/122479522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8380066ed0 msgr2=0x7f83800fd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f837f7fe700 1 --2- 172.21.15.6:0/122479522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8380066ed0 0x7f83800fd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f837f7fe700 1 -- 172.21.15.6:0/122479522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8370005040 con 0x7f83800fff80 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f837f7fe700 1 --2- 172.21.15.6:0/122479522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83800fff80 0x7f83800fb940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f836800eb40 tx=0x7f836800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f837cff9700 1 -- 172.21.15.6:0/122479522 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8368007cc0 con 0x7f83800fff80 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f837cff9700 1 -- 172.21.15.6:0/122479522 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f836800f040 con 0x7f83800fff80 2024-03-22T06:21:33.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f8386687700 1 -- 172.21.15.6:0/122479522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83800fc100 con 0x7f83800fff80 2024-03-22T06:21:33.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.118+0000 7f837cff9700 1 -- 172.21.15.6:0/122479522 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f836800ca10 con 0x7f83800fff80 2024-03-22T06:21:33.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.119+0000 7f8386687700 1 -- 172.21.15.6:0/122479522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83800fc640 con 0x7f83800fff80 2024-03-22T06:21:33.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.119+0000 7f8386687700 1 -- 172.21.15.6:0/122479522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838004c710 con 0x7f83800fff80 2024-03-22T06:21:33.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.120+0000 7f837cff9700 1 -- 172.21.15.6:0/122479522 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f836800cc80 con 0x7f83800fff80 2024-03-22T06:21:33.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.120+0000 7f837cff9700 1 --2- 172.21.15.6:0/122479522 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f836c06e840 0x7f836c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:33.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.120+0000 7f837cff9700 1 -- 172.21.15.6:0/122479522 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f836808e0c0 con 0x7f83800fff80 2024-03-22T06:21:33.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.120+0000 7f837ffff700 1 --2- 172.21.15.6:0/122479522 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f836c06e840 0x7f836c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:33.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.121+0000 7f837ffff700 1 --2- 172.21.15.6:0/122479522 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f836c06e840 0x7f836c070cf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8370004890 tx=0x7f8370009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:33.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.125+0000 7f837cff9700 1 -- 172.21.15.6:0/122479522 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f836805ab20 con 0x7f83800fff80 2024-03-22T06:21:33.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.419+0000 7f8386687700 1 -- 172.21.15.6:0/122479522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f83801058b0 con 0x7f83800fff80 2024-03-22T06:21:33.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.420+0000 7f837cff9700 1 -- 172.21.15.6:0/122479522 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f836805a6b0 con 0x7f83800fff80 2024-03-22T06:21:33.420 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:21:33.421 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:21:33.421 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:21:33.421 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:21:33.421 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:21:33.421 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:21:33.421 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:21:33.421 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:21:33.421 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T06:21:33.421 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 4 2024-03-22T06:21:33.421 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:21:33.422 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:21:33.422 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:21:33.422 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:21:33.422 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:21:33.422 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 5, 2024-03-22T06:21:33.422 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 8 2024-03-22T06:21:33.423 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:21:33.423 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:21:33.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.422+0000 7f8386687700 1 -- 172.21.15.6:0/122479522 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f836c06e840 msgr2=0x7f836c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.422+0000 7f8386687700 1 --2- 172.21.15.6:0/122479522 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f836c06e840 0x7f836c070cf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8370004890 tx=0x7f8370009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:33.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.422+0000 7f8386687700 1 -- 172.21.15.6:0/122479522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83800fff80 msgr2=0x7f83800fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.422+0000 7f8386687700 1 --2- 172.21.15.6:0/122479522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83800fff80 0x7f83800fb940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f836800eb40 tx=0x7f836800b040 comp rx=0 tx=0).stop 2024-03-22T06:21:33.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.422+0000 7f8386687700 1 -- 172.21.15.6:0/122479522 shutdown_connections 2024-03-22T06:21:33.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.422+0000 7f8386687700 1 --2- 172.21.15.6:0/122479522 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f836c06e840 0x7f836c070cf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.422+0000 7f8386687700 1 --2- 172.21.15.6:0/122479522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8380066ed0 0x7f83800fd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.422+0000 7f8386687700 1 --2- 172.21.15.6:0/122479522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83800fff80 0x7f83800fb940 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.422+0000 7f8386687700 1 -- 172.21.15.6:0/122479522 >> 172.21.15.6:0/122479522 conn(0x7f83800f7450 msgr2=0x7f83800f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:33.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.422+0000 7f8386687700 1 -- 172.21.15.6:0/122479522 shutdown_connections 2024-03-22T06:21:33.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.422+0000 7f8386687700 1 -- 172.21.15.6:0/122479522 wait complete. 2024-03-22T06:21:33.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.525+0000 7fc6dbc49700 1 -- 172.21.15.6:0/1162063019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6d40fbce0 msgr2=0x7fc6d40fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.525+0000 7fc6dbc49700 1 --2- 172.21.15.6:0/1162063019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6d40fbce0 0x7fc6d40fc0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4004940 tx=0x7fc6c4009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:33.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.525+0000 7fc6dbc49700 1 -- 172.21.15.6:0/1162063019 shutdown_connections 2024-03-22T06:21:33.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.525+0000 7fc6dbc49700 1 --2- 172.21.15.6:0/1162063019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6d40fcf90 0x7fc6d40fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.525+0000 7fc6dbc49700 1 --2- 172.21.15.6:0/1162063019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6d40fbce0 0x7fc6d40fc0f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.525+0000 7fc6dbc49700 1 -- 172.21.15.6:0/1162063019 >> 172.21.15.6:0/1162063019 conn(0x7fc6d40f7440 msgr2=0x7fc6d40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:33.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.525+0000 7fc6dbc49700 1 -- 172.21.15.6:0/1162063019 shutdown_connections 2024-03-22T06:21:33.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.525+0000 7fc6dbc49700 1 -- 172.21.15.6:0/1162063019 wait complete. 2024-03-22T06:21:33.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6dbc49700 1 Processor -- start 2024-03-22T06:21:33.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6dbc49700 1 -- start start 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6dbc49700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6d40fbce0 0x7fc6d4066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6dbc49700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6d40fcf90 0x7fc6d4114ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6dbc49700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d410e4b0 con 0x7fc6d40fcf90 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6dbc49700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4115010 con 0x7fc6d40fbce0 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6d91e4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6d40fcf90 0x7fc6d4114ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6d91e4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6d40fcf90 0x7fc6d4114ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42928/0 (socket says 172.21.15.6:42928) 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6d91e4700 1 -- 172.21.15.6:0/3373201762 learned_addr learned my addr 172.21.15.6:0/3373201762 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6d99e5700 1 --2- 172.21.15.6:0/3373201762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6d40fbce0 0x7fc6d4066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6d91e4700 1 -- 172.21.15.6:0/3373201762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6d40fbce0 msgr2=0x7fc6d4066d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6d91e4700 1 --2- 172.21.15.6:0/3373201762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6d40fbce0 0x7fc6d4066d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.526+0000 7fc6d91e4700 1 -- 172.21.15.6:0/3373201762 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4005040 con 0x7fc6d40fcf90 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.527+0000 7fc6d91e4700 1 --2- 172.21.15.6:0/3373201762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6d40fcf90 0x7fc6d4114ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0002ab0 tx=0x7fc6d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.527+0000 7fc6ca7fc700 1 -- 172.21.15.6:0/3373201762 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6d0007cc0 con 0x7fc6d40fcf90 2024-03-22T06:21:33.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.527+0000 7fc6ca7fc700 1 -- 172.21.15.6:0/3373201762 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6d000f040 con 0x7fc6d40fcf90 2024-03-22T06:21:33.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.527+0000 7fc6dbc49700 1 -- 172.21.15.6:0/3373201762 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d4115210 con 0x7fc6d40fcf90 2024-03-22T06:21:33.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.527+0000 7fc6ca7fc700 1 -- 172.21.15.6:0/3373201762 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6d000db60 con 0x7fc6d40fcf90 2024-03-22T06:21:33.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.527+0000 7fc6dbc49700 1 -- 172.21.15.6:0/3373201762 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d4115780 con 0x7fc6d40fcf90 2024-03-22T06:21:33.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.528+0000 7fc6dbc49700 1 -- 172.21.15.6:0/3373201762 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d404c710 con 0x7fc6d40fcf90 2024-03-22T06:21:33.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.528+0000 7fc6ca7fc700 1 -- 172.21.15.6:0/3373201762 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fc6d000dcc0 con 0x7fc6d40fcf90 2024-03-22T06:21:33.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.529+0000 7fc6ca7fc700 1 --2- 172.21.15.6:0/3373201762 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc6c006e840 0x7fc6c0070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:33.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.529+0000 7fc6ca7fc700 1 -- 172.21.15.6:0/3373201762 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6614+0+0 (secure 0 0 0) 0x7fc6d008e010 con 0x7fc6d40fcf90 2024-03-22T06:21:33.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.529+0000 7fc6d99e5700 1 --2- 172.21.15.6:0/3373201762 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc6c006e840 0x7fc6c0070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:33.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.529+0000 7fc6d99e5700 1 --2- 172.21.15.6:0/3373201762 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc6c006e840 0x7fc6c0070cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009fd0 tx=0x7fc6c4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:33.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.533+0000 7fc6ca7fc700 1 -- 172.21.15.6:0/3373201762 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fc6d005aae0 con 0x7fc6d40fcf90 2024-03-22T06:21:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.783+0000 7fc6dbc49700 1 -- 172.21.15.6:0/3373201762 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc6d41047d0 con 0x7fc6d40fcf90 2024-03-22T06:21:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.783+0000 7fc6ca7fc700 1 -- 172.21.15.6:0/3373201762 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fc6d005a670 con 0x7fc6d40fcf90 2024-03-22T06:21:33.784 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:21:33.784 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:21:33.784 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:21:33.784 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:21:33.784 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:21:33.784 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:21:33.784 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:21:33.785 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:21:33.786 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:21:33.786 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:21:33.786 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:21:33.786 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:21:33.786 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:21:33.786 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:21:33.787 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:21:33.787 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:21:33.787 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:21:33.787 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:21:33.787 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:21:33.787 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:21:33.787 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:21:33.787 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:21:33.787 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 0 members: 2024-03-22T06:21:33.787 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:21:33.788 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:21:33.788 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:21:33.788 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:21:33.788 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:21:33.788 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:21:33.788 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:21:33.788 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:21:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.786+0000 7fc6dbc49700 1 -- 172.21.15.6:0/3373201762 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc6c006e840 msgr2=0x7fc6c0070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.786+0000 7fc6dbc49700 1 --2- 172.21.15.6:0/3373201762 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc6c006e840 0x7fc6c0070cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009fd0 tx=0x7fc6c4009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.786+0000 7fc6dbc49700 1 -- 172.21.15.6:0/3373201762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6d40fcf90 msgr2=0x7fc6d4114ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.786+0000 7fc6dbc49700 1 --2- 172.21.15.6:0/3373201762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6d40fcf90 0x7fc6d4114ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0002ab0 tx=0x7fc6d00046b0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.786+0000 7fc6dbc49700 1 -- 172.21.15.6:0/3373201762 shutdown_connections 2024-03-22T06:21:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.786+0000 7fc6dbc49700 1 --2- 172.21.15.6:0/3373201762 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc6c006e840 0x7fc6c0070cf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.786+0000 7fc6dbc49700 1 --2- 172.21.15.6:0/3373201762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6d40fbce0 0x7fc6d4066d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.786+0000 7fc6dbc49700 1 --2- 172.21.15.6:0/3373201762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6d40fcf90 0x7fc6d4114ad0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.786+0000 7fc6dbc49700 1 -- 172.21.15.6:0/3373201762 >> 172.21.15.6:0/3373201762 conn(0x7fc6d40f7440 msgr2=0x7fc6d40f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.786+0000 7fc6dbc49700 1 -- 172.21.15.6:0/3373201762 shutdown_connections 2024-03-22T06:21:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.786+0000 7fc6dbc49700 1 -- 172.21.15.6:0/3373201762 wait complete. 2024-03-22T06:21:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:21:33.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.889+0000 7f6d45f7d700 1 -- 172.21.15.6:0/2735515960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d400fbd00 msgr2=0x7f6d400fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.889+0000 7f6d45f7d700 1 --2- 172.21.15.6:0/2735515960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d400fbd00 0x7f6d400fc110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d30004940 tx=0x7f6d30009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:33.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.889+0000 7f6d45f7d700 1 -- 172.21.15.6:0/2735515960 shutdown_connections 2024-03-22T06:21:33.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.889+0000 7f6d45f7d700 1 --2- 172.21.15.6:0/2735515960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d400fcfb0 0x7f6d400fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.889+0000 7f6d45f7d700 1 --2- 172.21.15.6:0/2735515960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d400fbd00 0x7f6d400fc110 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.889+0000 7f6d45f7d700 1 -- 172.21.15.6:0/2735515960 >> 172.21.15.6:0/2735515960 conn(0x7f6d400f7480 msgr2=0x7f6d400f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:33.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.889+0000 7f6d45f7d700 1 -- 172.21.15.6:0/2735515960 shutdown_connections 2024-03-22T06:21:33.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.889+0000 7f6d45f7d700 1 -- 172.21.15.6:0/2735515960 wait complete. 2024-03-22T06:21:33.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.889+0000 7f6d45f7d700 1 Processor -- start 2024-03-22T06:21:33.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.889+0000 7f6d45f7d700 1 -- start start 2024-03-22T06:21:33.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d45f7d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d400fbd00 0x7f6d4010c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d45f7d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d400fcfb0 0x7f6d4010cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d45f7d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4010d360 con 0x7f6d400fcfb0 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d45f7d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4010d4d0 con 0x7f6d400fbd00 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d3effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d400fcfb0 0x7f6d4010cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d3effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d400fcfb0 0x7f6d4010cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35280/0 (socket says 172.21.15.6:35280) 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d3effd700 1 -- 172.21.15.6:0/1652279083 learned_addr learned my addr 172.21.15.6:0/1652279083 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d3f7fe700 1 --2- 172.21.15.6:0/1652279083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d400fbd00 0x7f6d4010c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d3effd700 1 -- 172.21.15.6:0/1652279083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d400fbd00 msgr2=0x7f6d4010c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d3effd700 1 --2- 172.21.15.6:0/1652279083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d400fbd00 0x7f6d4010c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d3effd700 1 -- 172.21.15.6:0/1652279083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d30005040 con 0x7f6d400fcfb0 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d3effd700 1 --2- 172.21.15.6:0/1652279083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d400fcfb0 0x7f6d4010cd60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6d34002a50 tx=0x7f6d340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d27fff700 1 -- 172.21.15.6:0/1652279083 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d34007cc0 con 0x7f6d400fcfb0 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d27fff700 1 -- 172.21.15.6:0/1652279083 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d3400f040 con 0x7f6d400fcfb0 2024-03-22T06:21:33.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d45f7d700 1 -- 172.21.15.6:0/1652279083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d40111f30 con 0x7f6d400fcfb0 2024-03-22T06:21:33.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d27fff700 1 -- 172.21.15.6:0/1652279083 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d3400da30 con 0x7f6d400fcfb0 2024-03-22T06:21:33.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.890+0000 7f6d45f7d700 1 -- 172.21.15.6:0/1652279083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d401124c0 con 0x7f6d400fcfb0 2024-03-22T06:21:33.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.891+0000 7f6d45f7d700 1 -- 172.21.15.6:0/1652279083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4004c710 con 0x7f6d400fcfb0 2024-03-22T06:21:33.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.892+0000 7f6d27fff700 1 -- 172.21.15.6:0/1652279083 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f6d3400db90 con 0x7f6d400fcfb0 2024-03-22T06:21:33.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.892+0000 7f6d27fff700 1 --2- 172.21.15.6:0/1652279083 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6d2806e840 0x7f6d28070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:33.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.892+0000 7f6d27fff700 1 -- 172.21.15.6:0/1652279083 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f6d3408e1e0 con 0x7f6d400fcfb0 2024-03-22T06:21:33.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.892+0000 7f6d3f7fe700 1 --2- 172.21.15.6:0/1652279083 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6d2806e840 0x7f6d28070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:33.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.893+0000 7f6d3f7fe700 1 --2- 172.21.15.6:0/1652279083 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6d2806e840 0x7f6d28070cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009fd0 tx=0x7f6d30009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:33.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:33.897+0000 7f6d27fff700 1 -- 172.21.15.6:0/1652279083 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f6d3405ac10 con 0x7f6d400fcfb0 2024-03-22T06:21:33.916 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:21:33 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:21:33.702+0000 7fd961eed700 -1 osd.4 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:21:33.916 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:33 smithi117 ceph-mon[82126]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:33.916 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:33 smithi117 ceph-mon[82126]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:33.917 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:33 smithi117 ceph-mon[82126]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T06:21:33.917 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:33 smithi117 ceph-mon[82126]: osdmap e84: 6 total, 5 up, 6 in 2024-03-22T06:21:33.917 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:33 smithi117 ceph-mon[82126]: from='osd.4 [v2:172.21.15.117:6808/208554150,v1:172.21.15.117:6809/208554150]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:21:33.917 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:33 smithi117 ceph-mon[82126]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:21:33.917 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:33 smithi117 ceph-mon[82126]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:33.917 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:33 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/122479522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:33.917 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:33 smithi117 ceph-mon[82126]: from='osd.4 ' entity='osd.4' 2024-03-22T06:21:34.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:33 smithi006 ceph-mon[94482]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:34.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:33 smithi006 ceph-mon[94482]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:34.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:33 smithi006 ceph-mon[94482]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T06:21:34.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:33 smithi006 ceph-mon[94482]: osdmap e84: 6 total, 5 up, 6 in 2024-03-22T06:21:34.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:33 smithi006 ceph-mon[94482]: from='osd.4 [v2:172.21.15.117:6808/208554150,v1:172.21.15.117:6809/208554150]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:21:34.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:33 smithi006 ceph-mon[94482]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:21:34.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:33 smithi006 ceph-mon[94482]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:34.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:33 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/122479522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:34.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:33 smithi006 ceph-mon[94482]: from='osd.4 ' entity='osd.4' 2024-03-22T06:21:34.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.127+0000 7f6d45f7d700 1 -- 172.21.15.6:0/1652279083 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d40101320 con 0x7f6d2806e840 2024-03-22T06:21:34.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.128+0000 7f6d27fff700 1 -- 172.21.15.6:0/1652279083 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f6d40101320 con 0x7f6d2806e840 2024-03-22T06:21:34.128 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:21:34.128 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:21:34.129 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:21:34.129 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:21:34.129 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:21:34.129 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:21:34.129 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:21:34.129 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:21:34.129 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:21:34.129 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "11/21 daemons upgraded", 2024-03-22T06:21:34.129 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T06:21:34.129 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:21:34.129 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:21:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.130+0000 7f6d45f7d700 1 -- 172.21.15.6:0/1652279083 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6d2806e840 msgr2=0x7f6d28070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.130+0000 7f6d45f7d700 1 --2- 172.21.15.6:0/1652279083 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6d2806e840 0x7f6d28070cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009fd0 tx=0x7f6d30009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.130+0000 7f6d45f7d700 1 -- 172.21.15.6:0/1652279083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d400fcfb0 msgr2=0x7f6d4010cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.130+0000 7f6d45f7d700 1 --2- 172.21.15.6:0/1652279083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d400fcfb0 0x7f6d4010cd60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6d34002a50 tx=0x7f6d340046b0 comp rx=0 tx=0).stop 2024-03-22T06:21:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.130+0000 7f6d45f7d700 1 -- 172.21.15.6:0/1652279083 shutdown_connections 2024-03-22T06:21:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.130+0000 7f6d45f7d700 1 --2- 172.21.15.6:0/1652279083 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6d2806e840 0x7f6d28070cf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.130+0000 7f6d45f7d700 1 --2- 172.21.15.6:0/1652279083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d400fbd00 0x7f6d4010c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.130+0000 7f6d45f7d700 1 --2- 172.21.15.6:0/1652279083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d400fcfb0 0x7f6d4010cd60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.130+0000 7f6d45f7d700 1 -- 172.21.15.6:0/1652279083 >> 172.21.15.6:0/1652279083 conn(0x7f6d400f7480 msgr2=0x7f6d400f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.130+0000 7f6d45f7d700 1 -- 172.21.15.6:0/1652279083 shutdown_connections 2024-03-22T06:21:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.130+0000 7f6d45f7d700 1 -- 172.21.15.6:0/1652279083 wait complete. 2024-03-22T06:21:34.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.234+0000 7ff1f69a2700 1 -- 172.21.15.6:0/3602165389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00fdec0 msgr2=0x7ff1f00fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:34.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.234+0000 7ff1f69a2700 1 --2- 172.21.15.6:0/3602165389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00fdec0 0x7ff1f00fe330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0004940 tx=0x7ff1e0009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:34.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.234+0000 7ff1f69a2700 1 -- 172.21.15.6:0/3602165389 shutdown_connections 2024-03-22T06:21:34.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.234+0000 7ff1f69a2700 1 --2- 172.21.15.6:0/3602165389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00fdec0 0x7ff1f00fe330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:34.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.234+0000 7ff1f69a2700 1 --2- 172.21.15.6:0/3602165389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f00fcc10 0x7ff1f00fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:34.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.234+0000 7ff1f69a2700 1 -- 172.21.15.6:0/3602165389 >> 172.21.15.6:0/3602165389 conn(0x7ff1f00f8390 msgr2=0x7ff1f00fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:34.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.234+0000 7ff1f69a2700 1 -- 172.21.15.6:0/3602165389 shutdown_connections 2024-03-22T06:21:34.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.234+0000 7ff1f69a2700 1 -- 172.21.15.6:0/3602165389 wait complete. 2024-03-22T06:21:34.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.234+0000 7ff1f69a2700 1 Processor -- start 2024-03-22T06:21:34.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.234+0000 7ff1f69a2700 1 -- start start 2024-03-22T06:21:34.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f69a2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00fcc10 0x7ff1f010d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:34.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f69a2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f00fdec0 0x7ff1f010dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:34.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f69a2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f010e380 con 0x7ff1f00fcc10 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f69a2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f010ebd0 con 0x7ff1f00fdec0 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f59a0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00fcc10 0x7ff1f010d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f59a0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00fcc10 0x7ff1f010d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35286/0 (socket says 172.21.15.6:35286) 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f59a0700 1 -- 172.21.15.6:0/714512719 learned_addr learned my addr 172.21.15.6:0/714512719 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f519f700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f00fdec0 0x7ff1f010dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f59a0700 1 -- 172.21.15.6:0/714512719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f00fdec0 msgr2=0x7ff1f010dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f59a0700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f00fdec0 0x7ff1f010dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f59a0700 1 -- 172.21.15.6:0/714512719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e0005040 con 0x7ff1f00fcc10 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f519f700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f00fdec0 0x7ff1f010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f59a0700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00fcc10 0x7ff1f010d840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00c810 tx=0x7ff1ec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1e67fc700 1 -- 172.21.15.6:0/714512719 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1ec012810 con 0x7ff1f00fcc10 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1e67fc700 1 -- 172.21.15.6:0/714512719 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1ec012e40 con 0x7ff1f00fcc10 2024-03-22T06:21:34.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f69a2700 1 -- 172.21.15.6:0/714512719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f010eeb0 con 0x7ff1f00fcc10 2024-03-22T06:21:34.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1e67fc700 1 -- 172.21.15.6:0/714512719 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1ec00a370 con 0x7ff1f00fcc10 2024-03-22T06:21:34.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.235+0000 7ff1f69a2700 1 -- 172.21.15.6:0/714512719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f010f3f0 con 0x7ff1f00fcc10 2024-03-22T06:21:34.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.236+0000 7ff1f69a2700 1 -- 172.21.15.6:0/714512719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f00fc010 con 0x7ff1f00fcc10 2024-03-22T06:21:34.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.237+0000 7ff1e67fc700 1 -- 172.21.15.6:0/714512719 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7ff1ec011810 con 0x7ff1f00fcc10 2024-03-22T06:21:34.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.237+0000 7ff1e67fc700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1dc06e840 0x7ff1dc070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:21:34.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.237+0000 7ff1e67fc700 1 -- 172.21.15.6:0/714512719 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6614+0+0 (secure 0 0 0) 0x7ff1ec08e800 con 0x7ff1f00fcc10 2024-03-22T06:21:34.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.237+0000 7ff1f519f700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1dc06e840 0x7ff1dc070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:21:34.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.238+0000 7ff1f519f700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1dc06e840 0x7ff1dc070cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000b3b0 tx=0x7ff1e0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:21:34.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.242+0000 7ff1e67fc700 1 -- 172.21.15.6:0/714512719 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7ff1ec05b2d0 con 0x7ff1f00fcc10 2024-03-22T06:21:34.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.543+0000 7ff1f69a2700 1 -- 172.21.15.6:0/714512719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff1f004bed0 con 0x7ff1f00fcc10 2024-03-22T06:21:34.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.544+0000 7ff1e67fc700 1 -- 172.21.15.6:0/714512719 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1196 (secure 0 0 0) 0x7ff1ec0164c0 con 0x7ff1f00fcc10 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 osds down; Degraded data redundancy: 36/219 objects degraded (16.438%), 14 pgs degraded 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout: osd.4 (root=default,host=smithi117) is down 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 36/219 objects degraded (16.438%), 14 pgs degraded 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.5 is active+undersized+degraded, acting [3,2] 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.7 is active+undersized+degraded, acting [1,2] 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-03-22T06:21:34.545 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.b is active+undersized+degraded, acting [1,5] 2024-03-22T06:21:34.546 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-03-22T06:21:34.546 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-03-22T06:21:34.546 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-03-22T06:21:34.546 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-03-22T06:21:34.546 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-03-22T06:21:34.546 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-03-22T06:21:34.546 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-03-22T06:21:34.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.546+0000 7ff1f69a2700 1 -- 172.21.15.6:0/714512719 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1dc06e840 msgr2=0x7ff1dc070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:34.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.546+0000 7ff1f69a2700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1dc06e840 0x7ff1dc070cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000b3b0 tx=0x7ff1e0009e30 comp rx=0 tx=0).stop 2024-03-22T06:21:34.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.546+0000 7ff1f69a2700 1 -- 172.21.15.6:0/714512719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00fcc10 msgr2=0x7ff1f010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:21:34.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.546+0000 7ff1f69a2700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00fcc10 0x7ff1f010d840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00c810 tx=0x7ff1ec004470 comp rx=0 tx=0).stop 2024-03-22T06:21:34.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.546+0000 7ff1f69a2700 1 -- 172.21.15.6:0/714512719 shutdown_connections 2024-03-22T06:21:34.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.546+0000 7ff1f69a2700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1dc06e840 0x7ff1dc070cf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:34.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.546+0000 7ff1f69a2700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00fcc10 0x7ff1f010d840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:34.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.546+0000 7ff1f69a2700 1 --2- 172.21.15.6:0/714512719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f00fdec0 0x7ff1f010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:21:34.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.546+0000 7ff1f69a2700 1 -- 172.21.15.6:0/714512719 >> 172.21.15.6:0/714512719 conn(0x7ff1f00f8390 msgr2=0x7ff1f00f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:21:34.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.546+0000 7ff1f69a2700 1 -- 172.21.15.6:0/714512719 shutdown_connections 2024-03-22T06:21:34.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:21:34.546+0000 7ff1f69a2700 1 -- 172.21.15.6:0/714512719 wait complete. 2024-03-22T06:21:35.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: pgmap v165: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:35.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: Health check update: Degraded data redundancy: 36/219 objects degraded (16.438%), 14 pgs degraded (PG_DEGRADED) 2024-03-22T06:21:35.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3373201762' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:21:35.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:35.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:35.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:35.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:21:35.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: pgmap v166: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: pgmap v167: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/714512719' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: osd.4 [v2:172.21.15.117:6808/208554150,v1:172.21.15.117:6809/208554150] boot 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: osdmap e85: 6 total, 6 up, 6 in 2024-03-22T06:21:35.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:21:35.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: pgmap v165: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:35.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: Health check update: Degraded data redundancy: 36/219 objects degraded (16.438%), 14 pgs degraded (PG_DEGRADED) 2024-03-22T06:21:35.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3373201762' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: pgmap v166: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: pgmap v167: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T06:21:35.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/714512719' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:21:35.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:21:35.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: osd.4 [v2:172.21.15.117:6808/208554150,v1:172.21.15.117:6809/208554150] boot 2024-03-22T06:21:35.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: osdmap e85: 6 total, 6 up, 6 in 2024-03-22T06:21:35.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T06:21:36.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:36 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T06:21:36.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:36 smithi006 ceph-mon[94482]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-03-22T06:21:36.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:36 smithi006 ceph-mon[94482]: osdmap e86: 6 total, 6 up, 6 in 2024-03-22T06:21:36.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:36 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T06:21:36.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:36 smithi117 ceph-mon[82126]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-03-22T06:21:36.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:36 smithi117 ceph-mon[82126]: osdmap e86: 6 total, 6 up, 6 in 2024-03-22T06:21:37.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:37 smithi006 ceph-mon[94482]: pgmap v170: 65 pgs: 8 peering, 10 active+undersized, 5 active+undersized+degraded, 42 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12/219 objects degraded (5.479%) 2024-03-22T06:21:37.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:37 smithi117 ceph-mon[82126]: pgmap v170: 65 pgs: 8 peering, 10 active+undersized, 5 active+undersized+degraded, 42 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12/219 objects degraded (5.479%) 2024-03-22T06:21:38.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:38.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:21:38.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:38 smithi117 ceph-mon[82126]: pgmap v171: 65 pgs: 8 peering, 1 active+undersized, 1 active+undersized+degraded, 55 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4/219 objects degraded (1.826%) 2024-03-22T06:21:38.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:38.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:21:38.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:38 smithi006 ceph-mon[94482]: pgmap v171: 65 pgs: 8 peering, 1 active+undersized, 1 active+undersized+degraded, 55 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4/219 objects degraded (1.826%) 2024-03-22T06:21:39.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:39 smithi117 ceph-mon[82126]: Health check update: Degraded data redundancy: 4/219 objects degraded (1.826%), 1 pg degraded (PG_DEGRADED) 2024-03-22T06:21:39.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:39 smithi006 ceph-mon[94482]: Health check update: Degraded data redundancy: 4/219 objects degraded (1.826%), 1 pg degraded (PG_DEGRADED) 2024-03-22T06:21:40.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:40 smithi117 ceph-mon[82126]: pgmap v172: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:40.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:40 smithi006 ceph-mon[94482]: pgmap v172: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:41.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:41 smithi117 ceph-mon[82126]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/219 objects degraded (1.826%), 1 pg degraded) 2024-03-22T06:21:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:41 smithi006 ceph-mon[94482]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/219 objects degraded (1.826%), 1 pg degraded) 2024-03-22T06:21:42.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:42 smithi117 ceph-mon[82126]: pgmap v173: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:42.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:42 smithi006 ceph-mon[94482]: pgmap v173: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:44.653 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:44 smithi006 ceph-mon[94482]: pgmap v174: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:44.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:44 smithi117 ceph-mon[82126]: pgmap v174: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:46.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:46 smithi117 ceph-mon[82126]: pgmap v175: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:46.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:46 smithi006 ceph-mon[94482]: pgmap v175: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:48.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:48.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:48 smithi117 ceph-mon[82126]: pgmap v176: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:48.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:48.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:48 smithi006 ceph-mon[94482]: pgmap v176: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:49.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:49 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T06:21:49.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:49 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T06:21:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:50 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T06:21:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:50 smithi117 ceph-mon[82126]: Upgrade: osd.5 is safe to restart 2024-03-22T06:21:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:50 smithi117 ceph-mon[82126]: pgmap v177: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:50.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:50 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T06:21:50.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:50 smithi006 ceph-mon[94482]: Upgrade: osd.5 is safe to restart 2024-03-22T06:21:50.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:50 smithi006 ceph-mon[94482]: pgmap v177: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:52.220 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:51 smithi117 ceph-mon[82126]: Upgrade: Updating osd.5 2024-03-22T06:21:52.220 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:51 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:52.220 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:51 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T06:21:52.220 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:51 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:21:52.220 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:51 smithi117 ceph-mon[82126]: Deploying daemon osd.5 on smithi117 2024-03-22T06:21:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:51 smithi006 ceph-mon[94482]: Upgrade: Updating osd.5 2024-03-22T06:21:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:51 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:51 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T06:21:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:51 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:21:52.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:51 smithi006 ceph-mon[94482]: Deploying daemon osd.5 on smithi117 2024-03-22T06:21:53.151 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:52 smithi117 ceph-mon[82126]: pgmap v178: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:53.151 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:53.151 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:21:53.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:52 smithi006 ceph-mon[94482]: pgmap v178: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:53.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:21:53.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:21:54.466 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:54 smithi117 ceph-mon[82126]: pgmap v179: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:54.466 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:54 smithi117 systemd[1]: Stopping Ceph osd.5 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:21:54.653 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:54 smithi006 ceph-mon[94482]: pgmap v179: 65 pgs: 65 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:54.755 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:54 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[55451]: 2024-03-22T06:21:54.465+0000 7fda0cf14700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:21:54.756 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:54 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[55451]: 2024-03-22T06:21:54.465+0000 7fda0cf14700 -1 osd.5 86 *** Got signal Terminated *** 2024-03-22T06:21:54.756 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:54 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[55451]: 2024-03-22T06:21:54.465+0000 7fda0cf14700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:21:55.702 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:55 smithi117 ceph-mon[82126]: osd.5 marked itself down and dead 2024-03-22T06:21:55.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:55 smithi006 ceph-mon[94482]: osd.5 marked itself down and dead 2024-03-22T06:21:55.957 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:55 smithi117 podman[101187]: 2024-03-22 06:21:55.702670065 +0000 UTC m=+1.372084089 container died ac90669ae5d2a1e24449f823525f4fa4be9f964dc9a3c0489e6b64c1b8a04bfb (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5, RELEASE=quincy-9c4e0c3, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-22T06:21:56.255 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:56 smithi117 podman[101187]: 2024-03-22 06:21:56.048080577 +0000 UTC m=+1.717494609 container cleanup ac90669ae5d2a1e24449f823525f4fa4be9f964dc9a3c0489e6b64c1b8a04bfb (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5, RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-22T06:21:56.256 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:56 smithi117 bash[101187]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5 2024-03-22T06:21:56.666 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:56 smithi117 podman[101207]: 2024-03-22 06:21:56.332549715 +0000 UTC m=+0.633505988 container remove ac90669ae5d2a1e24449f823525f4fa4be9f964dc9a3c0489e6b64c1b8a04bfb (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-22T06:21:56.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:56 smithi117 ceph-mon[82126]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:21:56.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:56 smithi117 ceph-mon[82126]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-22T06:21:56.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:56 smithi117 ceph-mon[82126]: osdmap e87: 6 total, 5 up, 6 in 2024-03-22T06:21:56.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:56 smithi117 ceph-mon[82126]: pgmap v181: 65 pgs: 5 peering, 11 stale+active+clean, 49 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:56.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:56 smithi006 ceph-mon[94482]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T06:21:56.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:56 smithi006 ceph-mon[94482]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-22T06:21:56.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:56 smithi006 ceph-mon[94482]: osdmap e87: 6 total, 5 up, 6 in 2024-03-22T06:21:56.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:56 smithi006 ceph-mon[94482]: pgmap v181: 65 pgs: 5 peering, 11 stale+active+clean, 49 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-22T06:21:56.968 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:56 smithi117 podman[101301]: 2024-03-22 06:21:56.666885077 +0000 UTC m=+0.111701217 container create 09871b7d2aa6e6ed983598e511c75b689585555dd15d29228186b6e75745bd7e (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-22T06:21:56.969 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:56 smithi117 podman[101301]: 2024-03-22 06:21:56.573718712 +0000 UTC m=+0.018534807 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:21:56.969 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:56 smithi117 podman[101301]: 2024-03-22 06:21:56.902663698 +0000 UTC m=+0.347479801 container init 09871b7d2aa6e6ed983598e511c75b689585555dd15d29228186b6e75745bd7e (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:21:56.969 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:56 smithi117 podman[101301]: 2024-03-22 06:21:56.90595501 +0000 UTC m=+0.350771105 container start 09871b7d2aa6e6ed983598e511c75b689585555dd15d29228186b6e75745bd7e (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-22T06:21:56.969 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:56 smithi117 podman[101301]: 2024-03-22 06:21:56.969112283 +0000 UTC m=+0.413928387 container attach 09871b7d2aa6e6ed983598e511c75b689585555dd15d29228186b6e75745bd7e (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T06:21:57.255 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:57 smithi117 podman[101301]: 2024-03-22 06:21:57.199445737 +0000 UTC m=+0.644261839 container died 09871b7d2aa6e6ed983598e511c75b689585555dd15d29228186b6e75745bd7e (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:21:57.705 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:57 smithi117 ceph-mon[82126]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-22T06:21:57.705 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:57 smithi117 ceph-mon[82126]: osdmap e88: 6 total, 5 up, 6 in 2024-03-22T06:21:57.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:57 smithi006 ceph-mon[94482]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-22T06:21:57.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:57 smithi006 ceph-mon[94482]: osdmap e88: 6 total, 5 up, 6 in 2024-03-22T06:21:58.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:57 smithi117 podman[101327]: 2024-03-22 06:21:57.706073738 +0000 UTC m=+0.501664923 container remove 09871b7d2aa6e6ed983598e511c75b689585555dd15d29228186b6e75745bd7e (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-22T06:21:58.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:57 smithi117 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-22T06:21:58.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:57 smithi117 systemd[1]: Stopped Ceph osd.5 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:21:58.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:57 smithi117 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.5.service: Consumed 1min 4.803s CPU time. 2024-03-22T06:21:58.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:57 smithi117 systemd[1]: Starting Ceph osd.5 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:21:58.442 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:58 smithi117 ceph-mon[82126]: pgmap v183: 65 pgs: 10 active+undersized, 5 active+undersized+degraded, 5 peering, 2 stale+active+clean, 43 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11/219 objects degraded (5.023%) 2024-03-22T06:21:58.443 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 podman[101427]: 2024-03-22 06:21:58.031498308 +0000 UTC m=+0.018696300 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:21:58.443 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 podman[101427]: 2024-03-22 06:21:58.132210592 +0000 UTC m=+0.119408624 container create 5957a92fd6c6d5a520067d71987d533a1a90f37a8c55708dd13847987a27ff99 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-22T06:21:58.443 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 podman[101427]: 2024-03-22 06:21:58.35119357 +0000 UTC m=+0.338391559 container init 5957a92fd6c6d5a520067d71987d533a1a90f37a8c55708dd13847987a27ff99 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-22T06:21:58.443 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 podman[101427]: 2024-03-22 06:21:58.354486598 +0000 UTC m=+0.341684598 container start 5957a92fd6c6d5a520067d71987d533a1a90f37a8c55708dd13847987a27ff99 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-22T06:21:58.443 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 podman[101427]: 2024-03-22 06:21:58.44295798 +0000 UTC m=+0.430156009 container attach 5957a92fd6c6d5a520067d71987d533a1a90f37a8c55708dd13847987a27ff99 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-22T06:21:58.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:58 smithi006 ceph-mon[94482]: pgmap v183: 65 pgs: 10 active+undersized, 5 active+undersized+degraded, 5 peering, 2 stale+active+clean, 43 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11/219 objects degraded (5.023%) 2024-03-22T06:21:59.134 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate[101442]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate[101442]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 bash[101427]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 bash[101427]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 bash[101427]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 bash[101427]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 bash[101427]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 bash[101427]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 bash[101427]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate[101442]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate[101442]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate[101442]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate[101442]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T06:21:59.135 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate[101442]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-22T06:21:59.136 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:58 smithi117 podman[101427]: 2024-03-22 06:21:58.815605261 +0000 UTC m=+0.802803255 container died 5957a92fd6c6d5a520067d71987d533a1a90f37a8c55708dd13847987a27ff99 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:21:59.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:21:59 smithi117 ceph-mon[82126]: Health check failed: Degraded data redundancy: 11/219 objects degraded (5.023%), 5 pgs degraded (PG_DEGRADED) 2024-03-22T06:21:59.702 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:59 smithi117 podman[101561]: 2024-03-22 06:21:59.392526221 +0000 UTC m=+0.571897595 container remove 5957a92fd6c6d5a520067d71987d533a1a90f37a8c55708dd13847987a27ff99 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-22T06:21:59.702 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:59 smithi117 podman[101620]: 2024-03-22 06:21:59.503291285 +0000 UTC m=+0.018157260 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:21:59.703 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:59 smithi117 podman[101620]: 2024-03-22 06:21:59.605806199 +0000 UTC m=+0.120672156 container create 1a6a0ee2a2289dcc8c6f9dd04f36b3c65ae4a04dfb7a18e30ea86e2e8e1c2540 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:21:59.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:21:59 smithi006 ceph-mon[94482]: Health check failed: Degraded data redundancy: 11/219 objects degraded (5.023%), 5 pgs degraded (PG_DEGRADED) 2024-03-22T06:22:00.005 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:59 smithi117 podman[101620]: 2024-03-22 06:21:59.800002314 +0000 UTC m=+0.314868287 container init 1a6a0ee2a2289dcc8c6f9dd04f36b3c65ae4a04dfb7a18e30ea86e2e8e1c2540 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-03-22T06:22:00.005 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:59 smithi117 podman[101620]: 2024-03-22 06:21:59.803120417 +0000 UTC m=+0.317986376 container start 1a6a0ee2a2289dcc8c6f9dd04f36b3c65ae4a04dfb7a18e30ea86e2e8e1c2540 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-22T06:22:00.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:59 smithi117 ceph-osd[101639]: -- 172.21.15.117:0/1204169761 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55774e3f9680 con 0x55774e3c2c00 2024-03-22T06:22:00.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:59 smithi117 bash[101620]: 1a6a0ee2a2289dcc8c6f9dd04f36b3c65ae4a04dfb7a18e30ea86e2e8e1c2540 2024-03-22T06:22:00.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:21:59 smithi117 systemd[1]: Started Ceph osd.5 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:22:00.756 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:22:00 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:22:00.344+0000 7f65c41c77c0 -1 Falling back to public interface 2024-03-22T06:22:01.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:01.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:01.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:22:01.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:00 smithi117 ceph-mon[82126]: pgmap v184: 65 pgs: 15 active+undersized, 8 active+undersized+degraded, 5 peering, 37 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21/219 objects degraded (9.589%) 2024-03-22T06:22:01.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:01.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:01.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:22:01.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:00 smithi006 ceph-mon[94482]: pgmap v184: 65 pgs: 15 active+undersized, 8 active+undersized+degraded, 5 peering, 37 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21/219 objects degraded (9.589%) 2024-03-22T06:22:02.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:02 smithi117 ceph-mon[82126]: pgmap v185: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:22:02.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:02 smithi117 ceph-mon[82126]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-03-22T06:22:02.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:02 smithi006 ceph-mon[94482]: pgmap v185: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:22:02.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:02 smithi006 ceph-mon[94482]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-03-22T06:22:04.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:03 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:04.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:03 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:04.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:03 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:04.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:03 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:04.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.664+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3349680420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b80fce70 msgr2=0x7ff1b80fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:04.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.664+0000 7ff1bdfd7700 1 --2- 172.21.15.6:0/3349680420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b80fce70 0x7ff1b80fd280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009ce0 tx=0x7ff1a8004d10 comp rx=0 tx=0).stop 2024-03-22T06:22:04.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.664+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3349680420 shutdown_connections 2024-03-22T06:22:04.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.664+0000 7ff1bdfd7700 1 --2- 172.21.15.6:0/3349680420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b80fdfe0 0x7ff1b80fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:04.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.664+0000 7ff1bdfd7700 1 --2- 172.21.15.6:0/3349680420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b80fce70 0x7ff1b80fd280 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:04.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.664+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3349680420 >> 172.21.15.6:0/3349680420 conn(0x7ff1b80f8550 msgr2=0x7ff1b80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:04.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.664+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3349680420 shutdown_connections 2024-03-22T06:22:04.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.664+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3349680420 wait complete. 2024-03-22T06:22:04.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.664+0000 7ff1bdfd7700 1 Processor -- start 2024-03-22T06:22:04.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.664+0000 7ff1bdfd7700 1 -- start start 2024-03-22T06:22:04.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1bdfd7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b80fce70 0x7ff1b810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:04.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1bdfd7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b80fdfe0 0x7ff1b810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:04.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1bdfd7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b810e4c0 con 0x7ff1b80fdfe0 2024-03-22T06:22:04.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1bdfd7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b810e630 con 0x7ff1b80fce70 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1b6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b80fdfe0 0x7ff1b810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1b77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b80fce70 0x7ff1b810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1b6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b80fdfe0 0x7ff1b810dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58274/0 (socket says 172.21.15.6:58274) 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1b6ffd700 1 -- 172.21.15.6:0/3731635653 learned_addr learned my addr 172.21.15.6:0/3731635653 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1b77fe700 1 -- 172.21.15.6:0/3731635653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b80fdfe0 msgr2=0x7ff1b810dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1b77fe700 1 --2- 172.21.15.6:0/3731635653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b80fdfe0 0x7ff1b810dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1b77fe700 1 -- 172.21.15.6:0/3731635653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a8009940 con 0x7ff1b80fce70 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.665+0000 7ff1b77fe700 1 --2- 172.21.15.6:0/3731635653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b80fce70 0x7ff1b810d980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009cb0 tx=0x7ff1a8004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.666+0000 7ff1affff700 1 -- 172.21.15.6:0/3731635653 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1a800e040 con 0x7ff1b80fce70 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.666+0000 7ff1affff700 1 -- 172.21.15.6:0/3731635653 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1a801cb20 con 0x7ff1b80fce70 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.666+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3731635653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b8113030 con 0x7ff1b80fce70 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.666+0000 7ff1affff700 1 -- 172.21.15.6:0/3731635653 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1a8012b10 con 0x7ff1b80fce70 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.666+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3731635653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b81135f0 con 0x7ff1b80fce70 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.667+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3731635653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b804c710 con 0x7ff1b80fce70 2024-03-22T06:22:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.667+0000 7ff1affff700 1 -- 172.21.15.6:0/3731635653 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7ff1a801cc90 con 0x7ff1b80fce70 2024-03-22T06:22:04.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.668+0000 7ff1affff700 1 --2- 172.21.15.6:0/3731635653 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1a406e890 0x7ff1a4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:04.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.668+0000 7ff1affff700 1 -- 172.21.15.6:0/3731635653 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6614+0+0 (secure 0 0 0) 0x7ff1a8090710 con 0x7ff1b80fce70 2024-03-22T06:22:04.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.668+0000 7ff1b6ffd700 1 --2- 172.21.15.6:0/3731635653 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1a406e890 0x7ff1a4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:04.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.668+0000 7ff1b6ffd700 1 --2- 172.21.15.6:0/3731635653 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1a406e890 0x7ff1a4070d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0008fd0 tx=0x7ff1a000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:04.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.673+0000 7ff1affff700 1 -- 172.21.15.6:0/3731635653 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7ff1a805d0c0 con 0x7ff1b80fce70 2024-03-22T06:22:04.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.905+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3731635653 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1b81023c0 con 0x7ff1a406e890 2024-03-22T06:22:04.906 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:04 smithi006 ceph-mon[94482]: Health check update: Degraded data redundancy: 31/219 objects degraded (14.155%), 12 pgs degraded (PG_DEGRADED) 2024-03-22T06:22:04.906 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:04 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:04.906 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:04 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:04.906 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:04 smithi006 ceph-mon[94482]: pgmap v186: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:22:04.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.906+0000 7ff1affff700 1 -- 172.21.15.6:0/3731635653 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff1b81023c0 con 0x7ff1a406e890 2024-03-22T06:22:04.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.909+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3731635653 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1a406e890 msgr2=0x7ff1a4070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:04.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.909+0000 7ff1bdfd7700 1 --2- 172.21.15.6:0/3731635653 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1a406e890 0x7ff1a4070d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0008fd0 tx=0x7ff1a000b040 comp rx=0 tx=0).stop 2024-03-22T06:22:04.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.909+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3731635653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b80fce70 msgr2=0x7ff1b810d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:04.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.909+0000 7ff1bdfd7700 1 --2- 172.21.15.6:0/3731635653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b80fce70 0x7ff1b810d980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009cb0 tx=0x7ff1a8004d10 comp rx=0 tx=0).stop 2024-03-22T06:22:04.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.909+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3731635653 shutdown_connections 2024-03-22T06:22:04.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.909+0000 7ff1bdfd7700 1 --2- 172.21.15.6:0/3731635653 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1a406e890 0x7ff1a4070d40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:04.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.909+0000 7ff1bdfd7700 1 --2- 172.21.15.6:0/3731635653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b80fce70 0x7ff1b810d980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:04.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.909+0000 7ff1bdfd7700 1 --2- 172.21.15.6:0/3731635653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b80fdfe0 0x7ff1b810dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:04.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.909+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3731635653 >> 172.21.15.6:0/3731635653 conn(0x7ff1b80f8550 msgr2=0x7ff1b80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:04.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.909+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3731635653 shutdown_connections 2024-03-22T06:22:04.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:04.909+0000 7ff1bdfd7700 1 -- 172.21.15.6:0/3731635653 wait complete. 2024-03-22T06:22:04.920 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:22:05.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:04 smithi117 ceph-mon[82126]: Health check update: Degraded data redundancy: 31/219 objects degraded (14.155%), 12 pgs degraded (PG_DEGRADED) 2024-03-22T06:22:05.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:04 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:05.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:04 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:05.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:04 smithi117 ceph-mon[82126]: pgmap v186: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 194 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:22:05.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:22:04 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:22:04.858+0000 7f65c41c77c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-22T06:22:05.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.013+0000 7fa34bd3e700 1 -- 172.21.15.6:0/1298609353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3440fcee0 msgr2=0x7fa3440fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:05.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.013+0000 7fa34bd3e700 1 --2- 172.21.15.6:0/1298609353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3440fcee0 0x7fa3440fd310 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009ce0 tx=0x7fa33c004d10 comp rx=0 tx=0).stop 2024-03-22T06:22:05.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.013+0000 7fa34bd3e700 1 -- 172.21.15.6:0/1298609353 shutdown_connections 2024-03-22T06:22:05.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.013+0000 7fa34bd3e700 1 --2- 172.21.15.6:0/1298609353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3440fe0d0 0x7fa3440fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.013+0000 7fa34bd3e700 1 --2- 172.21.15.6:0/1298609353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3440fcee0 0x7fa3440fd310 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.013+0000 7fa34bd3e700 1 -- 172.21.15.6:0/1298609353 >> 172.21.15.6:0/1298609353 conn(0x7fa3440f8560 msgr2=0x7fa3440fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:05.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.013+0000 7fa34bd3e700 1 -- 172.21.15.6:0/1298609353 shutdown_connections 2024-03-22T06:22:05.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.013+0000 7fa34bd3e700 1 -- 172.21.15.6:0/1298609353 wait complete. 2024-03-22T06:22:05.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.013+0000 7fa34bd3e700 1 Processor -- start 2024-03-22T06:22:05.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.013+0000 7fa34bd3e700 1 -- start start 2024-03-22T06:22:05.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa34bd3e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3440fcee0 0x7fa34410da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:05.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa34bd3e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3440fe0d0 0x7fa34410dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:05.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa34bd3e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34410e5a0 con 0x7fa3440fe0d0 2024-03-22T06:22:05.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa34bd3e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34410edf0 con 0x7fa3440fcee0 2024-03-22T06:22:05.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa3492d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3440fe0d0 0x7fa34410dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:05.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa349ada700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3440fcee0 0x7fa34410da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:05.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa3492d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3440fe0d0 0x7fa34410dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58286/0 (socket says 172.21.15.6:58286) 2024-03-22T06:22:05.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa3492d9700 1 -- 172.21.15.6:0/2817998208 learned_addr learned my addr 172.21.15.6:0/2817998208 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:05.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa3492d9700 1 -- 172.21.15.6:0/2817998208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3440fcee0 msgr2=0x7fa34410da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:05.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa3492d9700 1 --2- 172.21.15.6:0/2817998208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3440fcee0 0x7fa34410da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa3492d9700 1 -- 172.21.15.6:0/2817998208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c009940 con 0x7fa3440fe0d0 2024-03-22T06:22:05.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa349ada700 1 --2- 172.21.15.6:0/2817998208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3440fcee0 0x7fa34410da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:22:05.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa3492d9700 1 --2- 172.21.15.6:0/2817998208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3440fe0d0 0x7fa34410dfa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa33800eb40 tx=0x7fa33800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:05.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa3427fc700 1 -- 172.21.15.6:0/2817998208 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa338007cc0 con 0x7fa3440fe0d0 2024-03-22T06:22:05.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa3427fc700 1 -- 172.21.15.6:0/2817998208 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa33800f040 con 0x7fa3440fe0d0 2024-03-22T06:22:05.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa34bd3e700 1 -- 172.21.15.6:0/2817998208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34410f070 con 0x7fa3440fe0d0 2024-03-22T06:22:05.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa3427fc700 1 -- 172.21.15.6:0/2817998208 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa33800ca10 con 0x7fa3440fe0d0 2024-03-22T06:22:05.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.014+0000 7fa34bd3e700 1 -- 172.21.15.6:0/2817998208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34410f5b0 con 0x7fa3440fe0d0 2024-03-22T06:22:05.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.015+0000 7fa34bd3e700 1 -- 172.21.15.6:0/2817998208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34405f5f0 con 0x7fa3440fe0d0 2024-03-22T06:22:05.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.016+0000 7fa3427fc700 1 -- 172.21.15.6:0/2817998208 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fa33800cb70 con 0x7fa3440fe0d0 2024-03-22T06:22:05.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.016+0000 7fa3427fc700 1 --2- 172.21.15.6:0/2817998208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa32c06e840 0x7fa32c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:05.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.016+0000 7fa3427fc700 1 -- 172.21.15.6:0/2817998208 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6614+0+0 (secure 0 0 0) 0x7fa33808df70 con 0x7fa3440fe0d0 2024-03-22T06:22:05.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.016+0000 7fa349ada700 1 --2- 172.21.15.6:0/2817998208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa32c06e840 0x7fa32c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:05.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.017+0000 7fa349ada700 1 --2- 172.21.15.6:0/2817998208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa32c06e840 0x7fa32c070cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009460 tx=0x7fa33c004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:05.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.021+0000 7fa3427fc700 1 -- 172.21.15.6:0/2817998208 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fa3380c2aa0 con 0x7fa3440fe0d0 2024-03-22T06:22:05.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.249+0000 7fa34bd3e700 1 -- 172.21.15.6:0/2817998208 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa34410f990 con 0x7fa32c06e840 2024-03-22T06:22:05.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.250+0000 7fa3427fc700 1 -- 172.21.15.6:0/2817998208 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa34410f990 con 0x7fa32c06e840 2024-03-22T06:22:05.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.252+0000 7fa34bd3e700 1 -- 172.21.15.6:0/2817998208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa32c06e840 msgr2=0x7fa32c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:05.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.252+0000 7fa34bd3e700 1 --2- 172.21.15.6:0/2817998208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa32c06e840 0x7fa32c070cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009460 tx=0x7fa33c004d10 comp rx=0 tx=0).stop 2024-03-22T06:22:05.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.252+0000 7fa34bd3e700 1 -- 172.21.15.6:0/2817998208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3440fe0d0 msgr2=0x7fa34410dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:05.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.252+0000 7fa34bd3e700 1 --2- 172.21.15.6:0/2817998208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3440fe0d0 0x7fa34410dfa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa33800eb40 tx=0x7fa33800b040 comp rx=0 tx=0).stop 2024-03-22T06:22:05.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.252+0000 7fa34bd3e700 1 -- 172.21.15.6:0/2817998208 shutdown_connections 2024-03-22T06:22:05.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.252+0000 7fa34bd3e700 1 --2- 172.21.15.6:0/2817998208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa32c06e840 0x7fa32c070cf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.252+0000 7fa34bd3e700 1 --2- 172.21.15.6:0/2817998208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3440fcee0 0x7fa34410da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.252+0000 7fa34bd3e700 1 --2- 172.21.15.6:0/2817998208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3440fe0d0 0x7fa34410dfa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.252+0000 7fa34bd3e700 1 -- 172.21.15.6:0/2817998208 >> 172.21.15.6:0/2817998208 conn(0x7fa3440f8560 msgr2=0x7fa3440f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:05.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.252+0000 7fa34bd3e700 1 -- 172.21.15.6:0/2817998208 shutdown_connections 2024-03-22T06:22:05.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.252+0000 7fa34bd3e700 1 -- 172.21.15.6:0/2817998208 wait complete. 2024-03-22T06:22:05.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.354+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2611096286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b00fcf90 msgr2=0x7f58b00fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:05.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.354+0000 7f58b5d6b700 1 --2- 172.21.15.6:0/2611096286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b00fcf90 0x7f58b00fd400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f58a4004940 tx=0x7f58a4009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:05.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.354+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2611096286 shutdown_connections 2024-03-22T06:22:05.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.354+0000 7f58b5d6b700 1 --2- 172.21.15.6:0/2611096286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b00fcf90 0x7f58b00fd400 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.354+0000 7f58b5d6b700 1 --2- 172.21.15.6:0/2611096286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b00fbce0 0x7f58b00fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.354+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2611096286 >> 172.21.15.6:0/2611096286 conn(0x7f58b00f7440 msgr2=0x7f58b00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:05.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.354+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2611096286 shutdown_connections 2024-03-22T06:22:05.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.354+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2611096286 wait complete. 2024-03-22T06:22:05.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.355+0000 7f58b5d6b700 1 Processor -- start 2024-03-22T06:22:05.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.355+0000 7f58b5d6b700 1 -- start start 2024-03-22T06:22:05.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.355+0000 7f58b5d6b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b00fbce0 0x7f58b0066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:05.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.355+0000 7f58b5d6b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b00fcf90 0x7f58b0114b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:05.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.355+0000 7f58b5d6b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b010e520 con 0x7f58b00fbce0 2024-03-22T06:22:05.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.355+0000 7f58b5d6b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b0115080 con 0x7f58b00fcf90 2024-03-22T06:22:05.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.355+0000 7f58af7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b00fbce0 0x7f58b0066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:05.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.355+0000 7f58af7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b00fbce0 0x7f58b0066d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58310/0 (socket says 172.21.15.6:58310) 2024-03-22T06:22:05.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.355+0000 7f58af7fe700 1 -- 172.21.15.6:0/2954077383 learned_addr learned my addr 172.21.15.6:0/2954077383 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:05.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.355+0000 7f58aeffd700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b00fcf90 0x7f58b0114b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:05.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.356+0000 7f58aeffd700 1 -- 172.21.15.6:0/2954077383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b00fbce0 msgr2=0x7f58b0066d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:05.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.356+0000 7f58aeffd700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b00fbce0 0x7f58b0066d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.356+0000 7f58aeffd700 1 -- 172.21.15.6:0/2954077383 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a4005040 con 0x7f58b00fcf90 2024-03-22T06:22:05.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.356+0000 7f58af7fe700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b00fbce0 0x7f58b0066d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:22:05.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.356+0000 7f58aeffd700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b00fcf90 0x7f58b0114b40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f58a4000c00 tx=0x7f58a4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:05.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.356+0000 7f5897fff700 1 -- 172.21.15.6:0/2954077383 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58a400e040 con 0x7f58b00fcf90 2024-03-22T06:22:05.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.356+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2954077383 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b0115220 con 0x7f58b00fcf90 2024-03-22T06:22:05.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.356+0000 7f5897fff700 1 -- 172.21.15.6:0/2954077383 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58a400bbf0 con 0x7f58b00fcf90 2024-03-22T06:22:05.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.356+0000 7f5897fff700 1 -- 172.21.15.6:0/2954077383 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58a401c3f0 con 0x7f58b00fcf90 2024-03-22T06:22:05.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.356+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2954077383 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b0115790 con 0x7f58b00fcf90 2024-03-22T06:22:05.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.357+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2954077383 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b004c710 con 0x7f58b00fcf90 2024-03-22T06:22:05.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.358+0000 7f5897fff700 1 -- 172.21.15.6:0/2954077383 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f58a400bd60 con 0x7f58b00fcf90 2024-03-22T06:22:05.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.358+0000 7f5897fff700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f589806e840 0x7f5898070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:05.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.358+0000 7f5897fff700 1 -- 172.21.15.6:0/2954077383 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f58a408f780 con 0x7f58b00fcf90 2024-03-22T06:22:05.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.358+0000 7f58af7fe700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f589806e840 0x7f5898070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:05.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.358+0000 7f58af7fe700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f589806e840 0x7f5898070cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f58a00041d0 tx=0x7f58a000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:05.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.363+0000 7f5897fff700 1 -- 172.21.15.6:0/2954077383 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f58a405c1b0 con 0x7f58b00fcf90 2024-03-22T06:22:05.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.578+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2954077383 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f58b0101350 con 0x7f589806e840 2024-03-22T06:22:05.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.588+0000 7f5897fff700 1 -- 172.21.15.6:0/2954077383 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f58b0101350 con 0x7f589806e840 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (6m) 102s ago 15m 26.2M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (3m) 102s ago 15m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (3m) 2s ago 14m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 unknown 102s ago 14m - - 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (9m) 102s ago 9m 157M - 17.2.7-726-g9c4e0c34 837124da804f fa820b635d71 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (9m) 102s ago 9m 18.0M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (9m) 2s ago 9m 15.4M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (9m) 2s ago 9m 17.2M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (5m) 102s ago 16m 504M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (4m) 2s ago 14m 452M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (4m) 102s ago 16m 48.0M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:22:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (4m) 2s ago 14m 38.2M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:22:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (6m) 102s ago 15m 17.6M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:22:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (6m) 2s ago 14m 14.9M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:22:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (3m) 102s ago 13m 148M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:22:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (2m) 102s ago 12m 147M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:22:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (109s) 102s ago 12m 12.5M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 3ec1f219c19c 2024-03-22T06:22:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (72s) 2s ago 12m 99.5M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 9cba39b6f32c 2024-03-22T06:22:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (39s) 2s ago 11m 100M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1950fc21bd2e 2024-03-22T06:22:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (6s) 2s ago 11m 15.0M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1a6a0ee2a228 2024-03-22T06:22:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (5m) 102s ago 14m 64.4M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:22:05.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.590+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2954077383 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f589806e840 msgr2=0x7f5898070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:05.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.590+0000 7f58b5d6b700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f589806e840 0x7f5898070cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f58a00041d0 tx=0x7f58a000b040 comp rx=0 tx=0).stop 2024-03-22T06:22:05.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.590+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2954077383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b00fcf90 msgr2=0x7f58b0114b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:05.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.590+0000 7f58b5d6b700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b00fcf90 0x7f58b0114b40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f58a4000c00 tx=0x7f58a4009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:05.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.590+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2954077383 shutdown_connections 2024-03-22T06:22:05.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.590+0000 7f58b5d6b700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f589806e840 0x7f5898070cf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.590+0000 7f58b5d6b700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b00fbce0 0x7f58b0066d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.590+0000 7f58b5d6b700 1 --2- 172.21.15.6:0/2954077383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b00fcf90 0x7f58b0114b40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.590+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2954077383 >> 172.21.15.6:0/2954077383 conn(0x7f58b00f7440 msgr2=0x7f58b00f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:05.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.590+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2954077383 shutdown_connections 2024-03-22T06:22:05.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.590+0000 7f58b5d6b700 1 -- 172.21.15.6:0/2954077383 wait complete. 2024-03-22T06:22:05.642 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:22:05 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:22:05.310+0000 7f65c41c77c0 -1 osd.5 86 log_to_monitors true 2024-03-22T06:22:05.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.691+0000 7f635ce9f700 1 -- 172.21.15.6:0/1430819046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63580fdfe0 msgr2=0x7f63580fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.691+0000 7f635ce9f700 1 --2- 172.21.15.6:0/1430819046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63580fdfe0 0x7f63580fe450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f634c004940 tx=0x7f634c009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 -- 172.21.15.6:0/1430819046 shutdown_connections 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 --2- 172.21.15.6:0/1430819046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63580fdfe0 0x7f63580fe450 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 --2- 172.21.15.6:0/1430819046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63580fce70 0x7f63580fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 -- 172.21.15.6:0/1430819046 >> 172.21.15.6:0/1430819046 conn(0x7f63580f8550 msgr2=0x7f63580fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 -- 172.21.15.6:0/1430819046 shutdown_connections 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 -- 172.21.15.6:0/1430819046 wait complete. 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 Processor -- start 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 -- start start 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63580fce70 0x7f635810fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63580fdfe0 0x7f6358110020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6358110620 con 0x7f63580fce70 2024-03-22T06:22:05.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635ce9f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6358110790 con 0x7f63580fdfe0 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63580fce70 0x7f635810fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63580fce70 0x7f635810fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58326/0 (socket says 172.21.15.6:58326) 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.692+0000 7f635659c700 1 -- 172.21.15.6:0/2123440641 learned_addr learned my addr 172.21.15.6:0/2123440641 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.693+0000 7f6355d9b700 1 --2- 172.21.15.6:0/2123440641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63580fdfe0 0x7f6358110020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.693+0000 7f635659c700 1 -- 172.21.15.6:0/2123440641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63580fdfe0 msgr2=0x7f6358110020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.693+0000 7f635659c700 1 --2- 172.21.15.6:0/2123440641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63580fdfe0 0x7f6358110020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.693+0000 7f635659c700 1 -- 172.21.15.6:0/2123440641 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c005040 con 0x7f63580fce70 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.693+0000 7f635659c700 1 --2- 172.21.15.6:0/2123440641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63580fce70 0x7f635810fae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f634800eb40 tx=0x7f634800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.693+0000 7f6346ffd700 1 -- 172.21.15.6:0/2123440641 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6348007cc0 con 0x7f63580fce70 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.693+0000 7f6346ffd700 1 -- 172.21.15.6:0/2123440641 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f634800f040 con 0x7f63580fce70 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.693+0000 7f635ce9f700 1 -- 172.21.15.6:0/2123440641 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6358115190 con 0x7f63580fce70 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.693+0000 7f6346ffd700 1 -- 172.21.15.6:0/2123440641 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f634800ca10 con 0x7f63580fce70 2024-03-22T06:22:05.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.693+0000 7f635ce9f700 1 -- 172.21.15.6:0/2123440641 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63581156d0 con 0x7f63580fce70 2024-03-22T06:22:05.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.694+0000 7f635ce9f700 1 -- 172.21.15.6:0/2123440641 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6358101970 con 0x7f63580fce70 2024-03-22T06:22:05.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.695+0000 7f6346ffd700 1 -- 172.21.15.6:0/2123440641 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f634800cb70 con 0x7f63580fce70 2024-03-22T06:22:05.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.695+0000 7f6346ffd700 1 --2- 172.21.15.6:0/2123440641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f634006e840 0x7f6340070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:05.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.695+0000 7f6346ffd700 1 -- 172.21.15.6:0/2123440641 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f634808df70 con 0x7f63580fce70 2024-03-22T06:22:05.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.695+0000 7f6355d9b700 1 --2- 172.21.15.6:0/2123440641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f634006e840 0x7f6340070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:05.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.695+0000 7f6355d9b700 1 --2- 172.21.15.6:0/2123440641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f634006e840 0x7f6340070cf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f634c009b30 tx=0x7f634c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:05.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.700+0000 7f6346ffd700 1 -- 172.21.15.6:0/2123440641 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f63480c2aa0 con 0x7f63580fce70 2024-03-22T06:22:05.934 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:05 smithi117 ceph-mon[82126]: from='client.44265 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:05.934 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:05 smithi117 ceph-mon[82126]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:05.934 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:05 smithi117 ceph-mon[82126]: from='osd.5 [v2:172.21.15.117:6816/1416150897,v1:172.21.15.117:6817/1416150897]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T06:22:05.934 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:05 smithi117 ceph-mon[82126]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T06:22:05.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.996+0000 7f635ce9f700 1 -- 172.21.15.6:0/2123440641 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f635804c710 con 0x7f63580fce70 2024-03-22T06:22:05.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.997+0000 7f6346ffd700 1 -- 172.21.15.6:0/2123440641 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+732 (secure 0 0 0) 0x7f634805a860 con 0x7f63580fce70 2024-03-22T06:22:05.998 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:22:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:22:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:22:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:22:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:22:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 5 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4, 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 9 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:22:05.999 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:22:06.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.999+0000 7f635ce9f700 1 -- 172.21.15.6:0/2123440641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f634006e840 msgr2=0x7f6340070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.999+0000 7f635ce9f700 1 --2- 172.21.15.6:0/2123440641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f634006e840 0x7f6340070cf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f634c009b30 tx=0x7f634c009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:06.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.999+0000 7f635ce9f700 1 -- 172.21.15.6:0/2123440641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63580fce70 msgr2=0x7f635810fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.999+0000 7f635ce9f700 1 --2- 172.21.15.6:0/2123440641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63580fce70 0x7f635810fae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f634800eb40 tx=0x7f634800b040 comp rx=0 tx=0).stop 2024-03-22T06:22:06.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.999+0000 7f635ce9f700 1 -- 172.21.15.6:0/2123440641 shutdown_connections 2024-03-22T06:22:06.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.999+0000 7f635ce9f700 1 --2- 172.21.15.6:0/2123440641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f634006e840 0x7f6340070cf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.999+0000 7f635ce9f700 1 --2- 172.21.15.6:0/2123440641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63580fce70 0x7f635810fae0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.999+0000 7f635ce9f700 1 --2- 172.21.15.6:0/2123440641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63580fdfe0 0x7f6358110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.999+0000 7f635ce9f700 1 -- 172.21.15.6:0/2123440641 >> 172.21.15.6:0/2123440641 conn(0x7f63580f8550 msgr2=0x7f63580f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:06.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.999+0000 7f635ce9f700 1 -- 172.21.15.6:0/2123440641 shutdown_connections 2024-03-22T06:22:06.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:05.999+0000 7f635ce9f700 1 -- 172.21.15.6:0/2123440641 wait complete. 2024-03-22T06:22:06.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.102+0000 7f107d767700 1 -- 172.21.15.6:0/2639937088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10780fce70 msgr2=0x7f10780fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.102+0000 7f107d767700 1 --2- 172.21.15.6:0/2639937088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10780fce70 0x7f10780fd280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1068004940 tx=0x7f1068009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:06.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.102+0000 7f107d767700 1 -- 172.21.15.6:0/2639937088 shutdown_connections 2024-03-22T06:22:06.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.102+0000 7f107d767700 1 --2- 172.21.15.6:0/2639937088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10780fdfe0 0x7f10780fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.102+0000 7f107d767700 1 --2- 172.21.15.6:0/2639937088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10780fce70 0x7f10780fd280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.102+0000 7f107d767700 1 -- 172.21.15.6:0/2639937088 >> 172.21.15.6:0/2639937088 conn(0x7f10780f8550 msgr2=0x7f10780fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:06.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.103+0000 7f107d767700 1 -- 172.21.15.6:0/2639937088 shutdown_connections 2024-03-22T06:22:06.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.103+0000 7f107d767700 1 -- 172.21.15.6:0/2639937088 wait complete. 2024-03-22T06:22:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.103+0000 7f107d767700 1 Processor -- start 2024-03-22T06:22:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.103+0000 7f107d767700 1 -- start start 2024-03-22T06:22:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.103+0000 7f107d767700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10780fce70 0x7f1078071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.103+0000 7f107d767700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10780fdfe0 0x7f1078070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.103+0000 7f107d767700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1078072260 con 0x7f10780fdfe0 2024-03-22T06:22:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.103+0000 7f107d767700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1078070850 con 0x7f10780fce70 2024-03-22T06:22:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f10767fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10780fdfe0 0x7f1078070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f10767fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10780fdfe0 0x7f1078070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58354/0 (socket says 172.21.15.6:58354) 2024-03-22T06:22:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f10767fc700 1 -- 172.21.15.6:0/948430015 learned_addr learned my addr 172.21.15.6:0/948430015 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f1076ffd700 1 --2- 172.21.15.6:0/948430015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10780fce70 0x7f1078071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f10767fc700 1 -- 172.21.15.6:0/948430015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10780fce70 msgr2=0x7f1078071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f10767fc700 1 --2- 172.21.15.6:0/948430015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10780fce70 0x7f1078071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f10767fc700 1 -- 172.21.15.6:0/948430015 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068005040 con 0x7f10780fdfe0 2024-03-22T06:22:06.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f10767fc700 1 --2- 172.21.15.6:0/948430015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10780fdfe0 0x7f1078070310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1060002a50 tx=0x7f10600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:06.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f106f7fe700 1 -- 172.21.15.6:0/948430015 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1060007cc0 con 0x7f10780fdfe0 2024-03-22T06:22:06.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f107d767700 1 -- 172.21.15.6:0/948430015 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1078070b00 con 0x7f10780fdfe0 2024-03-22T06:22:06.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f106f7fe700 1 -- 172.21.15.6:0/948430015 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f106000f040 con 0x7f10780fdfe0 2024-03-22T06:22:06.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f106f7fe700 1 -- 172.21.15.6:0/948430015 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f106000da30 con 0x7f10780fdfe0 2024-03-22T06:22:06.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.104+0000 7f107d767700 1 -- 172.21.15.6:0/948430015 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10780710c0 con 0x7f10780fdfe0 2024-03-22T06:22:06.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.105+0000 7f107d767700 1 -- 172.21.15.6:0/948430015 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107804c710 con 0x7f10780fdfe0 2024-03-22T06:22:06.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.106+0000 7f106f7fe700 1 -- 172.21.15.6:0/948430015 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f106000dbb0 con 0x7f10780fdfe0 2024-03-22T06:22:06.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.106+0000 7f106f7fe700 1 --2- 172.21.15.6:0/948430015 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f106406e840 0x7f1064070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:06.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.107+0000 7f1076ffd700 1 --2- 172.21.15.6:0/948430015 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f106406e840 0x7f1064070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.107+0000 7f106f7fe700 1 -- 172.21.15.6:0/948430015 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f106008e1e0 con 0x7f10780fdfe0 2024-03-22T06:22:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.107+0000 7f1076ffd700 1 --2- 172.21.15.6:0/948430015 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f106406e840 0x7f1064070cf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f106800b970 tx=0x7f1068009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:06.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.111+0000 7f106f7fe700 1 -- 172.21.15.6:0/948430015 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f106005ac30 con 0x7f10780fdfe0 2024-03-22T06:22:06.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:05 smithi006 ceph-mon[94482]: from='client.44265 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:06.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:05 smithi006 ceph-mon[94482]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:06.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:05 smithi006 ceph-mon[94482]: from='osd.5 [v2:172.21.15.117:6816/1416150897,v1:172.21.15.117:6817/1416150897]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T06:22:06.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:05 smithi006 ceph-mon[94482]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T06:22:06.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.366+0000 7f107d767700 1 -- 172.21.15.6:0/948430015 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f10781058d0 con 0x7f10780fdfe0 2024-03-22T06:22:06.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.367+0000 7f106f7fe700 1 -- 172.21.15.6:0/948430015 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1875 (secure 0 0 0) 0x7f106005a7c0 con 0x7f10780fdfe0 2024-03-22T06:22:06.367 INFO:teuthology.orchestra.run.smithi006.stdout:e14 2024-03-22T06:22:06.367 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:22:06.367 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:22:06.367 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:22:06.367 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:22:06.367 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 14 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:12:36.515420+0000 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-03-22T06:22:06.368 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14466} 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 0 members: 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:14466} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.6:6826/1260089028,v1:172.21.15.6:6827/1260089028] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:22:06.369 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:22:06.370 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:22:06.370 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:22:06.370 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:14476} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:22:06.370 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:22:06.370 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.369+0000 7f107d767700 1 -- 172.21.15.6:0/948430015 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f106406e840 msgr2=0x7f1064070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.369+0000 7f107d767700 1 --2- 172.21.15.6:0/948430015 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f106406e840 0x7f1064070cf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f106800b970 tx=0x7f1068009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.369+0000 7f107d767700 1 -- 172.21.15.6:0/948430015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10780fdfe0 msgr2=0x7f1078070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.369+0000 7f107d767700 1 --2- 172.21.15.6:0/948430015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10780fdfe0 0x7f1078070310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1060002a50 tx=0x7f10600046b0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.369+0000 7f107d767700 1 -- 172.21.15.6:0/948430015 shutdown_connections 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.369+0000 7f107d767700 1 --2- 172.21.15.6:0/948430015 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f106406e840 0x7f1064070cf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.369+0000 7f107d767700 1 --2- 172.21.15.6:0/948430015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10780fce70 0x7f1078071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.369+0000 7f107d767700 1 --2- 172.21.15.6:0/948430015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10780fdfe0 0x7f1078070310 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.369+0000 7f107d767700 1 -- 172.21.15.6:0/948430015 >> 172.21.15.6:0/948430015 conn(0x7f10780f8550 msgr2=0x7f10780f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.369+0000 7f107d767700 1 -- 172.21.15.6:0/948430015 shutdown_connections 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.369+0000 7f107d767700 1 -- 172.21.15.6:0/948430015 wait complete. 2024-03-22T06:22:06.371 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 14 2024-03-22T06:22:06.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.472+0000 7f35c51f8700 1 -- 172.21.15.6:0/557890789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35c00fbe00 msgr2=0x7f35c00fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.472+0000 7f35c51f8700 1 --2- 172.21.15.6:0/557890789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35c00fbe00 0x7f35c00fc210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f35b0004940 tx=0x7f35b0009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:06.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.472+0000 7f35c51f8700 1 -- 172.21.15.6:0/557890789 shutdown_connections 2024-03-22T06:22:06.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.472+0000 7f35c51f8700 1 --2- 172.21.15.6:0/557890789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35c00fcf70 0x7f35c00fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.472+0000 7f35c51f8700 1 --2- 172.21.15.6:0/557890789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35c00fbe00 0x7f35c00fc210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.472+0000 7f35c51f8700 1 -- 172.21.15.6:0/557890789 >> 172.21.15.6:0/557890789 conn(0x7f35c00f74e0 msgr2=0x7f35c00f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:06.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.472+0000 7f35c51f8700 1 -- 172.21.15.6:0/557890789 shutdown_connections 2024-03-22T06:22:06.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.472+0000 7f35c51f8700 1 -- 172.21.15.6:0/557890789 wait complete. 2024-03-22T06:22:06.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.472+0000 7f35c51f8700 1 Processor -- start 2024-03-22T06:22:06.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.472+0000 7f35c51f8700 1 -- start start 2024-03-22T06:22:06.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35c51f8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35c00fbe00 0x7f35c010c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:06.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35c51f8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35c00fcf70 0x7f35c010ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:06.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35c51f8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c010d400 con 0x7f35c00fcf70 2024-03-22T06:22:06.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35c51f8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c010dc50 con 0x7f35c00fbe00 2024-03-22T06:22:06.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35be59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35c00fcf70 0x7f35c010ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:06.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35be59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35c00fcf70 0x7f35c010ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58378/0 (socket says 172.21.15.6:58378) 2024-03-22T06:22:06.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35be59c700 1 -- 172.21.15.6:0/789610273 learned_addr learned my addr 172.21.15.6:0/789610273 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:06.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35bed9d700 1 --2- 172.21.15.6:0/789610273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35c00fbe00 0x7f35c010c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:06.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35be59c700 1 -- 172.21.15.6:0/789610273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35c00fbe00 msgr2=0x7f35c010c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35be59c700 1 --2- 172.21.15.6:0/789610273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35c00fbe00 0x7f35c010c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35be59c700 1 -- 172.21.15.6:0/789610273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b0005040 con 0x7f35c00fcf70 2024-03-22T06:22:06.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35be59c700 1 --2- 172.21.15.6:0/789610273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35c00fcf70 0x7f35c010ce00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f35a800eb40 tx=0x7f35a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:06.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35b77fe700 1 -- 172.21.15.6:0/789610273 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a8007cc0 con 0x7f35c00fcf70 2024-03-22T06:22:06.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35b77fe700 1 -- 172.21.15.6:0/789610273 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35a800f040 con 0x7f35c00fcf70 2024-03-22T06:22:06.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35c51f8700 1 -- 172.21.15.6:0/789610273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c010ded0 con 0x7f35c00fcf70 2024-03-22T06:22:06.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35b77fe700 1 -- 172.21.15.6:0/789610273 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a800ca10 con 0x7f35c00fcf70 2024-03-22T06:22:06.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.473+0000 7f35c51f8700 1 -- 172.21.15.6:0/789610273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c010e410 con 0x7f35c00fcf70 2024-03-22T06:22:06.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.474+0000 7f35c51f8700 1 -- 172.21.15.6:0/789610273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c005f5f0 con 0x7f35c00fcf70 2024-03-22T06:22:06.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.475+0000 7f35b77fe700 1 -- 172.21.15.6:0/789610273 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f35a800cc80 con 0x7f35c00fcf70 2024-03-22T06:22:06.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.475+0000 7f35b77fe700 1 --2- 172.21.15.6:0/789610273 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35ac06e890 0x7f35ac070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:06.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.475+0000 7f35b77fe700 1 -- 172.21.15.6:0/789610273 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6614+0+0 (secure 0 0 0) 0x7f35a808e0f0 con 0x7f35c00fcf70 2024-03-22T06:22:06.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.475+0000 7f35bed9d700 1 --2- 172.21.15.6:0/789610273 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35ac06e890 0x7f35ac070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:06.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.476+0000 7f35bed9d700 1 --2- 172.21.15.6:0/789610273 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35ac06e890 0x7f35ac070d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009b30 tx=0x7f35b0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:06.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.480+0000 7f35b77fe700 1 -- 172.21.15.6:0/789610273 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f35a805ab50 con 0x7f35c00fcf70 2024-03-22T06:22:06.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.708+0000 7f35c51f8700 1 -- 172.21.15.6:0/789610273 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35c010e6f0 con 0x7f35ac06e890 2024-03-22T06:22:06.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.709+0000 7f35b77fe700 1 -- 172.21.15.6:0/789610273 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f35c010e6f0 con 0x7f35ac06e890 2024-03-22T06:22:06.709 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:22:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:22:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:22:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:22:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:22:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:22:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:22:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "osd", 2024-03-22T06:22:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:22:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:22:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "12/21 daemons upgraded", 2024-03-22T06:22:06.711 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T06:22:06.711 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:22:06.711 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:22:06.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.712+0000 7f35c51f8700 1 -- 172.21.15.6:0/789610273 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35ac06e890 msgr2=0x7f35ac070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.712+0000 7f35c51f8700 1 --2- 172.21.15.6:0/789610273 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35ac06e890 0x7f35ac070d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009b30 tx=0x7f35b0009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:06.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.712+0000 7f35c51f8700 1 -- 172.21.15.6:0/789610273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35c00fcf70 msgr2=0x7f35c010ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.712+0000 7f35c51f8700 1 --2- 172.21.15.6:0/789610273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35c00fcf70 0x7f35c010ce00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f35a800eb40 tx=0x7f35a800b040 comp rx=0 tx=0).stop 2024-03-22T06:22:06.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.712+0000 7f35c51f8700 1 -- 172.21.15.6:0/789610273 shutdown_connections 2024-03-22T06:22:06.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.712+0000 7f35c51f8700 1 --2- 172.21.15.6:0/789610273 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f35ac06e890 0x7f35ac070d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.712+0000 7f35c51f8700 1 --2- 172.21.15.6:0/789610273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35c00fbe00 0x7f35c010c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.712+0000 7f35c51f8700 1 --2- 172.21.15.6:0/789610273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35c00fcf70 0x7f35c010ce00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.712+0000 7f35c51f8700 1 -- 172.21.15.6:0/789610273 >> 172.21.15.6:0/789610273 conn(0x7f35c00f74e0 msgr2=0x7f35c00f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:06.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.712+0000 7f35c51f8700 1 -- 172.21.15.6:0/789610273 shutdown_connections 2024-03-22T06:22:06.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.712+0000 7f35c51f8700 1 -- 172.21.15.6:0/789610273 wait complete. 2024-03-22T06:22:06.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.815+0000 7fab5b939700 1 -- 172.21.15.6:0/1813966938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab54070870 msgr2=0x7fab54070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.815+0000 7fab5b939700 1 --2- 172.21.15.6:0/1813966938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab54070870 0x7fab54070ce0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab44004940 tx=0x7fab44009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:06.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.815+0000 7fab5b939700 1 -- 172.21.15.6:0/1813966938 shutdown_connections 2024-03-22T06:22:06.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.815+0000 7fab5b939700 1 --2- 172.21.15.6:0/1813966938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab54070870 0x7fab54070ce0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab5b939700 1 --2- 172.21.15.6:0/1813966938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab54071e20 0x7fab54070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab5b939700 1 -- 172.21.15.6:0/1813966938 >> 172.21.15.6:0/1813966938 conn(0x7fab540f8550 msgr2=0x7fab540fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:06.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab5b939700 1 -- 172.21.15.6:0/1813966938 shutdown_connections 2024-03-22T06:22:06.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab5b939700 1 -- 172.21.15.6:0/1813966938 wait complete. 2024-03-22T06:22:06.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab5b939700 1 Processor -- start 2024-03-22T06:22:06.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab5b939700 1 -- start start 2024-03-22T06:22:06.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab5b939700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab54070870 0x7fab5410fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:06.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab5b939700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab54071e20 0x7fab5410ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:06.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab5b939700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab541105d0 con 0x7fab54070870 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab5b939700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab54110740 con 0x7fab54071e20 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab596d5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab54070870 0x7fab5410fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab596d5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab54070870 0x7fab5410fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58388/0 (socket says 172.21.15.6:58388) 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.816+0000 7fab596d5700 1 -- 172.21.15.6:0/536306491 learned_addr learned my addr 172.21.15.6:0/536306491 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.817+0000 7fab58ed4700 1 --2- 172.21.15.6:0/536306491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab54071e20 0x7fab5410ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.817+0000 7fab596d5700 1 -- 172.21.15.6:0/536306491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab54071e20 msgr2=0x7fab5410ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.817+0000 7fab596d5700 1 --2- 172.21.15.6:0/536306491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab54071e20 0x7fab5410ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.817+0000 7fab596d5700 1 -- 172.21.15.6:0/536306491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab44005040 con 0x7fab54070870 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.817+0000 7fab596d5700 1 --2- 172.21.15.6:0/536306491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab54070870 0x7fab5410fa90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fab5000eb70 tx=0x7fab5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.817+0000 7fab49ffb700 1 -- 172.21.15.6:0/536306491 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab50007cc0 con 0x7fab54070870 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.817+0000 7fab49ffb700 1 -- 172.21.15.6:0/536306491 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab5000f040 con 0x7fab54070870 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.817+0000 7fab49ffb700 1 -- 172.21.15.6:0/536306491 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab5000cb20 con 0x7fab54070870 2024-03-22T06:22:06.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.817+0000 7fab5b939700 1 -- 172.21.15.6:0/536306491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab54115140 con 0x7fab54070870 2024-03-22T06:22:06.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.817+0000 7fab5b939700 1 -- 172.21.15.6:0/536306491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab54115680 con 0x7fab54070870 2024-03-22T06:22:06.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.818+0000 7fab5b939700 1 -- 172.21.15.6:0/536306491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5404c710 con 0x7fab54070870 2024-03-22T06:22:06.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.819+0000 7fab49ffb700 1 -- 172.21.15.6:0/536306491 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fab5000cc80 con 0x7fab54070870 2024-03-22T06:22:06.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.819+0000 7fab49ffb700 1 --2- 172.21.15.6:0/536306491 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab4006e840 0x7fab40070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:06.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.819+0000 7fab49ffb700 1 -- 172.21.15.6:0/536306491 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6614+0+0 (secure 0 0 0) 0x7fab5008de00 con 0x7fab54070870 2024-03-22T06:22:06.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.819+0000 7fab58ed4700 1 --2- 172.21.15.6:0/536306491 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab4006e840 0x7fab40070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:06.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.819+0000 7fab58ed4700 1 --2- 172.21.15.6:0/536306491 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab4006e840 0x7fab40070cf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fab44009b30 tx=0x7fab44009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:06.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:06.824+0000 7fab49ffb700 1 -- 172.21.15.6:0/536306491 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fab5005a980 con 0x7fab54070870 2024-03-22T06:22:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:06 smithi006 ceph-mon[94482]: from='client.44275 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:06 smithi006 ceph-mon[94482]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T06:22:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:06 smithi006 ceph-mon[94482]: osdmap e89: 6 total, 5 up, 6 in 2024-03-22T06:22:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:06 smithi006 ceph-mon[94482]: from='osd.5 [v2:172.21.15.117:6816/1416150897,v1:172.21.15.117:6817/1416150897]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:22:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:06 smithi006 ceph-mon[94482]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:22:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:06 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2123440641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:06 smithi006 ceph-mon[94482]: pgmap v188: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 194 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:22:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:06 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/948430015' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:22:07.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:06 smithi006 ceph-mon[94482]: from='osd.5 ' entity='osd.5' 2024-03-22T06:22:07.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.117+0000 7fab5b939700 1 -- 172.21.15.6:0/536306491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fab54115b20 con 0x7fab54070870 2024-03-22T06:22:07.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.117+0000 7fab49ffb700 1 -- 172.21.15.6:0/536306491 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1325 (secure 0 0 0) 0x7fab5005a510 con 0x7fab54070870 2024-03-22T06:22:07.118 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 31/219 objects degraded (14.155%), 12 pgs degraded 2024-03-22T06:22:07.118 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:22:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in unknown state 2024-03-22T06:22:07.118 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-22T06:22:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: osd.5 (root=default,host=smithi117) is down 2024-03-22T06:22:07.118 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-03-22T06:22:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-03-22T06:22:07.118 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 31/219 objects degraded (14.155%), 12 pgs degraded 2024-03-22T06:22:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-03-22T06:22:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-03-22T06:22:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.b is active+undersized+degraded, acting [1,4] 2024-03-22T06:22:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-03-22T06:22:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-03-22T06:22:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.16 is active+undersized+degraded, acting [1,2] 2024-03-22T06:22:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-03-22T06:22:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-03-22T06:22:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-03-22T06:22:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-03-22T06:22:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-03-22T06:22:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-03-22T06:22:07.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.119+0000 7fab5b939700 1 -- 172.21.15.6:0/536306491 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab4006e840 msgr2=0x7fab40070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:07.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.119+0000 7fab5b939700 1 --2- 172.21.15.6:0/536306491 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab4006e840 0x7fab40070cf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fab44009b30 tx=0x7fab44009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:07.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.119+0000 7fab5b939700 1 -- 172.21.15.6:0/536306491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab54070870 msgr2=0x7fab5410fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:07.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.119+0000 7fab5b939700 1 --2- 172.21.15.6:0/536306491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab54070870 0x7fab5410fa90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fab5000eb70 tx=0x7fab5000b040 comp rx=0 tx=0).stop 2024-03-22T06:22:07.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.119+0000 7fab5b939700 1 -- 172.21.15.6:0/536306491 shutdown_connections 2024-03-22T06:22:07.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.119+0000 7fab5b939700 1 --2- 172.21.15.6:0/536306491 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fab4006e840 0x7fab40070cf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:07.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.119+0000 7fab5b939700 1 --2- 172.21.15.6:0/536306491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab54070870 0x7fab5410fa90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:07.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.119+0000 7fab5b939700 1 --2- 172.21.15.6:0/536306491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab54071e20 0x7fab5410ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:07.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.119+0000 7fab5b939700 1 -- 172.21.15.6:0/536306491 >> 172.21.15.6:0/536306491 conn(0x7fab540f8550 msgr2=0x7fab540f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:07.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.120+0000 7fab5b939700 1 -- 172.21.15.6:0/536306491 shutdown_connections 2024-03-22T06:22:07.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:07.120+0000 7fab5b939700 1 -- 172.21.15.6:0/536306491 wait complete. 2024-03-22T06:22:07.157 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:22:06 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:22:06.850+0000 7f65b1617700 -1 osd.5 86 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T06:22:07.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:06 smithi117 ceph-mon[82126]: from='client.44275 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:07.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:06 smithi117 ceph-mon[82126]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T06:22:07.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:06 smithi117 ceph-mon[82126]: osdmap e89: 6 total, 5 up, 6 in 2024-03-22T06:22:07.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:06 smithi117 ceph-mon[82126]: from='osd.5 [v2:172.21.15.117:6816/1416150897,v1:172.21.15.117:6817/1416150897]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:22:07.158 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:06 smithi117 ceph-mon[82126]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-22T06:22:07.158 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:06 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2123440641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:07.158 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:06 smithi117 ceph-mon[82126]: pgmap v188: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 194 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:22:07.158 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:06 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/948430015' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:22:07.158 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:06 smithi117 ceph-mon[82126]: from='osd.5 ' entity='osd.5' 2024-03-22T06:22:08.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:08.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/536306491' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:22:08.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:08.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:22:08.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:08.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:08.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:22:08.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:22:08.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:08.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:22:08.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: osd.5 [v2:172.21.15.117:6816/1416150897,v1:172.21.15.117:6817/1416150897] boot 2024-03-22T06:22:08.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: osdmap e90: 6 total, 6 up, 6 in 2024-03-22T06:22:08.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:22:08.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:08.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/536306491' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:22:08.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:08.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:22:08.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:08.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:22:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:22:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T06:22:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: osd.5 [v2:172.21.15.117:6816/1416150897,v1:172.21.15.117:6817/1416150897] boot 2024-03-22T06:22:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: osdmap e90: 6 total, 6 up, 6 in 2024-03-22T06:22:08.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T06:22:09.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: pgmap v189: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 194 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:22:09.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: pgmap v190: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 194 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:22:09.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:22:09.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: Upgrade: Setting container_image for all osd 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-22T06:22:09.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-22T06:22:09.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-22T06:22:09.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-22T06:22:09.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: Upgrade: Setting require_osd_release to 19 squid 2024-03-22T06:22:09.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-22T06:22:09.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-22T06:22:09.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-22T06:22:09.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: osdmap e91: 6 total, 6 up, 6 in 2024-03-22T06:22:09.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi006.bztopc"]}]: dispatch 2024-03-22T06:22:09.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: pgmap v189: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 194 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:22:09.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: pgmap v190: 65 pgs: 16 active+undersized, 12 active+undersized+degraded, 37 active+clean; 194 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-22T06:22:09.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:22:09.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: Upgrade: Setting container_image for all osd 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-22T06:22:09.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-22T06:22:09.284 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-22T06:22:09.284 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-22T06:22:09.284 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: Upgrade: Setting require_osd_release to 19 squid 2024-03-22T06:22:09.284 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-22T06:22:09.284 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-22T06:22:09.284 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-22T06:22:09.284 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: osdmap e91: 6 total, 6 up, 6 in 2024-03-22T06:22:09.284 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi006.bztopc"]}]: dispatch 2024-03-22T06:22:10.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:09 smithi006 ceph-mon[94482]: Upgrade: It appears safe to stop mds.cephfs.smithi006.bztopc 2024-03-22T06:22:10.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:09 smithi117 ceph-mon[82126]: Upgrade: It appears safe to stop mds.cephfs.smithi006.bztopc 2024-03-22T06:22:11.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:10 smithi117 ceph-mon[82126]: pgmap v193: 65 pgs: 2 peering, 15 active+undersized, 11 active+undersized+degraded, 37 active+clean; 194 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 28/219 objects degraded (12.785%) 2024-03-22T06:22:11.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:10 smithi117 ceph-mon[82126]: Health check update: Degraded data redundancy: 28/219 objects degraded (12.785%), 11 pgs degraded (PG_DEGRADED) 2024-03-22T06:22:11.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:10 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:11.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:10 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.bztopc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:22:11.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:10 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:22:11.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:10 smithi006 ceph-mon[94482]: pgmap v193: 65 pgs: 2 peering, 15 active+undersized, 11 active+undersized+degraded, 37 active+clean; 194 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 28/219 objects degraded (12.785%) 2024-03-22T06:22:11.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:10 smithi006 ceph-mon[94482]: Health check update: Degraded data redundancy: 28/219 objects degraded (12.785%), 11 pgs degraded (PG_DEGRADED) 2024-03-22T06:22:11.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:10 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:11.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:10 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.bztopc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:22:11.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:10 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:22:12.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:11 smithi117 ceph-mon[82126]: Upgrade: Updating mds.cephfs.smithi006.bztopc 2024-03-22T06:22:12.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:11 smithi117 ceph-mon[82126]: Deploying daemon mds.cephfs.smithi006.bztopc on smithi006 2024-03-22T06:22:12.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:11 smithi006 ceph-mon[94482]: Upgrade: Updating mds.cephfs.smithi006.bztopc 2024-03-22T06:22:12.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:11 smithi006 ceph-mon[94482]: Deploying daemon mds.cephfs.smithi006.bztopc on smithi006 2024-03-22T06:22:13.008 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:12 smithi006 ceph-mon[94482]: pgmap v194: 65 pgs: 2 peering, 2 active+undersized, 2 active+undersized+degraded, 59 active+clean; 194 MiB data, 806 MiB used, 536 GiB / 536 GiB avail; 3/219 objects degraded (1.370%) 2024-03-22T06:22:13.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:12 smithi117 ceph-mon[82126]: pgmap v194: 65 pgs: 2 peering, 2 active+undersized, 2 active+undersized+degraded, 59 active+clean; 194 MiB data, 806 MiB used, 536 GiB / 536 GiB avail; 3/219 objects degraded (1.370%) 2024-03-22T06:22:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:13 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006[94477]: 2024-03-22T06:22:13.007+0000 7fe66190e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:22:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:13 smithi117 ceph-mon[82126]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T06:22:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:13 smithi117 ceph-mon[82126]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:22:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:13 smithi117 ceph-mon[82126]: osdmap e92: 6 total, 6 up, 6 in 2024-03-22T06:22:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:13 smithi117 ceph-mon[82126]: Standby daemon mds.cephfs.smithi006.dcjpla assigned to filesystem cephfs as rank 0 2024-03-22T06:22:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:13 smithi117 ceph-mon[82126]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T06:22:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:13 smithi117 ceph-mon[82126]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T06:22:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:13 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi006.dcjpla=up:replay} 2 up:standby 2024-03-22T06:22:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:13 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:22:14.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:13 smithi006 ceph-mon[94482]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T06:22:14.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:13 smithi006 ceph-mon[94482]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:22:14.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:13 smithi006 ceph-mon[94482]: osdmap e92: 6 total, 6 up, 6 in 2024-03-22T06:22:14.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:13 smithi006 ceph-mon[94482]: Standby daemon mds.cephfs.smithi006.dcjpla assigned to filesystem cephfs as rank 0 2024-03-22T06:22:14.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:13 smithi006 ceph-mon[94482]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T06:22:14.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:13 smithi006 ceph-mon[94482]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T06:22:14.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:13 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi006.dcjpla=up:replay} 2 up:standby 2024-03-22T06:22:14.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:13 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:22:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:14 smithi117 ceph-mon[82126]: pgmap v196: 65 pgs: 65 active+clean; 194 MiB data, 806 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 0 op/s 2024-03-22T06:22:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:14 smithi117 ceph-mon[82126]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/219 objects degraded (1.370%), 2 pgs degraded) 2024-03-22T06:22:15.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:14 smithi006 ceph-mon[94482]: pgmap v196: 65 pgs: 65 active+clean; 194 MiB data, 806 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 0 op/s 2024-03-22T06:22:15.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:14 smithi006 ceph-mon[94482]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/219 objects degraded (1.370%), 2 pgs degraded) 2024-03-22T06:22:17.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:16 smithi117 ceph-mon[82126]: pgmap v197: 65 pgs: 65 active+clean; 194 MiB data, 806 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 1 op/s 2024-03-22T06:22:17.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:16 smithi006 ceph-mon[94482]: pgmap v197: 65 pgs: 65 active+clean; 194 MiB data, 806 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 1 op/s 2024-03-22T06:22:19.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:18 smithi117 ceph-mon[82126]: pgmap v198: 65 pgs: 65 active+clean; 194 MiB data, 806 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-03-22T06:22:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:18 smithi006 ceph-mon[94482]: pgmap v198: 65 pgs: 65 active+clean; 194 MiB data, 806 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-03-22T06:22:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:20 smithi117 ceph-mon[82126]: pgmap v199: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-22T06:22:21.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:20 smithi006 ceph-mon[94482]: pgmap v199: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-22T06:22:23.111 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:22 smithi006 ceph-mon[94482]: pgmap v200: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-22T06:22:23.111 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:22 smithi006 ceph-mon[94482]: reconnect by client.24313 v1:172.21.15.6:0/1969668997 after 0.00099999 2024-03-22T06:22:23.111 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:22 smithi006 ceph-mon[94482]: reconnect by client.14528 v1:172.21.15.117:0/3855819009 after 0.00099999 2024-03-22T06:22:23.111 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:22 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] up:reconnect 2024-03-22T06:22:23.112 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:22 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi006.dcjpla=up:reconnect} 2 up:standby 2024-03-22T06:22:23.112 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:22 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:23.112 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:22 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:22:23.112 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:22 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:22 smithi117 ceph-mon[82126]: pgmap v200: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-22T06:22:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:22 smithi117 ceph-mon[82126]: reconnect by client.24313 v1:172.21.15.6:0/1969668997 after 0.00099999 2024-03-22T06:22:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:22 smithi117 ceph-mon[82126]: reconnect by client.14528 v1:172.21.15.117:0/3855819009 after 0.00099999 2024-03-22T06:22:23.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:22 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] up:reconnect 2024-03-22T06:22:23.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:22 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi006.dcjpla=up:reconnect} 2 up:standby 2024-03-22T06:22:23.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:22 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:23.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:22 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:22:23.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:22 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:24.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:23 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] up:rejoin 2024-03-22T06:22:24.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:23 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi006.dcjpla=up:rejoin} 2 up:standby 2024-03-22T06:22:24.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:23 smithi117 ceph-mon[82126]: daemon mds.cephfs.smithi006.dcjpla is now active in filesystem cephfs as rank 0 2024-03-22T06:22:24.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:24.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:24.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:22:24.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:23 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] up:rejoin 2024-03-22T06:22:24.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:23 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi006.dcjpla=up:rejoin} 2 up:standby 2024-03-22T06:22:24.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:23 smithi006 ceph-mon[94482]: daemon mds.cephfs.smithi006.dcjpla is now active in filesystem cephfs as rank 0 2024-03-22T06:22:24.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:24.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:24.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:22:25.192 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:24 smithi006 ceph-mon[94482]: pgmap v201: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 95 B/s wr, 5 op/s 2024-03-22T06:22:25.192 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:24 smithi006 ceph-mon[94482]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T06:22:25.192 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:24 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] up:active 2024-03-22T06:22:25.192 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:24 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] up:boot 2024-03-22T06:22:25.192 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:24 smithi006 ceph-mon[94482]: fsmap cephfs:1 {0=cephfs.smithi006.dcjpla=up:active} 3 up:standby 2024-03-22T06:22:25.193 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:24 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:22:25.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:24 smithi117 ceph-mon[82126]: pgmap v201: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 95 B/s wr, 5 op/s 2024-03-22T06:22:25.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:24 smithi117 ceph-mon[82126]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T06:22:25.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:24 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] up:active 2024-03-22T06:22:25.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:24 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] up:boot 2024-03-22T06:22:25.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:24 smithi117 ceph-mon[82126]: fsmap cephfs:1 {0=cephfs.smithi006.dcjpla=up:active} 3 up:standby 2024-03-22T06:22:25.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:24 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:22:27.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:26 smithi117 ceph-mon[82126]: pgmap v202: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-22T06:22:27.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:26 smithi006 ceph-mon[94482]: pgmap v202: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-22T06:22:29.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:28 smithi117 ceph-mon[82126]: pgmap v203: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 341 B/s wr, 6 op/s 2024-03-22T06:22:29.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:28 smithi006 ceph-mon[94482]: pgmap v203: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 341 B/s wr, 6 op/s 2024-03-22T06:22:31.239 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:30 smithi006 ceph-mon[94482]: pgmap v204: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-03-22T06:22:31.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:30 smithi117 ceph-mon[82126]: pgmap v204: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-03-22T06:22:33.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:32 smithi117 ceph-mon[82126]: pgmap v205: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 5.2 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-03-22T06:22:33.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:32 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:33.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:32 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:33.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:32 smithi006 ceph-mon[94482]: pgmap v205: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 5.2 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-03-22T06:22:33.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:32 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:33.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:32 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:34.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:34.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:34 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:34.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:34 smithi117 ceph-mon[82126]: pgmap v206: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 917 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-03-22T06:22:34.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:34.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:34 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:34.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:34 smithi006 ceph-mon[94482]: pgmap v206: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 917 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-03-22T06:22:37.128 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:36 smithi006 ceph-mon[94482]: pgmap v207: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 37 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-03-22T06:22:37.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.235+0000 7f4bc9386700 1 -- 172.21.15.6:0/1334877645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc40fbce0 msgr2=0x7f4bc40fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:37.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.235+0000 7f4bc9386700 1 --2- 172.21.15.6:0/1334877645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc40fbce0 0x7f4bc40fc0f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4004940 tx=0x7f4bb4009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:37.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.235+0000 7f4bc9386700 1 -- 172.21.15.6:0/1334877645 shutdown_connections 2024-03-22T06:22:37.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.235+0000 7f4bc9386700 1 --2- 172.21.15.6:0/1334877645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc40fcf90 0x7f4bc40fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.235+0000 7f4bc9386700 1 --2- 172.21.15.6:0/1334877645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc40fbce0 0x7f4bc40fc0f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.235+0000 7f4bc9386700 1 -- 172.21.15.6:0/1334877645 >> 172.21.15.6:0/1334877645 conn(0x7f4bc40f7440 msgr2=0x7f4bc40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:37.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.235+0000 7f4bc9386700 1 -- 172.21.15.6:0/1334877645 shutdown_connections 2024-03-22T06:22:37.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.235+0000 7f4bc9386700 1 -- 172.21.15.6:0/1334877645 wait complete. 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc9386700 1 Processor -- start 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc9386700 1 -- start start 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc9386700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc40fbce0 0x7f4bc40ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc9386700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc40fcf90 0x7f4bc40fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc9386700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc4100180 con 0x7f4bc40fcf90 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc9386700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40fe810 con 0x7f4bc40fbce0 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc27fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc40fcf90 0x7f4bc40fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc27fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc40fcf90 0x7f4bc40fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41850/0 (socket says 172.21.15.6:41850) 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc27fc700 1 -- 172.21.15.6:0/3762251624 learned_addr learned my addr 172.21.15.6:0/3762251624 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc2ffd700 1 --2- 172.21.15.6:0/3762251624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc40fbce0 0x7f4bc40ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc27fc700 1 -- 172.21.15.6:0/3762251624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc40fbce0 msgr2=0x7f4bc40ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:37.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc27fc700 1 --2- 172.21.15.6:0/3762251624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc40fbce0 0x7f4bc40ffb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.236+0000 7f4bc27fc700 1 -- 172.21.15.6:0/3762251624 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb4005040 con 0x7f4bc40fcf90 2024-03-22T06:22:37.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.237+0000 7f4bc27fc700 1 --2- 172.21.15.6:0/3762251624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc40fcf90 0x7f4bc40fe2d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8002a80 tx=0x7f4bb80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:37.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.237+0000 7f4bab7fe700 1 -- 172.21.15.6:0/3762251624 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bb8007cc0 con 0x7f4bc40fcf90 2024-03-22T06:22:37.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.237+0000 7f4bc9386700 1 -- 172.21.15.6:0/3762251624 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc40feaf0 con 0x7f4bc40fcf90 2024-03-22T06:22:37.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.237+0000 7f4bab7fe700 1 -- 172.21.15.6:0/3762251624 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bb800f040 con 0x7f4bc40fcf90 2024-03-22T06:22:37.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.237+0000 7f4bab7fe700 1 -- 172.21.15.6:0/3762251624 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bb80096f0 con 0x7f4bc40fcf90 2024-03-22T06:22:37.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.237+0000 7f4bc9386700 1 -- 172.21.15.6:0/3762251624 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc40ff0b0 con 0x7f4bc40fcf90 2024-03-22T06:22:37.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.238+0000 7f4bc9386700 1 -- 172.21.15.6:0/3762251624 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc404c710 con 0x7f4bc40fcf90 2024-03-22T06:22:37.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.238+0000 7f4bab7fe700 1 -- 172.21.15.6:0/3762251624 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f4bb801c400 con 0x7f4bc40fcf90 2024-03-22T06:22:37.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.239+0000 7f4bab7fe700 1 --2- 172.21.15.6:0/3762251624 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4bac06e840 0x7f4bac070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:37.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.239+0000 7f4bab7fe700 1 -- 172.21.15.6:0/3762251624 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6700+0+0 (secure 0 0 0) 0x7f4bb808dcb0 con 0x7f4bc40fcf90 2024-03-22T06:22:37.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.239+0000 7f4bc2ffd700 1 --2- 172.21.15.6:0/3762251624 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4bac06e840 0x7f4bac070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:37.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.240+0000 7f4bc2ffd700 1 --2- 172.21.15.6:0/3762251624 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4bac06e840 0x7f4bac070cf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009b80 tx=0x7f4bb4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:37.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.244+0000 7f4bab7fe700 1 -- 172.21.15.6:0/3762251624 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f4bb805a720 con 0x7f4bc40fcf90 2024-03-22T06:22:37.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:36 smithi117 ceph-mon[82126]: pgmap v207: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 37 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-03-22T06:22:37.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.476+0000 7f4bc9386700 1 -- 172.21.15.6:0/3762251624 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bc405e400 con 0x7f4bac06e840 2024-03-22T06:22:37.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.478+0000 7f4bab7fe700 1 -- 172.21.15.6:0/3762251624 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4bc405e400 con 0x7f4bac06e840 2024-03-22T06:22:37.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.480+0000 7f4bc9386700 1 -- 172.21.15.6:0/3762251624 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4bac06e840 msgr2=0x7f4bac070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:37.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.480+0000 7f4bc9386700 1 --2- 172.21.15.6:0/3762251624 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4bac06e840 0x7f4bac070cf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009b80 tx=0x7f4bb4009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:37.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.480+0000 7f4bc9386700 1 -- 172.21.15.6:0/3762251624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc40fcf90 msgr2=0x7f4bc40fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:37.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.480+0000 7f4bc9386700 1 --2- 172.21.15.6:0/3762251624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc40fcf90 0x7f4bc40fe2d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8002a80 tx=0x7f4bb80046b0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.480+0000 7f4bc9386700 1 -- 172.21.15.6:0/3762251624 shutdown_connections 2024-03-22T06:22:37.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.480+0000 7f4bc9386700 1 --2- 172.21.15.6:0/3762251624 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4bac06e840 0x7f4bac070cf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.480+0000 7f4bc9386700 1 --2- 172.21.15.6:0/3762251624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc40fbce0 0x7f4bc40ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.480+0000 7f4bc9386700 1 --2- 172.21.15.6:0/3762251624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc40fcf90 0x7f4bc40fe2d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.480+0000 7f4bc9386700 1 -- 172.21.15.6:0/3762251624 >> 172.21.15.6:0/3762251624 conn(0x7f4bc40f7440 msgr2=0x7f4bc40668e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:37.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.480+0000 7f4bc9386700 1 -- 172.21.15.6:0/3762251624 shutdown_connections 2024-03-22T06:22:37.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.480+0000 7f4bc9386700 1 -- 172.21.15.6:0/3762251624 wait complete. 2024-03-22T06:22:37.492 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:22:37.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.587+0000 7f81013e2700 1 -- 172.21.15.6:0/1295168723 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80fc0fbd00 msgr2=0x7f80fc0fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:37.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.587+0000 7f81013e2700 1 --2- 172.21.15.6:0/1295168723 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80fc0fbd00 0x7f80fc0fc110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f80ec004940 tx=0x7f80ec009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:37.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.588+0000 7f81013e2700 1 -- 172.21.15.6:0/1295168723 shutdown_connections 2024-03-22T06:22:37.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.588+0000 7f81013e2700 1 --2- 172.21.15.6:0/1295168723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80fc0fcfb0 0x7f80fc0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.588+0000 7f81013e2700 1 --2- 172.21.15.6:0/1295168723 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80fc0fbd00 0x7f80fc0fc110 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.588+0000 7f81013e2700 1 -- 172.21.15.6:0/1295168723 >> 172.21.15.6:0/1295168723 conn(0x7f80fc0f7480 msgr2=0x7f80fc0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:37.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.588+0000 7f81013e2700 1 -- 172.21.15.6:0/1295168723 shutdown_connections 2024-03-22T06:22:37.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.588+0000 7f81013e2700 1 -- 172.21.15.6:0/1295168723 wait complete. 2024-03-22T06:22:37.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.588+0000 7f81013e2700 1 Processor -- start 2024-03-22T06:22:37.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.588+0000 7f81013e2700 1 -- start start 2024-03-22T06:22:37.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f81013e2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80fc0fbd00 0x7f80fc0ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:37.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f81013e2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80fc0fcfb0 0x7f80fc0fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:37.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f81013e2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc1001f0 con 0x7f80fc0fcfb0 2024-03-22T06:22:37.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f81013e2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0fe830 con 0x7f80fc0fbd00 2024-03-22T06:22:37.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f80fa7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80fc0fcfb0 0x7f80fc0fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:37.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f80fa7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80fc0fcfb0 0x7f80fc0fe2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41870/0 (socket says 172.21.15.6:41870) 2024-03-22T06:22:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f80fa7fc700 1 -- 172.21.15.6:0/1467357366 learned_addr learned my addr 172.21.15.6:0/1467357366 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f80faffd700 1 --2- 172.21.15.6:0/1467357366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80fc0fbd00 0x7f80fc0ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f80fa7fc700 1 -- 172.21.15.6:0/1467357366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80fc0fbd00 msgr2=0x7f80fc0ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f80fa7fc700 1 --2- 172.21.15.6:0/1467357366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80fc0fbd00 0x7f80fc0ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f80fa7fc700 1 -- 172.21.15.6:0/1467357366 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec005040 con 0x7f80fc0fcfb0 2024-03-22T06:22:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f80fa7fc700 1 --2- 172.21.15.6:0/1467357366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80fc0fcfb0 0x7f80fc0fe2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f80f0002a50 tx=0x7f80f00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f80e37fe700 1 -- 172.21.15.6:0/1467357366 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80f0007cc0 con 0x7f80fc0fcfb0 2024-03-22T06:22:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f81013e2700 1 -- 172.21.15.6:0/1467357366 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80fc0feb10 con 0x7f80fc0fcfb0 2024-03-22T06:22:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f80e37fe700 1 -- 172.21.15.6:0/1467357366 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80f000f040 con 0x7f80fc0fcfb0 2024-03-22T06:22:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f81013e2700 1 -- 172.21.15.6:0/1467357366 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80fc0ff050 con 0x7f80fc0fcfb0 2024-03-22T06:22:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.589+0000 7f80e37fe700 1 -- 172.21.15.6:0/1467357366 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80f0009640 con 0x7f80fc0fcfb0 2024-03-22T06:22:37.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.591+0000 7f81013e2700 1 -- 172.21.15.6:0/1467357366 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e80052e0 con 0x7f80fc0fcfb0 2024-03-22T06:22:37.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.591+0000 7f80e37fe700 1 -- 172.21.15.6:0/1467357366 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f80f00097a0 con 0x7f80fc0fcfb0 2024-03-22T06:22:37.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.591+0000 7f80e37fe700 1 --2- 172.21.15.6:0/1467357366 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f80e406e840 0x7f80e4070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:37.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.591+0000 7f80e37fe700 1 -- 172.21.15.6:0/1467357366 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6700+0+0 (secure 0 0 0) 0x7f80f008e120 con 0x7f80fc0fcfb0 2024-03-22T06:22:37.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.591+0000 7f80faffd700 1 --2- 172.21.15.6:0/1467357366 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f80e406e840 0x7f80e4070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:37.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.592+0000 7f80faffd700 1 --2- 172.21.15.6:0/1467357366 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f80e406e840 0x7f80e4070cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f80ec000c00 tx=0x7f80ec009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:37.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.596+0000 7f80e37fe700 1 -- 172.21.15.6:0/1467357366 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f80f005ac50 con 0x7f80fc0fcfb0 2024-03-22T06:22:37.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.828+0000 7f81013e2700 1 -- 172.21.15.6:0/1467357366 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80e8000bf0 con 0x7f80e406e840 2024-03-22T06:22:37.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.830+0000 7f80e37fe700 1 -- 172.21.15.6:0/1467357366 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f80e8000bf0 con 0x7f80e406e840 2024-03-22T06:22:37.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.832+0000 7f81013e2700 1 -- 172.21.15.6:0/1467357366 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f80e406e840 msgr2=0x7f80e4070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:37.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.832+0000 7f81013e2700 1 --2- 172.21.15.6:0/1467357366 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f80e406e840 0x7f80e4070cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f80ec000c00 tx=0x7f80ec009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:37.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.832+0000 7f81013e2700 1 -- 172.21.15.6:0/1467357366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80fc0fcfb0 msgr2=0x7f80fc0fe2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:37.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.832+0000 7f81013e2700 1 --2- 172.21.15.6:0/1467357366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80fc0fcfb0 0x7f80fc0fe2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f80f0002a50 tx=0x7f80f00046b0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.832+0000 7f81013e2700 1 -- 172.21.15.6:0/1467357366 shutdown_connections 2024-03-22T06:22:37.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.832+0000 7f81013e2700 1 --2- 172.21.15.6:0/1467357366 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f80e406e840 0x7f80e4070cf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.832+0000 7f81013e2700 1 --2- 172.21.15.6:0/1467357366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80fc0fbd00 0x7f80fc0ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.832+0000 7f81013e2700 1 --2- 172.21.15.6:0/1467357366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80fc0fcfb0 0x7f80fc0fe2f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.832+0000 7f81013e2700 1 -- 172.21.15.6:0/1467357366 >> 172.21.15.6:0/1467357366 conn(0x7f80fc0f7480 msgr2=0x7f80fc0667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:37.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.832+0000 7f81013e2700 1 -- 172.21.15.6:0/1467357366 shutdown_connections 2024-03-22T06:22:37.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.832+0000 7f81013e2700 1 -- 172.21.15.6:0/1467357366 wait complete. 2024-03-22T06:22:37.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.940+0000 7f3f815be700 1 -- 172.21.15.6:0/1587994196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f7c0fc720 msgr2=0x7f3f7c0fcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:37.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.940+0000 7f3f815be700 1 --2- 172.21.15.6:0/1587994196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f7c0fc720 0x7f3f7c0fcb90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3f64004a50 tx=0x7f3f6400af40 comp rx=0 tx=0).stop 2024-03-22T06:22:37.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.940+0000 7f3f815be700 1 -- 172.21.15.6:0/1587994196 shutdown_connections 2024-03-22T06:22:37.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.940+0000 7f3f815be700 1 --2- 172.21.15.6:0/1587994196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f7c0fc720 0x7f3f7c0fcb90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.940+0000 7f3f815be700 1 --2- 172.21.15.6:0/1587994196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f7c0fb470 0x7f3f7c0fb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.940+0000 7f3f815be700 1 -- 172.21.15.6:0/1587994196 >> 172.21.15.6:0/1587994196 conn(0x7f3f7c0f6bd0 msgr2=0x7f3f7c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:37.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.940+0000 7f3f815be700 1 -- 172.21.15.6:0/1587994196 shutdown_connections 2024-03-22T06:22:37.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.940+0000 7f3f815be700 1 -- 172.21.15.6:0/1587994196 wait complete. 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.941+0000 7f3f815be700 1 Processor -- start 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.941+0000 7f3f815be700 1 -- start start 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.941+0000 7f3f815be700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f7c0fb470 0x7f3f7c10b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.941+0000 7f3f815be700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f7c0fc720 0x7f3f7c10bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.941+0000 7f3f815be700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c10c230 con 0x7f3f7c0fb470 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.941+0000 7f3f815be700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c10c3a0 con 0x7f3f7c0fc720 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.941+0000 7f3f7affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f7c0fb470 0x7f3f7c10b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.941+0000 7f3f7affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f7c0fb470 0x7f3f7c10b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41890/0 (socket says 172.21.15.6:41890) 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.941+0000 7f3f7affd700 1 -- 172.21.15.6:0/1146645106 learned_addr learned my addr 172.21.15.6:0/1146645106 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.942+0000 7f3f7a7fc700 1 --2- 172.21.15.6:0/1146645106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f7c0fc720 0x7f3f7c10bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.942+0000 7f3f7affd700 1 -- 172.21.15.6:0/1146645106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f7c0fc720 msgr2=0x7f3f7c10bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.942+0000 7f3f7affd700 1 --2- 172.21.15.6:0/1146645106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f7c0fc720 0x7f3f7c10bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.942+0000 7f3f7affd700 1 -- 172.21.15.6:0/1146645106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f64005040 con 0x7f3f7c0fb470 2024-03-22T06:22:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.942+0000 7f3f7affd700 1 --2- 172.21.15.6:0/1146645106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f7c0fb470 0x7f3f7c10b6f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00cac0 tx=0x7f3f6c007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.942+0000 7f3f737fe700 1 -- 172.21.15.6:0/1146645106 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f6c00aa50 con 0x7f3f7c0fb470 2024-03-22T06:22:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.942+0000 7f3f737fe700 1 -- 172.21.15.6:0/1146645106 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f6c00abb0 con 0x7f3f7c0fb470 2024-03-22T06:22:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.942+0000 7f3f815be700 1 -- 172.21.15.6:0/1146645106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7c110e00 con 0x7f3f7c0fb470 2024-03-22T06:22:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.942+0000 7f3f737fe700 1 -- 172.21.15.6:0/1146645106 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f6c0115e0 con 0x7f3f7c0fb470 2024-03-22T06:22:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.942+0000 7f3f815be700 1 -- 172.21.15.6:0/1146645106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c1112f0 con 0x7f3f7c0fb470 2024-03-22T06:22:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.943+0000 7f3f815be700 1 -- 172.21.15.6:0/1146645106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7c04c710 con 0x7f3f7c0fb470 2024-03-22T06:22:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.943+0000 7f3f737fe700 1 -- 172.21.15.6:0/1146645106 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f3f6c00ad20 con 0x7f3f7c0fb470 2024-03-22T06:22:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.944+0000 7f3f737fe700 1 --2- 172.21.15.6:0/1146645106 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3f6806e840 0x7f3f68070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.944+0000 7f3f737fe700 1 -- 172.21.15.6:0/1146645106 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6700+0+0 (secure 0 0 0) 0x7f3f6c08ebd0 con 0x7f3f7c0fb470 2024-03-22T06:22:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.944+0000 7f3f7a7fc700 1 --2- 172.21.15.6:0/1146645106 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3f6806e840 0x7f3f68070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.944+0000 7f3f7a7fc700 1 --2- 172.21.15.6:0/1146645106 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3f6806e840 0x7f3f68070cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400c3b0 tx=0x7f3f6400af40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:37.948+0000 7f3f737fe700 1 -- 172.21.15.6:0/1146645106 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f3f6c05b5d0 con 0x7f3f7c0fb470 2024-03-22T06:22:38.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.172+0000 7f3f815be700 1 -- 172.21.15.6:0/1146645106 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f7c1009c0 con 0x7f3f6806e840 2024-03-22T06:22:38.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.184+0000 7f3f737fe700 1 -- 172.21.15.6:0/1146645106 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f3f7c1009c0 con 0x7f3f6806e840 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (7m) 5s ago 15m 26.4M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (4m) 5s ago 15m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (4m) 35s ago 14m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 error 5s ago 15m - - 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (15s) 5s ago 10m 13.8M - 19.0.0-2299-gea01217f aa461e7c5e7f 3e3319f646e6 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (10m) 5s ago 10m 82.7M - 17.2.7-726-g9c4e0c34 837124da804f 61d44311f193 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (10m) 35s ago 10m 15.4M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (10m) 35s ago 10m 17.2M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (6m) 5s ago 17m 508M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (5m) 35s ago 14m 452M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (5m) 5s ago 17m 50.1M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (4m) 35s ago 14m 38.2M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (7m) 5s ago 15m 17.2M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:22:38.185 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (6m) 35s ago 14m 14.9M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:22:38.186 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (3m) 5s ago 13m 153M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:22:38.186 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (2m) 5s ago 13m 135M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:22:38.186 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (2m) 5s ago 13m 101M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 3ec1f219c19c 2024-03-22T06:22:38.186 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (104s) 35s ago 12m 99.5M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 9cba39b6f32c 2024-03-22T06:22:38.186 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (71s) 35s ago 12m 100M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1950fc21bd2e 2024-03-22T06:22:38.186 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (38s) 35s ago 11m 15.0M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1a6a0ee2a228 2024-03-22T06:22:38.186 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (5m) 5s ago 15m 66.5M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:22:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.186+0000 7f3f815be700 1 -- 172.21.15.6:0/1146645106 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3f6806e840 msgr2=0x7f3f68070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.186+0000 7f3f815be700 1 --2- 172.21.15.6:0/1146645106 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3f6806e840 0x7f3f68070cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400c3b0 tx=0x7f3f6400af40 comp rx=0 tx=0).stop 2024-03-22T06:22:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.186+0000 7f3f815be700 1 -- 172.21.15.6:0/1146645106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f7c0fb470 msgr2=0x7f3f7c10b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.186+0000 7f3f815be700 1 --2- 172.21.15.6:0/1146645106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f7c0fb470 0x7f3f7c10b6f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00cac0 tx=0x7f3f6c007ee0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.186+0000 7f3f815be700 1 -- 172.21.15.6:0/1146645106 shutdown_connections 2024-03-22T06:22:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.186+0000 7f3f815be700 1 --2- 172.21.15.6:0/1146645106 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3f6806e840 0x7f3f68070cf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.186+0000 7f3f815be700 1 --2- 172.21.15.6:0/1146645106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f7c0fb470 0x7f3f7c10b6f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.186+0000 7f3f815be700 1 --2- 172.21.15.6:0/1146645106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f7c0fc720 0x7f3f7c10bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.186+0000 7f3f815be700 1 -- 172.21.15.6:0/1146645106 >> 172.21.15.6:0/1146645106 conn(0x7f3f7c0f6bd0 msgr2=0x7f3f7c0f8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.186+0000 7f3f815be700 1 -- 172.21.15.6:0/1146645106 shutdown_connections 2024-03-22T06:22:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.186+0000 7f3f815be700 1 -- 172.21.15.6:0/1146645106 wait complete. 2024-03-22T06:22:38.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.292+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3711389144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1d00fbd00 msgr2=0x7ff1d00fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:38.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.292+0000 7ff1d68a1700 1 --2- 172.21.15.6:0/3711389144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1d00fbd00 0x7ff1d00fc110 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0004940 tx=0x7ff1c0009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:38.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.293+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3711389144 shutdown_connections 2024-03-22T06:22:38.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.293+0000 7ff1d68a1700 1 --2- 172.21.15.6:0/3711389144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1d00fcfb0 0x7ff1d00fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.293+0000 7ff1d68a1700 1 --2- 172.21.15.6:0/3711389144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1d00fbd00 0x7ff1d00fc110 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.293+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3711389144 >> 172.21.15.6:0/3711389144 conn(0x7ff1d00f7480 msgr2=0x7ff1d00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:38.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.293+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3711389144 shutdown_connections 2024-03-22T06:22:38.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.293+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3711389144 wait complete. 2024-03-22T06:22:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.293+0000 7ff1d68a1700 1 Processor -- start 2024-03-22T06:22:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1d68a1700 1 -- start start 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1d68a1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1d00fbd00 0x7ff1d0066ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1d68a1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1d00fcfb0 0x7ff1d0112720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1d68a1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d0112c60 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1d68a1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d0112da0 con 0x7ff1d00fbd00 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1cf7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1d00fcfb0 0x7ff1d0112720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1cf7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1d00fcfb0 0x7ff1d0112720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41908/0 (socket says 172.21.15.6:41908) 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1cf7fe700 1 -- 172.21.15.6:0/3180476625 learned_addr learned my addr 172.21.15.6:0/3180476625 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1cffff700 1 --2- 172.21.15.6:0/3180476625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1d00fbd00 0x7ff1d0066ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1cf7fe700 1 -- 172.21.15.6:0/3180476625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1d00fbd00 msgr2=0x7ff1d0066ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1cf7fe700 1 --2- 172.21.15.6:0/3180476625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1d00fbd00 0x7ff1d0066ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.294+0000 7ff1cf7fe700 1 -- 172.21.15.6:0/3180476625 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c0005040 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.295+0000 7ff1cf7fe700 1 --2- 172.21.15.6:0/3180476625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1d00fcfb0 0x7ff1d0112720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400b7c0 tx=0x7ff1c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.295+0000 7ff1ccff9700 1 -- 172.21.15.6:0/3180476625 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1c400f6f0 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.295+0000 7ff1ccff9700 1 -- 172.21.15.6:0/3180476625 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1c400fd20 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.295+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3180476625 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d0113050 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.295+0000 7ff1ccff9700 1 -- 172.21.15.6:0/3180476625 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1c40103a0 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.295+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3180476625 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d01135e0 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.296+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3180476625 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d004c710 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.296+0000 7ff1ccff9700 1 -- 172.21.15.6:0/3180476625 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7ff1c4011400 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.297+0000 7ff1ccff9700 1 --2- 172.21.15.6:0/3180476625 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1b806e840 0x7ff1b8070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:38.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.297+0000 7ff1ccff9700 1 -- 172.21.15.6:0/3180476625 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6700+0+0 (secure 0 0 0) 0x7ff1c408e1a0 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.297+0000 7ff1cffff700 1 --2- 172.21.15.6:0/3180476625 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1b806e840 0x7ff1b8070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:38.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.297+0000 7ff1cffff700 1 --2- 172.21.15.6:0/3180476625 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1b806e840 0x7ff1b8070cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b970 tx=0x7ff1c0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:38.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.302+0000 7ff1ccff9700 1 -- 172.21.15.6:0/3180476625 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7ff1c405aa30 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: Detected new or changed devices on smithi006 2024-03-22T06:22:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:22:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:22:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: pgmap v208: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 38 KiB/s rd, 4.8 KiB/s wr, 5 op/s 2024-03-22T06:22:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: pgmap v209: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 45 KiB/s rd, 5.6 KiB/s wr, 4 op/s 2024-03-22T06:22:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:38.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:22:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:22:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi006.dcjpla"]}]: dispatch 2024-03-22T06:22:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: Detected new or changed devices on smithi006 2024-03-22T06:22:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:22:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:22:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: pgmap v208: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 38 KiB/s rd, 4.8 KiB/s wr, 5 op/s 2024-03-22T06:22:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: pgmap v209: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 45 KiB/s rd, 5.6 KiB/s wr, 4 op/s 2024-03-22T06:22:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:22:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:22:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:38.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi006.dcjpla"]}]: dispatch 2024-03-22T06:22:38.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.601+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3180476625 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff1d0104870 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.602+0000 7ff1ccff9700 1 -- 172.21.15.6:0/3180476625 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff1c405a850 con 0x7ff1d00fcfb0 2024-03-22T06:22:38.602 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:22:38.602 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:22:38.602 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:22:38.602 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 6 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 1 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 11 2024-03-22T06:22:38.603 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:22:38.604 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:22:38.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.604+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3180476625 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1b806e840 msgr2=0x7ff1b8070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.604+0000 7ff1d68a1700 1 --2- 172.21.15.6:0/3180476625 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1b806e840 0x7ff1b8070cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b970 tx=0x7ff1c0009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.604+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3180476625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1d00fcfb0 msgr2=0x7ff1d0112720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.604+0000 7ff1d68a1700 1 --2- 172.21.15.6:0/3180476625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1d00fcfb0 0x7ff1d0112720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400b7c0 tx=0x7ff1c4009250 comp rx=0 tx=0).stop 2024-03-22T06:22:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.604+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3180476625 shutdown_connections 2024-03-22T06:22:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.604+0000 7ff1d68a1700 1 --2- 172.21.15.6:0/3180476625 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff1b806e840 0x7ff1b8070cf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.604+0000 7ff1d68a1700 1 --2- 172.21.15.6:0/3180476625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1d00fbd00 0x7ff1d0066ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.604+0000 7ff1d68a1700 1 --2- 172.21.15.6:0/3180476625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1d00fcfb0 0x7ff1d0112720 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.604+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3180476625 >> 172.21.15.6:0/3180476625 conn(0x7ff1d00f7480 msgr2=0x7ff1d00f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.604+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3180476625 shutdown_connections 2024-03-22T06:22:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.604+0000 7ff1d68a1700 1 -- 172.21.15.6:0/3180476625 wait complete. 2024-03-22T06:22:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.707+0000 7fc478572700 1 -- 172.21.15.6:0/1541981795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4700fdff0 msgr2=0x7fc4700fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:38.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.707+0000 7fc478572700 1 --2- 172.21.15.6:0/1541981795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4700fdff0 0x7fc4700fe460 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc464004940 tx=0x7fc464009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:38.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.707+0000 7fc478572700 1 -- 172.21.15.6:0/1541981795 shutdown_connections 2024-03-22T06:22:38.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.707+0000 7fc478572700 1 --2- 172.21.15.6:0/1541981795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4700fdff0 0x7fc4700fe460 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.707+0000 7fc478572700 1 --2- 172.21.15.6:0/1541981795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4700fce80 0x7fc4700fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.707+0000 7fc478572700 1 -- 172.21.15.6:0/1541981795 >> 172.21.15.6:0/1541981795 conn(0x7fc4700f8560 msgr2=0x7fc4700fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:38.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.707+0000 7fc478572700 1 -- 172.21.15.6:0/1541981795 shutdown_connections 2024-03-22T06:22:38.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.707+0000 7fc478572700 1 -- 172.21.15.6:0/1541981795 wait complete. 2024-03-22T06:22:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc478572700 1 Processor -- start 2024-03-22T06:22:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc478572700 1 -- start start 2024-03-22T06:22:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc478572700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4700fce80 0x7fc47010da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc478572700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4700fdff0 0x7fc47010df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc478572700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47010e550 con 0x7fc4700fce80 2024-03-22T06:22:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc478572700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47010eda0 con 0x7fc4700fdff0 2024-03-22T06:22:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc47630e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4700fce80 0x7fc47010da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc47630e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4700fce80 0x7fc47010da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41926/0 (socket says 172.21.15.6:41926) 2024-03-22T06:22:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc47630e700 1 -- 172.21.15.6:0/3213105289 learned_addr learned my addr 172.21.15.6:0/3213105289 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc475b0d700 1 --2- 172.21.15.6:0/3213105289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4700fdff0 0x7fc47010df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc47630e700 1 -- 172.21.15.6:0/3213105289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4700fdff0 msgr2=0x7fc47010df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc47630e700 1 --2- 172.21.15.6:0/3213105289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4700fdff0 0x7fc47010df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.708+0000 7fc47630e700 1 -- 172.21.15.6:0/3213105289 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc464005040 con 0x7fc4700fce80 2024-03-22T06:22:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.709+0000 7fc47630e700 1 --2- 172.21.15.6:0/3213105289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4700fce80 0x7fc47010da10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc46000cb10 tx=0x7fc460007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.709+0000 7fc46effd700 1 -- 172.21.15.6:0/3213105289 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc460004200 con 0x7fc4700fce80 2024-03-22T06:22:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.709+0000 7fc46effd700 1 -- 172.21.15.6:0/3213105289 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc46000ac70 con 0x7fc4700fce80 2024-03-22T06:22:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.709+0000 7fc478572700 1 -- 172.21.15.6:0/3213105289 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47010f080 con 0x7fc4700fce80 2024-03-22T06:22:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.709+0000 7fc46effd700 1 -- 172.21.15.6:0/3213105289 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc460003d30 con 0x7fc4700fce80 2024-03-22T06:22:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.709+0000 7fc478572700 1 -- 172.21.15.6:0/3213105289 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47010f5c0 con 0x7fc4700fce80 2024-03-22T06:22:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.710+0000 7fc478572700 1 -- 172.21.15.6:0/3213105289 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47005f5f0 con 0x7fc4700fce80 2024-03-22T06:22:38.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.710+0000 7fc46effd700 1 -- 172.21.15.6:0/3213105289 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fc46000a7a0 con 0x7fc4700fce80 2024-03-22T06:22:38.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.711+0000 7fc46effd700 1 --2- 172.21.15.6:0/3213105289 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc45806e850 0x7fc458070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:38.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.711+0000 7fc46effd700 1 -- 172.21.15.6:0/3213105289 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6700+0+0 (secure 0 0 0) 0x7fc46008e9a0 con 0x7fc4700fce80 2024-03-22T06:22:38.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.711+0000 7fc475b0d700 1 --2- 172.21.15.6:0/3213105289 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc45806e850 0x7fc458070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:38.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.711+0000 7fc475b0d700 1 --2- 172.21.15.6:0/3213105289 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc45806e850 0x7fc458070d00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc464009fd0 tx=0x7fc464009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:38.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.715+0000 7fc46effd700 1 -- 172.21.15.6:0/3213105289 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fc46005b3a0 con 0x7fc4700fce80 2024-03-22T06:22:38.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.970+0000 7fc478572700 1 -- 172.21.15.6:0/3213105289 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc47010f9a0 con 0x7fc4700fce80 2024-03-22T06:22:38.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.970+0000 7fc46effd700 1 -- 172.21.15.6:0/3213105289 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1885 (secure 0 0 0) 0x7fc460004360 con 0x7fc4700fce80 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:e19 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 19 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:22:23.926858+0000 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:22:38.971 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 92 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=14476} 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:22:38.972 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 14476 members: 14476 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{0:14476} state up:active seq 151 join_fscid=1 addr [v2:172.21.15.6:6828/947853733,v1:172.21.15.6:6829/947853733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:24281} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:22:38.973 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{-1:34406} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:22:38.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.972+0000 7fc478572700 1 -- 172.21.15.6:0/3213105289 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc45806e850 msgr2=0x7fc458070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:38.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.972+0000 7fc478572700 1 --2- 172.21.15.6:0/3213105289 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc45806e850 0x7fc458070d00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc464009fd0 tx=0x7fc464009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:38.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.972+0000 7fc478572700 1 -- 172.21.15.6:0/3213105289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4700fce80 msgr2=0x7fc47010da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:38.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.972+0000 7fc478572700 1 --2- 172.21.15.6:0/3213105289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4700fce80 0x7fc47010da10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc46000cb10 tx=0x7fc460007f90 comp rx=0 tx=0).stop 2024-03-22T06:22:38.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.972+0000 7fc478572700 1 -- 172.21.15.6:0/3213105289 shutdown_connections 2024-03-22T06:22:38.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.972+0000 7fc478572700 1 --2- 172.21.15.6:0/3213105289 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fc45806e850 0x7fc458070d00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.972+0000 7fc478572700 1 --2- 172.21.15.6:0/3213105289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4700fce80 0x7fc47010da10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.972+0000 7fc478572700 1 --2- 172.21.15.6:0/3213105289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4700fdff0 0x7fc47010df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:38.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.972+0000 7fc478572700 1 -- 172.21.15.6:0/3213105289 >> 172.21.15.6:0/3213105289 conn(0x7fc4700f8560 msgr2=0x7fc4700f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:38.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.972+0000 7fc478572700 1 -- 172.21.15.6:0/3213105289 shutdown_connections 2024-03-22T06:22:38.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:38.972+0000 7fc478572700 1 -- 172.21.15.6:0/3213105289 wait complete. 2024-03-22T06:22:38.975 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 19 2024-03-22T06:22:39.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.076+0000 7f65e168d700 1 -- 172.21.15.6:0/2299057886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65dc0fbcd0 msgr2=0x7f65dc0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:39.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.076+0000 7f65e168d700 1 --2- 172.21.15.6:0/2299057886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65dc0fbcd0 0x7f65dc0fc0e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65cc004940 tx=0x7f65cc009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:39.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.076+0000 7f65e168d700 1 -- 172.21.15.6:0/2299057886 shutdown_connections 2024-03-22T06:22:39.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.076+0000 7f65e168d700 1 --2- 172.21.15.6:0/2299057886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65dc0fcf80 0x7f65dc0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.076+0000 7f65e168d700 1 --2- 172.21.15.6:0/2299057886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65dc0fbcd0 0x7f65dc0fc0e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.076+0000 7f65e168d700 1 -- 172.21.15.6:0/2299057886 >> 172.21.15.6:0/2299057886 conn(0x7f65dc0f7450 msgr2=0x7f65dc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:39.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.076+0000 7f65e168d700 1 -- 172.21.15.6:0/2299057886 shutdown_connections 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.076+0000 7f65e168d700 1 -- 172.21.15.6:0/2299057886 wait complete. 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.077+0000 7f65e168d700 1 Processor -- start 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.077+0000 7f65e168d700 1 -- start start 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.077+0000 7f65e168d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65dc0fbcd0 0x7f65dc10c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.077+0000 7f65e168d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65dc0fcf80 0x7f65dc10cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.077+0000 7f65e168d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc10d360 con 0x7f65dc0fcf80 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.077+0000 7f65e168d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc10d4d0 con 0x7f65dc0fbcd0 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.077+0000 7f65da7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65dc0fcf80 0x7f65dc10cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.077+0000 7f65da7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65dc0fcf80 0x7f65dc10cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41954/0 (socket says 172.21.15.6:41954) 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.077+0000 7f65da7fc700 1 -- 172.21.15.6:0/3069282434 learned_addr learned my addr 172.21.15.6:0/3069282434 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.078+0000 7f65daffd700 1 --2- 172.21.15.6:0/3069282434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65dc0fbcd0 0x7f65dc10c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.078+0000 7f65da7fc700 1 -- 172.21.15.6:0/3069282434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65dc0fbcd0 msgr2=0x7f65dc10c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.078+0000 7f65da7fc700 1 --2- 172.21.15.6:0/3069282434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65dc0fbcd0 0x7f65dc10c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.078+0000 7f65da7fc700 1 -- 172.21.15.6:0/3069282434 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc005040 con 0x7f65dc0fcf80 2024-03-22T06:22:39.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.078+0000 7f65da7fc700 1 --2- 172.21.15.6:0/3069282434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65dc0fcf80 0x7f65dc10cd60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65d0002a50 tx=0x7f65d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:39.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.078+0000 7f65c37fe700 1 -- 172.21.15.6:0/3069282434 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d0007cc0 con 0x7f65dc0fcf80 2024-03-22T06:22:39.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.078+0000 7f65c37fe700 1 -- 172.21.15.6:0/3069282434 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65d000f040 con 0x7f65dc0fcf80 2024-03-22T06:22:39.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.078+0000 7f65e168d700 1 -- 172.21.15.6:0/3069282434 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc111f30 con 0x7f65dc0fcf80 2024-03-22T06:22:39.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.078+0000 7f65c37fe700 1 -- 172.21.15.6:0/3069282434 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d000da30 con 0x7f65dc0fcf80 2024-03-22T06:22:39.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.078+0000 7f65e168d700 1 -- 172.21.15.6:0/3069282434 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc1124c0 con 0x7f65dc0fcf80 2024-03-22T06:22:39.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.079+0000 7f65e168d700 1 -- 172.21.15.6:0/3069282434 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65dc04c710 con 0x7f65dc0fcf80 2024-03-22T06:22:39.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.080+0000 7f65c37fe700 1 -- 172.21.15.6:0/3069282434 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f65d000db90 con 0x7f65dc0fcf80 2024-03-22T06:22:39.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.080+0000 7f65c37fe700 1 --2- 172.21.15.6:0/3069282434 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f65c406e840 0x7f65c4070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:39.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.080+0000 7f65c37fe700 1 -- 172.21.15.6:0/3069282434 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6700+0+0 (secure 0 0 0) 0x7f65d008e270 con 0x7f65dc0fcf80 2024-03-22T06:22:39.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.080+0000 7f65daffd700 1 --2- 172.21.15.6:0/3069282434 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f65c406e840 0x7f65c4070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:39.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.080+0000 7f65daffd700 1 --2- 172.21.15.6:0/3069282434 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f65c406e840 0x7f65c4070cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009fd0 tx=0x7f65cc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:39.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.085+0000 7f65c37fe700 1 -- 172.21.15.6:0/3069282434 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f65d005ac70 con 0x7f65dc0fcf80 2024-03-22T06:22:39.220 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:39 smithi006 ceph-mon[94482]: Upgrade: It appears safe to stop mds.cephfs.smithi006.dcjpla 2024-03-22T06:22:39.220 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:39 smithi006 ceph-mon[94482]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:39.221 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:39 smithi006 ceph-mon[94482]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:39.221 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:39 smithi006 ceph-mon[94482]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:39.221 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:39 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3180476625' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:39.221 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:39 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3213105289' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:22:39.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.317+0000 7f65e168d700 1 -- 172.21.15.6:0/3069282434 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f65dc101290 con 0x7f65c406e840 2024-03-22T06:22:39.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.319+0000 7f65c37fe700 1 -- 172.21.15.6:0/3069282434 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f65dc101290 con 0x7f65c406e840 2024-03-22T06:22:39.319 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:22:39.319 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:22:39.319 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:22:39.319 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:22:39.319 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:22:39.319 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:22:39.319 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:22:39.319 INFO:teuthology.orchestra.run.smithi006.stdout: "osd", 2024-03-22T06:22:39.320 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:22:39.320 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:22:39.320 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "13/21 daemons upgraded", 2024-03-22T06:22:39.320 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading mds daemons", 2024-03-22T06:22:39.320 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:22:39.320 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:22:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.321+0000 7f65e168d700 1 -- 172.21.15.6:0/3069282434 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f65c406e840 msgr2=0x7f65c4070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.321+0000 7f65e168d700 1 --2- 172.21.15.6:0/3069282434 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f65c406e840 0x7f65c4070cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009fd0 tx=0x7f65cc009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.321+0000 7f65e168d700 1 -- 172.21.15.6:0/3069282434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65dc0fcf80 msgr2=0x7f65dc10cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.321+0000 7f65e168d700 1 --2- 172.21.15.6:0/3069282434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65dc0fcf80 0x7f65dc10cd60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65d0002a50 tx=0x7f65d00046b0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.321+0000 7f65e168d700 1 -- 172.21.15.6:0/3069282434 shutdown_connections 2024-03-22T06:22:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.321+0000 7f65e168d700 1 --2- 172.21.15.6:0/3069282434 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f65c406e840 0x7f65c4070cf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.321+0000 7f65e168d700 1 --2- 172.21.15.6:0/3069282434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65dc0fbcd0 0x7f65dc10c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.321+0000 7f65e168d700 1 --2- 172.21.15.6:0/3069282434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65dc0fcf80 0x7f65dc10cd60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.321+0000 7f65e168d700 1 -- 172.21.15.6:0/3069282434 >> 172.21.15.6:0/3069282434 conn(0x7f65dc0f7450 msgr2=0x7f65dc0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.321+0000 7f65e168d700 1 -- 172.21.15.6:0/3069282434 shutdown_connections 2024-03-22T06:22:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.321+0000 7f65e168d700 1 -- 172.21.15.6:0/3069282434 wait complete. 2024-03-22T06:22:39.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.428+0000 7f36735c3700 1 -- 172.21.15.6:0/3424318725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366c0fe0d0 msgr2=0x7f366c0fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:39.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.428+0000 7f36735c3700 1 --2- 172.21.15.6:0/3424318725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366c0fe0d0 0x7f366c0fe540 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f365c004940 tx=0x7f365c009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:39.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.428+0000 7f36735c3700 1 -- 172.21.15.6:0/3424318725 shutdown_connections 2024-03-22T06:22:39.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.428+0000 7f36735c3700 1 --2- 172.21.15.6:0/3424318725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366c0fe0d0 0x7f366c0fe540 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.428+0000 7f36735c3700 1 --2- 172.21.15.6:0/3424318725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0fcee0 0x7f366c0fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.428+0000 7f36735c3700 1 -- 172.21.15.6:0/3424318725 >> 172.21.15.6:0/3424318725 conn(0x7f366c0f8560 msgr2=0x7f366c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:39.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.428+0000 7f36735c3700 1 -- 172.21.15.6:0/3424318725 shutdown_connections 2024-03-22T06:22:39.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.428+0000 7f36735c3700 1 -- 172.21.15.6:0/3424318725 wait complete. 2024-03-22T06:22:39.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.428+0000 7f36735c3700 1 Processor -- start 2024-03-22T06:22:39.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f36735c3700 1 -- start start 2024-03-22T06:22:39.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f36735c3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366c0fcee0 0x7f366c10da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:39.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f36735c3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0fe0d0 0x7f366c10dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:39.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f36735c3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c10e5a0 con 0x7f366c0fcee0 2024-03-22T06:22:39.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f36735c3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c10edf0 con 0x7f366c0fe0d0 2024-03-22T06:22:39.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f367135f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366c0fcee0 0x7f366c10da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f367135f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366c0fcee0 0x7f366c10da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41982/0 (socket says 172.21.15.6:41982) 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f367135f700 1 -- 172.21.15.6:0/54814509 learned_addr learned my addr 172.21.15.6:0/54814509 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f3670b5e700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0fe0d0 0x7f366c10dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f367135f700 1 -- 172.21.15.6:0/54814509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0fe0d0 msgr2=0x7f366c10dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f367135f700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0fe0d0 0x7f366c10dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f367135f700 1 -- 172.21.15.6:0/54814509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c005040 con 0x7f366c0fcee0 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f3670b5e700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0fe0d0 0x7f366c10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.429+0000 7f367135f700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366c0fcee0 0x7f366c10da60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f366800c800 tx=0x7f3668004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.430+0000 7f3661ffb700 1 -- 172.21.15.6:0/54814509 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f366800a760 con 0x7f366c0fcee0 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.430+0000 7f3661ffb700 1 -- 172.21.15.6:0/54814509 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f366800ad90 con 0x7f366c0fcee0 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.430+0000 7f36735c3700 1 -- 172.21.15.6:0/54814509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366c10f0d0 con 0x7f366c0fcee0 2024-03-22T06:22:39.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.430+0000 7f3661ffb700 1 -- 172.21.15.6:0/54814509 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36680113a0 con 0x7f366c0fcee0 2024-03-22T06:22:39.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.430+0000 7f36735c3700 1 -- 172.21.15.6:0/54814509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366c10f610 con 0x7f366c0fcee0 2024-03-22T06:22:39.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.431+0000 7f36735c3700 1 -- 172.21.15.6:0/54814509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366c05f5f0 con 0x7f366c0fcee0 2024-03-22T06:22:39.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.431+0000 7f3661ffb700 1 -- 172.21.15.6:0/54814509 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f3668012400 con 0x7f366c0fcee0 2024-03-22T06:22:39.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.432+0000 7f3661ffb700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f365806e840 0x7f3658070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:22:39.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.432+0000 7f3661ffb700 1 -- 172.21.15.6:0/54814509 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6700+0+0 (secure 0 0 0) 0x7f366808e530 con 0x7f366c0fcee0 2024-03-22T06:22:39.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.432+0000 7f3670b5e700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f365806e840 0x7f3658070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:22:39.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.432+0000 7f3670b5e700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f365806e840 0x7f3658070cf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f365c000c00 tx=0x7f365c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:22:39.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.437+0000 7f3661ffb700 1 -- 172.21.15.6:0/54814509 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f366805af50 con 0x7f366c0fcee0 2024-03-22T06:22:39.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:39 smithi117 ceph-mon[82126]: Upgrade: It appears safe to stop mds.cephfs.smithi006.dcjpla 2024-03-22T06:22:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:39 smithi117 ceph-mon[82126]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:39 smithi117 ceph-mon[82126]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:39 smithi117 ceph-mon[82126]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:39 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3180476625' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:22:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:39 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3213105289' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:22:39.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.728+0000 7f36735c3700 1 -- 172.21.15.6:0/54814509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f366c04bed0 con 0x7f366c0fcee0 2024-03-22T06:22:39.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.729+0000 7f3661ffb700 1 -- 172.21.15.6:0/54814509 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+155 (secure 0 0 0) 0x7f366805aae0 con 0x7f366c0fcee0 2024-03-22T06:22:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:22:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:22:39.730 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in error state 2024-03-22T06:22:39.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.731+0000 7f36735c3700 1 -- 172.21.15.6:0/54814509 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f365806e840 msgr2=0x7f3658070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:39.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.731+0000 7f36735c3700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f365806e840 0x7f3658070cf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f365c000c00 tx=0x7f365c009e30 comp rx=0 tx=0).stop 2024-03-22T06:22:39.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.731+0000 7f36735c3700 1 -- 172.21.15.6:0/54814509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366c0fcee0 msgr2=0x7f366c10da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:22:39.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.731+0000 7f36735c3700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366c0fcee0 0x7f366c10da60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f366800c800 tx=0x7f3668004470 comp rx=0 tx=0).stop 2024-03-22T06:22:39.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.731+0000 7f36735c3700 1 -- 172.21.15.6:0/54814509 shutdown_connections 2024-03-22T06:22:39.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.731+0000 7f36735c3700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f365806e840 0x7f3658070cf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.731+0000 7f36735c3700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366c0fcee0 0x7f366c10da60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.731+0000 7f36735c3700 1 --2- 172.21.15.6:0/54814509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0fe0d0 0x7f366c10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:22:39.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.732+0000 7f36735c3700 1 -- 172.21.15.6:0/54814509 >> 172.21.15.6:0/54814509 conn(0x7f366c0f8560 msgr2=0x7f366c0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:22:39.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.732+0000 7f36735c3700 1 -- 172.21.15.6:0/54814509 shutdown_connections 2024-03-22T06:22:39.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:22:39.732+0000 7f36735c3700 1 -- 172.21.15.6:0/54814509 wait complete. 2024-03-22T06:22:40.577 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:40 smithi006 ceph-mon[94482]: pgmap v210: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:22:40.578 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:40 smithi006 ceph-mon[94482]: Upgrade: Updating mds.cephfs.smithi006.dcjpla 2024-03-22T06:22:40.578 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:40 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:40.578 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:40 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.dcjpla", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:22:40.578 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:40 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:22:40.578 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:40 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/54814509' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:22:40.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:40 smithi117 ceph-mon[82126]: pgmap v210: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:22:40.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:40 smithi117 ceph-mon[82126]: Upgrade: Updating mds.cephfs.smithi006.dcjpla 2024-03-22T06:22:40.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:40 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:40.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:40 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi006.dcjpla", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:22:40.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:40 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:22:40.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:40 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/54814509' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:22:41.442 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:41 smithi006 ceph-mon[94482]: Deploying daemon mds.cephfs.smithi006.dcjpla on smithi006 2024-03-22T06:22:41.442 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:41 smithi006 ceph-mon[94482]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:41.442 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:41 smithi006 ceph-mon[94482]: pgmap v211: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:22:41.755 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:22:41 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:22:41.468+0000 7f74a9906700 -1 bdev(0x55dce8fcc000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 2 2024-03-22T06:22:41.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:41 smithi117 ceph-mon[82126]: Deploying daemon mds.cephfs.smithi006.dcjpla on smithi006 2024-03-22T06:22:41.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:41 smithi117 ceph-mon[82126]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:22:41.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:41 smithi117 ceph-mon[82126]: pgmap v211: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:22:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:41 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006[94477]: 2024-03-22T06:22:41.441+0000 7fe66190e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:22:42.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:42 smithi117 ceph-mon[82126]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T06:22:42.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:42 smithi117 ceph-mon[82126]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:22:42.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:42 smithi117 ceph-mon[82126]: osdmap e93: 6 total, 6 up, 6 in 2024-03-22T06:22:42.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:42 smithi117 ceph-mon[82126]: Standby daemon mds.cephfs.smithi117.fwkech assigned to filesystem cephfs as rank 0 2024-03-22T06:22:42.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:42 smithi117 ceph-mon[82126]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T06:22:42.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:42 smithi117 ceph-mon[82126]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T06:22:42.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:42 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi117.fwkech=up:replay} 2 up:standby 2024-03-22T06:22:42.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:42 smithi006 ceph-mon[94482]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T06:22:42.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:42 smithi006 ceph-mon[94482]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:22:42.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:42 smithi006 ceph-mon[94482]: osdmap e93: 6 total, 6 up, 6 in 2024-03-22T06:22:42.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:42 smithi006 ceph-mon[94482]: Standby daemon mds.cephfs.smithi117.fwkech assigned to filesystem cephfs as rank 0 2024-03-22T06:22:42.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:42 smithi006 ceph-mon[94482]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T06:22:42.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:42 smithi006 ceph-mon[94482]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T06:22:42.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:42 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi117.fwkech=up:replay} 2 up:standby 2024-03-22T06:22:43.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:43 smithi117 ceph-mon[82126]: pgmap v213: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 1 op/s 2024-03-22T06:22:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:43 smithi006 ceph-mon[94482]: pgmap v213: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 1 op/s 2024-03-22T06:22:46.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:46 smithi117 ceph-mon[82126]: pgmap v214: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s rd, 2 op/s 2024-03-22T06:22:46.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:46 smithi006 ceph-mon[94482]: pgmap v214: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s rd, 2 op/s 2024-03-22T06:22:48.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:48 smithi006 ceph-mon[94482]: pgmap v215: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-22T06:22:48.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:48 smithi117 ceph-mon[82126]: pgmap v215: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-22T06:22:50.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:50 smithi006 ceph-mon[94482]: pgmap v216: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-22T06:22:50.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:50 smithi117 ceph-mon[82126]: pgmap v216: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-22T06:22:51.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:51 smithi006 ceph-mon[94482]: reconnect by client.14528 v1:172.21.15.117:0/3855819009 after 0.00099998 2024-03-22T06:22:51.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:51 smithi006 ceph-mon[94482]: reconnect by client.24313 v1:172.21.15.6:0/1969668997 after 0.00099998 2024-03-22T06:22:51.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:51 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] up:reconnect 2024-03-22T06:22:51.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:51 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi117.fwkech=up:reconnect} 2 up:standby 2024-03-22T06:22:51.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:51 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:51.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:51 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:51.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:51 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:22:51.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:51 smithi117 ceph-mon[82126]: reconnect by client.14528 v1:172.21.15.117:0/3855819009 after 0.00099998 2024-03-22T06:22:51.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:51 smithi117 ceph-mon[82126]: reconnect by client.24313 v1:172.21.15.6:0/1969668997 after 0.00099998 2024-03-22T06:22:51.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:51 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] up:reconnect 2024-03-22T06:22:51.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:51 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi117.fwkech=up:reconnect} 2 up:standby 2024-03-22T06:22:51.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:51 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:51.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:51 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:51.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:51 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:22:52.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:52 smithi006 ceph-mon[94482]: pgmap v217: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-22T06:22:52.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:52 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] up:rejoin 2024-03-22T06:22:52.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:52 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.6:6828/1244315045,v1:172.21.15.6:6829/1244315045] up:boot 2024-03-22T06:22:52.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:52 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi117.fwkech=up:rejoin} 3 up:standby 2024-03-22T06:22:52.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:22:52.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:52 smithi006 ceph-mon[94482]: daemon mds.cephfs.smithi117.fwkech is now active in filesystem cephfs as rank 0 2024-03-22T06:22:52.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:52 smithi117 ceph-mon[82126]: pgmap v217: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-22T06:22:52.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:52 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] up:rejoin 2024-03-22T06:22:52.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:52 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.6:6828/1244315045,v1:172.21.15.6:6829/1244315045] up:boot 2024-03-22T06:22:52.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:52 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi117.fwkech=up:rejoin} 3 up:standby 2024-03-22T06:22:52.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:22:52.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:52 smithi117 ceph-mon[82126]: daemon mds.cephfs.smithi117.fwkech is now active in filesystem cephfs as rank 0 2024-03-22T06:22:53.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:53 smithi006 ceph-mon[94482]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T06:22:53.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:53 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] up:active 2024-03-22T06:22:53.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:53 smithi006 ceph-mon[94482]: fsmap cephfs:1 {0=cephfs.smithi117.fwkech=up:active} 3 up:standby 2024-03-22T06:22:53.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:22:53.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:53 smithi117 ceph-mon[82126]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T06:22:53.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:53 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.117:6824/1251227895,v1:172.21.15.117:6825/1251227895] up:active 2024-03-22T06:22:53.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:53 smithi117 ceph-mon[82126]: fsmap cephfs:1 {0=cephfs.smithi117.fwkech=up:active} 3 up:standby 2024-03-22T06:22:53.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:22:54.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:54 smithi006 ceph-mon[94482]: pgmap v218: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 87 B/s wr, 6 op/s 2024-03-22T06:22:54.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:54 smithi117 ceph-mon[82126]: pgmap v218: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 87 B/s wr, 6 op/s 2024-03-22T06:22:56.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:55 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:56.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:55 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:56.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:55 smithi117 ceph-mon[82126]: pgmap v219: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T06:22:56.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:55 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:56.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:55 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:56.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:55 smithi006 ceph-mon[94482]: pgmap v219: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T06:22:57.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:57.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:57.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:22:58.390 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:22:58 smithi006 ceph-mon[94482]: pgmap v220: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T06:22:58.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:22:58 smithi117 ceph-mon[82126]: pgmap v220: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T06:23:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: pgmap v221: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 4.6 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-22T06:23:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:23:00.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:23:00.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:00.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:00.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:00.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:00.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:00.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:00.533 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi117.fwkech"]}]: dispatch 2024-03-22T06:23:00.628 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: pgmap v221: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 4.6 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-22T06:23:00.628 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:00.628 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:00.628 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:23:00.628 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:23:00.628 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:00.628 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:00.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:00.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:00.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:00.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:00.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi117.fwkech"]}]: dispatch 2024-03-22T06:23:01.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:01 smithi006 ceph-mon[94482]: pgmap v222: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-03-22T06:23:01.548 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:01 smithi006 ceph-mon[94482]: pgmap v223: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-03-22T06:23:01.548 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:01 smithi006 ceph-mon[94482]: Upgrade: It appears safe to stop mds.cephfs.smithi117.fwkech 2024-03-22T06:23:01.575 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:01 smithi117 ceph-mon[82126]: pgmap v222: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-03-22T06:23:01.576 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:01 smithi117 ceph-mon[82126]: pgmap v223: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-03-22T06:23:01.576 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:01 smithi117 ceph-mon[82126]: Upgrade: It appears safe to stop mds.cephfs.smithi117.fwkech 2024-03-22T06:23:02.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:02 smithi117 ceph-mon[82126]: Upgrade: Updating mds.cephfs.smithi117.fwkech 2024-03-22T06:23:02.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:02.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.fwkech", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:23:02.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:23:02.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:02 smithi117 ceph-mon[82126]: Deploying daemon mds.cephfs.smithi117.fwkech on smithi117 2024-03-22T06:23:02.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:02 smithi117 ceph-mon[82126]: pgmap v224: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 5.9 KiB/s wr, 4 op/s 2024-03-22T06:23:03.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:02 smithi006 ceph-mon[94482]: Upgrade: Updating mds.cephfs.smithi117.fwkech 2024-03-22T06:23:03.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:03.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.fwkech", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:23:03.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:23:03.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:02 smithi006 ceph-mon[94482]: Deploying daemon mds.cephfs.smithi117.fwkech on smithi117 2024-03-22T06:23:03.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:02 smithi006 ceph-mon[94482]: pgmap v224: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 5.9 KiB/s wr, 4 op/s 2024-03-22T06:23:04.006 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:23:03 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:23:03.683+0000 7f74ad90e700 -1 bdev(0x55dce8fcc000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 3 2024-03-22T06:23:04.006 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:23:03 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:23:03.683+0000 7f74a8904700 -1 bdev(0x55dce8fcc000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 3 2024-03-22T06:23:04.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:03 smithi117 ceph-mon[82126]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T06:23:04.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:03 smithi117 ceph-mon[82126]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:23:04.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:03 smithi117 ceph-mon[82126]: osdmap e94: 6 total, 6 up, 6 in 2024-03-22T06:23:04.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:03 smithi117 ceph-mon[82126]: Standby daemon mds.cephfs.smithi117.zqxyql assigned to filesystem cephfs as rank 0 2024-03-22T06:23:04.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:03 smithi117 ceph-mon[82126]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T06:23:04.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:03 smithi117 ceph-mon[82126]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T06:23:04.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:03 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi117.zqxyql=up:replay} 2 up:standby 2024-03-22T06:23:04.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:03 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006[94477]: 2024-03-22T06:23:03.661+0000 7fe66190e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:23:04.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:03 smithi006 ceph-mon[94482]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T06:23:04.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:03 smithi006 ceph-mon[94482]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:23:04.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:03 smithi006 ceph-mon[94482]: osdmap e94: 6 total, 6 up, 6 in 2024-03-22T06:23:04.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:03 smithi006 ceph-mon[94482]: Standby daemon mds.cephfs.smithi117.zqxyql assigned to filesystem cephfs as rank 0 2024-03-22T06:23:04.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:03 smithi006 ceph-mon[94482]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T06:23:04.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:03 smithi006 ceph-mon[94482]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T06:23:04.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:03 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi117.zqxyql=up:replay} 2 up:standby 2024-03-22T06:23:05.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:04 smithi117 ceph-mon[82126]: pgmap v226: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 7.5 KiB/s wr, 3 op/s 2024-03-22T06:23:05.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:04 smithi006 ceph-mon[94482]: pgmap v226: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 7.5 KiB/s wr, 3 op/s 2024-03-22T06:23:07.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:06 smithi117 ceph-mon[82126]: pgmap v227: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 7.8 MiB/s rd, 0 B/s wr, 3 op/s 2024-03-22T06:23:07.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:06 smithi006 ceph-mon[94482]: pgmap v227: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 7.8 MiB/s rd, 0 B/s wr, 3 op/s 2024-03-22T06:23:08.530 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:08.530 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:23:08.530 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:08 smithi117 ceph-mon[82126]: pgmap v228: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 9.7 MiB/s rd, 0 B/s wr, 3 op/s 2024-03-22T06:23:08.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:08.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:23:08.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:08 smithi006 ceph-mon[94482]: pgmap v228: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 9.7 MiB/s rd, 0 B/s wr, 3 op/s 2024-03-22T06:23:09.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.851+0000 7f5e64ee5700 1 -- 172.21.15.6:0/2234722203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e600fff00 msgr2=0x7f5e60102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:09.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.851+0000 7f5e64ee5700 1 --2- 172.21.15.6:0/2234722203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e600fff00 0x7f5e60102320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5e48004940 tx=0x7f5e48009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:09.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 -- 172.21.15.6:0/2234722203 shutdown_connections 2024-03-22T06:23:09.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 --2- 172.21.15.6:0/2234722203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e600fff00 0x7f5e60102320 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:09.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 --2- 172.21.15.6:0/2234722203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e600662e0 0x7f5e600ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:09.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 -- 172.21.15.6:0/2234722203 >> 172.21.15.6:0/2234722203 conn(0x7f5e600f7430 msgr2=0x7f5e600f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:09.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 -- 172.21.15.6:0/2234722203 shutdown_connections 2024-03-22T06:23:09.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 -- 172.21.15.6:0/2234722203 wait complete. 2024-03-22T06:23:09.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 Processor -- start 2024-03-22T06:23:09.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 -- start start 2024-03-22T06:23:09.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e600662e0 0x7f5e600fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:09.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e600fff00 0x7f5e600fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:09.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600fbe00 con 0x7f5e600662e0 2024-03-22T06:23:09.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e64ee5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600fbf40 con 0x7f5e600fff00 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.852+0000 7f5e5e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e600662e0 0x7f5e600fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e5e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e600662e0 0x7f5e600fd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49348/0 (socket says 172.21.15.6:49348) 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e5e59c700 1 -- 172.21.15.6:0/3657162092 learned_addr learned my addr 172.21.15.6:0/3657162092 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e5dd9b700 1 --2- 172.21.15.6:0/3657162092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e600fff00 0x7f5e600fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e5e59c700 1 -- 172.21.15.6:0/3657162092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e600fff00 msgr2=0x7f5e600fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e5e59c700 1 --2- 172.21.15.6:0/3657162092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e600fff00 0x7f5e600fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e5e59c700 1 -- 172.21.15.6:0/3657162092 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e48005040 con 0x7f5e600662e0 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e5e59c700 1 --2- 172.21.15.6:0/3657162092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e600662e0 0x7f5e600fd350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000ec00 tx=0x7f5e5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e56ffd700 1 -- 172.21.15.6:0/3657162092 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e50007b30 con 0x7f5e600662e0 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e56ffd700 1 -- 172.21.15.6:0/3657162092 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e5000f040 con 0x7f5e600662e0 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e64ee5700 1 -- 172.21.15.6:0/3657162092 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e600fc220 con 0x7f5e600662e0 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e56ffd700 1 -- 172.21.15.6:0/3657162092 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e5000cc00 con 0x7f5e600662e0 2024-03-22T06:23:09.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.853+0000 7f5e64ee5700 1 -- 172.21.15.6:0/3657162092 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e60115ac0 con 0x7f5e600662e0 2024-03-22T06:23:09.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.854+0000 7f5e64ee5700 1 -- 172.21.15.6:0/3657162092 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6005f5f0 con 0x7f5e600662e0 2024-03-22T06:23:09.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.855+0000 7f5e56ffd700 1 -- 172.21.15.6:0/3657162092 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f5e50007c90 con 0x7f5e600662e0 2024-03-22T06:23:09.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.855+0000 7f5e56ffd700 1 --2- 172.21.15.6:0/3657162092 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5e4c06e840 0x7f5e4c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:09.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.855+0000 7f5e56ffd700 1 -- 172.21.15.6:0/3657162092 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f5e5008e9d0 con 0x7f5e600662e0 2024-03-22T06:23:09.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.855+0000 7f5e5dd9b700 1 --2- 172.21.15.6:0/3657162092 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5e4c06e840 0x7f5e4c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:09.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.856+0000 7f5e5dd9b700 1 --2- 172.21.15.6:0/3657162092 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5e4c06e840 0x7f5e4c070cf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009b30 tx=0x7f5e48009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:09.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:09.860+0000 7f5e56ffd700 1 -- 172.21.15.6:0/3657162092 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f5e500577f0 con 0x7f5e600662e0 2024-03-22T06:23:10.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.086+0000 7f5e64ee5700 1 -- 172.21.15.6:0/3657162092 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e600f8f70 con 0x7f5e4c06e840 2024-03-22T06:23:10.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.088+0000 7f5e56ffd700 1 -- 172.21.15.6:0/3657162092 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5e600f8f70 con 0x7f5e4c06e840 2024-03-22T06:23:10.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.090+0000 7f5e64ee5700 1 -- 172.21.15.6:0/3657162092 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5e4c06e840 msgr2=0x7f5e4c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.090+0000 7f5e64ee5700 1 --2- 172.21.15.6:0/3657162092 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5e4c06e840 0x7f5e4c070cf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009b30 tx=0x7f5e48009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:10.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.090+0000 7f5e64ee5700 1 -- 172.21.15.6:0/3657162092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e600662e0 msgr2=0x7f5e600fd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.090+0000 7f5e64ee5700 1 --2- 172.21.15.6:0/3657162092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e600662e0 0x7f5e600fd350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000ec00 tx=0x7f5e5000b040 comp rx=0 tx=0).stop 2024-03-22T06:23:10.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.090+0000 7f5e64ee5700 1 -- 172.21.15.6:0/3657162092 shutdown_connections 2024-03-22T06:23:10.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.090+0000 7f5e64ee5700 1 --2- 172.21.15.6:0/3657162092 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5e4c06e840 0x7f5e4c070cf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.090+0000 7f5e64ee5700 1 --2- 172.21.15.6:0/3657162092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e600662e0 0x7f5e600fd350 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.090+0000 7f5e64ee5700 1 --2- 172.21.15.6:0/3657162092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e600fff00 0x7f5e600fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.090+0000 7f5e64ee5700 1 -- 172.21.15.6:0/3657162092 >> 172.21.15.6:0/3657162092 conn(0x7f5e600f7430 msgr2=0x7f5e600f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.090+0000 7f5e64ee5700 1 -- 172.21.15.6:0/3657162092 shutdown_connections 2024-03-22T06:23:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.090+0000 7f5e64ee5700 1 -- 172.21.15.6:0/3657162092 wait complete. 2024-03-22T06:23:10.101 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:23:10.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.193+0000 7f9f0c28a700 1 -- 172.21.15.6:0/3054548744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f040fdfe0 msgr2=0x7f9f040fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.193+0000 7f9f0c28a700 1 --2- 172.21.15.6:0/3054548744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f040fdfe0 0x7f9f040fe450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4004940 tx=0x7f9ef4009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 -- 172.21.15.6:0/3054548744 shutdown_connections 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 --2- 172.21.15.6:0/3054548744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f040fdfe0 0x7f9f040fe450 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 --2- 172.21.15.6:0/3054548744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f040fce70 0x7f9f040fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 -- 172.21.15.6:0/3054548744 >> 172.21.15.6:0/3054548744 conn(0x7f9f040f8550 msgr2=0x7f9f040fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 -- 172.21.15.6:0/3054548744 shutdown_connections 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 -- 172.21.15.6:0/3054548744 wait complete. 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 Processor -- start 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 -- start start 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f040fce70 0x7f9f04071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f040fdfe0 0x7f9f04070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04072260 con 0x7f9f040fdfe0 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.194+0000 7f9f0c28a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04070850 con 0x7f9f040fce70 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9f09825700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f040fdfe0 0x7f9f04070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:10.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9f09825700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f040fdfe0 0x7f9f04070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49364/0 (socket says 172.21.15.6:49364) 2024-03-22T06:23:10.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9f09825700 1 -- 172.21.15.6:0/627236081 learned_addr learned my addr 172.21.15.6:0/627236081 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:10.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9f0a026700 1 --2- 172.21.15.6:0/627236081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f040fce70 0x7f9f04071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:10.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9f09825700 1 -- 172.21.15.6:0/627236081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f040fce70 msgr2=0x7f9f04071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9f09825700 1 --2- 172.21.15.6:0/627236081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f040fce70 0x7f9f04071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9f09825700 1 -- 172.21.15.6:0/627236081 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4005040 con 0x7f9f040fdfe0 2024-03-22T06:23:10.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9f09825700 1 --2- 172.21.15.6:0/627236081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f040fdfe0 0x7f9f04070310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4013870 tx=0x7f9ef4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:10.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9efaffd700 1 -- 172.21.15.6:0/627236081 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ef4021070 con 0x7f9f040fdfe0 2024-03-22T06:23:10.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9efaffd700 1 -- 172.21.15.6:0/627236081 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ef4009430 con 0x7f9f040fdfe0 2024-03-22T06:23:10.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9f0c28a700 1 -- 172.21.15.6:0/627236081 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f04070aa0 con 0x7f9f040fdfe0 2024-03-22T06:23:10.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9efaffd700 1 -- 172.21.15.6:0/627236081 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ef400e070 con 0x7f9f040fdfe0 2024-03-22T06:23:10.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.195+0000 7f9f0c28a700 1 -- 172.21.15.6:0/627236081 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f04070fe0 con 0x7f9f040fdfe0 2024-03-22T06:23:10.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.196+0000 7f9f0c28a700 1 -- 172.21.15.6:0/627236081 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee80052e0 con 0x7f9f040fdfe0 2024-03-22T06:23:10.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.197+0000 7f9efaffd700 1 -- 172.21.15.6:0/627236081 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f9ef4013af0 con 0x7f9f040fdfe0 2024-03-22T06:23:10.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.197+0000 7f9efaffd700 1 --2- 172.21.15.6:0/627236081 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9ef006e840 0x7f9ef0070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:10.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.198+0000 7f9efaffd700 1 -- 172.21.15.6:0/627236081 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f9ef408f530 con 0x7f9f040fdfe0 2024-03-22T06:23:10.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.198+0000 7f9f0a026700 1 --2- 172.21.15.6:0/627236081 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9ef006e840 0x7f9ef0070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:10.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.198+0000 7f9f0a026700 1 --2- 172.21.15.6:0/627236081 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9ef006e840 0x7f9ef0070cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009b80 tx=0x7f9f0000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:10.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.203+0000 7f9efaffd700 1 -- 172.21.15.6:0/627236081 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f9ef405bfa0 con 0x7f9f040fdfe0 2024-03-22T06:23:10.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.427+0000 7f9f0c28a700 1 -- 172.21.15.6:0/627236081 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ee8000bf0 con 0x7f9ef006e840 2024-03-22T06:23:10.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.428+0000 7f9efaffd700 1 -- 172.21.15.6:0/627236081 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9ee8000bf0 con 0x7f9ef006e840 2024-03-22T06:23:10.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.430+0000 7f9f0c28a700 1 -- 172.21.15.6:0/627236081 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9ef006e840 msgr2=0x7f9ef0070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.430+0000 7f9f0c28a700 1 --2- 172.21.15.6:0/627236081 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9ef006e840 0x7f9ef0070cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009b80 tx=0x7f9f0000b040 comp rx=0 tx=0).stop 2024-03-22T06:23:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.430+0000 7f9f0c28a700 1 -- 172.21.15.6:0/627236081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f040fdfe0 msgr2=0x7f9f04070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.430+0000 7f9f0c28a700 1 --2- 172.21.15.6:0/627236081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f040fdfe0 0x7f9f04070310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4013870 tx=0x7f9ef4009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.430+0000 7f9f0c28a700 1 -- 172.21.15.6:0/627236081 shutdown_connections 2024-03-22T06:23:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.430+0000 7f9f0c28a700 1 --2- 172.21.15.6:0/627236081 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f9ef006e840 0x7f9ef0070cf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.430+0000 7f9f0c28a700 1 --2- 172.21.15.6:0/627236081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f040fce70 0x7f9f04071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.430+0000 7f9f0c28a700 1 --2- 172.21.15.6:0/627236081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f040fdfe0 0x7f9f04070310 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.430+0000 7f9f0c28a700 1 -- 172.21.15.6:0/627236081 >> 172.21.15.6:0/627236081 conn(0x7f9f040f8550 msgr2=0x7f9f040f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.430+0000 7f9f0c28a700 1 -- 172.21.15.6:0/627236081 shutdown_connections 2024-03-22T06:23:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.430+0000 7f9f0c28a700 1 -- 172.21.15.6:0/627236081 wait complete. 2024-03-22T06:23:10.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 -- 172.21.15.6:0/3919259283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17b40fff00 msgr2=0x7f17b4102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 --2- 172.21.15.6:0/3919259283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17b40fff00 0x7f17b4102320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f17a8004940 tx=0x7f17a8009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 -- 172.21.15.6:0/3919259283 shutdown_connections 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 --2- 172.21.15.6:0/3919259283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17b40fff00 0x7f17b4102320 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 --2- 172.21.15.6:0/3919259283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b40662e0 0x7f17b40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 -- 172.21.15.6:0/3919259283 >> 172.21.15.6:0/3919259283 conn(0x7f17b40f7430 msgr2=0x7f17b40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 -- 172.21.15.6:0/3919259283 shutdown_connections 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 -- 172.21.15.6:0/3919259283 wait complete. 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 Processor -- start 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 -- start start 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17b40662e0 0x7f17b40fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b40fff00 0x7f17b40fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b40fbe50 con 0x7f17b40662e0 2024-03-22T06:23:10.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.533+0000 7f17b89fc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b40fbf90 con 0x7f17b40fff00 2024-03-22T06:23:10.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17b259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17b40662e0 0x7f17b40fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:10.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17b259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17b40662e0 0x7f17b40fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49380/0 (socket says 172.21.15.6:49380) 2024-03-22T06:23:10.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17b259c700 1 -- 172.21.15.6:0/445398330 learned_addr learned my addr 172.21.15.6:0/445398330 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:10.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17b1d9b700 1 --2- 172.21.15.6:0/445398330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b40fff00 0x7f17b40fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:10.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17b259c700 1 -- 172.21.15.6:0/445398330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b40fff00 msgr2=0x7f17b40fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17b259c700 1 --2- 172.21.15.6:0/445398330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b40fff00 0x7f17b40fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17b259c700 1 -- 172.21.15.6:0/445398330 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a8005040 con 0x7f17b40662e0 2024-03-22T06:23:10.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17b259c700 1 --2- 172.21.15.6:0/445398330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17b40662e0 0x7f17b40fd3a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f17a400eb40 tx=0x7f17a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:10.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17a2ffd700 1 -- 172.21.15.6:0/445398330 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a4007cc0 con 0x7f17b40662e0 2024-03-22T06:23:10.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17b89fc700 1 -- 172.21.15.6:0/445398330 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b40fc210 con 0x7f17b40662e0 2024-03-22T06:23:10.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17a2ffd700 1 -- 172.21.15.6:0/445398330 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17a400f040 con 0x7f17b40662e0 2024-03-22T06:23:10.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17a2ffd700 1 -- 172.21.15.6:0/445398330 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a400c9d0 con 0x7f17b40662e0 2024-03-22T06:23:10.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.534+0000 7f17b89fc700 1 -- 172.21.15.6:0/445398330 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b40fc720 con 0x7f17b40662e0 2024-03-22T06:23:10.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.535+0000 7f17b89fc700 1 -- 172.21.15.6:0/445398330 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b405f5f0 con 0x7f17b40662e0 2024-03-22T06:23:10.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.536+0000 7f17a2ffd700 1 -- 172.21.15.6:0/445398330 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f17a400cc40 con 0x7f17b40662e0 2024-03-22T06:23:10.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.536+0000 7f17a2ffd700 1 --2- 172.21.15.6:0/445398330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f179c06e840 0x7f179c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:10.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.536+0000 7f17a2ffd700 1 -- 172.21.15.6:0/445398330 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f17a408dfd0 con 0x7f17b40662e0 2024-03-22T06:23:10.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.536+0000 7f17b1d9b700 1 --2- 172.21.15.6:0/445398330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f179c06e840 0x7f179c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:10.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.537+0000 7f17b1d9b700 1 --2- 172.21.15.6:0/445398330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f179c06e840 0x7f179c070cf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009b30 tx=0x7f17a8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:10.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.541+0000 7f17a2ffd700 1 -- 172.21.15.6:0/445398330 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f17a4056ea0 con 0x7f17b40662e0 2024-03-22T06:23:10.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.757+0000 7f17b89fc700 1 -- 172.21.15.6:0/445398330 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f17b40f8f70 con 0x7f179c06e840 2024-03-22T06:23:10.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.767+0000 7f17a2ffd700 1 -- 172.21.15.6:0/445398330 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f17b40f8f70 con 0x7f179c06e840 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (7m) 15s ago 16m 26.5M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (4m) 15s ago 16m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (4m) 68s ago 15m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 error 15s ago 15m - - 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (47s) 15s ago 10m 14.0M - 19.0.0-2299-gea01217f aa461e7c5e7f 3e3319f646e6 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (20s) 15s ago 10m 11.3M - 19.0.0-2299-gea01217f aa461e7c5e7f 09ee1e271adb 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (10m) 68s ago 10m 15.4M - 17.2.7-726-g9c4e0c34 837124da804f dc1e3bd9f352 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (10m) 68s ago 10m 17.2M - 17.2.7-726-g9c4e0c34 837124da804f 431c65fe0bd8 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (6m) 15s ago 17m 509M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (6m) 68s ago 15m 452M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (5m) 15s ago 17m 50.9M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (5m) 68s ago 15m 38.2M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:23:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (8m) 15s ago 16m 17.5M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:23:10.769 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (7m) 68s ago 15m 14.9M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:23:10.769 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (4m) 15s ago 14m 154M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:23:10.769 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (3m) 15s ago 14m 135M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:23:10.769 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (2m) 15s ago 13m 102M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 3ec1f219c19c 2024-03-22T06:23:10.769 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (2m) 68s ago 13m 99.5M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 9cba39b6f32c 2024-03-22T06:23:10.769 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (104s) 68s ago 12m 100M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1950fc21bd2e 2024-03-22T06:23:10.769 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (71s) 68s ago 12m 15.0M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1a6a0ee2a228 2024-03-22T06:23:10.769 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (6m) 15s ago 15m 66.5M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:23:10.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.769+0000 7f17b89fc700 1 -- 172.21.15.6:0/445398330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f179c06e840 msgr2=0x7f179c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.769+0000 7f17b89fc700 1 --2- 172.21.15.6:0/445398330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f179c06e840 0x7f179c070cf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009b30 tx=0x7f17a8009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:10.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.769+0000 7f17b89fc700 1 -- 172.21.15.6:0/445398330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17b40662e0 msgr2=0x7f17b40fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.769+0000 7f17b89fc700 1 --2- 172.21.15.6:0/445398330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17b40662e0 0x7f17b40fd3a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f17a400eb40 tx=0x7f17a400b040 comp rx=0 tx=0).stop 2024-03-22T06:23:10.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.769+0000 7f17b89fc700 1 -- 172.21.15.6:0/445398330 shutdown_connections 2024-03-22T06:23:10.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.769+0000 7f17b89fc700 1 --2- 172.21.15.6:0/445398330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f179c06e840 0x7f179c070cf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.769+0000 7f17b89fc700 1 --2- 172.21.15.6:0/445398330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17b40662e0 0x7f17b40fd3a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.769+0000 7f17b89fc700 1 --2- 172.21.15.6:0/445398330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b40fff00 0x7f17b40fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.769+0000 7f17b89fc700 1 -- 172.21.15.6:0/445398330 >> 172.21.15.6:0/445398330 conn(0x7f17b40f7430 msgr2=0x7f17b40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:10.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.769+0000 7f17b89fc700 1 -- 172.21.15.6:0/445398330 shutdown_connections 2024-03-22T06:23:10.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.769+0000 7f17b89fc700 1 -- 172.21.15.6:0/445398330 wait complete. 2024-03-22T06:23:10.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.870+0000 7f1ae043e700 1 -- 172.21.15.6:0/220437430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ad80662e0 msgr2=0x7f1ad80ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.870+0000 7f1ae043e700 1 --2- 172.21.15.6:0/220437430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ad80662e0 0x7f1ad80ff9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4004940 tx=0x7f1ad4009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:10.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.870+0000 7f1ae043e700 1 -- 172.21.15.6:0/220437430 shutdown_connections 2024-03-22T06:23:10.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.870+0000 7f1ae043e700 1 --2- 172.21.15.6:0/220437430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad80fff00 0x7f1ad8102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.870+0000 7f1ae043e700 1 --2- 172.21.15.6:0/220437430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ad80662e0 0x7f1ad80ff9c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.870+0000 7f1ae043e700 1 -- 172.21.15.6:0/220437430 >> 172.21.15.6:0/220437430 conn(0x7f1ad80f7430 msgr2=0x7f1ad80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:10.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.871+0000 7f1ae043e700 1 -- 172.21.15.6:0/220437430 shutdown_connections 2024-03-22T06:23:10.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.871+0000 7f1ae043e700 1 -- 172.21.15.6:0/220437430 wait complete. 2024-03-22T06:23:10.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.871+0000 7f1ae043e700 1 Processor -- start 2024-03-22T06:23:10.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1ae043e700 1 -- start start 2024-03-22T06:23:10.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1ae043e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad80662e0 0x7f1ad80fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:10.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1ae043e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ad80fff00 0x7f1ad80fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:10.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1ae043e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad80fbe50 con 0x7f1ad80fff00 2024-03-22T06:23:10.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1ae043e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad80fbf90 con 0x7f1ad80662e0 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1add9d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ad80fff00 0x7f1ad80fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1add9d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ad80fff00 0x7f1ad80fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49412/0 (socket says 172.21.15.6:49412) 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1add9d9700 1 -- 172.21.15.6:0/2739472014 learned_addr learned my addr 172.21.15.6:0/2739472014 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1ade1da700 1 --2- 172.21.15.6:0/2739472014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad80662e0 0x7f1ad80fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1add9d9700 1 -- 172.21.15.6:0/2739472014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad80662e0 msgr2=0x7f1ad80fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1add9d9700 1 --2- 172.21.15.6:0/2739472014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad80662e0 0x7f1ad80fd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1add9d9700 1 -- 172.21.15.6:0/2739472014 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad4005040 con 0x7f1ad80fff00 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1add9d9700 1 --2- 172.21.15.6:0/2739472014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ad80fff00 0x7f1ad80fb8c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800eb40 tx=0x7f1ac800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.872+0000 7f1aceffd700 1 -- 172.21.15.6:0/2739472014 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac8007cc0 con 0x7f1ad80fff00 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.873+0000 7f1ae043e700 1 -- 172.21.15.6:0/2739472014 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad80fc210 con 0x7f1ad80fff00 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.873+0000 7f1aceffd700 1 -- 172.21.15.6:0/2739472014 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ac800f040 con 0x7f1ad80fff00 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.873+0000 7f1aceffd700 1 -- 172.21.15.6:0/2739472014 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac800ca10 con 0x7f1ad80fff00 2024-03-22T06:23:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.873+0000 7f1ae043e700 1 -- 172.21.15.6:0/2739472014 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad8115a30 con 0x7f1ad80fff00 2024-03-22T06:23:10.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.874+0000 7f1ae043e700 1 -- 172.21.15.6:0/2739472014 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad805f5f0 con 0x7f1ad80fff00 2024-03-22T06:23:10.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.875+0000 7f1aceffd700 1 -- 172.21.15.6:0/2739472014 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f1ac800cc80 con 0x7f1ad80fff00 2024-03-22T06:23:10.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.875+0000 7f1aceffd700 1 --2- 172.21.15.6:0/2739472014 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1ac406e840 0x7f1ac4070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:10.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.875+0000 7f1aceffd700 1 -- 172.21.15.6:0/2739472014 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f1ac808e130 con 0x7f1ad80fff00 2024-03-22T06:23:10.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.875+0000 7f1ade1da700 1 --2- 172.21.15.6:0/2739472014 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1ac406e840 0x7f1ac4070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:10.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.876+0000 7f1ade1da700 1 --2- 172.21.15.6:0/2739472014 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1ac406e840 0x7f1ac4070cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4013740 tx=0x7f1ad4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:10.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:10.880+0000 7f1aceffd700 1 -- 172.21.15.6:0/2739472014 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f1ac8057000 con 0x7f1ad80fff00 2024-03-22T06:23:11.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.178+0000 7f1ae043e700 1 -- 172.21.15.6:0/2739472014 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1ad804bed0 con 0x7f1ad80fff00 2024-03-22T06:23:11.179 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:10 smithi006 ceph-mon[94482]: pgmap v229: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 0 B/s wr, 4 op/s 2024-03-22T06:23:11.179 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:10 smithi006 ceph-mon[94482]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:11.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.178+0000 7f1aceffd700 1 -- 172.21.15.6:0/2739472014 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f1ac805a6c0 con 0x7f1ad80fff00 2024-03-22T06:23:11.179 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:23:11.179 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:23:11.179 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:23:11.179 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:23:11.179 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 6 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 12 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:23:11.180 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:23:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.181+0000 7f1ae043e700 1 -- 172.21.15.6:0/2739472014 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1ac406e840 msgr2=0x7f1ac4070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.181+0000 7f1ae043e700 1 --2- 172.21.15.6:0/2739472014 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1ac406e840 0x7f1ac4070cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4013740 tx=0x7f1ad4009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.181+0000 7f1ae043e700 1 -- 172.21.15.6:0/2739472014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ad80fff00 msgr2=0x7f1ad80fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.181+0000 7f1ae043e700 1 --2- 172.21.15.6:0/2739472014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ad80fff00 0x7f1ad80fb8c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800eb40 tx=0x7f1ac800b040 comp rx=0 tx=0).stop 2024-03-22T06:23:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.181+0000 7f1ae043e700 1 -- 172.21.15.6:0/2739472014 shutdown_connections 2024-03-22T06:23:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.181+0000 7f1ae043e700 1 --2- 172.21.15.6:0/2739472014 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1ac406e840 0x7f1ac4070cf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.181+0000 7f1ae043e700 1 --2- 172.21.15.6:0/2739472014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad80662e0 0x7f1ad80fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.181+0000 7f1ae043e700 1 --2- 172.21.15.6:0/2739472014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ad80fff00 0x7f1ad80fb8c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.181+0000 7f1ae043e700 1 -- 172.21.15.6:0/2739472014 >> 172.21.15.6:0/2739472014 conn(0x7f1ad80f7430 msgr2=0x7f1ad80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.181+0000 7f1ae043e700 1 -- 172.21.15.6:0/2739472014 shutdown_connections 2024-03-22T06:23:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.181+0000 7f1ae043e700 1 -- 172.21.15.6:0/2739472014 wait complete. 2024-03-22T06:23:11.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:10 smithi117 ceph-mon[82126]: pgmap v229: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 0 B/s wr, 4 op/s 2024-03-22T06:23:11.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:10 smithi117 ceph-mon[82126]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:11.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.283+0000 7f57492bc700 1 -- 172.21.15.6:0/3785176100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57440fe440 msgr2=0x7f57440fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:11.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.283+0000 7f57492bc700 1 --2- 172.21.15.6:0/3785176100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57440fe440 0x7f57440fe850 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5734004940 tx=0x7f5734009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:11.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.283+0000 7f57492bc700 1 -- 172.21.15.6:0/3785176100 shutdown_connections 2024-03-22T06:23:11.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.283+0000 7f57492bc700 1 --2- 172.21.15.6:0/3785176100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57440fed90 0x7f57440faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.283+0000 7f57492bc700 1 --2- 172.21.15.6:0/3785176100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57440fe440 0x7f57440fe850 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.283+0000 7f57492bc700 1 -- 172.21.15.6:0/3785176100 >> 172.21.15.6:0/3785176100 conn(0x7f57440f6be0 msgr2=0x7f57440f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:11.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.283+0000 7f57492bc700 1 -- 172.21.15.6:0/3785176100 shutdown_connections 2024-03-22T06:23:11.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.283+0000 7f57492bc700 1 -- 172.21.15.6:0/3785176100 wait complete. 2024-03-22T06:23:11.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.284+0000 7f57492bc700 1 Processor -- start 2024-03-22T06:23:11.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.284+0000 7f57492bc700 1 -- start start 2024-03-22T06:23:11.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.284+0000 7f57492bc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57440fe440 0x7f574410fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:11.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.284+0000 7f57492bc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57440fed90 0x7f5744110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.284+0000 7f57492bc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5744110610 con 0x7f57440fed90 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.284+0000 7f57492bc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5744110780 con 0x7f57440fe440 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.284+0000 7f57427fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57440fed90 0x7f5744110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.284+0000 7f5742ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57440fe440 0x7f574410fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.284+0000 7f57427fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57440fed90 0x7f5744110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49430/0 (socket says 172.21.15.6:49430) 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.284+0000 7f57427fc700 1 -- 172.21.15.6:0/1203872330 learned_addr learned my addr 172.21.15.6:0/1203872330 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.285+0000 7f5742ffd700 1 -- 172.21.15.6:0/1203872330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57440fed90 msgr2=0x7f5744110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.285+0000 7f5742ffd700 1 --2- 172.21.15.6:0/1203872330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57440fed90 0x7f5744110010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.285+0000 7f5742ffd700 1 -- 172.21.15.6:0/1203872330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5734005040 con 0x7f57440fe440 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.285+0000 7f57427fc700 1 --2- 172.21.15.6:0/1203872330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57440fed90 0x7f5744110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.285+0000 7f5742ffd700 1 --2- 172.21.15.6:0/1203872330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57440fe440 0x7f574410fad0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5734000c00 tx=0x7f5734009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:11.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.285+0000 7f572b7fe700 1 -- 172.21.15.6:0/1203872330 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573400e040 con 0x7f57440fe440 2024-03-22T06:23:11.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.285+0000 7f57492bc700 1 -- 172.21.15.6:0/1203872330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5744115180 con 0x7f57440fe440 2024-03-22T06:23:11.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.285+0000 7f572b7fe700 1 -- 172.21.15.6:0/1203872330 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f573400bbf0 con 0x7f57440fe440 2024-03-22T06:23:11.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.285+0000 7f572b7fe700 1 -- 172.21.15.6:0/1203872330 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573401c650 con 0x7f57440fe440 2024-03-22T06:23:11.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.285+0000 7f57492bc700 1 -- 172.21.15.6:0/1203872330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5744115740 con 0x7f57440fe440 2024-03-22T06:23:11.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.286+0000 7f57492bc700 1 -- 172.21.15.6:0/1203872330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5744101b80 con 0x7f57440fe440 2024-03-22T06:23:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.287+0000 7f572b7fe700 1 -- 172.21.15.6:0/1203872330 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f573400bd60 con 0x7f57440fe440 2024-03-22T06:23:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.287+0000 7f572b7fe700 1 --2- 172.21.15.6:0/1203872330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f572c06e840 0x7f572c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.287+0000 7f572b7fe700 1 -- 172.21.15.6:0/1203872330 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f573408f8d0 con 0x7f57440fe440 2024-03-22T06:23:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.287+0000 7f57427fc700 1 --2- 172.21.15.6:0/1203872330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f572c06e840 0x7f572c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.288+0000 7f57427fc700 1 --2- 172.21.15.6:0/1203872330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f572c06e840 0x7f572c070cf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f573800e5e0 tx=0x7f573800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:11.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.296+0000 7f572b7fe700 1 -- 172.21.15.6:0/1203872330 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f573405c200 con 0x7f57440fe440 2024-03-22T06:23:11.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.552+0000 7f57492bc700 1 -- 172.21.15.6:0/1203872330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f57441015a0 con 0x7f57440fe440 2024-03-22T06:23:11.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.553+0000 7f572b7fe700 1 -- 172.21.15.6:0/1203872330 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1698 (secure 0 0 0) 0x7f573405bd90 con 0x7f57440fe440 2024-03-22T06:23:11.553 INFO:teuthology.orchestra.run.smithi006.stdout:e26 2024-03-22T06:23:11.553 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:23:11.553 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:23:11.553 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:23:11.553 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:23:11.554 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:23:11.554 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:23:11.554 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 26 2024-03-22T06:23:11.554 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:23:11.554 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:23:11.554 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:23:03.668034+0000 2024-03-22T06:23:11.554 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:23:11.554 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:23:11.554 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:23:11.554 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:23:11.554 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 94 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=24281} 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:23:11.555 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:23:11.556 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:23:11.556 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:23:11.556 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:23:11.556 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 0 members: 2024-03-22T06:23:11.556 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{0:24281} state up:replay seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T06:23:11.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:23:11.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:23:11.556 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:23:11.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:23:11.556 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{-1:34406} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:23:11.556 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:34438} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.6:6828/1244315045,v1:172.21.15.6:6829/1244315045] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:23:11.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.555+0000 7f57492bc700 1 -- 172.21.15.6:0/1203872330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f572c06e840 msgr2=0x7f572c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:11.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.555+0000 7f57492bc700 1 --2- 172.21.15.6:0/1203872330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f572c06e840 0x7f572c070cf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f573800e5e0 tx=0x7f573800b040 comp rx=0 tx=0).stop 2024-03-22T06:23:11.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.555+0000 7f57492bc700 1 -- 172.21.15.6:0/1203872330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57440fe440 msgr2=0x7f574410fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:11.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.555+0000 7f57492bc700 1 --2- 172.21.15.6:0/1203872330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57440fe440 0x7f574410fad0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5734000c00 tx=0x7f5734009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:11.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.555+0000 7f57492bc700 1 -- 172.21.15.6:0/1203872330 shutdown_connections 2024-03-22T06:23:11.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.555+0000 7f57492bc700 1 --2- 172.21.15.6:0/1203872330 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f572c06e840 0x7f572c070cf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.555+0000 7f57492bc700 1 --2- 172.21.15.6:0/1203872330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57440fe440 0x7f574410fad0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.555+0000 7f57492bc700 1 --2- 172.21.15.6:0/1203872330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57440fed90 0x7f5744110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.555+0000 7f57492bc700 1 -- 172.21.15.6:0/1203872330 >> 172.21.15.6:0/1203872330 conn(0x7f57440f6be0 msgr2=0x7f57440f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:11.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.555+0000 7f57492bc700 1 -- 172.21.15.6:0/1203872330 shutdown_connections 2024-03-22T06:23:11.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.555+0000 7f57492bc700 1 -- 172.21.15.6:0/1203872330 wait complete. 2024-03-22T06:23:11.558 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-03-22T06:23:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.659+0000 7f3cbc8da700 1 -- 172.21.15.6:0/3077082834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cb80fbcd0 msgr2=0x7f3cb80fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.659+0000 7f3cbc8da700 1 --2- 172.21.15.6:0/3077082834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cb80fbcd0 0x7f3cb80fc0e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0004940 tx=0x7f3ca0009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.659+0000 7f3cbc8da700 1 -- 172.21.15.6:0/3077082834 shutdown_connections 2024-03-22T06:23:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.659+0000 7f3cbc8da700 1 --2- 172.21.15.6:0/3077082834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb80fcf80 0x7f3cb80fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.659+0000 7f3cbc8da700 1 --2- 172.21.15.6:0/3077082834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cb80fbcd0 0x7f3cb80fc0e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.659+0000 7f3cbc8da700 1 -- 172.21.15.6:0/3077082834 >> 172.21.15.6:0/3077082834 conn(0x7f3cb80f7450 msgr2=0x7f3cb80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.659+0000 7f3cbc8da700 1 -- 172.21.15.6:0/3077082834 shutdown_connections 2024-03-22T06:23:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.659+0000 7f3cbc8da700 1 -- 172.21.15.6:0/3077082834 wait complete. 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.659+0000 7f3cbc8da700 1 Processor -- start 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cbc8da700 1 -- start start 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cbc8da700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb80fbcd0 0x7f3cb8066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cbc8da700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cb80fcf80 0x7f3cb8112660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cbc8da700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb8112ba0 con 0x7f3cb80fcf80 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cbc8da700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb8112ce0 con 0x7f3cb80fbcd0 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cb5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cb80fcf80 0x7f3cb8112660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cb5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cb80fcf80 0x7f3cb8112660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49444/0 (socket says 172.21.15.6:49444) 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cb5d9b700 1 -- 172.21.15.6:0/4114876510 learned_addr learned my addr 172.21.15.6:0/4114876510 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cb659c700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb80fbcd0 0x7f3cb8066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cb5d9b700 1 -- 172.21.15.6:0/4114876510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb80fbcd0 msgr2=0x7f3cb8066d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cb5d9b700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb80fbcd0 0x7f3cb8066d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cb5d9b700 1 -- 172.21.15.6:0/4114876510 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca0005040 con 0x7f3cb80fcf80 2024-03-22T06:23:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cb659c700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb80fbcd0 0x7f3cb8066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:23:11.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3cb5d9b700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cb80fcf80 0x7f3cb8112660 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800eb40 tx=0x7f3ca800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:11.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.660+0000 7f3caeffd700 1 -- 172.21.15.6:0/4114876510 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ca8007cc0 con 0x7f3cb80fcf80 2024-03-22T06:23:11.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.661+0000 7f3caeffd700 1 -- 172.21.15.6:0/4114876510 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ca800f040 con 0x7f3cb80fcf80 2024-03-22T06:23:11.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.661+0000 7f3caeffd700 1 -- 172.21.15.6:0/4114876510 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ca8009620 con 0x7f3cb80fcf80 2024-03-22T06:23:11.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.661+0000 7f3cbc8da700 1 -- 172.21.15.6:0/4114876510 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb8112f60 con 0x7f3cb80fcf80 2024-03-22T06:23:11.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.661+0000 7f3cbc8da700 1 -- 172.21.15.6:0/4114876510 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb8113420 con 0x7f3cb80fcf80 2024-03-22T06:23:11.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.662+0000 7f3cbc8da700 1 -- 172.21.15.6:0/4114876510 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb804c710 con 0x7f3cb80fcf80 2024-03-22T06:23:11.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.662+0000 7f3caeffd700 1 -- 172.21.15.6:0/4114876510 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f3ca8009780 con 0x7f3cb80fcf80 2024-03-22T06:23:11.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.662+0000 7f3caeffd700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3ca406e840 0x7f3ca4070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:11.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.663+0000 7f3cb659c700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3ca406e840 0x7f3ca4070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:11.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.663+0000 7f3caeffd700 1 -- 172.21.15.6:0/4114876510 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f3ca808e210 con 0x7f3cb80fcf80 2024-03-22T06:23:11.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.663+0000 7f3cb659c700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3ca406e840 0x7f3ca4070cf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000b0d0 tx=0x7f3ca0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:11.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.667+0000 7f3caeffd700 1 -- 172.21.15.6:0/4114876510 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f3ca805ac10 con 0x7f3cb80fcf80 2024-03-22T06:23:11.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.894+0000 7f3cbc8da700 1 -- 172.21.15.6:0/4114876510 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cb8101220 con 0x7f3ca406e840 2024-03-22T06:23:11.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.896+0000 7f3caeffd700 1 -- 172.21.15.6:0/4114876510 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3cb8101220 con 0x7f3ca406e840 2024-03-22T06:23:11.896 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:23:11.896 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:23:11.896 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:23:11.896 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:23:11.896 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:23:11.896 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:23:11.896 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:23:11.896 INFO:teuthology.orchestra.run.smithi006.stdout: "osd", 2024-03-22T06:23:11.896 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:23:11.897 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:23:11.897 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "14/21 daemons upgraded", 2024-03-22T06:23:11.897 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading mds daemons", 2024-03-22T06:23:11.897 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:23:11.897 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:23:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.898+0000 7f3cbc8da700 1 -- 172.21.15.6:0/4114876510 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3ca406e840 msgr2=0x7f3ca4070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.898+0000 7f3cbc8da700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3ca406e840 0x7f3ca4070cf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000b0d0 tx=0x7f3ca0009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.898+0000 7f3cbc8da700 1 -- 172.21.15.6:0/4114876510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cb80fcf80 msgr2=0x7f3cb8112660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.898+0000 7f3cbc8da700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cb80fcf80 0x7f3cb8112660 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800eb40 tx=0x7f3ca800b040 comp rx=0 tx=0).stop 2024-03-22T06:23:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.898+0000 7f3cbc8da700 1 -- 172.21.15.6:0/4114876510 shutdown_connections 2024-03-22T06:23:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.898+0000 7f3cbc8da700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3ca406e840 0x7f3ca4070cf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.898+0000 7f3cbc8da700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb80fbcd0 0x7f3cb8066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.898+0000 7f3cbc8da700 1 --2- 172.21.15.6:0/4114876510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cb80fcf80 0x7f3cb8112660 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:11.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.898+0000 7f3cbc8da700 1 -- 172.21.15.6:0/4114876510 >> 172.21.15.6:0/4114876510 conn(0x7f3cb80f7450 msgr2=0x7f3cb80f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:11.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.898+0000 7f3cbc8da700 1 -- 172.21.15.6:0/4114876510 shutdown_connections 2024-03-22T06:23:11.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:11.898+0000 7f3cbc8da700 1 -- 172.21.15.6:0/4114876510 wait complete. 2024-03-22T06:23:12.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.003+0000 7eff9ecca700 1 -- 172.21.15.6:0/1476037578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff980fbcd0 msgr2=0x7eff980fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.003+0000 7eff9ecca700 1 --2- 172.21.15.6:0/1476037578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff980fbcd0 0x7eff980fc0e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff8c004940 tx=0x7eff8c009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.003+0000 7eff9ecca700 1 -- 172.21.15.6:0/1476037578 shutdown_connections 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.003+0000 7eff9ecca700 1 --2- 172.21.15.6:0/1476037578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff980fcf80 0x7eff980fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.003+0000 7eff9ecca700 1 --2- 172.21.15.6:0/1476037578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff980fbcd0 0x7eff980fc0e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.003+0000 7eff9ecca700 1 -- 172.21.15.6:0/1476037578 >> 172.21.15.6:0/1476037578 conn(0x7eff980f7450 msgr2=0x7eff980f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.003+0000 7eff9ecca700 1 -- 172.21.15.6:0/1476037578 shutdown_connections 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.004+0000 7eff9ecca700 1 -- 172.21.15.6:0/1476037578 wait complete. 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.004+0000 7eff9ecca700 1 Processor -- start 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.004+0000 7eff9ecca700 1 -- start start 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff9ecca700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff980fbcd0 0x7eff98066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff9ecca700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff980fcf80 0x7eff981126f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff9ecca700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff98112cc0 con 0x7eff980fcf80 2024-03-22T06:23:12.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff9ecca700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff98112e00 con 0x7eff980fbcd0 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff97fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff980fcf80 0x7eff981126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff97fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff980fcf80 0x7eff981126f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49454/0 (socket says 172.21.15.6:49454) 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff97fff700 1 -- 172.21.15.6:0/826798270 learned_addr learned my addr 172.21.15.6:0/826798270 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff9ca66700 1 --2- 172.21.15.6:0/826798270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff980fbcd0 0x7eff98066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff97fff700 1 -- 172.21.15.6:0/826798270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff980fbcd0 msgr2=0x7eff98066d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff97fff700 1 --2- 172.21.15.6:0/826798270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff980fbcd0 0x7eff98066d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff97fff700 1 -- 172.21.15.6:0/826798270 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c005040 con 0x7eff980fcf80 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff97fff700 1 --2- 172.21.15.6:0/826798270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff980fcf80 0x7eff981126f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff8400b790 tx=0x7eff84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff957fa700 1 -- 172.21.15.6:0/826798270 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff8400f770 con 0x7eff980fcf80 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff9ecca700 1 -- 172.21.15.6:0/826798270 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff981130e0 con 0x7eff980fcf80 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff957fa700 1 -- 172.21.15.6:0/826798270 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff8400fda0 con 0x7eff980fcf80 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff957fa700 1 -- 172.21.15.6:0/826798270 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff8400d310 con 0x7eff980fcf80 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.005+0000 7eff9ecca700 1 -- 172.21.15.6:0/826798270 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff981136a0 con 0x7eff980fcf80 2024-03-22T06:23:12.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.006+0000 7eff9ecca700 1 -- 172.21.15.6:0/826798270 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9804c710 con 0x7eff980fcf80 2024-03-22T06:23:12.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.007+0000 7eff957fa700 1 -- 172.21.15.6:0/826798270 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7eff84010420 con 0x7eff980fcf80 2024-03-22T06:23:12.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.007+0000 7eff957fa700 1 --2- 172.21.15.6:0/826798270 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff8806e840 0x7eff88070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:12.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.007+0000 7eff957fa700 1 -- 172.21.15.6:0/826798270 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6872+0+0 (secure 0 0 0) 0x7eff8408e4d0 con 0x7eff980fcf80 2024-03-22T06:23:12.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.007+0000 7eff9ca66700 1 --2- 172.21.15.6:0/826798270 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff8806e840 0x7eff88070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:12.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.008+0000 7eff9ca66700 1 --2- 172.21.15.6:0/826798270 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff8806e840 0x7eff88070cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00b3b0 tx=0x7eff8c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:12.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.012+0000 7eff957fa700 1 -- 172.21.15.6:0/826798270 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7eff8405af40 con 0x7eff980fcf80 2024-03-22T06:23:12.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:11 smithi117 ceph-mon[82126]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:12.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:11 smithi117 ceph-mon[82126]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:12.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:11 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2739472014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:12.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:11 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1203872330' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:23:12.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:11 smithi006 ceph-mon[94482]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:12.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:11 smithi006 ceph-mon[94482]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:12.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:11 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2739472014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:12.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:11 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1203872330' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:23:12.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.305+0000 7eff9ecca700 1 -- 172.21.15.6:0/826798270 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7eff98113b90 con 0x7eff980fcf80 2024-03-22T06:23:12.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.305+0000 7eff957fa700 1 -- 172.21.15.6:0/826798270 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+251 (secure 0 0 0) 0x7eff8405aad0 con 0x7eff980fcf80 2024-03-22T06:23:12.306 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem is degraded 2024-03-22T06:23:12.306 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:23:12.306 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in error state 2024-03-22T06:23:12.306 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-22T06:23:12.306 INFO:teuthology.orchestra.run.smithi006.stdout: fs cephfs is degraded 2024-03-22T06:23:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.307+0000 7eff9ecca700 1 -- 172.21.15.6:0/826798270 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff8806e840 msgr2=0x7eff88070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.307+0000 7eff9ecca700 1 --2- 172.21.15.6:0/826798270 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff8806e840 0x7eff88070cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00b3b0 tx=0x7eff8c009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.307+0000 7eff9ecca700 1 -- 172.21.15.6:0/826798270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff980fcf80 msgr2=0x7eff981126f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.307+0000 7eff9ecca700 1 --2- 172.21.15.6:0/826798270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff980fcf80 0x7eff981126f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff8400b790 tx=0x7eff84009250 comp rx=0 tx=0).stop 2024-03-22T06:23:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.307+0000 7eff9ecca700 1 -- 172.21.15.6:0/826798270 shutdown_connections 2024-03-22T06:23:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.307+0000 7eff9ecca700 1 --2- 172.21.15.6:0/826798270 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7eff8806e840 0x7eff88070cf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.307+0000 7eff9ecca700 1 --2- 172.21.15.6:0/826798270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff980fbcd0 0x7eff98066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.307+0000 7eff9ecca700 1 --2- 172.21.15.6:0/826798270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff980fcf80 0x7eff981126f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.308+0000 7eff9ecca700 1 -- 172.21.15.6:0/826798270 >> 172.21.15.6:0/826798270 conn(0x7eff980f7450 msgr2=0x7eff980f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.308+0000 7eff9ecca700 1 -- 172.21.15.6:0/826798270 shutdown_connections 2024-03-22T06:23:12.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:12.308+0000 7eff9ecca700 1 -- 172.21.15.6:0/826798270 wait complete. 2024-03-22T06:23:13.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:12 smithi117 ceph-mon[82126]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:12 smithi117 ceph-mon[82126]: pgmap v230: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-22T06:23:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:12 smithi117 ceph-mon[82126]: reconnect by client.14528 v1:172.21.15.117:0/3855819009 after 0.00099998 2024-03-22T06:23:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:12 smithi117 ceph-mon[82126]: reconnect by client.24313 v1:172.21.15.6:0/1969668997 after 0.00099998 2024-03-22T06:23:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:12 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] up:reconnect 2024-03-22T06:23:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:12 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi117.zqxyql=up:reconnect} 2 up:standby 2024-03-22T06:23:13.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:12 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/826798270' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:23:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:12 smithi006 ceph-mon[94482]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:12 smithi006 ceph-mon[94482]: pgmap v230: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-22T06:23:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:12 smithi006 ceph-mon[94482]: reconnect by client.14528 v1:172.21.15.117:0/3855819009 after 0.00099998 2024-03-22T06:23:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:12 smithi006 ceph-mon[94482]: reconnect by client.24313 v1:172.21.15.6:0/1969668997 after 0.00099998 2024-03-22T06:23:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:12 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] up:reconnect 2024-03-22T06:23:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:12 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi117.zqxyql=up:reconnect} 2 up:standby 2024-03-22T06:23:13.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:12 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/826798270' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:23:14.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:13 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] up:rejoin 2024-03-22T06:23:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:13 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi117.zqxyql=up:rejoin} 2 up:standby 2024-03-22T06:23:14.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:13 smithi117 ceph-mon[82126]: daemon mds.cephfs.smithi117.zqxyql is now active in filesystem cephfs as rank 0 2024-03-22T06:23:14.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:13 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] up:rejoin 2024-03-22T06:23:14.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:13 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi117.zqxyql=up:rejoin} 2 up:standby 2024-03-22T06:23:14.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:13 smithi006 ceph-mon[94482]: daemon mds.cephfs.smithi117.zqxyql is now active in filesystem cephfs as rank 0 2024-03-22T06:23:15.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:14 smithi117 ceph-mon[82126]: pgmap v231: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 100 B/s wr, 6 op/s 2024-03-22T06:23:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:14 smithi117 ceph-mon[82126]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T06:23:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:14 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] up:active 2024-03-22T06:23:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:14 smithi117 ceph-mon[82126]: fsmap cephfs:1 {0=cephfs.smithi117.zqxyql=up:active} 2 up:standby 2024-03-22T06:23:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:14 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:14 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:15.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:14 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:15.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:14 smithi006 ceph-mon[94482]: pgmap v231: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 100 B/s wr, 6 op/s 2024-03-22T06:23:15.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:14 smithi006 ceph-mon[94482]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T06:23:15.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:14 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.117:6826/1010515354,v1:172.21.15.117:6827/1010515354] up:active 2024-03-22T06:23:15.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:14 smithi006 ceph-mon[94482]: fsmap cephfs:1 {0=cephfs.smithi117.zqxyql=up:active} 2 up:standby 2024-03-22T06:23:15.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:14 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:15.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:14 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:15.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:14 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:16.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:15 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.117:6824/1194667779,v1:172.21.15.117:6825/1194667779] up:boot 2024-03-22T06:23:16.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:15 smithi117 ceph-mon[82126]: fsmap cephfs:1 {0=cephfs.smithi117.zqxyql=up:active} 3 up:standby 2024-03-22T06:23:16.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:15 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:23:16.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:15 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.117:6824/1194667779,v1:172.21.15.117:6825/1194667779] up:boot 2024-03-22T06:23:16.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:15 smithi006 ceph-mon[94482]: fsmap cephfs:1 {0=cephfs.smithi117.zqxyql=up:active} 3 up:standby 2024-03-22T06:23:16.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:15 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:23:17.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:16 smithi117 ceph-mon[82126]: pgmap v232: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T06:23:17.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:16 smithi006 ceph-mon[94482]: pgmap v232: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T06:23:19.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:18 smithi117 ceph-mon[82126]: pgmap v233: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-22T06:23:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:18 smithi006 ceph-mon[94482]: pgmap v233: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-22T06:23:21.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:20 smithi117 ceph-mon[82126]: pgmap v234: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-03-22T06:23:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:20 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:21.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:20 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:21.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:20 smithi006 ceph-mon[94482]: pgmap v234: 65 pgs: 65 active+clean; 194 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-03-22T06:23:21.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:20 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:21.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:20 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:22.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:22 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:22.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:22 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:22.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:22 smithi117 ceph-mon[82126]: pgmap v235: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 6.9 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-22T06:23:22.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:22 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:22.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:22 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:23:22.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:22 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:22.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:22 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:22.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:22 smithi006 ceph-mon[94482]: pgmap v235: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 6.9 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-22T06:23:22.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:22 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:22.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:22 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:23:25.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:24 smithi117 ceph-mon[82126]: pgmap v236: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-22T06:23:25.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:24 smithi006 ceph-mon[94482]: pgmap v236: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-22T06:23:26.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: Detected new or changed devices on smithi117 2024-03-22T06:23:26.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:26.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:26.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:23:26.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:23:26.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: pgmap v237: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 40 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-03-22T06:23:26.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: pgmap v238: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 47 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-03-22T06:23:26.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:26.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:26.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:26.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:26.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:26.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:26.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi117.zqxyql"]}]: dispatch 2024-03-22T06:23:26.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:25 smithi117 ceph-mon[82126]: Upgrade: It appears safe to stop mds.cephfs.smithi117.zqxyql 2024-03-22T06:23:26.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: Detected new or changed devices on smithi117 2024-03-22T06:23:26.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:26.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:26.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:23:26.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:23:26.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: pgmap v237: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 40 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-03-22T06:23:26.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: pgmap v238: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 47 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-03-22T06:23:26.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:26.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:26.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:26.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:26.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:26.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:26.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi117.zqxyql"]}]: dispatch 2024-03-22T06:23:26.283 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:25 smithi006 ceph-mon[94482]: Upgrade: It appears safe to stop mds.cephfs.smithi117.zqxyql 2024-03-22T06:23:27.897 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:27 smithi117 ceph-mon[82126]: Upgrade: Updating mds.cephfs.smithi117.zqxyql 2024-03-22T06:23:27.897 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:27 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:27.897 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:27 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.zqxyql", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:23:27.897 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:27 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:23:27.897 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:27 smithi117 ceph-mon[82126]: Deploying daemon mds.cephfs.smithi117.zqxyql on smithi117 2024-03-22T06:23:27.897 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:27 smithi117 ceph-mon[82126]: pgmap v239: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 46 KiB/s rd, 5.5 KiB/s wr, 4 op/s 2024-03-22T06:23:28.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:27 smithi006 ceph-mon[94482]: Upgrade: Updating mds.cephfs.smithi117.zqxyql 2024-03-22T06:23:28.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:27 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:28.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:27 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.zqxyql", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T06:23:28.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:27 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:23:28.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:27 smithi006 ceph-mon[94482]: Deploying daemon mds.cephfs.smithi117.zqxyql on smithi117 2024-03-22T06:23:28.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:27 smithi006 ceph-mon[94482]: pgmap v239: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 46 KiB/s rd, 5.5 KiB/s wr, 4 op/s 2024-03-22T06:23:29.006 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:23:28 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:23:28.665+0000 7f74ad90e700 -1 bdev(0x55dce8fcc000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 7 2024-03-22T06:23:29.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:28 smithi117 ceph-mon[82126]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T06:23:29.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:28 smithi117 ceph-mon[82126]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:23:29.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:28 smithi117 ceph-mon[82126]: osdmap e95: 6 total, 6 up, 6 in 2024-03-22T06:23:29.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:28 smithi117 ceph-mon[82126]: Standby daemon mds.cephfs.smithi006.bztopc assigned to filesystem cephfs as rank 0 2024-03-22T06:23:29.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:28 smithi117 ceph-mon[82126]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T06:23:29.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:28 smithi117 ceph-mon[82126]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T06:23:29.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:28 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi006.bztopc=up:replay} 2 up:standby 2024-03-22T06:23:29.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:28 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006[94477]: 2024-03-22T06:23:28.643+0000 7fe66190e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:23:29.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:28 smithi006 ceph-mon[94482]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T06:23:29.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:28 smithi006 ceph-mon[94482]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T06:23:29.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:28 smithi006 ceph-mon[94482]: osdmap e95: 6 total, 6 up, 6 in 2024-03-22T06:23:29.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:28 smithi006 ceph-mon[94482]: Standby daemon mds.cephfs.smithi006.bztopc assigned to filesystem cephfs as rank 0 2024-03-22T06:23:29.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:28 smithi006 ceph-mon[94482]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T06:23:29.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:28 smithi006 ceph-mon[94482]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T06:23:29.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:28 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi006.bztopc=up:replay} 2 up:standby 2024-03-22T06:23:30.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:29 smithi117 ceph-mon[82126]: pgmap v241: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:23:30.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:29 smithi006 ceph-mon[94482]: pgmap v241: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:23:32.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:32 smithi117 ceph-mon[82126]: pgmap v242: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-03-22T06:23:32.531 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:32 smithi006 ceph-mon[94482]: pgmap v242: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-03-22T06:23:34.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:34 smithi117 ceph-mon[82126]: pgmap v243: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 9.7 MiB/s rd, 3 op/s 2024-03-22T06:23:34.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:34 smithi006 ceph-mon[94482]: pgmap v243: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 9.7 MiB/s rd, 3 op/s 2024-03-22T06:23:36.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:36 smithi117 ceph-mon[82126]: pgmap v244: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-03-22T06:23:36.531 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:36 smithi006 ceph-mon[94482]: pgmap v244: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-03-22T06:23:38.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:38 smithi117 ceph-mon[82126]: pgmap v245: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-22T06:23:38.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:38 smithi117 ceph-mon[82126]: reconnect by client.24313 v1:172.21.15.6:0/1969668997 after 0 2024-03-22T06:23:38.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:38 smithi117 ceph-mon[82126]: reconnect by client.14528 v1:172.21.15.117:0/3855819009 after 0.00099999 2024-03-22T06:23:38.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:38 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] up:reconnect 2024-03-22T06:23:38.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:38 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi006.bztopc=up:reconnect} 2 up:standby 2024-03-22T06:23:38.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:38.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:38 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:23:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:38 smithi006 ceph-mon[94482]: pgmap v245: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-22T06:23:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:38 smithi006 ceph-mon[94482]: reconnect by client.24313 v1:172.21.15.6:0/1969668997 after 0 2024-03-22T06:23:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:38 smithi006 ceph-mon[94482]: reconnect by client.14528 v1:172.21.15.117:0/3855819009 after 0.00099999 2024-03-22T06:23:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:38 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] up:reconnect 2024-03-22T06:23:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:38 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi006.bztopc=up:reconnect} 2 up:standby 2024-03-22T06:23:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:38.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:38 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:23:39.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:39 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] up:rejoin 2024-03-22T06:23:39.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:39 smithi117 ceph-mon[82126]: fsmap cephfs:1/1 {0=cephfs.smithi006.bztopc=up:rejoin} 2 up:standby 2024-03-22T06:23:39.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:39 smithi117 ceph-mon[82126]: daemon mds.cephfs.smithi006.bztopc is now active in filesystem cephfs as rank 0 2024-03-22T06:23:39.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:39 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] up:rejoin 2024-03-22T06:23:39.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:39 smithi006 ceph-mon[94482]: fsmap cephfs:1/1 {0=cephfs.smithi006.bztopc=up:rejoin} 2 up:standby 2024-03-22T06:23:39.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:39 smithi006 ceph-mon[94482]: daemon mds.cephfs.smithi006.bztopc is now active in filesystem cephfs as rank 0 2024-03-22T06:23:40.239 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:40 smithi117 ceph-mon[82126]: pgmap v246: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 98 B/s wr, 5 op/s 2024-03-22T06:23:40.239 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:40 smithi117 ceph-mon[82126]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T06:23:40.239 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:40 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] up:active 2024-03-22T06:23:40.239 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:40 smithi117 ceph-mon[82126]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 2 up:standby 2024-03-22T06:23:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:40 smithi006 ceph-mon[94482]: pgmap v246: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 98 B/s wr, 5 op/s 2024-03-22T06:23:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:40 smithi006 ceph-mon[94482]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T06:23:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:40 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] up:active 2024-03-22T06:23:40.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:40 smithi006 ceph-mon[94482]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 2 up:standby 2024-03-22T06:23:41.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:41 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:41 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:41 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:41 smithi117 ceph-mon[82126]: mds.? [v2:172.21.15.117:6826/2788388155,v1:172.21.15.117:6827/2788388155] up:boot 2024-03-22T06:23:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:41 smithi117 ceph-mon[82126]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:23:41.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:41 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:23:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:41 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:41 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:41 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:41 smithi006 ceph-mon[94482]: mds.? [v2:172.21.15.117:6826/2788388155,v1:172.21.15.117:6827/2788388155] up:boot 2024-03-22T06:23:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:41 smithi006 ceph-mon[94482]: fsmap cephfs:1 {0=cephfs.smithi006.bztopc=up:active} 3 up:standby 2024-03-22T06:23:41.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:41 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:23:42.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2568967578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3e80662e0 msgr2=0x7ff3e80ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:42.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 --2- 172.21.15.6:0/2568967578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3e80662e0 0x7ff3e80ff9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc004940 tx=0x7ff3dc009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:42.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2568967578 shutdown_connections 2024-03-22T06:23:42.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 --2- 172.21.15.6:0/2568967578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3e80fff00 0x7ff3e8102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:42.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 --2- 172.21.15.6:0/2568967578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3e80662e0 0x7ff3e80ff9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:42.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2568967578 >> 172.21.15.6:0/2568967578 conn(0x7ff3e80f7430 msgr2=0x7ff3e80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:42.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2568967578 shutdown_connections 2024-03-22T06:23:42.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2568967578 wait complete. 2024-03-22T06:23:42.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 Processor -- start 2024-03-22T06:23:42.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 -- start start 2024-03-22T06:23:42.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3e80662e0 0x7ff3e810fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3e80fff00 0x7ff3e81100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e81106f0 con 0x7ff3e80662e0 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.414+0000 7ff3eefe0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e8110f40 con 0x7ff3e80fff00 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3ecd7c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3e80662e0 0x7ff3e810fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3ecd7c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3e80662e0 0x7ff3e810fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41064/0 (socket says 172.21.15.6:41064) 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3ecd7c700 1 -- 172.21.15.6:0/2039362615 learned_addr learned my addr 172.21.15.6:0/2039362615 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3e7fff700 1 --2- 172.21.15.6:0/2039362615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3e80fff00 0x7ff3e81100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3ecd7c700 1 -- 172.21.15.6:0/2039362615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3e80fff00 msgr2=0x7ff3e81100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3ecd7c700 1 --2- 172.21.15.6:0/2039362615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3e80fff00 0x7ff3e81100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3ecd7c700 1 -- 172.21.15.6:0/2039362615 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc005040 con 0x7ff3e80662e0 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3ecd7c700 1 --2- 172.21.15.6:0/2039362615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3e80662e0 0x7ff3e810fbb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc000c00 tx=0x7ff3dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3e57fa700 1 -- 172.21.15.6:0/2039362615 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3dc00e040 con 0x7ff3e80662e0 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3e57fa700 1 -- 172.21.15.6:0/2039362615 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3dc00bbf0 con 0x7ff3e80662e0 2024-03-22T06:23:42.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2039362615 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e81111c0 con 0x7ff3e80662e0 2024-03-22T06:23:42.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3e57fa700 1 -- 172.21.15.6:0/2039362615 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3dc01c630 con 0x7ff3e80662e0 2024-03-22T06:23:42.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.415+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2039362615 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e8111700 con 0x7ff3e80662e0 2024-03-22T06:23:42.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.416+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2039362615 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e805f5f0 con 0x7ff3e80662e0 2024-03-22T06:23:42.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.417+0000 7ff3e57fa700 1 -- 172.21.15.6:0/2039362615 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7ff3dc00bd60 con 0x7ff3e80662e0 2024-03-22T06:23:42.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.417+0000 7ff3e57fa700 1 --2- 172.21.15.6:0/2039362615 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff3d806e840 0x7ff3d8070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:42.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.418+0000 7ff3e57fa700 1 -- 172.21.15.6:0/2039362615 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7ff3dc08f7a0 con 0x7ff3e80662e0 2024-03-22T06:23:42.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.418+0000 7ff3e7fff700 1 --2- 172.21.15.6:0/2039362615 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff3d806e840 0x7ff3d8070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:42.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.418+0000 7ff3e7fff700 1 --2- 172.21.15.6:0/2039362615 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff3d806e840 0x7ff3d8070cf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009fd0 tx=0x7ff3d400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:42.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.422+0000 7ff3e57fa700 1 -- 172.21.15.6:0/2039362615 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7ff3dc05c1c0 con 0x7ff3e80662e0 2024-03-22T06:23:42.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.746+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2039362615 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3e8111af0 con 0x7ff3d806e840 2024-03-22T06:23:42.747 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:42 smithi006 ceph-mon[94482]: pgmap v247: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T06:23:42.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.747+0000 7ff3e57fa700 1 -- 172.21.15.6:0/2039362615 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff3e8111af0 con 0x7ff3d806e840 2024-03-22T06:23:42.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.749+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2039362615 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff3d806e840 msgr2=0x7ff3d8070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:42.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.749+0000 7ff3eefe0700 1 --2- 172.21.15.6:0/2039362615 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff3d806e840 0x7ff3d8070cf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009fd0 tx=0x7ff3d400b040 comp rx=0 tx=0).stop 2024-03-22T06:23:42.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.749+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2039362615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3e80662e0 msgr2=0x7ff3e810fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:42.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.749+0000 7ff3eefe0700 1 --2- 172.21.15.6:0/2039362615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3e80662e0 0x7ff3e810fbb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc000c00 tx=0x7ff3dc009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:42.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.750+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2039362615 shutdown_connections 2024-03-22T06:23:42.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.750+0000 7ff3eefe0700 1 --2- 172.21.15.6:0/2039362615 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff3d806e840 0x7ff3d8070cf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:42.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.750+0000 7ff3eefe0700 1 --2- 172.21.15.6:0/2039362615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3e80662e0 0x7ff3e810fbb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:42.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.750+0000 7ff3eefe0700 1 --2- 172.21.15.6:0/2039362615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3e80fff00 0x7ff3e81100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:42.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.750+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2039362615 >> 172.21.15.6:0/2039362615 conn(0x7ff3e80f7430 msgr2=0x7ff3e80fb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:42.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.750+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2039362615 shutdown_connections 2024-03-22T06:23:42.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.750+0000 7ff3eefe0700 1 -- 172.21.15.6:0/2039362615 wait complete. 2024-03-22T06:23:42.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:42 smithi117 ceph-mon[82126]: pgmap v247: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T06:23:42.763 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:23:42.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.856+0000 7faaf7742700 1 -- 172.21.15.6:0/1120559257 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faaf00fbe00 msgr2=0x7faaf00fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:42.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.856+0000 7faaf7742700 1 --2- 172.21.15.6:0/1120559257 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faaf00fbe00 0x7faaf00fc210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faaec004940 tx=0x7faaec009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:42.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.856+0000 7faaf7742700 1 -- 172.21.15.6:0/1120559257 shutdown_connections 2024-03-22T06:23:42.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.856+0000 7faaf7742700 1 --2- 172.21.15.6:0/1120559257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf00fcf70 0x7faaf00fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.856+0000 7faaf7742700 1 --2- 172.21.15.6:0/1120559257 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faaf00fbe00 0x7faaf00fc210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.856+0000 7faaf7742700 1 -- 172.21.15.6:0/1120559257 >> 172.21.15.6:0/1120559257 conn(0x7faaf00f74e0 msgr2=0x7faaf00f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.856+0000 7faaf7742700 1 -- 172.21.15.6:0/1120559257 shutdown_connections 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.856+0000 7faaf7742700 1 -- 172.21.15.6:0/1120559257 wait complete. 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.857+0000 7faaf7742700 1 Processor -- start 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.857+0000 7faaf7742700 1 -- start start 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.857+0000 7faaf7742700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf00fbe00 0x7faaf010c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.857+0000 7faaf7742700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faaf00fcf70 0x7faaf010ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.857+0000 7faaf7742700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf010d400 con 0x7faaf00fcf70 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.857+0000 7faaf7742700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf010dc50 con 0x7faaf00fbe00 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.857+0000 7faaf4cdd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faaf00fcf70 0x7faaf010ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.857+0000 7faaf4cdd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faaf00fcf70 0x7faaf010ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41096/0 (socket says 172.21.15.6:41096) 2024-03-22T06:23:42.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.857+0000 7faaf4cdd700 1 -- 172.21.15.6:0/1912291477 learned_addr learned my addr 172.21.15.6:0/1912291477 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:42.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.857+0000 7faaf54de700 1 --2- 172.21.15.6:0/1912291477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf00fbe00 0x7faaf010c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:42.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.858+0000 7faaf4cdd700 1 -- 172.21.15.6:0/1912291477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf00fbe00 msgr2=0x7faaf010c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:42.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.858+0000 7faaf4cdd700 1 --2- 172.21.15.6:0/1912291477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf00fbe00 0x7faaf010c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:42.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.858+0000 7faaf4cdd700 1 -- 172.21.15.6:0/1912291477 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec005040 con 0x7faaf00fcf70 2024-03-22T06:23:42.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.858+0000 7faaf4cdd700 1 --2- 172.21.15.6:0/1912291477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faaf00fcf70 0x7faaf010ce00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faae000eb40 tx=0x7faae000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:42.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.858+0000 7faae5ffb700 1 -- 172.21.15.6:0/1912291477 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae0007cc0 con 0x7faaf00fcf70 2024-03-22T06:23:42.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.858+0000 7faae5ffb700 1 -- 172.21.15.6:0/1912291477 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faae000f040 con 0x7faaf00fcf70 2024-03-22T06:23:42.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.858+0000 7faaf7742700 1 -- 172.21.15.6:0/1912291477 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf010ded0 con 0x7faaf00fcf70 2024-03-22T06:23:42.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.858+0000 7faae5ffb700 1 -- 172.21.15.6:0/1912291477 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae000ca10 con 0x7faaf00fcf70 2024-03-22T06:23:42.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.858+0000 7faaf7742700 1 -- 172.21.15.6:0/1912291477 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf010e410 con 0x7faaf00fcf70 2024-03-22T06:23:42.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.859+0000 7faaf7742700 1 -- 172.21.15.6:0/1912291477 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf005f5f0 con 0x7faaf00fcf70 2024-03-22T06:23:42.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.860+0000 7faae5ffb700 1 -- 172.21.15.6:0/1912291477 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7faae000cc80 con 0x7faaf00fcf70 2024-03-22T06:23:42.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.860+0000 7faae5ffb700 1 --2- 172.21.15.6:0/1912291477 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faadc06e840 0x7faadc070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:42.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.860+0000 7faae5ffb700 1 -- 172.21.15.6:0/1912291477 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7faae008e180 con 0x7faaf00fcf70 2024-03-22T06:23:42.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.860+0000 7faaf54de700 1 --2- 172.21.15.6:0/1912291477 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faadc06e840 0x7faadc070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:42.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.861+0000 7faaf54de700 1 --2- 172.21.15.6:0/1912291477 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faadc06e840 0x7faadc070cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faaec009b30 tx=0x7faaec009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:42.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:42.869+0000 7faae5ffb700 1 -- 172.21.15.6:0/1912291477 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7faae005ab30 con 0x7faaf00fcf70 2024-03-22T06:23:43.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.114+0000 7faaf7742700 1 -- 172.21.15.6:0/1912291477 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faaf010e630 con 0x7faadc06e840 2024-03-22T06:23:43.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.115+0000 7faae5ffb700 1 -- 172.21.15.6:0/1912291477 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faaf010e630 con 0x7faadc06e840 2024-03-22T06:23:43.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.118+0000 7faaf7742700 1 -- 172.21.15.6:0/1912291477 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faadc06e840 msgr2=0x7faadc070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:43.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.118+0000 7faaf7742700 1 --2- 172.21.15.6:0/1912291477 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faadc06e840 0x7faadc070cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faaec009b30 tx=0x7faaec009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:43.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.118+0000 7faaf7742700 1 -- 172.21.15.6:0/1912291477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faaf00fcf70 msgr2=0x7faaf010ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:43.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.118+0000 7faaf7742700 1 --2- 172.21.15.6:0/1912291477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faaf00fcf70 0x7faaf010ce00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faae000eb40 tx=0x7faae000b040 comp rx=0 tx=0).stop 2024-03-22T06:23:43.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.118+0000 7faaf7742700 1 -- 172.21.15.6:0/1912291477 shutdown_connections 2024-03-22T06:23:43.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.118+0000 7faaf7742700 1 --2- 172.21.15.6:0/1912291477 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faadc06e840 0x7faadc070cf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.118+0000 7faaf7742700 1 --2- 172.21.15.6:0/1912291477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf00fbe00 0x7faaf010c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.118+0000 7faaf7742700 1 --2- 172.21.15.6:0/1912291477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faaf00fcf70 0x7faaf010ce00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.118+0000 7faaf7742700 1 -- 172.21.15.6:0/1912291477 >> 172.21.15.6:0/1912291477 conn(0x7faaf00f74e0 msgr2=0x7faaf00f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.118+0000 7faaf7742700 1 -- 172.21.15.6:0/1912291477 shutdown_connections 2024-03-22T06:23:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.118+0000 7faaf7742700 1 -- 172.21.15.6:0/1912291477 wait complete. 2024-03-22T06:23:43.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.224+0000 7f0912f4a700 1 -- 172.21.15.6:0/1753169465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0fbcd0 msgr2=0x7f090c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:43.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.224+0000 7f0912f4a700 1 --2- 172.21.15.6:0/1753169465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0fbcd0 0x7f090c0fc0e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0908004940 tx=0x7f0908009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:43.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.224+0000 7f0912f4a700 1 -- 172.21.15.6:0/1753169465 shutdown_connections 2024-03-22T06:23:43.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.224+0000 7f0912f4a700 1 --2- 172.21.15.6:0/1753169465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090c0fcf80 0x7f090c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.224+0000 7f0912f4a700 1 --2- 172.21.15.6:0/1753169465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0fbcd0 0x7f090c0fc0e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.224+0000 7f0912f4a700 1 -- 172.21.15.6:0/1753169465 >> 172.21.15.6:0/1753169465 conn(0x7f090c0f7430 msgr2=0x7f090c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:43.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.224+0000 7f0912f4a700 1 -- 172.21.15.6:0/1753169465 shutdown_connections 2024-03-22T06:23:43.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.224+0000 7f0912f4a700 1 -- 172.21.15.6:0/1753169465 wait complete. 2024-03-22T06:23:43.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.224+0000 7f0912f4a700 1 Processor -- start 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0912f4a700 1 -- start start 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0912f4a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090c0fbcd0 0x7f090c110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0912f4a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0fcf80 0x7f090c112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0912f4a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c066350 con 0x7f090c0fcf80 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0912f4a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c066490 con 0x7f090c0fbcd0 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0903fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0fcf80 0x7f090c112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0903fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0fcf80 0x7f090c112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41104/0 (socket says 172.21.15.6:41104) 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0903fff700 1 -- 172.21.15.6:0/2570544511 learned_addr learned my addr 172.21.15.6:0/2570544511 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0910ce6700 1 --2- 172.21.15.6:0/2570544511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090c0fbcd0 0x7f090c110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0903fff700 1 -- 172.21.15.6:0/2570544511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090c0fbcd0 msgr2=0x7f090c110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0903fff700 1 --2- 172.21.15.6:0/2570544511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090c0fbcd0 0x7f090c110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.225+0000 7f0903fff700 1 -- 172.21.15.6:0/2570544511 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0908005040 con 0x7f090c0fcf80 2024-03-22T06:23:43.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.226+0000 7f0903fff700 1 --2- 172.21.15.6:0/2570544511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0fcf80 0x7f090c112c70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f08fc002a50 tx=0x7f08fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:43.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.226+0000 7f09017fa700 1 -- 172.21.15.6:0/2570544511 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08fc007cc0 con 0x7f090c0fcf80 2024-03-22T06:23:43.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.226+0000 7f09017fa700 1 -- 172.21.15.6:0/2570544511 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08fc00f040 con 0x7f090c0fcf80 2024-03-22T06:23:43.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.226+0000 7f0912f4a700 1 -- 172.21.15.6:0/2570544511 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c113210 con 0x7f090c0fcf80 2024-03-22T06:23:43.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.226+0000 7f0912f4a700 1 -- 172.21.15.6:0/2570544511 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c1137d0 con 0x7f090c0fcf80 2024-03-22T06:23:43.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.226+0000 7f09017fa700 1 -- 172.21.15.6:0/2570544511 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08fc00da30 con 0x7f090c0fcf80 2024-03-22T06:23:43.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.227+0000 7f0912f4a700 1 -- 172.21.15.6:0/2570544511 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090c05f5f0 con 0x7f090c0fcf80 2024-03-22T06:23:43.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.227+0000 7f09017fa700 1 -- 172.21.15.6:0/2570544511 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f08fc00db90 con 0x7f090c0fcf80 2024-03-22T06:23:43.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.228+0000 7f09017fa700 1 --2- 172.21.15.6:0/2570544511 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f08f406e850 0x7f08f4070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:43.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.228+0000 7f09017fa700 1 -- 172.21.15.6:0/2570544511 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f08fc08e330 con 0x7f090c0fcf80 2024-03-22T06:23:43.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.228+0000 7f0910ce6700 1 --2- 172.21.15.6:0/2570544511 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f08f406e850 0x7f08f4070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:43.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.228+0000 7f0910ce6700 1 --2- 172.21.15.6:0/2570544511 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f08f406e850 0x7f08f4070d00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f090800b3b0 tx=0x7f0908009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:43.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.232+0000 7f09017fa700 1 -- 172.21.15.6:0/2570544511 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f08fc057100 con 0x7f090c0fcf80 2024-03-22T06:23:43.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.455+0000 7f0912f4a700 1 -- 172.21.15.6:0/2570544511 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f090c101290 con 0x7f08f406e850 2024-03-22T06:23:43.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.466+0000 7f09017fa700 1 -- 172.21.15.6:0/2570544511 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f090c101290 con 0x7f08f406e850 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (8m) 48s ago 16m 26.5M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (5m) 48s ago 17m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (5m) 0s ago 16m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 error 48s ago 16m - - 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (80s) 48s ago 11m 14.0M - 19.0.0-2299-gea01217f aa461e7c5e7f 3e3319f646e6 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (53s) 48s ago 11m 11.3M - 19.0.0-2299-gea01217f aa461e7c5e7f 09ee1e271adb 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (29s) 0s ago 11m 14.9M - 19.0.0-2299-gea01217f aa461e7c5e7f a9149dea3f8e 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (3s) 0s ago 11m 11.3M - 19.0.0-2299-gea01217f aa461e7c5e7f 0abdc3f226cc 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (7m) 48s ago 18m 509M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (6m) 0s ago 15m 452M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (6m) 48s ago 18m 50.9M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:23:43.467 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (5m) 0s ago 15m 40.4M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:23:43.468 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (8m) 48s ago 16m 17.5M - 1.5.0 0da6a335fe13 ac5907c81a0d 2024-03-22T06:23:43.468 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (7m) 0s ago 15m 15.1M - 1.5.0 0da6a335fe13 ca60b5ff067c 2024-03-22T06:23:43.468 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (4m) 48s ago 15m 154M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:23:43.468 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (4m) 48s ago 14m 135M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:23:43.468 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (3m) 48s ago 14m 102M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 3ec1f219c19c 2024-03-22T06:23:43.468 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (2m) 0s ago 13m 114M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 9cba39b6f32c 2024-03-22T06:23:43.468 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (2m) 0s ago 13m 104M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1950fc21bd2e 2024-03-22T06:23:43.468 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (103s) 0s ago 12m 108M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1a6a0ee2a228 2024-03-22T06:23:43.468 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (6m) 48s ago 16m 66.5M - 2.43.0 a07b618ecd1d 2634e4ac5791 2024-03-22T06:23:43.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.468+0000 7f0912f4a700 1 -- 172.21.15.6:0/2570544511 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f08f406e850 msgr2=0x7f08f4070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:43.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.468+0000 7f0912f4a700 1 --2- 172.21.15.6:0/2570544511 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f08f406e850 0x7f08f4070d00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f090800b3b0 tx=0x7f0908009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:43.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.468+0000 7f0912f4a700 1 -- 172.21.15.6:0/2570544511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0fcf80 msgr2=0x7f090c112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:43.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.468+0000 7f0912f4a700 1 --2- 172.21.15.6:0/2570544511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0fcf80 0x7f090c112c70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f08fc002a50 tx=0x7f08fc0046b0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.468+0000 7f0912f4a700 1 -- 172.21.15.6:0/2570544511 shutdown_connections 2024-03-22T06:23:43.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.468+0000 7f0912f4a700 1 --2- 172.21.15.6:0/2570544511 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f08f406e850 0x7f08f4070d00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.468+0000 7f0912f4a700 1 --2- 172.21.15.6:0/2570544511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090c0fbcd0 0x7f090c110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.468+0000 7f0912f4a700 1 --2- 172.21.15.6:0/2570544511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0fcf80 0x7f090c112c70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.468+0000 7f0912f4a700 1 -- 172.21.15.6:0/2570544511 >> 172.21.15.6:0/2570544511 conn(0x7f090c0f7430 msgr2=0x7f090c0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:43.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.468+0000 7f0912f4a700 1 -- 172.21.15.6:0/2570544511 shutdown_connections 2024-03-22T06:23:43.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.468+0000 7f0912f4a700 1 -- 172.21.15.6:0/2570544511 wait complete. 2024-03-22T06:23:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.583+0000 7f176f07a700 1 -- 172.21.15.6:0/3342584422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17680fce80 msgr2=0x7f17680fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.583+0000 7f176f07a700 1 --2- 172.21.15.6:0/3342584422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17680fce80 0x7f17680fd290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f175c004940 tx=0x7f175c009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.583+0000 7f176f07a700 1 -- 172.21.15.6:0/3342584422 shutdown_connections 2024-03-22T06:23:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.583+0000 7f176f07a700 1 --2- 172.21.15.6:0/3342584422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17680fdff0 0x7f17680fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.583+0000 7f176f07a700 1 --2- 172.21.15.6:0/3342584422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17680fce80 0x7f17680fd290 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.583+0000 7f176f07a700 1 -- 172.21.15.6:0/3342584422 >> 172.21.15.6:0/3342584422 conn(0x7f17680f8580 msgr2=0x7f17680fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.583+0000 7f176f07a700 1 -- 172.21.15.6:0/3342584422 shutdown_connections 2024-03-22T06:23:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.583+0000 7f176f07a700 1 -- 172.21.15.6:0/3342584422 wait complete. 2024-03-22T06:23:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.583+0000 7f176f07a700 1 Processor -- start 2024-03-22T06:23:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.583+0000 7f176f07a700 1 -- start start 2024-03-22T06:23:43.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f176f07a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17680fce80 0x7f176810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:43.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f176f07a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17680fdff0 0x7f176810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f176f07a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176810e550 con 0x7f17680fce80 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f176f07a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176810ed50 con 0x7f17680fdff0 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f176ce16700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17680fce80 0x7f176810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f176ce16700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17680fce80 0x7f176810da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41118/0 (socket says 172.21.15.6:41118) 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f176ce16700 1 -- 172.21.15.6:0/4291144905 learned_addr learned my addr 172.21.15.6:0/4291144905 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f1767fff700 1 --2- 172.21.15.6:0/4291144905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17680fdff0 0x7f176810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f176ce16700 1 -- 172.21.15.6:0/4291144905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17680fdff0 msgr2=0x7f176810df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f176ce16700 1 --2- 172.21.15.6:0/4291144905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17680fdff0 0x7f176810df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f176ce16700 1 -- 172.21.15.6:0/4291144905 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c005040 con 0x7f17680fce80 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f176ce16700 1 --2- 172.21.15.6:0/4291144905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17680fce80 0x7f176810da10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f175c013870 tx=0x7f175c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.584+0000 7f17657fa700 1 -- 172.21.15.6:0/4291144905 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f175c00e070 con 0x7f17680fce80 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.585+0000 7f17657fa700 1 -- 172.21.15.6:0/4291144905 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f175c013de0 con 0x7f17680fce80 2024-03-22T06:23:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.585+0000 7f176f07a700 1 -- 172.21.15.6:0/4291144905 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176810efd0 con 0x7f17680fce80 2024-03-22T06:23:43.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.585+0000 7f17657fa700 1 -- 172.21.15.6:0/4291144905 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f175c01cea0 con 0x7f17680fce80 2024-03-22T06:23:43.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.585+0000 7f176f07a700 1 -- 172.21.15.6:0/4291144905 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176810f4b0 con 0x7f17680fce80 2024-03-22T06:23:43.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.585+0000 7f176f07a700 1 -- 172.21.15.6:0/4291144905 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176805f5f0 con 0x7f17680fce80 2024-03-22T06:23:43.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.586+0000 7f17657fa700 1 -- 172.21.15.6:0/4291144905 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f175c01c690 con 0x7f17680fce80 2024-03-22T06:23:43.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.586+0000 7f17657fa700 1 --2- 172.21.15.6:0/4291144905 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1758072c60 0x7f1758075110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:43.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.587+0000 7f17657fa700 1 -- 172.21.15.6:0/4291144905 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f175c08fce0 con 0x7f17680fce80 2024-03-22T06:23:43.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.587+0000 7f1767fff700 1 --2- 172.21.15.6:0/4291144905 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1758072c60 0x7f1758075110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:43.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.587+0000 7f1767fff700 1 --2- 172.21.15.6:0/4291144905 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1758072c60 0x7f1758075110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1754009b80 tx=0x7f175400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:43.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.591+0000 7f17657fa700 1 -- 172.21.15.6:0/4291144905 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f175c05c660 con 0x7f17680fce80 2024-03-22T06:23:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:43 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:43 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:43 smithi006 ceph-mon[94482]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:43 smithi006 ceph-mon[94482]: pgmap v248: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 9.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-22T06:23:43.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:43 smithi006 ceph-mon[94482]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:43.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.898+0000 7f176f07a700 1 -- 172.21.15.6:0/4291144905 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f176810f6d0 con 0x7f17680fce80 2024-03-22T06:23:43.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.898+0000 7f17657fa700 1 -- 172.21.15.6:0/4291144905 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f175c004490 con 0x7f17680fce80 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 6 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:23:43.899 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 4 2024-03-22T06:23:43.900 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:23:43.900 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:23:43.900 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 14 2024-03-22T06:23:43.900 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:23:43.900 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:23:43.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.901+0000 7f176f07a700 1 -- 172.21.15.6:0/4291144905 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1758072c60 msgr2=0x7f1758075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.901+0000 7f176f07a700 1 --2- 172.21.15.6:0/4291144905 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1758072c60 0x7f1758075110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1754009b80 tx=0x7f175400b040 comp rx=0 tx=0).stop 2024-03-22T06:23:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.901+0000 7f176f07a700 1 -- 172.21.15.6:0/4291144905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17680fce80 msgr2=0x7f176810da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.901+0000 7f176f07a700 1 --2- 172.21.15.6:0/4291144905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17680fce80 0x7f176810da10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f175c013870 tx=0x7f175c009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.901+0000 7f176f07a700 1 -- 172.21.15.6:0/4291144905 shutdown_connections 2024-03-22T06:23:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.901+0000 7f176f07a700 1 --2- 172.21.15.6:0/4291144905 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f1758072c60 0x7f1758075110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.901+0000 7f176f07a700 1 --2- 172.21.15.6:0/4291144905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17680fce80 0x7f176810da10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.901+0000 7f176f07a700 1 --2- 172.21.15.6:0/4291144905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17680fdff0 0x7f176810df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.901+0000 7f176f07a700 1 -- 172.21.15.6:0/4291144905 >> 172.21.15.6:0/4291144905 conn(0x7f17680f8580 msgr2=0x7f17680f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.901+0000 7f176f07a700 1 -- 172.21.15.6:0/4291144905 shutdown_connections 2024-03-22T06:23:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:43.901+0000 7f176f07a700 1 -- 172.21.15.6:0/4291144905 wait complete. 2024-03-22T06:23:44.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:43 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:44.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:43 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:44.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:43 smithi117 ceph-mon[82126]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:44.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:43 smithi117 ceph-mon[82126]: pgmap v248: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 9.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-22T06:23:44.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:43 smithi117 ceph-mon[82126]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:44.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.007+0000 7f7389cc8700 1 -- 172.21.15.6:0/4285276051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840fce80 msgr2=0x7f73840fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:44.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.007+0000 7f7389cc8700 1 --2- 172.21.15.6:0/4285276051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840fce80 0x7f73840fd290 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7374004940 tx=0x7f7374009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:44.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.007+0000 7f7389cc8700 1 -- 172.21.15.6:0/4285276051 shutdown_connections 2024-03-22T06:23:44.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.007+0000 7f7389cc8700 1 --2- 172.21.15.6:0/4285276051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73840fdff0 0x7f73840fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.007+0000 7f7389cc8700 1 --2- 172.21.15.6:0/4285276051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840fce80 0x7f73840fd290 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.007+0000 7f7389cc8700 1 -- 172.21.15.6:0/4285276051 >> 172.21.15.6:0/4285276051 conn(0x7f73840f8580 msgr2=0x7f73840fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:44.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.007+0000 7f7389cc8700 1 -- 172.21.15.6:0/4285276051 shutdown_connections 2024-03-22T06:23:44.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.007+0000 7f7389cc8700 1 -- 172.21.15.6:0/4285276051 wait complete. 2024-03-22T06:23:44.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f7389cc8700 1 Processor -- start 2024-03-22T06:23:44.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f7389cc8700 1 -- start start 2024-03-22T06:23:44.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f7389cc8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840fce80 0x7f738410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:44.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f7389cc8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73840fdff0 0x7f738410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:44.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f7389cc8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738410e590 con 0x7f73840fce80 2024-03-22T06:23:44.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f7389cc8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738410ede0 con 0x7f73840fdff0 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f73837fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840fce80 0x7f738410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f73837fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840fce80 0x7f738410da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44634/0 (socket says 172.21.15.6:44634) 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f73837fe700 1 -- 172.21.15.6:0/4029613078 learned_addr learned my addr 172.21.15.6:0/4029613078 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f7382ffd700 1 --2- 172.21.15.6:0/4029613078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73840fdff0 0x7f738410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f73837fe700 1 -- 172.21.15.6:0/4029613078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73840fdff0 msgr2=0x7f738410df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f73837fe700 1 --2- 172.21.15.6:0/4029613078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73840fdff0 0x7f738410df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.008+0000 7f73837fe700 1 -- 172.21.15.6:0/4029613078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7374005040 con 0x7f73840fce80 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.009+0000 7f73837fe700 1 --2- 172.21.15.6:0/4029613078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840fce80 0x7f738410da50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7374009150 tx=0x7f7374009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.009+0000 7f736bfff700 1 -- 172.21.15.6:0/4029613078 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f737400e040 con 0x7f73840fce80 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.009+0000 7f736bfff700 1 -- 172.21.15.6:0/4029613078 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7374004290 con 0x7f73840fce80 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.009+0000 7f7389cc8700 1 -- 172.21.15.6:0/4029613078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738410f060 con 0x7f73840fce80 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.009+0000 7f736bfff700 1 -- 172.21.15.6:0/4029613078 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f737401c630 con 0x7f73840fce80 2024-03-22T06:23:44.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.009+0000 7f7389cc8700 1 -- 172.21.15.6:0/4029613078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738410f5a0 con 0x7f73840fce80 2024-03-22T06:23:44.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.010+0000 7f7389cc8700 1 -- 172.21.15.6:0/4029613078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738405f5f0 con 0x7f73840fce80 2024-03-22T06:23:44.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.010+0000 7f736bfff700 1 -- 172.21.15.6:0/4029613078 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f737400bbf0 con 0x7f73840fce80 2024-03-22T06:23:44.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.011+0000 7f736bfff700 1 --2- 172.21.15.6:0/4029613078 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f736c06e840 0x7f736c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:44.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.011+0000 7f736bfff700 1 -- 172.21.15.6:0/4029613078 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f737408f150 con 0x7f73840fce80 2024-03-22T06:23:44.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.011+0000 7f7382ffd700 1 --2- 172.21.15.6:0/4029613078 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f736c06e840 0x7f736c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:44.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.011+0000 7f7382ffd700 1 --2- 172.21.15.6:0/4029613078 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f736c06e840 0x7f736c070cf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7378004f40 tx=0x7f737800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:44.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.016+0000 7f736bfff700 1 -- 172.21.15.6:0/4029613078 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f737405bad0 con 0x7f73840fce80 2024-03-22T06:23:44.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.268+0000 7f7389cc8700 1 -- 172.21.15.6:0/4029613078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f738410f920 con 0x7f73840fce80 2024-03-22T06:23:44.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.269+0000 7f736bfff700 1 -- 172.21.15.6:0/4029613078 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1908 (secure 0 0 0) 0x7f737405b660 con 0x7f73840fce80 2024-03-22T06:23:44.269 INFO:teuthology.orchestra.run.smithi006.stdout:e36 2024-03-22T06:23:44.269 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:23:44.269 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:23:44.269 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 35 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:23:39.065820+0000 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:23:44.270 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 95 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=34406} 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:23:44.271 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:23:44.272 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:23:44.272 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:23:44.272 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 34406 members: 34406 2024-03-22T06:23:44.272 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:34406} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:23:44.272 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:23:44.272 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:23:44.272 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:23:44.272 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:23:44.272 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:34438} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.6:6828/1244315045,v1:172.21.15.6:6829/1244315045] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:23:44.272 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6824/1194667779,v1:172.21.15.117:6825/1194667779] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:23:44.272 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2788388155,v1:172.21.15.117:6827/2788388155] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:23:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.271+0000 7f7389cc8700 1 -- 172.21.15.6:0/4029613078 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f736c06e840 msgr2=0x7f736c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.271+0000 7f7389cc8700 1 --2- 172.21.15.6:0/4029613078 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f736c06e840 0x7f736c070cf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7378004f40 tx=0x7f737800b040 comp rx=0 tx=0).stop 2024-03-22T06:23:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.271+0000 7f7389cc8700 1 -- 172.21.15.6:0/4029613078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840fce80 msgr2=0x7f738410da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.271+0000 7f7389cc8700 1 --2- 172.21.15.6:0/4029613078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840fce80 0x7f738410da50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7374009150 tx=0x7f7374009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.271+0000 7f7389cc8700 1 -- 172.21.15.6:0/4029613078 shutdown_connections 2024-03-22T06:23:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.271+0000 7f7389cc8700 1 --2- 172.21.15.6:0/4029613078 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f736c06e840 0x7f736c070cf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.271+0000 7f7389cc8700 1 --2- 172.21.15.6:0/4029613078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840fce80 0x7f738410da50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.271+0000 7f7389cc8700 1 --2- 172.21.15.6:0/4029613078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73840fdff0 0x7f738410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.271+0000 7f7389cc8700 1 -- 172.21.15.6:0/4029613078 >> 172.21.15.6:0/4029613078 conn(0x7f73840f8580 msgr2=0x7f73840f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.271+0000 7f7389cc8700 1 -- 172.21.15.6:0/4029613078 shutdown_connections 2024-03-22T06:23:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.271+0000 7f7389cc8700 1 -- 172.21.15.6:0/4029613078 wait complete. 2024-03-22T06:23:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 36 2024-03-22T06:23:44.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 -- 172.21.15.6:0/2636210153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88040662e0 msgr2=0x7f88040ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:44.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 --2- 172.21.15.6:0/2636210153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88040662e0 0x7f88040ff9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8800004940 tx=0x7f8800009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:44.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 -- 172.21.15.6:0/2636210153 shutdown_connections 2024-03-22T06:23:44.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 --2- 172.21.15.6:0/2636210153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88040fff00 0x7f8804102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 --2- 172.21.15.6:0/2636210153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88040662e0 0x7f88040ff9c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 -- 172.21.15.6:0/2636210153 >> 172.21.15.6:0/2636210153 conn(0x7f88040f7430 msgr2=0x7f88040f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:44.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 -- 172.21.15.6:0/2636210153 shutdown_connections 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 -- 172.21.15.6:0/2636210153 wait complete. 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 Processor -- start 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 -- start start 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88040662e0 0x7f880410da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88040fff00 0x7f880410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880410e570 con 0x7f88040662e0 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.376+0000 7f880c650700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880410edc0 con 0x7f88040fff00 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f880a3ec700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88040662e0 0x7f880410da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f880a3ec700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88040662e0 0x7f880410da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44658/0 (socket says 172.21.15.6:44658) 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f880a3ec700 1 -- 172.21.15.6:0/1641894859 learned_addr learned my addr 172.21.15.6:0/1641894859 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f8809beb700 1 --2- 172.21.15.6:0/1641894859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88040fff00 0x7f880410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f880a3ec700 1 -- 172.21.15.6:0/1641894859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88040fff00 msgr2=0x7f880410df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:44.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f880a3ec700 1 --2- 172.21.15.6:0/1641894859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88040fff00 0x7f880410df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f880a3ec700 1 -- 172.21.15.6:0/1641894859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8800005040 con 0x7f88040662e0 2024-03-22T06:23:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f880a3ec700 1 --2- 172.21.15.6:0/1641894859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88040662e0 0x7f880410da30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f880000b970 tx=0x7f8800009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f87faffd700 1 -- 172.21.15.6:0/1641894859 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f880000e040 con 0x7f88040662e0 2024-03-22T06:23:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f87faffd700 1 -- 172.21.15.6:0/1641894859 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f880000bbf0 con 0x7f88040662e0 2024-03-22T06:23:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f880c650700 1 -- 172.21.15.6:0/1641894859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880410f040 con 0x7f88040662e0 2024-03-22T06:23:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f880c650700 1 -- 172.21.15.6:0/1641894859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880410f580 con 0x7f88040662e0 2024-03-22T06:23:44.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.377+0000 7f87faffd700 1 -- 172.21.15.6:0/1641894859 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f880001c3f0 con 0x7f88040662e0 2024-03-22T06:23:44.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.378+0000 7f880c650700 1 -- 172.21.15.6:0/1641894859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880405f5f0 con 0x7f88040662e0 2024-03-22T06:23:44.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.379+0000 7f87faffd700 1 -- 172.21.15.6:0/1641894859 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f880000bd60 con 0x7f88040662e0 2024-03-22T06:23:44.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.379+0000 7f87faffd700 1 --2- 172.21.15.6:0/1641894859 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87f006e850 0x7f87f0070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:44.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.380+0000 7f8809beb700 1 --2- 172.21.15.6:0/1641894859 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87f006e850 0x7f87f0070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:44.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.380+0000 7f87faffd700 1 -- 172.21.15.6:0/1641894859 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f880008f7e0 con 0x7f88040662e0 2024-03-22T06:23:44.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.380+0000 7f8809beb700 1 --2- 172.21.15.6:0/1641894859 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87f006e850 0x7f87f0070d00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87f4007fd0 tx=0x7f87f4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:44.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.384+0000 7f87faffd700 1 -- 172.21.15.6:0/1641894859 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f880005c200 con 0x7f88040662e0 2024-03-22T06:23:44.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.642+0000 7f880c650700 1 -- 172.21.15.6:0/1641894859 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f880410f8b0 con 0x7f87f006e850 2024-03-22T06:23:44.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.644+0000 7f87faffd700 1 -- 172.21.15.6:0/1641894859 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f880410f8b0 con 0x7f87f006e850 2024-03-22T06:23:44.644 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:23:44.644 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:23:44.644 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:23:44.644 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:23:44.644 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:23:44.644 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:23:44.644 INFO:teuthology.orchestra.run.smithi006.stdout: "mds", 2024-03-22T06:23:44.644 INFO:teuthology.orchestra.run.smithi006.stdout: "osd", 2024-03-22T06:23:44.644 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:23:44.644 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:23:44.645 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:23:44.645 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "16/21 daemons upgraded", 2024-03-22T06:23:44.645 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading mds daemons", 2024-03-22T06:23:44.645 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:23:44.645 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:23:44.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.646+0000 7f880c650700 1 -- 172.21.15.6:0/1641894859 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87f006e850 msgr2=0x7f87f0070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:44.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.646+0000 7f880c650700 1 --2- 172.21.15.6:0/1641894859 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87f006e850 0x7f87f0070d00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87f4007fd0 tx=0x7f87f4009040 comp rx=0 tx=0).stop 2024-03-22T06:23:44.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.646+0000 7f880c650700 1 -- 172.21.15.6:0/1641894859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88040662e0 msgr2=0x7f880410da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:44.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.646+0000 7f880c650700 1 --2- 172.21.15.6:0/1641894859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88040662e0 0x7f880410da30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f880000b970 tx=0x7f8800009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:44.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.646+0000 7f880c650700 1 -- 172.21.15.6:0/1641894859 shutdown_connections 2024-03-22T06:23:44.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.646+0000 7f880c650700 1 --2- 172.21.15.6:0/1641894859 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87f006e850 0x7f87f0070d00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.646+0000 7f880c650700 1 --2- 172.21.15.6:0/1641894859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88040662e0 0x7f880410da30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.646+0000 7f880c650700 1 --2- 172.21.15.6:0/1641894859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88040fff00 0x7f880410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.646+0000 7f880c650700 1 -- 172.21.15.6:0/1641894859 >> 172.21.15.6:0/1641894859 conn(0x7f88040f7430 msgr2=0x7f88040f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:44.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.646+0000 7f880c650700 1 -- 172.21.15.6:0/1641894859 shutdown_connections 2024-03-22T06:23:44.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.646+0000 7f880c650700 1 -- 172.21.15.6:0/1641894859 wait complete. 2024-03-22T06:23:44.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.750+0000 7f18b8460700 1 -- 172.21.15.6:0/1554185990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b01002b0 msgr2=0x7f18b0100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.750+0000 7f18b8460700 1 --2- 172.21.15.6:0/1554185990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b01002b0 0x7f18b0100720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f18a0004940 tx=0x7f18a0009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.750+0000 7f18b8460700 1 -- 172.21.15.6:0/1554185990 shutdown_connections 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.750+0000 7f18b8460700 1 --2- 172.21.15.6:0/1554185990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b01002b0 0x7f18b0100720 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.750+0000 7f18b8460700 1 --2- 172.21.15.6:0/1554185990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18b00ff000 0x7f18b00ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.750+0000 7f18b8460700 1 -- 172.21.15.6:0/1554185990 >> 172.21.15.6:0/1554185990 conn(0x7f18b00fa950 msgr2=0x7f18b00fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.750+0000 7f18b8460700 1 -- 172.21.15.6:0/1554185990 shutdown_connections 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.750+0000 7f18b8460700 1 -- 172.21.15.6:0/1554185990 wait complete. 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.750+0000 7f18b8460700 1 Processor -- start 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.750+0000 7f18b8460700 1 -- start start 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b8460700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18b00ff000 0x7f18b010fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b8460700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b01002b0 0x7f18b01100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:44.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b8460700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b01106b0 con 0x7f18b01002b0 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b8460700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0110f00 con 0x7f18b00ff000 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b59fb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b01002b0 0x7f18b01100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b59fb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b01002b0 0x7f18b01100b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44666/0 (socket says 172.21.15.6:44666) 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b59fb700 1 -- 172.21.15.6:0/2406027447 learned_addr learned my addr 172.21.15.6:0/2406027447 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b61fc700 1 --2- 172.21.15.6:0/2406027447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18b00ff000 0x7f18b010fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b59fb700 1 -- 172.21.15.6:0/2406027447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18b00ff000 msgr2=0x7f18b010fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b59fb700 1 --2- 172.21.15.6:0/2406027447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18b00ff000 0x7f18b010fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b59fb700 1 -- 172.21.15.6:0/2406027447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a0005040 con 0x7f18b01002b0 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b59fb700 1 --2- 172.21.15.6:0/2406027447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b01002b0 0x7f18b01100b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009150 tx=0x7f18a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18a6ffd700 1 -- 172.21.15.6:0/2406027447 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18a000e040 con 0x7f18b01002b0 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18a6ffd700 1 -- 172.21.15.6:0/2406027447 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18a0004290 con 0x7f18b01002b0 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18b8460700 1 -- 172.21.15.6:0/2406027447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b0111180 con 0x7f18b01002b0 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.751+0000 7f18a6ffd700 1 -- 172.21.15.6:0/2406027447 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18a001c630 con 0x7f18b01002b0 2024-03-22T06:23:44.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.752+0000 7f18b8460700 1 -- 172.21.15.6:0/2406027447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b01116c0 con 0x7f18b01002b0 2024-03-22T06:23:44.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.752+0000 7f18b8460700 1 -- 172.21.15.6:0/2406027447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b005f5f0 con 0x7f18b01002b0 2024-03-22T06:23:44.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.753+0000 7f18a6ffd700 1 -- 172.21.15.6:0/2406027447 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f18a000bbf0 con 0x7f18b01002b0 2024-03-22T06:23:44.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.753+0000 7f18a6ffd700 1 --2- 172.21.15.6:0/2406027447 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189c06e840 0x7f189c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:23:44.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.754+0000 7f18a6ffd700 1 -- 172.21.15.6:0/2406027447 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f18a008f090 con 0x7f18b01002b0 2024-03-22T06:23:44.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.754+0000 7f18b61fc700 1 --2- 172.21.15.6:0/2406027447 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189c06e840 0x7f189c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:23:44.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.754+0000 7f18b61fc700 1 --2- 172.21.15.6:0/2406027447 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189c06e840 0x7f189c070cf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009fd0 tx=0x7f18ac00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:23:44.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:44.762+0000 7f18a6ffd700 1 -- 172.21.15.6:0/2406027447 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f18a005bab0 con 0x7f18b01002b0 2024-03-22T06:23:45.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:44 smithi117 ceph-mon[82126]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:45.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:44 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:45.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:44 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:45.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:44 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/4291144905' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:45.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:44 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/4029613078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:23:45.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:44 smithi006 ceph-mon[94482]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:45.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:44 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:45.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:44 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:45.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:44 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/4291144905' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:45.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:44 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/4029613078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:23:45.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.071+0000 7f18b8460700 1 -- 172.21.15.6:0/2406027447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f18b004bed0 con 0x7f18b01002b0 2024-03-22T06:23:45.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.071+0000 7f18a6ffd700 1 -- 172.21.15.6:0/2406027447 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+155 (secure 0 0 0) 0x7f18a005b640 con 0x7f18b01002b0 2024-03-22T06:23:45.071 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:23:45.072 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:23:45.072 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in error state 2024-03-22T06:23:45.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.073+0000 7f18b8460700 1 -- 172.21.15.6:0/2406027447 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189c06e840 msgr2=0x7f189c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:45.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.073+0000 7f18b8460700 1 --2- 172.21.15.6:0/2406027447 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189c06e840 0x7f189c070cf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009fd0 tx=0x7f18ac00b040 comp rx=0 tx=0).stop 2024-03-22T06:23:45.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.073+0000 7f18b8460700 1 -- 172.21.15.6:0/2406027447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b01002b0 msgr2=0x7f18b01100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:23:45.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.073+0000 7f18b8460700 1 --2- 172.21.15.6:0/2406027447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b01002b0 0x7f18b01100b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009150 tx=0x7f18a0009e30 comp rx=0 tx=0).stop 2024-03-22T06:23:45.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.073+0000 7f18b8460700 1 -- 172.21.15.6:0/2406027447 shutdown_connections 2024-03-22T06:23:45.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.073+0000 7f18b8460700 1 --2- 172.21.15.6:0/2406027447 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189c06e840 0x7f189c070cf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:45.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.073+0000 7f18b8460700 1 --2- 172.21.15.6:0/2406027447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18b00ff000 0x7f18b010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:45.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.073+0000 7f18b8460700 1 --2- 172.21.15.6:0/2406027447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b01002b0 0x7f18b01100b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:23:45.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.073+0000 7f18b8460700 1 -- 172.21.15.6:0/2406027447 >> 172.21.15.6:0/2406027447 conn(0x7f18b00fa950 msgr2=0x7f18b00fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:23:45.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.073+0000 7f18b8460700 1 -- 172.21.15.6:0/2406027447 shutdown_connections 2024-03-22T06:23:45.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:23:45.073+0000 7f18b8460700 1 -- 172.21.15.6:0/2406027447 wait complete. 2024-03-22T06:23:45.842 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:45 smithi117 ceph-mon[82126]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:45.842 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:45 smithi117 ceph-mon[82126]: pgmap v249: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-22T06:23:45.842 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:45 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2406027447' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:23:46.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:45 smithi006 ceph-mon[94482]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:23:46.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:45 smithi006 ceph-mon[94482]: pgmap v249: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-22T06:23:46.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:45 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2406027447' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:23:48.385 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: pgmap v250: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 6.9 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-22T06:23:48.385 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.385 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.385 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:23:48.385 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: pgmap v251: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 199 B/s wr, 4 op/s 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: pgmap v252: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s rd, 123 B/s wr, 4 op/s 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi006.bztopc"}]': finished 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:23:48.386 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi006.dcjpla"}]': finished 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.fwkech"}]': finished 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.zqxyql"}]': finished 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.387 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.388 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.388 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:48 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: pgmap v250: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 6.9 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-22T06:23:48.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:23:48.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:23:48.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: pgmap v251: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 199 B/s wr, 4 op/s 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: pgmap v252: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s rd, 123 B/s wr, 4 op/s 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi006.bztopc"}]: dispatch 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi006.bztopc"}]': finished 2024-03-22T06:23:48.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi006.dcjpla"}]: dispatch 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi006.dcjpla"}]': finished 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.fwkech"}]: dispatch 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.fwkech"}]': finished 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.zqxyql"}]: dispatch 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.zqxyql"}]': finished 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:48.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:23:48.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:48 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:49.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:49 smithi006 ceph-mon[94482]: Upgrade: Setting container_image for all mds 2024-03-22T06:23:49.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:49 smithi006 ceph-mon[94482]: Upgrade: Setting container_image for all rgw 2024-03-22T06:23:49.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:49 smithi006 ceph-mon[94482]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T06:23:49.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:49 smithi006 ceph-mon[94482]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T06:23:49.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:49 smithi006 ceph-mon[94482]: Upgrade: Setting container_image for all iscsi 2024-03-22T06:23:49.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:49 smithi006 ceph-mon[94482]: Upgrade: Setting container_image for all nfs 2024-03-22T06:23:49.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:49 smithi006 ceph-mon[94482]: Upgrade: Setting container_image for all nvmeof 2024-03-22T06:23:49.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:49 smithi117 ceph-mon[82126]: Upgrade: Setting container_image for all mds 2024-03-22T06:23:49.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:49 smithi117 ceph-mon[82126]: Upgrade: Setting container_image for all rgw 2024-03-22T06:23:49.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:49 smithi117 ceph-mon[82126]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T06:23:49.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:49 smithi117 ceph-mon[82126]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T06:23:49.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:49 smithi117 ceph-mon[82126]: Upgrade: Setting container_image for all iscsi 2024-03-22T06:23:49.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:49 smithi117 ceph-mon[82126]: Upgrade: Setting container_image for all nfs 2024-03-22T06:23:49.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:49 smithi117 ceph-mon[82126]: Upgrade: Setting container_image for all nvmeof 2024-03-22T06:23:50.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:50 smithi006 ceph-mon[94482]: Upgrade: Updating node-exporter.smithi006 (1/2) 2024-03-22T06:23:50.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:50 smithi006 ceph-mon[94482]: Deploying daemon node-exporter.smithi006 on smithi006 2024-03-22T06:23:50.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:50 smithi006 ceph-mon[94482]: pgmap v253: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-03-22T06:23:50.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:50 smithi117 ceph-mon[82126]: Upgrade: Updating node-exporter.smithi006 (1/2) 2024-03-22T06:23:50.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:50 smithi117 ceph-mon[82126]: Deploying daemon node-exporter.smithi006 on smithi006 2024-03-22T06:23:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:50 smithi117 ceph-mon[82126]: pgmap v253: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-03-22T06:23:52.531 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:52 smithi006 ceph-mon[94482]: pgmap v254: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s rd, 6.2 KiB/s wr, 4 op/s 2024-03-22T06:23:52.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:52.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:52.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:52 smithi117 ceph-mon[82126]: pgmap v254: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s rd, 6.2 KiB/s wr, 4 op/s 2024-03-22T06:23:52.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:52.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:53.548 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:53.548 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:23:53.548 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:53 smithi006 ceph-mon[94482]: pgmap v255: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s wr, 2 op/s 2024-03-22T06:23:53.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:53.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:23:53.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:53 smithi117 ceph-mon[82126]: pgmap v255: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s wr, 2 op/s 2024-03-22T06:23:55.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:54 smithi117 ceph-mon[82126]: Upgrade: Updating node-exporter.smithi117 (2/2) 2024-03-22T06:23:55.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:54 smithi117 ceph-mon[82126]: Deploying daemon node-exporter.smithi117 on smithi117 2024-03-22T06:23:55.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:54 smithi006 ceph-mon[94482]: Upgrade: Updating node-exporter.smithi117 (2/2) 2024-03-22T06:23:55.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:54 smithi006 ceph-mon[94482]: Deploying daemon node-exporter.smithi117 on smithi117 2024-03-22T06:23:56.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:55 smithi117 ceph-mon[82126]: pgmap v256: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s wr, 2 op/s 2024-03-22T06:23:56.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:55 smithi006 ceph-mon[94482]: pgmap v256: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s wr, 2 op/s 2024-03-22T06:23:57.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:57.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:57.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:57.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:57.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:57.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:23:58.440 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:58 smithi006 ceph-mon[94482]: pgmap v257: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 5.1 KiB/s wr, 2 op/s 2024-03-22T06:23:58.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:58 smithi117 ceph-mon[82126]: pgmap v257: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 5.1 KiB/s wr, 2 op/s 2024-03-22T06:23:59.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:59.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:23:59 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:59.531 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:23:59.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:23:59 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:00.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:00 smithi117 ceph-mon[82126]: pgmap v258: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 4.2 KiB/s wr, 1 op/s 2024-03-22T06:24:00.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:00.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:00.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:00.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:00 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:00.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:00 smithi006 ceph-mon[94482]: pgmap v258: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 4.2 KiB/s wr, 1 op/s 2024-03-22T06:24:00.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:00.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:00.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:00.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:00 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:02.423 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:02 smithi006 ceph-mon[94482]: pgmap v259: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:02.424 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:02.424 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:02 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:02.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:02 smithi117 ceph-mon[82126]: pgmap v259: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:02.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:02.573 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:02 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:03.995 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:03 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:03.995 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:03 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:03.995 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:03 smithi006 ceph-mon[94482]: pgmap v260: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:04.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:03 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:04.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:03 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:04.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:03 smithi117 ceph-mon[82126]: pgmap v260: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:06.365 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: pgmap v261: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:06.366 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:06.366 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:06.366 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:06.366 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:06.366 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: pgmap v262: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:06.366 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: pgmap v263: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:06.367 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:06.367 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:06.367 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.367 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.367 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.367 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.367 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.368 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.368 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.368 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.368 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.368 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.368 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.369 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.369 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:06 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: pgmap v261: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: pgmap v262: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: pgmap v263: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:06.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:06.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:06 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:07.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:07 smithi117 ceph-mon[82126]: Upgrade: Updating prometheus.smithi006 2024-03-22T06:24:07.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:07 smithi117 ceph-mon[82126]: pgmap v264: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:07.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:24:07.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:07 smithi006 ceph-mon[94482]: Upgrade: Updating prometheus.smithi006 2024-03-22T06:24:07.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:07 smithi006 ceph-mon[94482]: pgmap v264: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:07.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:24:08.488 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:08 smithi006 ceph-mon[94482]: Deploying daemon prometheus.smithi006 on smithi006 2024-03-22T06:24:08.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:08 smithi117 ceph-mon[82126]: Deploying daemon prometheus.smithi006 on smithi006 2024-03-22T06:24:09.629 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:09 smithi006 ceph-mon[94482]: pgmap v265: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:09.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:09 smithi117 ceph-mon[82126]: pgmap v265: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:11.641 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:11 smithi006 ceph-mon[94482]: pgmap v266: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:11.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:11 smithi117 ceph-mon[82126]: pgmap v266: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:13.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:13 smithi117 ceph-mon[82126]: pgmap v267: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:13.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:13 smithi006 ceph-mon[94482]: pgmap v267: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:14.499 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:14 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:14.499 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:14 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:14.499 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:14 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:14.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:14 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:14.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:14 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:14.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:14 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:15.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.184+0000 7f39f5887700 1 -- 172.21.15.6:0/3407858657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f00fed90 msgr2=0x7f39f00faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:15.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.184+0000 7f39f5887700 1 --2- 172.21.15.6:0/3407858657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f00fed90 0x7f39f00faed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f39e4004940 tx=0x7f39e4009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:15.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.184+0000 7f39f5887700 1 -- 172.21.15.6:0/3407858657 shutdown_connections 2024-03-22T06:24:15.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.184+0000 7f39f5887700 1 --2- 172.21.15.6:0/3407858657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f00fed90 0x7f39f00faed0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.184+0000 7f39f5887700 1 --2- 172.21.15.6:0/3407858657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f00fe440 0x7f39f00fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.184+0000 7f39f5887700 1 -- 172.21.15.6:0/3407858657 >> 172.21.15.6:0/3407858657 conn(0x7f39f00f6bc0 msgr2=0x7f39f00f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:15.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.184+0000 7f39f5887700 1 -- 172.21.15.6:0/3407858657 shutdown_connections 2024-03-22T06:24:15.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.184+0000 7f39f5887700 1 -- 172.21.15.6:0/3407858657 wait complete. 2024-03-22T06:24:15.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.185+0000 7f39f5887700 1 Processor -- start 2024-03-22T06:24:15.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.185+0000 7f39f5887700 1 -- start start 2024-03-22T06:24:15.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.185+0000 7f39f5887700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f00fe440 0x7f39f00fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:15.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.185+0000 7f39f5887700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f00fed90 0x7f39f00fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:15.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.185+0000 7f39f5887700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00fe050 con 0x7f39f00fed90 2024-03-22T06:24:15.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.185+0000 7f39f5887700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00fc640 con 0x7f39f00fe440 2024-03-22T06:24:15.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.185+0000 7f39ee7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f00fed90 0x7f39f00fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:15.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.185+0000 7f39ee7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f00fed90 0x7f39f00fc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35522/0 (socket says 172.21.15.6:35522) 2024-03-22T06:24:15.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.185+0000 7f39eeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f00fe440 0x7f39f00fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:15.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.185+0000 7f39ee7fc700 1 -- 172.21.15.6:0/2064945127 learned_addr learned my addr 172.21.15.6:0/2064945127 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:15.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.186+0000 7f39ee7fc700 1 -- 172.21.15.6:0/2064945127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f00fe440 msgr2=0x7f39f00fda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:15.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.186+0000 7f39ee7fc700 1 --2- 172.21.15.6:0/2064945127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f00fe440 0x7f39f00fda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.186+0000 7f39ee7fc700 1 -- 172.21.15.6:0/2064945127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e4005040 con 0x7f39f00fed90 2024-03-22T06:24:15.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.186+0000 7f39ee7fc700 1 --2- 172.21.15.6:0/2064945127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f00fed90 0x7f39f00fc100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009150 tx=0x7f39e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:15.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.186+0000 7f39d77fe700 1 -- 172.21.15.6:0/2064945127 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39e400e040 con 0x7f39f00fed90 2024-03-22T06:24:15.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.186+0000 7f39d77fe700 1 -- 172.21.15.6:0/2064945127 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39e400bbf0 con 0x7f39f00fed90 2024-03-22T06:24:15.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.186+0000 7f39d77fe700 1 -- 172.21.15.6:0/2064945127 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39e401c630 con 0x7f39f00fed90 2024-03-22T06:24:15.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.186+0000 7f39f5887700 1 -- 172.21.15.6:0/2064945127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f00fc890 con 0x7f39f00fed90 2024-03-22T06:24:15.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.186+0000 7f39f5887700 1 -- 172.21.15.6:0/2064945127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f00fce50 con 0x7f39f00fed90 2024-03-22T06:24:15.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.187+0000 7f39f5887700 1 -- 172.21.15.6:0/2064945127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f004c710 con 0x7f39f00fed90 2024-03-22T06:24:15.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.188+0000 7f39d77fe700 1 -- 172.21.15.6:0/2064945127 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f39e400bd60 con 0x7f39f00fed90 2024-03-22T06:24:15.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.188+0000 7f39d77fe700 1 --2- 172.21.15.6:0/2064945127 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f39d806e840 0x7f39d8070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:15.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.188+0000 7f39d77fe700 1 -- 172.21.15.6:0/2064945127 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f39e408f800 con 0x7f39f00fed90 2024-03-22T06:24:15.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.188+0000 7f39eeffd700 1 --2- 172.21.15.6:0/2064945127 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f39d806e840 0x7f39d8070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:15.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.189+0000 7f39eeffd700 1 --2- 172.21.15.6:0/2064945127 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f39d806e840 0x7f39d8070cf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f39e00041d0 tx=0x7f39e00043c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:15.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.193+0000 7f39d77fe700 1 -- 172.21.15.6:0/2064945127 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f39e405c220 con 0x7f39f00fed90 2024-03-22T06:24:15.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.427+0000 7f39f5887700 1 -- 172.21.15.6:0/2064945127 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f39f01024e0 con 0x7f39d806e840 2024-03-22T06:24:15.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.429+0000 7f39d77fe700 1 -- 172.21.15.6:0/2064945127 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f39f01024e0 con 0x7f39d806e840 2024-03-22T06:24:15.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.431+0000 7f39f5887700 1 -- 172.21.15.6:0/2064945127 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f39d806e840 msgr2=0x7f39d8070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:15.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.431+0000 7f39f5887700 1 --2- 172.21.15.6:0/2064945127 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f39d806e840 0x7f39d8070cf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f39e00041d0 tx=0x7f39e00043c0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.431+0000 7f39f5887700 1 -- 172.21.15.6:0/2064945127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f00fed90 msgr2=0x7f39f00fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:15.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.431+0000 7f39f5887700 1 --2- 172.21.15.6:0/2064945127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f00fed90 0x7f39f00fc100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009150 tx=0x7f39e4009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:15.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.431+0000 7f39f5887700 1 -- 172.21.15.6:0/2064945127 shutdown_connections 2024-03-22T06:24:15.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.431+0000 7f39f5887700 1 --2- 172.21.15.6:0/2064945127 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f39d806e840 0x7f39d8070cf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.431+0000 7f39f5887700 1 --2- 172.21.15.6:0/2064945127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39f00fe440 0x7f39f00fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.431+0000 7f39f5887700 1 --2- 172.21.15.6:0/2064945127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39f00fed90 0x7f39f00fc100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.431+0000 7f39f5887700 1 -- 172.21.15.6:0/2064945127 >> 172.21.15.6:0/2064945127 conn(0x7f39f00f6bc0 msgr2=0x7f39f00f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:15.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.431+0000 7f39f5887700 1 -- 172.21.15.6:0/2064945127 shutdown_connections 2024-03-22T06:24:15.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.431+0000 7f39f5887700 1 -- 172.21.15.6:0/2064945127 wait complete. 2024-03-22T06:24:15.443 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:24:15.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.539+0000 7f0792aa5700 1 -- 172.21.15.6:0/228343948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f078c0662e0 msgr2=0x7f078c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:15.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.539+0000 7f0792aa5700 1 --2- 172.21.15.6:0/228343948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f078c0662e0 0x7f078c0ff9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0780004940 tx=0x7f0780009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:15.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.539+0000 7f0792aa5700 1 -- 172.21.15.6:0/228343948 shutdown_connections 2024-03-22T06:24:15.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.539+0000 7f0792aa5700 1 --2- 172.21.15.6:0/228343948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f078c0fff00 0x7f078c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.539+0000 7f0792aa5700 1 --2- 172.21.15.6:0/228343948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f078c0662e0 0x7f078c0ff9c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.539+0000 7f0792aa5700 1 -- 172.21.15.6:0/228343948 >> 172.21.15.6:0/228343948 conn(0x7f078c0f7430 msgr2=0x7f078c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:15.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.539+0000 7f0792aa5700 1 -- 172.21.15.6:0/228343948 shutdown_connections 2024-03-22T06:24:15.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.540+0000 7f0792aa5700 1 -- 172.21.15.6:0/228343948 wait complete. 2024-03-22T06:24:15.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.540+0000 7f0792aa5700 1 Processor -- start 2024-03-22T06:24:15.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.540+0000 7f0792aa5700 1 -- start start 2024-03-22T06:24:15.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.540+0000 7f0792aa5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f078c0662e0 0x7f078c10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:15.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.540+0000 7f0792aa5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f078c0fff00 0x7f078c10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:15.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.540+0000 7f0792aa5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c10e570 con 0x7f078c0662e0 2024-03-22T06:24:15.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.540+0000 7f0792aa5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c10edc0 con 0x7f078c0fff00 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.540+0000 7f0790841700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f078c0662e0 0x7f078c10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f0790841700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f078c0662e0 0x7f078c10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35534/0 (socket says 172.21.15.6:35534) 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f0790841700 1 -- 172.21.15.6:0/1811988067 learned_addr learned my addr 172.21.15.6:0/1811988067 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f078bfff700 1 --2- 172.21.15.6:0/1811988067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f078c0fff00 0x7f078c10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f0790841700 1 -- 172.21.15.6:0/1811988067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f078c0fff00 msgr2=0x7f078c10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f0790841700 1 --2- 172.21.15.6:0/1811988067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f078c0fff00 0x7f078c10df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f0790841700 1 -- 172.21.15.6:0/1811988067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0780005040 con 0x7f078c0662e0 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f0790841700 1 --2- 172.21.15.6:0/1811988067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f078c0662e0 0x7f078c10da30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f078000b970 tx=0x7f0780009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f07897fa700 1 -- 172.21.15.6:0/1811988067 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f078000e040 con 0x7f078c0662e0 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f07897fa700 1 -- 172.21.15.6:0/1811988067 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0780004290 con 0x7f078c0662e0 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f07897fa700 1 -- 172.21.15.6:0/1811988067 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f078001c3f0 con 0x7f078c0662e0 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f0792aa5700 1 -- 172.21.15.6:0/1811988067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c10f040 con 0x7f078c0662e0 2024-03-22T06:24:15.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.541+0000 7f0792aa5700 1 -- 172.21.15.6:0/1811988067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c10f530 con 0x7f078c0662e0 2024-03-22T06:24:15.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.543+0000 7f0792aa5700 1 -- 172.21.15.6:0/1811988067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c05f5f0 con 0x7f078c0662e0 2024-03-22T06:24:15.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.543+0000 7f07897fa700 1 -- 172.21.15.6:0/1811988067 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f078000bbf0 con 0x7f078c0662e0 2024-03-22T06:24:15.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.543+0000 7f07897fa700 1 --2- 172.21.15.6:0/1811988067 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f077c06e850 0x7f077c070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:15.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.543+0000 7f07897fa700 1 -- 172.21.15.6:0/1811988067 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f078008f840 con 0x7f078c0662e0 2024-03-22T06:24:15.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.543+0000 7f078bfff700 1 --2- 172.21.15.6:0/1811988067 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f077c06e850 0x7f077c070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:15.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.544+0000 7f078bfff700 1 --2- 172.21.15.6:0/1811988067 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f077c06e850 0x7f077c070d00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0778004f40 tx=0x7f077800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:15.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.549+0000 7f07897fa700 1 -- 172.21.15.6:0/1811988067 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f078005c1c0 con 0x7f078c0662e0 2024-03-22T06:24:15.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:15 smithi117 ceph-mon[82126]: pgmap v268: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:15.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:15 smithi006 ceph-mon[94482]: pgmap v268: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:15.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.807+0000 7f0792aa5700 1 -- 172.21.15.6:0/1811988067 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f078c10f8b0 con 0x7f077c06e850 2024-03-22T06:24:15.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.809+0000 7f07897fa700 1 -- 172.21.15.6:0/1811988067 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f078c10f8b0 con 0x7f077c06e850 2024-03-22T06:24:15.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.811+0000 7f0792aa5700 1 -- 172.21.15.6:0/1811988067 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f077c06e850 msgr2=0x7f077c070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:15.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.811+0000 7f0792aa5700 1 --2- 172.21.15.6:0/1811988067 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f077c06e850 0x7f077c070d00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0778004f40 tx=0x7f077800b040 comp rx=0 tx=0).stop 2024-03-22T06:24:15.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.811+0000 7f0792aa5700 1 -- 172.21.15.6:0/1811988067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f078c0662e0 msgr2=0x7f078c10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:15.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.811+0000 7f0792aa5700 1 --2- 172.21.15.6:0/1811988067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f078c0662e0 0x7f078c10da30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f078000b970 tx=0x7f0780009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:15.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.811+0000 7f0792aa5700 1 -- 172.21.15.6:0/1811988067 shutdown_connections 2024-03-22T06:24:15.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.811+0000 7f0792aa5700 1 --2- 172.21.15.6:0/1811988067 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f077c06e850 0x7f077c070d00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.811+0000 7f0792aa5700 1 --2- 172.21.15.6:0/1811988067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f078c0662e0 0x7f078c10da30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.811+0000 7f0792aa5700 1 --2- 172.21.15.6:0/1811988067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f078c0fff00 0x7f078c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.811+0000 7f0792aa5700 1 -- 172.21.15.6:0/1811988067 >> 172.21.15.6:0/1811988067 conn(0x7f078c0f7430 msgr2=0x7f078c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:15.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.811+0000 7f0792aa5700 1 -- 172.21.15.6:0/1811988067 shutdown_connections 2024-03-22T06:24:15.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.811+0000 7f0792aa5700 1 -- 172.21.15.6:0/1811988067 wait complete. 2024-03-22T06:24:15.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.926+0000 7f914159a700 1 -- 172.21.15.6:0/929331927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0fcf80 msgr2=0x7f913c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:15.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.926+0000 7f914159a700 1 --2- 172.21.15.6:0/929331927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0fcf80 0x7f913c0fd3f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9130004940 tx=0x7f9130009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:15.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.927+0000 7f914159a700 1 -- 172.21.15.6:0/929331927 shutdown_connections 2024-03-22T06:24:15.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.927+0000 7f914159a700 1 --2- 172.21.15.6:0/929331927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0fcf80 0x7f913c0fd3f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.927+0000 7f914159a700 1 --2- 172.21.15.6:0/929331927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913c0fbcd0 0x7f913c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.927+0000 7f914159a700 1 -- 172.21.15.6:0/929331927 >> 172.21.15.6:0/929331927 conn(0x7f913c0f7430 msgr2=0x7f913c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:15.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.927+0000 7f914159a700 1 -- 172.21.15.6:0/929331927 shutdown_connections 2024-03-22T06:24:15.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.927+0000 7f914159a700 1 -- 172.21.15.6:0/929331927 wait complete. 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f914159a700 1 Processor -- start 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f914159a700 1 -- start start 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f914159a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0fbcd0 0x7f913c0ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f914159a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913c0fcf80 0x7f913c0fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f914159a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0fe850 con 0x7f913c0fbcd0 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f914159a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0fe990 con 0x7f913c0fcf80 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f913affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0fbcd0 0x7f913c0ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f913affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0fbcd0 0x7f913c0ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35544/0 (socket says 172.21.15.6:35544) 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f913affd700 1 -- 172.21.15.6:0/1226026080 learned_addr learned my addr 172.21.15.6:0/1226026080 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f913a7fc700 1 --2- 172.21.15.6:0/1226026080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913c0fcf80 0x7f913c0fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f913affd700 1 -- 172.21.15.6:0/1226026080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913c0fcf80 msgr2=0x7f913c0fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:15.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.928+0000 7f913affd700 1 --2- 172.21.15.6:0/1226026080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913c0fcf80 0x7f913c0fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:15.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.929+0000 7f913affd700 1 -- 172.21.15.6:0/1226026080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9130005040 con 0x7f913c0fbcd0 2024-03-22T06:24:15.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.929+0000 7f913affd700 1 --2- 172.21.15.6:0/1226026080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0fbcd0 0x7f913c0ffda0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f912c002a80 tx=0x7f912c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:15.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.929+0000 7f91237fe700 1 -- 172.21.15.6:0/1226026080 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912c007cc0 con 0x7f913c0fbcd0 2024-03-22T06:24:15.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.929+0000 7f914159a700 1 -- 172.21.15.6:0/1226026080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c0fec70 con 0x7f913c0fbcd0 2024-03-22T06:24:15.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.929+0000 7f91237fe700 1 -- 172.21.15.6:0/1226026080 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f912c00f040 con 0x7f913c0fbcd0 2024-03-22T06:24:15.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.929+0000 7f91237fe700 1 -- 172.21.15.6:0/1226026080 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912c00dac0 con 0x7f913c0fbcd0 2024-03-22T06:24:15.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.929+0000 7f914159a700 1 -- 172.21.15.6:0/1226026080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c1159f0 con 0x7f913c0fbcd0 2024-03-22T06:24:15.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.930+0000 7f914159a700 1 -- 172.21.15.6:0/1226026080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c101b10 con 0x7f913c0fbcd0 2024-03-22T06:24:15.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.931+0000 7f91237fe700 1 -- 172.21.15.6:0/1226026080 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f912c00dcc0 con 0x7f913c0fbcd0 2024-03-22T06:24:15.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.931+0000 7f91237fe700 1 --2- 172.21.15.6:0/1226026080 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f912406e840 0x7f9124070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:15.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.931+0000 7f91237fe700 1 -- 172.21.15.6:0/1226026080 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f912c08e120 con 0x7f913c0fbcd0 2024-03-22T06:24:15.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.931+0000 7f913a7fc700 1 --2- 172.21.15.6:0/1226026080 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f912406e840 0x7f9124070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:15.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.931+0000 7f913a7fc700 1 --2- 172.21.15.6:0/1226026080 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f912406e840 0x7f9124070cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f913000b970 tx=0x7f9130009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:15.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:15.936+0000 7f91237fe700 1 -- 172.21.15.6:0/1226026080 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f912c056f70 con 0x7f913c0fbcd0 2024-03-22T06:24:16.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.160+0000 7f914159a700 1 -- 172.21.15.6:0/1226026080 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f913c104970 con 0x7f912406e840 2024-03-22T06:24:16.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.170+0000 7f91237fe700 1 -- 172.21.15.6:0/1226026080 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f9130003dd0 con 0x7f912406e840 2024-03-22T06:24:16.170 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:24:16.170 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (8m) 15s ago 17m 26.5M - 0.25.0 c8568f914cd2 cd9352aee257 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (5m) 15s ago 17m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (5m) 17s ago 16m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 error 15s ago 17m - - 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (113s) 15s ago 11m 82.1M - 19.0.0-2299-gea01217f aa461e7c5e7f 3e3319f646e6 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (86s) 15s ago 11m 12.9M - 19.0.0-2299-gea01217f aa461e7c5e7f 09ee1e271adb 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (62s) 17s ago 11m 15.0M - 19.0.0-2299-gea01217f aa461e7c5e7f a9149dea3f8e 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (36s) 17s ago 11m 12.7M - 19.0.0-2299-gea01217f aa461e7c5e7f 0abdc3f226cc 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (8m) 15s ago 18m 511M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (7m) 17s ago 16m 452M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (6m) 15s ago 18m 56.2M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (6m) 17s ago 16m 44.7M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (24s) 15s ago 17m 8066k - 1.5.0 0da6a335fe13 7eb15a15b4c3 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (20s) 17s ago 16m 3275k - 1.5.0 0da6a335fe13 df7460dcdb08 2024-03-22T06:24:16.171 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (5m) 15s ago 15m 155M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:24:16.172 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (4m) 15s ago 15m 138M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:24:16.172 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (3m) 15s ago 14m 106M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 3ec1f219c19c 2024-03-22T06:24:16.172 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (3m) 17s ago 14m 114M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 9cba39b6f32c 2024-03-22T06:24:16.172 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (2m) 17s ago 13m 104M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1950fc21bd2e 2024-03-22T06:24:16.172 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (2m) 17s ago 13m 108M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1a6a0ee2a228 2024-03-22T06:24:16.172 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 starting - - - - 2024-03-22T06:24:16.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.172+0000 7f914159a700 1 -- 172.21.15.6:0/1226026080 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f912406e840 msgr2=0x7f9124070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:16.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.172+0000 7f914159a700 1 --2- 172.21.15.6:0/1226026080 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f912406e840 0x7f9124070cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f913000b970 tx=0x7f9130009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:16.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.172+0000 7f914159a700 1 -- 172.21.15.6:0/1226026080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0fbcd0 msgr2=0x7f913c0ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:16.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.172+0000 7f914159a700 1 --2- 172.21.15.6:0/1226026080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0fbcd0 0x7f913c0ffda0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f912c002a80 tx=0x7f912c0046b0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.173+0000 7f914159a700 1 -- 172.21.15.6:0/1226026080 shutdown_connections 2024-03-22T06:24:16.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.173+0000 7f914159a700 1 --2- 172.21.15.6:0/1226026080 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f912406e840 0x7f9124070cf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.173+0000 7f914159a700 1 --2- 172.21.15.6:0/1226026080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0fbcd0 0x7f913c0ffda0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.173+0000 7f914159a700 1 --2- 172.21.15.6:0/1226026080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913c0fcf80 0x7f913c0fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.173+0000 7f914159a700 1 -- 172.21.15.6:0/1226026080 >> 172.21.15.6:0/1226026080 conn(0x7f913c0f7430 msgr2=0x7f913c065b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:16.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.173+0000 7f914159a700 1 -- 172.21.15.6:0/1226026080 shutdown_connections 2024-03-22T06:24:16.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.173+0000 7f914159a700 1 -- 172.21.15.6:0/1226026080 wait complete. 2024-03-22T06:24:16.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.276+0000 7f8df60f7700 1 -- 172.21.15.6:0/3500534365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8df00fdff0 msgr2=0x7f8df00fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.276+0000 7f8df60f7700 1 --2- 172.21.15.6:0/3500534365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8df00fdff0 0x7f8df00fe460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8de4004940 tx=0x7f8de4009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.276+0000 7f8df60f7700 1 -- 172.21.15.6:0/3500534365 shutdown_connections 2024-03-22T06:24:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.276+0000 7f8df60f7700 1 --2- 172.21.15.6:0/3500534365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8df00fdff0 0x7f8df00fe460 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.276+0000 7f8df60f7700 1 --2- 172.21.15.6:0/3500534365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8df00fce80 0x7f8df00fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.276+0000 7f8df60f7700 1 -- 172.21.15.6:0/3500534365 >> 172.21.15.6:0/3500534365 conn(0x7f8df00f8580 msgr2=0x7f8df00fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.276+0000 7f8df60f7700 1 -- 172.21.15.6:0/3500534365 shutdown_connections 2024-03-22T06:24:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.276+0000 7f8df60f7700 1 -- 172.21.15.6:0/3500534365 wait complete. 2024-03-22T06:24:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.277+0000 7f8df60f7700 1 Processor -- start 2024-03-22T06:24:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.277+0000 7f8df60f7700 1 -- start start 2024-03-22T06:24:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.277+0000 7f8df60f7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8df00fce80 0x7f8df010da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.277+0000 7f8df60f7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8df00fdff0 0x7f8df010df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.277+0000 7f8df60f7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df010e550 con 0x7f8df00fce80 2024-03-22T06:24:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.277+0000 7f8df60f7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df010eda0 con 0x7f8df00fdff0 2024-03-22T06:24:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.277+0000 7f8def7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8df00fce80 0x7f8df010da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.277+0000 7f8def7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8df00fce80 0x7f8df010da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35554/0 (socket says 172.21.15.6:35554) 2024-03-22T06:24:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.277+0000 7f8def7fe700 1 -- 172.21.15.6:0/3598685384 learned_addr learned my addr 172.21.15.6:0/3598685384 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.278+0000 7f8deeffd700 1 --2- 172.21.15.6:0/3598685384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8df00fdff0 0x7f8df010df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.278+0000 7f8def7fe700 1 -- 172.21.15.6:0/3598685384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8df00fdff0 msgr2=0x7f8df010df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.278+0000 7f8def7fe700 1 --2- 172.21.15.6:0/3598685384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8df00fdff0 0x7f8df010df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.278+0000 7f8def7fe700 1 -- 172.21.15.6:0/3598685384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de4005040 con 0x7f8df00fce80 2024-03-22T06:24:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.278+0000 7f8def7fe700 1 --2- 172.21.15.6:0/3598685384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8df00fce80 0x7f8df010da10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8de0002a50 tx=0x7f8de00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.278+0000 7f8dd7fff700 1 -- 172.21.15.6:0/3598685384 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8de0007cc0 con 0x7f8df00fce80 2024-03-22T06:24:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.278+0000 7f8df60f7700 1 -- 172.21.15.6:0/3598685384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df010f080 con 0x7f8df00fce80 2024-03-22T06:24:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.278+0000 7f8dd7fff700 1 -- 172.21.15.6:0/3598685384 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8de000f040 con 0x7f8df00fce80 2024-03-22T06:24:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.278+0000 7f8df60f7700 1 -- 172.21.15.6:0/3598685384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df010f5c0 con 0x7f8df00fce80 2024-03-22T06:24:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.279+0000 7f8dd7fff700 1 -- 172.21.15.6:0/3598685384 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8de000d9f0 con 0x7f8df00fce80 2024-03-22T06:24:16.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.279+0000 7f8df60f7700 1 -- 172.21.15.6:0/3598685384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df005f5f0 con 0x7f8df00fce80 2024-03-22T06:24:16.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.280+0000 7f8dd7fff700 1 -- 172.21.15.6:0/3598685384 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f8de000db50 con 0x7f8df00fce80 2024-03-22T06:24:16.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.280+0000 7f8dd7fff700 1 --2- 172.21.15.6:0/3598685384 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8dd806e840 0x7f8dd8070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:16.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.281+0000 7f8deeffd700 1 --2- 172.21.15.6:0/3598685384 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8dd806e840 0x7f8dd8070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:16.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.281+0000 7f8dd7fff700 1 -- 172.21.15.6:0/3598685384 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f8de008e300 con 0x7f8df00fce80 2024-03-22T06:24:16.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.281+0000 7f8deeffd700 1 --2- 172.21.15.6:0/3598685384 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8dd806e840 0x7f8dd8070cf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8de400b970 tx=0x7f8de4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:16.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.288+0000 7f8dd7fff700 1 -- 172.21.15.6:0/3598685384 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f8de005ac00 con 0x7f8df00fce80 2024-03-22T06:24:16.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.707+0000 7f8df60f7700 1 -- 172.21.15.6:0/3598685384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8df004bed0 con 0x7f8df00fce80 2024-03-22T06:24:16.708 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:16 smithi006 ceph-mon[94482]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:16.708 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:16 smithi006 ceph-mon[94482]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:16.708 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:16 smithi006 ceph-mon[94482]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:16.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.708+0000 7f8dd7fff700 1 -- 172.21.15.6:0/3598685384 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8de005a790 con 0x7f8df00fce80 2024-03-22T06:24:16.709 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:24:16.709 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:24:16.709 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:24:16.709 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:24:16.709 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:24:16.709 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:24:16.709 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:24:16.709 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:24:16.709 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 6 2024-03-22T06:24:16.710 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:24:16.710 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:24:16.710 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 4 2024-03-22T06:24:16.710 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:24:16.710 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:24:16.710 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 14 2024-03-22T06:24:16.710 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:24:16.710 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:24:16.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.710+0000 7f8df60f7700 1 -- 172.21.15.6:0/3598685384 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8dd806e840 msgr2=0x7f8dd8070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:16.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.710+0000 7f8df60f7700 1 --2- 172.21.15.6:0/3598685384 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8dd806e840 0x7f8dd8070cf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8de400b970 tx=0x7f8de4009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:16.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.710+0000 7f8df60f7700 1 -- 172.21.15.6:0/3598685384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8df00fce80 msgr2=0x7f8df010da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:16.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.710+0000 7f8df60f7700 1 --2- 172.21.15.6:0/3598685384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8df00fce80 0x7f8df010da10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8de0002a50 tx=0x7f8de00046b0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.710+0000 7f8df60f7700 1 -- 172.21.15.6:0/3598685384 shutdown_connections 2024-03-22T06:24:16.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.710+0000 7f8df60f7700 1 --2- 172.21.15.6:0/3598685384 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8dd806e840 0x7f8dd8070cf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.710+0000 7f8df60f7700 1 --2- 172.21.15.6:0/3598685384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8df00fce80 0x7f8df010da10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.710+0000 7f8df60f7700 1 --2- 172.21.15.6:0/3598685384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8df00fdff0 0x7f8df010df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.710+0000 7f8df60f7700 1 -- 172.21.15.6:0/3598685384 >> 172.21.15.6:0/3598685384 conn(0x7f8df00f8580 msgr2=0x7f8df00f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:16.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.710+0000 7f8df60f7700 1 -- 172.21.15.6:0/3598685384 shutdown_connections 2024-03-22T06:24:16.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.710+0000 7f8df60f7700 1 -- 172.21.15.6:0/3598685384 wait complete. 2024-03-22T06:24:16.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:16 smithi117 ceph-mon[82126]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:16.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:16 smithi117 ceph-mon[82126]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:16.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:16 smithi117 ceph-mon[82126]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:16.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.842+0000 7feaa1c37700 1 -- 172.21.15.6:0/599133254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea9c0fce50 msgr2=0x7fea9c0fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:16.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.842+0000 7feaa1c37700 1 --2- 172.21.15.6:0/599133254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea9c0fce50 0x7fea9c0fd260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fea8c004940 tx=0x7fea8c009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:16.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.842+0000 7feaa1c37700 1 -- 172.21.15.6:0/599133254 shutdown_connections 2024-03-22T06:24:16.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.842+0000 7feaa1c37700 1 --2- 172.21.15.6:0/599133254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c0fe050 0x7fea9c0fe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.842+0000 7feaa1c37700 1 --2- 172.21.15.6:0/599133254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea9c0fce50 0x7fea9c0fd260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.842+0000 7feaa1c37700 1 -- 172.21.15.6:0/599133254 >> 172.21.15.6:0/599133254 conn(0x7fea9c0f8560 msgr2=0x7fea9c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:16.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.842+0000 7feaa1c37700 1 -- 172.21.15.6:0/599133254 shutdown_connections 2024-03-22T06:24:16.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.842+0000 7feaa1c37700 1 -- 172.21.15.6:0/599133254 wait complete. 2024-03-22T06:24:16.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.843+0000 7feaa1c37700 1 Processor -- start 2024-03-22T06:24:16.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.843+0000 7feaa1c37700 1 -- start start 2024-03-22T06:24:16.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.843+0000 7feaa1c37700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea9c0fe050 0x7fea9c10db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:16.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.843+0000 7feaa1c37700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c10e050 0x7fea9c113080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.843+0000 7feaa1c37700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c10e5a0 con 0x7fea9c0fe050 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.843+0000 7feaa1c37700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c10e6e0 con 0x7fea9c10e050 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.843+0000 7fea9b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea9c0fe050 0x7fea9c10db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.843+0000 7fea9b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea9c0fe050 0x7fea9c10db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35576/0 (socket says 172.21.15.6:35576) 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.843+0000 7fea9b7fe700 1 -- 172.21.15.6:0/3539916493 learned_addr learned my addr 172.21.15.6:0/3539916493 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.844+0000 7fea9affd700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c10e050 0x7fea9c113080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.844+0000 7fea9b7fe700 1 -- 172.21.15.6:0/3539916493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c10e050 msgr2=0x7fea9c113080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.844+0000 7fea9b7fe700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c10e050 0x7fea9c113080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.844+0000 7fea9b7fe700 1 -- 172.21.15.6:0/3539916493 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c005040 con 0x7fea9c0fe050 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.844+0000 7fea9affd700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c10e050 0x7fea9c113080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.844+0000 7fea9b7fe700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea9c0fe050 0x7fea9c10db10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00b970 tx=0x7fea8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.844+0000 7fea83fff700 1 -- 172.21.15.6:0/3539916493 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea8c00e070 con 0x7fea9c0fe050 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.844+0000 7fea83fff700 1 -- 172.21.15.6:0/3539916493 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea8c00bbf0 con 0x7fea9c0fe050 2024-03-22T06:24:16.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.844+0000 7fea83fff700 1 -- 172.21.15.6:0/3539916493 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea8c01c400 con 0x7fea9c0fe050 2024-03-22T06:24:16.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.844+0000 7feaa1c37700 1 -- 172.21.15.6:0/3539916493 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c113620 con 0x7fea9c0fe050 2024-03-22T06:24:16.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.844+0000 7feaa1c37700 1 -- 172.21.15.6:0/3539916493 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c113bb0 con 0x7fea9c0fe050 2024-03-22T06:24:16.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.845+0000 7feaa1c37700 1 -- 172.21.15.6:0/3539916493 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9c04c710 con 0x7fea9c0fe050 2024-03-22T06:24:16.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.846+0000 7fea83fff700 1 -- 172.21.15.6:0/3539916493 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fea8c013420 con 0x7fea9c0fe050 2024-03-22T06:24:16.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.847+0000 7fea83fff700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fea8406e880 0x7fea84070d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:16.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.847+0000 7fea83fff700 1 -- 172.21.15.6:0/3539916493 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fea8c08fe40 con 0x7fea9c0fe050 2024-03-22T06:24:16.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.847+0000 7fea9affd700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fea8406e880 0x7fea84070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:16.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.848+0000 7fea9affd700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fea8406e880 0x7fea84070d30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea90009b10 tx=0x7fea90008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:16.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:16.856+0000 7fea83fff700 1 -- 172.21.15.6:0/3539916493 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fea8c05c860 con 0x7fea9c0fe050 2024-03-22T06:24:17.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.124+0000 7feaa1c37700 1 -- 172.21.15.6:0/3539916493 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fea9c105960 con 0x7fea9c0fe050 2024-03-22T06:24:17.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.124+0000 7fea83fff700 1 -- 172.21.15.6:0/3539916493 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1908 (secure 0 0 0) 0x7fea8c05c3f0 con 0x7fea9c0fe050 2024-03-22T06:24:17.125 INFO:teuthology.orchestra.run.smithi006.stdout:e36 2024-03-22T06:24:17.125 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:24:17.125 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:24:17.125 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:24:17.125 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:24:17.125 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:24:17.125 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:24:17.125 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 35 2024-03-22T06:24:17.125 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:24:17.125 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:24:17.125 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:23:39.065820+0000 2024-03-22T06:24:17.126 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:24:17.126 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:24:17.126 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:24:17.126 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:24:17.126 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:24:17.126 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:24:17.126 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:24:17.126 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:24:17.126 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 95 2024-03-22T06:24:17.126 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T06:24:17.126 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:24:17.127 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:24:17.127 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=34406} 2024-03-22T06:24:17.127 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:24:17.127 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:24:17.127 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 34406 members: 34406 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:34406} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:24:17.128 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:34438} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.6:6828/1244315045,v1:172.21.15.6:6829/1244315045] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:24:17.129 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6824/1194667779,v1:172.21.15.117:6825/1194667779] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:24:17.129 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2788388155,v1:172.21.15.117:6827/2788388155] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:24:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.126+0000 7feaa1c37700 1 -- 172.21.15.6:0/3539916493 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fea8406e880 msgr2=0x7fea84070d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.126+0000 7feaa1c37700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fea8406e880 0x7fea84070d30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea90009b10 tx=0x7fea90008040 comp rx=0 tx=0).stop 2024-03-22T06:24:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.126+0000 7feaa1c37700 1 -- 172.21.15.6:0/3539916493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea9c0fe050 msgr2=0x7fea9c10db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.126+0000 7feaa1c37700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea9c0fe050 0x7fea9c10db10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00b970 tx=0x7fea8c009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.126+0000 7feaa1c37700 1 -- 172.21.15.6:0/3539916493 shutdown_connections 2024-03-22T06:24:17.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.126+0000 7feaa1c37700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fea8406e880 0x7fea84070d30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.126+0000 7feaa1c37700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea9c0fe050 0x7fea9c10db10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.126+0000 7feaa1c37700 1 --2- 172.21.15.6:0/3539916493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c10e050 0x7fea9c113080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.126+0000 7feaa1c37700 1 -- 172.21.15.6:0/3539916493 >> 172.21.15.6:0/3539916493 conn(0x7fea9c0f8560 msgr2=0x7fea9c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:17.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.126+0000 7feaa1c37700 1 -- 172.21.15.6:0/3539916493 shutdown_connections 2024-03-22T06:24:17.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.126+0000 7feaa1c37700 1 -- 172.21.15.6:0/3539916493 wait complete. 2024-03-22T06:24:17.131 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 36 2024-03-22T06:24:17.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.235+0000 7f32531eb700 1 -- 172.21.15.6:0/2042589278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f324c0fcef0 msgr2=0x7f324c0fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:17.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.235+0000 7f32531eb700 1 --2- 172.21.15.6:0/2042589278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f324c0fcef0 0x7f324c0fd300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3240004940 tx=0x7f3240009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:17.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.235+0000 7f32531eb700 1 -- 172.21.15.6:0/2042589278 shutdown_connections 2024-03-22T06:24:17.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.235+0000 7f32531eb700 1 --2- 172.21.15.6:0/2042589278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0fe060 0x7f324c0fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.235+0000 7f32531eb700 1 --2- 172.21.15.6:0/2042589278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f324c0fcef0 0x7f324c0fd300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.235+0000 7f32531eb700 1 -- 172.21.15.6:0/2042589278 >> 172.21.15.6:0/2042589278 conn(0x7f324c0f85b0 msgr2=0x7f324c0fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:17.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.235+0000 7f32531eb700 1 -- 172.21.15.6:0/2042589278 shutdown_connections 2024-03-22T06:24:17.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.235+0000 7f32531eb700 1 -- 172.21.15.6:0/2042589278 wait complete. 2024-03-22T06:24:17.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.236+0000 7f32531eb700 1 Processor -- start 2024-03-22T06:24:17.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.236+0000 7f32531eb700 1 -- start start 2024-03-22T06:24:17.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.236+0000 7f32531eb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f324c0fcef0 0x7f324c10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:17.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.236+0000 7f32531eb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0fe060 0x7f324c10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:17.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.236+0000 7f32531eb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c10e5d0 con 0x7f324c0fcef0 2024-03-22T06:24:17.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.236+0000 7f32531eb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c10edc0 con 0x7f324c0fe060 2024-03-22T06:24:17.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.236+0000 7f3250f87700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f324c0fcef0 0x7f324c10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:17.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.236+0000 7f3250f87700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f324c0fcef0 0x7f324c10da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35584/0 (socket says 172.21.15.6:35584) 2024-03-22T06:24:17.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.236+0000 7f3250f87700 1 -- 172.21.15.6:0/8803946 learned_addr learned my addr 172.21.15.6:0/8803946 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:17.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.237+0000 7f324bfff700 1 --2- 172.21.15.6:0/8803946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0fe060 0x7f324c10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.237+0000 7f3250f87700 1 -- 172.21.15.6:0/8803946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0fe060 msgr2=0x7f324c10df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.237+0000 7f3250f87700 1 --2- 172.21.15.6:0/8803946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0fe060 0x7f324c10df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.237+0000 7f3250f87700 1 -- 172.21.15.6:0/8803946 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3240005040 con 0x7f324c0fcef0 2024-03-22T06:24:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.237+0000 7f3250f87700 1 --2- 172.21.15.6:0/8803946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f324c0fcef0 0x7f324c10da00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f324000b3b0 tx=0x7f3240009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.237+0000 7f32497fa700 1 -- 172.21.15.6:0/8803946 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f324000e040 con 0x7f324c0fcef0 2024-03-22T06:24:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.237+0000 7f32497fa700 1 -- 172.21.15.6:0/8803946 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3240004290 con 0x7f324c0fcef0 2024-03-22T06:24:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.237+0000 7f32531eb700 1 -- 172.21.15.6:0/8803946 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324c10f040 con 0x7f324c0fcef0 2024-03-22T06:24:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.237+0000 7f32497fa700 1 -- 172.21.15.6:0/8803946 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f324001c630 con 0x7f324c0fcef0 2024-03-22T06:24:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.237+0000 7f32531eb700 1 -- 172.21.15.6:0/8803946 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324c10f600 con 0x7f324c0fcef0 2024-03-22T06:24:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.238+0000 7f32531eb700 1 -- 172.21.15.6:0/8803946 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324c05f5f0 con 0x7f324c0fcef0 2024-03-22T06:24:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.239+0000 7f32497fa700 1 -- 172.21.15.6:0/8803946 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f324000bbf0 con 0x7f324c0fcef0 2024-03-22T06:24:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.239+0000 7f32497fa700 1 --2- 172.21.15.6:0/8803946 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f323c06e850 0x7f323c070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.239+0000 7f32497fa700 1 -- 172.21.15.6:0/8803946 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f324008f9f0 con 0x7f324c0fcef0 2024-03-22T06:24:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.239+0000 7f324bfff700 1 --2- 172.21.15.6:0/8803946 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f323c06e850 0x7f323c070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.240+0000 7f324bfff700 1 --2- 172.21.15.6:0/8803946 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f323c06e850 0x7f323c070d00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3238006fd0 tx=0x7f3238008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:17.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.244+0000 7f32497fa700 1 -- 172.21.15.6:0/8803946 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f3240091e60 con 0x7f324c0fcef0 2024-03-22T06:24:17.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.479+0000 7f32531eb700 1 -- 172.21.15.6:0/8803946 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f324c10f8e0 con 0x7f323c06e850 2024-03-22T06:24:17.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.480+0000 7f32497fa700 1 -- 172.21.15.6:0/8803946 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f324c10f8e0 con 0x7f323c06e850 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "mds", 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "osd", 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "16/21 daemons upgraded", 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading prometheus daemons", 2024-03-22T06:24:17.481 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:24:17.482 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:24:17.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.482+0000 7f32531eb700 1 -- 172.21.15.6:0/8803946 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f323c06e850 msgr2=0x7f323c070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:17.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.482+0000 7f32531eb700 1 --2- 172.21.15.6:0/8803946 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f323c06e850 0x7f323c070d00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3238006fd0 tx=0x7f3238008040 comp rx=0 tx=0).stop 2024-03-22T06:24:17.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.482+0000 7f32531eb700 1 -- 172.21.15.6:0/8803946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f324c0fcef0 msgr2=0x7f324c10da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:17.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.482+0000 7f32531eb700 1 --2- 172.21.15.6:0/8803946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f324c0fcef0 0x7f324c10da00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f324000b3b0 tx=0x7f3240009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:17.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.483+0000 7f32531eb700 1 -- 172.21.15.6:0/8803946 shutdown_connections 2024-03-22T06:24:17.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.483+0000 7f32531eb700 1 --2- 172.21.15.6:0/8803946 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f323c06e850 0x7f323c070d00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.483+0000 7f32531eb700 1 --2- 172.21.15.6:0/8803946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f324c0fcef0 0x7f324c10da00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.483+0000 7f32531eb700 1 --2- 172.21.15.6:0/8803946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0fe060 0x7f324c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.483+0000 7f32531eb700 1 -- 172.21.15.6:0/8803946 >> 172.21.15.6:0/8803946 conn(0x7f324c0f85b0 msgr2=0x7f324c0f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:17.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.483+0000 7f32531eb700 1 -- 172.21.15.6:0/8803946 shutdown_connections 2024-03-22T06:24:17.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.483+0000 7f32531eb700 1 -- 172.21.15.6:0/8803946 wait complete. 2024-03-22T06:24:17.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.618+0000 7f37a4651700 1 -- 172.21.15.6:0/4108890513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c0fbcd0 msgr2=0x7f379c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:17.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.618+0000 7f37a4651700 1 --2- 172.21.15.6:0/4108890513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c0fbcd0 0x7f379c0fc0e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3798004940 tx=0x7f3798009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:17.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 -- 172.21.15.6:0/4108890513 shutdown_connections 2024-03-22T06:24:17.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 --2- 172.21.15.6:0/4108890513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0fcf80 0x7f379c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 --2- 172.21.15.6:0/4108890513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c0fbcd0 0x7f379c0fc0e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 -- 172.21.15.6:0/4108890513 >> 172.21.15.6:0/4108890513 conn(0x7f379c0f7430 msgr2=0x7f379c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:17.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 -- 172.21.15.6:0/4108890513 shutdown_connections 2024-03-22T06:24:17.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 -- 172.21.15.6:0/4108890513 wait complete. 2024-03-22T06:24:17.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 Processor -- start 2024-03-22T06:24:17.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 -- start start 2024-03-22T06:24:17.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c0fbcd0 0x7f379c10daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:17.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0fcf80 0x7f379c10dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:17.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c10e5e0 con 0x7f379c0fbcd0 2024-03-22T06:24:17.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.619+0000 7f37a4651700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c10ed90 con 0x7f379c0fcf80 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f37a23ed700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c0fbcd0 0x7f379c10daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f37a23ed700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c0fbcd0 0x7f379c10daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35600/0 (socket says 172.21.15.6:35600) 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f37a23ed700 1 -- 172.21.15.6:0/1444535794 learned_addr learned my addr 172.21.15.6:0/1444535794 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f37a1bec700 1 --2- 172.21.15.6:0/1444535794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0fcf80 0x7f379c10dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f37a23ed700 1 -- 172.21.15.6:0/1444535794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0fcf80 msgr2=0x7f379c10dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f37a23ed700 1 --2- 172.21.15.6:0/1444535794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0fcf80 0x7f379c10dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f37a23ed700 1 -- 172.21.15.6:0/1444535794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798005040 con 0x7f379c0fbcd0 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f37a23ed700 1 --2- 172.21.15.6:0/1444535794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c0fbcd0 0x7f379c10daa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3798009150 tx=0x7f3798009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f3792ffd700 1 -- 172.21.15.6:0/1444535794 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f379800e040 con 0x7f379c0fbcd0 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f37a4651700 1 -- 172.21.15.6:0/1444535794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c10f010 con 0x7f379c0fbcd0 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f37a4651700 1 -- 172.21.15.6:0/1444535794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c10f550 con 0x7f379c0fbcd0 2024-03-22T06:24:17.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.620+0000 7f3792ffd700 1 -- 172.21.15.6:0/1444535794 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f379800bbf0 con 0x7f379c0fbcd0 2024-03-22T06:24:17.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.621+0000 7f3792ffd700 1 -- 172.21.15.6:0/1444535794 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f379801c630 con 0x7f379c0fbcd0 2024-03-22T06:24:17.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.621+0000 7f37a4651700 1 -- 172.21.15.6:0/1444535794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c101b10 con 0x7f379c0fbcd0 2024-03-22T06:24:17.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.622+0000 7f3792ffd700 1 -- 172.21.15.6:0/1444535794 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f379800bd60 con 0x7f379c0fbcd0 2024-03-22T06:24:17.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.623+0000 7f3792ffd700 1 --2- 172.21.15.6:0/1444535794 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f378806e850 0x7f3788070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:17.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.623+0000 7f37a1bec700 1 --2- 172.21.15.6:0/1444535794 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f378806e850 0x7f3788070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:17.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.623+0000 7f3792ffd700 1 -- 172.21.15.6:0/1444535794 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f379808f990 con 0x7f379c0fbcd0 2024-03-22T06:24:17.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.623+0000 7f37a1bec700 1 --2- 172.21.15.6:0/1444535794 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f378806e850 0x7f3788070d00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f378c004f40 tx=0x7f378c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:17.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.627+0000 7f3792ffd700 1 -- 172.21.15.6:0/1444535794 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f379805c270 con 0x7f379c0fbcd0 2024-03-22T06:24:17.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:17 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3598685384' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:17.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:17 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3539916493' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:24:17.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:17 smithi117 ceph-mon[82126]: pgmap v269: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:17.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:17 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3598685384' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:17.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:17 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3539916493' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:24:17.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:17 smithi006 ceph-mon[94482]: pgmap v269: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:17.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.987+0000 7f37a4651700 1 -- 172.21.15.6:0/1444535794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f379c05f5f0 con 0x7f379c0fbcd0 2024-03-22T06:24:17.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.988+0000 7f3792ffd700 1 -- 172.21.15.6:0/1444535794 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+155 (secure 0 0 0) 0x7f3798013e60 con 0x7f379c0fbcd0 2024-03-22T06:24:17.988 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:24:17.989 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:24:17.989 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in error state 2024-03-22T06:24:17.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.990+0000 7f37a4651700 1 -- 172.21.15.6:0/1444535794 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f378806e850 msgr2=0x7f3788070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:17.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.990+0000 7f37a4651700 1 --2- 172.21.15.6:0/1444535794 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f378806e850 0x7f3788070d00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f378c004f40 tx=0x7f378c00b040 comp rx=0 tx=0).stop 2024-03-22T06:24:17.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.990+0000 7f37a4651700 1 -- 172.21.15.6:0/1444535794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c0fbcd0 msgr2=0x7f379c10daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:17.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.990+0000 7f37a4651700 1 --2- 172.21.15.6:0/1444535794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c0fbcd0 0x7f379c10daa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3798009150 tx=0x7f3798009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:17.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.990+0000 7f37a4651700 1 -- 172.21.15.6:0/1444535794 shutdown_connections 2024-03-22T06:24:17.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.990+0000 7f37a4651700 1 --2- 172.21.15.6:0/1444535794 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f378806e850 0x7f3788070d00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.990+0000 7f37a4651700 1 --2- 172.21.15.6:0/1444535794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c0fbcd0 0x7f379c10daa0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.990+0000 7f37a4651700 1 --2- 172.21.15.6:0/1444535794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f379c0fcf80 0x7f379c10dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:17.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.990+0000 7f37a4651700 1 -- 172.21.15.6:0/1444535794 >> 172.21.15.6:0/1444535794 conn(0x7f379c0f7430 msgr2=0x7f379c065b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:17.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.990+0000 7f37a4651700 1 -- 172.21.15.6:0/1444535794 shutdown_connections 2024-03-22T06:24:17.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:17.990+0000 7f37a4651700 1 -- 172.21.15.6:0/1444535794 wait complete. 2024-03-22T06:24:19.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:18 smithi117 ceph-mon[82126]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:19.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:18 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:19.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:18 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:19.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:18 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1444535794' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:24:19.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:18 smithi006 ceph-mon[94482]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:18 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:18 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:19.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:18 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1444535794' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:24:20.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:19 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:20.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:19 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:20.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:19 smithi117 ceph-mon[82126]: pgmap v270: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:20.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:19 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:20.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:19 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:20.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:19 smithi006 ceph-mon[94482]: pgmap v270: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:21.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:21 smithi006 ceph-mon[94482]: pgmap v271: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:21.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:21 smithi117 ceph-mon[82126]: pgmap v271: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:22.531 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:22 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:24:22.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:22 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:24:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: pgmap v272: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: pgmap v273: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:24:24.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:23 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:24.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:24.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:24.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:24.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: pgmap v272: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: pgmap v273: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:24.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:23 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:26.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:25 smithi117 ceph-mon[82126]: Upgrade: Updating alertmanager.smithi006 2024-03-22T06:24:26.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:25 smithi117 ceph-mon[82126]: Deploying daemon alertmanager.smithi006 on smithi006 2024-03-22T06:24:26.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:25 smithi117 ceph-mon[82126]: pgmap v274: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:26.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:25 smithi006 ceph-mon[94482]: Upgrade: Updating alertmanager.smithi006 2024-03-22T06:24:26.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:25 smithi006 ceph-mon[94482]: Deploying daemon alertmanager.smithi006 on smithi006 2024-03-22T06:24:26.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:25 smithi006 ceph-mon[94482]: pgmap v274: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:27.913 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:27 smithi006 ceph-mon[94482]: pgmap v275: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:28.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:27 smithi117 ceph-mon[82126]: pgmap v275: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:30.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:29 smithi117 ceph-mon[82126]: pgmap v276: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:30.028 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:29 smithi006 ceph-mon[94482]: pgmap v276: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:31.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:31 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:31.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:31 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:31.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:31 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:31.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:31 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:31.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:31 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:31.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:31 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:32.718 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:32 smithi006 ceph-mon[94482]: pgmap v277: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:32.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:32 smithi117 ceph-mon[82126]: pgmap v277: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:34.653 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:34 smithi006 ceph-mon[94482]: pgmap v278: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:34.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:34 smithi117 ceph-mon[82126]: pgmap v278: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:35.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:35 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:35.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:35 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:35.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:35 smithi117 ceph-mon[82126]: pgmap v279: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:35.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:35 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:35.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:35 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:35.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:35 smithi006 ceph-mon[94482]: pgmap v279: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:37.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:36 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:37.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:36 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:37.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:36 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:37.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:36 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:38.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:37 smithi117 ceph-mon[82126]: pgmap v280: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:38.017 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:37 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:24:38.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:37 smithi006 ceph-mon[94482]: pgmap v280: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:38.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:37 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:24:40.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: pgmap v281: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:40.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:40.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:40.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:40.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:40.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:40.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:24:40.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:40.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.006 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:39 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: pgmap v281: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:40.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:40.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:40.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:40.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.033 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.034 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:39 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:40.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:40 smithi006 ceph-mon[94482]: pgmap v282: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:40.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:40 smithi006 ceph-mon[94482]: pgmap v283: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:40.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:40 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:24:41.003 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:40 smithi117 ceph-mon[82126]: pgmap v282: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:41.004 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:40 smithi117 ceph-mon[82126]: pgmap v283: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:41.004 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:40 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T06:24:41.986 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:41 smithi006 ceph-mon[94482]: Upgrade: Updating grafana.smithi006 2024-03-22T06:24:41.986 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:41 smithi006 ceph-mon[94482]: Deploying daemon grafana.smithi006 on smithi006 2024-03-22T06:24:42.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:41 smithi117 ceph-mon[82126]: Upgrade: Updating grafana.smithi006 2024-03-22T06:24:42.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:41 smithi117 ceph-mon[82126]: Deploying daemon grafana.smithi006 on smithi006 2024-03-22T06:24:42.949 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:42 smithi006 ceph-mon[94482]: pgmap v284: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:43.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:42 smithi117 ceph-mon[82126]: pgmap v284: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:44.968 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:44 smithi006 ceph-mon[94482]: pgmap v285: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:45.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:44 smithi117 ceph-mon[82126]: pgmap v285: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:46.336 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:46 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:46.336 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:46 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:46.336 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:46 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:46.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:46 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:46.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:46 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:46.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:46 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:47.479 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:47 smithi006 ceph-mon[94482]: pgmap v286: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:47.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:47 smithi117 ceph-mon[82126]: pgmap v286: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.110+0000 7f7725fca700 1 -- 172.21.15.6:0/3273542719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77200662e0 msgr2=0x7f77200ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.110+0000 7f7725fca700 1 --2- 172.21.15.6:0/3273542719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77200662e0 0x7f77200ff9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7710004940 tx=0x7f7710009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.110+0000 7f7725fca700 1 -- 172.21.15.6:0/3273542719 shutdown_connections 2024-03-22T06:24:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.110+0000 7f7725fca700 1 --2- 172.21.15.6:0/3273542719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77200fff00 0x7f7720102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.110+0000 7f7725fca700 1 --2- 172.21.15.6:0/3273542719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77200662e0 0x7f77200ff9c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.110+0000 7f7725fca700 1 -- 172.21.15.6:0/3273542719 >> 172.21.15.6:0/3273542719 conn(0x7f77200f7430 msgr2=0x7f77200f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.110+0000 7f7725fca700 1 -- 172.21.15.6:0/3273542719 shutdown_connections 2024-03-22T06:24:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.110+0000 7f7725fca700 1 -- 172.21.15.6:0/3273542719 wait complete. 2024-03-22T06:24:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f7725fca700 1 Processor -- start 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f7725fca700 1 -- start start 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f7725fca700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77200662e0 0x7f772010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f7725fca700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77200fff00 0x7f772010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f7725fca700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772010e570 con 0x7f77200662e0 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f7725fca700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772010edc0 con 0x7f77200fff00 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f771f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77200662e0 0x7f772010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f771f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77200662e0 0x7f772010da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44114/0 (socket says 172.21.15.6:44114) 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f771f7fe700 1 -- 172.21.15.6:0/3066768047 learned_addr learned my addr 172.21.15.6:0/3066768047 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f771effd700 1 --2- 172.21.15.6:0/3066768047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77200fff00 0x7f772010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f771f7fe700 1 -- 172.21.15.6:0/3066768047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77200fff00 msgr2=0x7f772010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f771f7fe700 1 --2- 172.21.15.6:0/3066768047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77200fff00 0x7f772010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.111+0000 7f771f7fe700 1 -- 172.21.15.6:0/3066768047 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7710005040 con 0x7f77200662e0 2024-03-22T06:24:48.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.112+0000 7f771f7fe700 1 --2- 172.21.15.6:0/3066768047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77200662e0 0x7f772010da30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f771000b3b0 tx=0x7f7710009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:48.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.112+0000 7f7707fff700 1 -- 172.21.15.6:0/3066768047 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f771000e040 con 0x7f77200662e0 2024-03-22T06:24:48.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.112+0000 7f7725fca700 1 -- 172.21.15.6:0/3066768047 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772010f040 con 0x7f77200662e0 2024-03-22T06:24:48.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.112+0000 7f7707fff700 1 -- 172.21.15.6:0/3066768047 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f771000bbf0 con 0x7f77200662e0 2024-03-22T06:24:48.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.112+0000 7f7707fff700 1 -- 172.21.15.6:0/3066768047 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f771001c630 con 0x7f77200662e0 2024-03-22T06:24:48.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.112+0000 7f7725fca700 1 -- 172.21.15.6:0/3066768047 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772010f580 con 0x7f77200662e0 2024-03-22T06:24:48.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.113+0000 7f7725fca700 1 -- 172.21.15.6:0/3066768047 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772005f5f0 con 0x7f77200662e0 2024-03-22T06:24:48.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.114+0000 7f7707fff700 1 -- 172.21.15.6:0/3066768047 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f771000bd60 con 0x7f77200662e0 2024-03-22T06:24:48.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.114+0000 7f7707fff700 1 --2- 172.21.15.6:0/3066768047 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f770806e840 0x7f7708070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:48.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.114+0000 7f7707fff700 1 -- 172.21.15.6:0/3066768047 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f771008f160 con 0x7f77200662e0 2024-03-22T06:24:48.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.114+0000 7f771effd700 1 --2- 172.21.15.6:0/3066768047 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f770806e840 0x7f7708070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:48.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.114+0000 7f771effd700 1 --2- 172.21.15.6:0/3066768047 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f770806e840 0x7f7708070cf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7714004f40 tx=0x7f771400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:48.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.123+0000 7f7707fff700 1 -- 172.21.15.6:0/3066768047 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f771005bae0 con 0x7f77200662e0 2024-03-22T06:24:48.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.375+0000 7f7725fca700 1 -- 172.21.15.6:0/3066768047 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f772010f970 con 0x7f770806e840 2024-03-22T06:24:48.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.377+0000 7f7707fff700 1 -- 172.21.15.6:0/3066768047 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f772010f970 con 0x7f770806e840 2024-03-22T06:24:48.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.378+0000 7f7725fca700 1 -- 172.21.15.6:0/3066768047 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f770806e840 msgr2=0x7f7708070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:48.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.378+0000 7f7725fca700 1 --2- 172.21.15.6:0/3066768047 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f770806e840 0x7f7708070cf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7714004f40 tx=0x7f771400b040 comp rx=0 tx=0).stop 2024-03-22T06:24:48.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.379+0000 7f7725fca700 1 -- 172.21.15.6:0/3066768047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77200662e0 msgr2=0x7f772010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:48.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.379+0000 7f7725fca700 1 --2- 172.21.15.6:0/3066768047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77200662e0 0x7f772010da30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f771000b3b0 tx=0x7f7710009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:48.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.379+0000 7f7725fca700 1 -- 172.21.15.6:0/3066768047 shutdown_connections 2024-03-22T06:24:48.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.379+0000 7f7725fca700 1 --2- 172.21.15.6:0/3066768047 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f770806e840 0x7f7708070cf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.379+0000 7f7725fca700 1 --2- 172.21.15.6:0/3066768047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77200662e0 0x7f772010da30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.379+0000 7f7725fca700 1 --2- 172.21.15.6:0/3066768047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77200fff00 0x7f772010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.379+0000 7f7725fca700 1 -- 172.21.15.6:0/3066768047 >> 172.21.15.6:0/3066768047 conn(0x7f77200f7430 msgr2=0x7f77200f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:48.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.379+0000 7f7725fca700 1 -- 172.21.15.6:0/3066768047 shutdown_connections 2024-03-22T06:24:48.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.379+0000 7f7725fca700 1 -- 172.21.15.6:0/3066768047 wait complete. 2024-03-22T06:24:48.389 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:24:48.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.503+0000 7f7d1fa94700 1 -- 172.21.15.6:0/1444619083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d180fff10 msgr2=0x7f7d18102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.503+0000 7f7d1fa94700 1 --2- 172.21.15.6:0/1444619083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d180fff10 0x7f7d18102330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7d08004940 tx=0x7f7d08009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.503+0000 7f7d1fa94700 1 -- 172.21.15.6:0/1444619083 shutdown_connections 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.503+0000 7f7d1fa94700 1 --2- 172.21.15.6:0/1444619083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d180fff10 0x7f7d18102330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.503+0000 7f7d1fa94700 1 --2- 172.21.15.6:0/1444619083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d18066b70 0x7f7d180ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.503+0000 7f7d1fa94700 1 -- 172.21.15.6:0/1444619083 >> 172.21.15.6:0/1444619083 conn(0x7f7d180f7440 msgr2=0x7f7d180f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.504+0000 7f7d1fa94700 1 -- 172.21.15.6:0/1444619083 shutdown_connections 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.504+0000 7f7d1fa94700 1 -- 172.21.15.6:0/1444619083 wait complete. 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.504+0000 7f7d1fa94700 1 Processor -- start 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.504+0000 7f7d1fa94700 1 -- start start 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.504+0000 7f7d1fa94700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d18066b70 0x7f7d1810da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.504+0000 7f7d1fa94700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d180fff10 0x7f7d1810dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.504+0000 7f7d1fa94700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1810e5d0 con 0x7f7d180fff10 2024-03-22T06:24:48.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.504+0000 7f7d1fa94700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1810ee20 con 0x7f7d18066b70 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.504+0000 7f7d1d02f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d180fff10 0x7f7d1810dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.504+0000 7f7d1d02f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d180fff10 0x7f7d1810dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44136/0 (socket says 172.21.15.6:44136) 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.505+0000 7f7d1d02f700 1 -- 172.21.15.6:0/2417371914 learned_addr learned my addr 172.21.15.6:0/2417371914 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.505+0000 7f7d1d830700 1 --2- 172.21.15.6:0/2417371914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d18066b70 0x7f7d1810da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.505+0000 7f7d1d02f700 1 -- 172.21.15.6:0/2417371914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d18066b70 msgr2=0x7f7d1810da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.505+0000 7f7d1d02f700 1 --2- 172.21.15.6:0/2417371914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d18066b70 0x7f7d1810da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.505+0000 7f7d1d02f700 1 -- 172.21.15.6:0/2417371914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d08005040 con 0x7f7d180fff10 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.505+0000 7f7d1d02f700 1 --2- 172.21.15.6:0/2417371914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d180fff10 0x7f7d1810dfd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009150 tx=0x7f7d08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.505+0000 7f7d0e7fc700 1 -- 172.21.15.6:0/2417371914 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d0800e040 con 0x7f7d180fff10 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.505+0000 7f7d1fa94700 1 -- 172.21.15.6:0/2417371914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1810f0a0 con 0x7f7d180fff10 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.505+0000 7f7d1fa94700 1 -- 172.21.15.6:0/2417371914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1810f5e0 con 0x7f7d180fff10 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.505+0000 7f7d0e7fc700 1 -- 172.21.15.6:0/2417371914 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d08004290 con 0x7f7d180fff10 2024-03-22T06:24:48.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.505+0000 7f7d0e7fc700 1 -- 172.21.15.6:0/2417371914 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d0801c630 con 0x7f7d180fff10 2024-03-22T06:24:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.506+0000 7f7d1fa94700 1 -- 172.21.15.6:0/2417371914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1805f5f0 con 0x7f7d180fff10 2024-03-22T06:24:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.507+0000 7f7d0e7fc700 1 -- 172.21.15.6:0/2417371914 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f7d0800bbf0 con 0x7f7d180fff10 2024-03-22T06:24:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.508+0000 7f7d0e7fc700 1 --2- 172.21.15.6:0/2417371914 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7d0406e840 0x7f7d04070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.508+0000 7f7d0e7fc700 1 -- 172.21.15.6:0/2417371914 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f7d0808f090 con 0x7f7d180fff10 2024-03-22T06:24:48.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.508+0000 7f7d1d830700 1 --2- 172.21.15.6:0/2417371914 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7d0406e840 0x7f7d04070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:48.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.508+0000 7f7d1d830700 1 --2- 172.21.15.6:0/2417371914 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7d0406e840 0x7f7d04070cf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d180668b0 tx=0x7f7d1400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:48.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.516+0000 7f7d0e7fc700 1 -- 172.21.15.6:0/2417371914 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f7d0805bab0 con 0x7f7d180fff10 2024-03-22T06:24:48.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.763+0000 7f7d1fa94700 1 -- 172.21.15.6:0/2417371914 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d1810f9c0 con 0x7f7d0406e840 2024-03-22T06:24:48.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.765+0000 7f7d0e7fc700 1 -- 172.21.15.6:0/2417371914 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f7d1810f9c0 con 0x7f7d0406e840 2024-03-22T06:24:48.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.767+0000 7f7d1fa94700 1 -- 172.21.15.6:0/2417371914 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7d0406e840 msgr2=0x7f7d04070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:48.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.767+0000 7f7d1fa94700 1 --2- 172.21.15.6:0/2417371914 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7d0406e840 0x7f7d04070cf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d180668b0 tx=0x7f7d1400b040 comp rx=0 tx=0).stop 2024-03-22T06:24:48.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.767+0000 7f7d1fa94700 1 -- 172.21.15.6:0/2417371914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d180fff10 msgr2=0x7f7d1810dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:48.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.767+0000 7f7d1fa94700 1 --2- 172.21.15.6:0/2417371914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d180fff10 0x7f7d1810dfd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009150 tx=0x7f7d08009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:48.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.767+0000 7f7d1fa94700 1 -- 172.21.15.6:0/2417371914 shutdown_connections 2024-03-22T06:24:48.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.767+0000 7f7d1fa94700 1 --2- 172.21.15.6:0/2417371914 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f7d0406e840 0x7f7d04070cf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.767+0000 7f7d1fa94700 1 --2- 172.21.15.6:0/2417371914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d18066b70 0x7f7d1810da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.767+0000 7f7d1fa94700 1 --2- 172.21.15.6:0/2417371914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d180fff10 0x7f7d1810dfd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.767+0000 7f7d1fa94700 1 -- 172.21.15.6:0/2417371914 >> 172.21.15.6:0/2417371914 conn(0x7f7d180f7440 msgr2=0x7f7d180f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:48.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.768+0000 7f7d1fa94700 1 -- 172.21.15.6:0/2417371914 shutdown_connections 2024-03-22T06:24:48.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.768+0000 7f7d1fa94700 1 -- 172.21.15.6:0/2417371914 wait complete. 2024-03-22T06:24:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.881+0000 7f155ce9c700 1 -- 172.21.15.6:0/2539640024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15580fe400 msgr2=0x7f15580fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.881+0000 7f155ce9c700 1 --2- 172.21.15.6:0/2539640024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15580fe400 0x7f15580fe810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1548004940 tx=0x7f1548009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.881+0000 7f155ce9c700 1 -- 172.21.15.6:0/2539640024 shutdown_connections 2024-03-22T06:24:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.881+0000 7f155ce9c700 1 --2- 172.21.15.6:0/2539640024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15580fed50 0x7f15580fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.881+0000 7f155ce9c700 1 --2- 172.21.15.6:0/2539640024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15580fe400 0x7f15580fe810 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.881+0000 7f155ce9c700 1 -- 172.21.15.6:0/2539640024 >> 172.21.15.6:0/2539640024 conn(0x7f15580f6ba0 msgr2=0x7f15580f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.882+0000 7f155ce9c700 1 -- 172.21.15.6:0/2539640024 shutdown_connections 2024-03-22T06:24:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.882+0000 7f155ce9c700 1 -- 172.21.15.6:0/2539640024 wait complete. 2024-03-22T06:24:48.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.882+0000 7f155ce9c700 1 Processor -- start 2024-03-22T06:24:48.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.882+0000 7f155ce9c700 1 -- start start 2024-03-22T06:24:48.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.882+0000 7f155ce9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15580fe400 0x7f155810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:48.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.882+0000 7f155ce9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15580fed50 0x7f155810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:48.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.882+0000 7f155ce9c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155810e4c0 con 0x7f15580fed50 2024-03-22T06:24:48.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.882+0000 7f155ce9c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155810ed10 con 0x7f15580fe400 2024-03-22T06:24:48.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.883+0000 7f1555d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15580fed50 0x7f155810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:48.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.883+0000 7f1555d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15580fed50 0x7f155810dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44150/0 (socket says 172.21.15.6:44150) 2024-03-22T06:24:48.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.883+0000 7f1555d9b700 1 -- 172.21.15.6:0/2849938336 learned_addr learned my addr 172.21.15.6:0/2849938336 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:48.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.883+0000 7f155659c700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15580fe400 0x7f155810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:48.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.883+0000 7f1555d9b700 1 -- 172.21.15.6:0/2849938336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15580fe400 msgr2=0x7f155810d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:48.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.883+0000 7f1555d9b700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15580fe400 0x7f155810d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:48.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.883+0000 7f1555d9b700 1 -- 172.21.15.6:0/2849938336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1548005040 con 0x7f15580fed50 2024-03-22T06:24:48.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.883+0000 7f155659c700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15580fe400 0x7f155810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:24:48.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.883+0000 7f1555d9b700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15580fed50 0x7f155810dec0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f154c00eb40 tx=0x7f154c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:48.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.883+0000 7f1546ffd700 1 -- 172.21.15.6:0/2849938336 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f154c007cc0 con 0x7f15580fed50 2024-03-22T06:24:48.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.884+0000 7f155ce9c700 1 -- 172.21.15.6:0/2849938336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155810ef90 con 0x7f15580fed50 2024-03-22T06:24:48.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.884+0000 7f1546ffd700 1 -- 172.21.15.6:0/2849938336 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f154c00f040 con 0x7f15580fed50 2024-03-22T06:24:48.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.884+0000 7f155ce9c700 1 -- 172.21.15.6:0/2849938336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155810f4d0 con 0x7f15580fed50 2024-03-22T06:24:48.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.884+0000 7f1546ffd700 1 -- 172.21.15.6:0/2849938336 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f154c009620 con 0x7f15580fed50 2024-03-22T06:24:48.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.885+0000 7f155ce9c700 1 -- 172.21.15.6:0/2849938336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155805f5f0 con 0x7f15580fed50 2024-03-22T06:24:48.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.885+0000 7f1546ffd700 1 -- 172.21.15.6:0/2849938336 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f154c009780 con 0x7f15580fed50 2024-03-22T06:24:48.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.886+0000 7f1546ffd700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f154006e840 0x7f1540070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:48.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.886+0000 7f1546ffd700 1 -- 172.21.15.6:0/2849938336 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f154c08e250 con 0x7f15580fed50 2024-03-22T06:24:48.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.886+0000 7f155659c700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f154006e840 0x7f1540070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:48.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.887+0000 7f155659c700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f154006e840 0x7f1540070cf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1548009b30 tx=0x7f1548009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:48.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:48.895+0000 7f1546ffd700 1 -- 172.21.15.6:0/2849938336 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f154c05ac00 con 0x7f15580fed50 2024-03-22T06:24:49.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.113+0000 7f155ce9c700 1 -- 172.21.15.6:0/2849938336 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f155810f800 con 0x7f154006e840 2024-03-22T06:24:49.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.123+0000 7f1546ffd700 1 -- 172.21.15.6:0/2849938336 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f155810f800 con 0x7f154006e840 2024-03-22T06:24:49.123 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:24:49.123 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (19s) 14s ago 17m 16.5M - 0.25.0 c8568f914cd2 5fd2ca09ada6 2024-03-22T06:24:49.123 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (6m) 14s ago 18m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:24:49.123 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (6m) 50s ago 17m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 starting - - - - 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (2m) 14s ago 12m 82.1M - 19.0.0-2299-gea01217f aa461e7c5e7f 3e3319f646e6 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (118s) 14s ago 12m 13.2M - 19.0.0-2299-gea01217f aa461e7c5e7f 09ee1e271adb 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (95s) 50s ago 12m 15.0M - 19.0.0-2299-gea01217f aa461e7c5e7f a9149dea3f8e 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (69s) 50s ago 12m 12.7M - 19.0.0-2299-gea01217f aa461e7c5e7f 0abdc3f226cc 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (8m) 14s ago 19m 513M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (7m) 50s ago 16m 452M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (7m) 14s ago 19m 56.2M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (6m) 50s ago 16m 44.7M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (57s) 14s ago 18m 8782k - 1.5.0 0da6a335fe13 7eb15a15b4c3 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (53s) 50s ago 17m 3275k - 1.5.0 0da6a335fe13 df7460dcdb08 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (5m) 14s ago 16m 155M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (5m) 14s ago 15m 138M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (4m) 14s ago 15m 106M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 3ec1f219c19c 2024-03-22T06:24:49.124 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (3m) 50s ago 14m 114M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 9cba39b6f32c 2024-03-22T06:24:49.125 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (3m) 50s ago 14m 104M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1950fc21bd2e 2024-03-22T06:24:49.125 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (2m) 50s ago 13m 108M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1a6a0ee2a228 2024-03-22T06:24:49.125 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (36s) 14s ago 17m 53.4M - 2.43.0 a07b618ecd1d 831f4be843c5 2024-03-22T06:24:49.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.125+0000 7f155ce9c700 1 -- 172.21.15.6:0/2849938336 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f154006e840 msgr2=0x7f1540070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:49.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.125+0000 7f155ce9c700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f154006e840 0x7f1540070cf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1548009b30 tx=0x7f1548009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:49.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.125+0000 7f155ce9c700 1 -- 172.21.15.6:0/2849938336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15580fed50 msgr2=0x7f155810dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:49.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.125+0000 7f155ce9c700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15580fed50 0x7f155810dec0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f154c00eb40 tx=0x7f154c00b040 comp rx=0 tx=0).stop 2024-03-22T06:24:49.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.125+0000 7f155ce9c700 1 -- 172.21.15.6:0/2849938336 shutdown_connections 2024-03-22T06:24:49.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.125+0000 7f155ce9c700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f154006e840 0x7f1540070cf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.125+0000 7f155ce9c700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15580fe400 0x7f155810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.125+0000 7f155ce9c700 1 --2- 172.21.15.6:0/2849938336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15580fed50 0x7f155810dec0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.125+0000 7f155ce9c700 1 -- 172.21.15.6:0/2849938336 >> 172.21.15.6:0/2849938336 conn(0x7f15580f6ba0 msgr2=0x7f15580f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:49.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.125+0000 7f155ce9c700 1 -- 172.21.15.6:0/2849938336 shutdown_connections 2024-03-22T06:24:49.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.125+0000 7f155ce9c700 1 -- 172.21.15.6:0/2849938336 wait complete. 2024-03-22T06:24:49.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.240+0000 7f87be542700 1 -- 172.21.15.6:0/453722721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b80fffa0 msgr2=0x7f87b8102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:49.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.240+0000 7f87be542700 1 --2- 172.21.15.6:0/453722721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b80fffa0 0x7f87b8102380 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f87ac004940 tx=0x7f87ac009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:49.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.240+0000 7f87be542700 1 -- 172.21.15.6:0/453722721 shutdown_connections 2024-03-22T06:24:49.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.240+0000 7f87be542700 1 --2- 172.21.15.6:0/453722721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b80fffa0 0x7f87b8102380 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.240+0000 7f87be542700 1 --2- 172.21.15.6:0/453722721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87b8066240 0x7f87b80ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.240+0000 7f87be542700 1 -- 172.21.15.6:0/453722721 >> 172.21.15.6:0/453722721 conn(0x7f87b80f7430 msgr2=0x7f87b80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:49.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.240+0000 7f87be542700 1 -- 172.21.15.6:0/453722721 shutdown_connections 2024-03-22T06:24:49.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.240+0000 7f87be542700 1 -- 172.21.15.6:0/453722721 wait complete. 2024-03-22T06:24:49.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.241+0000 7f87be542700 1 Processor -- start 2024-03-22T06:24:49.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.241+0000 7f87be542700 1 -- start start 2024-03-22T06:24:49.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87be542700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87b8066240 0x7f87b810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87be542700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b80fffa0 0x7f87b810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87be542700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b810e5a0 con 0x7f87b80fffa0 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87be542700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b810ed50 con 0x7f87b8066240 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87b77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b80fffa0 0x7f87b810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87b77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b80fffa0 0x7f87b810dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44178/0 (socket says 172.21.15.6:44178) 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87b77fe700 1 -- 172.21.15.6:0/1071969972 learned_addr learned my addr 172.21.15.6:0/1071969972 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87b7fff700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87b8066240 0x7f87b810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87b77fe700 1 -- 172.21.15.6:0/1071969972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87b8066240 msgr2=0x7f87b810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87b77fe700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87b8066240 0x7f87b810da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87b77fe700 1 -- 172.21.15.6:0/1071969972 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac005040 con 0x7f87b80fffa0 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87b7fff700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87b8066240 0x7f87b810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87b77fe700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b80fffa0 0x7f87b810dfa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00b3b0 tx=0x7f87ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.242+0000 7f87b4ff9700 1 -- 172.21.15.6:0/1071969972 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87ac00e040 con 0x7f87b80fffa0 2024-03-22T06:24:49.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.243+0000 7f87be542700 1 -- 172.21.15.6:0/1071969972 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b810efd0 con 0x7f87b80fffa0 2024-03-22T06:24:49.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.243+0000 7f87be542700 1 -- 172.21.15.6:0/1071969972 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b810f510 con 0x7f87b80fffa0 2024-03-22T06:24:49.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.243+0000 7f87b4ff9700 1 -- 172.21.15.6:0/1071969972 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87ac00bcc0 con 0x7f87b80fffa0 2024-03-22T06:24:49.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.243+0000 7f87b4ff9700 1 -- 172.21.15.6:0/1071969972 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87ac026620 con 0x7f87b80fffa0 2024-03-22T06:24:49.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.244+0000 7f87be542700 1 -- 172.21.15.6:0/1071969972 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b805f5f0 con 0x7f87b80fffa0 2024-03-22T06:24:49.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.245+0000 7f87b4ff9700 1 -- 172.21.15.6:0/1071969972 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f87ac0139f0 con 0x7f87b80fffa0 2024-03-22T06:24:49.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.245+0000 7f87b4ff9700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87a006e840 0x7f87a0070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:49.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.245+0000 7f87b4ff9700 1 -- 172.21.15.6:0/1071969972 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f87ac08fab0 con 0x7f87b80fffa0 2024-03-22T06:24:49.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.246+0000 7f87b7fff700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87a006e840 0x7f87a0070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:49.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.246+0000 7f87b7fff700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87a006e840 0x7f87a0070cf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f87b8065f80 tx=0x7f87a8007ec0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:49.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.250+0000 7f87b4ff9700 1 -- 172.21.15.6:0/1071969972 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f87ac05c4e0 con 0x7f87b80fffa0 2024-03-22T06:24:49.498 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:49 smithi006 ceph-mon[94482]: pgmap v287: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:49.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:49 smithi117 ceph-mon[82126]: pgmap v287: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:49.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.572+0000 7f87be542700 1 -- 172.21.15.6:0/1071969972 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f87b804bed0 con 0x7f87b80fffa0 2024-03-22T06:24:49.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.573+0000 7f87b4ff9700 1 -- 172.21.15.6:0/1071969972 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f87ac05c070 con 0x7f87b80fffa0 2024-03-22T06:24:49.573 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:24:49.573 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:24:49.574 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:24:49.574 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:24:49.574 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:24:49.574 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:24:49.574 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:24:49.574 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:24:49.574 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 6 2024-03-22T06:24:49.574 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:24:49.574 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:24:49.574 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 4 2024-03-22T06:24:49.574 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:24:49.575 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:24:49.575 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 14 2024-03-22T06:24:49.575 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:24:49.575 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:24:49.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.575+0000 7f87be542700 1 -- 172.21.15.6:0/1071969972 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87a006e840 msgr2=0x7f87a0070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:49.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.575+0000 7f87be542700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87a006e840 0x7f87a0070cf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f87b8065f80 tx=0x7f87a8007ec0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.575+0000 7f87be542700 1 -- 172.21.15.6:0/1071969972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b80fffa0 msgr2=0x7f87b810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:49.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.575+0000 7f87be542700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b80fffa0 0x7f87b810dfa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00b3b0 tx=0x7f87ac009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:49.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.575+0000 7f87be542700 1 -- 172.21.15.6:0/1071969972 shutdown_connections 2024-03-22T06:24:49.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.575+0000 7f87be542700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f87a006e840 0x7f87a0070cf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.575+0000 7f87be542700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87b8066240 0x7f87b810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.575+0000 7f87be542700 1 --2- 172.21.15.6:0/1071969972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b80fffa0 0x7f87b810dfa0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.575+0000 7f87be542700 1 -- 172.21.15.6:0/1071969972 >> 172.21.15.6:0/1071969972 conn(0x7f87b80f7430 msgr2=0x7f87b80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:49.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.575+0000 7f87be542700 1 -- 172.21.15.6:0/1071969972 shutdown_connections 2024-03-22T06:24:49.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.575+0000 7f87be542700 1 -- 172.21.15.6:0/1071969972 wait complete. 2024-03-22T06:24:49.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.685+0000 7fd6f8721700 1 -- 172.21.15.6:0/327255857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f00fcf80 msgr2=0x7fd6f00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:49.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.685+0000 7fd6f8721700 1 --2- 172.21.15.6:0/327255857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f00fcf80 0x7fd6f00fd3f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0004940 tx=0x7fd6e0009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:49.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.685+0000 7fd6f8721700 1 -- 172.21.15.6:0/327255857 shutdown_connections 2024-03-22T06:24:49.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.685+0000 7fd6f8721700 1 --2- 172.21.15.6:0/327255857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f00fcf80 0x7fd6f00fd3f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.685+0000 7fd6f8721700 1 --2- 172.21.15.6:0/327255857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f00fbcd0 0x7fd6f00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.685+0000 7fd6f8721700 1 -- 172.21.15.6:0/327255857 >> 172.21.15.6:0/327255857 conn(0x7fd6f00f7430 msgr2=0x7fd6f00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:49.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.685+0000 7fd6f8721700 1 -- 172.21.15.6:0/327255857 shutdown_connections 2024-03-22T06:24:49.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.685+0000 7fd6f8721700 1 -- 172.21.15.6:0/327255857 wait complete. 2024-03-22T06:24:49.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.686+0000 7fd6f8721700 1 Processor -- start 2024-03-22T06:24:49.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.686+0000 7fd6f8721700 1 -- start start 2024-03-22T06:24:49.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.686+0000 7fd6f8721700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f00fbcd0 0x7fd6f00ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:49.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.686+0000 7fd6f8721700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f00fcf80 0x7fd6f00fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:49.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.686+0000 7fd6f8721700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00fe870 con 0x7fd6f00fcf80 2024-03-22T06:24:49.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.686+0000 7fd6f8721700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00fe9e0 con 0x7fd6f00fbcd0 2024-03-22T06:24:49.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.686+0000 7fd6f5cbc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f00fcf80 0x7fd6f00fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:49.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.686+0000 7fd6f5cbc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f00fcf80 0x7fd6f00fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44182/0 (socket says 172.21.15.6:44182) 2024-03-22T06:24:49.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.686+0000 7fd6f5cbc700 1 -- 172.21.15.6:0/3726023271 learned_addr learned my addr 172.21.15.6:0/3726023271 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:49.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.686+0000 7fd6f64bd700 1 --2- 172.21.15.6:0/3726023271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f00fbcd0 0x7fd6f00ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:49.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.686+0000 7fd6f5cbc700 1 -- 172.21.15.6:0/3726023271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f00fbcd0 msgr2=0x7fd6f00ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:49.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.687+0000 7fd6f5cbc700 1 --2- 172.21.15.6:0/3726023271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f00fbcd0 0x7fd6f00ffdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.687+0000 7fd6f5cbc700 1 -- 172.21.15.6:0/3726023271 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e0005040 con 0x7fd6f00fcf80 2024-03-22T06:24:49.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.687+0000 7fd6f5cbc700 1 --2- 172.21.15.6:0/3726023271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f00fcf80 0x7fd6f00fe2c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000b3b0 tx=0x7fd6e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:49.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.687+0000 7fd6e6ffd700 1 -- 172.21.15.6:0/3726023271 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6e000e040 con 0x7fd6f00fcf80 2024-03-22T06:24:49.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.687+0000 7fd6e6ffd700 1 -- 172.21.15.6:0/3726023271 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6e000bcc0 con 0x7fd6f00fcf80 2024-03-22T06:24:49.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.687+0000 7fd6f8721700 1 -- 172.21.15.6:0/3726023271 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f00fec60 con 0x7fd6f00fcf80 2024-03-22T06:24:49.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.687+0000 7fd6e6ffd700 1 -- 172.21.15.6:0/3726023271 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6e001c630 con 0x7fd6f00fcf80 2024-03-22T06:24:49.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.687+0000 7fd6f8721700 1 -- 172.21.15.6:0/3726023271 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f0115a40 con 0x7fd6f00fcf80 2024-03-22T06:24:49.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.688+0000 7fd6f8721700 1 -- 172.21.15.6:0/3726023271 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f0101b10 con 0x7fd6f00fcf80 2024-03-22T06:24:49.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.688+0000 7fd6e6ffd700 1 -- 172.21.15.6:0/3726023271 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fd6e0013420 con 0x7fd6f00fcf80 2024-03-22T06:24:49.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.689+0000 7fd6e6ffd700 1 --2- 172.21.15.6:0/3726023271 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd6dc06e640 0x7fd6dc070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:49.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.689+0000 7fd6e6ffd700 1 -- 172.21.15.6:0/3726023271 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fd6e008f150 con 0x7fd6f00fcf80 2024-03-22T06:24:49.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.689+0000 7fd6f64bd700 1 --2- 172.21.15.6:0/3726023271 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd6dc06e640 0x7fd6dc070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:49.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.689+0000 7fd6f64bd700 1 --2- 172.21.15.6:0/3726023271 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd6dc06e640 0x7fd6dc070af0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0041d0 tx=0x7fd6ec009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:49.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.693+0000 7fd6e6ffd700 1 -- 172.21.15.6:0/3726023271 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fd6e0057fa0 con 0x7fd6f00fcf80 2024-03-22T06:24:49.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.950+0000 7fd6f8721700 1 -- 172.21.15.6:0/3726023271 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd6f0104910 con 0x7fd6f00fcf80 2024-03-22T06:24:49.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.951+0000 7fd6e6ffd700 1 -- 172.21.15.6:0/3726023271 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1908 (secure 0 0 0) 0x7fd6e0004490 con 0x7fd6f00fcf80 2024-03-22T06:24:49.951 INFO:teuthology.orchestra.run.smithi006.stdout:e36 2024-03-22T06:24:49.951 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:24:49.951 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:24:49.951 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:24:49.951 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 35 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:23:39.065820+0000 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:24:49.952 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:24:49.953 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:24:49.953 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:24:49.953 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 95 2024-03-22T06:24:49.953 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T06:24:49.953 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:24:49.953 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:24:49.953 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=34406} 2024-03-22T06:24:49.953 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:24:49.953 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:24:49.953 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:24:49.953 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 34406 members: 34406 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:34406} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:24:49.954 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:34438} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.6:6828/1244315045,v1:172.21.15.6:6829/1244315045] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:24:49.955 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6824/1194667779,v1:172.21.15.117:6825/1194667779] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:24:49.955 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2788388155,v1:172.21.15.117:6827/2788388155] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:24:49.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.953+0000 7fd6f8721700 1 -- 172.21.15.6:0/3726023271 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd6dc06e640 msgr2=0x7fd6dc070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:49.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.953+0000 7fd6f8721700 1 --2- 172.21.15.6:0/3726023271 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd6dc06e640 0x7fd6dc070af0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0041d0 tx=0x7fd6ec009040 comp rx=0 tx=0).stop 2024-03-22T06:24:49.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.953+0000 7fd6f8721700 1 -- 172.21.15.6:0/3726023271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f00fcf80 msgr2=0x7fd6f00fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:49.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.953+0000 7fd6f8721700 1 --2- 172.21.15.6:0/3726023271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f00fcf80 0x7fd6f00fe2c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000b3b0 tx=0x7fd6e0009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:49.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.954+0000 7fd6f8721700 1 -- 172.21.15.6:0/3726023271 shutdown_connections 2024-03-22T06:24:49.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.954+0000 7fd6f8721700 1 --2- 172.21.15.6:0/3726023271 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd6dc06e640 0x7fd6dc070af0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.954+0000 7fd6f8721700 1 --2- 172.21.15.6:0/3726023271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f00fbcd0 0x7fd6f00ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.954+0000 7fd6f8721700 1 --2- 172.21.15.6:0/3726023271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6f00fcf80 0x7fd6f00fe2c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:49.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.954+0000 7fd6f8721700 1 -- 172.21.15.6:0/3726023271 >> 172.21.15.6:0/3726023271 conn(0x7fd6f00f7430 msgr2=0x7fd6f0065b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:49.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.954+0000 7fd6f8721700 1 -- 172.21.15.6:0/3726023271 shutdown_connections 2024-03-22T06:24:49.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:49.954+0000 7fd6f8721700 1 -- 172.21.15.6:0/3726023271 wait complete. 2024-03-22T06:24:49.957 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 36 2024-03-22T06:24:50.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.062+0000 7f72a744b700 1 -- 172.21.15.6:0/2035157561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a00fff00 msgr2=0x7f72a0102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:50.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.062+0000 7f72a744b700 1 --2- 172.21.15.6:0/2035157561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a00fff00 0x7f72a0102320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7290004940 tx=0x7f7290009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:50.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.062+0000 7f72a744b700 1 -- 172.21.15.6:0/2035157561 shutdown_connections 2024-03-22T06:24:50.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.062+0000 7f72a744b700 1 --2- 172.21.15.6:0/2035157561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a00fff00 0x7f72a0102320 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.062+0000 7f72a744b700 1 --2- 172.21.15.6:0/2035157561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72a0066ef0 0x7f72a00ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.062+0000 7f72a744b700 1 -- 172.21.15.6:0/2035157561 >> 172.21.15.6:0/2035157561 conn(0x7f72a00f7430 msgr2=0x7f72a00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:50.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.062+0000 7f72a744b700 1 -- 172.21.15.6:0/2035157561 shutdown_connections 2024-03-22T06:24:50.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.063+0000 7f72a744b700 1 -- 172.21.15.6:0/2035157561 wait complete. 2024-03-22T06:24:50.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.063+0000 7f72a744b700 1 Processor -- start 2024-03-22T06:24:50.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.063+0000 7f72a744b700 1 -- start start 2024-03-22T06:24:50.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.063+0000 7f72a744b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a0066ef0 0x7f72a010d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:50.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.063+0000 7f72a744b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72a00fff00 0x7f72a010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:50.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.063+0000 7f72a744b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a010e490 con 0x7f72a0066ef0 2024-03-22T06:24:50.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.063+0000 7f72a744b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a010e600 con 0x7f72a00fff00 2024-03-22T06:24:50.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.064+0000 7f72a51e7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a0066ef0 0x7f72a010d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:50.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.064+0000 7f72a51e7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a0066ef0 0x7f72a010d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44208/0 (socket says 172.21.15.6:44208) 2024-03-22T06:24:50.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.064+0000 7f72a51e7700 1 -- 172.21.15.6:0/592393755 learned_addr learned my addr 172.21.15.6:0/592393755 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:50.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.064+0000 7f72a49e6700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72a00fff00 0x7f72a010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:50.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.064+0000 7f72a51e7700 1 -- 172.21.15.6:0/592393755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72a00fff00 msgr2=0x7f72a010de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:50.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.064+0000 7f72a51e7700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72a00fff00 0x7f72a010de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.064+0000 7f72a51e7700 1 -- 172.21.15.6:0/592393755 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290005040 con 0x7f72a0066ef0 2024-03-22T06:24:50.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.064+0000 7f72a49e6700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72a00fff00 0x7f72a010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:24:50.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.064+0000 7f72a51e7700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a0066ef0 0x7f72a010d950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f729c002a50 tx=0x7f729c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:50.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.065+0000 7f7295ffb700 1 -- 172.21.15.6:0/592393755 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f729c007cc0 con 0x7f72a0066ef0 2024-03-22T06:24:50.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.065+0000 7f7295ffb700 1 -- 172.21.15.6:0/592393755 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f729c00f040 con 0x7f72a0066ef0 2024-03-22T06:24:50.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.065+0000 7f72a744b700 1 -- 172.21.15.6:0/592393755 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a0113060 con 0x7f72a0066ef0 2024-03-22T06:24:50.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.065+0000 7f7295ffb700 1 -- 172.21.15.6:0/592393755 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f729c00daf0 con 0x7f72a0066ef0 2024-03-22T06:24:50.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.065+0000 7f72a744b700 1 -- 172.21.15.6:0/592393755 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a00fb9e0 con 0x7f72a0066ef0 2024-03-22T06:24:50.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.066+0000 7f72a744b700 1 -- 172.21.15.6:0/592393755 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a004c710 con 0x7f72a0066ef0 2024-03-22T06:24:50.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.067+0000 7f7295ffb700 1 -- 172.21.15.6:0/592393755 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f729c009810 con 0x7f72a0066ef0 2024-03-22T06:24:50.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.067+0000 7f7295ffb700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f728c072c50 0x7f728c075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:50.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.068+0000 7f7295ffb700 1 -- 172.21.15.6:0/592393755 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f729c08e290 con 0x7f72a0066ef0 2024-03-22T06:24:50.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.068+0000 7f72a49e6700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f728c072c50 0x7f728c075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:50.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.068+0000 7f72a49e6700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f728c072c50 0x7f728c075100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f729000b020 tx=0x7f7290009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:50.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.074+0000 7f7295ffb700 1 -- 172.21.15.6:0/592393755 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f729c05ac10 con 0x7f72a0066ef0 2024-03-22T06:24:50.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.343+0000 7f72a744b700 1 -- 172.21.15.6:0/592393755 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72a01131f0 con 0x7f728c072c50 2024-03-22T06:24:50.344 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:50 smithi006 ceph-mon[94482]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:50.344 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:50 smithi006 ceph-mon[94482]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:50.344 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:50 smithi006 ceph-mon[94482]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:50.344 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:50 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1071969972' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:50.344 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:50 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3726023271' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:24:50.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.345+0000 7f7295ffb700 1 -- 172.21.15.6:0/592393755 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f72a01131f0 con 0x7f728c072c50 2024-03-22T06:24:50.345 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:24:50.345 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9", 2024-03-22T06:24:50.345 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": true, 2024-03-22T06:24:50.345 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T06:24:50.345 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [ 2024-03-22T06:24:50.345 INFO:teuthology.orchestra.run.smithi006.stdout: "mon", 2024-03-22T06:24:50.345 INFO:teuthology.orchestra.run.smithi006.stdout: "mds", 2024-03-22T06:24:50.346 INFO:teuthology.orchestra.run.smithi006.stdout: "osd", 2024-03-22T06:24:50.346 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-03-22T06:24:50.346 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr" 2024-03-22T06:24:50.346 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-03-22T06:24:50.346 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": "16/21 daemons upgraded", 2024-03-22T06:24:50.346 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "Currently upgrading grafana daemons", 2024-03-22T06:24:50.346 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:24:50.346 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:24:50.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.347+0000 7f72a744b700 1 -- 172.21.15.6:0/592393755 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f728c072c50 msgr2=0x7f728c075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:50.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.347+0000 7f72a744b700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f728c072c50 0x7f728c075100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f729000b020 tx=0x7f7290009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.347+0000 7f72a744b700 1 -- 172.21.15.6:0/592393755 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a0066ef0 msgr2=0x7f72a010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.347+0000 7f72a744b700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a0066ef0 0x7f72a010d950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f729c002a50 tx=0x7f729c004600 comp rx=0 tx=0).stop 2024-03-22T06:24:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.347+0000 7f72a744b700 1 -- 172.21.15.6:0/592393755 shutdown_connections 2024-03-22T06:24:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.347+0000 7f72a744b700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f728c072c50 0x7f728c075100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.347+0000 7f72a744b700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a0066ef0 0x7f72a010d950 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.347+0000 7f72a744b700 1 --2- 172.21.15.6:0/592393755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72a00fff00 0x7f72a010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.347+0000 7f72a744b700 1 -- 172.21.15.6:0/592393755 >> 172.21.15.6:0/592393755 conn(0x7f72a00f7430 msgr2=0x7f72a00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.347+0000 7f72a744b700 1 -- 172.21.15.6:0/592393755 shutdown_connections 2024-03-22T06:24:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.347+0000 7f72a744b700 1 -- 172.21.15.6:0/592393755 wait complete. 2024-03-22T06:24:50.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.457+0000 7fa577cb0700 1 -- 172.21.15.6:0/2472359116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5700fcfa0 msgr2=0x7fa5700fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:50.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.457+0000 7fa577cb0700 1 --2- 172.21.15.6:0/2472359116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5700fcfa0 0x7fa5700fd410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa560004940 tx=0x7fa560009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:50.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.457+0000 7fa577cb0700 1 -- 172.21.15.6:0/2472359116 shutdown_connections 2024-03-22T06:24:50.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.457+0000 7fa577cb0700 1 --2- 172.21.15.6:0/2472359116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5700fcfa0 0x7fa5700fd410 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.457+0000 7fa577cb0700 1 --2- 172.21.15.6:0/2472359116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5700fbcf0 0x7fa5700fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.457+0000 7fa577cb0700 1 -- 172.21.15.6:0/2472359116 >> 172.21.15.6:0/2472359116 conn(0x7fa5700f7450 msgr2=0x7fa5700f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:50.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.457+0000 7fa577cb0700 1 -- 172.21.15.6:0/2472359116 shutdown_connections 2024-03-22T06:24:50.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.457+0000 7fa577cb0700 1 -- 172.21.15.6:0/2472359116 wait complete. 2024-03-22T06:24:50.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.457+0000 7fa577cb0700 1 Processor -- start 2024-03-22T06:24:50.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.457+0000 7fa577cb0700 1 -- start start 2024-03-22T06:24:50.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa577cb0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5700fbcf0 0x7fa5700ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa577cb0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5700fcfa0 0x7fa5700fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa577cb0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700fe870 con 0x7fa5700fbcf0 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa577cb0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700fe9b0 con 0x7fa5700fcfa0 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa575a4c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5700fbcf0 0x7fa5700ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa575a4c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5700fbcf0 0x7fa5700ffdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44226/0 (socket says 172.21.15.6:44226) 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa575a4c700 1 -- 172.21.15.6:0/1822210787 learned_addr learned my addr 172.21.15.6:0/1822210787 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa57524b700 1 --2- 172.21.15.6:0/1822210787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5700fcfa0 0x7fa5700fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa575a4c700 1 -- 172.21.15.6:0/1822210787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5700fcfa0 msgr2=0x7fa5700fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa575a4c700 1 --2- 172.21.15.6:0/1822210787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5700fcfa0 0x7fa5700fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa575a4c700 1 -- 172.21.15.6:0/1822210787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa560005040 con 0x7fa5700fbcf0 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa575a4c700 1 --2- 172.21.15.6:0/1822210787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5700fbcf0 0x7fa5700ffdc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa56c002a50 tx=0x7fa56c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa5667fc700 1 -- 172.21.15.6:0/1822210787 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa56c007cc0 con 0x7fa5700fbcf0 2024-03-22T06:24:50.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa577cb0700 1 -- 172.21.15.6:0/1822210787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5700fec90 con 0x7fa5700fbcf0 2024-03-22T06:24:50.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa577cb0700 1 -- 172.21.15.6:0/1822210787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa570115ad0 con 0x7fa5700fbcf0 2024-03-22T06:24:50.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.458+0000 7fa5667fc700 1 -- 172.21.15.6:0/1822210787 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa56c00f040 con 0x7fa5700fbcf0 2024-03-22T06:24:50.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.459+0000 7fa5667fc700 1 -- 172.21.15.6:0/1822210787 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa56c009640 con 0x7fa5700fbcf0 2024-03-22T06:24:50.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.459+0000 7fa577cb0700 1 -- 172.21.15.6:0/1822210787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa570101b30 con 0x7fa5700fbcf0 2024-03-22T06:24:50.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.460+0000 7fa5667fc700 1 -- 172.21.15.6:0/1822210787 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fa56c0097c0 con 0x7fa5700fbcf0 2024-03-22T06:24:50.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.461+0000 7fa5667fc700 1 --2- 172.21.15.6:0/1822210787 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa55c06e840 0x7fa55c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:24:50.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.461+0000 7fa5667fc700 1 -- 172.21.15.6:0/1822210787 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fa56c08e290 con 0x7fa5700fbcf0 2024-03-22T06:24:50.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.461+0000 7fa57524b700 1 --2- 172.21.15.6:0/1822210787 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa55c06e840 0x7fa55c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:24:50.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.461+0000 7fa57524b700 1 --2- 172.21.15.6:0/1822210787 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa55c06e840 0x7fa55c070cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa5701003a0 tx=0x7fa560009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:24:50.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.470+0000 7fa5667fc700 1 -- 172.21.15.6:0/1822210787 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fa56c0570e0 con 0x7fa5700fbcf0 2024-03-22T06:24:50.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:50 smithi117 ceph-mon[82126]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:50.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:50 smithi117 ceph-mon[82126]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:50 smithi117 ceph-mon[82126]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:50 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1071969972' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:50.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:50 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3726023271' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:24:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.770+0000 7fa577cb0700 1 -- 172.21.15.6:0/1822210787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa57005f5f0 con 0x7fa5700fbcf0 2024-03-22T06:24:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.771+0000 7fa5667fc700 1 -- 172.21.15.6:0/1822210787 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+155 (secure 0 0 0) 0x7fa56c05a7a0 con 0x7fa5700fbcf0 2024-03-22T06:24:50.771 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-03-22T06:24:50.771 INFO:teuthology.orchestra.run.smithi006.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-22T06:24:50.771 INFO:teuthology.orchestra.run.smithi006.stdout: daemon grafana.smithi006 on smithi006 is in error state 2024-03-22T06:24:50.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.773+0000 7fa577cb0700 1 -- 172.21.15.6:0/1822210787 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa55c06e840 msgr2=0x7fa55c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:50.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.773+0000 7fa577cb0700 1 --2- 172.21.15.6:0/1822210787 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa55c06e840 0x7fa55c070cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa5701003a0 tx=0x7fa560009e30 comp rx=0 tx=0).stop 2024-03-22T06:24:50.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.773+0000 7fa577cb0700 1 -- 172.21.15.6:0/1822210787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5700fbcf0 msgr2=0x7fa5700ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:24:50.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.773+0000 7fa577cb0700 1 --2- 172.21.15.6:0/1822210787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5700fbcf0 0x7fa5700ffdc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa56c002a50 tx=0x7fa56c0046b0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.774+0000 7fa577cb0700 1 -- 172.21.15.6:0/1822210787 shutdown_connections 2024-03-22T06:24:50.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.774+0000 7fa577cb0700 1 --2- 172.21.15.6:0/1822210787 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa55c06e840 0x7fa55c070cf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.774+0000 7fa577cb0700 1 --2- 172.21.15.6:0/1822210787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5700fbcf0 0x7fa5700ffdc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.774+0000 7fa577cb0700 1 --2- 172.21.15.6:0/1822210787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5700fcfa0 0x7fa5700fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:24:50.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.774+0000 7fa577cb0700 1 -- 172.21.15.6:0/1822210787 >> 172.21.15.6:0/1822210787 conn(0x7fa5700f7450 msgr2=0x7fa5700663e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:24:50.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.774+0000 7fa577cb0700 1 -- 172.21.15.6:0/1822210787 shutdown_connections 2024-03-22T06:24:50.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:24:50.774+0000 7fa577cb0700 1 -- 172.21.15.6:0/1822210787 wait complete. 2024-03-22T06:24:51.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:51 smithi117 ceph-mon[82126]: pgmap v288: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:51.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:51 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:51.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:51 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:51.506 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:51 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1822210787' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:24:51.531 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:51 smithi006 ceph-mon[94482]: pgmap v288: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:51.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:51 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:51.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:51 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:51.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:51 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1822210787' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:24:52.434 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:52 smithi006 ceph-mon[94482]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:52.435 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:52.435 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:52.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:52 smithi117 ceph-mon[82126]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:24:52.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:52.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:53.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:53 smithi006 ceph-mon[94482]: pgmap v289: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:53.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:53 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:24:53.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:53 smithi117 ceph-mon[82126]: pgmap v289: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:53.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:53 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:24:55.532 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:55 smithi006 ceph-mon[94482]: pgmap v290: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:55.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:55 smithi117 ceph-mon[82126]: pgmap v290: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:56.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:56.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:56.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: pgmap v291: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: Upgrade: Finalizing container_image settings 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T06:24:56.759 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T06:24:56.760 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T06:24:56.760 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T06:24:56.760 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.760 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T06:24:56.760 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T06:24:56.760 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T06:24:56.760 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T06:24:56.760 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T06:24:56.760 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T06:24:56.760 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T06:24:56.760 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: Upgrade: Complete! 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:56.761 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:56.762 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:56.762 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:56.762 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:56.762 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:56.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:56.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:56.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:56.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: pgmap v291: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.783 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: Upgrade: Finalizing container_image settings 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.784 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T06:24:56.785 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: Upgrade: Complete! 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:56.786 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:56.787 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:56.787 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:56.787 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:24:56.787 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:24:56.787 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:24:56.787 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:57.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:57 smithi117 ceph-mon[82126]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-22T06:24:57.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:57 smithi117 ceph-mon[82126]: Cluster is now healthy 2024-03-22T06:24:57.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:57 smithi006 ceph-mon[94482]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-22T06:24:57.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:57 smithi006 ceph-mon[94482]: Cluster is now healthy 2024-03-22T06:24:59.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:58 smithi117 ceph-mon[82126]: pgmap v292: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:59.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:24:58 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:24:59.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:58 smithi006 ceph-mon[94482]: pgmap v292: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:24:59.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:24:58 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:25:01.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:00 smithi117 ceph-mon[82126]: pgmap v293: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:01.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:00 smithi006 ceph-mon[94482]: pgmap v293: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:02.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:02 smithi006 ceph-mon[94482]: pgmap v294: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:03.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:02 smithi117 ceph-mon[82126]: pgmap v294: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:05.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:04 smithi117 ceph-mon[82126]: pgmap v295: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:05.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:04 smithi006 ceph-mon[94482]: pgmap v295: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:07.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:06 smithi117 ceph-mon[82126]: pgmap v296: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:07.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:06 smithi006 ceph-mon[94482]: pgmap v296: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:08.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:25:08.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:08 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:25:08.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:08 smithi117 ceph-mon[82126]: pgmap v297: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:08.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:25:08.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:08 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:25:08.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:08 smithi006 ceph-mon[94482]: pgmap v297: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:10.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:10 smithi117 ceph-mon[82126]: pgmap v298: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:10.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:10 smithi006 ceph-mon[94482]: pgmap v298: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:12.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:12 smithi117 ceph-mon[82126]: pgmap v299: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:12.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:12 smithi006 ceph-mon[94482]: pgmap v299: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:14.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:14 smithi117 ceph-mon[82126]: pgmap v300: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:14.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:14 smithi006 ceph-mon[94482]: pgmap v300: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:16.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:16 smithi117 ceph-mon[82126]: pgmap v301: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:16.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:16 smithi006 ceph-mon[94482]: pgmap v301: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:18.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:18 smithi117 ceph-mon[82126]: pgmap v302: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:18.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:18 smithi006 ceph-mon[94482]: pgmap v302: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:20.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:20 smithi117 ceph-mon[82126]: pgmap v303: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:20.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:20 smithi006 ceph-mon[94482]: pgmap v303: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:20.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.882+0000 7f0872ec8700 1 -- 172.21.15.6:0/3170373901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c0fbce0 msgr2=0x7f086c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:20.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.882+0000 7f0872ec8700 1 --2- 172.21.15.6:0/3170373901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c0fbce0 0x7f086c0fc0f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0860004940 tx=0x7f0860009e30 comp rx=0 tx=0).stop 2024-03-22T06:25:20.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.882+0000 7f0872ec8700 1 -- 172.21.15.6:0/3170373901 shutdown_connections 2024-03-22T06:25:20.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.882+0000 7f0872ec8700 1 --2- 172.21.15.6:0/3170373901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c0fcf90 0x7f086c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:20.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.882+0000 7f0872ec8700 1 --2- 172.21.15.6:0/3170373901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c0fbce0 0x7f086c0fc0f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:20.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.882+0000 7f0872ec8700 1 -- 172.21.15.6:0/3170373901 >> 172.21.15.6:0/3170373901 conn(0x7f086c0f7480 msgr2=0x7f086c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:25:20.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.882+0000 7f0872ec8700 1 -- 172.21.15.6:0/3170373901 shutdown_connections 2024-03-22T06:25:20.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.882+0000 7f0872ec8700 1 -- 172.21.15.6:0/3170373901 wait complete. 2024-03-22T06:25:20.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.882+0000 7f0872ec8700 1 Processor -- start 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.882+0000 7f0872ec8700 1 -- start start 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0872ec8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c0fbce0 0x7f086c10c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0872ec8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c0fcf90 0x7f086c10ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0872ec8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c10d450 con 0x7f086c0fbce0 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0872ec8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c10dca0 con 0x7f086c0fcf90 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0870c64700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c0fbce0 0x7f086c10c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0870c64700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c0fbce0 0x7f086c10c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44170/0 (socket says 172.21.15.6:44170) 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0870c64700 1 -- 172.21.15.6:0/3601358886 learned_addr learned my addr 172.21.15.6:0/3601358886 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f086bfff700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c0fcf90 0x7f086c10ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0870c64700 1 -- 172.21.15.6:0/3601358886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c0fcf90 msgr2=0x7f086c10ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0870c64700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c0fcf90 0x7f086c10ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0870c64700 1 -- 172.21.15.6:0/3601358886 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0860005040 con 0x7f086c0fbce0 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f086bfff700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c0fcf90 0x7f086c10ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:25:20.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0870c64700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c0fbce0 0x7f086c10c910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0860009150 tx=0x7f0860009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:25:20.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f08697fa700 1 -- 172.21.15.6:0/3601358886 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f086000e040 con 0x7f086c0fbce0 2024-03-22T06:25:20.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f08697fa700 1 -- 172.21.15.6:0/3601358886 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f086000bcc0 con 0x7f086c0fbce0 2024-03-22T06:25:20.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f0872ec8700 1 -- 172.21.15.6:0/3601358886 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086c10df20 con 0x7f086c0fbce0 2024-03-22T06:25:20.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.883+0000 7f08697fa700 1 -- 172.21.15.6:0/3601358886 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f086001c630 con 0x7f086c0fbce0 2024-03-22T06:25:20.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.884+0000 7f0872ec8700 1 -- 172.21.15.6:0/3601358886 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086c10e460 con 0x7f086c0fbce0 2024-03-22T06:25:20.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.884+0000 7f0872ec8700 1 -- 172.21.15.6:0/3601358886 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086c05f5f0 con 0x7f086c0fbce0 2024-03-22T06:25:20.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.885+0000 7f08697fa700 1 -- 172.21.15.6:0/3601358886 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f0860013420 con 0x7f086c0fbce0 2024-03-22T06:25:20.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.885+0000 7f08697fa700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f085c06e840 0x7f085c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:20.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.886+0000 7f08697fa700 1 -- 172.21.15.6:0/3601358886 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f086008f170 con 0x7f086c0fbce0 2024-03-22T06:25:20.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.886+0000 7f086bfff700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f085c06e840 0x7f085c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:20.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.886+0000 7f086bfff700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f085c06e840 0x7f085c070cf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f08580041d0 tx=0x7f0858009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:25:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:20.894+0000 7f08697fa700 1 -- 172.21.15.6:0/3601358886 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f086005bb90 con 0x7f086c0fbce0 2024-03-22T06:25:21.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.141+0000 7f0872ec8700 1 -- 172.21.15.6:0/3601358886 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f086c10e850 con 0x7f085c06e840 2024-03-22T06:25:21.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.142+0000 7f08697fa700 1 -- 172.21.15.6:0/3601358886 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f086c10e850 con 0x7f085c06e840 2024-03-22T06:25:21.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.144+0000 7f0872ec8700 1 -- 172.21.15.6:0/3601358886 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f085c06e840 msgr2=0x7f085c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:21.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.144+0000 7f0872ec8700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f085c06e840 0x7f085c070cf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f08580041d0 tx=0x7f0858009040 comp rx=0 tx=0).stop 2024-03-22T06:25:21.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.144+0000 7f0872ec8700 1 -- 172.21.15.6:0/3601358886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c0fbce0 msgr2=0x7f086c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:21.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.144+0000 7f0872ec8700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c0fbce0 0x7f086c10c910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0860009150 tx=0x7f0860009e30 comp rx=0 tx=0).stop 2024-03-22T06:25:21.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.145+0000 7f0872ec8700 1 -- 172.21.15.6:0/3601358886 shutdown_connections 2024-03-22T06:25:21.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.145+0000 7f0872ec8700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f085c06e840 0x7f085c070cf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:21.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.145+0000 7f0872ec8700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c0fbce0 0x7f086c10c910 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:21.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.145+0000 7f0872ec8700 1 --2- 172.21.15.6:0/3601358886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c0fcf90 0x7f086c10ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:21.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.145+0000 7f0872ec8700 1 -- 172.21.15.6:0/3601358886 >> 172.21.15.6:0/3601358886 conn(0x7f086c0f7480 msgr2=0x7f086c0f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:25:21.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.145+0000 7f0872ec8700 1 -- 172.21.15.6:0/3601358886 shutdown_connections 2024-03-22T06:25:21.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:21.145+0000 7f0872ec8700 1 -- 172.21.15.6:0/3601358886 wait complete. 2024-03-22T06:25:21.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:21 smithi117 ceph-mon[82126]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:25:21.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:21 smithi006 ceph-mon[94482]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:25:21.998 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph orch ps' 2024-03-22T06:25:22.274 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:25:22.554 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:22 smithi006 ceph-mon[94482]: pgmap v304: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:22.554 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:22 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:25:22.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:22 smithi117 ceph-mon[82126]: pgmap v304: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:22.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:22 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:25:24.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.064+0000 7f73db2ff700 1 -- 172.21.15.6:0/3589396564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d40fde90 msgr2=0x7f73d40fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:24.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.064+0000 7f73db2ff700 1 --2- 172.21.15.6:0/3589396564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d40fde90 0x7f73d40fe260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f73d0004940 tx=0x7f73d0009e30 comp rx=0 tx=0).stop 2024-03-22T06:25:24.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.064+0000 7f73db2ff700 1 -- 172.21.15.6:0/3589396564 shutdown_connections 2024-03-22T06:25:24.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.064+0000 7f73db2ff700 1 --2- 172.21.15.6:0/3589396564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d40fe830 0x7f73d4106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:24.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.064+0000 7f73db2ff700 1 --2- 172.21.15.6:0/3589396564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d40fde90 0x7f73d40fe260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:24.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.064+0000 7f73db2ff700 1 -- 172.21.15.6:0/3589396564 >> 172.21.15.6:0/3589396564 conn(0x7f73d4072bc0 msgr2=0x7f73d4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:25:24.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.064+0000 7f73db2ff700 1 -- 172.21.15.6:0/3589396564 shutdown_connections 2024-03-22T06:25:24.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.064+0000 7f73db2ff700 1 -- 172.21.15.6:0/3589396564 wait complete. 2024-03-22T06:25:24.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.065+0000 7f73db2ff700 1 Processor -- start 2024-03-22T06:25:24.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.065+0000 7f73db2ff700 1 -- start start 2024-03-22T06:25:24.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.065+0000 7f73db2ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d40fde90 0x7f73d4115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:24.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.065+0000 7f73db2ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d40fe830 0x7f73d41162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:24.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.065+0000 7f73db2ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d4116910 con 0x7f73d40fe830 2024-03-22T06:25:24.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.065+0000 7f73db2ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d410fd20 con 0x7f73d40fde90 2024-03-22T06:25:24.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.065+0000 7f73d889a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d40fe830 0x7f73d41162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:24.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.065+0000 7f73d889a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d40fe830 0x7f73d41162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41630/0 (socket says 172.21.15.6:41630) 2024-03-22T06:25:24.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.065+0000 7f73d889a700 1 -- 172.21.15.6:0/1917456735 learned_addr learned my addr 172.21.15.6:0/1917456735 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:25:24.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.065+0000 7f73d909b700 1 --2- 172.21.15.6:0/1917456735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d40fde90 0x7f73d4115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:24.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.066+0000 7f73d889a700 1 -- 172.21.15.6:0/1917456735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d40fde90 msgr2=0x7f73d4115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:24.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.066+0000 7f73d889a700 1 --2- 172.21.15.6:0/1917456735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d40fde90 0x7f73d4115d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:24.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.066+0000 7f73d889a700 1 -- 172.21.15.6:0/1917456735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d0005040 con 0x7f73d40fe830 2024-03-22T06:25:24.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.066+0000 7f73d889a700 1 --2- 172.21.15.6:0/1917456735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d40fe830 0x7f73d41162c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f73c800c8c0 tx=0x7f73c80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:25:24.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.066+0000 7f73c5ffb700 1 -- 172.21.15.6:0/1917456735 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c800a760 con 0x7f73d40fe830 2024-03-22T06:25:24.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.066+0000 7f73c5ffb700 1 -- 172.21.15.6:0/1917456735 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73c800ad90 con 0x7f73d40fe830 2024-03-22T06:25:24.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.066+0000 7f73db2ff700 1 -- 172.21.15.6:0/1917456735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d4110000 con 0x7f73d40fe830 2024-03-22T06:25:24.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.066+0000 7f73c5ffb700 1 -- 172.21.15.6:0/1917456735 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c80113a0 con 0x7f73d40fe830 2024-03-22T06:25:24.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.066+0000 7f73db2ff700 1 -- 172.21.15.6:0/1917456735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d4110540 con 0x7f73d40fe830 2024-03-22T06:25:24.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.067+0000 7f73db2ff700 1 -- 172.21.15.6:0/1917456735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d4106a20 con 0x7f73d40fe830 2024-03-22T06:25:24.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.068+0000 7f73c5ffb700 1 -- 172.21.15.6:0/1917456735 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f73c8012400 con 0x7f73d40fe830 2024-03-22T06:25:24.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.068+0000 7f73c5ffb700 1 --2- 172.21.15.6:0/1917456735 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f73c006e840 0x7f73c0070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:24.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.069+0000 7f73c5ffb700 1 -- 172.21.15.6:0/1917456735 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f73c808e5d0 con 0x7f73d40fe830 2024-03-22T06:25:24.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.069+0000 7f73d909b700 1 --2- 172.21.15.6:0/1917456735 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f73c006e840 0x7f73c0070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:24.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.069+0000 7f73d909b700 1 --2- 172.21.15.6:0/1917456735 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f73c006e840 0x7f73c0070cf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f73d000b3b0 tx=0x7f73d0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:25:24.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.073+0000 7f73c5ffb700 1 -- 172.21.15.6:0/1917456735 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f73c8057420 con 0x7f73d40fe830 2024-03-22T06:25:24.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.325+0000 7f73db2ff700 1 -- 172.21.15.6:0/1917456735 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f73d4111410 con 0x7f73c006e840 2024-03-22T06:25:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.338+0000 7f73c5ffb700 1 -- 172.21.15.6:0/1917456735 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f73d4111410 con 0x7f73c006e840 2024-03-22T06:25:24.338 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (54s) 34s ago 18m 16.7M - 0.25.0 c8568f914cd2 5fd2ca09ada6 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (6m) 34s ago 18m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (6m) 86s ago 17m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 running (39s) 34s ago 18m 51.4M - 9.4.12 2bacad6d85d8 05b7909e5d8a 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (3m) 34s ago 12m 82.1M - 19.0.0-2299-gea01217f aa461e7c5e7f 3e3319f646e6 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (2m) 34s ago 12m 13.2M - 19.0.0-2299-gea01217f aa461e7c5e7f 09ee1e271adb 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (2m) 86s ago 12m 15.0M - 19.0.0-2299-gea01217f aa461e7c5e7f a9149dea3f8e 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (104s) 86s ago 12m 12.7M - 19.0.0-2299-gea01217f aa461e7c5e7f 0abdc3f226cc 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (9m) 34s ago 19m 520M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (8m) 86s ago 17m 452M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (7m) 34s ago 19m 56.8M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (7m) 86s ago 17m 44.7M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (92s) 34s ago 18m 16.0M - 1.5.0 0da6a335fe13 7eb15a15b4c3 2024-03-22T06:25:24.339 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (88s) 86s ago 17m 3275k - 1.5.0 0da6a335fe13 df7460dcdb08 2024-03-22T06:25:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (6m) 34s ago 16m 155M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:25:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (5m) 34s ago 16m 138M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:25:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (5m) 34s ago 15m 106M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 3ec1f219c19c 2024-03-22T06:25:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (4m) 86s ago 15m 114M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 9cba39b6f32c 2024-03-22T06:25:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (3m) 86s ago 14m 104M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1950fc21bd2e 2024-03-22T06:25:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (3m) 86s ago 14m 108M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1a6a0ee2a228 2024-03-22T06:25:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (71s) 34s ago 18m 53.5M - 2.43.0 a07b618ecd1d 831f4be843c5 2024-03-22T06:25:24.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.341+0000 7f73db2ff700 1 -- 172.21.15.6:0/1917456735 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f73c006e840 msgr2=0x7f73c0070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:24.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.341+0000 7f73db2ff700 1 --2- 172.21.15.6:0/1917456735 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f73c006e840 0x7f73c0070cf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f73d000b3b0 tx=0x7f73d0009e30 comp rx=0 tx=0).stop 2024-03-22T06:25:24.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.341+0000 7f73db2ff700 1 -- 172.21.15.6:0/1917456735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d40fe830 msgr2=0x7f73d41162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:24.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.341+0000 7f73db2ff700 1 --2- 172.21.15.6:0/1917456735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d40fe830 0x7f73d41162c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f73c800c8c0 tx=0x7f73c80044c0 comp rx=0 tx=0).stop 2024-03-22T06:25:24.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.341+0000 7f73db2ff700 1 -- 172.21.15.6:0/1917456735 shutdown_connections 2024-03-22T06:25:24.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.341+0000 7f73db2ff700 1 --2- 172.21.15.6:0/1917456735 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f73c006e840 0x7f73c0070cf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:24.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.341+0000 7f73db2ff700 1 --2- 172.21.15.6:0/1917456735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d40fde90 0x7f73d4115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:24.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.341+0000 7f73db2ff700 1 --2- 172.21.15.6:0/1917456735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d40fe830 0x7f73d41162c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:24.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.341+0000 7f73db2ff700 1 -- 172.21.15.6:0/1917456735 >> 172.21.15.6:0/1917456735 conn(0x7f73d4072bc0 msgr2=0x7f73d40fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:25:24.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.341+0000 7f73db2ff700 1 -- 172.21.15.6:0/1917456735 shutdown_connections 2024-03-22T06:25:24.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:24.341+0000 7f73db2ff700 1 -- 172.21.15.6:0/1917456735 wait complete. 2024-03-22T06:25:24.653 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:24 smithi006 ceph-mon[94482]: pgmap v305: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:24.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:24 smithi117 ceph-mon[82126]: pgmap v305: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:25.052 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph orch upgrade status' 2024-03-22T06:25:25.323 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:25:25.704 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:25 smithi006 ceph-mon[94482]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:25:25.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:25 smithi117 ceph-mon[82126]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:25:26.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:26 smithi117 ceph-mon[82126]: pgmap v306: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:26.765 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:26 smithi006 ceph-mon[94482]: pgmap v306: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.314+0000 7f8f33f38700 1 -- 172.21.15.6:0/2630407582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f2c101d70 msgr2=0x7f8f2c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.314+0000 7f8f33f38700 1 --2- 172.21.15.6:0/2630407582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f2c101d70 0x7f8f2c102140 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8f28004940 tx=0x7f8f28009e30 comp rx=0 tx=0).stop 2024-03-22T06:25:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.314+0000 7f8f33f38700 1 -- 172.21.15.6:0/2630407582 shutdown_connections 2024-03-22T06:25:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.314+0000 7f8f33f38700 1 --2- 172.21.15.6:0/2630407582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f2c102710 0x7f8f2c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.314+0000 7f8f33f38700 1 --2- 172.21.15.6:0/2630407582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f2c101d70 0x7f8f2c102140 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.314+0000 7f8f33f38700 1 -- 172.21.15.6:0/2630407582 >> 172.21.15.6:0/2630407582 conn(0x7f8f2c072bc0 msgr2=0x7f8f2c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:25:27.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.314+0000 7f8f33f38700 1 -- 172.21.15.6:0/2630407582 shutdown_connections 2024-03-22T06:25:27.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.314+0000 7f8f33f38700 1 -- 172.21.15.6:0/2630407582 wait complete. 2024-03-22T06:25:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.315+0000 7f8f33f38700 1 Processor -- start 2024-03-22T06:25:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.315+0000 7f8f33f38700 1 -- start start 2024-03-22T06:25:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.315+0000 7f8f33f38700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f2c101d70 0x7f8f2c112a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.315+0000 7f8f33f38700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f2c102710 0x7f8f2c112f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.315+0000 7f8f33f38700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c1135b0 con 0x7f8f2c102710 2024-03-22T06:25:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.315+0000 7f8f33f38700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c10c9a0 con 0x7f8f2c101d70 2024-03-22T06:25:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.315+0000 7f8f314d3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f2c102710 0x7f8f2c112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.315+0000 7f8f314d3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f2c102710 0x7f8f2c112f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41652/0 (socket says 172.21.15.6:41652) 2024-03-22T06:25:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.315+0000 7f8f314d3700 1 -- 172.21.15.6:0/4225681641 learned_addr learned my addr 172.21.15.6:0/4225681641 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:25:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.316+0000 7f8f31cd4700 1 --2- 172.21.15.6:0/4225681641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f2c101d70 0x7f8f2c112a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.316+0000 7f8f314d3700 1 -- 172.21.15.6:0/4225681641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f2c101d70 msgr2=0x7f8f2c112a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.316+0000 7f8f314d3700 1 --2- 172.21.15.6:0/4225681641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f2c101d70 0x7f8f2c112a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.316+0000 7f8f314d3700 1 -- 172.21.15.6:0/4225681641 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f28005040 con 0x7f8f2c102710 2024-03-22T06:25:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.316+0000 7f8f314d3700 1 --2- 172.21.15.6:0/4225681641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f2c102710 0x7f8f2c112f60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c010d50 tx=0x7f8f1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:25:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.316+0000 7f8f227fc700 1 -- 172.21.15.6:0/4225681641 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f1c007ab0 con 0x7f8f2c102710 2024-03-22T06:25:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.316+0000 7f8f227fc700 1 -- 172.21.15.6:0/4225681641 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f1c007c10 con 0x7f8f2c102710 2024-03-22T06:25:27.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.316+0000 7f8f33f38700 1 -- 172.21.15.6:0/4225681641 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2c10cc20 con 0x7f8f2c102710 2024-03-22T06:25:27.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.316+0000 7f8f227fc700 1 -- 172.21.15.6:0/4225681641 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f1c012700 con 0x7f8f2c102710 2024-03-22T06:25:27.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.316+0000 7f8f33f38700 1 -- 172.21.15.6:0/4225681641 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2c10d160 con 0x7f8f2c102710 2024-03-22T06:25:27.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.317+0000 7f8f33f38700 1 -- 172.21.15.6:0/4225681641 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2c0fd4c0 con 0x7f8f2c102710 2024-03-22T06:25:27.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.318+0000 7f8f227fc700 1 -- 172.21.15.6:0/4225681641 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f8f1c018030 con 0x7f8f2c102710 2024-03-22T06:25:27.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.318+0000 7f8f227fc700 1 --2- 172.21.15.6:0/4225681641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8f1806e840 0x7f8f18070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:27.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.318+0000 7f8f227fc700 1 -- 172.21.15.6:0/4225681641 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f8f1c016070 con 0x7f8f2c102710 2024-03-22T06:25:27.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.318+0000 7f8f31cd4700 1 --2- 172.21.15.6:0/4225681641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8f1806e840 0x7f8f18070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:27.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.319+0000 7f8f31cd4700 1 --2- 172.21.15.6:0/4225681641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8f1806e840 0x7f8f18070cf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800b0d0 tx=0x7f8f28009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:25:27.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.323+0000 7f8f227fc700 1 -- 172.21.15.6:0/4225681641 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f8f1c05d300 con 0x7f8f2c102710 2024-03-22T06:25:27.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.550+0000 7f8f33f38700 1 -- 172.21.15.6:0/4225681641 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f2c108e70 con 0x7f8f1806e840 2024-03-22T06:25:27.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.552+0000 7f8f227fc700 1 -- 172.21.15.6:0/4225681641 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f8f2c108e70 con 0x7f8f1806e840 2024-03-22T06:25:27.552 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:25:27.552 INFO:teuthology.orchestra.run.smithi006.stdout: "target_image": null, 2024-03-22T06:25:27.552 INFO:teuthology.orchestra.run.smithi006.stdout: "in_progress": false, 2024-03-22T06:25:27.552 INFO:teuthology.orchestra.run.smithi006.stdout: "which": "", 2024-03-22T06:25:27.552 INFO:teuthology.orchestra.run.smithi006.stdout: "services_complete": [], 2024-03-22T06:25:27.552 INFO:teuthology.orchestra.run.smithi006.stdout: "progress": null, 2024-03-22T06:25:27.552 INFO:teuthology.orchestra.run.smithi006.stdout: "message": "", 2024-03-22T06:25:27.553 INFO:teuthology.orchestra.run.smithi006.stdout: "is_paused": false 2024-03-22T06:25:27.553 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:25:27.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.554+0000 7f8f33f38700 1 -- 172.21.15.6:0/4225681641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8f1806e840 msgr2=0x7f8f18070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:27.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.554+0000 7f8f33f38700 1 --2- 172.21.15.6:0/4225681641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8f1806e840 0x7f8f18070cf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800b0d0 tx=0x7f8f28009e30 comp rx=0 tx=0).stop 2024-03-22T06:25:27.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.554+0000 7f8f33f38700 1 -- 172.21.15.6:0/4225681641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f2c102710 msgr2=0x7f8f2c112f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:27.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.554+0000 7f8f33f38700 1 --2- 172.21.15.6:0/4225681641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f2c102710 0x7f8f2c112f60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c010d50 tx=0x7f8f1c00b040 comp rx=0 tx=0).stop 2024-03-22T06:25:27.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.554+0000 7f8f33f38700 1 -- 172.21.15.6:0/4225681641 shutdown_connections 2024-03-22T06:25:27.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.554+0000 7f8f33f38700 1 --2- 172.21.15.6:0/4225681641 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f8f1806e840 0x7f8f18070cf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:27.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.554+0000 7f8f33f38700 1 --2- 172.21.15.6:0/4225681641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f2c101d70 0x7f8f2c112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:27.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.554+0000 7f8f33f38700 1 --2- 172.21.15.6:0/4225681641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f2c102710 0x7f8f2c112f60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.554+0000 7f8f33f38700 1 -- 172.21.15.6:0/4225681641 >> 172.21.15.6:0/4225681641 conn(0x7f8f2c072bc0 msgr2=0x7f8f2c0faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:25:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.554+0000 7f8f33f38700 1 -- 172.21.15.6:0/4225681641 shutdown_connections 2024-03-22T06:25:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:27.554+0000 7f8f33f38700 1 -- 172.21.15.6:0/4225681641 wait complete. 2024-03-22T06:25:28.475 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph health detail' 2024-03-22T06:25:28.746 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:25:28.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:28 smithi117 ceph-mon[82126]: pgmap v307: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:28.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:28 smithi117 ceph-mon[82126]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:25:28.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:28 smithi006 ceph-mon[94482]: pgmap v307: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:28.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:28 smithi006 ceph-mon[94482]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:25:30.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.434+0000 7f78f9719700 1 -- 172.21.15.6:0/1098966114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78f40fe880 msgr2=0x7f78f4106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:30.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.434+0000 7f78f9719700 1 --2- 172.21.15.6:0/1098966114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78f40fe880 0x7f78f4106ec0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f78e8004940 tx=0x7f78e8009e30 comp rx=0 tx=0).stop 2024-03-22T06:25:30.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 -- 172.21.15.6:0/1098966114 shutdown_connections 2024-03-22T06:25:30.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 --2- 172.21.15.6:0/1098966114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78f40fe880 0x7f78f4106ec0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:30.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 --2- 172.21.15.6:0/1098966114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f40fdee0 0x7f78f40fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:30.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 -- 172.21.15.6:0/1098966114 >> 172.21.15.6:0/1098966114 conn(0x7f78f40f78f0 msgr2=0x7f78f40f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:25:30.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 -- 172.21.15.6:0/1098966114 shutdown_connections 2024-03-22T06:25:30.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 -- 172.21.15.6:0/1098966114 wait complete. 2024-03-22T06:25:30.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 Processor -- start 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 -- start start 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f40fdee0 0x7f78f41138a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78f40fe880 0x7f78f4113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f41144c0 con 0x7f78f40fe880 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.435+0000 7f78f9719700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f410da30 con 0x7f78f40fdee0 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78f27fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78f40fe880 0x7f78f4113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78f27fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78f40fe880 0x7f78f4113de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41666/0 (socket says 172.21.15.6:41666) 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78f27fc700 1 -- 172.21.15.6:0/791913803 learned_addr learned my addr 172.21.15.6:0/791913803 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78f2ffd700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f40fdee0 0x7f78f41138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78f27fc700 1 -- 172.21.15.6:0/791913803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f40fdee0 msgr2=0x7f78f41138a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78f27fc700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f40fdee0 0x7f78f41138a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78f27fc700 1 -- 172.21.15.6:0/791913803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8005040 con 0x7f78f40fe880 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78f2ffd700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f40fdee0 0x7f78f41138a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78f27fc700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78f40fe880 0x7f78f4113de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f78e8004890 tx=0x7f78e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:25:30.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78db7fe700 1 -- 172.21.15.6:0/791913803 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e800e040 con 0x7f78f40fe880 2024-03-22T06:25:30.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78f9719700 1 -- 172.21.15.6:0/791913803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f410dd10 con 0x7f78f40fe880 2024-03-22T06:25:30.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78db7fe700 1 -- 172.21.15.6:0/791913803 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78e8009af0 con 0x7f78f40fe880 2024-03-22T06:25:30.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78db7fe700 1 -- 172.21.15.6:0/791913803 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e801c630 con 0x7f78f40fe880 2024-03-22T06:25:30.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.436+0000 7f78f9719700 1 -- 172.21.15.6:0/791913803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f410e250 con 0x7f78f40fe880 2024-03-22T06:25:30.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.437+0000 7f78f9719700 1 -- 172.21.15.6:0/791913803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f4066a90 con 0x7f78f40fe880 2024-03-22T06:25:30.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.438+0000 7f78db7fe700 1 -- 172.21.15.6:0/791913803 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f78e800bbc0 con 0x7f78f40fe880 2024-03-22T06:25:30.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.438+0000 7f78db7fe700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f78dc06e840 0x7f78dc070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:30.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.439+0000 7f78db7fe700 1 -- 172.21.15.6:0/791913803 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f78e8090380 con 0x7f78f40fe880 2024-03-22T06:25:30.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.439+0000 7f78f2ffd700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f78dc06e840 0x7f78dc070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:30.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.439+0000 7f78f2ffd700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f78dc06e840 0x7f78dc070cf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009b10 tx=0x7f78e4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:25:30.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.443+0000 7f78db7fe700 1 -- 172.21.15.6:0/791913803 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f78e805cda0 con 0x7f78f40fe880 2024-03-22T06:25:30.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.738+0000 7f78f9719700 1 -- 172.21.15.6:0/791913803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f78f410e470 con 0x7f78f40fe880 2024-03-22T06:25:30.739 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:30 smithi006 ceph-mon[94482]: pgmap v308: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:30.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.739+0000 7f78db7fe700 1 -- 172.21.15.6:0/791913803 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f78e805c930 con 0x7f78f40fe880 2024-03-22T06:25:30.740 INFO:teuthology.orchestra.run.smithi006.stdout:HEALTH_OK 2024-03-22T06:25:30.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.741+0000 7f78f9719700 1 -- 172.21.15.6:0/791913803 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f78dc06e840 msgr2=0x7f78dc070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:30.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.741+0000 7f78f9719700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f78dc06e840 0x7f78dc070cf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009b10 tx=0x7f78e4008040 comp rx=0 tx=0).stop 2024-03-22T06:25:30.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.741+0000 7f78f9719700 1 -- 172.21.15.6:0/791913803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78f40fe880 msgr2=0x7f78f4113de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:30.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.741+0000 7f78f9719700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78f40fe880 0x7f78f4113de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f78e8004890 tx=0x7f78e8009e30 comp rx=0 tx=0).stop 2024-03-22T06:25:30.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.741+0000 7f78f9719700 1 -- 172.21.15.6:0/791913803 shutdown_connections 2024-03-22T06:25:30.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.741+0000 7f78f9719700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f78dc06e840 0x7f78dc070cf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:30.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.741+0000 7f78f9719700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f40fdee0 0x7f78f41138a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:30.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.741+0000 7f78f9719700 1 --2- 172.21.15.6:0/791913803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78f40fe880 0x7f78f4113de0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:30.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.741+0000 7f78f9719700 1 -- 172.21.15.6:0/791913803 >> 172.21.15.6:0/791913803 conn(0x7f78f40f78f0 msgr2=0x7f78f41036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:25:30.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.741+0000 7f78f9719700 1 -- 172.21.15.6:0/791913803 shutdown_connections 2024-03-22T06:25:30.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:30.741+0000 7f78f9719700 1 -- 172.21.15.6:0/791913803 wait complete. 2024-03-22T06:25:30.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:30 smithi117 ceph-mon[82126]: pgmap v308: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:31.513 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph versions' 2024-03-22T06:25:31.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:31 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/791913803' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:25:31.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:31 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/791913803' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T06:25:31.784 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:25:32.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:32 smithi117 ceph-mon[82126]: pgmap v309: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:32.781 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:32 smithi006 ceph-mon[94482]: pgmap v309: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:33.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.543+0000 7f2d0acc5700 1 -- 172.21.15.6:0/2675288295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d041037a0 msgr2=0x7f2d040fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.543+0000 7f2d0acc5700 1 --2- 172.21.15.6:0/2675288295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d041037a0 0x7f2d040fce10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0004940 tx=0x7f2cf0009e30 comp rx=0 tx=0).stop 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.543+0000 7f2d0acc5700 1 -- 172.21.15.6:0/2675288295 shutdown_connections 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.543+0000 7f2d0acc5700 1 --2- 172.21.15.6:0/2675288295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d041037a0 0x7f2d040fce10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.543+0000 7f2d0acc5700 1 --2- 172.21.15.6:0/2675288295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d04102e00 0x7f2d041031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.543+0000 7f2d0acc5700 1 -- 172.21.15.6:0/2675288295 >> 172.21.15.6:0/2675288295 conn(0x7f2d040f89b0 msgr2=0x7f2d040fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.543+0000 7f2d0acc5700 1 -- 172.21.15.6:0/2675288295 shutdown_connections 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.543+0000 7f2d0acc5700 1 -- 172.21.15.6:0/2675288295 wait complete. 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d0acc5700 1 Processor -- start 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d0acc5700 1 -- start start 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d0acc5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d04102e00 0x7f2d04113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d0acc5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d041037a0 0x7f2d041140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d0acc5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d04114710 con 0x7f2d041037a0 2024-03-22T06:25:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d0acc5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0410db70 con 0x7f2d04102e00 2024-03-22T06:25:33.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d03fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d041037a0 0x7f2d041140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:33.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d03fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d041037a0 0x7f2d041140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41680/0 (socket says 172.21.15.6:41680) 2024-03-22T06:25:33.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d03fff700 1 -- 172.21.15.6:0/3284794414 learned_addr learned my addr 172.21.15.6:0/3284794414 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:25:33.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d08a61700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d04102e00 0x7f2d04113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:33.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d03fff700 1 -- 172.21.15.6:0/3284794414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d04102e00 msgr2=0x7f2d04113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:33.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d03fff700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d04102e00 0x7f2d04113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:33.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d03fff700 1 -- 172.21.15.6:0/3284794414 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0005040 con 0x7f2d041037a0 2024-03-22T06:25:33.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.544+0000 7f2d08a61700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d04102e00 0x7f2d04113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:25:33.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.545+0000 7f2d03fff700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d041037a0 0x7f2d041140c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009100 tx=0x7f2cf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:25:33.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.545+0000 7f2d017fa700 1 -- 172.21.15.6:0/3284794414 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cf000e040 con 0x7f2d041037a0 2024-03-22T06:25:33.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.545+0000 7f2d017fa700 1 -- 172.21.15.6:0/3284794414 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cf0013420 con 0x7f2d041037a0 2024-03-22T06:25:33.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.545+0000 7f2d0acc5700 1 -- 172.21.15.6:0/3284794414 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0410ddf0 con 0x7f2d041037a0 2024-03-22T06:25:33.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.545+0000 7f2d017fa700 1 -- 172.21.15.6:0/3284794414 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cf001c610 con 0x7f2d041037a0 2024-03-22T06:25:33.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.545+0000 7f2d0acc5700 1 -- 172.21.15.6:0/3284794414 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0410e3b0 con 0x7f2d041037a0 2024-03-22T06:25:33.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.546+0000 7f2d0acc5700 1 -- 172.21.15.6:0/3284794414 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d041052d0 con 0x7f2d041037a0 2024-03-22T06:25:33.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.546+0000 7f2d017fa700 1 -- 172.21.15.6:0/3284794414 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f2cf0013590 con 0x7f2d041037a0 2024-03-22T06:25:33.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.547+0000 7f2d017fa700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2cf406e840 0x7f2cf4070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:25:33.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.547+0000 7f2d017fa700 1 -- 172.21.15.6:0/3284794414 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f2cf008f830 con 0x7f2d041037a0 2024-03-22T06:25:33.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.547+0000 7f2d08a61700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2cf406e840 0x7f2cf4070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:25:33.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.547+0000 7f2d08a61700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2cf406e840 0x7f2cf4070cf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009fd0 tx=0x7f2cf8009e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:25:33.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.551+0000 7f2d017fa700 1 -- 172.21.15.6:0/3284794414 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f2cf0058680 con 0x7f2d041037a0 2024-03-22T06:25:33.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.853+0000 7f2d0acc5700 1 -- 172.21.15.6:0/3284794414 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2d0410f1d0 con 0x7f2d041037a0 2024-03-22T06:25:33.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.853+0000 7f2d017fa700 1 -- 172.21.15.6:0/3284794414 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2cf005bd40 con 0x7f2d041037a0 2024-03-22T06:25:33.854 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:25:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:25:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:25:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:25:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:25:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:25:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:25:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:25:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 6 2024-03-22T06:25:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:25:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:25:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 4 2024-03-22T06:25:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:25:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:25:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 14 2024-03-22T06:25:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:25:33.855 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:25:33.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.856+0000 7f2d0acc5700 1 -- 172.21.15.6:0/3284794414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2cf406e840 msgr2=0x7f2cf4070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:33.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.856+0000 7f2d0acc5700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2cf406e840 0x7f2cf4070cf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009fd0 tx=0x7f2cf8009e50 comp rx=0 tx=0).stop 2024-03-22T06:25:33.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.856+0000 7f2d0acc5700 1 -- 172.21.15.6:0/3284794414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d041037a0 msgr2=0x7f2d041140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:25:33.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.856+0000 7f2d0acc5700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d041037a0 0x7f2d041140c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009100 tx=0x7f2cf0009e30 comp rx=0 tx=0).stop 2024-03-22T06:25:33.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.856+0000 7f2d0acc5700 1 -- 172.21.15.6:0/3284794414 shutdown_connections 2024-03-22T06:25:33.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.856+0000 7f2d0acc5700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2cf406e840 0x7f2cf4070cf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:33.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.856+0000 7f2d0acc5700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d04102e00 0x7f2d04113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:33.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.856+0000 7f2d0acc5700 1 --2- 172.21.15.6:0/3284794414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d041037a0 0x7f2d041140c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:25:33.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.856+0000 7f2d0acc5700 1 -- 172.21.15.6:0/3284794414 >> 172.21.15.6:0/3284794414 conn(0x7f2d040f89b0 msgr2=0x7f2d04102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:25:33.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.856+0000 7f2d0acc5700 1 -- 172.21.15.6:0/3284794414 shutdown_connections 2024-03-22T06:25:33.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:25:33.856+0000 7f2d0acc5700 1 -- 172.21.15.6:0/3284794414 wait complete. 2024-03-22T06:25:34.512 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-22T06:25:34.813 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:25:35.098 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:34 smithi006 ceph-mon[94482]: pgmap v310: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:35.098 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:34 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3284794414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:25:35.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:34 smithi117 ceph-mon[82126]: pgmap v310: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:35.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:34 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3284794414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:25:36.724 INFO:teuthology.orchestra.run.smithi006.stdout:wait for servicemap items w/ changing names to refresh 2024-03-22T06:25:37.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:36 smithi006 ceph-mon[94482]: pgmap v311: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:37.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:36 smithi117 ceph-mon[82126]: pgmap v311: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:37.343 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'sleep 60' 2024-03-22T06:25:37.633 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:25:38.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:37 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:25:38.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:37 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:25:38.994 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:38 smithi006 ceph-mon[94482]: pgmap v312: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:39.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:38 smithi117 ceph-mon[82126]: pgmap v312: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:41.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:40 smithi117 ceph-mon[82126]: pgmap v313: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:41.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:40 smithi006 ceph-mon[94482]: pgmap v313: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:43.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:42 smithi117 ceph-mon[82126]: pgmap v314: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:43.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:42 smithi006 ceph-mon[94482]: pgmap v314: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:45.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:44 smithi006 ceph-mon[94482]: pgmap v315: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:45.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:44 smithi117 ceph-mon[82126]: pgmap v315: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:47.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:46 smithi117 ceph-mon[82126]: pgmap v316: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:47.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:46 smithi006 ceph-mon[94482]: pgmap v316: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:49.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:48 smithi117 ceph-mon[82126]: pgmap v317: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:49.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:48 smithi006 ceph-mon[94482]: pgmap v317: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:51.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:50 smithi117 ceph-mon[82126]: pgmap v318: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:51.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:50 smithi006 ceph-mon[94482]: pgmap v318: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:53.107 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:52 smithi006 ceph-mon[94482]: pgmap v319: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:53.107 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:25:53.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:52 smithi117 ceph-mon[82126]: pgmap v319: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:53.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:25:55.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:54 smithi117 ceph-mon[82126]: pgmap v320: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:55.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:54 smithi006 ceph-mon[94482]: pgmap v320: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:56.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:55 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:25:56.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:55 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:25:57.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:56 smithi117 ceph-mon[82126]: pgmap v321: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:57.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:25:57.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:25:57.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:25:57.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:56 smithi006 ceph-mon[94482]: pgmap v321: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:57.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:25:57.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:25:57.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:25:59.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:25:58 smithi117 ceph-mon[82126]: pgmap v322: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:25:59.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:25:58 smithi006 ceph-mon[94482]: pgmap v322: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:01.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:00 smithi117 ceph-mon[82126]: pgmap v323: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:01.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:00 smithi006 ceph-mon[94482]: pgmap v323: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:03.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:02 smithi117 ceph-mon[82126]: pgmap v324: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:03.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:02 smithi006 ceph-mon[94482]: pgmap v324: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:05.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:04 smithi117 ceph-mon[82126]: pgmap v325: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:05.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:04 smithi006 ceph-mon[94482]: pgmap v325: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:07.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:06 smithi117 ceph-mon[82126]: pgmap v326: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:07.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:06 smithi006 ceph-mon[94482]: pgmap v326: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:08.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:26:08.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:26:09.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:08 smithi117 ceph-mon[82126]: pgmap v327: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:09.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:08 smithi006 ceph-mon[94482]: pgmap v327: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:11.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:10 smithi117 ceph-mon[82126]: pgmap v328: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:11.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:10 smithi006 ceph-mon[94482]: pgmap v328: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:13.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:12 smithi117 ceph-mon[82126]: pgmap v329: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:13.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:12 smithi006 ceph-mon[94482]: pgmap v329: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:15.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:14 smithi006 ceph-mon[94482]: pgmap v330: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:15.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:14 smithi117 ceph-mon[82126]: pgmap v330: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:17.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:16 smithi117 ceph-mon[82126]: pgmap v331: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:17.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:16 smithi006 ceph-mon[94482]: pgmap v331: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:19.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:18 smithi117 ceph-mon[82126]: pgmap v332: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:19.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:18 smithi006 ceph-mon[94482]: pgmap v332: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:21.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:20 smithi117 ceph-mon[82126]: pgmap v333: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:21.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:20 smithi006 ceph-mon[94482]: pgmap v333: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:23.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:22 smithi117 ceph-mon[82126]: pgmap v334: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:23.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:22 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:26:23.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:22 smithi006 ceph-mon[94482]: pgmap v334: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:23.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:22 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:26:25.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:24 smithi006 ceph-mon[94482]: pgmap v335: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:25.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:24 smithi117 ceph-mon[82126]: pgmap v335: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:27.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:26 smithi117 ceph-mon[82126]: pgmap v336: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:27.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:26 smithi006 ceph-mon[94482]: pgmap v336: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:29.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:28 smithi117 ceph-mon[82126]: pgmap v337: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:29.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:28 smithi006 ceph-mon[94482]: pgmap v337: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:31.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:30 smithi117 ceph-mon[82126]: pgmap v338: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:31.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:30 smithi006 ceph-mon[94482]: pgmap v338: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:33.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:32 smithi117 ceph-mon[82126]: pgmap v339: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:33.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:32 smithi006 ceph-mon[94482]: pgmap v339: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:35.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:34 smithi117 ceph-mon[82126]: pgmap v340: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:35.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:34 smithi006 ceph-mon[94482]: pgmap v340: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:37.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:36 smithi117 ceph-mon[82126]: pgmap v341: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:37.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:36 smithi006 ceph-mon[94482]: pgmap v341: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:38.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:37 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:26:38.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:37 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:26:39.246 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:38 smithi006 ceph-mon[94482]: pgmap v342: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:39.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:38 smithi117 ceph-mon[82126]: pgmap v342: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:39.983 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph orch ps' 2024-03-22T06:26:40.265 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:26:41.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:40 smithi006 ceph-mon[94482]: pgmap v343: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:41.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:40 smithi117 ceph-mon[82126]: pgmap v343: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:42.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.110+0000 7f0468229700 1 -- 172.21.15.6:0/2661282176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04601026a0 msgr2=0x7f04600fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:42.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.110+0000 7f0468229700 1 --2- 172.21.15.6:0/2661282176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04601026a0 0x7f04600fbd10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0454004940 tx=0x7f0454009e30 comp rx=0 tx=0).stop 2024-03-22T06:26:42.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.111+0000 7f0468229700 1 -- 172.21.15.6:0/2661282176 shutdown_connections 2024-03-22T06:26:42.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.111+0000 7f0468229700 1 --2- 172.21.15.6:0/2661282176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04601026a0 0x7f04600fbd10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.111+0000 7f0468229700 1 --2- 172.21.15.6:0/2661282176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0460101d00 0x7f04601020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.111+0000 7f0468229700 1 -- 172.21.15.6:0/2661282176 >> 172.21.15.6:0/2661282176 conn(0x7f04600f78d0 msgr2=0x7f04600f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.111+0000 7f0468229700 1 -- 172.21.15.6:0/2661282176 shutdown_connections 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.111+0000 7f0468229700 1 -- 172.21.15.6:0/2661282176 wait complete. 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.111+0000 7f0468229700 1 Processor -- start 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.111+0000 7f0468229700 1 -- start start 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f0468229700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0460101d00 0x7f0460102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f0468229700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04601026a0 0x7f04601030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f0468229700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04601036c0 con 0x7f0460101d00 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f0468229700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0460103800 con 0x7f04601026a0 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f0465fc5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0460101d00 0x7f0460102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f0465fc5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0460101d00 0x7f0460102b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58974/0 (socket says 172.21.15.6:58974) 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f0465fc5700 1 -- 172.21.15.6:0/1471376102 learned_addr learned my addr 172.21.15.6:0/1471376102 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f04657c4700 1 --2- 172.21.15.6:0/1471376102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04601026a0 0x7f04601030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f0465fc5700 1 -- 172.21.15.6:0/1471376102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04601026a0 msgr2=0x7f04601030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:42.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f0465fc5700 1 --2- 172.21.15.6:0/1471376102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04601026a0 0x7f04601030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:42.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f0465fc5700 1 -- 172.21.15.6:0/1471376102 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0454005040 con 0x7f0460101d00 2024-03-22T06:26:42.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.112+0000 7f0465fc5700 1 --2- 172.21.15.6:0/1471376102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0460101d00 0x7f0460102b60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f045800b790 tx=0x7f0458009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:42.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.113+0000 7f045e7fc700 1 -- 172.21.15.6:0/1471376102 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f045800f770 con 0x7f0460101d00 2024-03-22T06:26:42.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.113+0000 7f045e7fc700 1 -- 172.21.15.6:0/1471376102 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f045800fda0 con 0x7f0460101d00 2024-03-22T06:26:42.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.113+0000 7f0468229700 1 -- 172.21.15.6:0/1471376102 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0460119eb0 con 0x7f0460101d00 2024-03-22T06:26:42.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.113+0000 7f045e7fc700 1 -- 172.21.15.6:0/1471376102 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f045800d310 con 0x7f0460101d00 2024-03-22T06:26:42.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.113+0000 7f0468229700 1 -- 172.21.15.6:0/1471376102 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046011a390 con 0x7f0460101d00 2024-03-22T06:26:42.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.114+0000 7f0468229700 1 -- 172.21.15.6:0/1471376102 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04600fd450 con 0x7f0460101d00 2024-03-22T06:26:42.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.114+0000 7f045e7fc700 1 -- 172.21.15.6:0/1471376102 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f0458010420 con 0x7f0460101d00 2024-03-22T06:26:42.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.115+0000 7f045e7fc700 1 --2- 172.21.15.6:0/1471376102 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f044806e840 0x7f0448070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:42.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.115+0000 7f045e7fc700 1 -- 172.21.15.6:0/1471376102 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f045808e4e0 con 0x7f0460101d00 2024-03-22T06:26:42.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.115+0000 7f04657c4700 1 --2- 172.21.15.6:0/1471376102 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f044806e840 0x7f0448070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.115+0000 7f04657c4700 1 --2- 172.21.15.6:0/1471376102 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f044806e840 0x7f0448070cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f045400b3b0 tx=0x7f0454009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:42.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.120+0000 7f045e7fc700 1 -- 172.21.15.6:0/1471376102 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f04580573d0 con 0x7f0460101d00 2024-03-22T06:26:42.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.367+0000 7f0468229700 1 -- 172.21.15.6:0/1471376102 --> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0460104250 con 0x7f044806e840 2024-03-22T06:26:42.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.380+0000 7f045e7fc700 1 -- 172.21.15.6:0/1471376102 <== mgr.34102 v2:172.21.15.6:6800/3762089760 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f0460104250 con 0x7f044806e840 2024-03-22T06:26:42.380 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T06:26:42.380 INFO:teuthology.orchestra.run.smithi006.stdout:alertmanager.smithi006 smithi006 *:9093,9094 running (2m) 112s ago 19m 16.7M - 0.25.0 c8568f914cd2 5fd2ca09ada6 2024-03-22T06:26:42.380 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi006 smithi006 running (8m) 112s ago 20m 6665k - 19.0.0-2299-gea01217f aa461e7c5e7f cf7152d834da 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:crash.smithi117 smithi117 running (8m) 2m ago 18m 6660k - 19.0.0-2299-gea01217f aa461e7c5e7f 785cc0cc8b0d 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:grafana.smithi006 smithi006 *:3000 running (117s) 112s ago 19m 51.4M - 9.4.12 2bacad6d85d8 05b7909e5d8a 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.bztopc smithi006 running (4m) 112s ago 14m 82.1M - 19.0.0-2299-gea01217f aa461e7c5e7f 3e3319f646e6 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi006.dcjpla smithi006 running (3m) 112s ago 14m 13.2M - 19.0.0-2299-gea01217f aa461e7c5e7f 09ee1e271adb 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.fwkech smithi117 running (3m) 2m ago 14m 15.0M - 19.0.0-2299-gea01217f aa461e7c5e7f a9149dea3f8e 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:mds.cephfs.smithi117.zqxyql smithi117 running (3m) 2m ago 14m 12.7M - 19.0.0-2299-gea01217f aa461e7c5e7f 0abdc3f226cc 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi006.sfpxdy smithi006 *:8443,9283,8765 running (10m) 112s ago 21m 520M - 19.0.0-2299-gea01217f aa461e7c5e7f c620e2e2f964 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.smithi117.bqeskk smithi117 *:8443,9283,8765 running (9m) 2m ago 18m 452M - 19.0.0-2299-gea01217f aa461e7c5e7f b88a5bef6285 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi006 smithi006 running (9m) 112s ago 21m 56.8M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 7eafc6be6dca 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:mon.smithi117 smithi117 running (8m) 2m ago 18m 44.7M 2048M 19.0.0-2299-gea01217f aa461e7c5e7f 4766be58b3b2 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi006 smithi006 *:9100 running (2m) 112s ago 19m 16.0M - 1.5.0 0da6a335fe13 7eb15a15b4c3 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:node-exporter.smithi117 smithi117 *:9100 running (2m) 2m ago 18m 3275k - 1.5.0 0da6a335fe13 df7460dcdb08 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (7m) 112s ago 18m 155M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f a106f4647f4e 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi006 running (7m) 112s ago 17m 138M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 2b0a2f2a205f 2024-03-22T06:26:42.381 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi006 running (6m) 112s ago 17m 106M 1559M 19.0.0-2299-gea01217f aa461e7c5e7f 3ec1f219c19c 2024-03-22T06:26:42.382 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi117 running (5m) 2m ago 16m 114M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 9cba39b6f32c 2024-03-22T06:26:42.382 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi117 running (5m) 2m ago 16m 104M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1950fc21bd2e 2024-03-22T06:26:42.382 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi117 running (4m) 2m ago 15m 108M 2581M 19.0.0-2299-gea01217f aa461e7c5e7f 1a6a0ee2a228 2024-03-22T06:26:42.382 INFO:teuthology.orchestra.run.smithi006.stdout:prometheus.smithi006 smithi006 *:9095 running (2m) 112s ago 19m 53.5M - 2.43.0 a07b618ecd1d 831f4be843c5 2024-03-22T06:26:42.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.382+0000 7f0468229700 1 -- 172.21.15.6:0/1471376102 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f044806e840 msgr2=0x7f0448070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:42.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.382+0000 7f0468229700 1 --2- 172.21.15.6:0/1471376102 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f044806e840 0x7f0448070cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f045400b3b0 tx=0x7f0454009e30 comp rx=0 tx=0).stop 2024-03-22T06:26:42.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.382+0000 7f0468229700 1 -- 172.21.15.6:0/1471376102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0460101d00 msgr2=0x7f0460102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:42.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.382+0000 7f0468229700 1 --2- 172.21.15.6:0/1471376102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0460101d00 0x7f0460102b60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f045800b790 tx=0x7f0458009250 comp rx=0 tx=0).stop 2024-03-22T06:26:42.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.382+0000 7f0468229700 1 -- 172.21.15.6:0/1471376102 shutdown_connections 2024-03-22T06:26:42.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.382+0000 7f0468229700 1 --2- 172.21.15.6:0/1471376102 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f044806e840 0x7f0448070cf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:42.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.382+0000 7f0468229700 1 --2- 172.21.15.6:0/1471376102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0460101d00 0x7f0460102b60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:42.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.382+0000 7f0468229700 1 --2- 172.21.15.6:0/1471376102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04601026a0 0x7f04601030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:42.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.382+0000 7f0468229700 1 -- 172.21.15.6:0/1471376102 >> 172.21.15.6:0/1471376102 conn(0x7f04600f78d0 msgr2=0x7f04600f8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:42.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.383+0000 7f0468229700 1 -- 172.21.15.6:0/1471376102 shutdown_connections 2024-03-22T06:26:42.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:42.383+0000 7f0468229700 1 -- 172.21.15.6:0/1471376102 wait complete. 2024-03-22T06:26:43.007 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:42 smithi006 ceph-mon[94482]: pgmap v344: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:43.032 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph versions' 2024-03-22T06:26:43.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:42 smithi117 ceph-mon[82126]: pgmap v344: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:43.355 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:26:44.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:43 smithi117 ceph-mon[82126]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:26:44.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:43 smithi006 ceph-mon[94482]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T06:26:45.219 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:44 smithi006 ceph-mon[94482]: pgmap v345: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:45.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:44 smithi117 ceph-mon[82126]: pgmap v345: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:45.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.319+0000 7ff140646700 1 -- 172.21.15.6:0/3026249469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1381037b0 msgr2=0x7ff1380fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:45.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.319+0000 7ff140646700 1 --2- 172.21.15.6:0/3026249469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1381037b0 0x7ff1380fce20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff128004940 tx=0x7ff128009e30 comp rx=0 tx=0).stop 2024-03-22T06:26:45.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 -- 172.21.15.6:0/3026249469 shutdown_connections 2024-03-22T06:26:45.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 --2- 172.21.15.6:0/3026249469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1381037b0 0x7ff1380fce20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:45.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 --2- 172.21.15.6:0/3026249469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff138102e10 0x7ff1381031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:45.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 -- 172.21.15.6:0/3026249469 >> 172.21.15.6:0/3026249469 conn(0x7ff1380f89a0 msgr2=0x7ff1380fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:45.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 -- 172.21.15.6:0/3026249469 shutdown_connections 2024-03-22T06:26:45.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 -- 172.21.15.6:0/3026249469 wait complete. 2024-03-22T06:26:45.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 Processor -- start 2024-03-22T06:26:45.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 -- start start 2024-03-22T06:26:45.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff138102e10 0x7ff138115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:45.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1381037b0 0x7ff138116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:45.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff138116950 con 0x7ff1381037b0 2024-03-22T06:26:45.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.320+0000 7ff140646700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13810fd20 con 0x7ff138102e10 2024-03-22T06:26:45.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff13e3e2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff138102e10 0x7ff138115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:45.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff13e3e2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff138102e10 0x7ff138115dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.6:59596/0 (socket says 172.21.15.6:59596) 2024-03-22T06:26:45.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff13e3e2700 1 -- 172.21.15.6:0/1668714987 learned_addr learned my addr 172.21.15.6:0/1668714987 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:45.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff13dbe1700 1 --2- 172.21.15.6:0/1668714987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1381037b0 0x7ff138116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:45.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff13dbe1700 1 -- 172.21.15.6:0/1668714987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff138102e10 msgr2=0x7ff138115dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:45.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff13dbe1700 1 --2- 172.21.15.6:0/1668714987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff138102e10 0x7ff138115dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:45.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff13dbe1700 1 -- 172.21.15.6:0/1668714987 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128005040 con 0x7ff1381037b0 2024-03-22T06:26:45.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff13dbe1700 1 --2- 172.21.15.6:0/1668714987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1381037b0 0x7ff138116300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff128009fd0 tx=0x7ff128009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:45.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff12effd700 1 -- 172.21.15.6:0/1668714987 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff12800e040 con 0x7ff1381037b0 2024-03-22T06:26:45.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff12effd700 1 -- 172.21.15.6:0/1668714987 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff128013420 con 0x7ff1381037b0 2024-03-22T06:26:45.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff12effd700 1 -- 172.21.15.6:0/1668714987 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff12801c630 con 0x7ff1381037b0 2024-03-22T06:26:45.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.321+0000 7ff140646700 1 -- 172.21.15.6:0/1668714987 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13810ffa0 con 0x7ff1381037b0 2024-03-22T06:26:45.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.322+0000 7ff140646700 1 -- 172.21.15.6:0/1668714987 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff138110530 con 0x7ff1381037b0 2024-03-22T06:26:45.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.322+0000 7ff140646700 1 -- 172.21.15.6:0/1668714987 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1380fe560 con 0x7ff1381037b0 2024-03-22T06:26:45.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.323+0000 7ff12effd700 1 -- 172.21.15.6:0/1668714987 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7ff12800baf0 con 0x7ff1381037b0 2024-03-22T06:26:45.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.323+0000 7ff12effd700 1 --2- 172.21.15.6:0/1668714987 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff12406e840 0x7ff124070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:45.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.324+0000 7ff13e3e2700 1 --2- 172.21.15.6:0/1668714987 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff12406e840 0x7ff124070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:45.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.324+0000 7ff12effd700 1 -- 172.21.15.6:0/1668714987 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7ff12808f620 con 0x7ff1381037b0 2024-03-22T06:26:45.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.324+0000 7ff13e3e2700 1 --2- 172.21.15.6:0/1668714987 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff12406e840 0x7ff124070cf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff1340041d0 tx=0x7ff13400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:45.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.328+0000 7ff12effd700 1 -- 172.21.15.6:0/1668714987 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7ff12805c0f0 con 0x7ff1381037b0 2024-03-22T06:26:45.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.622+0000 7ff140646700 1 -- 172.21.15.6:0/1668714987 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff1380714a0 con 0x7ff1381037b0 2024-03-22T06:26:45.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.623+0000 7ff12effd700 1 -- 172.21.15.6:0/1668714987 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff12805bc80 con 0x7ff1381037b0 2024-03-22T06:26:45.626 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-22T06:26:45.626 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": { 2024-03-22T06:26:45.626 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:26:45.626 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:26:45.626 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": { 2024-03-22T06:26:45.626 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 2 2024-03-22T06:26:45.626 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:26:45.626 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": { 2024-03-22T06:26:45.626 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 6 2024-03-22T06:26:45.626 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:26:45.627 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-03-22T06:26:45.627 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 4 2024-03-22T06:26:45.627 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-03-22T06:26:45.627 INFO:teuthology.orchestra.run.smithi006.stdout: "overall": { 2024-03-22T06:26:45.627 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)": 14 2024-03-22T06:26:45.627 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-03-22T06:26:45.627 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-22T06:26:45.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.625+0000 7ff140646700 1 -- 172.21.15.6:0/1668714987 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff12406e840 msgr2=0x7ff124070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:45.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.625+0000 7ff140646700 1 --2- 172.21.15.6:0/1668714987 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff12406e840 0x7ff124070cf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff1340041d0 tx=0x7ff13400b040 comp rx=0 tx=0).stop 2024-03-22T06:26:45.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.625+0000 7ff140646700 1 -- 172.21.15.6:0/1668714987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1381037b0 msgr2=0x7ff138116300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:45.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.625+0000 7ff140646700 1 --2- 172.21.15.6:0/1668714987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1381037b0 0x7ff138116300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff128009fd0 tx=0x7ff128009e30 comp rx=0 tx=0).stop 2024-03-22T06:26:45.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.625+0000 7ff140646700 1 -- 172.21.15.6:0/1668714987 shutdown_connections 2024-03-22T06:26:45.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.625+0000 7ff140646700 1 --2- 172.21.15.6:0/1668714987 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff12406e840 0x7ff124070cf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:45.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.625+0000 7ff140646700 1 --2- 172.21.15.6:0/1668714987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff138102e10 0x7ff138115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:45.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.625+0000 7ff140646700 1 --2- 172.21.15.6:0/1668714987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1381037b0 0x7ff138116300 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:45.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.625+0000 7ff140646700 1 -- 172.21.15.6:0/1668714987 >> 172.21.15.6:0/1668714987 conn(0x7ff1380f89a0 msgr2=0x7ff1380f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:45.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.625+0000 7ff140646700 1 -- 172.21.15.6:0/1668714987 shutdown_connections 2024-03-22T06:26:45.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:45.625+0000 7ff140646700 1 -- 172.21.15.6:0/1668714987 wait complete. 2024-03-22T06:26:46.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:45 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1668714987' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:26:46.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:45 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1668714987' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:26:46.348 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-22T06:26:46.610 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:26:47.000 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:46 smithi006 ceph-mon[94482]: pgmap v346: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:47.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:46 smithi117 ceph-mon[82126]: pgmap v346: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.280+0000 7f267cb9e700 1 -- 172.21.15.6:0/4247792105 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2678102710 msgr2=0x7f26780fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.280+0000 7f267cb9e700 1 --2- 172.21.15.6:0/4247792105 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2678102710 0x7f26780fbd80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2660004940 tx=0x7f2660009e30 comp rx=0 tx=0).stop 2024-03-22T06:26:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.280+0000 7f267cb9e700 1 -- 172.21.15.6:0/4247792105 shutdown_connections 2024-03-22T06:26:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.280+0000 7f267cb9e700 1 --2- 172.21.15.6:0/4247792105 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2678102710 0x7f26780fbd80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.280+0000 7f267cb9e700 1 --2- 172.21.15.6:0/4247792105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678101d70 0x7f2678102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.280+0000 7f267cb9e700 1 -- 172.21.15.6:0/4247792105 >> 172.21.15.6:0/4247792105 conn(0x7f2678072bc0 msgr2=0x7f2678072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.280+0000 7f267cb9e700 1 -- 172.21.15.6:0/4247792105 shutdown_connections 2024-03-22T06:26:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.280+0000 7f267cb9e700 1 -- 172.21.15.6:0/4247792105 wait complete. 2024-03-22T06:26:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f267cb9e700 1 Processor -- start 2024-03-22T06:26:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f267cb9e700 1 -- start start 2024-03-22T06:26:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f267cb9e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678101d70 0x7f267810eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f267cb9e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2678102710 0x7f267810f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f267cb9e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267810f720 con 0x7f2678102710 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f267cb9e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267810f860 con 0x7f2678101d70 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f2675d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2678102710 0x7f267810f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f2675d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2678102710 0x7f267810f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36480/0 (socket says 172.21.15.6:36480) 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f2675d9b700 1 -- 172.21.15.6:0/2300386278 learned_addr learned my addr 172.21.15.6:0/2300386278 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f267659c700 1 --2- 172.21.15.6:0/2300386278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678101d70 0x7f267810eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f2675d9b700 1 -- 172.21.15.6:0/2300386278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678101d70 msgr2=0x7f267810eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f2675d9b700 1 --2- 172.21.15.6:0/2300386278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678101d70 0x7f267810eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.281+0000 7f2675d9b700 1 -- 172.21.15.6:0/2300386278 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2660005040 con 0x7f2678102710 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.282+0000 7f2675d9b700 1 --2- 172.21.15.6:0/2300386278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2678102710 0x7f267810f1e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2660009180 tx=0x7f2660009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.282+0000 7f266effd700 1 -- 172.21.15.6:0/2300386278 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f266000e040 con 0x7f2678102710 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.282+0000 7f266effd700 1 -- 172.21.15.6:0/2300386278 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2660013420 con 0x7f2678102710 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.282+0000 7f267cb9e700 1 -- 172.21.15.6:0/2300386278 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2678117b50 con 0x7f2678102710 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.282+0000 7f266effd700 1 -- 172.21.15.6:0/2300386278 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f266001c610 con 0x7f2678102710 2024-03-22T06:26:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.282+0000 7f267cb9e700 1 -- 172.21.15.6:0/2300386278 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26781180b0 con 0x7f2678102710 2024-03-22T06:26:48.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.283+0000 7f267cb9e700 1 -- 172.21.15.6:0/2300386278 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2678104240 con 0x7f2678102710 2024-03-22T06:26:48.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.283+0000 7f266effd700 1 -- 172.21.15.6:0/2300386278 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f266001c880 con 0x7f2678102710 2024-03-22T06:26:48.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.284+0000 7f266effd700 1 --2- 172.21.15.6:0/2300386278 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f266406e840 0x7f2664070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:48.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.284+0000 7f266effd700 1 -- 172.21.15.6:0/2300386278 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f266008f600 con 0x7f2678102710 2024-03-22T06:26:48.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.284+0000 7f267659c700 1 --2- 172.21.15.6:0/2300386278 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f266406e840 0x7f2664070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:48.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.284+0000 7f267659c700 1 --2- 172.21.15.6:0/2300386278 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f266406e840 0x7f2664070cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2668007a40 tx=0x7f2668009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:48.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.289+0000 7f266effd700 1 -- 172.21.15.6:0/2300386278 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f2660059580 con 0x7f2678102710 2024-03-22T06:26:48.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.590+0000 7f267cb9e700 1 -- 172.21.15.6:0/2300386278 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2678065e70 con 0x7f2678102710 2024-03-22T06:26:48.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.590+0000 7f266effd700 1 -- 172.21.15.6:0/2300386278 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f266005be80 con 0x7f2678102710 2024-03-22T06:26:48.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.592+0000 7f267cb9e700 1 -- 172.21.15.6:0/2300386278 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f266406e840 msgr2=0x7f2664070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:48.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.592+0000 7f267cb9e700 1 --2- 172.21.15.6:0/2300386278 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f266406e840 0x7f2664070cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2668007a40 tx=0x7f2668009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:48.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.592+0000 7f267cb9e700 1 -- 172.21.15.6:0/2300386278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2678102710 msgr2=0x7f267810f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:48.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.592+0000 7f267cb9e700 1 --2- 172.21.15.6:0/2300386278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2678102710 0x7f267810f1e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2660009180 tx=0x7f2660009e30 comp rx=0 tx=0).stop 2024-03-22T06:26:48.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.592+0000 7f267cb9e700 1 -- 172.21.15.6:0/2300386278 shutdown_connections 2024-03-22T06:26:48.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.592+0000 7f267cb9e700 1 --2- 172.21.15.6:0/2300386278 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f266406e840 0x7f2664070cf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:48.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.592+0000 7f267cb9e700 1 --2- 172.21.15.6:0/2300386278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678101d70 0x7f267810eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:48.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.592+0000 7f267cb9e700 1 --2- 172.21.15.6:0/2300386278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2678102710 0x7f267810f1e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:48.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.592+0000 7f267cb9e700 1 -- 172.21.15.6:0/2300386278 >> 172.21.15.6:0/2300386278 conn(0x7f2678072bc0 msgr2=0x7f26780fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:48.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.592+0000 7f267cb9e700 1 -- 172.21.15.6:0/2300386278 shutdown_connections 2024-03-22T06:26:48.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:48.592+0000 7f267cb9e700 1 -- 172.21.15.6:0/2300386278 wait complete. 2024-03-22T06:26:48.602 INFO:teuthology.orchestra.run.smithi006.stdout:true 2024-03-22T06:26:49.237 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:48 smithi006 ceph-mon[94482]: pgmap v347: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:49.237 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:48 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2300386278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:26:49.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:48 smithi117 ceph-mon[82126]: pgmap v347: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:49.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:48 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2300386278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:26:49.263 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -e sha1=ea01217f8a79678e64ef6a5d68a043a4ef21ca37 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-22T06:26:49.526 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:26:51.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:50 smithi006 ceph-mon[94482]: pgmap v348: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:51.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:50 smithi117 ceph-mon[82126]: pgmap v348: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:51.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.306+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3752406978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46a4070300 msgr2=0x7f46a40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:51.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.306+0000 7f46aa3ea700 1 --2- 172.21.15.6:0/3752406978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46a4070300 0x7f46a40706d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4694004970 tx=0x7f4694009e30 comp rx=0 tx=0).stop 2024-03-22T06:26:51.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.306+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3752406978 shutdown_connections 2024-03-22T06:26:51.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.306+0000 7f46aa3ea700 1 --2- 172.21.15.6:0/3752406978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46a4070c10 0x7f46a4109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:51.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.306+0000 7f46aa3ea700 1 --2- 172.21.15.6:0/3752406978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46a4070300 0x7f46a40706d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:51.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.306+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3752406978 >> 172.21.15.6:0/3752406978 conn(0x7f46a40f8980 msgr2=0x7f46a40fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:51.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.306+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3752406978 shutdown_connections 2024-03-22T06:26:51.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.306+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3752406978 wait complete. 2024-03-22T06:26:51.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.306+0000 7f46aa3ea700 1 Processor -- start 2024-03-22T06:26:51.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.306+0000 7f46aa3ea700 1 -- start start 2024-03-22T06:26:51.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46aa3ea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46a4070300 0x7f46a4115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:51.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46aa3ea700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46a4070c10 0x7f46a4116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:51.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46aa3ea700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4116970 con 0x7f46a4070300 2024-03-22T06:26:51.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46aa3ea700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a410fd40 con 0x7f46a4070c10 2024-03-22T06:26:51.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46a3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46a4070300 0x7f46a4115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:51.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46a3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46a4070300 0x7f46a4115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36494/0 (socket says 172.21.15.6:36494) 2024-03-22T06:26:51.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46a3fff700 1 -- 172.21.15.6:0/3331428098 learned_addr learned my addr 172.21.15.6:0/3331428098 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:51.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46a37fe700 1 --2- 172.21.15.6:0/3331428098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46a4070c10 0x7f46a4116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:51.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46a3fff700 1 -- 172.21.15.6:0/3331428098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46a4070c10 msgr2=0x7f46a4116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:51.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46a3fff700 1 --2- 172.21.15.6:0/3331428098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46a4070c10 0x7f46a4116320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:51.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46a3fff700 1 -- 172.21.15.6:0/3331428098 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694005040 con 0x7f46a4070300 2024-03-22T06:26:51.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46a3fff700 1 --2- 172.21.15.6:0/3331428098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46a4070300 0x7f46a4115de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4694000c00 tx=0x7f4694009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:51.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46a0ff9700 1 -- 172.21.15.6:0/3331428098 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f469400e040 con 0x7f46a4070300 2024-03-22T06:26:51.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46a0ff9700 1 -- 172.21.15.6:0/3331428098 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f469400bc10 con 0x7f46a4070300 2024-03-22T06:26:51.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3331428098 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a410ffc0 con 0x7f46a4070300 2024-03-22T06:26:51.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.307+0000 7f46a0ff9700 1 -- 172.21.15.6:0/3331428098 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f469401c630 con 0x7f46a4070300 2024-03-22T06:26:51.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.308+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3331428098 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a4110500 con 0x7f46a4070300 2024-03-22T06:26:51.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.308+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3331428098 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a4071a40 con 0x7f46a4070300 2024-03-22T06:26:51.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.309+0000 7f46a0ff9700 1 -- 172.21.15.6:0/3331428098 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f469400bd80 con 0x7f46a4070300 2024-03-22T06:26:51.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.309+0000 7f46a0ff9700 1 --2- 172.21.15.6:0/3331428098 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f468c06e840 0x7f468c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:51.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.309+0000 7f46a0ff9700 1 -- 172.21.15.6:0/3331428098 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f469408f910 con 0x7f46a4070300 2024-03-22T06:26:51.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.309+0000 7f46a37fe700 1 --2- 172.21.15.6:0/3331428098 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f468c06e840 0x7f468c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:51.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.310+0000 7f46a37fe700 1 --2- 172.21.15.6:0/3331428098 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f468c06e840 0x7f468c070cf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4698007fd0 tx=0x7f4698009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:51.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.317+0000 7f46a0ff9700 1 -- 172.21.15.6:0/3331428098 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f469405c1f0 con 0x7f46a4070300 2024-03-22T06:26:51.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.612+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3331428098 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f46a410c190 con 0x7f46a4070300 2024-03-22T06:26:51.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.612+0000 7f46a0ff9700 1 -- 172.21.15.6:0/3331428098 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4694004de0 con 0x7f46a4070300 2024-03-22T06:26:51.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.615+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3331428098 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f468c06e840 msgr2=0x7f468c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:51.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.615+0000 7f46aa3ea700 1 --2- 172.21.15.6:0/3331428098 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f468c06e840 0x7f468c070cf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4698007fd0 tx=0x7f4698009040 comp rx=0 tx=0).stop 2024-03-22T06:26:51.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.615+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3331428098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46a4070300 msgr2=0x7f46a4115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:51.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.615+0000 7f46aa3ea700 1 --2- 172.21.15.6:0/3331428098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46a4070300 0x7f46a4115de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4694000c00 tx=0x7f4694009e30 comp rx=0 tx=0).stop 2024-03-22T06:26:51.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.615+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3331428098 shutdown_connections 2024-03-22T06:26:51.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.615+0000 7f46aa3ea700 1 --2- 172.21.15.6:0/3331428098 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f468c06e840 0x7f468c070cf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:51.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.615+0000 7f46aa3ea700 1 --2- 172.21.15.6:0/3331428098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46a4070300 0x7f46a4115de0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:51.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.615+0000 7f46aa3ea700 1 --2- 172.21.15.6:0/3331428098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46a4070c10 0x7f46a4116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:51.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.615+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3331428098 >> 172.21.15.6:0/3331428098 conn(0x7f46a40f8980 msgr2=0x7f46a40f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:51.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.615+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3331428098 shutdown_connections 2024-03-22T06:26:51.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:51.616+0000 7f46aa3ea700 1 -- 172.21.15.6:0/3331428098 wait complete. 2024-03-22T06:26:51.626 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph version 19.0.0-2299-gea01217f (ea01217f8a79678e64ef6a5d68a043a4ef21ca37) squid (dev)" 2024-03-22T06:26:52.188 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:51 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3331428098' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:26:52.208 DEBUG:teuthology.parallel:result is None 2024-03-22T06:26:52.208 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T06:26:52.219 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:26:52.219 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T06:26:52.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:51 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3331428098' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T06:26:52.494 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/mon.smithi006/config 2024-03-22T06:26:53.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:52 smithi117 ceph-mon[82126]: pgmap v349: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:53.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:52 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:26:53.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:52 smithi006 ceph-mon[94482]: pgmap v349: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:53.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:52 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:26:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.210+0000 7f261cd26700 1 -- 172.21.15.6:0/1601123873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2618102e20 msgr2=0x7f26181031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:54.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.210+0000 7f261cd26700 1 --2- 172.21.15.6:0/1601123873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2618102e20 0x7f26181031f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2610004940 tx=0x7f2610009e30 comp rx=0 tx=0).stop 2024-03-22T06:26:54.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.210+0000 7f261cd26700 1 -- 172.21.15.6:0/1601123873 shutdown_connections 2024-03-22T06:26:54.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.210+0000 7f261cd26700 1 --2- 172.21.15.6:0/1601123873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26181037c0 0x7f26180fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:54.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.210+0000 7f261cd26700 1 --2- 172.21.15.6:0/1601123873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2618102e20 0x7f26181031f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:54.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.210+0000 7f261cd26700 1 -- 172.21.15.6:0/1601123873 >> 172.21.15.6:0/1601123873 conn(0x7f26180f89d0 msgr2=0x7f26180fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:54.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.210+0000 7f261cd26700 1 -- 172.21.15.6:0/1601123873 shutdown_connections 2024-03-22T06:26:54.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.210+0000 7f261cd26700 1 -- 172.21.15.6:0/1601123873 wait complete. 2024-03-22T06:26:54.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.211+0000 7f261cd26700 1 Processor -- start 2024-03-22T06:26:54.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.211+0000 7f261cd26700 1 -- start start 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.211+0000 7f261cd26700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618102e20 0x7f2618113c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.211+0000 7f261cd26700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26181037c0 0x7f2618114150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.211+0000 7f261cd26700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26181147a0 con 0x7f26181037c0 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.211+0000 7f261cd26700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261810db90 con 0x7f2618102e20 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.211+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26181037c0 0x7f2618114150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.211+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26181037c0 0x7f2618114150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35298/0 (socket says 172.21.15.6:35298) 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.211+0000 7f2615d9b700 1 -- 172.21.15.6:0/3432204212 learned_addr learned my addr 172.21.15.6:0/3432204212 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.211+0000 7f261659c700 1 --2- 172.21.15.6:0/3432204212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618102e20 0x7f2618113c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.212+0000 7f2615d9b700 1 -- 172.21.15.6:0/3432204212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618102e20 msgr2=0x7f2618113c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.212+0000 7f2615d9b700 1 --2- 172.21.15.6:0/3432204212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618102e20 0x7f2618113c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.212+0000 7f2615d9b700 1 -- 172.21.15.6:0/3432204212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2610005040 con 0x7f26181037c0 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.212+0000 7f2615d9b700 1 --2- 172.21.15.6:0/3432204212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26181037c0 0x7f2618114150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2600002a80 tx=0x7f2600004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:54.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.212+0000 7f260effd700 1 -- 172.21.15.6:0/3432204212 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2600004e80 con 0x7f26181037c0 2024-03-22T06:26:54.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.212+0000 7f261cd26700 1 -- 172.21.15.6:0/3432204212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261810de70 con 0x7f26181037c0 2024-03-22T06:26:54.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.212+0000 7f260effd700 1 -- 172.21.15.6:0/3432204212 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f260000f040 con 0x7f26181037c0 2024-03-22T06:26:54.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.212+0000 7f260effd700 1 -- 172.21.15.6:0/3432204212 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f260000daa0 con 0x7f26181037c0 2024-03-22T06:26:54.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.212+0000 7f261cd26700 1 -- 172.21.15.6:0/3432204212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261810e430 con 0x7f26181037c0 2024-03-22T06:26:54.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.213+0000 7f261cd26700 1 -- 172.21.15.6:0/3432204212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2618100130 con 0x7f26181037c0 2024-03-22T06:26:54.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.214+0000 7f260effd700 1 -- 172.21.15.6:0/3432204212 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f2600007cc0 con 0x7f26181037c0 2024-03-22T06:26:54.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.214+0000 7f260effd700 1 --2- 172.21.15.6:0/3432204212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f260406e840 0x7f2604070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:54.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.215+0000 7f260effd700 1 -- 172.21.15.6:0/3432204212 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f260008de00 con 0x7f26181037c0 2024-03-22T06:26:54.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.215+0000 7f261659c700 1 --2- 172.21.15.6:0/3432204212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f260406e840 0x7f2604070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:54.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.215+0000 7f261659c700 1 --2- 172.21.15.6:0/3432204212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f260406e840 0x7f2604070cf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2610004f90 tx=0x7f2610009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:54.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.218+0000 7f260effd700 1 -- 172.21.15.6:0/3432204212 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f260005a780 con 0x7f26181037c0 2024-03-22T06:26:54.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.470+0000 7f261cd26700 1 -- 172.21.15.6:0/3432204212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f261810f1f0 con 0x7f26181037c0 2024-03-22T06:26:54.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.470+0000 7f260effd700 1 -- 172.21.15.6:0/3432204212 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1908 (secure 0 0 0) 0x7f260005a310 con 0x7f26181037c0 2024-03-22T06:26:54.470 INFO:teuthology.orchestra.run.smithi006.stdout:e36 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 35 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-03-22T06:12:20.386024+0000 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-03-22T06:23:39.065820+0000 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 60 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-03-22T06:26:54.471 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 95 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=34406} 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-03-22T06:26:54.472 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 1 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 34406 members: 34406 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.bztopc{0:34406} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.6:6826/192114945,v1:172.21.15.6:6827/192114945] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi006.dcjpla{-1:34438} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.6:6828/1244315045,v1:172.21.15.6:6829/1244315045] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.fwkech{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6824/1194667779,v1:172.21.15.117:6825/1194667779] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:26:54.473 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.cephfs.smithi117.zqxyql{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2788388155,v1:172.21.15.117:6827/2788388155] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T06:26:54.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.472+0000 7f261cd26700 1 -- 172.21.15.6:0/3432204212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f260406e840 msgr2=0x7f2604070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:54.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.472+0000 7f261cd26700 1 --2- 172.21.15.6:0/3432204212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f260406e840 0x7f2604070cf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2610004f90 tx=0x7f2610009e30 comp rx=0 tx=0).stop 2024-03-22T06:26:54.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.472+0000 7f261cd26700 1 -- 172.21.15.6:0/3432204212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26181037c0 msgr2=0x7f2618114150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:54.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.472+0000 7f261cd26700 1 --2- 172.21.15.6:0/3432204212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26181037c0 0x7f2618114150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2600002a80 tx=0x7f2600004630 comp rx=0 tx=0).stop 2024-03-22T06:26:54.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.472+0000 7f261cd26700 1 -- 172.21.15.6:0/3432204212 shutdown_connections 2024-03-22T06:26:54.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.472+0000 7f261cd26700 1 --2- 172.21.15.6:0/3432204212 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f260406e840 0x7f2604070cf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:54.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.472+0000 7f261cd26700 1 --2- 172.21.15.6:0/3432204212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618102e20 0x7f2618113c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:54.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.472+0000 7f261cd26700 1 --2- 172.21.15.6:0/3432204212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26181037c0 0x7f2618114150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:54.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.472+0000 7f261cd26700 1 -- 172.21.15.6:0/3432204212 >> 172.21.15.6:0/3432204212 conn(0x7f26180f89d0 msgr2=0x7f2618107410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:54.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.472+0000 7f261cd26700 1 -- 172.21.15.6:0/3432204212 shutdown_connections 2024-03-22T06:26:54.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:54.472+0000 7f261cd26700 1 -- 172.21.15.6:0/3432204212 wait complete. 2024-03-22T06:26:54.475 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 36 2024-03-22T06:26:55.129 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:54 smithi006 ceph-mon[94482]: pgmap v350: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:55.129 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:54 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3432204212' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:26:55.153 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-22T06:26:55.166 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T06:26:55.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:54 smithi117 ceph-mon[82126]: pgmap v350: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:55.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:54 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3432204212' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T06:26:55.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.273+0000 7f2cb055d640 1 -- 172.21.15.6:0/969200831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca8064cb0 msgr2=0x7f2ca8108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:55.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.273+0000 7f2cb055d640 1 --2- 172.21.15.6:0/969200831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca8064cb0 0x7f2ca8108780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2c980048d0 tx=0x7f2c98009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.273+0000 7f2cb055d640 1 -- 172.21.15.6:0/969200831 shutdown_connections 2024-03-22T06:26:55.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.273+0000 7f2cb055d640 1 --2- 172.21.15.6:0/969200831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca8064cb0 0x7f2ca8108780 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.273+0000 7f2cb055d640 1 --2- 172.21.15.6:0/969200831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca80643a0 0x7f2ca8064770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.273+0000 7f2cb055d640 1 -- 172.21.15.6:0/969200831 >> 172.21.15.6:0/969200831 conn(0x7f2ca8071130 msgr2=0x7f2ca8071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:55.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.273+0000 7f2cb055d640 1 -- 172.21.15.6:0/969200831 shutdown_connections 2024-03-22T06:26:55.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.273+0000 7f2cb055d640 1 -- 172.21.15.6:0/969200831 wait complete. 2024-03-22T06:26:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.274+0000 7f2cb055d640 1 Processor -- start 2024-03-22T06:26:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.274+0000 7f2cb055d640 1 -- start start 2024-03-22T06:26:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.274+0000 7f2cb055d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca80643a0 0x7f2ca8114eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.274+0000 7f2cb055d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca8064cb0 0x7f2ca81153f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.274+0000 7f2cb055d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca81159f0 con 0x7f2ca8064cb0 2024-03-22T06:26:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.274+0000 7f2cb055d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca810f040 con 0x7f2ca80643a0 2024-03-22T06:26:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2cadaf8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca8064cb0 0x7f2ca81153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2cadaf8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca8064cb0 0x7f2ca81153f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35320/0 (socket says 172.21.15.6:35320) 2024-03-22T06:26:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2cadaf8640 1 -- 172.21.15.6:0/70534690 learned_addr learned my addr 172.21.15.6:0/70534690 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2cae2f9640 1 --2- 172.21.15.6:0/70534690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca80643a0 0x7f2ca8114eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2cadaf8640 1 -- 172.21.15.6:0/70534690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca80643a0 msgr2=0x7f2ca8114eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2cadaf8640 1 --2- 172.21.15.6:0/70534690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca80643a0 0x7f2ca8114eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2cadaf8640 1 -- 172.21.15.6:0/70534690 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c98005040 con 0x7f2ca8064cb0 2024-03-22T06:26:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2cadaf8640 1 --2- 172.21.15.6:0/70534690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca8064cb0 0x7f2ca81153f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c98004930 tx=0x7f2c98009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2c9effd640 1 -- 172.21.15.6:0/70534690 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c980094c0 con 0x7f2ca8064cb0 2024-03-22T06:26:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2cb055d640 1 -- 172.21.15.6:0/70534690 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca810f2c0 con 0x7f2ca8064cb0 2024-03-22T06:26:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2c9effd640 1 -- 172.21.15.6:0/70534690 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c98032e50 con 0x7f2ca8064cb0 2024-03-22T06:26:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2c9effd640 1 -- 172.21.15.6:0/70534690 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c980433b0 con 0x7f2ca8064cb0 2024-03-22T06:26:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.275+0000 7f2cb055d640 1 -- 172.21.15.6:0/70534690 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca810f780 con 0x7f2ca8064cb0 2024-03-22T06:26:55.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.277+0000 7f2cb055d640 1 -- 172.21.15.6:0/70534690 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca80fb730 con 0x7f2ca8064cb0 2024-03-22T06:26:55.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.277+0000 7f2c9effd640 1 -- 172.21.15.6:0/70534690 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f2c9802f070 con 0x7f2ca8064cb0 2024-03-22T06:26:55.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.277+0000 7f2c9effd640 1 --2- 172.21.15.6:0/70534690 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2c9406e7b0 0x7f2c94070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:55.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.277+0000 7f2c9effd640 1 -- 172.21.15.6:0/70534690 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f2c980b0410 con 0x7f2ca8064cb0 2024-03-22T06:26:55.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.277+0000 7f2cae2f9640 1 --2- 172.21.15.6:0/70534690 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2c9406e7b0 0x7f2c94070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:55.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.278+0000 7f2cae2f9640 1 --2- 172.21.15.6:0/70534690 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2c9406e7b0 0x7f2c94070c60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009fd0 tx=0x7f2ca400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:55.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.282+0000 7f2c9effd640 1 -- 172.21.15.6:0/70534690 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f2c98079230 con 0x7f2ca8064cb0 2024-03-22T06:26:55.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.610+0000 7f2cb055d640 1 -- 172.21.15.6:0/70534690 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ca810b4f0 con 0x7f2ca8064cb0 2024-03-22T06:26:55.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.611+0000 7f2c9effd640 1 -- 172.21.15.6:0/70534690 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+17295 (secure 0 0 0) 0x7f2c9807c8f0 con 0x7f2ca8064cb0 2024-03-22T06:26:55.612 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:55.612 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":95,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","created":"2024-03-22T06:05:27.379207+0000","modified":"2024-03-22T06:23:28.644346+0000","last_up_change":"2024-03-22T06:22:07.841620+0000","last_in_change":"2024-03-22T06:10:39.982933+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T06:09:46.449392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T06:12:19.302857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T06:12:19.628730+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ecd88bb1-d643-4505-a098-3728178c89e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":90,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6803","nonce":1666131505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6805","nonce":1666131505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6809","nonce":1666131505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6807","nonce":1666131505}]},"public_addr":"172.21.15.6:6803/1666131505","cluster_addr":"172.21.15.6:6805/1666131505","heartbeat_back_addr":"172.21.15.6:6809/1666131505","heartbeat_front_addr":"172.21.15.6:6807/1666131505","state":["exists","up"]},{"osd":1,"uuid":"f1b55307-2c3b-4ab8-a893-af8072f333ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":66,"up_thru":90,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6811","nonce":1922945925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6813","nonce":1922945925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6817","nonce":1922945925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6815","nonce":1922945925}]},"public_addr":"172.21.15.6:6811/1922945925","cluster_addr":"172.21.15.6:6813/1922945925","heartbeat_back_addr":"172.21.15.6:6817/1922945925","heartbeat_front_addr":"172.21.15.6:6815/1922945925","state":["exists","up"]},{"osd":2,"uuid":"9bacfe42-3c78-4d86-bac0-0142cde76ff5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6819","nonce":195584623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6821","nonce":195584623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6825","nonce":195584623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6823","nonce":195584623}]},"public_addr":"172.21.15.6:6819/195584623","cluster_addr":"172.21.15.6:6821/195584623","heartbeat_back_addr":"172.21.15.6:6825/195584623","heartbeat_front_addr":"172.21.15.6:6823/195584623","state":["exists","up"]},{"osd":3,"uuid":"2c5a6acd-de41-455d-b6c5-d53bb76197ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6801","nonce":1104353694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6803","nonce":1104353694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6807","nonce":1104353694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6805","nonce":1104353694}]},"public_addr":"172.21.15.117:6801/1104353694","cluster_addr":"172.21.15.117:6803/1104353694","heartbeat_back_addr":"172.21.15.117:6807/1104353694","heartbeat_front_addr":"172.21.15.117:6805/1104353694","state":["exists","up"]},{"osd":4,"uuid":"be424041-1e07-41f4-abe0-68fdd21b1eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6809","nonce":208554150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6811","nonce":208554150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6815","nonce":208554150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6813","nonce":208554150}]},"public_addr":"172.21.15.117:6809/208554150","cluster_addr":"172.21.15.117:6811/208554150","heartbeat_back_addr":"172.21.15.117:6815/208554150","heartbeat_front_addr":"172.21.15.117:6813/208554150","state":["exists","up"]},{"osd":5,"uuid":"bf61ff67-1f07-47ed-ad58-e6c946369d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6817","nonce":1416150897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6819","nonce":1416150897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6823","nonce":1416150897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6821","nonce":1416150897}]},"public_addr":"172.21.15.117:6817/1416150897","cluster_addr":"172.21.15.117:6819/1416150897","heartbeat_back_addr":"172.21.15.117:6823/1416150897","heartbeat_front_addr":"172.21.15.117:6821/1416150897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:08:45.922485+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:13.118065+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:42.409903+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:05.760002+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:31.339234+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:57.286867+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":0,"to":2}]},{"pgid":"2.7","mappings":[{"from":3,"to":1}]},{"pgid":"2.b","mappings":[{"from":3,"to":1}]},{"pgid":"2.e","mappings":[{"from":0,"to":1}]},{"pgid":"2.12","mappings":[{"from":0,"to":2}]},{"pgid":"2.16","mappings":[{"from":3,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":4,"to":0}]},{"pgid":"3.1d","mappings":[{"from":4,"to":0}]},{"pgid":"3.1f","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6827/1010515354":"2024-03-23T06:23:28.644314+0000","172.21.15.117:6826/1010515354":"2024-03-23T06:23:28.644314+0000","172.21.15.117:6825/1251227895":"2024-03-23T06:23:03.662256+0000","172.21.15.6:6829/947853733":"2024-03-23T06:22:41.442153+0000","172.21.15.6:6828/947853733":"2024-03-23T06:22:41.442153+0000","172.21.15.6:6827/1260089028":"2024-03-23T06:22:13.008321+0000","172.21.15.117:6829/1963654099":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/2826360973":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6800/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:6801/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2042811963":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6801/602685778":"2024-03-23T06:14:38.572050+0000","172.21.15.6:6801/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.117:0/45595122":"2024-03-23T06:16:28.822651+0000","172.21.15.117:6824/1251227895":"2024-03-23T06:23:03.662256+0000","172.21.15.6:0/1533067608":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/3502710787":"2024-03-23T06:17:37.246941+0000","172.21.15.6:6800/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/791727884":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/3606143293":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/254878217":"2024-03-23T06:14:38.572050+0000","172.21.15.6:0/3489834288":"2024-03-23T06:07:32.107836+0000","172.21.15.117:0/3344206523":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/2742769039":"2024-03-23T06:14:38.572050+0000","172.21.15.6:6801/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.117:0/435100846":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/1354987702":"2024-03-23T06:14:38.572050+0000","172.21.15.6:0/2939515329":"2024-03-23T06:05:54.100943+0000","172.21.15.117:0/3477422362":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/3665872066":"2024-03-23T06:06:24.832062+0000","172.21.15.6:0/619515244":"2024-03-23T06:05:54.100943+0000","172.21.15.6:6801/3502710787":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/1270793675":"2024-03-23T06:14:38.572050+0000","172.21.15.117:6828/1963654099":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/5416477":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/3809759464":"2024-03-23T06:17:37.246941+0000","172.21.15.6:6800/602685778":"2024-03-23T06:14:38.572050+0000","172.21.15.6:0/4154587507":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/1448591806":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/2671132128":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/1248395428":"2024-03-23T06:17:37.246941+0000","172.21.15.6:6826/1260089028":"2024-03-23T06:22:13.008321+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T06:26:55.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.614+0000 7f2cb055d640 1 -- 172.21.15.6:0/70534690 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2c9406e7b0 msgr2=0x7f2c94070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:55.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.614+0000 7f2cb055d640 1 --2- 172.21.15.6:0/70534690 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2c9406e7b0 0x7f2c94070c60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009fd0 tx=0x7f2ca400b040 comp rx=0 tx=0).stop 2024-03-22T06:26:55.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.614+0000 7f2cb055d640 1 -- 172.21.15.6:0/70534690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca8064cb0 msgr2=0x7f2ca81153f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:55.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.614+0000 7f2cb055d640 1 --2- 172.21.15.6:0/70534690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca8064cb0 0x7f2ca81153f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c98004930 tx=0x7f2c98009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.614+0000 7f2cb055d640 1 -- 172.21.15.6:0/70534690 shutdown_connections 2024-03-22T06:26:55.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.614+0000 7f2cb055d640 1 --2- 172.21.15.6:0/70534690 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2c9406e7b0 0x7f2c94070c60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.614+0000 7f2cb055d640 1 --2- 172.21.15.6:0/70534690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca8064cb0 0x7f2ca81153f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.614+0000 7f2cb055d640 1 --2- 172.21.15.6:0/70534690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca80643a0 0x7f2ca8114eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.614+0000 7f2cb055d640 1 -- 172.21.15.6:0/70534690 >> 172.21.15.6:0/70534690 conn(0x7f2ca8071130 msgr2=0x7f2ca80f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:55.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.614+0000 7f2cb055d640 1 -- 172.21.15.6:0/70534690 shutdown_connections 2024-03-22T06:26:55.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.614+0000 7f2cb055d640 1 -- 172.21.15.6:0/70534690 wait complete. 2024-03-22T06:26:55.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T06:09:46.449392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T06:12:19.302857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T06:12:19.628730+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T06:26:55.627 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T06:26:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.731+0000 7ff00106e640 1 -- 172.21.15.6:0/2719166840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feffc102250 msgr2=0x7feffc102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.731+0000 7ff00106e640 1 --2- 172.21.15.6:0/2719166840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feffc102250 0x7feffc102620 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fefec009c50 tx=0x7fefec004d10 comp rx=0 tx=0).stop 2024-03-22T06:26:55.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.731+0000 7ff00106e640 1 -- 172.21.15.6:0/2719166840 shutdown_connections 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.731+0000 7ff00106e640 1 --2- 172.21.15.6:0/2719166840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feffc102bf0 0x7feffc0fc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.731+0000 7ff00106e640 1 --2- 172.21.15.6:0/2719166840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feffc102250 0x7feffc102620 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.731+0000 7ff00106e640 1 -- 172.21.15.6:0/2719166840 >> 172.21.15.6:0/2719166840 conn(0x7feffc0f7e60 msgr2=0x7feffc0fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.731+0000 7ff00106e640 1 -- 172.21.15.6:0/2719166840 shutdown_connections 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.731+0000 7ff00106e640 1 -- 172.21.15.6:0/2719166840 wait complete. 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.732+0000 7ff00106e640 1 Processor -- start 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.732+0000 7ff00106e640 1 -- start start 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.732+0000 7ff00106e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feffc102250 0x7feffc06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.732+0000 7ff00106e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feffc102bf0 0x7feffc06fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.732+0000 7ff00106e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc0716b0 con 0x7feffc102250 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.732+0000 7ff00106e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc070250 con 0x7feffc102bf0 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.732+0000 7feffad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feffc102250 0x7feffc06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.732+0000 7feffad9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feffc102250 0x7feffc06f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35334/0 (socket says 172.21.15.6:35334) 2024-03-22T06:26:55.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.732+0000 7feffad9d640 1 -- 172.21.15.6:0/4093470048 learned_addr learned my addr 172.21.15.6:0/4093470048 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:55.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.733+0000 7feffa59c640 1 --2- 172.21.15.6:0/4093470048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feffc102bf0 0x7feffc06fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:55.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.733+0000 7feffad9d640 1 -- 172.21.15.6:0/4093470048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feffc102bf0 msgr2=0x7feffc06fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:55.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.733+0000 7feffad9d640 1 --2- 172.21.15.6:0/4093470048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feffc102bf0 0x7feffc06fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.733+0000 7feffad9d640 1 -- 172.21.15.6:0/4093470048 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec0098b0 con 0x7feffc102250 2024-03-22T06:26:55.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.733+0000 7feffad9d640 1 --2- 172.21.15.6:0/4093470048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feffc102250 0x7feffc06f7d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fefec004f80 tx=0x7fefec004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:55.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.733+0000 7fefe37fe640 1 -- 172.21.15.6:0/4093470048 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefec03a070 con 0x7feffc102250 2024-03-22T06:26:55.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.733+0000 7ff00106e640 1 -- 172.21.15.6:0/4093470048 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feffc0704d0 con 0x7feffc102250 2024-03-22T06:26:55.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.733+0000 7fefe37fe640 1 -- 172.21.15.6:0/4093470048 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefec009740 con 0x7feffc102250 2024-03-22T06:26:55.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.733+0000 7fefe37fe640 1 -- 172.21.15.6:0/4093470048 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefec03ea60 con 0x7feffc102250 2024-03-22T06:26:55.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.733+0000 7ff00106e640 1 -- 172.21.15.6:0/4093470048 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc119400 con 0x7feffc102250 2024-03-22T06:26:55.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.734+0000 7ff00106e640 1 -- 172.21.15.6:0/4093470048 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feffc0fd9c0 con 0x7feffc102250 2024-03-22T06:26:55.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.735+0000 7fefe37fe640 1 -- 172.21.15.6:0/4093470048 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fefec032070 con 0x7feffc102250 2024-03-22T06:26:55.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.735+0000 7fefe37fe640 1 --2- 172.21.15.6:0/4093470048 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fefe406e7b0 0x7fefe4070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:55.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.735+0000 7fefe37fe640 1 -- 172.21.15.6:0/4093470048 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fefec0b0690 con 0x7feffc102250 2024-03-22T06:26:55.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.735+0000 7feffa59c640 1 --2- 172.21.15.6:0/4093470048 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fefe406e7b0 0x7fefe4070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:55.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.736+0000 7feffa59c640 1 --2- 172.21.15.6:0/4093470048 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fefe406e7b0 0x7fefe4070c60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feffc070f00 tx=0x7feff000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:55.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.740+0000 7fefe37fe640 1 -- 172.21.15.6:0/4093470048 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fefec079510 con 0x7feffc102250 2024-03-22T06:26:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.928+0000 7ff00106e640 1 -- 172.21.15.6:0/4093470048 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feffc04bcc0 con 0x7feffc102250 2024-03-22T06:26:55.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.928+0000 7fefe37fe640 1 -- 172.21.15.6:0/4093470048 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7feffc04bcc0 con 0x7feffc102250 2024-03-22T06:26:55.928 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-03-22T06:26:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.930+0000 7ff00106e640 1 -- 172.21.15.6:0/4093470048 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fefe406e7b0 msgr2=0x7fefe4070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.930+0000 7ff00106e640 1 --2- 172.21.15.6:0/4093470048 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fefe406e7b0 0x7fefe4070c60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feffc070f00 tx=0x7feff000b040 comp rx=0 tx=0).stop 2024-03-22T06:26:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.930+0000 7ff00106e640 1 -- 172.21.15.6:0/4093470048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feffc102250 msgr2=0x7feffc06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.930+0000 7ff00106e640 1 --2- 172.21.15.6:0/4093470048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feffc102250 0x7feffc06f7d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fefec004f80 tx=0x7fefec004d10 comp rx=0 tx=0).stop 2024-03-22T06:26:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.930+0000 7ff00106e640 1 -- 172.21.15.6:0/4093470048 shutdown_connections 2024-03-22T06:26:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.930+0000 7ff00106e640 1 --2- 172.21.15.6:0/4093470048 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fefe406e7b0 0x7fefe4070c60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.930+0000 7ff00106e640 1 --2- 172.21.15.6:0/4093470048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feffc102bf0 0x7feffc06fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.930+0000 7ff00106e640 1 --2- 172.21.15.6:0/4093470048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feffc102250 0x7feffc06f7d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.930+0000 7ff00106e640 1 -- 172.21.15.6:0/4093470048 >> 172.21.15.6:0/4093470048 conn(0x7feffc0f7e60 msgr2=0x7feffc0f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.930+0000 7ff00106e640 1 -- 172.21.15.6:0/4093470048 shutdown_connections 2024-03-22T06:26:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:55.930+0000 7ff00106e640 1 -- 172.21.15.6:0/4093470048 wait complete. 2024-03-22T06:26:55.941 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T06:26:56.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.051+0000 7fb05ef5d640 1 -- 172.21.15.6:0/1872613611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0580fb110 msgr2=0x7fb0580fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.051+0000 7fb05ef5d640 1 --2- 172.21.15.6:0/1872613611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0580fb110 0x7fb0580fb560 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0048d0 tx=0x7fb04c009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.051+0000 7fb05ef5d640 1 -- 172.21.15.6:0/1872613611 shutdown_connections 2024-03-22T06:26:56.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.051+0000 7fb05ef5d640 1 --2- 172.21.15.6:0/1872613611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0580fb110 0x7fb0580fb560 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.051+0000 7fb05ef5d640 1 --2- 172.21.15.6:0/1872613611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb058101140 0x7fb058101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.051+0000 7fb05ef5d640 1 -- 172.21.15.6:0/1872613611 >> 172.21.15.6:0/1872613611 conn(0x7fb0580710b0 msgr2=0x7fb0580714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:56.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.051+0000 7fb05ef5d640 1 -- 172.21.15.6:0/1872613611 shutdown_connections 2024-03-22T06:26:56.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.051+0000 7fb05ef5d640 1 -- 172.21.15.6:0/1872613611 wait complete. 2024-03-22T06:26:56.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.051+0000 7fb05ef5d640 1 Processor -- start 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.051+0000 7fb05ef5d640 1 -- start start 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.052+0000 7fb05ef5d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0580fb110 0x7fb058111b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.052+0000 7fb05ef5d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb058101140 0x7fb058112090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.052+0000 7fb05ef5d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb058112690 con 0x7fb0580fb110 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.052+0000 7fb05ef5d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb058112800 con 0x7fb058101140 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.052+0000 7fb05ccf9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0580fb110 0x7fb058111b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.052+0000 7fb05ccf9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0580fb110 0x7fb058111b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35346/0 (socket says 172.21.15.6:35346) 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.052+0000 7fb05ccf9640 1 -- 172.21.15.6:0/2592790208 learned_addr learned my addr 172.21.15.6:0/2592790208 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.052+0000 7fb057fff640 1 --2- 172.21.15.6:0/2592790208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb058101140 0x7fb058112090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.052+0000 7fb05ccf9640 1 -- 172.21.15.6:0/2592790208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb058101140 msgr2=0x7fb058112090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.052+0000 7fb05ccf9640 1 --2- 172.21.15.6:0/2592790208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb058101140 0x7fb058112090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.052+0000 7fb05ccf9640 1 -- 172.21.15.6:0/2592790208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c005040 con 0x7fb0580fb110 2024-03-22T06:26:56.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.053+0000 7fb05ccf9640 1 --2- 172.21.15.6:0/2592790208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0580fb110 0x7fb058111b50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb05000ed20 tx=0x7fb050007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:56.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.053+0000 7fb0557fa640 1 -- 172.21.15.6:0/2592790208 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb050004020 con 0x7fb0580fb110 2024-03-22T06:26:56.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.053+0000 7fb0557fa640 1 -- 172.21.15.6:0/2592790208 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb050004180 con 0x7fb0580fb110 2024-03-22T06:26:56.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.053+0000 7fb0557fa640 1 -- 172.21.15.6:0/2592790208 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb050019740 con 0x7fb0580fb110 2024-03-22T06:26:56.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.053+0000 7fb05ef5d640 1 -- 172.21.15.6:0/2592790208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05810bf00 con 0x7fb0580fb110 2024-03-22T06:26:56.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.053+0000 7fb05ef5d640 1 -- 172.21.15.6:0/2592790208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05810c420 con 0x7fb0580fb110 2024-03-22T06:26:56.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.054+0000 7fb05ef5d640 1 -- 172.21.15.6:0/2592790208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0580fc800 con 0x7fb0580fb110 2024-03-22T06:26:56.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.054+0000 7fb0557fa640 1 -- 172.21.15.6:0/2592790208 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fb050011ad0 con 0x7fb0580fb110 2024-03-22T06:26:56.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.055+0000 7fb0557fa640 1 --2- 172.21.15.6:0/2592790208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb04006e7b0 0x7fb040070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:56.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.055+0000 7fb0557fa640 1 -- 172.21.15.6:0/2592790208 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fb050015070 con 0x7fb0580fb110 2024-03-22T06:26:56.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.055+0000 7fb057fff640 1 --2- 172.21.15.6:0/2592790208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb04006e7b0 0x7fb040070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:56.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.055+0000 7fb057fff640 1 --2- 172.21.15.6:0/2592790208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb04006e7b0 0x7fb040070c60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb058100110 tx=0x7fb04c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:56.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.062+0000 7fb0557fa640 1 -- 172.21.15.6:0/2592790208 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fb05005a020 con 0x7fb0580fb110 2024-03-22T06:26:56.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.249+0000 7fb05ef5d640 1 -- 172.21.15.6:0/2592790208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fb0581082b0 con 0x7fb0580fb110 2024-03-22T06:26:56.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.249+0000 7fb0557fa640 1 -- 172.21.15.6:0/2592790208 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb05005d6e0 con 0x7fb0580fb110 2024-03-22T06:26:56.249 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-03-22T06:26:56.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.251+0000 7fb05ef5d640 1 -- 172.21.15.6:0/2592790208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb04006e7b0 msgr2=0x7fb040070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.251+0000 7fb05ef5d640 1 --2- 172.21.15.6:0/2592790208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb04006e7b0 0x7fb040070c60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb058100110 tx=0x7fb04c009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.251+0000 7fb05ef5d640 1 -- 172.21.15.6:0/2592790208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0580fb110 msgr2=0x7fb058111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.251+0000 7fb05ef5d640 1 --2- 172.21.15.6:0/2592790208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0580fb110 0x7fb058111b50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb05000ed20 tx=0x7fb050007f70 comp rx=0 tx=0).stop 2024-03-22T06:26:56.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.252+0000 7fb05ef5d640 1 -- 172.21.15.6:0/2592790208 shutdown_connections 2024-03-22T06:26:56.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.252+0000 7fb05ef5d640 1 --2- 172.21.15.6:0/2592790208 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fb04006e7b0 0x7fb040070c60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.252+0000 7fb05ef5d640 1 --2- 172.21.15.6:0/2592790208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb058101140 0x7fb058112090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.252+0000 7fb05ef5d640 1 --2- 172.21.15.6:0/2592790208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0580fb110 0x7fb058111b50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.252+0000 7fb05ef5d640 1 -- 172.21.15.6:0/2592790208 >> 172.21.15.6:0/2592790208 conn(0x7fb0580710b0 msgr2=0x7fb0580f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:56.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.252+0000 7fb05ef5d640 1 -- 172.21.15.6:0/2592790208 shutdown_connections 2024-03-22T06:26:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.252+0000 7fb05ef5d640 1 -- 172.21.15.6:0/2592790208 wait complete. 2024-03-22T06:26:56.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:55 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/70534690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:26:56.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:55 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/4093470048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T06:26:56.261 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:55 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/70534690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:26:56.262 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:55 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/4093470048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T06:26:56.263 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T06:26:56.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.366+0000 7f6cebb9d640 1 -- 172.21.15.6:0/3806508873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce4102240 msgr2=0x7f6ce4102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.366+0000 7f6cebb9d640 1 --2- 172.21.15.6:0/3806508873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce4102240 0x7f6ce4102610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00048d0 tx=0x7f6ce0009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.366+0000 7f6cebb9d640 1 -- 172.21.15.6:0/3806508873 shutdown_connections 2024-03-22T06:26:56.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.366+0000 7f6cebb9d640 1 --2- 172.21.15.6:0/3806508873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce40643a0 0x7f6ce40647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.366+0000 7f6cebb9d640 1 --2- 172.21.15.6:0/3806508873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce4102240 0x7f6ce4102610 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.366+0000 7f6cebb9d640 1 -- 172.21.15.6:0/3806508873 >> 172.21.15.6:0/3806508873 conn(0x7f6ce40710b0 msgr2=0x7f6ce40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:56.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6cebb9d640 1 -- 172.21.15.6:0/3806508873 shutdown_connections 2024-03-22T06:26:56.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6cebb9d640 1 -- 172.21.15.6:0/3806508873 wait complete. 2024-03-22T06:26:56.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6cebb9d640 1 Processor -- start 2024-03-22T06:26:56.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6cebb9d640 1 -- start start 2024-03-22T06:26:56.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6cebb9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce40643a0 0x7f6ce4112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:56.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6cebb9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4102240 0x7f6ce4113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:56.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6cebb9d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce4113850 con 0x7f6ce40643a0 2024-03-22T06:26:56.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6cebb9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce410cea0 con 0x7f6ce4102240 2024-03-22T06:26:56.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6ce9939640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce40643a0 0x7f6ce4112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:56.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6ce9939640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce40643a0 0x7f6ce4112d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35370/0 (socket says 172.21.15.6:35370) 2024-03-22T06:26:56.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6ce9939640 1 -- 172.21.15.6:0/332004953 learned_addr learned my addr 172.21.15.6:0/332004953 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:56.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.367+0000 7f6ce9138640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4102240 0x7f6ce4113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:56.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.368+0000 7f6ce9939640 1 -- 172.21.15.6:0/332004953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4102240 msgr2=0x7f6ce4113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.368+0000 7f6ce9939640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4102240 0x7f6ce4113250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.368+0000 7f6ce9939640 1 -- 172.21.15.6:0/332004953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce0005040 con 0x7f6ce40643a0 2024-03-22T06:26:56.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.368+0000 7f6ce9138640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4102240 0x7f6ce4113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:26:56.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.368+0000 7f6ce9939640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce40643a0 0x7f6ce4112d10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0004930 tx=0x7f6ce0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:56.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.368+0000 7f6cda7fc640 1 -- 172.21.15.6:0/332004953 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ce00094c0 con 0x7f6ce40643a0 2024-03-22T06:26:56.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.368+0000 7f6cda7fc640 1 -- 172.21.15.6:0/332004953 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ce0032e50 con 0x7f6ce40643a0 2024-03-22T06:26:56.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.368+0000 7f6cebb9d640 1 -- 172.21.15.6:0/332004953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce410d120 con 0x7f6ce40643a0 2024-03-22T06:26:56.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.368+0000 7f6cda7fc640 1 -- 172.21.15.6:0/332004953 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ce00433b0 con 0x7f6ce40643a0 2024-03-22T06:26:56.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.368+0000 7f6cebb9d640 1 -- 172.21.15.6:0/332004953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce410d5e0 con 0x7f6ce40643a0 2024-03-22T06:26:56.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.369+0000 7f6cebb9d640 1 -- 172.21.15.6:0/332004953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce40fd940 con 0x7f6ce40643a0 2024-03-22T06:26:56.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.370+0000 7f6cda7fc640 1 -- 172.21.15.6:0/332004953 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f6ce002f070 con 0x7f6ce40643a0 2024-03-22T06:26:56.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.370+0000 7f6cda7fc640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6cd006e7b0 0x7f6cd0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:56.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.370+0000 7f6cda7fc640 1 -- 172.21.15.6:0/332004953 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f6ce00b0590 con 0x7f6ce40643a0 2024-03-22T06:26:56.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.370+0000 7f6ce9138640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6cd006e7b0 0x7f6cd0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:56.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.371+0000 7f6ce9138640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6cd006e7b0 0x7f6cd0070c60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ce410e5d0 tx=0x7f6cd4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:56.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.375+0000 7f6cda7fc640 1 -- 172.21.15.6:0/332004953 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f6ce0079360 con 0x7f6ce40643a0 2024-03-22T06:26:56.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.561+0000 7f6cebb9d640 1 -- 172.21.15.6:0/332004953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f6ce410e380 con 0x7f6ce40643a0 2024-03-22T06:26:56.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.562+0000 7f6cda7fc640 1 -- 172.21.15.6:0/332004953 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7f6ce007ca20 con 0x7f6ce40643a0 2024-03-22T06:26:56.562 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-03-22T06:26:56.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.564+0000 7f6cebb9d640 1 -- 172.21.15.6:0/332004953 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6cd006e7b0 msgr2=0x7f6cd0070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.564+0000 7f6cebb9d640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6cd006e7b0 0x7f6cd0070c60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ce410e5d0 tx=0x7f6cd4009040 comp rx=0 tx=0).stop 2024-03-22T06:26:56.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.564+0000 7f6cebb9d640 1 -- 172.21.15.6:0/332004953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce40643a0 msgr2=0x7f6ce4112d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.564+0000 7f6cebb9d640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce40643a0 0x7f6ce4112d10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0004930 tx=0x7f6ce0009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.564+0000 7f6cebb9d640 1 -- 172.21.15.6:0/332004953 shutdown_connections 2024-03-22T06:26:56.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.564+0000 7f6cebb9d640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f6cd006e7b0 0x7f6cd0070c60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.564+0000 7f6cebb9d640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4102240 0x7f6ce4113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.564+0000 7f6cebb9d640 1 --2- 172.21.15.6:0/332004953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce40643a0 0x7f6ce4112d10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.564+0000 7f6cebb9d640 1 -- 172.21.15.6:0/332004953 >> 172.21.15.6:0/332004953 conn(0x7f6ce40710b0 msgr2=0x7f6ce40f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:56.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.564+0000 7f6cebb9d640 1 -- 172.21.15.6:0/332004953 shutdown_connections 2024-03-22T06:26:56.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.564+0000 7f6cebb9d640 1 -- 172.21.15.6:0/332004953 wait complete. 2024-03-22T06:26:56.575 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T06:26:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.675+0000 7f00d631d640 1 -- 172.21.15.6:0/1583685098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00d00643a0 msgr2=0x7f00d00647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.675+0000 7f00d631d640 1 --2- 172.21.15.6:0/1583685098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00d00643a0 0x7f00d00647d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f00c40048d0 tx=0x7f00c4009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.675+0000 7f00d631d640 1 -- 172.21.15.6:0/1583685098 shutdown_connections 2024-03-22T06:26:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.675+0000 7f00d631d640 1 --2- 172.21.15.6:0/1583685098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00d00643a0 0x7f00d00647d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.675+0000 7f00d631d640 1 --2- 172.21.15.6:0/1583685098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d0102240 0x7f00d0102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.675+0000 7f00d631d640 1 -- 172.21.15.6:0/1583685098 >> 172.21.15.6:0/1583685098 conn(0x7f00d0071130 msgr2=0x7f00d0071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.675+0000 7f00d631d640 1 -- 172.21.15.6:0/1583685098 shutdown_connections 2024-03-22T06:26:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.675+0000 7f00d631d640 1 -- 172.21.15.6:0/1583685098 wait complete. 2024-03-22T06:26:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.675+0000 7f00d631d640 1 Processor -- start 2024-03-22T06:26:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.675+0000 7f00d631d640 1 -- start start 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00d631d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d00643a0 0x7f00d0112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00d631d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00d0102240 0x7f00d0113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00d631d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d0113820 con 0x7f00d0102240 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00d631d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d010ce70 con 0x7f00d00643a0 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00cf7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00d0102240 0x7f00d0113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00cf7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00d0102240 0x7f00d0113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35386/0 (socket says 172.21.15.6:35386) 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00cf7fe640 1 -- 172.21.15.6:0/469337124 learned_addr learned my addr 172.21.15.6:0/469337124 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00cffff640 1 --2- 172.21.15.6:0/469337124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d00643a0 0x7f00d0112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00cf7fe640 1 -- 172.21.15.6:0/469337124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d00643a0 msgr2=0x7f00d0112ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00cf7fe640 1 --2- 172.21.15.6:0/469337124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d00643a0 0x7f00d0112ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00cf7fe640 1 -- 172.21.15.6:0/469337124 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c4005040 con 0x7f00d0102240 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00cf7fe640 1 --2- 172.21.15.6:0/469337124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00d0102240 0x7f00d0113220 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f00c4034040 tx=0x7f00c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00ccff9640 1 -- 172.21.15.6:0/469337124 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00c40092b0 con 0x7f00d0102240 2024-03-22T06:26:56.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00ccff9640 1 -- 172.21.15.6:0/469337124 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00c403b400 con 0x7f00d0102240 2024-03-22T06:26:56.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.676+0000 7f00d631d640 1 -- 172.21.15.6:0/469337124 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d010d0f0 con 0x7f00d0102240 2024-03-22T06:26:56.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.677+0000 7f00ccff9640 1 -- 172.21.15.6:0/469337124 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00c40435d0 con 0x7f00d0102240 2024-03-22T06:26:56.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.677+0000 7f00d631d640 1 -- 172.21.15.6:0/469337124 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d010d5b0 con 0x7f00d0102240 2024-03-22T06:26:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.678+0000 7f00d631d640 1 -- 172.21.15.6:0/469337124 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d00fd940 con 0x7f00d0102240 2024-03-22T06:26:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.678+0000 7f00ccff9640 1 -- 172.21.15.6:0/469337124 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f00c402f070 con 0x7f00d0102240 2024-03-22T06:26:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.679+0000 7f00ccff9640 1 --2- 172.21.15.6:0/469337124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f00b806e7b0 0x7f00b8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.679+0000 7f00ccff9640 1 -- 172.21.15.6:0/469337124 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f00c40b02b0 con 0x7f00d0102240 2024-03-22T06:26:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.679+0000 7f00cffff640 1 --2- 172.21.15.6:0/469337124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f00b806e7b0 0x7f00b8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:56.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.679+0000 7f00cffff640 1 --2- 172.21.15.6:0/469337124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f00b806e7b0 0x7f00b8070c60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009af0 tx=0x7f00c000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:56.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.684+0000 7f00ccff9640 1 -- 172.21.15.6:0/469337124 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f00c4079130 con 0x7f00d0102240 2024-03-22T06:26:56.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.958+0000 7f00d631d640 1 -- 172.21.15.6:0/469337124 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00d0109320 con 0x7f00d0102240 2024-03-22T06:26:56.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.959+0000 7f00ccff9640 1 -- 172.21.15.6:0/469337124 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5034 (secure 0 0 0) 0x7f00c407c7f0 con 0x7f00d0102240 2024-03-22T06:26:56.959 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:56.959 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44335,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/1194667779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1194667779},{"type":"v1","addr":"172.21.15.117:6825","nonce":1194667779}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44339,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/2788388155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2788388155},{"type":"v1","addr":"172.21.15.117:6827","nonce":2788388155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:23:39.065820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":0,"incarnation":32,"state":"up:active","state_seq":22,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-03-22T06:26:56.960 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 36 2024-03-22T06:26:56.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.961+0000 7f00d631d640 1 -- 172.21.15.6:0/469337124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f00b806e7b0 msgr2=0x7f00b8070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.961+0000 7f00d631d640 1 --2- 172.21.15.6:0/469337124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f00b806e7b0 0x7f00b8070c60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009af0 tx=0x7f00c000b040 comp rx=0 tx=0).stop 2024-03-22T06:26:56.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.961+0000 7f00d631d640 1 -- 172.21.15.6:0/469337124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00d0102240 msgr2=0x7f00d0113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:56.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.961+0000 7f00d631d640 1 --2- 172.21.15.6:0/469337124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00d0102240 0x7f00d0113220 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f00c4034040 tx=0x7f00c4009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.961+0000 7f00d631d640 1 -- 172.21.15.6:0/469337124 shutdown_connections 2024-03-22T06:26:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.961+0000 7f00d631d640 1 --2- 172.21.15.6:0/469337124 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f00b806e7b0 0x7f00b8070c60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.961+0000 7f00d631d640 1 --2- 172.21.15.6:0/469337124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00d0102240 0x7f00d0113220 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.961+0000 7f00d631d640 1 --2- 172.21.15.6:0/469337124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d00643a0 0x7f00d0112ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.961+0000 7f00d631d640 1 -- 172.21.15.6:0/469337124 >> 172.21.15.6:0/469337124 conn(0x7f00d0071130 msgr2=0x7f00d00f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.961+0000 7f00d631d640 1 -- 172.21.15.6:0/469337124 shutdown_connections 2024-03-22T06:26:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:56.961+0000 7f00d631d640 1 -- 172.21.15.6:0/469337124 wait complete. 2024-03-22T06:26:56.973 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 14, 'max_mds': 1, 'flags': 18} 2024-03-22T06:26:56.974 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-03-22T06:26:57.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.076+0000 7fd5cee6e640 1 -- 172.21.15.6:0/425559239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5c8102b70 msgr2=0x7fd5c80fc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.076+0000 7fd5cee6e640 1 --2- 172.21.15.6:0/425559239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5c8102b70 0x7fd5c80fc200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40048d0 tx=0x7fd5b4009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.076+0000 7fd5cee6e640 1 -- 172.21.15.6:0/425559239 shutdown_connections 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.076+0000 7fd5cee6e640 1 --2- 172.21.15.6:0/425559239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5c8102b70 0x7fd5c80fc200 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.076+0000 7fd5cee6e640 1 --2- 172.21.15.6:0/425559239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c81021d0 0x7fd5c81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.076+0000 7fd5cee6e640 1 -- 172.21.15.6:0/425559239 >> 172.21.15.6:0/425559239 conn(0x7fd5c80f7dc0 msgr2=0x7fd5c80fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.076+0000 7fd5cee6e640 1 -- 172.21.15.6:0/425559239 shutdown_connections 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.076+0000 7fd5cee6e640 1 -- 172.21.15.6:0/425559239 wait complete. 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.076+0000 7fd5cee6e640 1 Processor -- start 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.076+0000 7fd5cee6e640 1 -- start start 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5cee6e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c81021d0 0x7fd5c8112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5cee6e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5c8102b70 0x7fd5c8113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:57.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5cee6e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c8113830 con 0x7fd5c8102b70 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5cee6e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c810ce80 con 0x7fd5c81021d0 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5c7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5c8102b70 0x7fd5c8113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5c7fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5c8102b70 0x7fd5c8113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35402/0 (socket says 172.21.15.6:35402) 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5c7fff640 1 -- 172.21.15.6:0/208778907 learned_addr learned my addr 172.21.15.6:0/208778907 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5ccc0a640 1 --2- 172.21.15.6:0/208778907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c81021d0 0x7fd5c8112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5c7fff640 1 -- 172.21.15.6:0/208778907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c81021d0 msgr2=0x7fd5c8112cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5c7fff640 1 --2- 172.21.15.6:0/208778907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c81021d0 0x7fd5c8112cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5c7fff640 1 -- 172.21.15.6:0/208778907 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b4005040 con 0x7fd5c8102b70 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5c7fff640 1 --2- 172.21.15.6:0/208778907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5c8102b70 0x7fd5c8113230 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4034040 tx=0x7fd5b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.077+0000 7fd5c57fa640 1 -- 172.21.15.6:0/208778907 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5b40092b0 con 0x7fd5c8102b70 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.078+0000 7fd5cee6e640 1 -- 172.21.15.6:0/208778907 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c810d100 con 0x7fd5c8102b70 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.078+0000 7fd5c57fa640 1 -- 172.21.15.6:0/208778907 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5b403b400 con 0x7fd5c8102b70 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.078+0000 7fd5c57fa640 1 -- 172.21.15.6:0/208778907 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5b40435d0 con 0x7fd5c8102b70 2024-03-22T06:26:57.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.078+0000 7fd5cee6e640 1 -- 172.21.15.6:0/208778907 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c810d5c0 con 0x7fd5c8102b70 2024-03-22T06:26:57.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.079+0000 7fd5cee6e640 1 -- 172.21.15.6:0/208778907 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c80fd940 con 0x7fd5c8102b70 2024-03-22T06:26:57.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.079+0000 7fd5c57fa640 1 -- 172.21.15.6:0/208778907 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fd5b402f070 con 0x7fd5c8102b70 2024-03-22T06:26:57.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.080+0000 7fd5c57fa640 1 --2- 172.21.15.6:0/208778907 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd5b806e7b0 0x7fd5b8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:57.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.080+0000 7fd5ccc0a640 1 --2- 172.21.15.6:0/208778907 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd5b806e7b0 0x7fd5b8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:57.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.080+0000 7fd5c57fa640 1 -- 172.21.15.6:0/208778907 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fd5b40b02b0 con 0x7fd5c8102b70 2024-03-22T06:26:57.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.080+0000 7fd5ccc0a640 1 --2- 172.21.15.6:0/208778907 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd5b806e7b0 0x7fd5b8070c60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009af0 tx=0x7fd5bc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:57.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.084+0000 7fd5c57fa640 1 -- 172.21.15.6:0/208778907 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fd5b4079130 con 0x7fd5c8102b70 2024-03-22T06:26:57.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:56 smithi117 ceph-mon[82126]: pgmap v351: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:57.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:56 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2592790208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T06:26:57.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:56 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/332004953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T06:26:57.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:56 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:26:57.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:56 smithi006 ceph-mon[94482]: pgmap v351: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:57.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:56 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2592790208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T06:26:57.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:56 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/332004953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T06:26:57.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:56 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T06:26:57.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.326+0000 7fd5cee6e640 1 -- 172.21.15.6:0/208778907 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7fd5c804bcc0 con 0x7fd5c8102b70 2024-03-22T06:26:57.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.327+0000 7fd5c57fa640 1 -- 172.21.15.6:0/208778907 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v36) v1 ==== 107+0+4067 (secure 0 0 0) 0x7fd5b407c7f0 con 0x7fd5c8102b70 2024-03-22T06:26:57.327 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:57.327 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14476,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.6:6829/947853733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":947853733},{"type":"v1","addr":"172.21.15.6:6829","nonce":947853733}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24269,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6825/1251227895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1251227895},{"type":"v1","addr":"172.21.15.117:6825","nonce":1251227895}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:22:13.008332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:26:57.327 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 15 2024-03-22T06:26:57.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.329+0000 7fd5cee6e640 1 -- 172.21.15.6:0/208778907 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd5b806e7b0 msgr2=0x7fd5b8070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:57.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.329+0000 7fd5cee6e640 1 --2- 172.21.15.6:0/208778907 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd5b806e7b0 0x7fd5b8070c60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009af0 tx=0x7fd5bc00b040 comp rx=0 tx=0).stop 2024-03-22T06:26:57.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.329+0000 7fd5cee6e640 1 -- 172.21.15.6:0/208778907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5c8102b70 msgr2=0x7fd5c8113230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:57.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.329+0000 7fd5cee6e640 1 --2- 172.21.15.6:0/208778907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5c8102b70 0x7fd5c8113230 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4034040 tx=0x7fd5b4009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.329+0000 7fd5cee6e640 1 -- 172.21.15.6:0/208778907 shutdown_connections 2024-03-22T06:26:57.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.329+0000 7fd5cee6e640 1 --2- 172.21.15.6:0/208778907 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd5b806e7b0 0x7fd5b8070c60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.329+0000 7fd5cee6e640 1 --2- 172.21.15.6:0/208778907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5c8102b70 0x7fd5c8113230 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.329+0000 7fd5cee6e640 1 --2- 172.21.15.6:0/208778907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c81021d0 0x7fd5c8112cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.329+0000 7fd5cee6e640 1 -- 172.21.15.6:0/208778907 >> 172.21.15.6:0/208778907 conn(0x7fd5c80f7dc0 msgr2=0x7fd5c80f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:57.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.329+0000 7fd5cee6e640 1 -- 172.21.15.6:0/208778907 shutdown_connections 2024-03-22T06:26:57.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.329+0000 7fd5cee6e640 1 -- 172.21.15.6:0/208778907 wait complete. 2024-03-22T06:26:57.341 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-03-22T06:26:57.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.446+0000 7fd586f14640 1 -- 172.21.15.6:0/308909431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5800fb110 msgr2=0x7fd5800fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:57.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.446+0000 7fd586f14640 1 --2- 172.21.15.6:0/308909431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5800fb110 0x7fd5800fb560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0048d0 tx=0x7fd56c009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.446+0000 7fd586f14640 1 -- 172.21.15.6:0/308909431 shutdown_connections 2024-03-22T06:26:57.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.446+0000 7fd586f14640 1 --2- 172.21.15.6:0/308909431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5800fb110 0x7fd5800fb560 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.446+0000 7fd586f14640 1 --2- 172.21.15.6:0/308909431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd580101140 0x7fd580101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.446+0000 7fd586f14640 1 -- 172.21.15.6:0/308909431 >> 172.21.15.6:0/308909431 conn(0x7fd5800710b0 msgr2=0x7fd5800714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:57.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.446+0000 7fd586f14640 1 -- 172.21.15.6:0/308909431 shutdown_connections 2024-03-22T06:26:57.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.446+0000 7fd586f14640 1 -- 172.21.15.6:0/308909431 wait complete. 2024-03-22T06:26:57.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.447+0000 7fd586f14640 1 Processor -- start 2024-03-22T06:26:57.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.447+0000 7fd586f14640 1 -- start start 2024-03-22T06:26:57.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.447+0000 7fd586f14640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5800fb110 0x7fd580112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:57.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.447+0000 7fd586f14640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd580101140 0x7fd580113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:57.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.447+0000 7fd586f14640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd580113820 con 0x7fd5800fb110 2024-03-22T06:26:57.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.447+0000 7fd586f14640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58010ce70 con 0x7fd580101140 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd584cb0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5800fb110 0x7fd580112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd584cb0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5800fb110 0x7fd580112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35426/0 (socket says 172.21.15.6:35426) 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd584cb0640 1 -- 172.21.15.6:0/825136378 learned_addr learned my addr 172.21.15.6:0/825136378 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd57ffff640 1 --2- 172.21.15.6:0/825136378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd580101140 0x7fd580113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd584cb0640 1 -- 172.21.15.6:0/825136378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd580101140 msgr2=0x7fd580113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd584cb0640 1 --2- 172.21.15.6:0/825136378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd580101140 0x7fd580113220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd584cb0640 1 -- 172.21.15.6:0/825136378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c005040 con 0x7fd5800fb110 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd584cb0640 1 --2- 172.21.15.6:0/825136378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5800fb110 0x7fd580112ce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd57400eb70 tx=0x7fd57400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd57d7fa640 1 -- 172.21.15.6:0/825136378 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd574007c30 con 0x7fd5800fb110 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd57d7fa640 1 -- 172.21.15.6:0/825136378 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd57400f040 con 0x7fd5800fb110 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd57d7fa640 1 -- 172.21.15.6:0/825136378 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5740095b0 con 0x7fd5800fb110 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd586f14640 1 -- 172.21.15.6:0/825136378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58010d0f0 con 0x7fd5800fb110 2024-03-22T06:26:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.448+0000 7fd586f14640 1 -- 172.21.15.6:0/825136378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58010d610 con 0x7fd5800fb110 2024-03-22T06:26:57.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.450+0000 7fd586f14640 1 -- 172.21.15.6:0/825136378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5800fc800 con 0x7fd5800fb110 2024-03-22T06:26:57.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.450+0000 7fd57d7fa640 1 -- 172.21.15.6:0/825136378 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fd574009710 con 0x7fd5800fb110 2024-03-22T06:26:57.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.450+0000 7fd57d7fa640 1 --2- 172.21.15.6:0/825136378 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd57006e7b0 0x7fd570070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:57.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.451+0000 7fd57d7fa640 1 -- 172.21.15.6:0/825136378 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fd57408e250 con 0x7fd5800fb110 2024-03-22T06:26:57.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.451+0000 7fd57ffff640 1 --2- 172.21.15.6:0/825136378 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd57006e7b0 0x7fd570070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:57.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.451+0000 7fd57ffff640 1 --2- 172.21.15.6:0/825136378 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd57006e7b0 0x7fd570070c60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd58010e5a0 tx=0x7fd56c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:57.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.455+0000 7fd57d7fa640 1 -- 172.21.15.6:0/825136378 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fd5740570a0 con 0x7fd5800fb110 2024-03-22T06:26:57.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.734+0000 7fd586f14640 1 -- 172.21.15.6:0/825136378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7fd58010e350 con 0x7fd5800fb110 2024-03-22T06:26:57.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.735+0000 7fd57d7fa640 1 -- 172.21.15.6:0/825136378 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v36) v1 ==== 107+0+4078 (secure 0 0 0) 0x7fd57405a760 con 0x7fd5800fb110 2024-03-22T06:26:57.735 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:57.735 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24269,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6825/1251227895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1251227895},{"type":"v1","addr":"172.21.15.117:6825","nonce":1251227895}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:22:13.013361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi006.dcjpla","rank":0,"incarnation":16,"state":"up:replay","state_seq":2,"addr":"172.21.15.6:6829/947853733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":947853733},{"type":"v1","addr":"172.21.15.6:6829","nonce":947853733}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:26:57.735 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 16 2024-03-22T06:26:57.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.737+0000 7fd586f14640 1 -- 172.21.15.6:0/825136378 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd57006e7b0 msgr2=0x7fd570070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:57.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.737+0000 7fd586f14640 1 --2- 172.21.15.6:0/825136378 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd57006e7b0 0x7fd570070c60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd58010e5a0 tx=0x7fd56c009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.737+0000 7fd586f14640 1 -- 172.21.15.6:0/825136378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5800fb110 msgr2=0x7fd580112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:57.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.737+0000 7fd586f14640 1 --2- 172.21.15.6:0/825136378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5800fb110 0x7fd580112ce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd57400eb70 tx=0x7fd57400b040 comp rx=0 tx=0).stop 2024-03-22T06:26:57.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.737+0000 7fd586f14640 1 -- 172.21.15.6:0/825136378 shutdown_connections 2024-03-22T06:26:57.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.737+0000 7fd586f14640 1 --2- 172.21.15.6:0/825136378 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd57006e7b0 0x7fd570070c60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.737+0000 7fd586f14640 1 --2- 172.21.15.6:0/825136378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd580101140 0x7fd580113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.737+0000 7fd586f14640 1 --2- 172.21.15.6:0/825136378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5800fb110 0x7fd580112ce0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.737+0000 7fd586f14640 1 -- 172.21.15.6:0/825136378 >> 172.21.15.6:0/825136378 conn(0x7fd5800710b0 msgr2=0x7fd5800f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.737+0000 7fd586f14640 1 -- 172.21.15.6:0/825136378 shutdown_connections 2024-03-22T06:26:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.737+0000 7fd586f14640 1 -- 172.21.15.6:0/825136378 wait complete. 2024-03-22T06:26:57.748 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-03-22T06:26:57.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.855+0000 7f0fc019d640 1 -- 172.21.15.6:0/2175642558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb8102b50 msgr2=0x7f0fb80fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:57.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.855+0000 7f0fc019d640 1 --2- 172.21.15.6:0/2175642558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb8102b50 0x7f0fb80fc1e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80048d0 tx=0x7f0fa8009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.855+0000 7f0fc019d640 1 -- 172.21.15.6:0/2175642558 shutdown_connections 2024-03-22T06:26:57.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.855+0000 7f0fc019d640 1 --2- 172.21.15.6:0/2175642558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb8102b50 0x7f0fb80fc1e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.855+0000 7f0fc019d640 1 --2- 172.21.15.6:0/2175642558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb81021b0 0x7f0fb8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.855+0000 7f0fc019d640 1 -- 172.21.15.6:0/2175642558 >> 172.21.15.6:0/2175642558 conn(0x7f0fb80f7dc0 msgr2=0x7f0fb80fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:57.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.855+0000 7f0fc019d640 1 -- 172.21.15.6:0/2175642558 shutdown_connections 2024-03-22T06:26:57.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.855+0000 7f0fc019d640 1 -- 172.21.15.6:0/2175642558 wait complete. 2024-03-22T06:26:57.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.856+0000 7f0fc019d640 1 Processor -- start 2024-03-22T06:26:57.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.856+0000 7f0fc019d640 1 -- start start 2024-03-22T06:26:57.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.856+0000 7f0fc019d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb81021b0 0x7f0fb8112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:57.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.856+0000 7f0fc019d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb8102b50 0x7f0fb8113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:57.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.856+0000 7f0fc019d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb8113800 con 0x7f0fb8102b50 2024-03-22T06:26:57.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.856+0000 7f0fc019d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb810ce50 con 0x7f0fb81021b0 2024-03-22T06:26:57.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.856+0000 7f0fbd738640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb8102b50 0x7f0fb8113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:57.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.856+0000 7f0fbd738640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb8102b50 0x7f0fb8113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35438/0 (socket says 172.21.15.6:35438) 2024-03-22T06:26:57.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.856+0000 7f0fbd738640 1 -- 172.21.15.6:0/1339392930 learned_addr learned my addr 172.21.15.6:0/1339392930 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:57.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.856+0000 7f0fbdf39640 1 --2- 172.21.15.6:0/1339392930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb81021b0 0x7f0fb8112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:57.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.857+0000 7f0fbd738640 1 -- 172.21.15.6:0/1339392930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb81021b0 msgr2=0x7f0fb8112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:57.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.857+0000 7f0fbd738640 1 --2- 172.21.15.6:0/1339392930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb81021b0 0x7f0fb8112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:57.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.857+0000 7f0fbd738640 1 -- 172.21.15.6:0/1339392930 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa8005040 con 0x7f0fb8102b50 2024-03-22T06:26:57.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.857+0000 7f0fbd738640 1 --2- 172.21.15.6:0/1339392930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb8102b50 0x7f0fb8113200 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8004930 tx=0x7f0fa8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:57.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.857+0000 7f0fae7fc640 1 -- 172.21.15.6:0/1339392930 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fa80094c0 con 0x7f0fb8102b50 2024-03-22T06:26:57.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.857+0000 7f0fae7fc640 1 -- 172.21.15.6:0/1339392930 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fa8032e50 con 0x7f0fb8102b50 2024-03-22T06:26:57.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.857+0000 7f0fae7fc640 1 -- 172.21.15.6:0/1339392930 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fa80433b0 con 0x7f0fb8102b50 2024-03-22T06:26:57.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.857+0000 7f0fc019d640 1 -- 172.21.15.6:0/1339392930 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb810d0d0 con 0x7f0fb8102b50 2024-03-22T06:26:57.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.857+0000 7f0fc019d640 1 -- 172.21.15.6:0/1339392930 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb810d590 con 0x7f0fb8102b50 2024-03-22T06:26:57.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.859+0000 7f0fc019d640 1 -- 172.21.15.6:0/1339392930 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb80fd920 con 0x7f0fb8102b50 2024-03-22T06:26:57.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.859+0000 7f0fae7fc640 1 -- 172.21.15.6:0/1339392930 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f0fa802f070 con 0x7f0fb8102b50 2024-03-22T06:26:57.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.859+0000 7f0fae7fc640 1 --2- 172.21.15.6:0/1339392930 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0fa406e7b0 0x7f0fa4070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:57.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.860+0000 7f0fae7fc640 1 -- 172.21.15.6:0/1339392930 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f0fa80afd10 con 0x7f0fb8102b50 2024-03-22T06:26:57.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.860+0000 7f0fbdf39640 1 --2- 172.21.15.6:0/1339392930 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0fa406e7b0 0x7f0fa4070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:57.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.860+0000 7f0fbdf39640 1 --2- 172.21.15.6:0/1339392930 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0fa406e7b0 0x7f0fa4070c60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4004ed0 tx=0x7f0fb400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:57.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:57.864+0000 7f0fae7fc640 1 -- 172.21.15.6:0/1339392930 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f0fa8078b90 con 0x7f0fb8102b50 2024-03-22T06:26:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:57 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/469337124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T06:26:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:26:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:26:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:57 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/208778907' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-03-22T06:26:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:57 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/825136378' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-03-22T06:26:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:26:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:26:58.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:57 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:26:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.114+0000 7f0fc019d640 1 -- 172.21.15.6:0/1339392930 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f0fb8109310 con 0x7f0fb8102b50 2024-03-22T06:26:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.114+0000 7f0fae7fc640 1 -- 172.21.15.6:0/1339392930 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v36) v1 ==== 107+0+4083 (secure 0 0 0) 0x7f0fa807c250 con 0x7f0fb8102b50 2024-03-22T06:26:58.115 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:58.115 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24269,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6825/1251227895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1251227895},{"type":"v1","addr":"172.21.15.117:6825","nonce":1251227895}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:22:21.040224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi006.dcjpla","rank":0,"incarnation":16,"state":"up:reconnect","state_seq":149,"addr":"172.21.15.6:6829/947853733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":947853733},{"type":"v1","addr":"172.21.15.6:6829","nonce":947853733}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:26:58.115 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 17 2024-03-22T06:26:58.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.116+0000 7f0fc019d640 1 -- 172.21.15.6:0/1339392930 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0fa406e7b0 msgr2=0x7f0fa4070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.116+0000 7f0fc019d640 1 --2- 172.21.15.6:0/1339392930 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0fa406e7b0 0x7f0fa4070c60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4004ed0 tx=0x7f0fb400b040 comp rx=0 tx=0).stop 2024-03-22T06:26:58.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.116+0000 7f0fc019d640 1 -- 172.21.15.6:0/1339392930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb8102b50 msgr2=0x7f0fb8113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.116+0000 7f0fc019d640 1 --2- 172.21.15.6:0/1339392930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb8102b50 0x7f0fb8113200 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8004930 tx=0x7f0fa8009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.116+0000 7f0fc019d640 1 -- 172.21.15.6:0/1339392930 shutdown_connections 2024-03-22T06:26:58.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.116+0000 7f0fc019d640 1 --2- 172.21.15.6:0/1339392930 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f0fa406e7b0 0x7f0fa4070c60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.116+0000 7f0fc019d640 1 --2- 172.21.15.6:0/1339392930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb8102b50 0x7f0fb8113200 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.116+0000 7f0fc019d640 1 --2- 172.21.15.6:0/1339392930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb81021b0 0x7f0fb8112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.116+0000 7f0fc019d640 1 -- 172.21.15.6:0/1339392930 >> 172.21.15.6:0/1339392930 conn(0x7f0fb80f7dc0 msgr2=0x7f0fb80f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:58.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.116+0000 7f0fc019d640 1 -- 172.21.15.6:0/1339392930 shutdown_connections 2024-03-22T06:26:58.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.117+0000 7f0fc019d640 1 -- 172.21.15.6:0/1339392930 wait complete. 2024-03-22T06:26:58.127 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-03-22T06:26:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.227+0000 7f3a41b14640 1 -- 172.21.15.6:0/2622163270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a3c0fb8d0 msgr2=0x7f3a3c108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.227+0000 7f3a41b14640 1 --2- 172.21.15.6:0/2622163270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a3c0fb8d0 0x7f3a3c108630 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0048d0 tx=0x7f3a2c009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.227+0000 7f3a41b14640 1 -- 172.21.15.6:0/2622163270 shutdown_connections 2024-03-22T06:26:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.227+0000 7f3a41b14640 1 --2- 172.21.15.6:0/2622163270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a3c0fb8d0 0x7f3a3c108630 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.227+0000 7f3a41b14640 1 --2- 172.21.15.6:0/2622163270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3c0fafc0 0x7f3a3c0fb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.227+0000 7f3a41b14640 1 -- 172.21.15.6:0/2622163270 >> 172.21.15.6:0/2622163270 conn(0x7f3a3c0f6cb0 msgr2=0x7f3a3c0f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.227+0000 7f3a41b14640 1 -- 172.21.15.6:0/2622163270 shutdown_connections 2024-03-22T06:26:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.227+0000 7f3a41b14640 1 -- 172.21.15.6:0/2622163270 wait complete. 2024-03-22T06:26:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.227+0000 7f3a41b14640 1 Processor -- start 2024-03-22T06:26:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.227+0000 7f3a41b14640 1 -- start start 2024-03-22T06:26:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a41b14640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a3c0fafc0 0x7f3a3c114dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a41b14640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3c0fb8d0 0x7f3a3c115310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a41b14640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c115910 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a41b14640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c10ef60 con 0x7f3a3c0fb8d0 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a3b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a3c0fafc0 0x7f3a3c114dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a3b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a3c0fafc0 0x7f3a3c114dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35456/0 (socket says 172.21.15.6:35456) 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a3b7fe640 1 -- 172.21.15.6:0/2549266361 learned_addr learned my addr 172.21.15.6:0/2549266361 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a3affd640 1 --2- 172.21.15.6:0/2549266361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3c0fb8d0 0x7f3a3c115310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a3b7fe640 1 -- 172.21.15.6:0/2549266361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3c0fb8d0 msgr2=0x7f3a3c115310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a3b7fe640 1 --2- 172.21.15.6:0/2549266361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3c0fb8d0 0x7f3a3c115310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a3b7fe640 1 -- 172.21.15.6:0/2549266361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c005040 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a3b7fe640 1 --2- 172.21.15.6:0/2549266361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a3c0fafc0 0x7f3a3c114dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c830 tx=0x7f3a30004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a23fff640 1 -- 172.21.15.6:0/2549266361 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a3000a850 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a23fff640 1 -- 172.21.15.6:0/2549266361 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a3000ae80 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a41b14640 1 -- 172.21.15.6:0/2549266361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c10f240 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a23fff640 1 -- 172.21.15.6:0/2549266361 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a30012ea0 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.228+0000 7f3a41b14640 1 -- 172.21.15.6:0/2549266361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c10f7e0 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.229+0000 7f3a41b14640 1 -- 172.21.15.6:0/2549266361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c065920 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.230+0000 7f3a23fff640 1 -- 172.21.15.6:0/2549266361 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f3a30012400 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.230+0000 7f3a23fff640 1 --2- 172.21.15.6:0/2549266361 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3a2406e7b0 0x7f3a24070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.231+0000 7f3a3affd640 1 --2- 172.21.15.6:0/2549266361 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3a2406e7b0 0x7f3a24070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.231+0000 7f3a23fff640 1 -- 172.21.15.6:0/2549266361 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f3a3008e840 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.231+0000 7f3a3affd640 1 --2- 172.21.15.6:0/2549266361 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3a2406e7b0 0x7f3a24070c60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c007ac0 tx=0x7f3a2c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:58.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.235+0000 7f3a23fff640 1 -- 172.21.15.6:0/2549266361 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f3a30057690 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:57 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/469337124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T06:26:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:26:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:26:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:57 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/208778907' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-03-22T06:26:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:57 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/825136378' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-03-22T06:26:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:26:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T06:26:58.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:57 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' 2024-03-22T06:26:58.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.464+0000 7f3a41b14640 1 -- 172.21.15.6:0/2549266361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f3a3c04bcc0 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.465+0000 7f3a23fff640 1 -- 172.21.15.6:0/2549266361 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v36) v1 ==== 107+0+4080 (secure 0 0 0) 0x7f3a300126b0 con 0x7f3a3c0fafc0 2024-03-22T06:26:58.465 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:58.466 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24269,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6825/1251227895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1251227895},{"type":"v1","addr":"172.21.15.117:6825","nonce":1251227895}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:22:21.923176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi006.dcjpla","rank":0,"incarnation":16,"state":"up:rejoin","state_seq":150,"addr":"172.21.15.6:6829/947853733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":947853733},{"type":"v1","addr":"172.21.15.6:6829","nonce":947853733}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:26:58.466 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 18 2024-03-22T06:26:58.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.467+0000 7f3a41b14640 1 -- 172.21.15.6:0/2549266361 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3a2406e7b0 msgr2=0x7f3a24070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.467+0000 7f3a41b14640 1 --2- 172.21.15.6:0/2549266361 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3a2406e7b0 0x7f3a24070c60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c007ac0 tx=0x7f3a2c009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.467+0000 7f3a41b14640 1 -- 172.21.15.6:0/2549266361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a3c0fafc0 msgr2=0x7f3a3c114dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.467+0000 7f3a41b14640 1 --2- 172.21.15.6:0/2549266361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a3c0fafc0 0x7f3a3c114dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c830 tx=0x7f3a30004430 comp rx=0 tx=0).stop 2024-03-22T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.467+0000 7f3a41b14640 1 -- 172.21.15.6:0/2549266361 shutdown_connections 2024-03-22T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.467+0000 7f3a41b14640 1 --2- 172.21.15.6:0/2549266361 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3a2406e7b0 0x7f3a24070c60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.467+0000 7f3a41b14640 1 --2- 172.21.15.6:0/2549266361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a3c0fb8d0 0x7f3a3c115310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.467+0000 7f3a41b14640 1 --2- 172.21.15.6:0/2549266361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a3c0fafc0 0x7f3a3c114dd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.467+0000 7f3a41b14640 1 -- 172.21.15.6:0/2549266361 >> 172.21.15.6:0/2549266361 conn(0x7f3a3c0f6cb0 msgr2=0x7f3a3c0f7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.467+0000 7f3a41b14640 1 -- 172.21.15.6:0/2549266361 shutdown_connections 2024-03-22T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.467+0000 7f3a41b14640 1 -- 172.21.15.6:0/2549266361 wait complete. 2024-03-22T06:26:58.478 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-22T06:26:58.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.578+0000 7fcf08f5d640 1 -- 172.21.15.6:0/1211385460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf04101140 msgr2=0x7fcf04101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.578+0000 7fcf08f5d640 1 --2- 172.21.15.6:0/1211385460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf04101140 0x7fcf04101510 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcef40048d0 tx=0x7fcef4009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.578+0000 7fcf08f5d640 1 -- 172.21.15.6:0/1211385460 shutdown_connections 2024-03-22T06:26:58.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.578+0000 7fcf08f5d640 1 --2- 172.21.15.6:0/1211385460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf040fb110 0x7fcf040fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.578+0000 7fcf08f5d640 1 --2- 172.21.15.6:0/1211385460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf04101140 0x7fcf04101510 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.578+0000 7fcf08f5d640 1 -- 172.21.15.6:0/1211385460 >> 172.21.15.6:0/1211385460 conn(0x7fcf040710b0 msgr2=0x7fcf040714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:58.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.578+0000 7fcf08f5d640 1 -- 172.21.15.6:0/1211385460 shutdown_connections 2024-03-22T06:26:58.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.579+0000 7fcf08f5d640 1 -- 172.21.15.6:0/1211385460 wait complete. 2024-03-22T06:26:58.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.579+0000 7fcf08f5d640 1 Processor -- start 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.579+0000 7fcf08f5d640 1 -- start start 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.579+0000 7fcf08f5d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf040fb110 0x7fcf0410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.579+0000 7fcf08f5d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf04101140 0x7fcf0410e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.579+0000 7fcf08f5d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf04065310 con 0x7fcf04101140 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.579+0000 7fcf08f5d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0410e900 con 0x7fcf040fb110 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.579+0000 7fcefbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf04101140 0x7fcf0410e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.579+0000 7fcefbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf04101140 0x7fcf0410e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35468/0 (socket says 172.21.15.6:35468) 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.579+0000 7fcefbfff640 1 -- 172.21.15.6:0/584215637 learned_addr learned my addr 172.21.15.6:0/584215637 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.579+0000 7fcf0259c640 1 --2- 172.21.15.6:0/584215637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf040fb110 0x7fcf0410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.580+0000 7fcefbfff640 1 -- 172.21.15.6:0/584215637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf040fb110 msgr2=0x7fcf0410de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.580+0000 7fcefbfff640 1 --2- 172.21.15.6:0/584215637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf040fb110 0x7fcf0410de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.580+0000 7fcefbfff640 1 -- 172.21.15.6:0/584215637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4005040 con 0x7fcf04101140 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.580+0000 7fcefbfff640 1 --2- 172.21.15.6:0/584215637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf04101140 0x7fcf0410e3c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fceec002a50 tx=0x7fceec004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:58.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.580+0000 7fcefaffd640 1 -- 172.21.15.6:0/584215637 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceec007c30 con 0x7fcf04101140 2024-03-22T06:26:58.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.580+0000 7fcefaffd640 1 -- 172.21.15.6:0/584215637 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fceec00f040 con 0x7fcf04101140 2024-03-22T06:26:58.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.580+0000 7fcf08f5d640 1 -- 172.21.15.6:0/584215637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0410ebe0 con 0x7fcf04101140 2024-03-22T06:26:58.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.580+0000 7fcefaffd640 1 -- 172.21.15.6:0/584215637 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceec00db50 con 0x7fcf04101140 2024-03-22T06:26:58.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.580+0000 7fcf08f5d640 1 -- 172.21.15.6:0/584215637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf041171a0 con 0x7fcf04101140 2024-03-22T06:26:58.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.581+0000 7fcf08f5d640 1 -- 172.21.15.6:0/584215637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf040fc800 con 0x7fcf04101140 2024-03-22T06:26:58.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.581+0000 7fcefaffd640 1 -- 172.21.15.6:0/584215637 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fceec01c400 con 0x7fcf04101140 2024-03-22T06:26:58.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.582+0000 7fcefaffd640 1 --2- 172.21.15.6:0/584215637 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7b0 0x7fcef0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:58.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.582+0000 7fcf0259c640 1 --2- 172.21.15.6:0/584215637 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7b0 0x7fcef0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:58.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.582+0000 7fcefaffd640 1 -- 172.21.15.6:0/584215637 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fceec08df00 con 0x7fcf04101140 2024-03-22T06:26:58.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.583+0000 7fcf0259c640 1 --2- 172.21.15.6:0/584215637 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7b0 0x7fcef0070c60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009990 tx=0x7fcef4009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:58.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.589+0000 7fcefaffd640 1 -- 172.21.15.6:0/584215637 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fceec056c70 con 0x7fcf04101140 2024-03-22T06:26:58.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.812+0000 7fcf08f5d640 1 -- 172.21.15.6:0/584215637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fcf0404bcc0 con 0x7fcf04101140 2024-03-22T06:26:58.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.812+0000 7fcefaffd640 1 -- 172.21.15.6:0/584215637 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v36) v1 ==== 107+0+4896 (secure 0 0 0) 0x7fceec05a330 con 0x7fcf04101140 2024-03-22T06:26:58.813 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:58.813 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24269,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6825/1251227895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1251227895},{"type":"v1","addr":"172.21.15.117:6825","nonce":1251227895}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:22:23.926858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi006.dcjpla","rank":0,"incarnation":16,"state":"up:active","state_seq":151,"addr":"172.21.15.6:6829/947853733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":947853733},{"type":"v1","addr":"172.21.15.6:6829","nonce":947853733}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14476,"qdb_cluster":[14476]},"id":1}]} 2024-03-22T06:26:58.813 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 19 2024-03-22T06:26:58.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.814+0000 7fcf08f5d640 1 -- 172.21.15.6:0/584215637 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7b0 msgr2=0x7fcef0070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.814+0000 7fcf08f5d640 1 --2- 172.21.15.6:0/584215637 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7b0 0x7fcef0070c60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009990 tx=0x7fcef4009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.814+0000 7fcf08f5d640 1 -- 172.21.15.6:0/584215637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf04101140 msgr2=0x7fcf0410e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.814+0000 7fcf08f5d640 1 --2- 172.21.15.6:0/584215637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf04101140 0x7fcf0410e3c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fceec002a50 tx=0x7fceec004600 comp rx=0 tx=0).stop 2024-03-22T06:26:58.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.814+0000 7fcf08f5d640 1 -- 172.21.15.6:0/584215637 shutdown_connections 2024-03-22T06:26:58.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.814+0000 7fcf08f5d640 1 --2- 172.21.15.6:0/584215637 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7b0 0x7fcef0070c60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.814+0000 7fcf08f5d640 1 --2- 172.21.15.6:0/584215637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf04101140 0x7fcf0410e3c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.814+0000 7fcf08f5d640 1 --2- 172.21.15.6:0/584215637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf040fb110 0x7fcf0410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.814+0000 7fcf08f5d640 1 -- 172.21.15.6:0/584215637 >> 172.21.15.6:0/584215637 conn(0x7fcf040710b0 msgr2=0x7fcf040f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:58.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.815+0000 7fcf08f5d640 1 -- 172.21.15.6:0/584215637 shutdown_connections 2024-03-22T06:26:58.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.815+0000 7fcf08f5d640 1 -- 172.21.15.6:0/584215637 wait complete. 2024-03-22T06:26:58.826 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-22T06:26:58.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.925+0000 7f5d97d9d640 1 -- 172.21.15.6:0/2685090649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d90101140 msgr2=0x7f5d90101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.925+0000 7f5d97d9d640 1 --2- 172.21.15.6:0/2685090649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d90101140 0x7f5d90101510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0048d0 tx=0x7f5d8c009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.925+0000 7f5d97d9d640 1 -- 172.21.15.6:0/2685090649 shutdown_connections 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.925+0000 7f5d97d9d640 1 --2- 172.21.15.6:0/2685090649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d900fb110 0x7f5d900fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.925+0000 7f5d97d9d640 1 --2- 172.21.15.6:0/2685090649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d90101140 0x7f5d90101510 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.925+0000 7f5d97d9d640 1 -- 172.21.15.6:0/2685090649 >> 172.21.15.6:0/2685090649 conn(0x7f5d900710b0 msgr2=0x7f5d900714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d97d9d640 1 -- 172.21.15.6:0/2685090649 shutdown_connections 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d97d9d640 1 -- 172.21.15.6:0/2685090649 wait complete. 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d97d9d640 1 Processor -- start 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d97d9d640 1 -- start start 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d97d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d900fb110 0x7f5d90112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d97d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d90101140 0x7f5d901131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d97d9d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d901137f0 con 0x7f5d900fb110 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d97d9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9010ce40 con 0x7f5d90101140 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d95b39640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d900fb110 0x7f5d90112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d95b39640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d900fb110 0x7f5d90112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35486/0 (socket says 172.21.15.6:35486) 2024-03-22T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d95b39640 1 -- 172.21.15.6:0/3016060579 learned_addr learned my addr 172.21.15.6:0/3016060579 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:58.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.926+0000 7f5d95338640 1 --2- 172.21.15.6:0/3016060579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d90101140 0x7f5d901131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:58.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.927+0000 7f5d95b39640 1 -- 172.21.15.6:0/3016060579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d90101140 msgr2=0x7f5d901131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:58.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.927+0000 7f5d95b39640 1 --2- 172.21.15.6:0/3016060579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d90101140 0x7f5d901131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:58.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.927+0000 7f5d95b39640 1 -- 172.21.15.6:0/3016060579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c005040 con 0x7f5d900fb110 2024-03-22T06:26:58.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.927+0000 7f5d95b39640 1 --2- 172.21.15.6:0/3016060579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d900fb110 0x7f5d90112cb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0041d0 tx=0x7f5d8c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:58.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.927+0000 7f5d867fc640 1 -- 172.21.15.6:0/3016060579 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8c0092e0 con 0x7f5d900fb110 2024-03-22T06:26:58.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.927+0000 7f5d867fc640 1 -- 172.21.15.6:0/3016060579 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d8c009440 con 0x7f5d900fb110 2024-03-22T06:26:58.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.927+0000 7f5d97d9d640 1 -- 172.21.15.6:0/3016060579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9010d0c0 con 0x7f5d900fb110 2024-03-22T06:26:58.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.927+0000 7f5d867fc640 1 -- 172.21.15.6:0/3016060579 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8c043640 con 0x7f5d900fb110 2024-03-22T06:26:58.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.927+0000 7f5d97d9d640 1 -- 172.21.15.6:0/3016060579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9010d5e0 con 0x7f5d900fb110 2024-03-22T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.928+0000 7f5d97d9d640 1 -- 172.21.15.6:0/3016060579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d900fc800 con 0x7f5d900fb110 2024-03-22T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.928+0000 7f5d867fc640 1 -- 172.21.15.6:0/3016060579 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f5d8c03b980 con 0x7f5d900fb110 2024-03-22T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.929+0000 7f5d867fc640 1 --2- 172.21.15.6:0/3016060579 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5d7c06e7c0 0x7f5d7c070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.929+0000 7f5d867fc640 1 -- 172.21.15.6:0/3016060579 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f5d8c0b12c0 con 0x7f5d900fb110 2024-03-22T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.929+0000 7f5d95338640 1 --2- 172.21.15.6:0/3016060579 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5d7c06e7c0 0x7f5d7c070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.929+0000 7f5d95338640 1 --2- 172.21.15.6:0/3016060579 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5d7c06e7c0 0x7f5d7c070c70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d9010e570 tx=0x7f5d80009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:58.934+0000 7f5d867fc640 1 -- 172.21.15.6:0/3016060579 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f5d8c07a1b0 con 0x7f5d900fb110 2024-03-22T06:26:59.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:58 smithi117 ceph-mon[82126]: pgmap v352: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:59.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:58 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1339392930' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-22T06:26:59.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:58 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2549266361' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-22T06:26:59.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:58 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/584215637' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-22T06:26:59.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.273+0000 7f5d97d9d640 1 -- 172.21.15.6:0/3016060579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f5d9004bcc0 con 0x7f5d900fb110 2024-03-22T06:26:59.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.273+0000 7f5d867fc640 1 -- 172.21.15.6:0/3016060579 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v36) v1 ==== 107+0+4101 (secure 0 0 0) 0x7f5d8c07d870 con 0x7f5d900fb110 2024-03-22T06:26:59.274 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:59.274 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24269,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6825/1251227895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1251227895},{"type":"v1","addr":"172.21.15.117:6825","nonce":1251227895}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:22:41.442397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:26:59.274 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2024-03-22T06:26:59.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.275+0000 7f5d97d9d640 1 -- 172.21.15.6:0/3016060579 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5d7c06e7c0 msgr2=0x7f5d7c070c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:59.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.275+0000 7f5d97d9d640 1 --2- 172.21.15.6:0/3016060579 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5d7c06e7c0 0x7f5d7c070c70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d9010e570 tx=0x7f5d80009040 comp rx=0 tx=0).stop 2024-03-22T06:26:59.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.275+0000 7f5d97d9d640 1 -- 172.21.15.6:0/3016060579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d900fb110 msgr2=0x7f5d90112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:59.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.275+0000 7f5d97d9d640 1 --2- 172.21.15.6:0/3016060579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d900fb110 0x7f5d90112cb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0041d0 tx=0x7f5d8c009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.275+0000 7f5d97d9d640 1 -- 172.21.15.6:0/3016060579 shutdown_connections 2024-03-22T06:26:59.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.275+0000 7f5d97d9d640 1 --2- 172.21.15.6:0/3016060579 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f5d7c06e7c0 0x7f5d7c070c70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.275+0000 7f5d97d9d640 1 --2- 172.21.15.6:0/3016060579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d90101140 0x7f5d901131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.275+0000 7f5d97d9d640 1 --2- 172.21.15.6:0/3016060579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d900fb110 0x7f5d90112cb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.275+0000 7f5d97d9d640 1 -- 172.21.15.6:0/3016060579 >> 172.21.15.6:0/3016060579 conn(0x7f5d900710b0 msgr2=0x7f5d900f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:59.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.275+0000 7f5d97d9d640 1 -- 172.21.15.6:0/3016060579 shutdown_connections 2024-03-22T06:26:59.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.275+0000 7f5d97d9d640 1 -- 172.21.15.6:0/3016060579 wait complete. 2024-03-22T06:26:59.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:58 smithi006 ceph-mon[94482]: pgmap v352: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:26:59.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:58 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1339392930' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-22T06:26:59.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:58 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2549266361' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-22T06:26:59.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:58 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/584215637' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-22T06:26:59.290 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-22T06:26:59.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.392+0000 7f0415c07640 1 -- 172.21.15.6:0/3430228599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0410101140 msgr2=0x7f0410101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:59.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.392+0000 7f0415c07640 1 --2- 172.21.15.6:0/3430228599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0410101140 0x7f0410101510 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f04000048d0 tx=0x7f0400009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.392+0000 7f0415c07640 1 -- 172.21.15.6:0/3430228599 shutdown_connections 2024-03-22T06:26:59.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.392+0000 7f0415c07640 1 --2- 172.21.15.6:0/3430228599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04100fb110 0x7f04100fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.392+0000 7f0415c07640 1 --2- 172.21.15.6:0/3430228599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0410101140 0x7f0410101510 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.392+0000 7f0415c07640 1 -- 172.21.15.6:0/3430228599 >> 172.21.15.6:0/3430228599 conn(0x7f04100710b0 msgr2=0x7f04100714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:59.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.392+0000 7f0415c07640 1 -- 172.21.15.6:0/3430228599 shutdown_connections 2024-03-22T06:26:59.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.392+0000 7f0415c07640 1 -- 172.21.15.6:0/3430228599 wait complete. 2024-03-22T06:26:59.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.392+0000 7f0415c07640 1 Processor -- start 2024-03-22T06:26:59.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f0415c07640 1 -- start start 2024-03-22T06:26:59.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f0415c07640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04100fb110 0x7f041010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:59.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f0415c07640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0410101140 0x7f041010e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:59.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f0415c07640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0410065310 con 0x7f04100fb110 2024-03-22T06:26:59.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f0415c07640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041010e900 con 0x7f0410101140 2024-03-22T06:26:59.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f040f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04100fb110 0x7f041010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:59.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f040f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04100fb110 0x7f041010de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35496/0 (socket says 172.21.15.6:35496) 2024-03-22T06:26:59.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f040f7fe640 1 -- 172.21.15.6:0/2262960808 learned_addr learned my addr 172.21.15.6:0/2262960808 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:59.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f040effd640 1 --2- 172.21.15.6:0/2262960808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0410101140 0x7f041010e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:59.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f040f7fe640 1 -- 172.21.15.6:0/2262960808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0410101140 msgr2=0x7f041010e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:59.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f040f7fe640 1 --2- 172.21.15.6:0/2262960808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0410101140 0x7f041010e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f040f7fe640 1 -- 172.21.15.6:0/2262960808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0400005040 con 0x7f04100fb110 2024-03-22T06:26:59.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f040f7fe640 1 --2- 172.21.15.6:0/2262960808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04100fb110 0x7f041010de80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0400030640 tx=0x7f0400009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:59.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.393+0000 7f03f7fff640 1 -- 172.21.15.6:0/2262960808 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0400030820 con 0x7f04100fb110 2024-03-22T06:26:59.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.394+0000 7f0415c07640 1 -- 172.21.15.6:0/2262960808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041010eb80 con 0x7f04100fb110 2024-03-22T06:26:59.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.394+0000 7f03f7fff640 1 -- 172.21.15.6:0/2262960808 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04000092e0 con 0x7f04100fb110 2024-03-22T06:26:59.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.394+0000 7f03f7fff640 1 -- 172.21.15.6:0/2262960808 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0400043400 con 0x7f04100fb110 2024-03-22T06:26:59.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.394+0000 7f0415c07640 1 -- 172.21.15.6:0/2262960808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0410117140 con 0x7f04100fb110 2024-03-22T06:26:59.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.395+0000 7f0415c07640 1 -- 172.21.15.6:0/2262960808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04100fc800 con 0x7f04100fb110 2024-03-22T06:26:59.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.395+0000 7f03f7fff640 1 -- 172.21.15.6:0/2262960808 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f040003b420 con 0x7f04100fb110 2024-03-22T06:26:59.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.395+0000 7f03f7fff640 1 --2- 172.21.15.6:0/2262960808 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f03f806e7b0 0x7f03f8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:59.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.396+0000 7f03f7fff640 1 -- 172.21.15.6:0/2262960808 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f04000b1310 con 0x7f04100fb110 2024-03-22T06:26:59.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.396+0000 7f040effd640 1 --2- 172.21.15.6:0/2262960808 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f03f806e7b0 0x7f03f8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:59.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.396+0000 7f040effd640 1 --2- 172.21.15.6:0/2262960808 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f03f806e7b0 0x7f03f8070c60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0410064b60 tx=0x7f040400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:59.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.400+0000 7f03f7fff640 1 -- 172.21.15.6:0/2262960808 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f040007a220 con 0x7f04100fb110 2024-03-22T06:26:59.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.624+0000 7f0415c07640 1 -- 172.21.15.6:0/2262960808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f041004bcc0 con 0x7f04100fb110 2024-03-22T06:26:59.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.624+0000 7f03f7fff640 1 -- 172.21.15.6:0/2262960808 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v36) v1 ==== 107+0+4112 (secure 0 0 0) 0x7f040007d8e0 con 0x7f04100fb110 2024-03-22T06:26:59.625 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:26:59.625 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:22:41.447049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24269":{"gid":24269,"name":"cephfs.smithi117.fwkech","rank":0,"incarnation":21,"state":"up:replay","state_seq":3,"addr":"172.21.15.117:6825/1251227895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1251227895},{"type":"v1","addr":"172.21.15.117:6825","nonce":1251227895}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:26:59.625 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 21 2024-03-22T06:26:59.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.626+0000 7f0415c07640 1 -- 172.21.15.6:0/2262960808 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f03f806e7b0 msgr2=0x7f03f8070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:59.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.626+0000 7f0415c07640 1 --2- 172.21.15.6:0/2262960808 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f03f806e7b0 0x7f03f8070c60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0410064b60 tx=0x7f040400b040 comp rx=0 tx=0).stop 2024-03-22T06:26:59.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.626+0000 7f0415c07640 1 -- 172.21.15.6:0/2262960808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04100fb110 msgr2=0x7f041010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.626+0000 7f0415c07640 1 --2- 172.21.15.6:0/2262960808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04100fb110 0x7f041010de80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0400030640 tx=0x7f0400009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.626+0000 7f0415c07640 1 -- 172.21.15.6:0/2262960808 shutdown_connections 2024-03-22T06:26:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.626+0000 7f0415c07640 1 --2- 172.21.15.6:0/2262960808 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f03f806e7b0 0x7f03f8070c60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.626+0000 7f0415c07640 1 --2- 172.21.15.6:0/2262960808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0410101140 0x7f041010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.626+0000 7f0415c07640 1 --2- 172.21.15.6:0/2262960808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04100fb110 0x7f041010de80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.626+0000 7f0415c07640 1 -- 172.21.15.6:0/2262960808 >> 172.21.15.6:0/2262960808 conn(0x7f04100710b0 msgr2=0x7f04100f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.627+0000 7f0415c07640 1 -- 172.21.15.6:0/2262960808 shutdown_connections 2024-03-22T06:26:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.627+0000 7f0415c07640 1 -- 172.21.15.6:0/2262960808 wait complete. 2024-03-22T06:26:59.638 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-22T06:26:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.767+0000 7faeeb06e640 1 -- 172.21.15.6:0/3863914971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faee4101140 msgr2=0x7faee4101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.767+0000 7faeeb06e640 1 --2- 172.21.15.6:0/3863914971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faee4101140 0x7faee4101510 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faed80048d0 tx=0x7faed8009da0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.767+0000 7faeeb06e640 1 -- 172.21.15.6:0/3863914971 shutdown_connections 2024-03-22T06:26:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.767+0000 7faeeb06e640 1 --2- 172.21.15.6:0/3863914971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee40fb110 0x7faee40fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.767+0000 7faeeb06e640 1 --2- 172.21.15.6:0/3863914971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faee4101140 0x7faee4101510 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.767+0000 7faeeb06e640 1 -- 172.21.15.6:0/3863914971 >> 172.21.15.6:0/3863914971 conn(0x7faee40710b0 msgr2=0x7faee40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:26:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.767+0000 7faeeb06e640 1 -- 172.21.15.6:0/3863914971 shutdown_connections 2024-03-22T06:26:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.767+0000 7faeeb06e640 1 -- 172.21.15.6:0/3863914971 wait complete. 2024-03-22T06:26:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.768+0000 7faeeb06e640 1 Processor -- start 2024-03-22T06:26:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.768+0000 7faeeb06e640 1 -- start start 2024-03-22T06:26:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.768+0000 7faeeb06e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faee40fb110 0x7faee4112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.768+0000 7faeeb06e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4101140 0x7faee41131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.768+0000 7faeeb06e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee41137a0 con 0x7faee40fb110 2024-03-22T06:26:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.768+0000 7faeeb06e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee4113910 con 0x7faee4101140 2024-03-22T06:26:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.768+0000 7faee8e0a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faee40fb110 0x7faee4112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.768+0000 7faee8e0a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faee40fb110 0x7faee4112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35512/0 (socket says 172.21.15.6:35512) 2024-03-22T06:26:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.768+0000 7faee8e0a640 1 -- 172.21.15.6:0/1211444926 learned_addr learned my addr 172.21.15.6:0/1211444926 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:26:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.769+0000 7faee3fff640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4101140 0x7faee41131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.769+0000 7faee8e0a640 1 -- 172.21.15.6:0/1211444926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4101140 msgr2=0x7faee41131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:26:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.769+0000 7faee8e0a640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4101140 0x7faee41131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:26:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.769+0000 7faee8e0a640 1 -- 172.21.15.6:0/1211444926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed8005040 con 0x7faee40fb110 2024-03-22T06:26:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.769+0000 7faee3fff640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4101140 0x7faee41131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:26:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.769+0000 7faee8e0a640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faee40fb110 0x7faee4112c60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faed80041d0 tx=0x7faed8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.769+0000 7faee17fa640 1 -- 172.21.15.6:0/1211444926 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faed80092e0 con 0x7faee40fb110 2024-03-22T06:26:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.769+0000 7faee17fa640 1 -- 172.21.15.6:0/1211444926 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faed8009440 con 0x7faee40fb110 2024-03-22T06:26:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.769+0000 7faeeb06e640 1 -- 172.21.15.6:0/1211444926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee410cf40 con 0x7faee40fb110 2024-03-22T06:26:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.769+0000 7faee17fa640 1 -- 172.21.15.6:0/1211444926 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faed8043640 con 0x7faee40fb110 2024-03-22T06:26:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.769+0000 7faeeb06e640 1 -- 172.21.15.6:0/1211444926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee410d460 con 0x7faee40fb110 2024-03-22T06:26:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.770+0000 7faeeb06e640 1 -- 172.21.15.6:0/1211444926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee40fc800 con 0x7faee40fb110 2024-03-22T06:26:59.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.770+0000 7faee17fa640 1 -- 172.21.15.6:0/1211444926 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7faed803b420 con 0x7faee40fb110 2024-03-22T06:26:59.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.771+0000 7faee17fa640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faed406e7b0 0x7faed4070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:26:59.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.771+0000 7faee17fa640 1 -- 172.21.15.6:0/1211444926 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7faed80b13c0 con 0x7faee40fb110 2024-03-22T06:26:59.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.771+0000 7faee3fff640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faed406e7b0 0x7faed4070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:26:59.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.771+0000 7faee3fff640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faed406e7b0 0x7faed4070c60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faee410e560 tx=0x7faed0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:26:59.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.776+0000 7faee17fa640 1 -- 172.21.15.6:0/1211444926 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7faed807a2d0 con 0x7faee40fb110 2024-03-22T06:26:59.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:26:59.999+0000 7faeeb06e640 1 -- 172.21.15.6:0/1211444926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7faee404bcc0 con 0x7faee40fb110 2024-03-22T06:27:00.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.000+0000 7faee17fa640 1 -- 172.21.15.6:0/1211444926 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v36) v1 ==== 107+0+4117 (secure 0 0 0) 0x7faed807d990 con 0x7faee40fb110 2024-03-22T06:27:00.000 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:00.000 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:22:49.366014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24269":{"gid":24269,"name":"cephfs.smithi117.fwkech","rank":0,"incarnation":21,"state":"up:reconnect","state_seq":157,"addr":"172.21.15.117:6825/1251227895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1251227895},{"type":"v1","addr":"172.21.15.117:6825","nonce":1251227895}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:27:00.000 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 22 2024-03-22T06:27:00.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.002+0000 7faeeb06e640 1 -- 172.21.15.6:0/1211444926 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faed406e7b0 msgr2=0x7faed4070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.002+0000 7faeeb06e640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faed406e7b0 0x7faed4070c60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faee410e560 tx=0x7faed0009040 comp rx=0 tx=0).stop 2024-03-22T06:27:00.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.002+0000 7faeeb06e640 1 -- 172.21.15.6:0/1211444926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faee40fb110 msgr2=0x7faee4112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.002+0000 7faeeb06e640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faee40fb110 0x7faee4112c60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faed80041d0 tx=0x7faed8009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.002+0000 7faeeb06e640 1 -- 172.21.15.6:0/1211444926 shutdown_connections 2024-03-22T06:27:00.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.002+0000 7faeeb06e640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7faed406e7b0 0x7faed4070c60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.002+0000 7faeeb06e640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4101140 0x7faee41131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.002+0000 7faeeb06e640 1 --2- 172.21.15.6:0/1211444926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faee40fb110 0x7faee4112c60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.002+0000 7faeeb06e640 1 -- 172.21.15.6:0/1211444926 >> 172.21.15.6:0/1211444926 conn(0x7faee40710b0 msgr2=0x7faee40643d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:00.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.002+0000 7faeeb06e640 1 -- 172.21.15.6:0/1211444926 shutdown_connections 2024-03-22T06:27:00.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.002+0000 7faeeb06e640 1 -- 172.21.15.6:0/1211444926 wait complete. 2024-03-22T06:27:00.013 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-22T06:27:00.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.117+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1009989079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a8101140 msgr2=0x7ff1a8101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.117+0000 7ff1aed5d640 1 --2- 172.21.15.6:0/1009989079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a8101140 0x7ff1a8101510 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0048d0 tx=0x7ff19c009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.117+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1009989079 shutdown_connections 2024-03-22T06:27:00.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.117+0000 7ff1aed5d640 1 --2- 172.21.15.6:0/1009989079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1a80fb110 0x7ff1a80fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.117+0000 7ff1aed5d640 1 --2- 172.21.15.6:0/1009989079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a8101140 0x7ff1a8101510 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.117+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1009989079 >> 172.21.15.6:0/1009989079 conn(0x7ff1a80710b0 msgr2=0x7ff1a80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:00.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.117+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1009989079 shutdown_connections 2024-03-22T06:27:00.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.117+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1009989079 wait complete. 2024-03-22T06:27:00.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.117+0000 7ff1aed5d640 1 Processor -- start 2024-03-22T06:27:00.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.117+0000 7ff1aed5d640 1 -- start start 2024-03-22T06:27:00.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1aed5d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a80fb110 0x7ff1a8111bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:00.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1aed5d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1a8101140 0x7ff1a81120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:00.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1aed5d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81126f0 con 0x7ff1a80fb110 2024-03-22T06:27:00.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1aed5d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a810bd40 con 0x7ff1a8101140 2024-03-22T06:27:00.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1acaf9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a80fb110 0x7ff1a8111bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:00.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1acaf9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a80fb110 0x7ff1a8111bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35526/0 (socket says 172.21.15.6:35526) 2024-03-22T06:27:00.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1acaf9640 1 -- 172.21.15.6:0/1349458713 learned_addr learned my addr 172.21.15.6:0/1349458713 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:00.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1acaf9640 1 -- 172.21.15.6:0/1349458713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1a8101140 msgr2=0x7ff1a81120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1acaf9640 1 --2- 172.21.15.6:0/1349458713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1a8101140 0x7ff1a81120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1acaf9640 1 -- 172.21.15.6:0/1349458713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c005040 con 0x7ff1a80fb110 2024-03-22T06:27:00.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1acaf9640 1 --2- 172.21.15.6:0/1349458713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a80fb110 0x7ff1a8111bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0041d0 tx=0x7ff19c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:00.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.118+0000 7ff1a57fa640 1 -- 172.21.15.6:0/1349458713 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff19c030870 con 0x7ff1a80fb110 2024-03-22T06:27:00.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.119+0000 7ff1a57fa640 1 -- 172.21.15.6:0/1349458713 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff19c0092e0 con 0x7ff1a80fb110 2024-03-22T06:27:00.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.119+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1349458713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a810bfc0 con 0x7ff1a80fb110 2024-03-22T06:27:00.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.119+0000 7ff1a57fa640 1 -- 172.21.15.6:0/1349458713 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff19c043400 con 0x7ff1a80fb110 2024-03-22T06:27:00.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.119+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1349458713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a810c4e0 con 0x7ff1a80fb110 2024-03-22T06:27:00.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.119+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1349458713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a80fc800 con 0x7ff1a80fb110 2024-03-22T06:27:00.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.120+0000 7ff1a57fa640 1 -- 172.21.15.6:0/1349458713 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7ff19c03b420 con 0x7ff1a80fb110 2024-03-22T06:27:00.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.120+0000 7ff1a57fa640 1 --2- 172.21.15.6:0/1349458713 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff19806e7b0 0x7ff198070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:00.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.121+0000 7ff1a57fa640 1 -- 172.21.15.6:0/1349458713 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7ff19c0b1320 con 0x7ff1a80fb110 2024-03-22T06:27:00.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.121+0000 7ff1a7fff640 1 --2- 172.21.15.6:0/1349458713 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff19806e7b0 0x7ff198070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:00.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.121+0000 7ff1a7fff640 1 --2- 172.21.15.6:0/1349458713 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff19806e7b0 0x7ff198070c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff194009af0 tx=0x7ff194009cd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:00.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.125+0000 7ff1a57fa640 1 -- 172.21.15.6:0/1349458713 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7ff19c07a230 con 0x7ff1a80fb110 2024-03-22T06:27:00.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:59 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3016060579' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-22T06:27:00.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:26:59 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2262960808' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-22T06:27:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:59 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3016060579' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-22T06:27:00.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:26:59 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2262960808' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-22T06:27:00.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.351+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1349458713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7ff1a8108260 con 0x7ff1a80fb110 2024-03-22T06:27:00.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.351+0000 7ff1a57fa640 1 -- 172.21.15.6:0/1349458713 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v36) v1 ==== 107+0+4924 (secure 0 0 0) 0x7ff19c07d8f0 con 0x7ff1a80fb110 2024-03-22T06:27:00.352 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:00.352 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:22:50.265657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24269":{"gid":24269,"name":"cephfs.smithi117.fwkech","rank":0,"incarnation":21,"state":"up:rejoin","state_seq":158,"addr":"172.21.15.117:6825/1251227895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1251227895},{"type":"v1","addr":"172.21.15.117:6825","nonce":1251227895}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:27:00.352 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 23 2024-03-22T06:27:00.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.353+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1349458713 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff19806e7b0 msgr2=0x7ff198070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.353+0000 7ff1aed5d640 1 --2- 172.21.15.6:0/1349458713 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff19806e7b0 0x7ff198070c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff194009af0 tx=0x7ff194009cd0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.353+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1349458713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a80fb110 msgr2=0x7ff1a8111bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.353+0000 7ff1aed5d640 1 --2- 172.21.15.6:0/1349458713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a80fb110 0x7ff1a8111bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0041d0 tx=0x7ff19c009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.353+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1349458713 shutdown_connections 2024-03-22T06:27:00.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.353+0000 7ff1aed5d640 1 --2- 172.21.15.6:0/1349458713 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff19806e7b0 0x7ff198070c60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.353+0000 7ff1aed5d640 1 --2- 172.21.15.6:0/1349458713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1a8101140 0x7ff1a81120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.353+0000 7ff1aed5d640 1 --2- 172.21.15.6:0/1349458713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a80fb110 0x7ff1a8111bb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.353+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1349458713 >> 172.21.15.6:0/1349458713 conn(0x7ff1a80710b0 msgr2=0x7ff1a80f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:00.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.353+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1349458713 shutdown_connections 2024-03-22T06:27:00.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.353+0000 7ff1aed5d640 1 -- 172.21.15.6:0/1349458713 wait complete. 2024-03-22T06:27:00.365 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-22T06:27:00.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.485+0000 7f18b0e65640 1 -- 172.21.15.6:0/3025010879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18ac0fb110 msgr2=0x7f18ac0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.485+0000 7f18b0e65640 1 --2- 172.21.15.6:0/3025010879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18ac0fb110 0x7f18ac0fb560 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18a00048d0 tx=0x7f18a0009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.485+0000 7f18b0e65640 1 -- 172.21.15.6:0/3025010879 shutdown_connections 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.485+0000 7f18b0e65640 1 --2- 172.21.15.6:0/3025010879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18ac0fb110 0x7f18ac0fb560 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.485+0000 7f18b0e65640 1 --2- 172.21.15.6:0/3025010879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ac101140 0x7f18ac101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.485+0000 7f18b0e65640 1 -- 172.21.15.6:0/3025010879 >> 172.21.15.6:0/3025010879 conn(0x7f18ac0710b0 msgr2=0x7f18ac0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.485+0000 7f18b0e65640 1 -- 172.21.15.6:0/3025010879 shutdown_connections 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.485+0000 7f18b0e65640 1 -- 172.21.15.6:0/3025010879 wait complete. 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.486+0000 7f18b0e65640 1 Processor -- start 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.486+0000 7f18b0e65640 1 -- start start 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.486+0000 7f18b0e65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ac0fb110 0x7f18ac0fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.486+0000 7f18b0e65640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18ac101140 0x7f18ac0fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.486+0000 7f18b0e65640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac0fec90 con 0x7f18ac101140 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.486+0000 7f18b0e65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac0fd7b0 con 0x7f18ac0fb110 2024-03-22T06:27:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.486+0000 7f18a9d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18ac101140 0x7f18ac0fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.486+0000 7f18a9d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18ac101140 0x7f18ac0fd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35546/0 (socket says 172.21.15.6:35546) 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.486+0000 7f18a9d9b640 1 -- 172.21.15.6:0/2918569936 learned_addr learned my addr 172.21.15.6:0/2918569936 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.486+0000 7f18aa59c640 1 --2- 172.21.15.6:0/2918569936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ac0fb110 0x7f18ac0fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.487+0000 7f18a9d9b640 1 -- 172.21.15.6:0/2918569936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ac0fb110 msgr2=0x7f18ac0fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.487+0000 7f18a9d9b640 1 --2- 172.21.15.6:0/2918569936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ac0fb110 0x7f18ac0fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.487+0000 7f18a9d9b640 1 -- 172.21.15.6:0/2918569936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a0005040 con 0x7f18ac101140 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.487+0000 7f18a9d9b640 1 --2- 172.21.15.6:0/2918569936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18ac101140 0x7f18ac0fd270 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18a00043d0 tx=0x7f18a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.487+0000 7f189affd640 1 -- 172.21.15.6:0/2918569936 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18a0030840 con 0x7f18ac101140 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.487+0000 7f189affd640 1 -- 172.21.15.6:0/2918569936 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18a00092e0 con 0x7f18ac101140 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.487+0000 7f18b0e65640 1 -- 172.21.15.6:0/2918569936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac0fda30 con 0x7f18ac101140 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.487+0000 7f189affd640 1 -- 172.21.15.6:0/2918569936 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18a0043640 con 0x7f18ac101140 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.487+0000 7f18b0e65640 1 -- 172.21.15.6:0/2918569936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac119450 con 0x7f18ac101140 2024-03-22T06:27:00.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.488+0000 7f18b0e65640 1 -- 172.21.15.6:0/2918569936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ac0fc800 con 0x7f18ac101140 2024-03-22T06:27:00.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.489+0000 7f189affd640 1 -- 172.21.15.6:0/2918569936 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f18a0009450 con 0x7f18ac101140 2024-03-22T06:27:00.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.489+0000 7f189affd640 1 --2- 172.21.15.6:0/2918569936 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189406e7c0 0x7f1894070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:00.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.489+0000 7f189affd640 1 -- 172.21.15.6:0/2918569936 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f18a00b11c0 con 0x7f18ac101140 2024-03-22T06:27:00.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.489+0000 7f18aa59c640 1 --2- 172.21.15.6:0/2918569936 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189406e7c0 0x7f1894070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:00.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.490+0000 7f18aa59c640 1 --2- 172.21.15.6:0/2918569936 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189406e7c0 0x7f1894070c70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f189c004ed0 tx=0x7f189c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:00.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.494+0000 7f189affd640 1 -- 172.21.15.6:0/2918569936 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f18a007a010 con 0x7f18ac101140 2024-03-22T06:27:00.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.718+0000 7f18b0e65640 1 -- 172.21.15.6:0/2918569936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f18ac04bcc0 con 0x7f18ac101140 2024-03-22T06:27:00.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.719+0000 7f189affd640 1 -- 172.21.15.6:0/2918569936 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v36) v1 ==== 107+0+4933 (secure 0 0 0) 0x7f18a007d6d0 con 0x7f18ac101140 2024-03-22T06:27:00.719 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:00.719 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:22:52.271472+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24269":{"gid":24269,"name":"cephfs.smithi117.fwkech","rank":0,"incarnation":21,"state":"up:active","state_seq":159,"addr":"172.21.15.117:6825/1251227895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1251227895},{"type":"v1","addr":"172.21.15.117:6825","nonce":1251227895}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24269,"qdb_cluster":[24269]},"id":1}]} 2024-03-22T06:27:00.720 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2024-03-22T06:27:00.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.721+0000 7f18b0e65640 1 -- 172.21.15.6:0/2918569936 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189406e7c0 msgr2=0x7f1894070c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.721+0000 7f18b0e65640 1 --2- 172.21.15.6:0/2918569936 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189406e7c0 0x7f1894070c70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f189c004ed0 tx=0x7f189c00b040 comp rx=0 tx=0).stop 2024-03-22T06:27:00.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.721+0000 7f18b0e65640 1 -- 172.21.15.6:0/2918569936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18ac101140 msgr2=0x7f18ac0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.721+0000 7f18b0e65640 1 --2- 172.21.15.6:0/2918569936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18ac101140 0x7f18ac0fd270 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18a00043d0 tx=0x7f18a0009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.721+0000 7f18b0e65640 1 -- 172.21.15.6:0/2918569936 shutdown_connections 2024-03-22T06:27:00.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.721+0000 7f18b0e65640 1 --2- 172.21.15.6:0/2918569936 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f189406e7c0 0x7f1894070c70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.721+0000 7f18b0e65640 1 --2- 172.21.15.6:0/2918569936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18ac101140 0x7f18ac0fd270 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.721+0000 7f18b0e65640 1 --2- 172.21.15.6:0/2918569936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ac0fb110 0x7f18ac0fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.721+0000 7f18b0e65640 1 -- 172.21.15.6:0/2918569936 >> 172.21.15.6:0/2918569936 conn(0x7f18ac0710b0 msgr2=0x7f18ac0643b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:00.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.721+0000 7f18b0e65640 1 -- 172.21.15.6:0/2918569936 shutdown_connections 2024-03-22T06:27:00.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.721+0000 7f18b0e65640 1 -- 172.21.15.6:0/2918569936 wait complete. 2024-03-22T06:27:00.732 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-22T06:27:00.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.834+0000 7f53a60ae640 1 -- 172.21.15.6:0/3636132900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00fd400 msgr2=0x7f53a00fd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.834+0000 7f53a60ae640 1 --2- 172.21.15.6:0/3636132900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00fd400 0x7f53a00fd7d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f53880048d0 tx=0x7f5388009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.834+0000 7f53a60ae640 1 -- 172.21.15.6:0/3636132900 shutdown_connections 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.834+0000 7f53a60ae640 1 --2- 172.21.15.6:0/3636132900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53a00fdd10 0x7f53a01063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.834+0000 7f53a60ae640 1 --2- 172.21.15.6:0/3636132900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00fd400 0x7f53a00fd7d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.834+0000 7f53a60ae640 1 -- 172.21.15.6:0/3636132900 >> 172.21.15.6:0/3636132900 conn(0x7f53a00f6e70 msgr2=0x7f53a00f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.834+0000 7f53a60ae640 1 -- 172.21.15.6:0/3636132900 shutdown_connections 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.834+0000 7f53a60ae640 1 -- 172.21.15.6:0/3636132900 wait complete. 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.834+0000 7f53a60ae640 1 Processor -- start 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f53a60ae640 1 -- start start 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f53a60ae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53a00fd400 0x7f53a0112f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f53a60ae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00fdd10 0x7f53a01134a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f53a60ae640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a0113aa0 con 0x7f53a00fdd10 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f53a60ae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a0113c10 con 0x7f53a00fd400 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f539effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00fdd10 0x7f53a01134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f539effd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00fdd10 0x7f53a01134a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35560/0 (socket says 172.21.15.6:35560) 2024-03-22T06:27:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f539effd640 1 -- 172.21.15.6:0/245630698 learned_addr learned my addr 172.21.15.6:0/245630698 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f539f7fe640 1 --2- 172.21.15.6:0/245630698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53a00fd400 0x7f53a0112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f539effd640 1 -- 172.21.15.6:0/245630698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53a00fd400 msgr2=0x7f53a0112f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f539effd640 1 --2- 172.21.15.6:0/245630698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53a00fd400 0x7f53a0112f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.835+0000 7f539effd640 1 -- 172.21.15.6:0/245630698 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5388005040 con 0x7f53a00fdd10 2024-03-22T06:27:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.836+0000 7f539effd640 1 --2- 172.21.15.6:0/245630698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00fdd10 0x7f53a01134a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f539000c830 tx=0x7f5390004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.836+0000 7f5397fff640 1 -- 172.21.15.6:0/245630698 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f539000a850 con 0x7f53a00fdd10 2024-03-22T06:27:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.836+0000 7f53a60ae640 1 -- 172.21.15.6:0/245630698 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a010d2a0 con 0x7f53a00fdd10 2024-03-22T06:27:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.836+0000 7f5397fff640 1 -- 172.21.15.6:0/245630698 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f539000ae80 con 0x7f53a00fdd10 2024-03-22T06:27:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.836+0000 7f5397fff640 1 -- 172.21.15.6:0/245630698 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5390012ea0 con 0x7f53a00fdd10 2024-03-22T06:27:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.836+0000 7f53a60ae640 1 -- 172.21.15.6:0/245630698 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a010d7c0 con 0x7f53a00fdd10 2024-03-22T06:27:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.837+0000 7f53a60ae640 1 -- 172.21.15.6:0/245630698 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a0065e30 con 0x7f53a00fdd10 2024-03-22T06:27:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.837+0000 7f5397fff640 1 -- 172.21.15.6:0/245630698 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f5390012400 con 0x7f53a00fdd10 2024-03-22T06:27:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.838+0000 7f5397fff640 1 --2- 172.21.15.6:0/245630698 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f538c06e7b0 0x7f538c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.838+0000 7f5397fff640 1 -- 172.21.15.6:0/245630698 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f539008e840 con 0x7f53a00fdd10 2024-03-22T06:27:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.838+0000 7f539f7fe640 1 --2- 172.21.15.6:0/245630698 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f538c06e7b0 0x7f538c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:00.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.838+0000 7f539f7fe640 1 --2- 172.21.15.6:0/245630698 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f538c06e7b0 0x7f538c070c60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5388007ac0 tx=0x7f5388009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:00.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:00.842+0000 7f5397fff640 1 -- 172.21.15.6:0/245630698 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f5390057690 con 0x7f53a00fdd10 2024-03-22T06:27:01.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:00 smithi006 ceph-mon[94482]: pgmap v353: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:27:01.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:00 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1211444926' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-22T06:27:01.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:00 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1349458713' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-22T06:27:01.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:00 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2918569936' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-22T06:27:01.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.068+0000 7f53a60ae640 1 -- 172.21.15.6:0/245630698 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f53a01095b0 con 0x7f53a00fdd10 2024-03-22T06:27:01.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.068+0000 7f5397fff640 1 -- 172.21.15.6:0/245630698 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v36) v1 ==== 107+0+4129 (secure 0 0 0) 0x7f53900126b0 con 0x7f53a00fdd10 2024-03-22T06:27:01.068 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:01.069 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:23:03.662568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:27:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 25 2024-03-22T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.070+0000 7f53a60ae640 1 -- 172.21.15.6:0/245630698 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f538c06e7b0 msgr2=0x7f538c070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.070+0000 7f53a60ae640 1 --2- 172.21.15.6:0/245630698 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f538c06e7b0 0x7f538c070c60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5388007ac0 tx=0x7f5388009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.070+0000 7f53a60ae640 1 -- 172.21.15.6:0/245630698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00fdd10 msgr2=0x7f53a01134a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.070+0000 7f53a60ae640 1 --2- 172.21.15.6:0/245630698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00fdd10 0x7f53a01134a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f539000c830 tx=0x7f5390004430 comp rx=0 tx=0).stop 2024-03-22T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.070+0000 7f53a60ae640 1 -- 172.21.15.6:0/245630698 shutdown_connections 2024-03-22T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.070+0000 7f53a60ae640 1 --2- 172.21.15.6:0/245630698 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f538c06e7b0 0x7f538c070c60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.070+0000 7f53a60ae640 1 --2- 172.21.15.6:0/245630698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00fdd10 0x7f53a01134a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.070+0000 7f53a60ae640 1 --2- 172.21.15.6:0/245630698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53a00fd400 0x7f53a0112f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.070+0000 7f53a60ae640 1 -- 172.21.15.6:0/245630698 >> 172.21.15.6:0/245630698 conn(0x7f53a00f6e70 msgr2=0x7f53a00f8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.070+0000 7f53a60ae640 1 -- 172.21.15.6:0/245630698 shutdown_connections 2024-03-22T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.071+0000 7f53a60ae640 1 -- 172.21.15.6:0/245630698 wait complete. 2024-03-22T06:27:01.082 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-22T06:27:01.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.184+0000 7f42550ae640 1 -- 172.21.15.6:0/2465497695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4250101140 msgr2=0x7f4250101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.184+0000 7f42550ae640 1 --2- 172.21.15.6:0/2465497695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4250101140 0x7f4250101510 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f42440048d0 tx=0x7f4244009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.184+0000 7f42550ae640 1 -- 172.21.15.6:0/2465497695 shutdown_connections 2024-03-22T06:27:01.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.184+0000 7f42550ae640 1 --2- 172.21.15.6:0/2465497695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42500fb110 0x7f42500fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.184+0000 7f42550ae640 1 --2- 172.21.15.6:0/2465497695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4250101140 0x7f4250101510 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.184+0000 7f42550ae640 1 -- 172.21.15.6:0/2465497695 >> 172.21.15.6:0/2465497695 conn(0x7f42500710b0 msgr2=0x7f42500714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:01.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.184+0000 7f42550ae640 1 -- 172.21.15.6:0/2465497695 shutdown_connections 2024-03-22T06:27:01.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.184+0000 7f42550ae640 1 -- 172.21.15.6:0/2465497695 wait complete. 2024-03-22T06:27:01.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.184+0000 7f42550ae640 1 Processor -- start 2024-03-22T06:27:01.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.184+0000 7f42550ae640 1 -- start start 2024-03-22T06:27:01.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.184+0000 7f42550ae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42500fb110 0x7f4250114ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:01.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f42550ae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4250101140 0x7f4250115400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f42550ae640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4250115a00 con 0x7f42500fb110 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f42550ae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425010f050 con 0x7f4250101140 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f424ed9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42500fb110 0x7f4250114ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f424ed9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42500fb110 0x7f4250114ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35574/0 (socket says 172.21.15.6:35574) 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f424ed9d640 1 -- 172.21.15.6:0/4058589957 learned_addr learned my addr 172.21.15.6:0/4058589957 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f424e59c640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4250101140 0x7f4250115400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f424ed9d640 1 -- 172.21.15.6:0/4058589957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4250101140 msgr2=0x7f4250115400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f424ed9d640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4250101140 0x7f4250115400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f424ed9d640 1 -- 172.21.15.6:0/4058589957 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4244005040 con 0x7f42500fb110 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f424e59c640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4250101140 0x7f4250115400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f424ed9d640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42500fb110 0x7f4250114ec0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42440043d0 tx=0x7f4244009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f42377fe640 1 -- 172.21.15.6:0/4058589957 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4244030840 con 0x7f42500fb110 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f42550ae640 1 -- 172.21.15.6:0/4058589957 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425010f2d0 con 0x7f42500fb110 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f42377fe640 1 -- 172.21.15.6:0/4058589957 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42440092e0 con 0x7f42500fb110 2024-03-22T06:27:01.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f42377fe640 1 -- 172.21.15.6:0/4058589957 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4244043640 con 0x7f42500fb110 2024-03-22T06:27:01.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.185+0000 7f42550ae640 1 -- 172.21.15.6:0/4058589957 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425010f7f0 con 0x7f42500fb110 2024-03-22T06:27:01.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.186+0000 7f42550ae640 1 -- 172.21.15.6:0/4058589957 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42500fc800 con 0x7f42500fb110 2024-03-22T06:27:01.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.187+0000 7f42377fe640 1 -- 172.21.15.6:0/4058589957 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f4244009450 con 0x7f42500fb110 2024-03-22T06:27:01.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.187+0000 7f42377fe640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f423806e7c0 0x7f4238070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:01.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.188+0000 7f42377fe640 1 -- 172.21.15.6:0/4058589957 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f42440b1320 con 0x7f42500fb110 2024-03-22T06:27:01.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.188+0000 7f424e59c640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f423806e7c0 0x7f4238070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:01.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.188+0000 7f424e59c640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f423806e7c0 0x7f4238070c70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4250110780 tx=0x7f424000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:01.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.192+0000 7f42377fe640 1 -- 172.21.15.6:0/4058589957 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f424407a170 con 0x7f42500fb110 2024-03-22T06:27:01.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:00 smithi117 ceph-mon[82126]: pgmap v353: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:27:01.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:00 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1211444926' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-22T06:27:01.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:00 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1349458713' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-22T06:27:01.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:00 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2918569936' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-22T06:27:01.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.416+0000 7f42550ae640 1 -- 172.21.15.6:0/4058589957 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f4250110530 con 0x7f42500fb110 2024-03-22T06:27:01.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.417+0000 7f42377fe640 1 -- 172.21.15.6:0/4058589957 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v36) v1 ==== 107+0+4140 (secure 0 0 0) 0x7f424407d830 con 0x7f42500fb110 2024-03-22T06:27:01.417 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:01.417 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:23:03.668034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24281":{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":0,"incarnation":26,"state":"up:replay","state_seq":2,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:27:01.417 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-03-22T06:27:01.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.419+0000 7f42550ae640 1 -- 172.21.15.6:0/4058589957 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f423806e7c0 msgr2=0x7f4238070c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.419+0000 7f42550ae640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f423806e7c0 0x7f4238070c70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4250110780 tx=0x7f424000b040 comp rx=0 tx=0).stop 2024-03-22T06:27:01.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.419+0000 7f42550ae640 1 -- 172.21.15.6:0/4058589957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42500fb110 msgr2=0x7f4250114ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.419+0000 7f42550ae640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42500fb110 0x7f4250114ec0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42440043d0 tx=0x7f4244009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.419+0000 7f42550ae640 1 -- 172.21.15.6:0/4058589957 shutdown_connections 2024-03-22T06:27:01.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.419+0000 7f42550ae640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f423806e7c0 0x7f4238070c70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.419+0000 7f42550ae640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4250101140 0x7f4250115400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.419+0000 7f42550ae640 1 --2- 172.21.15.6:0/4058589957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42500fb110 0x7f4250114ec0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.419+0000 7f42550ae640 1 -- 172.21.15.6:0/4058589957 >> 172.21.15.6:0/4058589957 conn(0x7f42500710b0 msgr2=0x7f42500f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:01.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.419+0000 7f42550ae640 1 -- 172.21.15.6:0/4058589957 shutdown_connections 2024-03-22T06:27:01.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.419+0000 7f42550ae640 1 -- 172.21.15.6:0/4058589957 wait complete. 2024-03-22T06:27:01.430 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-22T06:27:01.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.530+0000 7fd068b9d640 1 -- 172.21.15.6:0/2460281880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd064102240 msgr2=0x7fd064102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.530+0000 7fd068b9d640 1 --2- 172.21.15.6:0/2460281880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd064102240 0x7fd064102610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd0540048d0 tx=0x7fd054009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.530+0000 7fd068b9d640 1 -- 172.21.15.6:0/2460281880 shutdown_connections 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.530+0000 7fd068b9d640 1 --2- 172.21.15.6:0/2460281880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0640643a0 0x7fd0640647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.530+0000 7fd068b9d640 1 --2- 172.21.15.6:0/2460281880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd064102240 0x7fd064102610 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.530+0000 7fd068b9d640 1 -- 172.21.15.6:0/2460281880 >> 172.21.15.6:0/2460281880 conn(0x7fd0640710b0 msgr2=0x7fd0640714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.530+0000 7fd068b9d640 1 -- 172.21.15.6:0/2460281880 shutdown_connections 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.530+0000 7fd068b9d640 1 -- 172.21.15.6:0/2460281880 wait complete. 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd068b9d640 1 Processor -- start 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd068b9d640 1 -- start start 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd068b9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0640643a0 0x7fd064112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd068b9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064102240 0x7fd064113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd068b9d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064113850 con 0x7fd0640643a0 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd068b9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06410cea0 con 0x7fd064102240 2024-03-22T06:27:01.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd06259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0640643a0 0x7fd064112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd06259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0640643a0 0x7fd064112d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35584/0 (socket says 172.21.15.6:35584) 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd06259c640 1 -- 172.21.15.6:0/202658342 learned_addr learned my addr 172.21.15.6:0/202658342 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd061d9b640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064102240 0x7fd064113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd06259c640 1 -- 172.21.15.6:0/202658342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064102240 msgr2=0x7fd064113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd06259c640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064102240 0x7fd064113250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd06259c640 1 -- 172.21.15.6:0/202658342 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd054005040 con 0x7fd0640643a0 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.531+0000 7fd061d9b640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064102240 0x7fd064113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.532+0000 7fd06259c640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0640643a0 0x7fd064112d10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd054004930 tx=0x7fd054009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.532+0000 7fd052ffd640 1 -- 172.21.15.6:0/202658342 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0540094c0 con 0x7fd0640643a0 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.532+0000 7fd052ffd640 1 -- 172.21.15.6:0/202658342 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd054032e50 con 0x7fd0640643a0 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.532+0000 7fd068b9d640 1 -- 172.21.15.6:0/202658342 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06410d120 con 0x7fd0640643a0 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.532+0000 7fd052ffd640 1 -- 172.21.15.6:0/202658342 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0540433b0 con 0x7fd0640643a0 2024-03-22T06:27:01.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.532+0000 7fd068b9d640 1 -- 172.21.15.6:0/202658342 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06410d5e0 con 0x7fd0640643a0 2024-03-22T06:27:01.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.533+0000 7fd068b9d640 1 -- 172.21.15.6:0/202658342 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0640fd940 con 0x7fd0640643a0 2024-03-22T06:27:01.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.533+0000 7fd052ffd640 1 -- 172.21.15.6:0/202658342 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fd05402f070 con 0x7fd0640643a0 2024-03-22T06:27:01.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.534+0000 7fd052ffd640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd04c06e7b0 0x7fd04c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:01.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.534+0000 7fd061d9b640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd04c06e7b0 0x7fd04c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:01.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.534+0000 7fd052ffd640 1 -- 172.21.15.6:0/202658342 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fd0540b0360 con 0x7fd0640643a0 2024-03-22T06:27:01.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.534+0000 7fd061d9b640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd04c06e7b0 0x7fd04c070c60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd058009af0 tx=0x7fd05800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:01.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.542+0000 7fd052ffd640 1 -- 172.21.15.6:0/202658342 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fd054079130 con 0x7fd0640643a0 2024-03-22T06:27:01.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.769+0000 7fd068b9d640 1 -- 172.21.15.6:0/202658342 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fd06410e380 con 0x7fd0640643a0 2024-03-22T06:27:01.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.770+0000 7fd052ffd640 1 -- 172.21.15.6:0/202658342 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v36) v1 ==== 107+0+4145 (secure 0 0 0) 0x7fd05407c7f0 con 0x7fd0640643a0 2024-03-22T06:27:01.770 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:01.770 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:23:11.503720+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24281":{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":161,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:27:01.770 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-03-22T06:27:01.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.772+0000 7fd068b9d640 1 -- 172.21.15.6:0/202658342 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd04c06e7b0 msgr2=0x7fd04c070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.772+0000 7fd068b9d640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd04c06e7b0 0x7fd04c070c60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd058009af0 tx=0x7fd05800b040 comp rx=0 tx=0).stop 2024-03-22T06:27:01.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.772+0000 7fd068b9d640 1 -- 172.21.15.6:0/202658342 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0640643a0 msgr2=0x7fd064112d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.772+0000 7fd068b9d640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0640643a0 0x7fd064112d10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd054004930 tx=0x7fd054009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.772+0000 7fd068b9d640 1 -- 172.21.15.6:0/202658342 shutdown_connections 2024-03-22T06:27:01.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.772+0000 7fd068b9d640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fd04c06e7b0 0x7fd04c070c60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.772+0000 7fd068b9d640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064102240 0x7fd064113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.772+0000 7fd068b9d640 1 --2- 172.21.15.6:0/202658342 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0640643a0 0x7fd064112d10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.772+0000 7fd068b9d640 1 -- 172.21.15.6:0/202658342 >> 172.21.15.6:0/202658342 conn(0x7fd0640710b0 msgr2=0x7fd0640f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:01.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.772+0000 7fd068b9d640 1 -- 172.21.15.6:0/202658342 shutdown_connections 2024-03-22T06:27:01.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.772+0000 7fd068b9d640 1 -- 172.21.15.6:0/202658342 wait complete. 2024-03-22T06:27:01.784 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-22T06:27:01.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.883+0000 7f34eb55d640 1 -- 172.21.15.6:0/2631530699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34e40643a0 msgr2=0x7f34e40647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.883+0000 7f34eb55d640 1 --2- 172.21.15.6:0/2631530699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34e40643a0 0x7f34e40647d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f34d40048d0 tx=0x7f34d4009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.883+0000 7f34eb55d640 1 -- 172.21.15.6:0/2631530699 shutdown_connections 2024-03-22T06:27:01.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.883+0000 7f34eb55d640 1 --2- 172.21.15.6:0/2631530699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34e40643a0 0x7f34e40647d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.883+0000 7f34eb55d640 1 --2- 172.21.15.6:0/2631530699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34e4102240 0x7f34e4102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.883+0000 7f34eb55d640 1 -- 172.21.15.6:0/2631530699 >> 172.21.15.6:0/2631530699 conn(0x7f34e40710b0 msgr2=0x7f34e40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:01.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.883+0000 7f34eb55d640 1 -- 172.21.15.6:0/2631530699 shutdown_connections 2024-03-22T06:27:01.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.883+0000 7f34eb55d640 1 -- 172.21.15.6:0/2631530699 wait complete. 2024-03-22T06:27:01.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.884+0000 7f34eb55d640 1 Processor -- start 2024-03-22T06:27:01.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.884+0000 7f34eb55d640 1 -- start start 2024-03-22T06:27:01.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.884+0000 7f34eb55d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34e40643a0 0x7f34e4112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:01.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.884+0000 7f34eb55d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34e4102240 0x7f34e4113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:01.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.884+0000 7f34eb55d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e4113800 con 0x7f34e4102240 2024-03-22T06:27:01.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.884+0000 7f34eb55d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e410ce50 con 0x7f34e40643a0 2024-03-22T06:27:01.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.884+0000 7f34e8af8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34e4102240 0x7f34e4113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:01.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.884+0000 7f34e8af8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34e4102240 0x7f34e4113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35606/0 (socket says 172.21.15.6:35606) 2024-03-22T06:27:01.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.884+0000 7f34e8af8640 1 -- 172.21.15.6:0/2947115215 learned_addr learned my addr 172.21.15.6:0/2947115215 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:01.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.885+0000 7f34e92f9640 1 --2- 172.21.15.6:0/2947115215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34e40643a0 0x7f34e4112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:01.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.885+0000 7f34e8af8640 1 -- 172.21.15.6:0/2947115215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34e40643a0 msgr2=0x7f34e4112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:01.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.885+0000 7f34e8af8640 1 --2- 172.21.15.6:0/2947115215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34e40643a0 0x7f34e4112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:01.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.885+0000 7f34e8af8640 1 -- 172.21.15.6:0/2947115215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d4005040 con 0x7f34e4102240 2024-03-22T06:27:01.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.885+0000 7f34e8af8640 1 --2- 172.21.15.6:0/2947115215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34e4102240 0x7f34e4113200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f34d4004930 tx=0x7f34d4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:01.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.885+0000 7f34d9ffb640 1 -- 172.21.15.6:0/2947115215 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d40094c0 con 0x7f34e4102240 2024-03-22T06:27:01.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.885+0000 7f34d9ffb640 1 -- 172.21.15.6:0/2947115215 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34d4032e50 con 0x7f34e4102240 2024-03-22T06:27:01.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.885+0000 7f34eb55d640 1 -- 172.21.15.6:0/2947115215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e410d0d0 con 0x7f34e4102240 2024-03-22T06:27:01.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.885+0000 7f34d9ffb640 1 -- 172.21.15.6:0/2947115215 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d40433b0 con 0x7f34e4102240 2024-03-22T06:27:01.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.885+0000 7f34eb55d640 1 -- 172.21.15.6:0/2947115215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e410d590 con 0x7f34e4102240 2024-03-22T06:27:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.886+0000 7f34eb55d640 1 -- 172.21.15.6:0/2947115215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e40fd940 con 0x7f34e4102240 2024-03-22T06:27:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.887+0000 7f34d9ffb640 1 -- 172.21.15.6:0/2947115215 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f34d402f070 con 0x7f34e4102240 2024-03-22T06:27:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.887+0000 7f34d9ffb640 1 --2- 172.21.15.6:0/2947115215 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f34d006e7b0 0x7f34d0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.887+0000 7f34d9ffb640 1 -- 172.21.15.6:0/2947115215 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f34d40b0550 con 0x7f34e4102240 2024-03-22T06:27:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.887+0000 7f34e92f9640 1 --2- 172.21.15.6:0/2947115215 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f34d006e7b0 0x7f34d0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.888+0000 7f34e92f9640 1 --2- 172.21.15.6:0/2947115215 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f34d006e7b0 0x7f34d0070c60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009fd0 tx=0x7f34e0009e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:01.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:01.892+0000 7f34d9ffb640 1 -- 172.21.15.6:0/2947115215 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f34d4079280 con 0x7f34e4102240 2024-03-22T06:27:02.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.137+0000 7f34eb55d640 1 -- 172.21.15.6:0/2947115215 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f34e4109390 con 0x7f34e4102240 2024-03-22T06:27:02.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.137+0000 7f34d9ffb640 1 -- 172.21.15.6:0/2947115215 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v36) v1 ==== 107+0+4142 (secure 0 0 0) 0x7f34d407c940 con 0x7f34e4102240 2024-03-22T06:27:02.138 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:02.138 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:23:11.959564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24281":{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":162,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:27:02.139 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 28 2024-03-22T06:27:02.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.140+0000 7f34eb55d640 1 -- 172.21.15.6:0/2947115215 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f34d006e7b0 msgr2=0x7f34d0070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.140+0000 7f34eb55d640 1 --2- 172.21.15.6:0/2947115215 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f34d006e7b0 0x7f34d0070c60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009fd0 tx=0x7f34e0009e50 comp rx=0 tx=0).stop 2024-03-22T06:27:02.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.140+0000 7f34eb55d640 1 -- 172.21.15.6:0/2947115215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34e4102240 msgr2=0x7f34e4113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.140+0000 7f34eb55d640 1 --2- 172.21.15.6:0/2947115215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34e4102240 0x7f34e4113200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f34d4004930 tx=0x7f34d4009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.140+0000 7f34eb55d640 1 -- 172.21.15.6:0/2947115215 shutdown_connections 2024-03-22T06:27:02.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.140+0000 7f34eb55d640 1 --2- 172.21.15.6:0/2947115215 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f34d006e7b0 0x7f34d0070c60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.140+0000 7f34eb55d640 1 --2- 172.21.15.6:0/2947115215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34e4102240 0x7f34e4113200 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.140+0000 7f34eb55d640 1 --2- 172.21.15.6:0/2947115215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34e40643a0 0x7f34e4112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.140+0000 7f34eb55d640 1 -- 172.21.15.6:0/2947115215 >> 172.21.15.6:0/2947115215 conn(0x7f34e40710b0 msgr2=0x7f34e40f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:02.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.140+0000 7f34eb55d640 1 -- 172.21.15.6:0/2947115215 shutdown_connections 2024-03-22T06:27:02.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.140+0000 7f34eb55d640 1 -- 172.21.15.6:0/2947115215 wait complete. 2024-03-22T06:27:02.150 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:01 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/245630698' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-22T06:27:02.150 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:01 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/4058589957' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-22T06:27:02.150 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:01 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/202658342' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-22T06:27:02.151 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-22T06:27:02.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.254+0000 7fa93859d640 1 -- 172.21.15.6:0/602717724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9300fb110 msgr2=0x7fa9300fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.254+0000 7fa93859d640 1 --2- 172.21.15.6:0/602717724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9300fb110 0x7fa9300fb560 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0048d0 tx=0x7fa92c009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 -- 172.21.15.6:0/602717724 shutdown_connections 2024-03-22T06:27:02.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 --2- 172.21.15.6:0/602717724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9300fb110 0x7fa9300fb560 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 --2- 172.21.15.6:0/602717724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa930101140 0x7fa930101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 -- 172.21.15.6:0/602717724 >> 172.21.15.6:0/602717724 conn(0x7fa9300710b0 msgr2=0x7fa9300714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:02.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 -- 172.21.15.6:0/602717724 shutdown_connections 2024-03-22T06:27:02.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 -- 172.21.15.6:0/602717724 wait complete. 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 Processor -- start 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 -- start start 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9300fb110 0x7fa930103980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930101140 0x7fa930101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9301024c0 con 0x7fa930101140 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa93859d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa930102630 con 0x7fa9300fb110 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa935b38640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930101140 0x7fa930101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa935b38640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930101140 0x7fa930101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35628/0 (socket says 172.21.15.6:35628) 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa935b38640 1 -- 172.21.15.6:0/981835122 learned_addr learned my addr 172.21.15.6:0/981835122 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.255+0000 7fa936339640 1 --2- 172.21.15.6:0/981835122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9300fb110 0x7fa930103980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.256+0000 7fa935b38640 1 -- 172.21.15.6:0/981835122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9300fb110 msgr2=0x7fa930103980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.256+0000 7fa935b38640 1 --2- 172.21.15.6:0/981835122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9300fb110 0x7fa930103980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.256+0000 7fa935b38640 1 -- 172.21.15.6:0/981835122 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c005040 con 0x7fa930101140 2024-03-22T06:27:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.256+0000 7fa935b38640 1 --2- 172.21.15.6:0/981835122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930101140 0x7fa930101ef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0041d0 tx=0x7fa92c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.256+0000 7fa922ffd640 1 -- 172.21.15.6:0/981835122 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa92c0092e0 con 0x7fa930101140 2024-03-22T06:27:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.256+0000 7fa922ffd640 1 -- 172.21.15.6:0/981835122 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa92c009440 con 0x7fa930101140 2024-03-22T06:27:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.256+0000 7fa93859d640 1 -- 172.21.15.6:0/981835122 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa930102850 con 0x7fa930101140 2024-03-22T06:27:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.256+0000 7fa922ffd640 1 -- 172.21.15.6:0/981835122 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa92c043640 con 0x7fa930101140 2024-03-22T06:27:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.256+0000 7fa93859d640 1 -- 172.21.15.6:0/981835122 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa930102d70 con 0x7fa930101140 2024-03-22T06:27:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.257+0000 7fa93859d640 1 -- 172.21.15.6:0/981835122 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9300fc800 con 0x7fa930101140 2024-03-22T06:27:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.257+0000 7fa922ffd640 1 -- 172.21.15.6:0/981835122 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fa92c03b9f0 con 0x7fa930101140 2024-03-22T06:27:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.258+0000 7fa922ffd640 1 --2- 172.21.15.6:0/981835122 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa91c06e7b0 0x7fa91c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.258+0000 7fa922ffd640 1 -- 172.21.15.6:0/981835122 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fa92c0b1200 con 0x7fa930101140 2024-03-22T06:27:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.258+0000 7fa936339640 1 --2- 172.21.15.6:0/981835122 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa91c06e7b0 0x7fa91c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:02.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:01 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/245630698' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-22T06:27:02.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:01 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/4058589957' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-22T06:27:02.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:01 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/202658342' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-22T06:27:02.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.258+0000 7fa936339640 1 --2- 172.21.15.6:0/981835122 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa91c06e7b0 0x7fa91c070c60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa924009b50 tx=0x7fa924009120 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:02.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.267+0000 7fa922ffd640 1 -- 172.21.15.6:0/981835122 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fa92c07a110 con 0x7fa930101140 2024-03-22T06:27:02.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.491+0000 7fa93859d640 1 -- 172.21.15.6:0/981835122 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fa93004bcc0 con 0x7fa930101140 2024-03-22T06:27:02.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.491+0000 7fa922ffd640 1 -- 172.21.15.6:0/981835122 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v36) v1 ==== 107+0+4151 (secure 0 0 0) 0x7fa92c07d7d0 con 0x7fa930101140 2024-03-22T06:27:02.491 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:02.492 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:23:13.963052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24281":{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":0,"incarnation":26,"state":"up:active","state_seq":163,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24281,"qdb_cluster":[24281]},"id":1}]} 2024-03-22T06:27:02.492 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 29 2024-03-22T06:27:02.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.493+0000 7fa93859d640 1 -- 172.21.15.6:0/981835122 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa91c06e7b0 msgr2=0x7fa91c070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.493+0000 7fa93859d640 1 --2- 172.21.15.6:0/981835122 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa91c06e7b0 0x7fa91c070c60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa924009b50 tx=0x7fa924009120 comp rx=0 tx=0).stop 2024-03-22T06:27:02.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.493+0000 7fa93859d640 1 -- 172.21.15.6:0/981835122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930101140 msgr2=0x7fa930101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.493+0000 7fa93859d640 1 --2- 172.21.15.6:0/981835122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930101140 0x7fa930101ef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0041d0 tx=0x7fa92c009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.493+0000 7fa93859d640 1 -- 172.21.15.6:0/981835122 shutdown_connections 2024-03-22T06:27:02.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.493+0000 7fa93859d640 1 --2- 172.21.15.6:0/981835122 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fa91c06e7b0 0x7fa91c070c60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.493+0000 7fa93859d640 1 --2- 172.21.15.6:0/981835122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa930101140 0x7fa930101ef0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.493+0000 7fa93859d640 1 --2- 172.21.15.6:0/981835122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9300fb110 0x7fa930103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.493+0000 7fa93859d640 1 -- 172.21.15.6:0/981835122 >> 172.21.15.6:0/981835122 conn(0x7fa9300710b0 msgr2=0x7fa9300f9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:02.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.493+0000 7fa93859d640 1 -- 172.21.15.6:0/981835122 shutdown_connections 2024-03-22T06:27:02.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.493+0000 7fa93859d640 1 -- 172.21.15.6:0/981835122 wait complete. 2024-03-22T06:27:02.504 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-22T06:27:02.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.612+0000 7fcf0c26e640 1 -- 172.21.15.6:0/1395537482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf040643a0 msgr2=0x7fcf040647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.612+0000 7fcf0c26e640 1 --2- 172.21.15.6:0/1395537482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf040643a0 0x7fcf040647d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcef40048d0 tx=0x7fcef4009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.612+0000 7fcf0c26e640 1 -- 172.21.15.6:0/1395537482 shutdown_connections 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.612+0000 7fcf0c26e640 1 --2- 172.21.15.6:0/1395537482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf040643a0 0x7fcf040647d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.612+0000 7fcf0c26e640 1 --2- 172.21.15.6:0/1395537482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf04102240 0x7fcf04102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.612+0000 7fcf0c26e640 1 -- 172.21.15.6:0/1395537482 >> 172.21.15.6:0/1395537482 conn(0x7fcf040710b0 msgr2=0x7fcf040714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.612+0000 7fcf0c26e640 1 -- 172.21.15.6:0/1395537482 shutdown_connections 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.612+0000 7fcf0c26e640 1 -- 172.21.15.6:0/1395537482 wait complete. 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0c26e640 1 Processor -- start 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0c26e640 1 -- start start 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0c26e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf040643a0 0x7fcf04114eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0c26e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf04102240 0x7fcf041153f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0c26e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf041159f0 con 0x7fcf040643a0 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0c26e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0410f040 con 0x7fcf04102240 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0a00a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf040643a0 0x7fcf04114eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0a00a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf040643a0 0x7fcf04114eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35648/0 (socket says 172.21.15.6:35648) 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0a00a640 1 -- 172.21.15.6:0/832741657 learned_addr learned my addr 172.21.15.6:0/832741657 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:02.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0a00a640 1 -- 172.21.15.6:0/832741657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf04102240 msgr2=0x7fcf041153f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf09809640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf04102240 0x7fcf041153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:02.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0a00a640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf04102240 0x7fcf041153f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf0a00a640 1 -- 172.21.15.6:0/832741657 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4005040 con 0x7fcf040643a0 2024-03-22T06:27:02.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.613+0000 7fcf09809640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf04102240 0x7fcf041153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:27:02.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.614+0000 7fcf0a00a640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf040643a0 0x7fcf04114eb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000c790 tx=0x7fcf000043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:02.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.614+0000 7fcefaffd640 1 -- 172.21.15.6:0/832741657 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf0000a820 con 0x7fcf040643a0 2024-03-22T06:27:02.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.614+0000 7fcefaffd640 1 -- 172.21.15.6:0/832741657 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf0000ae50 con 0x7fcf040643a0 2024-03-22T06:27:02.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.614+0000 7fcf0c26e640 1 -- 172.21.15.6:0/832741657 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0410f320 con 0x7fcf040643a0 2024-03-22T06:27:02.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.614+0000 7fcefaffd640 1 -- 172.21.15.6:0/832741657 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf000115e0 con 0x7fcf040643a0 2024-03-22T06:27:02.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.614+0000 7fcf0c26e640 1 -- 172.21.15.6:0/832741657 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0410f840 con 0x7fcf040643a0 2024-03-22T06:27:02.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.615+0000 7fcf0c26e640 1 -- 172.21.15.6:0/832741657 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf040fd940 con 0x7fcf040643a0 2024-03-22T06:27:02.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.615+0000 7fcefaffd640 1 -- 172.21.15.6:0/832741657 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fcf00012400 con 0x7fcf040643a0 2024-03-22T06:27:02.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.616+0000 7fcefaffd640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7c0 0x7fcef0070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:02.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.616+0000 7fcf09809640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7c0 0x7fcef0070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:02.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.616+0000 7fcefaffd640 1 -- 172.21.15.6:0/832741657 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fcf0008e770 con 0x7fcf040643a0 2024-03-22T06:27:02.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.616+0000 7fcf09809640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7c0 0x7fcef0070c70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcef4000c00 tx=0x7fcef4009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:02.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.620+0000 7fcefaffd640 1 -- 172.21.15.6:0/832741657 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fcf000575c0 con 0x7fcf040643a0 2024-03-22T06:27:02.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.845+0000 7fcf0c26e640 1 -- 172.21.15.6:0/832741657 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fcf0404bcc0 con 0x7fcf040643a0 2024-03-22T06:27:02.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.846+0000 7fcefaffd640 1 -- 172.21.15.6:0/832741657 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v36) v1 ==== 107+0+4967 (secure 0 0 0) 0x7fcf0005ac80 con 0x7fcf040643a0 2024-03-22T06:27:02.847 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:02.847 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44335,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/1194667779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1194667779},{"type":"v1","addr":"172.21.15.117:6825","nonce":1194667779}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:23:13.963052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24281":{"gid":24281,"name":"cephfs.smithi117.zqxyql","rank":0,"incarnation":26,"state":"up:active","state_seq":163,"addr":"172.21.15.117:6827/1010515354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1010515354},{"type":"v1","addr":"172.21.15.117:6827","nonce":1010515354}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24281,"qdb_cluster":[24281]},"id":1}]} 2024-03-22T06:27:02.847 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 30 2024-03-22T06:27:02.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.848+0000 7fcf0c26e640 1 -- 172.21.15.6:0/832741657 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7c0 msgr2=0x7fcef0070c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.848+0000 7fcf0c26e640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7c0 0x7fcef0070c70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcef4000c00 tx=0x7fcef4009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.848+0000 7fcf0c26e640 1 -- 172.21.15.6:0/832741657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf040643a0 msgr2=0x7fcf04114eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.848+0000 7fcf0c26e640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf040643a0 0x7fcf04114eb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000c790 tx=0x7fcf000043c0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.848+0000 7fcf0c26e640 1 -- 172.21.15.6:0/832741657 shutdown_connections 2024-03-22T06:27:02.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.848+0000 7fcf0c26e640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fcef006e7c0 0x7fcef0070c70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.848+0000 7fcf0c26e640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf04102240 0x7fcf041153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.848+0000 7fcf0c26e640 1 --2- 172.21.15.6:0/832741657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf040643a0 0x7fcf04114eb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.848+0000 7fcf0c26e640 1 -- 172.21.15.6:0/832741657 >> 172.21.15.6:0/832741657 conn(0x7fcf040710b0 msgr2=0x7fcf040fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:02.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.848+0000 7fcf0c26e640 1 -- 172.21.15.6:0/832741657 shutdown_connections 2024-03-22T06:27:02.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.848+0000 7fcf0c26e640 1 -- 172.21.15.6:0/832741657 wait complete. 2024-03-22T06:27:02.859 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-22T06:27:02.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.958+0000 7f72e342e640 1 -- 172.21.15.6:0/192228212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72dc1012c0 msgr2=0x7f72dc101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.958+0000 7f72e342e640 1 --2- 172.21.15.6:0/192228212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72dc1012c0 0x7f72dc101690 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f72d80048d0 tx=0x7f72d8009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.958+0000 7f72e342e640 1 -- 172.21.15.6:0/192228212 shutdown_connections 2024-03-22T06:27:02.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.958+0000 7f72e342e640 1 --2- 172.21.15.6:0/192228212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72dc101c60 0x7f72dc0fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.958+0000 7f72e342e640 1 --2- 172.21.15.6:0/192228212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72dc1012c0 0x7f72dc101690 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.958+0000 7f72e342e640 1 -- 172.21.15.6:0/192228212 >> 172.21.15.6:0/192228212 conn(0x7f72dc0f6eb0 msgr2=0x7f72dc0f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.958+0000 7f72e342e640 1 -- 172.21.15.6:0/192228212 shutdown_connections 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.958+0000 7f72e342e640 1 -- 172.21.15.6:0/192228212 wait complete. 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e342e640 1 Processor -- start 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e342e640 1 -- start start 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e342e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72dc1012c0 0x7f72dc111e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e342e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72dc101c60 0x7f72dc112350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e342e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc112950 con 0x7f72dc1012c0 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e342e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc10bfa0 con 0x7f72dc101c60 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e11ca640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72dc1012c0 0x7f72dc111e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e11ca640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72dc1012c0 0x7f72dc111e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35664/0 (socket says 172.21.15.6:35664) 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e11ca640 1 -- 172.21.15.6:0/516181988 learned_addr learned my addr 172.21.15.6:0/516181988 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e11ca640 1 -- 172.21.15.6:0/516181988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72dc101c60 msgr2=0x7f72dc112350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e11ca640 1 --2- 172.21.15.6:0/516181988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72dc101c60 0x7f72dc112350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:02.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.959+0000 7f72e11ca640 1 -- 172.21.15.6:0/516181988 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d8005040 con 0x7f72dc1012c0 2024-03-22T06:27:02.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.960+0000 7f72e11ca640 1 --2- 172.21.15.6:0/516181988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72dc1012c0 0x7f72dc111e10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f72d8004930 tx=0x7f72d8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:02.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.960+0000 7f72d1ffb640 1 -- 172.21.15.6:0/516181988 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d80094c0 con 0x7f72dc1012c0 2024-03-22T06:27:02.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.960+0000 7f72d1ffb640 1 -- 172.21.15.6:0/516181988 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72d8032e50 con 0x7f72dc1012c0 2024-03-22T06:27:02.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.960+0000 7f72e342e640 1 -- 172.21.15.6:0/516181988 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc10c220 con 0x7f72dc1012c0 2024-03-22T06:27:02.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.960+0000 7f72d1ffb640 1 -- 172.21.15.6:0/516181988 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d80433b0 con 0x7f72dc1012c0 2024-03-22T06:27:02.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.960+0000 7f72e342e640 1 -- 172.21.15.6:0/516181988 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc10c6e0 con 0x7f72dc1012c0 2024-03-22T06:27:02.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.961+0000 7f72e342e640 1 -- 172.21.15.6:0/516181988 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc0fca30 con 0x7f72dc1012c0 2024-03-22T06:27:02.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.961+0000 7f72d1ffb640 1 -- 172.21.15.6:0/516181988 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f72d802f070 con 0x7f72dc1012c0 2024-03-22T06:27:02.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.962+0000 7f72d1ffb640 1 --2- 172.21.15.6:0/516181988 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f72c806e7b0 0x7f72c8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:02.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.962+0000 7f72d1ffb640 1 -- 172.21.15.6:0/516181988 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f72d80b04b0 con 0x7f72dc1012c0 2024-03-22T06:27:02.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.962+0000 7f72e09c9640 1 --2- 172.21.15.6:0/516181988 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f72c806e7b0 0x7f72c8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:02.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.962+0000 7f72e09c9640 1 --2- 172.21.15.6:0/516181988 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f72c806e7b0 0x7f72c8070c60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72dc10d6d0 tx=0x7f72cc009c50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:02.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:02.970+0000 7f72d1ffb640 1 -- 172.21.15.6:0/516181988 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f72d8079280 con 0x7f72dc1012c0 2024-03-22T06:27:03.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:02 smithi117 ceph-mon[82126]: pgmap v354: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:27:03.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:02 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2947115215' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-22T06:27:03.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:02 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/981835122' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-22T06:27:03.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:02 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/832741657' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-22T06:27:03.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:02 smithi006 ceph-mon[94482]: pgmap v354: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:27:03.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:02 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2947115215' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-22T06:27:03.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:02 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/981835122' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-22T06:27:03.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:02 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/832741657' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-22T06:27:03.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.323+0000 7f72e342e640 1 -- 172.21.15.6:0/516181988 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f72dc108450 con 0x7f72dc1012c0 2024-03-22T06:27:03.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.324+0000 7f72d1ffb640 1 -- 172.21.15.6:0/516181988 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v36) v1 ==== 107+0+4163 (secure 0 0 0) 0x7f72d807c940 con 0x7f72dc1012c0 2024-03-22T06:27:03.324 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:03.324 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44335,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/1194667779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1194667779},{"type":"v1","addr":"172.21.15.117:6825","nonce":1194667779}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:23:28.644616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:27:03.325 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 31 2024-03-22T06:27:03.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.326+0000 7f72e342e640 1 -- 172.21.15.6:0/516181988 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f72c806e7b0 msgr2=0x7f72c8070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:03.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.326+0000 7f72e342e640 1 --2- 172.21.15.6:0/516181988 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f72c806e7b0 0x7f72c8070c60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72dc10d6d0 tx=0x7f72cc009c50 comp rx=0 tx=0).stop 2024-03-22T06:27:03.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.326+0000 7f72e342e640 1 -- 172.21.15.6:0/516181988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72dc1012c0 msgr2=0x7f72dc111e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:03.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.326+0000 7f72e342e640 1 --2- 172.21.15.6:0/516181988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72dc1012c0 0x7f72dc111e10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f72d8004930 tx=0x7f72d8009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.326+0000 7f72e342e640 1 -- 172.21.15.6:0/516181988 shutdown_connections 2024-03-22T06:27:03.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.326+0000 7f72e342e640 1 --2- 172.21.15.6:0/516181988 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f72c806e7b0 0x7f72c8070c60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.326+0000 7f72e342e640 1 --2- 172.21.15.6:0/516181988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72dc101c60 0x7f72dc112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.326+0000 7f72e342e640 1 --2- 172.21.15.6:0/516181988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72dc1012c0 0x7f72dc111e10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.326+0000 7f72e342e640 1 -- 172.21.15.6:0/516181988 >> 172.21.15.6:0/516181988 conn(0x7f72dc0f6eb0 msgr2=0x7f72dc0f7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:03.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.326+0000 7f72e342e640 1 -- 172.21.15.6:0/516181988 shutdown_connections 2024-03-22T06:27:03.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.326+0000 7f72e342e640 1 -- 172.21.15.6:0/516181988 wait complete. 2024-03-22T06:27:03.341 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-22T06:27:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.442+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2522491042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b480fb110 msgr2=0x7f4b480fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.442+0000 7f4b4d59d640 1 --2- 172.21.15.6:0/2522491042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b480fb110 0x7f4b480fb560 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0048d0 tx=0x7f4b3c009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.442+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2522491042 shutdown_connections 2024-03-22T06:27:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.442+0000 7f4b4d59d640 1 --2- 172.21.15.6:0/2522491042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b480fb110 0x7f4b480fb560 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.442+0000 7f4b4d59d640 1 --2- 172.21.15.6:0/2522491042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b48101140 0x7f4b48101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.442+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2522491042 >> 172.21.15.6:0/2522491042 conn(0x7f4b480710b0 msgr2=0x7f4b480714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.442+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2522491042 shutdown_connections 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.442+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2522491042 wait complete. 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.443+0000 7f4b4d59d640 1 Processor -- start 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.443+0000 7f4b4d59d640 1 -- start start 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.443+0000 7f4b4d59d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b480fb110 0x7f4b4810ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.443+0000 7f4b4d59d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b48101140 0x7f4b4810e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.443+0000 7f4b4d59d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b48065310 con 0x7f4b480fb110 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.443+0000 7f4b4d59d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4810e950 con 0x7f4b48101140 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.443+0000 7f4b46ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b480fb110 0x7f4b4810ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.443+0000 7f4b46ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b480fb110 0x7f4b4810ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35680/0 (socket says 172.21.15.6:35680) 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.443+0000 7f4b46ffd640 1 -- 172.21.15.6:0/2556397306 learned_addr learned my addr 172.21.15.6:0/2556397306 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.444+0000 7f4b46ffd640 1 -- 172.21.15.6:0/2556397306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b48101140 msgr2=0x7f4b4810e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.444+0000 7f4b46ffd640 1 --2- 172.21.15.6:0/2556397306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b48101140 0x7f4b4810e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.444+0000 7f4b46ffd640 1 -- 172.21.15.6:0/2556397306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c005040 con 0x7f4b480fb110 2024-03-22T06:27:03.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.444+0000 7f4b46ffd640 1 --2- 172.21.15.6:0/2556397306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b480fb110 0x7f4b4810ded0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4b38002a50 tx=0x7f4b38004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:03.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.444+0000 7f4b2f7fe640 1 -- 172.21.15.6:0/2556397306 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b38007c30 con 0x7f4b480fb110 2024-03-22T06:27:03.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.444+0000 7f4b2f7fe640 1 -- 172.21.15.6:0/2556397306 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b3800f040 con 0x7f4b480fb110 2024-03-22T06:27:03.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.444+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2556397306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4810ec30 con 0x7f4b480fb110 2024-03-22T06:27:03.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.444+0000 7f4b2f7fe640 1 -- 172.21.15.6:0/2556397306 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b3800db50 con 0x7f4b480fb110 2024-03-22T06:27:03.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.444+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2556397306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b481171f0 con 0x7f4b480fb110 2024-03-22T06:27:03.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.445+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2556397306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b480fc800 con 0x7f4b480fb110 2024-03-22T06:27:03.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.446+0000 7f4b2f7fe640 1 -- 172.21.15.6:0/2556397306 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f4b3800dcf0 con 0x7f4b480fb110 2024-03-22T06:27:03.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.446+0000 7f4b2f7fe640 1 --2- 172.21.15.6:0/2556397306 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4b3006e7b0 0x7f4b30070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:03.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.446+0000 7f4b2f7fe640 1 -- 172.21.15.6:0/2556397306 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f4b3808ddb0 con 0x7f4b480fb110 2024-03-22T06:27:03.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.446+0000 7f4b467fc640 1 --2- 172.21.15.6:0/2556397306 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4b3006e7b0 0x7f4b30070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:03.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.447+0000 7f4b467fc640 1 --2- 172.21.15.6:0/2556397306 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4b3006e7b0 0x7f4b30070c60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009990 tx=0x7f4b3c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:03.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.451+0000 7f4b2f7fe640 1 -- 172.21.15.6:0/2556397306 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f4b38056c00 con 0x7f4b480fb110 2024-03-22T06:27:03.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.855+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2556397306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f4b48117430 con 0x7f4b480fb110 2024-03-22T06:27:03.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.856+0000 7f4b2f7fe640 1 -- 172.21.15.6:0/2556397306 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v36) v1 ==== 107+0+4208 (secure 0 0 0) 0x7f4b3805a2c0 con 0x7f4b480fb110 2024-03-22T06:27:03.856 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:03.857 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44335,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/1194667779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1194667779},{"type":"v1","addr":"172.21.15.117:6825","nonce":1194667779}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:23:28.649511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:27:03.857 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 32 2024-03-22T06:27:03.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.858+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2556397306 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4b3006e7b0 msgr2=0x7f4b30070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:03.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.858+0000 7f4b4d59d640 1 --2- 172.21.15.6:0/2556397306 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4b3006e7b0 0x7f4b30070c60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009990 tx=0x7f4b3c009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.858+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2556397306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b480fb110 msgr2=0x7f4b4810ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:03.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.858+0000 7f4b4d59d640 1 --2- 172.21.15.6:0/2556397306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b480fb110 0x7f4b4810ded0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4b38002a50 tx=0x7f4b38004600 comp rx=0 tx=0).stop 2024-03-22T06:27:03.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.858+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2556397306 shutdown_connections 2024-03-22T06:27:03.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.858+0000 7f4b4d59d640 1 --2- 172.21.15.6:0/2556397306 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f4b3006e7b0 0x7f4b30070c60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.858+0000 7f4b4d59d640 1 --2- 172.21.15.6:0/2556397306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b48101140 0x7f4b4810e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.858+0000 7f4b4d59d640 1 --2- 172.21.15.6:0/2556397306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b480fb110 0x7f4b4810ded0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.858+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2556397306 >> 172.21.15.6:0/2556397306 conn(0x7f4b480710b0 msgr2=0x7f4b480f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:03.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.858+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2556397306 shutdown_connections 2024-03-22T06:27:03.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.858+0000 7f4b4d59d640 1 -- 172.21.15.6:0/2556397306 wait complete. 2024-03-22T06:27:03.869 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-22T06:27:03.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 -- 172.21.15.6:0/158286300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2e8101c70 msgr2=0x7ff2e80fb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:03.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 --2- 172.21.15.6:0/158286300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2e8101c70 0x7ff2e80fb300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0048d0 tx=0x7ff2dc009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 -- 172.21.15.6:0/158286300 shutdown_connections 2024-03-22T06:27:03.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 --2- 172.21.15.6:0/158286300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2e8101c70 0x7ff2e80fb300 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 --2- 172.21.15.6:0/158286300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e81012d0 0x7ff2e81016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 -- 172.21.15.6:0/158286300 >> 172.21.15.6:0/158286300 conn(0x7ff2e80f6ea0 msgr2=0x7ff2e80f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:03.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 -- 172.21.15.6:0/158286300 shutdown_connections 2024-03-22T06:27:03.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 -- 172.21.15.6:0/158286300 wait complete. 2024-03-22T06:27:03.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 Processor -- start 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 -- start start 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e81012d0 0x7ff2e810e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2e8101c70 0x7ff2e810e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e810eb90 con 0x7ff2e8101c70 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.969+0000 7ff2f071d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e810ed00 con 0x7ff2e81012d0 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2edcb8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2e8101c70 0x7ff2e810e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2edcb8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2e8101c70 0x7ff2e810e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52858/0 (socket says 172.21.15.6:52858) 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2edcb8640 1 -- 172.21.15.6:0/2575582474 learned_addr learned my addr 172.21.15.6:0/2575582474 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2edcb8640 1 -- 172.21.15.6:0/2575582474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e81012d0 msgr2=0x7ff2e810e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2ee4b9640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e81012d0 0x7ff2e810e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2edcb8640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e81012d0 0x7ff2e810e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2edcb8640 1 -- 172.21.15.6:0/2575582474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc005040 con 0x7ff2e8101c70 2024-03-22T06:27:03.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2ee4b9640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e81012d0 0x7ff2e810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:27:03.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2edcb8640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2e8101c70 0x7ff2e810e650 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc004930 tx=0x7ff2dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:03.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2e6ffd640 1 -- 172.21.15.6:0/2575582474 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2dc0094c0 con 0x7ff2e8101c70 2024-03-22T06:27:03.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2f071d640 1 -- 172.21.15.6:0/2575582474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e810ef80 con 0x7ff2e8101c70 2024-03-22T06:27:03.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2e6ffd640 1 -- 172.21.15.6:0/2575582474 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2dc032e50 con 0x7ff2e8101c70 2024-03-22T06:27:03.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2e6ffd640 1 -- 172.21.15.6:0/2575582474 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2dc0433b0 con 0x7ff2e8101c70 2024-03-22T06:27:03.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.970+0000 7ff2f071d640 1 -- 172.21.15.6:0/2575582474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e81173d0 con 0x7ff2e8101c70 2024-03-22T06:27:03.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.971+0000 7ff2f071d640 1 -- 172.21.15.6:0/2575582474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e80fca40 con 0x7ff2e8101c70 2024-03-22T06:27:03.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.972+0000 7ff2e6ffd640 1 -- 172.21.15.6:0/2575582474 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7ff2dc02f070 con 0x7ff2e8101c70 2024-03-22T06:27:03.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.972+0000 7ff2e6ffd640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d006e7b0 0x7ff2d0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:03.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.972+0000 7ff2e6ffd640 1 -- 172.21.15.6:0/2575582474 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7ff2dc0b03c0 con 0x7ff2e8101c70 2024-03-22T06:27:03.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.972+0000 7ff2ee4b9640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d006e7b0 0x7ff2d0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:03.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.973+0000 7ff2ee4b9640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d006e7b0 0x7ff2d0070c60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0004e80 tx=0x7ff2e0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:03.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:03.980+0000 7ff2e6ffd640 1 -- 172.21.15.6:0/2575582474 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7ff2dc079190 con 0x7ff2e8101c70 2024-03-22T06:27:04.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.223+0000 7ff2f071d640 1 -- 172.21.15.6:0/2575582474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7ff2e80638c0 con 0x7ff2e8101c70 2024-03-22T06:27:04.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.224+0000 7ff2e6ffd640 1 -- 172.21.15.6:0/2575582474 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v36) v1 ==== 107+0+4212 (secure 0 0 0) 0x7ff2dc07c850 con 0x7ff2e8101c70 2024-03-22T06:27:04.224 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:04.224 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44335,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/1194667779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1194667779},{"type":"v1","addr":"172.21.15.117:6825","nonce":1194667779}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:23:36.738980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:27:04.225 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 33 2024-03-22T06:27:04.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.226+0000 7ff2f071d640 1 -- 172.21.15.6:0/2575582474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d006e7b0 msgr2=0x7ff2d0070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:04.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.226+0000 7ff2f071d640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d006e7b0 0x7ff2d0070c60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0004e80 tx=0x7ff2e0009040 comp rx=0 tx=0).stop 2024-03-22T06:27:04.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.226+0000 7ff2f071d640 1 -- 172.21.15.6:0/2575582474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2e8101c70 msgr2=0x7ff2e810e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:04.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.226+0000 7ff2f071d640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2e8101c70 0x7ff2e810e650 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc004930 tx=0x7ff2dc009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.226+0000 7ff2f071d640 1 -- 172.21.15.6:0/2575582474 shutdown_connections 2024-03-22T06:27:04.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.226+0000 7ff2f071d640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff2d006e7b0 0x7ff2d0070c60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.226+0000 7ff2f071d640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2e8101c70 0x7ff2e810e650 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.226+0000 7ff2f071d640 1 --2- 172.21.15.6:0/2575582474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e81012d0 0x7ff2e810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.226+0000 7ff2f071d640 1 -- 172.21.15.6:0/2575582474 >> 172.21.15.6:0/2575582474 conn(0x7ff2e80f6ea0 msgr2=0x7ff2e80f7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:04.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.226+0000 7ff2f071d640 1 -- 172.21.15.6:0/2575582474 shutdown_connections 2024-03-22T06:27:04.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.226+0000 7ff2f071d640 1 -- 172.21.15.6:0/2575582474 wait complete. 2024-03-22T06:27:04.236 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:03 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/516181988' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-22T06:27:04.236 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:03 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2556397306' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-22T06:27:04.237 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-22T06:27:04.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:03 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/516181988' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-22T06:27:04.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:03 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2556397306' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-22T06:27:04.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.336+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2066969653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad64101140 msgr2=0x7fad64101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:04.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 --2- 172.21.15.6:0/2066969653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad64101140 0x7fad64101510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fad600048d0 tx=0x7fad60009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2066969653 shutdown_connections 2024-03-22T06:27:04.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 --2- 172.21.15.6:0/2066969653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad640fb110 0x7fad640fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 --2- 172.21.15.6:0/2066969653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad64101140 0x7fad64101510 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2066969653 >> 172.21.15.6:0/2066969653 conn(0x7fad640710b0 msgr2=0x7fad640714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:04.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2066969653 shutdown_connections 2024-03-22T06:27:04.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2066969653 wait complete. 2024-03-22T06:27:04.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 Processor -- start 2024-03-22T06:27:04.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 -- start start 2024-03-22T06:27:04.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad640fb110 0x7fad64114e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad64101140 0x7fad641153a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad641159a0 con 0x7fad640fb110 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.337+0000 7fad6bd5d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad64115b10 con 0x7fad64101140 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad69af9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad640fb110 0x7fad64114e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad69af9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad640fb110 0x7fad64114e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52870/0 (socket says 172.21.15.6:52870) 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad69af9640 1 -- 172.21.15.6:0/2290007431 learned_addr learned my addr 172.21.15.6:0/2290007431 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad692f8640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad64101140 0x7fad641153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad69af9640 1 -- 172.21.15.6:0/2290007431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad64101140 msgr2=0x7fad641153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad69af9640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad64101140 0x7fad641153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad69af9640 1 -- 172.21.15.6:0/2290007431 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad60005040 con 0x7fad640fb110 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad692f8640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad64101140 0x7fad641153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad69af9640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad640fb110 0x7fad64114e60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fad600041d0 tx=0x7fad60009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad5a7fc640 1 -- 172.21.15.6:0/2290007431 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad600092e0 con 0x7fad640fb110 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad5a7fc640 1 -- 172.21.15.6:0/2290007431 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad60009440 con 0x7fad640fb110 2024-03-22T06:27:04.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2290007431 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6410f140 con 0x7fad640fb110 2024-03-22T06:27:04.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad5a7fc640 1 -- 172.21.15.6:0/2290007431 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad60043640 con 0x7fad640fb110 2024-03-22T06:27:04.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.338+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2290007431 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6410f660 con 0x7fad640fb110 2024-03-22T06:27:04.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.339+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2290007431 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad640fc800 con 0x7fad640fb110 2024-03-22T06:27:04.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.340+0000 7fad5a7fc640 1 -- 172.21.15.6:0/2290007431 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fad6003b9f0 con 0x7fad640fb110 2024-03-22T06:27:04.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.340+0000 7fad5a7fc640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fad5006e7b0 0x7fad50070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:04.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.340+0000 7fad5a7fc640 1 -- 172.21.15.6:0/2290007431 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fad600b12b0 con 0x7fad640fb110 2024-03-22T06:27:04.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.340+0000 7fad692f8640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fad5006e7b0 0x7fad50070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:04.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.341+0000 7fad692f8640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fad5006e7b0 0x7fad50070c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fad64110760 tx=0x7fad54009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:04.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.345+0000 7fad5a7fc640 1 -- 172.21.15.6:0/2290007431 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fad6007a1a0 con 0x7fad640fb110 2024-03-22T06:27:04.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.568+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2290007431 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fad6404bcc0 con 0x7fad640fb110 2024-03-22T06:27:04.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.569+0000 7fad5a7fc640 1 -- 172.21.15.6:0/2290007431 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v36) v1 ==== 107+0+4209 (secure 0 0 0) 0x7fad6007d860 con 0x7fad640fb110 2024-03-22T06:27:04.569 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:04.569 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi006.dcjpla","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6829/1244315045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1244315045},{"type":"v1","addr":"172.21.15.6:6829","nonce":1244315045}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44335,"name":"cephfs.smithi117.fwkech","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/1194667779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1194667779},{"type":"v1","addr":"172.21.15.117:6825","nonce":1194667779}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T06:12:20.386024+0000","modified":"2024-03-22T06:23:37.062275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi006.bztopc","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.6:6827/192114945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":192114945},{"type":"v1","addr":"172.21.15.6:6827","nonce":192114945}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T06:27:04.570 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 34 2024-03-22T06:27:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.571+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2290007431 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fad5006e7b0 msgr2=0x7fad50070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:04.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.571+0000 7fad6bd5d640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fad5006e7b0 0x7fad50070c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fad64110760 tx=0x7fad54009040 comp rx=0 tx=0).stop 2024-03-22T06:27:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.571+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2290007431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad640fb110 msgr2=0x7fad64114e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.571+0000 7fad6bd5d640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad640fb110 0x7fad64114e60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fad600041d0 tx=0x7fad60009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.571+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2290007431 shutdown_connections 2024-03-22T06:27:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.571+0000 7fad6bd5d640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fad5006e7b0 0x7fad50070c60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.571+0000 7fad6bd5d640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad64101140 0x7fad641153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.571+0000 7fad6bd5d640 1 --2- 172.21.15.6:0/2290007431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad640fb110 0x7fad64114e60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.571+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2290007431 >> 172.21.15.6:0/2290007431 conn(0x7fad640710b0 msgr2=0x7fad640643b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.571+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2290007431 shutdown_connections 2024-03-22T06:27:04.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.571+0000 7fad6bd5d640 1 -- 172.21.15.6:0/2290007431 wait complete. 2024-03-22T06:27:04.583 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-22T06:27:04.595 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-22T06:27:04.595 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:27:04.595 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T06:27:04.611 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-22T06:27:04.611 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T06:27:04.668 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T06:27:04.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.805+0000 7f13eca2e640 1 -- 172.21.15.6:0/3110305473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13e8102240 msgr2=0x7f13e8102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.805+0000 7f13eca2e640 1 --2- 172.21.15.6:0/3110305473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13e8102240 0x7f13e8102610 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f13d80048d0 tx=0x7f13d8009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.805+0000 7f13eca2e640 1 -- 172.21.15.6:0/3110305473 shutdown_connections 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.805+0000 7f13eca2e640 1 --2- 172.21.15.6:0/3110305473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e80643a0 0x7f13e80647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.805+0000 7f13eca2e640 1 --2- 172.21.15.6:0/3110305473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13e8102240 0x7f13e8102610 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.805+0000 7f13eca2e640 1 -- 172.21.15.6:0/3110305473 >> 172.21.15.6:0/3110305473 conn(0x7f13e8071130 msgr2=0x7f13e8071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.805+0000 7f13eca2e640 1 -- 172.21.15.6:0/3110305473 shutdown_connections 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.805+0000 7f13eca2e640 1 -- 172.21.15.6:0/3110305473 wait complete. 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.806+0000 7f13eca2e640 1 Processor -- start 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.806+0000 7f13eca2e640 1 -- start start 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.806+0000 7f13eca2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e80643a0 0x7f13e80fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.806+0000 7f13eca2e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13e8102240 0x7f13e80fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.806+0000 7f13eca2e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80fd0c0 con 0x7f13e8102240 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.806+0000 7f13eca2e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80fbc60 con 0x7f13e80643a0 2024-03-22T06:27:04.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.806+0000 7f13dffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13e8102240 0x7f13e80fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.806+0000 7f13dffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13e8102240 0x7f13e80fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52886/0 (socket says 172.21.15.6:52886) 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.806+0000 7f13dffff640 1 -- 172.21.15.6:0/1063191241 learned_addr learned my addr 172.21.15.6:0/1063191241 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.806+0000 7f13e659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e80643a0 0x7f13e80fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.807+0000 7f13dffff640 1 -- 172.21.15.6:0/1063191241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e80643a0 msgr2=0x7f13e80fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.807+0000 7f13dffff640 1 --2- 172.21.15.6:0/1063191241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e80643a0 0x7f13e80fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.807+0000 7f13dffff640 1 -- 172.21.15.6:0/1063191241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8005040 con 0x7f13e8102240 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.807+0000 7f13dffff640 1 --2- 172.21.15.6:0/1063191241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13e8102240 0x7f13e80fb720 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f13d000c790 tx=0x7f13d00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.807+0000 7f13deffd640 1 -- 172.21.15.6:0/1063191241 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13d000a7a0 con 0x7f13e8102240 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.807+0000 7f13deffd640 1 -- 172.21.15.6:0/1063191241 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13d000add0 con 0x7f13e8102240 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.807+0000 7f13eca2e640 1 -- 172.21.15.6:0/1063191241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e80fbf40 con 0x7f13e8102240 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.807+0000 7f13deffd640 1 -- 172.21.15.6:0/1063191241 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13d00115e0 con 0x7f13e8102240 2024-03-22T06:27:04.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.807+0000 7f13eca2e640 1 -- 172.21.15.6:0/1063191241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e81193d0 con 0x7f13e8102240 2024-03-22T06:27:04.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.808+0000 7f13eca2e640 1 -- 172.21.15.6:0/1063191241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e80fd940 con 0x7f13e8102240 2024-03-22T06:27:04.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.809+0000 7f13deffd640 1 -- 172.21.15.6:0/1063191241 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f13d0012400 con 0x7f13e8102240 2024-03-22T06:27:04.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.809+0000 7f13deffd640 1 --2- 172.21.15.6:0/1063191241 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f13d406e7b0 0x7f13d4070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:04.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.810+0000 7f13deffd640 1 -- 172.21.15.6:0/1063191241 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f13d008e630 con 0x7f13e8102240 2024-03-22T06:27:04.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.810+0000 7f13e659c640 1 --2- 172.21.15.6:0/1063191241 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f13d406e7b0 0x7f13d4070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:04.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.810+0000 7f13e659c640 1 --2- 172.21.15.6:0/1063191241 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f13d406e7b0 0x7f13d4070c60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13d8000c00 tx=0x7f13d8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:04.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:04.814+0000 7f13deffd640 1 -- 172.21.15.6:0/1063191241 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f13d0057480 con 0x7f13e8102240 2024-03-22T06:27:05.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.001+0000 7f13eca2e640 1 -- 172.21.15.6:0/1063191241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13e804bcc0 con 0x7f13e8102240 2024-03-22T06:27:05.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.003+0000 7f13deffd640 1 -- 172.21.15.6:0/1063191241 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+17295 (secure 0 0 0) 0x7f13d005ab40 con 0x7f13e8102240 2024-03-22T06:27:05.003 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:05.003 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":95,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","created":"2024-03-22T06:05:27.379207+0000","modified":"2024-03-22T06:23:28.644346+0000","last_up_change":"2024-03-22T06:22:07.841620+0000","last_in_change":"2024-03-22T06:10:39.982933+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T06:09:46.449392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T06:12:19.302857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T06:12:19.628730+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ecd88bb1-d643-4505-a098-3728178c89e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":90,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6803","nonce":1666131505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6805","nonce":1666131505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6809","nonce":1666131505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6807","nonce":1666131505}]},"public_addr":"172.21.15.6:6803/1666131505","cluster_addr":"172.21.15.6:6805/1666131505","heartbeat_back_addr":"172.21.15.6:6809/1666131505","heartbeat_front_addr":"172.21.15.6:6807/1666131505","state":["exists","up"]},{"osd":1,"uuid":"f1b55307-2c3b-4ab8-a893-af8072f333ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":66,"up_thru":90,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6811","nonce":1922945925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6813","nonce":1922945925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6817","nonce":1922945925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6815","nonce":1922945925}]},"public_addr":"172.21.15.6:6811/1922945925","cluster_addr":"172.21.15.6:6813/1922945925","heartbeat_back_addr":"172.21.15.6:6817/1922945925","heartbeat_front_addr":"172.21.15.6:6815/1922945925","state":["exists","up"]},{"osd":2,"uuid":"9bacfe42-3c78-4d86-bac0-0142cde76ff5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6819","nonce":195584623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6821","nonce":195584623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6825","nonce":195584623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6823","nonce":195584623}]},"public_addr":"172.21.15.6:6819/195584623","cluster_addr":"172.21.15.6:6821/195584623","heartbeat_back_addr":"172.21.15.6:6825/195584623","heartbeat_front_addr":"172.21.15.6:6823/195584623","state":["exists","up"]},{"osd":3,"uuid":"2c5a6acd-de41-455d-b6c5-d53bb76197ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6801","nonce":1104353694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6803","nonce":1104353694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6807","nonce":1104353694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6805","nonce":1104353694}]},"public_addr":"172.21.15.117:6801/1104353694","cluster_addr":"172.21.15.117:6803/1104353694","heartbeat_back_addr":"172.21.15.117:6807/1104353694","heartbeat_front_addr":"172.21.15.117:6805/1104353694","state":["exists","up"]},{"osd":4,"uuid":"be424041-1e07-41f4-abe0-68fdd21b1eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6809","nonce":208554150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6811","nonce":208554150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6815","nonce":208554150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6813","nonce":208554150}]},"public_addr":"172.21.15.117:6809/208554150","cluster_addr":"172.21.15.117:6811/208554150","heartbeat_back_addr":"172.21.15.117:6815/208554150","heartbeat_front_addr":"172.21.15.117:6813/208554150","state":["exists","up"]},{"osd":5,"uuid":"bf61ff67-1f07-47ed-ad58-e6c946369d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6817","nonce":1416150897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6819","nonce":1416150897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6823","nonce":1416150897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6821","nonce":1416150897}]},"public_addr":"172.21.15.117:6817/1416150897","cluster_addr":"172.21.15.117:6819/1416150897","heartbeat_back_addr":"172.21.15.117:6823/1416150897","heartbeat_front_addr":"172.21.15.117:6821/1416150897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:08:45.922485+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:13.118065+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:42.409903+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:05.760002+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:31.339234+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:57.286867+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":0,"to":2}]},{"pgid":"2.7","mappings":[{"from":3,"to":1}]},{"pgid":"2.b","mappings":[{"from":3,"to":1}]},{"pgid":"2.e","mappings":[{"from":0,"to":1}]},{"pgid":"2.12","mappings":[{"from":0,"to":2}]},{"pgid":"2.16","mappings":[{"from":3,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":4,"to":0}]},{"pgid":"3.1d","mappings":[{"from":4,"to":0}]},{"pgid":"3.1f","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6827/1010515354":"2024-03-23T06:23:28.644314+0000","172.21.15.117:6826/1010515354":"2024-03-23T06:23:28.644314+0000","172.21.15.117:6825/1251227895":"2024-03-23T06:23:03.662256+0000","172.21.15.6:6829/947853733":"2024-03-23T06:22:41.442153+0000","172.21.15.6:6828/947853733":"2024-03-23T06:22:41.442153+0000","172.21.15.6:6827/1260089028":"2024-03-23T06:22:13.008321+0000","172.21.15.117:6829/1963654099":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/2826360973":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6800/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:6801/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2042811963":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6801/602685778":"2024-03-23T06:14:38.572050+0000","172.21.15.6:6801/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.117:0/45595122":"2024-03-23T06:16:28.822651+0000","172.21.15.117:6824/1251227895":"2024-03-23T06:23:03.662256+0000","172.21.15.6:0/1533067608":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/3502710787":"2024-03-23T06:17:37.246941+0000","172.21.15.6:6800/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/791727884":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/3606143293":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/254878217":"2024-03-23T06:14:38.572050+0000","172.21.15.6:0/3489834288":"2024-03-23T06:07:32.107836+0000","172.21.15.117:0/3344206523":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/2742769039":"2024-03-23T06:14:38.572050+0000","172.21.15.6:6801/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.117:0/435100846":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/1354987702":"2024-03-23T06:14:38.572050+0000","172.21.15.6:0/2939515329":"2024-03-23T06:05:54.100943+0000","172.21.15.117:0/3477422362":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/3665872066":"2024-03-23T06:06:24.832062+0000","172.21.15.6:0/619515244":"2024-03-23T06:05:54.100943+0000","172.21.15.6:6801/3502710787":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/1270793675":"2024-03-23T06:14:38.572050+0000","172.21.15.117:6828/1963654099":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/5416477":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/3809759464":"2024-03-23T06:17:37.246941+0000","172.21.15.6:6800/602685778":"2024-03-23T06:14:38.572050+0000","172.21.15.6:0/4154587507":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/1448591806":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/2671132128":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/1248395428":"2024-03-23T06:17:37.246941+0000","172.21.15.6:6826/1260089028":"2024-03-23T06:22:13.008321+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T06:27:05.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.005+0000 7f13eca2e640 1 -- 172.21.15.6:0/1063191241 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f13d406e7b0 msgr2=0x7f13d4070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.005+0000 7f13eca2e640 1 --2- 172.21.15.6:0/1063191241 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f13d406e7b0 0x7f13d4070c60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13d8000c00 tx=0x7f13d8009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.005+0000 7f13eca2e640 1 -- 172.21.15.6:0/1063191241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13e8102240 msgr2=0x7f13e80fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.005+0000 7f13eca2e640 1 --2- 172.21.15.6:0/1063191241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13e8102240 0x7f13e80fb720 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f13d000c790 tx=0x7f13d00043c0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.005+0000 7f13eca2e640 1 -- 172.21.15.6:0/1063191241 shutdown_connections 2024-03-22T06:27:05.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.005+0000 7f13eca2e640 1 --2- 172.21.15.6:0/1063191241 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f13d406e7b0 0x7f13d4070c60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.005+0000 7f13eca2e640 1 --2- 172.21.15.6:0/1063191241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13e8102240 0x7f13e80fb720 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.005+0000 7f13eca2e640 1 --2- 172.21.15.6:0/1063191241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e80643a0 0x7f13e80fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.005+0000 7f13eca2e640 1 -- 172.21.15.6:0/1063191241 >> 172.21.15.6:0/1063191241 conn(0x7f13e8071130 msgr2=0x7f13e80f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:05.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.005+0000 7f13eca2e640 1 -- 172.21.15.6:0/1063191241 shutdown_connections 2024-03-22T06:27:05.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.005+0000 7f13eca2e640 1 -- 172.21.15.6:0/1063191241 wait complete. 2024-03-22T06:27:05.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T06:09:46.449392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T06:12:19.302857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T06:12:19.628730+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T06:27:05.016 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T06:27:05.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.117+0000 7fe8dea2e640 1 -- 172.21.15.6:0/3228603864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d8101140 msgr2=0x7fe8d8101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.117+0000 7fe8dea2e640 1 --2- 172.21.15.6:0/3228603864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d8101140 0x7fe8d8101510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80048d0 tx=0x7fe8c8009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.117+0000 7fe8dea2e640 1 -- 172.21.15.6:0/3228603864 shutdown_connections 2024-03-22T06:27:05.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.117+0000 7fe8dea2e640 1 --2- 172.21.15.6:0/3228603864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80fb9e0 0x7fe8d80fbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.117+0000 7fe8dea2e640 1 --2- 172.21.15.6:0/3228603864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d8101140 0x7fe8d8101510 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.117+0000 7fe8dea2e640 1 -- 172.21.15.6:0/3228603864 >> 172.21.15.6:0/3228603864 conn(0x7fe8d80708f0 msgr2=0x7fe8d8070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:05.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.117+0000 7fe8dea2e640 1 -- 172.21.15.6:0/3228603864 shutdown_connections 2024-03-22T06:27:05.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.117+0000 7fe8dea2e640 1 -- 172.21.15.6:0/3228603864 wait complete. 2024-03-22T06:27:05.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.118+0000 7fe8dea2e640 1 Processor -- start 2024-03-22T06:27:05.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.118+0000 7fe8dea2e640 1 -- start start 2024-03-22T06:27:05.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.118+0000 7fe8dea2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80fb9e0 0x7fe8d8111bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:05.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.118+0000 7fe8dea2e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d8101140 0x7fe8d8112130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:05.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.118+0000 7fe8dea2e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8112730 con 0x7fe8d8101140 2024-03-22T06:27:05.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.118+0000 7fe8dea2e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d810bd80 con 0x7fe8d80fb9e0 2024-03-22T06:27:05.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.118+0000 7fe8d77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d8101140 0x7fe8d8112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:05.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.118+0000 7fe8d77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d8101140 0x7fe8d8112130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52896/0 (socket says 172.21.15.6:52896) 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.118+0000 7fe8d77fe640 1 -- 172.21.15.6:0/1210685582 learned_addr learned my addr 172.21.15.6:0/1210685582 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.118+0000 7fe8d7fff640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80fb9e0 0x7fe8d8111bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.119+0000 7fe8d77fe640 1 -- 172.21.15.6:0/1210685582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80fb9e0 msgr2=0x7fe8d8111bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.119+0000 7fe8d77fe640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80fb9e0 0x7fe8d8111bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.119+0000 7fe8d77fe640 1 -- 172.21.15.6:0/1210685582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c8005040 con 0x7fe8d8101140 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.119+0000 7fe8d7fff640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80fb9e0 0x7fe8d8111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.119+0000 7fe8d77fe640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d8101140 0x7fe8d8112130 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c810 tx=0x7fe8cc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.119+0000 7fe8d4ff9640 1 -- 172.21.15.6:0/1210685582 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8cc00a820 con 0x7fe8d8101140 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.119+0000 7fe8dea2e640 1 -- 172.21.15.6:0/1210685582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d810c060 con 0x7fe8d8101140 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.119+0000 7fe8d4ff9640 1 -- 172.21.15.6:0/1210685582 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8cc00ae50 con 0x7fe8d8101140 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.119+0000 7fe8d4ff9640 1 -- 172.21.15.6:0/1210685582 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8cc0115c0 con 0x7fe8d8101140 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.119+0000 7fe8dea2e640 1 -- 172.21.15.6:0/1210685582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d810c580 con 0x7fe8d8101140 2024-03-22T06:27:05.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.120+0000 7fe8dea2e640 1 -- 172.21.15.6:0/1210685582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d80fc9f0 con 0x7fe8d8101140 2024-03-22T06:27:05.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.121+0000 7fe8d4ff9640 1 -- 172.21.15.6:0/1210685582 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fe8cc00a980 con 0x7fe8d8101140 2024-03-22T06:27:05.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.121+0000 7fe8d4ff9640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe8c006e7b0 0x7fe8c0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:05.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.121+0000 7fe8d7fff640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe8c006e7b0 0x7fe8c0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:05.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.121+0000 7fe8d4ff9640 1 -- 172.21.15.6:0/1210685582 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fe8cc08eb80 con 0x7fe8d8101140 2024-03-22T06:27:05.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.122+0000 7fe8d7fff640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe8c006e7b0 0x7fe8c0070c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8000c00 tx=0x7fe8c8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:05.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.126+0000 7fe8d4ff9640 1 -- 172.21.15.6:0/1210685582 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fe8cc057950 con 0x7fe8d8101140 2024-03-22T06:27:05.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:04 smithi117 ceph-mon[82126]: pgmap v355: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:27:05.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:04 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2575582474' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-22T06:27:05.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:04 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2290007431' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-22T06:27:05.272 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:04 smithi006 ceph-mon[94482]: pgmap v355: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:27:05.272 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:04 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2575582474' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-22T06:27:05.272 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:04 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2290007431' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-22T06:27:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.311+0000 7fe8dea2e640 1 -- 172.21.15.6:0/1210685582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8d810d260 con 0x7fe8d8101140 2024-03-22T06:27:05.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.311+0000 7fe8d4ff9640 1 -- 172.21.15.6:0/1210685582 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8cc05b010 con 0x7fe8d8101140 2024-03-22T06:27:05.312 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-03-22T06:27:05.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.313+0000 7fe8dea2e640 1 -- 172.21.15.6:0/1210685582 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe8c006e7b0 msgr2=0x7fe8c0070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.313+0000 7fe8dea2e640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe8c006e7b0 0x7fe8c0070c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8000c00 tx=0x7fe8c8009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.313+0000 7fe8dea2e640 1 -- 172.21.15.6:0/1210685582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d8101140 msgr2=0x7fe8d8112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.313+0000 7fe8dea2e640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d8101140 0x7fe8d8112130 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c810 tx=0x7fe8cc0043c0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.313+0000 7fe8dea2e640 1 -- 172.21.15.6:0/1210685582 shutdown_connections 2024-03-22T06:27:05.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.313+0000 7fe8dea2e640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe8c006e7b0 0x7fe8c0070c60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.313+0000 7fe8dea2e640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d8101140 0x7fe8d8112130 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.313+0000 7fe8dea2e640 1 --2- 172.21.15.6:0/1210685582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80fb9e0 0x7fe8d8111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.313+0000 7fe8dea2e640 1 -- 172.21.15.6:0/1210685582 >> 172.21.15.6:0/1210685582 conn(0x7fe8d80708f0 msgr2=0x7fe8d80f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.313+0000 7fe8dea2e640 1 -- 172.21.15.6:0/1210685582 shutdown_connections 2024-03-22T06:27:05.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.313+0000 7fe8dea2e640 1 -- 172.21.15.6:0/1210685582 wait complete. 2024-03-22T06:27:05.324 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T06:27:05.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.425+0000 7f8158e2e640 1 -- 172.21.15.6:0/3954185559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81540643a0 msgr2=0x7f81540647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.425+0000 7f8158e2e640 1 --2- 172.21.15.6:0/3954185559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81540643a0 0x7f81540647d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f813c0048d0 tx=0x7f813c009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.425+0000 7f8158e2e640 1 -- 172.21.15.6:0/3954185559 shutdown_connections 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.425+0000 7f8158e2e640 1 --2- 172.21.15.6:0/3954185559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81540643a0 0x7f81540647d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.425+0000 7f8158e2e640 1 --2- 172.21.15.6:0/3954185559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8154102240 0x7f8154102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.425+0000 7f8158e2e640 1 -- 172.21.15.6:0/3954185559 >> 172.21.15.6:0/3954185559 conn(0x7f81540710b0 msgr2=0x7f81540714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.425+0000 7f8158e2e640 1 -- 172.21.15.6:0/3954185559 shutdown_connections 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.425+0000 7f8158e2e640 1 -- 172.21.15.6:0/3954185559 wait complete. 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.426+0000 7f8158e2e640 1 Processor -- start 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.426+0000 7f8158e2e640 1 -- start start 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.426+0000 7f8158e2e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81540643a0 0x7f81540fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.426+0000 7f8158e2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8154102240 0x7f81540fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:05.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.426+0000 7f8158e2e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81540fd0c0 con 0x7f81540643a0 2024-03-22T06:27:05.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.426+0000 7f8158e2e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81540fbc60 con 0x7f8154102240 2024-03-22T06:27:05.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.426+0000 7f815259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81540643a0 0x7f81540fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:05.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.426+0000 7f815259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81540643a0 0x7f81540fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52922/0 (socket says 172.21.15.6:52922) 2024-03-22T06:27:05.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.426+0000 7f815259c640 1 -- 172.21.15.6:0/2801867623 learned_addr learned my addr 172.21.15.6:0/2801867623 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:05.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.426+0000 7f8149d9b640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8154102240 0x7f81540fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:05.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.427+0000 7f815259c640 1 -- 172.21.15.6:0/2801867623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8154102240 msgr2=0x7f81540fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.427+0000 7f815259c640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8154102240 0x7f81540fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.427+0000 7f815259c640 1 -- 172.21.15.6:0/2801867623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c005040 con 0x7f81540643a0 2024-03-22T06:27:05.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.427+0000 7f8149d9b640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8154102240 0x7f81540fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:27:05.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.427+0000 7f815259c640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81540643a0 0x7f81540fb1e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8144002a50 tx=0x7f8144004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:05.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.427+0000 7f814b7fe640 1 -- 172.21.15.6:0/2801867623 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8144007c30 con 0x7f81540643a0 2024-03-22T06:27:05.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.427+0000 7f8158e2e640 1 -- 172.21.15.6:0/2801867623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81540fbf40 con 0x7f81540643a0 2024-03-22T06:27:05.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.427+0000 7f814b7fe640 1 -- 172.21.15.6:0/2801867623 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f814400f040 con 0x7f81540643a0 2024-03-22T06:27:05.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.427+0000 7f814b7fe640 1 -- 172.21.15.6:0/2801867623 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f814400dbe0 con 0x7f81540643a0 2024-03-22T06:27:05.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.427+0000 7f8158e2e640 1 -- 172.21.15.6:0/2801867623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81541193e0 con 0x7f81540643a0 2024-03-22T06:27:05.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.428+0000 7f8158e2e640 1 -- 172.21.15.6:0/2801867623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81540fd940 con 0x7f81540643a0 2024-03-22T06:27:05.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.429+0000 7f814b7fe640 1 -- 172.21.15.6:0/2801867623 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f814400dd40 con 0x7f81540643a0 2024-03-22T06:27:05.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.429+0000 7f814b7fe640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f814006e7b0 0x7f8140070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:05.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.429+0000 7f814b7fe640 1 -- 172.21.15.6:0/2801867623 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f814408e590 con 0x7f81540643a0 2024-03-22T06:27:05.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.429+0000 7f8149d9b640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f814006e7b0 0x7f8140070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:05.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.429+0000 7f8149d9b640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f814006e7b0 0x7f8140070c60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f81540fc910 tx=0x7f813c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:05.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.434+0000 7f814b7fe640 1 -- 172.21.15.6:0/2801867623 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f8144057360 con 0x7f81540643a0 2024-03-22T06:27:05.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.616+0000 7f8158e2e640 1 -- 172.21.15.6:0/2801867623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f815404bcc0 con 0x7f81540643a0 2024-03-22T06:27:05.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.617+0000 7f814b7fe640 1 -- 172.21.15.6:0/2801867623 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7f814405aa20 con 0x7f81540643a0 2024-03-22T06:27:05.617 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-03-22T06:27:05.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.618+0000 7f8158e2e640 1 -- 172.21.15.6:0/2801867623 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f814006e7b0 msgr2=0x7f8140070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.618+0000 7f8158e2e640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f814006e7b0 0x7f8140070c60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f81540fc910 tx=0x7f813c009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.618+0000 7f8158e2e640 1 -- 172.21.15.6:0/2801867623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81540643a0 msgr2=0x7f81540fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.618+0000 7f8158e2e640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81540643a0 0x7f81540fb1e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8144002a50 tx=0x7f8144004600 comp rx=0 tx=0).stop 2024-03-22T06:27:05.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.618+0000 7f8158e2e640 1 -- 172.21.15.6:0/2801867623 shutdown_connections 2024-03-22T06:27:05.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.618+0000 7f8158e2e640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f814006e7b0 0x7f8140070c60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.618+0000 7f8158e2e640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8154102240 0x7f81540fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.618+0000 7f8158e2e640 1 --2- 172.21.15.6:0/2801867623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81540643a0 0x7f81540fb1e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.618+0000 7f8158e2e640 1 -- 172.21.15.6:0/2801867623 >> 172.21.15.6:0/2801867623 conn(0x7f81540710b0 msgr2=0x7f81540f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:05.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.619+0000 7f8158e2e640 1 -- 172.21.15.6:0/2801867623 shutdown_connections 2024-03-22T06:27:05.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.619+0000 7f8158e2e640 1 -- 172.21.15.6:0/2801867623 wait complete. 2024-03-22T06:27:05.629 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.730+0000 7f571ac2e640 1 -- 172.21.15.6:0/1902145321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5714101140 msgr2=0x7f5714101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.730+0000 7f571ac2e640 1 --2- 172.21.15.6:0/1902145321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5714101140 0x7f5714101510 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f57040048d0 tx=0x7f5704009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.731+0000 7f571ac2e640 1 -- 172.21.15.6:0/1902145321 shutdown_connections 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.731+0000 7f571ac2e640 1 --2- 172.21.15.6:0/1902145321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57140fb110 0x7f57140fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.731+0000 7f571ac2e640 1 --2- 172.21.15.6:0/1902145321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5714101140 0x7f5714101510 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.731+0000 7f571ac2e640 1 -- 172.21.15.6:0/1902145321 >> 172.21.15.6:0/1902145321 conn(0x7f57140710b0 msgr2=0x7f57140714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.731+0000 7f571ac2e640 1 -- 172.21.15.6:0/1902145321 shutdown_connections 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.731+0000 7f571ac2e640 1 -- 172.21.15.6:0/1902145321 wait complete. 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.731+0000 7f571ac2e640 1 Processor -- start 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.731+0000 7f571ac2e640 1 -- start start 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f571ac2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57140fb110 0x7f5714101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:05.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f571ac2e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5714101140 0x7f5714102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f571ac2e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5714103dd0 con 0x7f5714101140 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f571ac2e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5714102970 con 0x7f57140fb110 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f5713fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5714101140 0x7f5714102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f5713fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5714101140 0x7f5714102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52928/0 (socket says 172.21.15.6:52928) 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f5713fff640 1 -- 172.21.15.6:0/3441016072 learned_addr learned my addr 172.21.15.6:0/3441016072 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f57189ca640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57140fb110 0x7f5714101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f5713fff640 1 -- 172.21.15.6:0/3441016072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57140fb110 msgr2=0x7f5714101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f5713fff640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57140fb110 0x7f5714101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f5713fff640 1 -- 172.21.15.6:0/3441016072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5704005040 con 0x7f5714101140 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f57189ca640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57140fb110 0x7f5714101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f5713fff640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5714101140 0x7f5714102430 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5708002a50 tx=0x7f5708004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f57117fa640 1 -- 172.21.15.6:0/3441016072 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5708007c30 con 0x7f5714101140 2024-03-22T06:27:05.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f571ac2e640 1 -- 172.21.15.6:0/3441016072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5714102c50 con 0x7f5714101140 2024-03-22T06:27:05.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f57117fa640 1 -- 172.21.15.6:0/3441016072 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f570800f040 con 0x7f5714101140 2024-03-22T06:27:05.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f57117fa640 1 -- 172.21.15.6:0/3441016072 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f570800db50 con 0x7f5714101140 2024-03-22T06:27:05.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.732+0000 7f571ac2e640 1 -- 172.21.15.6:0/3441016072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5714119420 con 0x7f5714101140 2024-03-22T06:27:05.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.733+0000 7f571ac2e640 1 -- 172.21.15.6:0/3441016072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57140fc800 con 0x7f5714101140 2024-03-22T06:27:05.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.734+0000 7f57117fa640 1 -- 172.21.15.6:0/3441016072 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f570801c400 con 0x7f5714101140 2024-03-22T06:27:05.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.734+0000 7f57117fa640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f56fc06e7b0 0x7f56fc070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:05.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.734+0000 7f57117fa640 1 -- 172.21.15.6:0/3441016072 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f570808df00 con 0x7f5714101140 2024-03-22T06:27:05.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.734+0000 7f57189ca640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f56fc06e7b0 0x7f56fc070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:05.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.735+0000 7f57189ca640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f56fc06e7b0 0x7f56fc070c60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5704007ac0 tx=0x7f5704009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:05.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.739+0000 7f57117fa640 1 -- 172.21.15.6:0/3441016072 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f5708056d40 con 0x7f5714101140 2024-03-22T06:27:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.946+0000 7f571ac2e640 1 -- 172.21.15.6:0/3441016072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f5714108280 con 0x7f5714101140 2024-03-22T06:27:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.946+0000 7f57117fa640 1 -- 172.21.15.6:0/3441016072 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7f570805a400 con 0x7f5714101140 2024-03-22T06:27:05.947 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-03-22T06:27:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.949+0000 7f571ac2e640 1 -- 172.21.15.6:0/3441016072 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f56fc06e7b0 msgr2=0x7f56fc070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.949+0000 7f571ac2e640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f56fc06e7b0 0x7f56fc070c60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5704007ac0 tx=0x7f5704009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.949+0000 7f571ac2e640 1 -- 172.21.15.6:0/3441016072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5714101140 msgr2=0x7f5714102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.949+0000 7f571ac2e640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5714101140 0x7f5714102430 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5708002a50 tx=0x7f5708004600 comp rx=0 tx=0).stop 2024-03-22T06:27:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.949+0000 7f571ac2e640 1 -- 172.21.15.6:0/3441016072 shutdown_connections 2024-03-22T06:27:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.949+0000 7f571ac2e640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f56fc06e7b0 0x7f56fc070c60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.949+0000 7f571ac2e640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5714101140 0x7f5714102430 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.949+0000 7f571ac2e640 1 --2- 172.21.15.6:0/3441016072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57140fb110 0x7f5714101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.949+0000 7f571ac2e640 1 -- 172.21.15.6:0/3441016072 >> 172.21.15.6:0/3441016072 conn(0x7f57140710b0 msgr2=0x7f57140f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.949+0000 7f571ac2e640 1 -- 172.21.15.6:0/3441016072 shutdown_connections 2024-03-22T06:27:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:05.949+0000 7f571ac2e640 1 -- 172.21.15.6:0/3441016072 wait complete. 2024-03-22T06:27:05.961 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T06:27:06.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.061+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1914835634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bc4101a70 msgr2=0x7f2bc40fb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:06.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.061+0000 7f2bcb19d640 1 --2- 172.21.15.6:0/1914835634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bc4101a70 0x7f2bc40fb100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00048d0 tx=0x7f2bb0009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.061+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1914835634 shutdown_connections 2024-03-22T06:27:06.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.061+0000 7f2bcb19d640 1 --2- 172.21.15.6:0/1914835634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bc4101a70 0x7f2bc40fb100 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.061+0000 7f2bcb19d640 1 --2- 172.21.15.6:0/1914835634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc41010d0 0x7f2bc41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.061+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1914835634 >> 172.21.15.6:0/1914835634 conn(0x7f2bc40f6ca0 msgr2=0x7f2bc40f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:06.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.061+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1914835634 shutdown_connections 2024-03-22T06:27:06.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.061+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1914835634 wait complete. 2024-03-22T06:27:06.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.062+0000 7f2bcb19d640 1 Processor -- start 2024-03-22T06:27:06.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.062+0000 7f2bcb19d640 1 -- start start 2024-03-22T06:27:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.062+0000 7f2bcb19d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bc41010d0 0x7f2bc4111bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.062+0000 7f2bcb19d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc4101a70 0x7f2bc4112100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.062+0000 7f2bcb19d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4112700 con 0x7f2bc41010d0 2024-03-22T06:27:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.062+0000 7f2bcb19d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc410bd50 con 0x7f2bc4101a70 2024-03-22T06:27:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.062+0000 7f2bc8f39640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bc41010d0 0x7f2bc4111bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.062+0000 7f2bc8f39640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bc41010d0 0x7f2bc4111bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52944/0 (socket says 172.21.15.6:52944) 2024-03-22T06:27:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bc8f39640 1 -- 172.21.15.6:0/1814668549 learned_addr learned my addr 172.21.15.6:0/1814668549 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bc3fff640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc4101a70 0x7f2bc4112100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bc8f39640 1 -- 172.21.15.6:0/1814668549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc4101a70 msgr2=0x7f2bc4112100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:06.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bc8f39640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc4101a70 0x7f2bc4112100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bc8f39640 1 -- 172.21.15.6:0/1814668549 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0005040 con 0x7f2bc41010d0 2024-03-22T06:27:06.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bc3fff640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc4101a70 0x7f2bc4112100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T06:27:06.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bc8f39640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bc41010d0 0x7f2bc4111bc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800fd10 tx=0x7f2bb8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:06.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bc17fa640 1 -- 172.21.15.6:0/1814668549 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb8007a20 con 0x7f2bc41010d0 2024-03-22T06:27:06.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bc17fa640 1 -- 172.21.15.6:0/1814668549 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bb8007b80 con 0x7f2bc41010d0 2024-03-22T06:27:06.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1814668549 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc410bfd0 con 0x7f2bc41010d0 2024-03-22T06:27:06.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bc17fa640 1 -- 172.21.15.6:0/1814668549 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb80127a0 con 0x7f2bc41010d0 2024-03-22T06:27:06.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.063+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1814668549 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc410c4c0 con 0x7f2bc41010d0 2024-03-22T06:27:06.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.064+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1814668549 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc40fc840 con 0x7f2bc41010d0 2024-03-22T06:27:06.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.065+0000 7f2bc17fa640 1 -- 172.21.15.6:0/1814668549 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f2bb8007cf0 con 0x7f2bc41010d0 2024-03-22T06:27:06.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.065+0000 7f2bc17fa640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2bb406e7b0 0x7f2bb4070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:06.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.065+0000 7f2bc3fff640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2bb406e7b0 0x7f2bb4070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:06.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.065+0000 7f2bc17fa640 1 -- 172.21.15.6:0/1814668549 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f2bb8090ca0 con 0x7f2bc41010d0 2024-03-22T06:27:06.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.066+0000 7f2bc3fff640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2bb406e7b0 0x7f2bb4070c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40f9720 tx=0x7f2bb0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:06.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.070+0000 7f2bc17fa640 1 -- 172.21.15.6:0/1814668549 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f2bb8095070 con 0x7f2bc41010d0 2024-03-22T06:27:06.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:05 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1063191241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:27:06.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:05 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1210685582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T06:27:06.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:05 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2801867623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T06:27:06.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:05 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3441016072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T06:27:06.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:05 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1063191241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:27:06.281 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:05 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1210685582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T06:27:06.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:05 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2801867623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T06:27:06.282 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:05 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3441016072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T06:27:06.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.288+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1814668549 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2bc410d2c0 con 0x7f2bc41010d0 2024-03-22T06:27:06.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.288+0000 7f2bc17fa640 1 -- 172.21.15.6:0/1814668549 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v95) v1 ==== 81+0+2516 (secure 0 0 0) 0x7f2bb8059a70 con 0x7f2bc41010d0 2024-03-22T06:27:06.288 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6827/1010515354 2024-03-23T06:23:28.644314+0000 2024-03-22T06:27:06.288 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6826/1010515354 2024-03-23T06:23:28.644314+0000 2024-03-22T06:27:06.289 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6825/1251227895 2024-03-23T06:23:03.662256+0000 2024-03-22T06:27:06.289 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6829/947853733 2024-03-23T06:22:41.442153+0000 2024-03-22T06:27:06.289 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6828/947853733 2024-03-23T06:22:41.442153+0000 2024-03-22T06:27:06.289 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6827/1260089028 2024-03-23T06:22:13.008321+0000 2024-03-22T06:27:06.289 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6829/1963654099 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:06.289 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2826360973 2024-03-23T06:06:24.832062+0000 2024-03-22T06:27:06.289 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3099962970 2024-03-23T06:05:54.100943+0000 2024-03-22T06:27:06.289 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3099962970 2024-03-23T06:05:54.100943+0000 2024-03-22T06:27:06.289 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2042811963 2024-03-23T06:06:24.832062+0000 2024-03-22T06:27:06.289 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/602685778 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:06.289 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3903206979 2024-03-23T06:06:24.832062+0000 2024-03-22T06:27:06.290 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:0/45595122 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:06.290 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6824/1251227895 2024-03-23T06:23:03.662256+0000 2024-03-22T06:27:06.290 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1533067608 2024-03-23T06:07:32.107836+0000 2024-03-22T06:27:06.290 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3502710787 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:06.290 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/358732355 2024-03-23T06:07:32.107836+0000 2024-03-22T06:27:06.290 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/791727884 2024-03-23T06:07:32.107836+0000 2024-03-22T06:27:06.290 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3606143293 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:06.290 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/254878217 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:06.290 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3489834288 2024-03-23T06:07:32.107836+0000 2024-03-22T06:27:06.290 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:0/3344206523 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:06.290 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2742769039 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:06.291 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/358732355 2024-03-23T06:07:32.107836+0000 2024-03-22T06:27:06.291 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3903206979 2024-03-23T06:06:24.832062+0000 2024-03-22T06:27:06.291 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:0/435100846 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:06.291 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1354987702 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:06.291 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2939515329 2024-03-23T06:05:54.100943+0000 2024-03-22T06:27:06.291 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:0/3477422362 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:06.291 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3665872066 2024-03-23T06:06:24.832062+0000 2024-03-22T06:27:06.291 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/619515244 2024-03-23T06:05:54.100943+0000 2024-03-22T06:27:06.291 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3502710787 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:06.291 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1270793675 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:06.292 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6828/1963654099 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:06.292 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/5416477 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:06.292 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3809759464 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:06.292 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/602685778 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:06.292 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/4154587507 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:06.292 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1448591806 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:06.292 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2671132128 2024-03-23T06:05:54.100943+0000 2024-03-22T06:27:06.292 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1248395428 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:06.292 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6826/1260089028 2024-03-23T06:22:13.008321+0000 2024-03-22T06:27:06.293 INFO:teuthology.orchestra.run.smithi006.stderr:listed 43 entries 2024-03-22T06:27:06.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.290+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1814668549 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2bb406e7b0 msgr2=0x7f2bb4070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:06.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.290+0000 7f2bcb19d640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2bb406e7b0 0x7f2bb4070c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40f9720 tx=0x7f2bb0009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.290+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1814668549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bc41010d0 msgr2=0x7f2bc4111bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:06.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.290+0000 7f2bcb19d640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bc41010d0 0x7f2bc4111bc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800fd10 tx=0x7f2bb8004600 comp rx=0 tx=0).stop 2024-03-22T06:27:06.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.290+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1814668549 shutdown_connections 2024-03-22T06:27:06.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.290+0000 7f2bcb19d640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f2bb406e7b0 0x7f2bb4070c60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.290+0000 7f2bcb19d640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc4101a70 0x7f2bc4112100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.290+0000 7f2bcb19d640 1 --2- 172.21.15.6:0/1814668549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bc41010d0 0x7f2bc4111bc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.290+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1814668549 >> 172.21.15.6:0/1814668549 conn(0x7f2bc40f6ca0 msgr2=0x7f2bc40f7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:06.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.290+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1814668549 shutdown_connections 2024-03-22T06:27:06.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.291+0000 7f2bcb19d640 1 -- 172.21.15.6:0/1814668549 wait complete. 2024-03-22T06:27:06.302 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T06:27:06.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:27:06.302 DEBUG:teuthology.orchestra.run.smithi006:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T06:27:06.330 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:27:06.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:27:06.331 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T06:27:06.406 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:27:06.406 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T06:27:06.426 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-22T06:27:06.426 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T06:27:06.485 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T06:27:06.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.613+0000 7f79d8e58640 1 -- 172.21.15.6:0/3040713460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d4102b70 msgr2=0x7f79d40fc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:06.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.613+0000 7f79d8e58640 1 --2- 172.21.15.6:0/3040713460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d4102b70 0x7f79d40fc200 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0048d0 tx=0x7f79bc009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 -- 172.21.15.6:0/3040713460 shutdown_connections 2024-03-22T06:27:06.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 --2- 172.21.15.6:0/3040713460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d4102b70 0x7f79d40fc200 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 --2- 172.21.15.6:0/3040713460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79d41021d0 0x7f79d41025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 -- 172.21.15.6:0/3040713460 >> 172.21.15.6:0/3040713460 conn(0x7f79d40f7da0 msgr2=0x7f79d40fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:06.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 -- 172.21.15.6:0/3040713460 shutdown_connections 2024-03-22T06:27:06.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 -- 172.21.15.6:0/3040713460 wait complete. 2024-03-22T06:27:06.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 Processor -- start 2024-03-22T06:27:06.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 -- start start 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d41021d0 0x7f79d406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79d4102b70 0x7f79d406fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40716b0 con 0x7f79d41021d0 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d8e58640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d4070250 con 0x7f79d4102b70 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d41021d0 0x7f79d406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d259c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d41021d0 0x7f79d406f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52952/0 (socket says 172.21.15.6:52952) 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.614+0000 7f79d259c640 1 -- 172.21.15.6:0/2340802398 learned_addr learned my addr 172.21.15.6:0/2340802398 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.615+0000 7f79d259c640 1 -- 172.21.15.6:0/2340802398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79d4102b70 msgr2=0x7f79d406fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.615+0000 7f79d259c640 1 --2- 172.21.15.6:0/2340802398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79d4102b70 0x7f79d406fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.615+0000 7f79d259c640 1 -- 172.21.15.6:0/2340802398 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc005040 con 0x7f79d41021d0 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.615+0000 7f79d259c640 1 --2- 172.21.15.6:0/2340802398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d41021d0 0x7f79d406f7d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f79c400db00 tx=0x7f79c4004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.615+0000 7f79caffd640 1 -- 172.21.15.6:0/2340802398 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79c4017070 con 0x7f79d41021d0 2024-03-22T06:27:06.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.615+0000 7f79caffd640 1 -- 172.21.15.6:0/2340802398 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79c400f040 con 0x7f79d41021d0 2024-03-22T06:27:06.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.615+0000 7f79d8e58640 1 -- 172.21.15.6:0/2340802398 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d4070530 con 0x7f79d41021d0 2024-03-22T06:27:06.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.615+0000 7f79caffd640 1 -- 172.21.15.6:0/2340802398 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79c4003ae0 con 0x7f79d41021d0 2024-03-22T06:27:06.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.615+0000 7f79d8e58640 1 -- 172.21.15.6:0/2340802398 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d4119400 con 0x7f79d41021d0 2024-03-22T06:27:06.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.616+0000 7f79d8e58640 1 -- 172.21.15.6:0/2340802398 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d40fd940 con 0x7f79d41021d0 2024-03-22T06:27:06.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.616+0000 7f79caffd640 1 -- 172.21.15.6:0/2340802398 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f79c4003c40 con 0x7f79d41021d0 2024-03-22T06:27:06.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.617+0000 7f79caffd640 1 --2- 172.21.15.6:0/2340802398 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79c006e7b0 0x7f79c0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:06.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.617+0000 7f79caffd640 1 -- 172.21.15.6:0/2340802398 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f79c408e750 con 0x7f79d41021d0 2024-03-22T06:27:06.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.617+0000 7f79d1d9b640 1 --2- 172.21.15.6:0/2340802398 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79c006e7b0 0x7f79c0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:06.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.617+0000 7f79d1d9b640 1 --2- 172.21.15.6:0/2340802398 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79c006e7b0 0x7f79c0070c60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0043d0 tx=0x7f79bc009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:06.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.622+0000 7f79caffd640 1 -- 172.21.15.6:0/2340802398 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f79c4057520 con 0x7f79d41021d0 2024-03-22T06:27:06.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.809+0000 7f79d8e58640 1 -- 172.21.15.6:0/2340802398 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79d4070cb0 con 0x7f79d41021d0 2024-03-22T06:27:06.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.810+0000 7f79caffd640 1 -- 172.21.15.6:0/2340802398 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+17295 (secure 0 0 0) 0x7f79c405abe0 con 0x7f79d41021d0 2024-03-22T06:27:06.811 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-22T06:27:06.811 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":95,"fsid":"fa0e13c2-e811-11ee-95cd-87774f69a715","created":"2024-03-22T06:05:27.379207+0000","modified":"2024-03-22T06:23:28.644346+0000","last_up_change":"2024-03-22T06:22:07.841620+0000","last_in_change":"2024-03-22T06:10:39.982933+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T06:09:46.449392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T06:12:19.302857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T06:12:19.628730+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ecd88bb1-d643-4505-a098-3728178c89e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":90,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6803","nonce":1666131505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6805","nonce":1666131505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6809","nonce":1666131505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1666131505},{"type":"v1","addr":"172.21.15.6:6807","nonce":1666131505}]},"public_addr":"172.21.15.6:6803/1666131505","cluster_addr":"172.21.15.6:6805/1666131505","heartbeat_back_addr":"172.21.15.6:6809/1666131505","heartbeat_front_addr":"172.21.15.6:6807/1666131505","state":["exists","up"]},{"osd":1,"uuid":"f1b55307-2c3b-4ab8-a893-af8072f333ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":66,"up_thru":90,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6811","nonce":1922945925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6813","nonce":1922945925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6817","nonce":1922945925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1922945925},{"type":"v1","addr":"172.21.15.6:6815","nonce":1922945925}]},"public_addr":"172.21.15.6:6811/1922945925","cluster_addr":"172.21.15.6:6813/1922945925","heartbeat_back_addr":"172.21.15.6:6817/1922945925","heartbeat_front_addr":"172.21.15.6:6815/1922945925","state":["exists","up"]},{"osd":2,"uuid":"9bacfe42-3c78-4d86-bac0-0142cde76ff5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6819","nonce":195584623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6821","nonce":195584623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6825","nonce":195584623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":195584623},{"type":"v1","addr":"172.21.15.6:6823","nonce":195584623}]},"public_addr":"172.21.15.6:6819/195584623","cluster_addr":"172.21.15.6:6821/195584623","heartbeat_back_addr":"172.21.15.6:6825/195584623","heartbeat_front_addr":"172.21.15.6:6823/195584623","state":["exists","up"]},{"osd":3,"uuid":"2c5a6acd-de41-455d-b6c5-d53bb76197ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6801","nonce":1104353694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6803","nonce":1104353694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6807","nonce":1104353694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1104353694},{"type":"v1","addr":"172.21.15.117:6805","nonce":1104353694}]},"public_addr":"172.21.15.117:6801/1104353694","cluster_addr":"172.21.15.117:6803/1104353694","heartbeat_back_addr":"172.21.15.117:6807/1104353694","heartbeat_front_addr":"172.21.15.117:6805/1104353694","state":["exists","up"]},{"osd":4,"uuid":"be424041-1e07-41f4-abe0-68fdd21b1eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6809","nonce":208554150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6811","nonce":208554150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6815","nonce":208554150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":208554150},{"type":"v1","addr":"172.21.15.117:6813","nonce":208554150}]},"public_addr":"172.21.15.117:6809/208554150","cluster_addr":"172.21.15.117:6811/208554150","heartbeat_back_addr":"172.21.15.117:6815/208554150","heartbeat_front_addr":"172.21.15.117:6813/208554150","state":["exists","up"]},{"osd":5,"uuid":"bf61ff67-1f07-47ed-ad58-e6c946369d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6817","nonce":1416150897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6819","nonce":1416150897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6823","nonce":1416150897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1416150897},{"type":"v1","addr":"172.21.15.117:6821","nonce":1416150897}]},"public_addr":"172.21.15.117:6817/1416150897","cluster_addr":"172.21.15.117:6819/1416150897","heartbeat_back_addr":"172.21.15.117:6823/1416150897","heartbeat_front_addr":"172.21.15.117:6821/1416150897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:08:45.922485+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:13.118065+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:09:42.409903+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:05.760002+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:31.339234+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T06:10:57.286867+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":0,"to":2}]},{"pgid":"2.7","mappings":[{"from":3,"to":1}]},{"pgid":"2.b","mappings":[{"from":3,"to":1}]},{"pgid":"2.e","mappings":[{"from":0,"to":1}]},{"pgid":"2.12","mappings":[{"from":0,"to":2}]},{"pgid":"2.16","mappings":[{"from":3,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":4,"to":0}]},{"pgid":"3.1d","mappings":[{"from":4,"to":0}]},{"pgid":"3.1f","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6827/1010515354":"2024-03-23T06:23:28.644314+0000","172.21.15.117:6826/1010515354":"2024-03-23T06:23:28.644314+0000","172.21.15.117:6825/1251227895":"2024-03-23T06:23:03.662256+0000","172.21.15.6:6829/947853733":"2024-03-23T06:22:41.442153+0000","172.21.15.6:6828/947853733":"2024-03-23T06:22:41.442153+0000","172.21.15.6:6827/1260089028":"2024-03-23T06:22:13.008321+0000","172.21.15.117:6829/1963654099":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/2826360973":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6800/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:6801/3099962970":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/2042811963":"2024-03-23T06:06:24.832062+0000","172.21.15.6:6801/602685778":"2024-03-23T06:14:38.572050+0000","172.21.15.6:6801/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.117:0/45595122":"2024-03-23T06:16:28.822651+0000","172.21.15.117:6824/1251227895":"2024-03-23T06:23:03.662256+0000","172.21.15.6:0/1533067608":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/3502710787":"2024-03-23T06:17:37.246941+0000","172.21.15.6:6800/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/791727884":"2024-03-23T06:07:32.107836+0000","172.21.15.6:0/3606143293":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/254878217":"2024-03-23T06:14:38.572050+0000","172.21.15.6:0/3489834288":"2024-03-23T06:07:32.107836+0000","172.21.15.117:0/3344206523":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/2742769039":"2024-03-23T06:14:38.572050+0000","172.21.15.6:6801/358732355":"2024-03-23T06:07:32.107836+0000","172.21.15.6:6800/3903206979":"2024-03-23T06:06:24.832062+0000","172.21.15.117:0/435100846":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/1354987702":"2024-03-23T06:14:38.572050+0000","172.21.15.6:0/2939515329":"2024-03-23T06:05:54.100943+0000","172.21.15.117:0/3477422362":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/3665872066":"2024-03-23T06:06:24.832062+0000","172.21.15.6:0/619515244":"2024-03-23T06:05:54.100943+0000","172.21.15.6:6801/3502710787":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/1270793675":"2024-03-23T06:14:38.572050+0000","172.21.15.117:6828/1963654099":"2024-03-23T06:16:28.822651+0000","172.21.15.6:0/5416477":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/3809759464":"2024-03-23T06:17:37.246941+0000","172.21.15.6:6800/602685778":"2024-03-23T06:14:38.572050+0000","172.21.15.6:0/4154587507":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/1448591806":"2024-03-23T06:17:37.246941+0000","172.21.15.6:0/2671132128":"2024-03-23T06:05:54.100943+0000","172.21.15.6:0/1248395428":"2024-03-23T06:17:37.246941+0000","172.21.15.6:6826/1260089028":"2024-03-23T06:22:13.008321+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T06:27:06.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.812+0000 7f79d8e58640 1 -- 172.21.15.6:0/2340802398 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79c006e7b0 msgr2=0x7f79c0070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:06.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.812+0000 7f79d8e58640 1 --2- 172.21.15.6:0/2340802398 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79c006e7b0 0x7f79c0070c60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0043d0 tx=0x7f79bc009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.812+0000 7f79d8e58640 1 -- 172.21.15.6:0/2340802398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d41021d0 msgr2=0x7f79d406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:06.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.812+0000 7f79d8e58640 1 --2- 172.21.15.6:0/2340802398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d41021d0 0x7f79d406f7d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f79c400db00 tx=0x7f79c4004e60 comp rx=0 tx=0).stop 2024-03-22T06:27:06.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.812+0000 7f79d8e58640 1 -- 172.21.15.6:0/2340802398 shutdown_connections 2024-03-22T06:27:06.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.812+0000 7f79d8e58640 1 --2- 172.21.15.6:0/2340802398 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f79c006e7b0 0x7f79c0070c60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.812+0000 7f79d8e58640 1 --2- 172.21.15.6:0/2340802398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79d4102b70 0x7f79d406fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.812+0000 7f79d8e58640 1 --2- 172.21.15.6:0/2340802398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d41021d0 0x7f79d406f7d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.812+0000 7f79d8e58640 1 -- 172.21.15.6:0/2340802398 >> 172.21.15.6:0/2340802398 conn(0x7f79d40f7da0 msgr2=0x7f79d40f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:06.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.813+0000 7f79d8e58640 1 -- 172.21.15.6:0/2340802398 shutdown_connections 2024-03-22T06:27:06.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.813+0000 7f79d8e58640 1 -- 172.21.15.6:0/2340802398 wait complete. 2024-03-22T06:27:06.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T06:09:46.449392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T06:12:19.302857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T06:12:19.628730+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T06:27:06.824 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T06:27:06.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.925+0000 7fe2b715d640 1 -- 172.21.15.6:0/782272783 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2b0102240 msgr2=0x7fe2b0102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:06.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.925+0000 7fe2b715d640 1 --2- 172.21.15.6:0/782272783 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2b0102240 0x7fe2b0102610 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40048d0 tx=0x7fe2a4009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.925+0000 7fe2b715d640 1 -- 172.21.15.6:0/782272783 shutdown_connections 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.925+0000 7fe2b715d640 1 --2- 172.21.15.6:0/782272783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b00643a0 0x7fe2b00647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.925+0000 7fe2b715d640 1 --2- 172.21.15.6:0/782272783 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2b0102240 0x7fe2b0102610 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.925+0000 7fe2b715d640 1 -- 172.21.15.6:0/782272783 >> 172.21.15.6:0/782272783 conn(0x7fe2b00710b0 msgr2=0x7fe2b00714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.925+0000 7fe2b715d640 1 -- 172.21.15.6:0/782272783 shutdown_connections 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.926+0000 7fe2b715d640 1 -- 172.21.15.6:0/782272783 wait complete. 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.926+0000 7fe2b715d640 1 Processor -- start 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.926+0000 7fe2b715d640 1 -- start start 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.926+0000 7fe2b715d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b00643a0 0x7fe2b0112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.926+0000 7fe2b715d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2b0102240 0x7fe2b0113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.926+0000 7fe2b715d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b0113810 con 0x7fe2b0102240 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.926+0000 7fe2b715d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b010ce60 con 0x7fe2b00643a0 2024-03-22T06:27:06.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.926+0000 7fe2affff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2b0102240 0x7fe2b0113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.926+0000 7fe2affff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2b0102240 0x7fe2b0113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52970/0 (socket says 172.21.15.6:52970) 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.926+0000 7fe2affff640 1 -- 172.21.15.6:0/1427316002 learned_addr learned my addr 172.21.15.6:0/1427316002 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.927+0000 7fe2b4ef9640 1 --2- 172.21.15.6:0/1427316002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b00643a0 0x7fe2b0112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.927+0000 7fe2affff640 1 -- 172.21.15.6:0/1427316002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b00643a0 msgr2=0x7fe2b0112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.927+0000 7fe2affff640 1 --2- 172.21.15.6:0/1427316002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b00643a0 0x7fe2b0112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.927+0000 7fe2affff640 1 -- 172.21.15.6:0/1427316002 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4005040 con 0x7fe2b0102240 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.927+0000 7fe2affff640 1 --2- 172.21.15.6:0/1427316002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2b0102240 0x7fe2b0113210 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00eb70 tx=0x7fe29c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.927+0000 7fe2ad7fa640 1 -- 172.21.15.6:0/1427316002 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe29c007c30 con 0x7fe2b0102240 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.927+0000 7fe2ad7fa640 1 -- 172.21.15.6:0/1427316002 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe29c00f040 con 0x7fe2b0102240 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.927+0000 7fe2b715d640 1 -- 172.21.15.6:0/1427316002 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b010d0e0 con 0x7fe2b0102240 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.927+0000 7fe2ad7fa640 1 -- 172.21.15.6:0/1427316002 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe29c00cad0 con 0x7fe2b0102240 2024-03-22T06:27:06.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.927+0000 7fe2b715d640 1 -- 172.21.15.6:0/1427316002 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b010d600 con 0x7fe2b0102240 2024-03-22T06:27:06.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.928+0000 7fe2b715d640 1 -- 172.21.15.6:0/1427316002 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b00fd940 con 0x7fe2b0102240 2024-03-22T06:27:06.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.928+0000 7fe2ad7fa640 1 -- 172.21.15.6:0/1427316002 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7fe29c00cc50 con 0x7fe2b0102240 2024-03-22T06:27:06.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.929+0000 7fe2ad7fa640 1 --2- 172.21.15.6:0/1427316002 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe2a006e7c0 0x7fe2a0070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:06.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.929+0000 7fe2ad7fa640 1 -- 172.21.15.6:0/1427316002 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fe29c08e2c0 con 0x7fe2b0102240 2024-03-22T06:27:06.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.929+0000 7fe2b4ef9640 1 --2- 172.21.15.6:0/1427316002 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe2a006e7c0 0x7fe2a0070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:06.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.930+0000 7fe2b4ef9640 1 --2- 172.21.15.6:0/1427316002 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe2a006e7c0 0x7fe2a0070c70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40048a0 tx=0x7fe2a4009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:06.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:06.934+0000 7fe2ad7fa640 1 -- 172.21.15.6:0/1427316002 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7fe29c057090 con 0x7fe2b0102240 2024-03-22T06:27:07.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.118+0000 7fe2b715d640 1 -- 172.21.15.6:0/1427316002 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2b010e340 con 0x7fe2b0102240 2024-03-22T06:27:07.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.118+0000 7fe2ad7fa640 1 -- 172.21.15.6:0/1427316002 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe29c05a750 con 0x7fe2b0102240 2024-03-22T06:27:07.119 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-03-22T06:27:07.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.121+0000 7fe2b715d640 1 -- 172.21.15.6:0/1427316002 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe2a006e7c0 msgr2=0x7fe2a0070c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.121+0000 7fe2b715d640 1 --2- 172.21.15.6:0/1427316002 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe2a006e7c0 0x7fe2a0070c70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40048a0 tx=0x7fe2a4009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.121+0000 7fe2b715d640 1 -- 172.21.15.6:0/1427316002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2b0102240 msgr2=0x7fe2b0113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.121+0000 7fe2b715d640 1 --2- 172.21.15.6:0/1427316002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2b0102240 0x7fe2b0113210 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00eb70 tx=0x7fe29c00b040 comp rx=0 tx=0).stop 2024-03-22T06:27:07.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.121+0000 7fe2b715d640 1 -- 172.21.15.6:0/1427316002 shutdown_connections 2024-03-22T06:27:07.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.121+0000 7fe2b715d640 1 --2- 172.21.15.6:0/1427316002 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7fe2a006e7c0 0x7fe2a0070c70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.121+0000 7fe2b715d640 1 --2- 172.21.15.6:0/1427316002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2b0102240 0x7fe2b0113210 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.121+0000 7fe2b715d640 1 --2- 172.21.15.6:0/1427316002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b00643a0 0x7fe2b0112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.121+0000 7fe2b715d640 1 -- 172.21.15.6:0/1427316002 >> 172.21.15.6:0/1427316002 conn(0x7fe2b00710b0 msgr2=0x7fe2b00f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:07.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.121+0000 7fe2b715d640 1 -- 172.21.15.6:0/1427316002 shutdown_connections 2024-03-22T06:27:07.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.121+0000 7fe2b715d640 1 -- 172.21.15.6:0/1427316002 wait complete. 2024-03-22T06:27:07.131 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:06 smithi006 ceph-mon[94482]: pgmap v356: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:27:07.131 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:06 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1814668549' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T06:27:07.131 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:06 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2340802398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:27:07.132 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T06:27:07.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.233+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/31269957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b68102240 msgr2=0x7f3b68102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.233+0000 7f3b6ef9d640 1 --2- 172.21.15.6:0/31269957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b68102240 0x7f3b68102610 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0048d0 tx=0x7f3b5c009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.233+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/31269957 shutdown_connections 2024-03-22T06:27:07.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.233+0000 7f3b6ef9d640 1 --2- 172.21.15.6:0/31269957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b680643a0 0x7f3b680647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.233+0000 7f3b6ef9d640 1 --2- 172.21.15.6:0/31269957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b68102240 0x7f3b68102610 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.233+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/31269957 >> 172.21.15.6:0/31269957 conn(0x7f3b680710b0 msgr2=0x7f3b680714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.233+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/31269957 shutdown_connections 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.233+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/31269957 wait complete. 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.233+0000 7f3b6ef9d640 1 Processor -- start 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.233+0000 7f3b6ef9d640 1 -- start start 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b6ef9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b680643a0 0x7f3b68112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b6ef9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b68102240 0x7f3b68113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b6ef9d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b68113810 con 0x7f3b68102240 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b6ef9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6810ce60 con 0x7f3b680643a0 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b68102240 0x7f3b68113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b68102240 0x7f3b68113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52984/0 (socket says 172.21.15.6:52984) 2024-03-22T06:27:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b67fff640 1 -- 172.21.15.6:0/3555507776 learned_addr learned my addr 172.21.15.6:0/3555507776 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b6cd39640 1 --2- 172.21.15.6:0/3555507776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b680643a0 0x7f3b68112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b67fff640 1 -- 172.21.15.6:0/3555507776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b680643a0 msgr2=0x7f3b68112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b67fff640 1 --2- 172.21.15.6:0/3555507776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b680643a0 0x7f3b68112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b67fff640 1 -- 172.21.15.6:0/3555507776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c005040 con 0x7f3b68102240 2024-03-22T06:27:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.234+0000 7f3b67fff640 1 --2- 172.21.15.6:0/3555507776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b68102240 0x7f3b68113210 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3b54002a50 tx=0x7f3b54004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.235+0000 7f3b657fa640 1 -- 172.21.15.6:0/3555507776 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b54007c30 con 0x7f3b68102240 2024-03-22T06:27:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.235+0000 7f3b657fa640 1 -- 172.21.15.6:0/3555507776 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b5400f040 con 0x7f3b68102240 2024-03-22T06:27:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.235+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/3555507776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6810d140 con 0x7f3b68102240 2024-03-22T06:27:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.235+0000 7f3b657fa640 1 -- 172.21.15.6:0/3555507776 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b5400db50 con 0x7f3b68102240 2024-03-22T06:27:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.235+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/3555507776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6810d660 con 0x7f3b68102240 2024-03-22T06:27:07.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.236+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/3555507776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b680fd940 con 0x7f3b68102240 2024-03-22T06:27:07.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.236+0000 7f3b657fa640 1 -- 172.21.15.6:0/3555507776 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f3b5401c400 con 0x7f3b68102240 2024-03-22T06:27:07.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.237+0000 7f3b657fa640 1 --2- 172.21.15.6:0/3555507776 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3b5806e7b0 0x7f3b58070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:07.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.237+0000 7f3b6cd39640 1 --2- 172.21.15.6:0/3555507776 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3b5806e7b0 0x7f3b58070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:07.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.237+0000 7f3b657fa640 1 -- 172.21.15.6:0/3555507776 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f3b5408de90 con 0x7f3b68102240 2024-03-22T06:27:07.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.237+0000 7f3b6cd39640 1 --2- 172.21.15.6:0/3555507776 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3b5806e7b0 0x7f3b58070c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c003080 tx=0x7f3b5c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:07.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.241+0000 7f3b657fa640 1 -- 172.21.15.6:0/3555507776 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f3b54056c70 con 0x7f3b68102240 2024-03-22T06:27:07.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:06 smithi117 ceph-mon[82126]: pgmap v356: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:27:07.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:06 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1814668549' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T06:27:07.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:06 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2340802398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T06:27:07.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.432+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/3555507776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f3b68109320 con 0x7f3b68102240 2024-03-22T06:27:07.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.432+0000 7f3b657fa640 1 -- 172.21.15.6:0/3555507776 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3b5405fe60 con 0x7f3b68102240 2024-03-22T06:27:07.432 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-03-22T06:27:07.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.434+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/3555507776 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3b5806e7b0 msgr2=0x7f3b58070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.434+0000 7f3b6ef9d640 1 --2- 172.21.15.6:0/3555507776 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3b5806e7b0 0x7f3b58070c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c003080 tx=0x7f3b5c009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.434+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/3555507776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b68102240 msgr2=0x7f3b68113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.434+0000 7f3b6ef9d640 1 --2- 172.21.15.6:0/3555507776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b68102240 0x7f3b68113210 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3b54002a50 tx=0x7f3b54004600 comp rx=0 tx=0).stop 2024-03-22T06:27:07.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.434+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/3555507776 shutdown_connections 2024-03-22T06:27:07.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.434+0000 7f3b6ef9d640 1 --2- 172.21.15.6:0/3555507776 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f3b5806e7b0 0x7f3b58070c60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.434+0000 7f3b6ef9d640 1 --2- 172.21.15.6:0/3555507776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b68102240 0x7f3b68113210 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.434+0000 7f3b6ef9d640 1 --2- 172.21.15.6:0/3555507776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b680643a0 0x7f3b68112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.434+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/3555507776 >> 172.21.15.6:0/3555507776 conn(0x7f3b680710b0 msgr2=0x7f3b680f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:07.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.434+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/3555507776 shutdown_connections 2024-03-22T06:27:07.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.434+0000 7f3b6ef9d640 1 -- 172.21.15.6:0/3555507776 wait complete. 2024-03-22T06:27:07.445 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T06:27:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.548+0000 7ff0ba59d640 1 -- 172.21.15.6:0/2397963745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0b40fb110 msgr2=0x7ff0b40fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.548+0000 7ff0ba59d640 1 --2- 172.21.15.6:0/2397963745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0b40fb110 0x7ff0b40fb560 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80048d0 tx=0x7ff0a8009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.548+0000 7ff0ba59d640 1 -- 172.21.15.6:0/2397963745 shutdown_connections 2024-03-22T06:27:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.548+0000 7ff0ba59d640 1 --2- 172.21.15.6:0/2397963745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0b40fb110 0x7ff0b40fb560 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.548+0000 7ff0ba59d640 1 --2- 172.21.15.6:0/2397963745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b4101140 0x7ff0b4101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.548+0000 7ff0ba59d640 1 -- 172.21.15.6:0/2397963745 >> 172.21.15.6:0/2397963745 conn(0x7ff0b40710b0 msgr2=0x7ff0b40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.548+0000 7ff0ba59d640 1 -- 172.21.15.6:0/2397963745 shutdown_connections 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.548+0000 7ff0ba59d640 1 -- 172.21.15.6:0/2397963745 wait complete. 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.548+0000 7ff0ba59d640 1 Processor -- start 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0ba59d640 1 -- start start 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0ba59d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b40fb110 0x7ff0b410de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0ba59d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0b4101140 0x7ff0b410e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0ba59d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b410e8f0 con 0x7ff0b4101140 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0ba59d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b410ea60 con 0x7ff0b40fb110 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0b37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0b4101140 0x7ff0b410e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0b37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0b4101140 0x7ff0b410e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52988/0 (socket says 172.21.15.6:52988) 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0b37fe640 1 -- 172.21.15.6:0/4049977069 learned_addr learned my addr 172.21.15.6:0/4049977069 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0b3fff640 1 --2- 172.21.15.6:0/4049977069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b40fb110 0x7ff0b410de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0b37fe640 1 -- 172.21.15.6:0/4049977069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b40fb110 msgr2=0x7ff0b410de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0b37fe640 1 --2- 172.21.15.6:0/4049977069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b40fb110 0x7ff0b410de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0b37fe640 1 -- 172.21.15.6:0/4049977069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a8005040 con 0x7ff0b4101140 2024-03-22T06:27:07.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0b37fe640 1 --2- 172.21.15.6:0/4049977069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0b4101140 0x7ff0b410e3b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80041d0 tx=0x7ff0a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:07.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.549+0000 7ff0b0ff9640 1 -- 172.21.15.6:0/4049977069 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a8030870 con 0x7ff0b4101140 2024-03-22T06:27:07.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.550+0000 7ff0b0ff9640 1 -- 172.21.15.6:0/4049977069 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0a80092e0 con 0x7ff0b4101140 2024-03-22T06:27:07.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.550+0000 7ff0ba59d640 1 -- 172.21.15.6:0/4049977069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b410ece0 con 0x7ff0b4101140 2024-03-22T06:27:07.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.550+0000 7ff0b0ff9640 1 -- 172.21.15.6:0/4049977069 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a8043400 con 0x7ff0b4101140 2024-03-22T06:27:07.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.550+0000 7ff0ba59d640 1 -- 172.21.15.6:0/4049977069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b4117190 con 0x7ff0b4101140 2024-03-22T06:27:07.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.550+0000 7ff0ba59d640 1 -- 172.21.15.6:0/4049977069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b40fc800 con 0x7ff0b4101140 2024-03-22T06:27:07.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.551+0000 7ff0b0ff9640 1 -- 172.21.15.6:0/4049977069 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7ff0a803b420 con 0x7ff0b4101140 2024-03-22T06:27:07.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.552+0000 7ff0b0ff9640 1 --2- 172.21.15.6:0/4049977069 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff09c06e7b0 0x7ff09c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:07.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.552+0000 7ff0b0ff9640 1 -- 172.21.15.6:0/4049977069 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7ff0a80b11a0 con 0x7ff0b4101140 2024-03-22T06:27:07.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.552+0000 7ff0b3fff640 1 --2- 172.21.15.6:0/4049977069 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff09c06e7b0 0x7ff09c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:07.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.552+0000 7ff0b3fff640 1 --2- 172.21.15.6:0/4049977069 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff09c06e7b0 0x7ff09c070c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4007fd0 tx=0x7ff0a4009e90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:07.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.557+0000 7ff0b0ff9640 1 -- 172.21.15.6:0/4049977069 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7ff0a8079ff0 con 0x7ff0b4101140 2024-03-22T06:27:07.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.768+0000 7ff0ba59d640 1 -- 172.21.15.6:0/4049977069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7ff0b410ece0 con 0x7ff0b4101140 2024-03-22T06:27:07.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.768+0000 7ff0b0ff9640 1 -- 172.21.15.6:0/4049977069 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff0b410ece0 con 0x7ff0b4101140 2024-03-22T06:27:07.768 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-03-22T06:27:07.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.770+0000 7ff0ba59d640 1 -- 172.21.15.6:0/4049977069 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff09c06e7b0 msgr2=0x7ff09c070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.770+0000 7ff0ba59d640 1 --2- 172.21.15.6:0/4049977069 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff09c06e7b0 0x7ff09c070c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4007fd0 tx=0x7ff0a4009e90 comp rx=0 tx=0).stop 2024-03-22T06:27:07.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.770+0000 7ff0ba59d640 1 -- 172.21.15.6:0/4049977069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0b4101140 msgr2=0x7ff0b410e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.770+0000 7ff0ba59d640 1 --2- 172.21.15.6:0/4049977069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0b4101140 0x7ff0b410e3b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80041d0 tx=0x7ff0a8009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.770+0000 7ff0ba59d640 1 -- 172.21.15.6:0/4049977069 shutdown_connections 2024-03-22T06:27:07.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.771+0000 7ff0ba59d640 1 --2- 172.21.15.6:0/4049977069 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7ff09c06e7b0 0x7ff09c070c60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.771+0000 7ff0ba59d640 1 --2- 172.21.15.6:0/4049977069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0b4101140 0x7ff0b410e3b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.771+0000 7ff0ba59d640 1 --2- 172.21.15.6:0/4049977069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b40fb110 0x7ff0b410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.771+0000 7ff0ba59d640 1 -- 172.21.15.6:0/4049977069 >> 172.21.15.6:0/4049977069 conn(0x7ff0b40710b0 msgr2=0x7ff0b40f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:07.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.771+0000 7ff0ba59d640 1 -- 172.21.15.6:0/4049977069 shutdown_connections 2024-03-22T06:27:07.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.771+0000 7ff0ba59d640 1 -- 172.21.15.6:0/4049977069 wait complete. 2024-03-22T06:27:07.782 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T06:27:07.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.882+0000 7f05f3aae640 1 -- 172.21.15.6:0/2363517941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec0fb110 msgr2=0x7f05ec0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.882+0000 7f05f3aae640 1 --2- 172.21.15.6:0/2363517941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec0fb110 0x7f05ec0fb560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0048d0 tx=0x7f05dc009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.882+0000 7f05f3aae640 1 -- 172.21.15.6:0/2363517941 shutdown_connections 2024-03-22T06:27:07.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.882+0000 7f05f3aae640 1 --2- 172.21.15.6:0/2363517941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec0fb110 0x7f05ec0fb560 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.882+0000 7f05f3aae640 1 --2- 172.21.15.6:0/2363517941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05ec101140 0x7f05ec101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.882+0000 7f05f3aae640 1 -- 172.21.15.6:0/2363517941 >> 172.21.15.6:0/2363517941 conn(0x7f05ec0710b0 msgr2=0x7f05ec0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:07.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.882+0000 7f05f3aae640 1 -- 172.21.15.6:0/2363517941 shutdown_connections 2024-03-22T06:27:07.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.882+0000 7f05f3aae640 1 -- 172.21.15.6:0/2363517941 wait complete. 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f3aae640 1 Processor -- start 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f3aae640 1 -- start start 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f3aae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec0fb110 0x7f05ec10bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f3aae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05ec101140 0x7f05ec10c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f3aae640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec065120 con 0x7f05ec0fb110 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f3aae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec065290 con 0x7f05ec101140 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f184a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec0fb110 0x7f05ec10bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f184a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec0fb110 0x7f05ec10bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53014/0 (socket says 172.21.15.6:53014) 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f184a640 1 -- 172.21.15.6:0/2744922108 learned_addr learned my addr 172.21.15.6:0/2744922108 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f1049640 1 --2- 172.21.15.6:0/2744922108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05ec101140 0x7f05ec10c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f184a640 1 -- 172.21.15.6:0/2744922108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05ec101140 msgr2=0x7f05ec10c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f184a640 1 --2- 172.21.15.6:0/2744922108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05ec101140 0x7f05ec10c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.883+0000 7f05f184a640 1 -- 172.21.15.6:0/2744922108 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc005040 con 0x7f05ec0fb110 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.884+0000 7f05f184a640 1 --2- 172.21.15.6:0/2744922108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec0fb110 0x7f05ec10bcc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f05e8002a50 tx=0x7f05e8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:07.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.884+0000 7f05e27fc640 1 -- 172.21.15.6:0/2744922108 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e8007c30 con 0x7f05ec0fb110 2024-03-22T06:27:07.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.884+0000 7f05e27fc640 1 -- 172.21.15.6:0/2744922108 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05e800f040 con 0x7f05ec0fb110 2024-03-22T06:27:07.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.884+0000 7f05f3aae640 1 -- 172.21.15.6:0/2744922108 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec10c800 con 0x7f05ec0fb110 2024-03-22T06:27:07.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.884+0000 7f05e27fc640 1 -- 172.21.15.6:0/2744922108 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e800db50 con 0x7f05ec0fb110 2024-03-22T06:27:07.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.884+0000 7f05f3aae640 1 -- 172.21.15.6:0/2744922108 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec1194d0 con 0x7f05ec0fb110 2024-03-22T06:27:07.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.885+0000 7f05f3aae640 1 -- 172.21.15.6:0/2744922108 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec0fc800 con 0x7f05ec0fb110 2024-03-22T06:27:07.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.885+0000 7f05e27fc640 1 -- 172.21.15.6:0/2744922108 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 39) v1 ==== 92559+0+0 (secure 0 0 0) 0x7f05e801c400 con 0x7f05ec0fb110 2024-03-22T06:27:07.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.886+0000 7f05e27fc640 1 --2- 172.21.15.6:0/2744922108 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f05d806e7b0 0x7f05d8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:27:07.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.886+0000 7f05e27fc640 1 -- 172.21.15.6:0/2744922108 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f05e808ded0 con 0x7f05ec0fb110 2024-03-22T06:27:07.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.886+0000 7f05f1049640 1 --2- 172.21.15.6:0/2744922108 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f05d806e7b0 0x7f05d8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:27:07.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.886+0000 7f05f1049640 1 --2- 172.21.15.6:0/2744922108 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f05d806e7b0 0x7f05d8070c60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009fa0 tx=0x7f05dc009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:27:07.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:07.894+0000 7f05e27fc640 1 -- 172.21.15.6:0/2744922108 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187020 (secure 0 0 0) 0x7f05e8056ca0 con 0x7f05ec0fb110 2024-03-22T06:27:08.031 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:07 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/1427316002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T06:27:08.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:07 smithi006 ceph-mon[94482]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:27:08.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:07 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/3555507776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T06:27:08.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:07 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/4049977069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T06:27:08.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.093+0000 7f05f3aae640 1 -- 172.21.15.6:0/2744922108 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f05ec04bcc0 con 0x7f05ec0fb110 2024-03-22T06:27:08.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.093+0000 7f05e27fc640 1 -- 172.21.15.6:0/2744922108 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v95) v1 ==== 81+0+2516 (secure 0 0 0) 0x7f05e805a360 con 0x7f05ec0fb110 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6827/1010515354 2024-03-23T06:23:28.644314+0000 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6826/1010515354 2024-03-23T06:23:28.644314+0000 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6825/1251227895 2024-03-23T06:23:03.662256+0000 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6829/947853733 2024-03-23T06:22:41.442153+0000 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6828/947853733 2024-03-23T06:22:41.442153+0000 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6827/1260089028 2024-03-23T06:22:13.008321+0000 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6829/1963654099 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2826360973 2024-03-23T06:06:24.832062+0000 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3099962970 2024-03-23T06:05:54.100943+0000 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3099962970 2024-03-23T06:05:54.100943+0000 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2042811963 2024-03-23T06:06:24.832062+0000 2024-03-22T06:27:08.094 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/602685778 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3903206979 2024-03-23T06:06:24.832062+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:0/45595122 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6824/1251227895 2024-03-23T06:23:03.662256+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1533067608 2024-03-23T06:07:32.107836+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3502710787 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/358732355 2024-03-23T06:07:32.107836+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/791727884 2024-03-23T06:07:32.107836+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3606143293 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/254878217 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3489834288 2024-03-23T06:07:32.107836+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:0/3344206523 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2742769039 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/358732355 2024-03-23T06:07:32.107836+0000 2024-03-22T06:27:08.095 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3903206979 2024-03-23T06:06:24.832062+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:0/435100846 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1354987702 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2939515329 2024-03-23T06:05:54.100943+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:0/3477422362 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3665872066 2024-03-23T06:06:24.832062+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/619515244 2024-03-23T06:05:54.100943+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3502710787 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1270793675 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.117:6828/1963654099 2024-03-23T06:16:28.822651+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/5416477 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3809759464 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/602685778 2024-03-23T06:14:38.572050+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/4154587507 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:08.096 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1448591806 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:08.097 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2671132128 2024-03-23T06:05:54.100943+0000 2024-03-22T06:27:08.097 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1248395428 2024-03-23T06:17:37.246941+0000 2024-03-22T06:27:08.097 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6826/1260089028 2024-03-23T06:22:13.008321+0000 2024-03-22T06:27:08.097 INFO:teuthology.orchestra.run.smithi006.stderr:listed 43 entries 2024-03-22T06:27:08.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.095+0000 7f05f3aae640 1 -- 172.21.15.6:0/2744922108 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f05d806e7b0 msgr2=0x7f05d8070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:08.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.095+0000 7f05f3aae640 1 --2- 172.21.15.6:0/2744922108 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f05d806e7b0 0x7f05d8070c60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009fa0 tx=0x7f05dc009da0 comp rx=0 tx=0).stop 2024-03-22T06:27:08.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.095+0000 7f05f3aae640 1 -- 172.21.15.6:0/2744922108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec0fb110 msgr2=0x7f05ec10bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:27:08.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.095+0000 7f05f3aae640 1 --2- 172.21.15.6:0/2744922108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec0fb110 0x7f05ec10bcc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f05e8002a50 tx=0x7f05e8004600 comp rx=0 tx=0).stop 2024-03-22T06:27:08.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.095+0000 7f05f3aae640 1 -- 172.21.15.6:0/2744922108 shutdown_connections 2024-03-22T06:27:08.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.096+0000 7f05f3aae640 1 --2- 172.21.15.6:0/2744922108 >> [v2:172.21.15.6:6800/3762089760,v1:172.21.15.6:6801/3762089760] conn(0x7f05d806e7b0 0x7f05d8070c60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:08.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.096+0000 7f05f3aae640 1 --2- 172.21.15.6:0/2744922108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05ec101140 0x7f05ec10c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:08.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.096+0000 7f05f3aae640 1 --2- 172.21.15.6:0/2744922108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec0fb110 0x7f05ec10bcc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:27:08.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.096+0000 7f05f3aae640 1 -- 172.21.15.6:0/2744922108 >> 172.21.15.6:0/2744922108 conn(0x7f05ec0710b0 msgr2=0x7f05ec0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:27:08.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.096+0000 7f05f3aae640 1 -- 172.21.15.6:0/2744922108 shutdown_connections 2024-03-22T06:27:08.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-22T06:27:08.096+0000 7f05f3aae640 1 -- 172.21.15.6:0/2744922108 wait complete. 2024-03-22T06:27:08.107 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-22T06:27:08.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:27:08.107 DEBUG:teuthology.orchestra.run.smithi117:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-22T06:27:08.143 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-03-22T06:27:08.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:27:08.143 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-22T06:27:08.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:07 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/1427316002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T06:27:08.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:07 smithi117 ceph-mon[82126]: from='mgr.34102 172.21.15.6:0/237219720' entity='mgr.smithi006.sfpxdy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T06:27:08.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:07 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/3555507776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T06:27:08.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:07 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/4049977069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T06:27:08.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:27:08.362 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-03-22T06:27:08.378 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T06:27:08.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:27:08.378 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T06:27:08.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:27:08.445 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2024-03-22T06:27:08.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:27:08.545 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-22T06:27:08.563 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-22T06:27:08.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:27:08.563 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T06:27:08.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T06:27:08.636 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-03-22T06:27:08.732 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-22T06:27:08.745 INFO:tasks.cephadm:Teardown begin 2024-03-22T06:27:08.746 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T06:27:08.773 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T06:27:08.802 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-22T06:27:08.803 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-22T06:27:08.828 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-22T06:27:08.857 INFO:tasks.cephadm:Stopping all daemons... 2024-03-22T06:27:08.858 INFO:tasks.cephadm.mon.smithi006:Stopping mon.smithi006... 2024-03-22T06:27:08.858 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006 2024-03-22T06:27:09.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:08 smithi006 systemd[1]: Stopping Ceph mon.smithi006 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:27:09.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:08 smithi006 ceph-mon[94482]: pgmap v357: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:27:09.032 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:08 smithi006 ceph-mon[94482]: from='client.? 172.21.15.6:0/2744922108' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T06:27:09.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:08 smithi117 ceph-mon[82126]: pgmap v357: 65 pgs: 65 active+clean; 194 MiB data, 811 MiB used, 536 GiB / 536 GiB avail 2024-03-22T06:27:09.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:08 smithi117 ceph-mon[82126]: from='client.? 172.21.15.6:0/2744922108' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T06:27:09.503 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:09 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006[94477]: 2024-03-22T06:27:09.194+0000 7fe667119700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi006 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:27:09.503 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:09 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006[94477]: 2024-03-22T06:27:09.194+0000 7fe667119700 -1 mon.smithi006@0(leader) e3 *** Got Signal Terminated *** 2024-03-22T06:27:09.503 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:09 smithi006 podman[152107]: 2024-03-22 06:27:09.255988506 +0000 UTC m=+0.184207461 container died 7eafc6be6dcac266a1811663c7b778d83ea283998c5ca32058f9de56ed048877 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-22T06:27:09.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:09 smithi006 podman[152107]: 2024-03-22 06:27:09.695216621 +0000 UTC m=+0.623435587 container cleanup 7eafc6be6dcac266a1811663c7b778d83ea283998c5ca32058f9de56ed048877 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-03-22T06:27:09.782 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:09 smithi006 bash[152107]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006 2024-03-22T06:27:10.139 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006.service' 2024-03-22T06:27:10.166 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:09 smithi006 podman[152132]: 2024-03-22 06:27:09.897033657 +0000 UTC m=+0.636931461 container remove 7eafc6be6dcac266a1811663c7b778d83ea283998c5ca32058f9de56ed048877 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi006, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-22T06:27:10.166 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:10 smithi006 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006.service: Deactivated successfully. 2024-03-22T06:27:10.166 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:10 smithi006 systemd[1]: Stopped Ceph mon.smithi006 for fa0e13c2-e811-11ee-95cd-87774f69a715. 2024-03-22T06:27:10.167 INFO:journalctl@ceph.mon.smithi006.smithi006.stdout:Mar 22 06:27:10 smithi006 systemd[1]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi006.service: Consumed 9.343s CPU time. 2024-03-22T06:27:10.210 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T06:27:10.210 INFO:tasks.cephadm.mon.smithi006:Stopped mon.smithi006 2024-03-22T06:27:10.210 INFO:tasks.cephadm.mon.smithi117:Stopping mon.smithi117... 2024-03-22T06:27:10.210 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi117 2024-03-22T06:27:10.505 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:10 smithi117 systemd[1]: Stopping Ceph mon.smithi117 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:27:11.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:10 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117[82122]: 2024-03-22T06:27:10.535+0000 7ffb8b33b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi117 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:27:11.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:10 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117[82122]: 2024-03-22T06:27:10.535+0000 7ffb8b33b700 -1 mon.smithi117@1(peon) e3 *** Got Signal Terminated *** 2024-03-22T06:27:11.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:10 smithi117 podman[117973]: 2024-03-22 06:27:10.730172768 +0000 UTC m=+0.310561583 container died 4766be58b3b28999ca9d77f6e2f72e6a05ff8e14dc03f1aae47d21c9050743f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:27:11.675 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:11 smithi117 podman[117973]: 2024-03-22 06:27:11.279012655 +0000 UTC m=+0.859401463 container cleanup 4766be58b3b28999ca9d77f6e2f72e6a05ff8e14dc03f1aae47d21c9050743f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T06:27:11.675 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:11 smithi117 bash[117973]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117 2024-03-22T06:27:11.675 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Mar 22 06:27:11 smithi117 podman[117995]: 2024-03-22 06:27:11.472075212 +0000 UTC m=+0.739778523 container remove 4766be58b3b28999ca9d77f6e2f72e6a05ff8e14dc03f1aae47d21c9050743f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-mon-smithi117, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-22T06:27:11.694 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@mon.smithi117.service' 2024-03-22T06:27:11.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T06:27:11.746 INFO:tasks.cephadm.mon.smithi117:Stopped mon.smithi117 2024-03-22T06:27:11.746 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-22T06:27:11.746 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.0 2024-03-22T06:27:12.031 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:11 smithi006 systemd[1]: Stopping Ceph osd.0 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:27:12.531 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:12 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[103963]: 2024-03-22T06:27:12.065+0000 7f902327f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:27:12.532 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:12 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[103963]: 2024-03-22T06:27:12.065+0000 7f902327f700 -1 osd.0 95 *** Got signal Terminated *** 2024-03-22T06:27:12.532 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:12 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0[103963]: 2024-03-22T06:27:12.065+0000 7f902327f700 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:27:17.876 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:17 smithi006 podman[152237]: 2024-03-22 06:27:17.568596327 +0000 UTC m=+5.631334260 container died a106f4647f4e1fab1b8d3dea8ff76bb4365e5f4909644deb55bc81b8d3e73b6c (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-22T06:27:18.281 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:18 smithi006 podman[152237]: 2024-03-22 06:27:18.042265856 +0000 UTC m=+6.105003789 container cleanup a106f4647f4e1fab1b8d3dea8ff76bb4365e5f4909644deb55bc81b8d3e73b6c (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_BRANCH=HEAD) 2024-03-22T06:27:18.282 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:18 smithi006 bash[152237]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0 2024-03-22T06:27:18.683 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:18 smithi006 podman[152254]: 2024-03-22 06:27:18.335818314 +0000 UTC m=+0.766468696 container remove a106f4647f4e1fab1b8d3dea8ff76bb4365e5f4909644deb55bc81b8d3e73b6c (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319) 2024-03-22T06:27:18.947 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:18 smithi006 podman[152347]: 2024-03-22 06:27:18.587727901 +0000 UTC m=+0.019128290 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:27:18.947 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:18 smithi006 podman[152347]: 2024-03-22 06:27:18.711887414 +0000 UTC m=+0.143287796 container create 83ea15b725a99bd5a9a326cb7e8dccf474d27f7b3257b9e3aed432f26838aad8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-22T06:27:18.947 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:18 smithi006 podman[152347]: 2024-03-22 06:27:18.948233282 +0000 UTC m=+0.379633679 container init 83ea15b725a99bd5a9a326cb7e8dccf474d27f7b3257b9e3aed432f26838aad8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-22T06:27:19.264 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:18 smithi006 podman[152347]: 2024-03-22 06:27:18.951690739 +0000 UTC m=+0.383091119 container start 83ea15b725a99bd5a9a326cb7e8dccf474d27f7b3257b9e3aed432f26838aad8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:27:19.264 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:19 smithi006 podman[152347]: 2024-03-22 06:27:19.040091905 +0000 UTC m=+0.471492299 container attach 83ea15b725a99bd5a9a326cb7e8dccf474d27f7b3257b9e3aed432f26838aad8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-22T06:27:19.531 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 22 06:27:19 smithi006 podman[152347]: 2024-03-22 06:27:19.264660219 +0000 UTC m=+0.696060613 container died 83ea15b725a99bd5a9a326cb7e8dccf474d27f7b3257b9e3aed432f26838aad8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-22T06:27:19.942 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.0.service' 2024-03-22T06:27:19.984 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T06:27:19.984 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-22T06:27:19.984 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-22T06:27:19.984 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.1 2024-03-22T06:27:20.282 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:20 smithi006 systemd[1]: Stopping Ceph osd.1 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:27:20.632 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:20 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[110081]: 2024-03-22T06:27:20.381+0000 7fe251c54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:27:20.633 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:20 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[110081]: 2024-03-22T06:27:20.381+0000 7fe251c54700 -1 osd.1 95 *** Got signal Terminated *** 2024-03-22T06:27:20.633 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:20 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1[110081]: 2024-03-22T06:27:20.381+0000 7fe251c54700 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:27:26.249 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:25 smithi006 podman[152464]: 2024-03-22 06:27:25.913600426 +0000 UTC m=+5.678208174 container died 2b0a2f2a205f37f4bd4acdb700ca6a794f6a963416e95f98cbefda9c237a848d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-22T06:27:26.531 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:26 smithi006 podman[152464]: 2024-03-22 06:27:26.406802952 +0000 UTC m=+6.171410686 container cleanup 2b0a2f2a205f37f4bd4acdb700ca6a794f6a963416e95f98cbefda9c237a848d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-22T06:27:26.532 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:26 smithi006 bash[152464]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1 2024-03-22T06:27:26.996 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:26 smithi006 podman[152483]: 2024-03-22 06:27:26.650083933 +0000 UTC m=+0.731657309 container remove 2b0a2f2a205f37f4bd4acdb700ca6a794f6a963416e95f98cbefda9c237a848d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:27:27.253 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:26 smithi006 podman[152572]: 2024-03-22 06:27:26.900121356 +0000 UTC m=+0.020485450 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:27:27.254 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:27 smithi006 podman[152572]: 2024-03-22 06:27:27.001075675 +0000 UTC m=+0.121439777 container create 2e2e22610efb4cce7524fdc91375adc6c083211c3b64bc4e1bb11ca6ecabfc61 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-22T06:27:27.254 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:27 smithi006 podman[152572]: 2024-03-22 06:27:27.253872411 +0000 UTC m=+0.374236520 container init 2e2e22610efb4cce7524fdc91375adc6c083211c3b64bc4e1bb11ca6ecabfc61 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-22T06:27:27.532 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:27 smithi006 podman[152572]: 2024-03-22 06:27:27.257659624 +0000 UTC m=+0.378023726 container start 2e2e22610efb4cce7524fdc91375adc6c083211c3b64bc4e1bb11ca6ecabfc61 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-22T06:27:27.532 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:27 smithi006 podman[152572]: 2024-03-22 06:27:27.34542634 +0000 UTC m=+0.465790449 container attach 2e2e22610efb4cce7524fdc91375adc6c083211c3b64bc4e1bb11ca6ecabfc61 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-22T06:27:27.931 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 22 06:27:27 smithi006 podman[152572]: 2024-03-22 06:27:27.56788989 +0000 UTC m=+0.688253993 container died 2e2e22610efb4cce7524fdc91375adc6c083211c3b64bc4e1bb11ca6ecabfc61 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-22T06:27:28.242 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.1.service' 2024-03-22T06:27:28.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T06:27:28.286 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-22T06:27:28.286 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-22T06:27:28.286 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.2 2024-03-22T06:27:28.532 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:28 smithi006 systemd[1]: Stopping Ceph osd.2 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:27:29.032 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:28 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[116178]: 2024-03-22T06:27:28.635+0000 7f1c9fe05700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:27:29.032 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:28 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[116178]: 2024-03-22T06:27:28.635+0000 7f1c9fe05700 -1 osd.2 95 *** Got signal Terminated *** 2024-03-22T06:27:29.032 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:28 smithi006 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2[116178]: 2024-03-22T06:27:28.635+0000 7f1c9fe05700 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:27:34.427 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:34 smithi006 podman[152679]: 2024-03-22 06:27:34.119398063 +0000 UTC m=+5.598681228 container died 3ec1f219c19c1d692b154426a32ffc08a0b0336fba6196e9348a023264d3307b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:27:34.781 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:34 smithi006 podman[152679]: 2024-03-22 06:27:34.627458816 +0000 UTC m=+6.106741973 container cleanup 3ec1f219c19c1d692b154426a32ffc08a0b0336fba6196e9348a023264d3307b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-22T06:27:34.782 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:34 smithi006 bash[152679]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2 2024-03-22T06:27:35.120 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:34 smithi006 podman[152697]: 2024-03-22 06:27:34.862645505 +0000 UTC m=+0.741525586 container remove 3ec1f219c19c1d692b154426a32ffc08a0b0336fba6196e9348a023264d3307b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-22T06:27:35.373 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:35 smithi006 podman[152789]: 2024-03-22 06:27:35.105952448 +0000 UTC m=+0.023621270 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:27:35.373 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:35 smithi006 podman[152789]: 2024-03-22 06:27:35.238761752 +0000 UTC m=+0.156430573 container create e538422ba1e3cbe586be98897f5348ccd9ada2d0337dc75d424e45a821674fd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-22T06:27:35.761 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:35 smithi006 podman[152789]: 2024-03-22 06:27:35.457875027 +0000 UTC m=+0.375543859 container init e538422ba1e3cbe586be98897f5348ccd9ada2d0337dc75d424e45a821674fd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:27:35.762 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:35 smithi006 podman[152789]: 2024-03-22 06:27:35.461179527 +0000 UTC m=+0.378848355 container start e538422ba1e3cbe586be98897f5348ccd9ada2d0337dc75d424e45a821674fd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:27:35.762 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:35 smithi006 podman[152789]: 2024-03-22 06:27:35.532861879 +0000 UTC m=+0.450530729 container attach e538422ba1e3cbe586be98897f5348ccd9ada2d0337dc75d424e45a821674fd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, CEPH_POINT_RELEASE=) 2024-03-22T06:27:36.031 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 22 06:27:35 smithi006 podman[152789]: 2024-03-22 06:27:35.761880139 +0000 UTC m=+0.679548961 container died e538422ba1e3cbe586be98897f5348ccd9ada2d0337dc75d424e45a821674fd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-22T06:27:36.417 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.2.service' 2024-03-22T06:27:36.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T06:27:36.461 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-22T06:27:36.461 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-22T06:27:36.461 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.3 2024-03-22T06:27:36.756 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:36 smithi117 systemd[1]: Stopping Ceph osd.3 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:27:36.756 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:36 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:27:36.738+0000 7f74c5b43700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:27:36.756 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:36 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:27:36.738+0000 7f74c5b43700 -1 osd.3 95 *** Got signal Terminated *** 2024-03-22T06:27:36.756 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:36 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:27:36.738+0000 7f74c5b43700 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:27:38.505 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:38 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:38.226+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:39.505 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:39 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:39.236+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:40.505 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:40 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:40.194+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:41.234 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:40 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3[91785]: 2024-03-22T06:27:40.937+0000 7f74c2137700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.749430+0000 front 2024-03-22T06:27:16.749464+0000 (oldest deadline 2024-03-22T06:27:40.849590+0000) 2024-03-22T06:27:41.505 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:41 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:41.240+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:41.505 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:41 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:41.234+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:42.355 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:42 smithi117 podman[118107]: 2024-03-22 06:27:42.078273183 +0000 UTC m=+5.426165526 container died 9cba39b6f32c7854368a20769dca6ec44ecf646b0335f2cdc314f8db750f398b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:27:42.355 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:42 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:42.237+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:42.356 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:42 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:42.217+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:42.680 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:42 smithi117 podman[118107]: 2024-03-22 06:27:42.446176538 +0000 UTC m=+5.794068884 container cleanup 9cba39b6f32c7854368a20769dca6ec44ecf646b0335f2cdc314f8db750f398b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-22T06:27:42.680 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:42 smithi117 bash[118107]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3 2024-03-22T06:27:43.004 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:42 smithi117 podman[118136]: 2024-03-22 06:27:42.6808487 +0000 UTC m=+0.603545770 container remove 9cba39b6f32c7854368a20769dca6ec44ecf646b0335f2cdc314f8db750f398b (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-22T06:27:43.255 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:43 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:43.222+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:43.255 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:43 smithi117 podman[118227]: 2024-03-22 06:27:42.908626048 +0000 UTC m=+0.017179231 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:27:43.256 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:43 smithi117 podman[118227]: 2024-03-22 06:27:43.00654915 +0000 UTC m=+0.115102324 container create a8e56b16d401aea89b4705e8318c84d7843f6b64f61cf6a38da6d77d05198e63 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:27:43.256 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:43 smithi117 podman[118227]: 2024-03-22 06:27:43.209165322 +0000 UTC m=+0.317718503 container init a8e56b16d401aea89b4705e8318c84d7843f6b64f61cf6a38da6d77d05198e63 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-deactivate, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-22T06:27:43.256 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:43 smithi117 podman[118227]: 2024-03-22 06:27:43.212538166 +0000 UTC m=+0.321091344 container start a8e56b16d401aea89b4705e8318c84d7843f6b64f61cf6a38da6d77d05198e63 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-22T06:27:43.755 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:43 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:43.258+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:43.755 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:43 smithi117 podman[118227]: 2024-03-22 06:27:43.309198189 +0000 UTC m=+0.417751377 container attach a8e56b16d401aea89b4705e8318c84d7843f6b64f61cf6a38da6d77d05198e63 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-22T06:27:43.756 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 22 06:27:43 smithi117 podman[118227]: 2024-03-22 06:27:43.50247416 +0000 UTC m=+0.611027326 container died a8e56b16d401aea89b4705e8318c84d7843f6b64f61cf6a38da6d77d05198e63 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:27:44.115 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.3.service' 2024-03-22T06:27:44.157 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T06:27:44.157 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-22T06:27:44.157 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-22T06:27:44.157 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.4 2024-03-22T06:27:44.473 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:44 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:44.179+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:44.474 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:44 smithi117 systemd[1]: Stopping Ceph osd.4 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:27:44.474 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:44 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:44.302+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:44.474 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:44 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:44.474+0000 7fd96b104700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:27:44.755 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:44 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:44.474+0000 7fd96b104700 -1 osd.4 95 *** Got signal Terminated *** 2024-03-22T06:27:44.755 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:44 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:44.474+0000 7fd96b104700 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:27:45.506 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:45 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:45.295+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:45.506 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:45 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:45.161+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:46.505 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:46 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:46.339+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:46.506 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:46 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:46.135+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:47.506 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:47 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:47.389+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:47.506 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:47 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:47.168+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:47.506 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:47 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:47.168+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:25.354249+0000 front 2024-03-22T06:27:25.354259+0000 (oldest deadline 2024-03-22T06:27:47.054108+0000) 2024-03-22T06:27:48.505 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:48 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:48.395+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:48.505 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:48 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:48.395+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:22.818470+0000 front 2024-03-22T06:27:22.818481+0000 (oldest deadline 2024-03-22T06:27:48.118423+0000) 2024-03-22T06:27:48.506 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:48 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:48.189+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:48.506 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:48 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:48.189+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:25.354249+0000 front 2024-03-22T06:27:25.354259+0000 (oldest deadline 2024-03-22T06:27:47.054108+0000) 2024-03-22T06:27:49.505 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:49 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:49.351+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:14.017641+0000 front 2024-03-22T06:27:14.017625+0000 (oldest deadline 2024-03-22T06:27:38.117562+0000) 2024-03-22T06:27:49.505 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:49 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4[96636]: 2024-03-22T06:27:49.351+0000 7fd9636f0700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:22.818470+0000 front 2024-03-22T06:27:22.818481+0000 (oldest deadline 2024-03-22T06:27:48.118423+0000) 2024-03-22T06:27:49.506 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:49 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:49.145+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:49.506 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:49 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:49.145+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:25.354249+0000 front 2024-03-22T06:27:25.354259+0000 (oldest deadline 2024-03-22T06:27:47.054108+0000) 2024-03-22T06:27:50.239 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:50 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:50.118+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:50.239 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:50 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:50.118+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:25.354249+0000 front 2024-03-22T06:27:25.354259+0000 (oldest deadline 2024-03-22T06:27:47.054108+0000) 2024-03-22T06:27:50.239 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:49 smithi117 podman[118334]: 2024-03-22 06:27:49.927999803 +0000 UTC m=+5.546813845 container died 1950fc21bd2e9f45ba5fb986f27f31def11ba958a84b6a1d9a23dbdaf3d61c2d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:27:50.239 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:50 smithi117 podman[118334]: 2024-03-22 06:27:50.2398667 +0000 UTC m=+5.858680750 container cleanup 1950fc21bd2e9f45ba5fb986f27f31def11ba958a84b6a1d9a23dbdaf3d61c2d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:27:50.505 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:50 smithi117 bash[118334]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4 2024-03-22T06:27:50.505 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:50 smithi117 podman[118352]: 2024-03-22 06:27:50.399423633 +0000 UTC m=+0.470378825 container remove 1950fc21bd2e9f45ba5fb986f27f31def11ba958a84b6a1d9a23dbdaf3d61c2d (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:27:51.080 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:50 smithi117 podman[118440]: 2024-03-22 06:27:50.700235937 +0000 UTC m=+0.017859427 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:27:51.081 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:50 smithi117 podman[118440]: 2024-03-22 06:27:50.808716621 +0000 UTC m=+0.126340109 container create cc43912069b8be85740be4f8ced4eca8292a16137e8309acaef6ad014c81a5b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-22T06:27:51.081 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:51 smithi117 podman[118440]: 2024-03-22 06:27:51.036002721 +0000 UTC m=+0.353626218 container init cc43912069b8be85740be4f8ced4eca8292a16137e8309acaef6ad014c81a5b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:27:51.081 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:51 smithi117 podman[118440]: 2024-03-22 06:27:51.039140907 +0000 UTC m=+0.356764399 container start cc43912069b8be85740be4f8ced4eca8292a16137e8309acaef6ad014c81a5b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-22T06:27:51.336 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:51 smithi117 podman[118440]: 2024-03-22 06:27:51.127883881 +0000 UTC m=+0.445507370 container attach cc43912069b8be85740be4f8ced4eca8292a16137e8309acaef6ad014c81a5b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:27:51.337 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:51 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:51.080+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:51.337 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:51 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:51.080+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:25.354249+0000 front 2024-03-22T06:27:25.354259+0000 (oldest deadline 2024-03-22T06:27:47.054108+0000) 2024-03-22T06:27:51.613 INFO:journalctl@ceph.osd.4.smithi117.stdout:Mar 22 06:27:51 smithi117 podman[118440]: 2024-03-22 06:27:51.337835504 +0000 UTC m=+0.655459012 container died cc43912069b8be85740be4f8ced4eca8292a16137e8309acaef6ad014c81a5b5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-22T06:27:51.884 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.4.service' 2024-03-22T06:27:51.926 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T06:27:51.926 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-22T06:27:51.927 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-22T06:27:51.927 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.5 2024-03-22T06:27:52.242 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:52 smithi117 systemd[1]: Stopping Ceph osd.5 for fa0e13c2-e811-11ee-95cd-87774f69a715... 2024-03-22T06:27:52.242 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:52.071+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:52.242 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:52.071+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:25.354249+0000 front 2024-03-22T06:27:25.354259+0000 (oldest deadline 2024-03-22T06:27:47.054108+0000) 2024-03-22T06:27:52.242 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:52.241+0000 7f65ba82e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:27:52.242 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:52.241+0000 7f65ba82e700 -1 osd.5 95 *** Got signal Terminated *** 2024-03-22T06:27:52.242 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:52 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:52.241+0000 7f65ba82e700 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T06:27:53.505 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:53 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:53.064+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:53.505 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:53 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:53.064+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:25.354249+0000 front 2024-03-22T06:27:25.354259+0000 (oldest deadline 2024-03-22T06:27:47.054108+0000) 2024-03-22T06:27:54.505 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:54 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:54.091+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:54.506 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:54 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:54.091+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:25.354249+0000 front 2024-03-22T06:27:25.354259+0000 (oldest deadline 2024-03-22T06:27:47.054108+0000) 2024-03-22T06:27:55.507 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:55 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:55.045+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:55.508 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:55 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:55.045+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:25.354249+0000 front 2024-03-22T06:27:25.354259+0000 (oldest deadline 2024-03-22T06:27:47.054108+0000) 2024-03-22T06:27:56.505 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:56 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:56.021+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:56.505 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:56 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:56.021+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:25.354249+0000 front 2024-03-22T06:27:25.354259+0000 (oldest deadline 2024-03-22T06:27:47.054108+0000) 2024-03-22T06:27:56.505 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:56 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:56.021+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-22T06:27:29.855094+0000 front 2024-03-22T06:27:29.855112+0000 (oldest deadline 2024-03-22T06:27:55.155016+0000) 2024-03-22T06:27:57.255 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:57 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:57.003+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-22T06:27:16.053130+0000 front 2024-03-22T06:27:16.053011+0000 (oldest deadline 2024-03-22T06:27:40.753012+0000) 2024-03-22T06:27:57.255 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:57 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:57.003+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-22T06:27:25.354249+0000 front 2024-03-22T06:27:25.354259+0000 (oldest deadline 2024-03-22T06:27:47.054108+0000) 2024-03-22T06:27:57.255 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:57 smithi117 ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5[101635]: 2024-03-22T06:27:57.003+0000 7f65b2e1a700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-22T06:27:29.855094+0000 front 2024-03-22T06:27:29.855112+0000 (oldest deadline 2024-03-22T06:27:55.155016+0000) 2024-03-22T06:27:57.999 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:57 smithi117 podman[118545]: 2024-03-22 06:27:57.71582402 +0000 UTC m=+5.568666771 container died 1a6a0ee2a2289dcc8c6f9dd04f36b3c65ae4a04dfb7a18e30ea86e2e8e1c2540 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217) 2024-03-22T06:27:57.999 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:58 smithi117 podman[118545]: 2024-03-22 06:27:57.999958522 +0000 UTC m=+5.852801270 container cleanup 1a6a0ee2a2289dcc8c6f9dd04f36b3c65ae4a04dfb7a18e30ea86e2e8e1c2540 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-22T06:27:58.255 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:58 smithi117 bash[118545]: ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5 2024-03-22T06:27:58.255 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:58 smithi117 podman[118563]: 2024-03-22 06:27:58.159668741 +0000 UTC m=+0.436819085 container remove 1a6a0ee2a2289dcc8c6f9dd04f36b3c65ae4a04dfb7a18e30ea86e2e8e1c2540 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T06:27:59.005 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:58 smithi117 podman[118659]: 2024-03-22 06:27:58.560340962 +0000 UTC m=+0.105002319 container create 68f177a3ac82ca2643adad69d9bede34f142817ab851573987c6b53eaa3a1206 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-22T06:27:59.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:58 smithi117 podman[118659]: 2024-03-22 06:27:58.473256221 +0000 UTC m=+0.017917583 image pull aa461e7c5e7f322b169c789b389d594b994ea7ffcbc9e24e563a3ec9b69bc3c1 quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9 2024-03-22T06:27:59.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:58 smithi117 podman[118659]: 2024-03-22 06:27:58.746230267 +0000 UTC m=+0.290891628 container init 68f177a3ac82ca2643adad69d9bede34f142817ab851573987c6b53eaa3a1206 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-22T06:27:59.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:58 smithi117 podman[118659]: 2024-03-22 06:27:58.749372446 +0000 UTC m=+0.294033803 container start 68f177a3ac82ca2643adad69d9bede34f142817ab851573987c6b53eaa3a1206 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-22T06:27:59.006 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:58 smithi117 podman[118659]: 2024-03-22 06:27:58.804273457 +0000 UTC m=+0.348934815 container attach 68f177a3ac82ca2643adad69d9bede34f142817ab851573987c6b53eaa3a1206 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-22T06:27:59.314 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:59 smithi117 podman[118659]: 2024-03-22 06:27:59.033096179 +0000 UTC m=+0.577757538 container died 68f177a3ac82ca2643adad69d9bede34f142817ab851573987c6b53eaa3a1206 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-22T06:27:59.564 INFO:journalctl@ceph.osd.5.smithi117.stdout:Mar 22 06:27:59 smithi117 podman[118685]: 2024-03-22 06:27:59.549170359 +0000 UTC m=+0.510842816 container remove 68f177a3ac82ca2643adad69d9bede34f142817ab851573987c6b53eaa3a1206 (image=quay.ceph.io/ceph-ci/ceph@sha256:86a7734db939dbe0061161a071f862ec15f51e06c205e37a9b0f00b8755315a9, name=ceph-fa0e13c2-e811-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-03-21-1835-ea01217, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:27:59.567 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fa0e13c2-e811-11ee-95cd-87774f69a715@osd.5.service' 2024-03-22T06:27:59.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T06:27:59.608 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-22T06:27:59.608 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-22T06:28:20.077 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-22T06:28:33.462 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T06:28:33.497 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T06:28:33.527 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-22T06:28:33.529 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090/remote/smithi006/crash 2024-03-22T06:28:33.530 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/crash -- . 2024-03-22T06:28:33.566 INFO:teuthology.orchestra.run.smithi006.stderr:tar: /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-22T06:28:33.567 INFO:teuthology.orchestra.run.smithi006.stderr:tar: Error is not recoverable: exiting now 2024-03-22T06:28:33.568 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090/remote/smithi117/crash 2024-03-22T06:28:33.569 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/crash -- . 2024-03-22T06:28:33.595 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-22T06:28:33.596 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-03-22T06:28:33.597 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-22T06:28:33.597 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-22T06:28:33.643 INFO:teuthology.orchestra.run.smithi006.stdout:1711088189.6958392 mon.smithi006 (mon.0) 890 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-22T06:28:33.644 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-22T06:28:33.644 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[SEC\]' /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-22T06:28:33.718 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[ERR\]' /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-22T06:28:33.795 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[WRN\]' /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-22T06:28:33.867 INFO:teuthology.orchestra.run.smithi006.stdout:1711088189.6958392 mon.smithi006 (mon.0) 890 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-22T06:28:33.867 INFO:tasks.cephadm:Compressing logs... 2024-03-22T06:28:33.867 DEBUG:teuthology.orchestra.run.smithi006:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-22T06:28:33.913 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-22T06:28:33.938 INFO:teuthology.orchestra.run.smithi006.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-03-22T06:28:33.939 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-03-22T06:28:33.939 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi006.dcjpla.log 2024-03-22T06:28:33.939 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi006.bztopc.log 2024-03-22T06:28:33.939 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi006.dcjpla.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-22T06:28:33.939 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi006.bztopc.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-22T06:28:33.941 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mon.smithi006.log 2024-03-22T06:28:33.941 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-22T06:28:33.941 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mon.smithi006.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-22T06:28:33.946 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-22T06:28:33.946 INFO:teuthology.orchestra.run.smithi117.stderr:: No such file or directory 2024-03-22T06:28:33.946 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi117.fwkech.log 2024-03-22T06:28:33.946 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi117.fwkech.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.4.log: /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mon.smithi117.log 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mon.smithi117.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.log 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.cephadm.log: 85.5% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi117.zqxyql.log 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi117.zqxyql.log: 91.2% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-22T06:28:33.947 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mgr.smithi117.bqeskk.log 2024-03-22T06:28:33.949 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-volume.log: 87.8% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-22T06:28:33.951 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.0.log: 91.1% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-22T06:28:33.951 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.log 2024-03-22T06:28:33.951 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.cephadm.log: 85.5% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-22T06:28:33.951 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-22T06:28:33.951 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.log: 87.8% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-22T06:28:33.951 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mgr.smithi006.sfpxdy.log 2024-03-22T06:28:33.967 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-22T06:28:33.968 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mgr.smithi117.bqeskk.log: 91.1% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-22T06:28:33.995 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mgr.smithi006.sfpxdy.log: /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-volume.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-22T06:28:33.998 INFO:teuthology.orchestra.run.smithi117.stderr: 89.6% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mgr.smithi117.bqeskk.log.gz 2024-03-22T06:28:34.029 INFO:teuthology.orchestra.run.smithi006.stderr: 91.0% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-22T06:28:34.338 INFO:teuthology.orchestra.run.smithi117.stderr: 92.5% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mon.smithi117.log.gz 2024-03-22T06:28:34.341 INFO:teuthology.orchestra.run.smithi006.stderr: 88.3% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mgr.smithi006.sfpxdy.log.gz 2024-03-22T06:28:35.039 INFO:teuthology.orchestra.run.smithi006.stderr: 90.5% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mon.smithi006.log.gz 2024-03-22T06:28:39.097 INFO:teuthology.orchestra.run.smithi117.stderr: 94.9% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi117.zqxyql.log.gz 2024-03-22T06:28:39.576 INFO:teuthology.orchestra.run.smithi006.stderr: 94.9% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi006.dcjpla.log.gz 2024-03-22T06:28:40.025 INFO:teuthology.orchestra.run.smithi117.stderr: 93.4% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-22T06:28:40.211 INFO:teuthology.orchestra.run.smithi006.stderr: 93.4% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-22T06:28:40.251 INFO:teuthology.orchestra.run.smithi117.stderr: 93.4% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-22T06:28:40.461 INFO:teuthology.orchestra.run.smithi006.stderr: 93.5% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-22T06:28:40.874 INFO:teuthology.orchestra.run.smithi117.stderr: 94.9% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi117.fwkech.log.gz 2024-03-22T06:28:41.082 INFO:teuthology.orchestra.run.smithi117.stderr: 92.9% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-22T06:28:41.084 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-03-22T06:28:41.084 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m7.157s 2024-03-22T06:28:41.084 INFO:teuthology.orchestra.run.smithi117.stderr:user 0m31.021s 2024-03-22T06:28:41.084 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m1.050s 2024-03-22T06:28:41.501 INFO:teuthology.orchestra.run.smithi006.stderr: 93.2% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-22T06:30:12.945 INFO:teuthology.orchestra.run.smithi006.stderr: 92.7% -- replaced with /var/log/ceph/fa0e13c2-e811-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi006.bztopc.log.gz 2024-03-22T06:30:12.947 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-03-22T06:30:12.947 INFO:teuthology.orchestra.run.smithi006.stderr:real 1m39.022s 2024-03-22T06:30:12.947 INFO:teuthology.orchestra.run.smithi006.stderr:user 2m2.807s 2024-03-22T06:30:12.947 INFO:teuthology.orchestra.run.smithi006.stderr:sys 0m3.483s 2024-03-22T06:30:12.947 INFO:tasks.cephadm:Archiving logs... 2024-03-22T06:30:12.948 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/log/ceph to /home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090/remote/smithi006/log 2024-03-22T06:30:12.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-22T06:30:30.823 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090/remote/smithi117/log 2024-03-22T06:30:30.824 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-22T06:30:34.976 INFO:tasks.cephadm:Removing cluster... 2024-03-22T06:30:34.976 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 --force 2024-03-22T06:30:35.590 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fa0e13c2-e811-11ee-95cd-87774f69a715 --force 2024-03-22T06:30:36.198 INFO:tasks.cephadm:Removing cephadm ... 2024-03-22T06:30:36.198 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-22T06:30:36.215 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-22T06:30:36.230 INFO:tasks.cephadm:Teardown complete 2024-03-22T06:30:36.230 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-22T06:30:36.241 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-22T06:30:36.241 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-22T06:30:36.259 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-22T06:30:36.346 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-22T06:30:36.357 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-22T06:30:36.357 DEBUG:teuthology.orchestra.run.smithi006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T06:30:36.360 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T06:30:36.372 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-03-22T06:30:36.373 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-03-22T06:30:36.377 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T06:30:36.377 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-03-22T06:30:36.377 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 55 +449us[ +433us] +/- 79ms 2024-03-22T06:30:36.377 INFO:teuthology.orchestra.run.smithi006.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -925us[ -941us] +/- 51ms 2024-03-22T06:30:36.377 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +837us[ +837us] +/- 79ms 2024-03-22T06:30:36.377 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:30:36.378 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T06:30:36.379 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-03-22T06:30:36.379 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 56 +488us[ +488us] +/- 79ms 2024-03-22T06:30:36.379 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv02.front.sepia.ceph.com 3 6 377 57 -880us[ -867us] +/- 51ms 2024-03-22T06:30:36.379 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +873us[ +873us] +/- 79ms 2024-03-22T06:30:36.379 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:30:36.379 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-22T06:30:36.390 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-22T06:30:36.391 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-22T06:30:36.415 DEBUG:teuthology.orchestra.run.smithi006:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-22T06:30:36.509 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-22T06:30:36.586 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-22T06:30:36.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:30:36.619 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-22T06:30:36.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:30:36.649 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-22T06:30:36.661 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-22T06:30:36.692 INFO:teuthology.task.internal:Duration was 2001.370124 seconds 2024-03-22T06:30:36.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-22T06:30:36.724 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-22T06:30:36.724 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T06:30:36.727 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T06:30:36.768 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T06:30:36.769 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T06:30:37.076 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-22T06:30:37.076 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi006.front.sepia.ceph.com 2024-03-22T06:30:37.076 DEBUG:teuthology.orchestra.run.smithi006:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-22T06:30:37.137 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-03-22T06:30:37.138 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-22T06:30:37.163 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-22T06:30:37.163 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T06:30:37.180 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T06:30:37.233 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-22T06:30:37.233 DEBUG:teuthology.orchestra.run.smithi006:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-22T06:30:37.248 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-22T06:30:37.904 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-22T06:30:37.916 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-22T06:30:37.916 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-22T06:30:37.943 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-22T06:30:37.972 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-22T06:30:37.984 DEBUG:teuthology.orchestra.run.smithi006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T06:30:37.987 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T06:30:38.009 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = core 2024-03-22T06:30:38.038 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-03-22T06:30:38.053 DEBUG:teuthology.orchestra.run.smithi006:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-22T06:30:38.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:30:38.080 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-22T06:30:38.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:30:38.108 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-22T06:30:38.120 INFO:teuthology.task.internal:Transferring archived files... 2024-03-22T06:30:38.121 DEBUG:teuthology.misc:Transferring archived files from smithi006:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090/remote/smithi006 2024-03-22T06:30:38.122 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-22T06:30:38.178 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090/remote/smithi117 2024-03-22T06:30:38.178 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-22T06:30:38.218 INFO:teuthology.task.internal:Removing archive directory... 2024-03-22T06:30:38.218 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-22T06:30:38.221 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-22T06:30:38.266 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-22T06:30:38.283 INFO:teuthology.task.internal:Not uploading archives. 2024-03-22T06:30:38.284 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-22T06:30:38.342 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-22T06:30:38.342 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T06:30:38.344 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T06:30:38.356 INFO:teuthology.orchestra.run.smithi006.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 22 06:30 /home/ubuntu/cephtest 2024-03-22T06:30:38.358 INFO:teuthology.orchestra.run.smithi117.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 22 06:30 /home/ubuntu/cephtest 2024-03-22T06:30:38.359 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-22T06:30:38.374 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-22T06:30:38.442 INFO:teuthology.nuke:Checking targets against current locks 2024-03-22T06:30:38.470 DEBUG:teuthology.nuke:shortname: smithi006 2024-03-22T06:30:38.470 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T06:30:38.496 DEBUG:teuthology.nuke:shortname: smithi117 2024-03-22T06:30:38.496 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T06:30:38.500 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 05:47:43.154049', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL7Rh2nhxdiKheDchRY+s+PZcNNLncc8JtyB/NpcJI2YkbwhsG/vLRl4Ex7h/vWFnvtTLSdRhiuvrd8TdxoU5ok='} 2024-03-22T06:30:38.520 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-03-22_03:02:08-orch:cephadm-wip-phlogistonjohn-testing-2024-03-21-1835-distro-default-smithi/7616090', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 05:47:43.151797', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLSrAIKihkpI38B332XvAQERRk0xewPUHrkVozw9iWrIVEJAogL1Vf9QTCD4D1w34az4Mhh1zPLDzDksRuKdlig='} 2024-03-22T06:30:38.556 INFO:teuthology.orchestra.console.smithi006:Power off 2024-03-22T06:30:38.556 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-22T06:30:38.594 INFO:teuthology.orchestra.console.smithi117:Power off 2024-03-22T06:30:38.594 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-22T06:30:38.602 DEBUG:teuthology.orchestra.console.smithi006:power off output: Chassis Power Control: Down/Off 2024-03-22T06:30:38.603 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T06:30:38.617 DEBUG:teuthology.orchestra.console.smithi117:power off output: Chassis Power Control: Down/Off 2024-03-22T06:30:38.618 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T06:30:38.641 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-03-22T06:30:38.646 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2024-03-22T06:30:42.642 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T06:30:42.650 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T06:30:42.753 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-03-22T06:30:42.760 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2024-03-22T06:30:46.758 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T06:30:46.773 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T06:30:46.875 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-03-22T06:30:46.887 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2024-03-22T06:30:46.988 INFO:teuthology.orchestra.console.smithi006:Power off completed 2024-03-22T06:30:50.879 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T06:30:50.989 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is off 2024-03-22T06:30:51.090 INFO:teuthology.orchestra.console.smithi117:Power off completed 2024-03-22T06:30:51.195 INFO:teuthology.run:Summary data: description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2001.3701238632202 failure_reason: '"1711088189.6958392 mon.smithi006 (mon.0) 890 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log' flavor: default owner: scheduled_phlogistonjohn@teuthology success: false 2024-03-22T06:30:51.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T06:30:51.291 INFO:teuthology.run:FAIL